Doing style transfer with grounding OpenAI's CLIP.
Under normal circumstances, one would use a method like that of Gatys et al. to transfer the style of one image to the content of another. This only encompasses one photo and has no true concept of space.
Using a similar technique to that of Gatys et al. we trade Gram Matrices for CLIP activations. Given your image and a phrase, CLIP can transfer (somewhat successfully) the artist or painting's style that you specify!
- There is an understanding of spatial relationships. 🌌
- Ghost faces and self-portraits. 👻
- Abstract/general style and motif comprehension. 🎨