I Lived 5 Minutes From the Ghibli Museum and Here’s My Thoughts on the Ghibli AI Trend

I quickly became a fan. Not a huge fan where it consumed my life, but enough of a fan where I checked out a few of the studio’s well-known productions (e.g., Spirited Away) and enjoyed them.

So when the internet became flooded with Ghibli style art last week after OpenAI released their newest AI image generator, I was immediately taken back there – cerebrally speaking that is.

Needless to say, my reaction was probably more visceral than the thousands (millions?) of others sharing their thoughts about the trend. After spending some time Ghibli-ficating myself, reading others’ opinions, and having a few days to reflect on it all, here are my thoughts.

Initial reaction 🤯

It’d be hard to deny that the sudden ability to Ghiblify any image didn’t immediately elicit a “woah, this is awesome!” from most people – myself very much included. The resulting trend is a testament to that.

My guess is that even those who have vocally protested against it (other than perhaps people who are closely associated with Studio Ghibli itself) were probably initially impressed as well.

There’s something emotional about being able to transform yourself into a character from an anime universe that you’ve watched and appreciated in the past.

It somehow reminds me of how I felt as a kid when the first Teenage Mutant Ninja Turtles movie was released back in 1990. All of a sudden my favorite cartoon characters were transformed into “real” creatures.

But even without overthinking the underlying psychology, the facts remain:

• It’s genuinely enjoyable to Ghiblify yourself.
• OpenAI’s new image generator is mighty impressive.

What makes the new OpenAI image generator different

Aside from the fun factor, there are three main things that stood out to me about the new OpenAI image generator. In my opinion, this isn’t just another update. At the risk of sounding cliché – it’s a gamechanger. Here’s why…

Text finally works

For the first time since AI image tools hit the internet, we can finally generate an image where the text isn’t some illegible alphabet slop.

You can instruct the new OpenAI tool to write specific text on parts of your image and it will generally do it without errors. If there are errors, they are minor and fixable.

And if you are using it to Ghiblify an image (or apply some other effect), it’ll also pick up on existing text in your uploaded image, and duplicate it over without you even asking.

Contextual awareness

Another interesting difference I noticed from previous image generation models is that the new OpenAI tool also exhibits contextual awareness. For example, one of the photos I asked it to Ghiblify was of me standing in front of the Chureito Pagoda in Arakurayama Sengen Park.

In the original photo, Mt. Fuji isn’t visible in the background and more than half of the pagoda is cropped out.

Yet the tool was able to scan the image, correctly identify the exact location and recreate the invisible portions of it accurately. If I had taken the real-life photo on a less cloudy day and at a slightly different angle, it would look exactly the way the tool sketched it.

Much better at applying feedback

Most new AI image models have a feedback function that lets you describe what you’d like to change after the initial output you are given. From personal experience – and I’m sure many of you can relate – those feedback features can easily suck you into an endless loop of iterations that eventually end in you closing your tab out of frustration.

That all just changed.

Somehow OpenAI’s new image generator actually understands your feedback and applies it. It’s not always perfect and it might still take a few tries but you eventually get your desired outcome – or something close to it.

The ethics and legality of it all ⚖️

Ability is one thing, but morality and legality are entirely separate things.

While you’ll probably have a tough time finding anyone who is going to dispute the capabilities of OpenAI’s new model (and its ability to Ghiblify images), you don’t have to look far to see that not everyone feels so warm and fuzzy about it. Least of all is probably the founder of Studio Ghibli itself – Hayao Miyazaki.

Although he hasn’t publicly come out to comment on the Ghibli AI trend specifically, his name immediately began popping up as the trend took off. Critics started quoting what he said in a documentary from almost a decade ago. In the documentary, after being shown a crude AI-generated animation demo of a zombie, he responded with:

“Whoever creates this stuff has no idea what pain is whatsoever. I am utterly disgusted… I strongly feel that this is an insult to life itself.”

Those are some harsh words. Many people are understandably upset and they are quoting Miyazaki as a kind of authoritative validation that confirms their views. Like “hey, look, the guy who’s style you’re emulating would be against this.”

What the law says

Unfortunately, personal opinions and legal interpretations are not the same – hence why Altman and Co. have been able to effectively get away with using the Ghibli style in their AI models.

The OpenAI CEO even seemed to mock naysayers on his X account by posting “just because you should doesn’t mean you can” – a play on words that reverses the position of should and can to change the meaning of the classic appeal to morality.

Now I’m no lawyer here, but I don’t need to be one to understand that OpenAI and their legal team believe they can handle the smoke. Or at least that the fire will bring them enough money to offset any potential damage they might have to deal with later. This is the approach they have historically taken. And even though they (and others) have been slapped with lawsuits, it doesn’t seem to be changing their mind about altering course. ¹

From what I do understand in my capacity as a non-legal expert, is that above all, the legality of AI models using existing works is not set in stone – but it also depends on where you’re talking about.

A matter of jurisdiction

Legally speaking, OpenAI is a U.S. company headquartered in San Francisco. But in practice, it’s a transnational corporate behemoth. The work they train their models on comes from every corner of the web-connected Earth. In scenarios where another U.S. entity is upset with OpenAI about their work being used without permission and decides to sue them, it’s easy to establish jurisdiction.

But what if – hypothetically speaking – a Japanese company like Studio Ghibli wanted to sue OpenAI for similar reasons?

Unless OpenAI had substantial assets in Japan, enforcing a Japanese judgment against a U.S. company would be challenging, if not impossible. Not only that, but Japanese laws would offer them little protection anyway. Japan’s Article 30-4 explicitly permits the use of copyrighted works for AI training purposes, even for commercial use. ²

So if they wanted some kind of legal recourse, Studio Ghibli’s only option would be to file a suit in U.S. courts. The case would be tried under American copyright law, where there are two main factors at play:

• Generative AI can potentially violate copyright law if the program has access to a copyright owner’s work and is generating output that is “substantially similar” to the copyright owner’s existing work. But – and this is a big BUT – there is no federal legal consensus that determines what constitutes “substantial similarity.” ³
• Beyond that, the use of copyrighted material for AI training falls under the “fair use” doctrine. This is a legal principle that allows limited use of copyrighted material without permission under certain circumstances.

In short, it’s a legal grey area and it’s likely going to stay a legal grey area for the foreseeable future. In OpenAI’s view, that doesn’t equate to an orange light, but a very green one. Ethics be damned.

Where we are heading 🛣️

As a fan and user of AI tools, I have mixed feelings about their impact on society. I think about it often and have even written about it on multiple occasions. In one particular post, from July of 2023, I wrote about how AI will eventually overtake humanity. Within the post, I added seven AI-generated images that were created using OpenAI’s DALL E-2. Below are three of them:

As terrible as these are, they are a representation of how good AI image generation was at that time – with the exception of MidJourney, which was marginally better and the quality frontrunner back then. I say “back then,” but this isn’t even two whole years ago.

And where are we now?

We’re here:

In. Less. Than. Two. Years. 🤯

Where are we headed?

In my opinion, we’re probably three iterations away from nearly flawless image generation. By then, OpenAI’s Sora might even be able to generate entire Studio Ghibli mini cartoons from a few prompts.

What that means for all of us is an entirely separate topic. But one thing is certain: wherever we’re headed, we’re heading there fast.

So get ready and buckle up.

OpenAI and the other tech giants aren’t letting any of us out of the car. Might as well figure out how you can enjoy the ride.

Are you a fan of Studio Ghibli? What do you think of the Ghibli AI trend and the new OpenAI image generator? Let me know in the comments below.