A leaked Google memo provides a point-by-point summary of why Google is losing to open-source AI and suggests a path back to platform dominance and ownership.

The memo begins by confirming that their competitor has never been OpenAI and will always be open source.

Cannot compete with open source

Furthermore, they admit that they are in no way positioned to compete against open source and concede that they have already lost the battle for AI dominance.

They write:

“We looked over our shoulders a lot at OpenAI. Who will cross the next milestone? What will be the next step?

But the uncomfortable truth is that we are in no position to win this arms race, and neither is OpenAI. As we argue, a third faction quietly eats our lunch.

Of course I’m talking about open source.

Put simply, they lap us. Things that we consider to be ‘big open problems’ are now resolved and in people’s hands.”

Most of the memo is spent describing how open source is trumping Google.

And while Google has a slight advantage over open source, the memo’s author concedes that it’s ebbing away and will never come back.

Self-analysis of the metaphorical cards they’ve dealt themselves is quite dejected:

“While our models still have a small head start in terms of quality, the gap is closing surprisingly quickly.

Open-source models are faster, more customizable, more private, and pound for pound more powerful.

They’re doing things with $100 and $13B parameters that we struggle with at $10M and $540B.

And that in weeks, not months.”

Large language models are not an advantage

Perhaps the most startling realization expressed in the memo is that Google’s size is no longer an asset.

The unusually large size of their models is now seen as a disadvantage, and not at all the insurmountable advantage they thought it was.

The leaked memo lists a series of events that suggest Google’s (and OpenAI’s) control over AI could be over quickly.

It is reported that barely a month ago, in March 2023, the open source community received a leaked major language open source model developed by Meta called LLaMA.

Within days and weeks, the global open source community developed all the necessary building blocks to create Bard and ChatGPT clones.

Sophisticated steps like instruction tuning and human feedback reinforcement learning (RLHF) were quickly replicated by the global open source community, no less cheaply.

• Voting Instructions
  A process of fine-tuning a language model to do something specific that it was not originally trained to do.
• Reinforcement Learning from Human Feedback (RLHF)
  A technique in which humans evaluate the output of a language model so that it learns which outputs are satisfactory to humans.

RLHF is the technique used by OpenAI to create InstructGPT, a model underlying ChatGPT that allows the GPT-3.5 and GPT-4 models to take instructions and complete tasks.

RLHF is the fire that open source has taken over

Scope of open source scares Google

What particularly scares Google is the fact that the open source movement can scale its projects in ways that closed source cannot.

The question and answer dataset used to create the open source ChatGPT clone Dolly 2.0 was created entirely by thousands of volunteer collaborators.

Google and OpenAI relied in part on questions and answers from sites like Reddit.

The open source question and answer dataset created by Databricks is said to be of higher quality because the people who contributed to its creation were professionals and the answers they provided were longer and more meaningful than what which is found in a typical question-and-answer record scraped from a public forum.

The leaked memo noted:

“In early March, the open source community got their hands on their first truly powerful base model when Metas LLaMA was leaked to the public.

It had no instruction or conversation vote and no RLHF.

Nonetheless, the community immediately understood the importance of what they had been given.

A tremendous tide of innovation followed, with only a few days between major developments…

Here we are, barely a month later, and there are flavors with instruction tuning, quantization, quality improvements, human scoring, multimodality, RLHF, etc. etc., many of which build on top of each other.

Most importantly, they’ve solved the scaling problem to the point where anyone can tinker.

Many of the new ideas come from ordinary people.

The barrier to entry for training and experimentation has dropped from the overall performance of a large research organization to one person, one evening and a beefy laptop.”

In other words, what took Google and OpenAI months and years to train and build took just days for the open source community.

That must be a really scary scenario for Google.

This is one of the reasons I’ve written so much about the open source AI movement, as it really looks like the future of generative AI is fairly close at hand.

Open source has historically overtaken closed source

The memo cites OpenAI’s recent experience with DALL-E, the deep-learning model used to create images, versus open-source stable diffusion as a harbinger of what’s currently affecting generative AI like Bard and ChatGPT.

Dall-e was released by OpenAI in January 2021. Stable Diffusion, the open-source version, was released a year and a half later in August 2022 and overtook Dall-E in popularity in a matter of weeks.

This timing chart shows how quickly Stable Diffusion overtook Dall-E:

The Google Trends timeline above shows how interest in the open-source stable-diffusion model far exceeded that of Dall-E within three weeks of its release.

And even though Dall-E had been out for a year and a half, interest in Stable Diffusion continued to grow exponentially while OpenAI’s Dall-E stagnated.

The existential threat of similar events overtaking Bard (and OpenAI) is giving Google nightmares.

The creation process of the open source model is superior

Another factor that alarms engineers at Google is that the process of creating and improving open source models is fast, inexpensive, and perfectly suited to a global collaborative approach common to open source projects.

The memo notes that new techniques such as LoRA (Low-Rank Adaptation of Large Language Models) allow language models to be fine-tuned in a matter of days at an extremely low cost, with the final LLM being comparable to the extremely expensive LLMs created by Google and OpenAI.

Another benefit is that open-source engineers can build on and iterate on previous work, rather than having to start from scratch.

Creating large language models with billions of parameters like OpenAI and Google did is no longer necessary today.

This might be the point Sam Alton alluded to recently when he said that the era of massive large language models is over.

The author of the Google memo contrasted the cheap and fast LoRA approach to creating LLMs with the current big AI approach.

The memo author reflects on Google’s shortcoming:

“By contrast, training huge models from scratch not only throws away the pre-training, but also any iterative improvements made on top of it. In the open source world, it doesn’t take long for these improvements to become dominant, making complete retraining extremely costly.

We should think about whether each new application or idea really needs a whole new model.

… Indeed, the rate of improvement of these models in terms of engineering hours far exceeds what we can achieve with our largest variants, and the best are already largely indistinguishable from ChatGPT.”

The author concludes by recognizing that what they believed to be their advantage, their huge models and the prohibitive costs involved, were actually a disadvantage.

The global, collaborative nature of open source is more efficient and orders of magnitude faster to innovate.

How can a closed source system compete against the overwhelming multitude of engineers around the world?

The author concludes that they cannot compete and that, in their words, direct competition is a “losing bid”.

This is the crisis, the storm, unfolding outside of Google.

If you can’t beat open source, join them

The only solace the memo author finds in open source is that because open source innovations are free, Google can benefit from them too.

In conclusion, the author concludes that the only approach open to Google is to own the platform in the same way they dominate the Chrome and Android open-source platforms.

They point out how Meta benefits from releasing their large LLaMA language model for research and how they now have thousands of people doing their work for free.

Perhaps the big takeaway from the memo is that in the near future, Google could look to replicate its open-source dominance by releasing its projects to open-source, thereby owning the platform.

The memo concludes that moving to open source is the most viable option:

“Google should establish itself as a leader in the open source community and take the lead by collaborating with, rather than ignoring, the broader discussion.

This likely means taking some awkward steps, such as: B. the publication of the model weights for small ULM variants. This inevitably means giving up control of our models.

But this compromise is inevitable.

We cannot hope to both drive innovation and control it.”

Open source is going away with the AI ​​fire

Last week, I made a nod to the Greek myth of the human hero Prometheus stealing fire from the gods on Mount Olympus, and pitted the Prometheus open source against the “Olympic gods” of Google and OpenAI:

I tweeted:

“While Google, Microsoft, and Open AI bicker and turn their backs on each other, does open source walk away with its fire?”

The leak of Google’s memo confirms this observation, but also points to a possible shift in strategy by Google to join the open source movement and thereby co-opt and dominate it, just as they did with Chrome and Android.

Read the leaked Google memo here:

Google “We don’t have a moat and neither does OpenAI”