Gemini vs GPT-4: How Google's and OpenAI's Models Really Compare

Gemini (Google) and GPT-4 (OpenAI) are both large transformer-based language models, and on most everyday tasks they perform at a similar level. The real differences are in how they handle images, audio and video, how much text they accept in one go, and which products and clouds they plug into. Which one is "better" depends on your task, and the answer has changed with almost every release since 2023.

I published the first version of this comparison two days after Gemini was announced. A few of the claims floating around that week turned out to be wrong, so I have rewritten it to stick to what is documented.

When were GPT-4 and Gemini released?

• GPT-4: released by OpenAI on 14 March 2023. It powered ChatGPT Plus and Microsoft's Bing Chat (later Copilot).
• Gemini 1.0: announced by Google DeepMind on 6 December 2023, in Ultra, Pro and Nano sizes.

Both families have had several newer versions since. OpenAI released GPT-4o in May 2024, and Google released Gemini 1.5 in February 2024, followed by later generations. If you are reading this to choose a model today, compare the current versions using each vendor's documentation, not the 2023 names.

How do their architectures differ?

Both are transformer models, the architecture introduced by Google researchers in 2017. Neither company has published full architecture details or parameter counts for its flagship models.

The one documented design difference at launch was multimodality. Google trained Gemini on text, images, audio and video together from the start. GPT-4 launched as a text model with image input added (as GPT-4V), and OpenAI's later GPT-4o moved to a single model that handles text, audio and images natively. So by mid 2024 this gap had largely closed.

You may have seen posts saying Gemini mixes recurrent neural networks with transformers. Google's technical report doesn't say that, and I'd ignore it.

Which model scored higher on benchmarks?

Google's Gemini report gave Gemini Ultra 90.0% on MMLU against 86.4% for GPT-4. The catch is that Gemini's number used chain-of-thought prompting with 32 samples, while GPT-4's was a standard 5-shot score. Under the same 5-shot setup, the report showed Gemini Ultra at 83.7%. That is close enough that the honest summary is "roughly level", with each model ahead on different tests.

Benchmarks also go stale fast. Models get tuned for popular tests, and newer evaluations replace older ones. For your decision, a test on twenty of your own real prompts is worth more than any leaderboard.

How much text can each model handle?

Context length is the amount of text (measured in tokens) a model can consider at once. The original GPT-4 offered 8,000 and 32,000-token versions, and GPT-4 Turbo raised this to 128,000 in November 2023. Gemini 1.5 Pro previewed a one-million-token window in February 2024.

In practice a long context lets you paste a whole policy manual or a long transcript and ask questions about it. It doesn't guarantee the model pays equal attention to every page, so still check answers against the source.

Which is better for everyday work?

From what I see in our training batches in Anand, the choice usually comes down to where you already work:

• If your company lives in Google Workspace (Gmail, Docs, Sheets), Gemini is built into those tools.
• If you work in Microsoft 365, Copilot uses OpenAI models inside Word, Excel, Outlook and Teams.
• For coding help, both are capable. Try each on your own code base.
• For data work, both can explain SQL, DAX or Python. Neither should be trusted with numbers you haven't checked.

What limitations do both share?

Both can produce wrong facts with total confidence. Both reflect biases from their training data. Both send your prompts to the vendor's servers unless you use an enterprise setup with different terms. And both change often enough that a workflow tuned to one version may behave differently after an update.

How should you choose between Gemini and GPT-4?

Pick three or four tasks you actually do each week. Run them on both, with the same prompts. Score the answers for accuracy and how much editing they needed. Then factor in cost, data policy and which tools your team already uses.

We do this exercise in our Generative AI training at TechNet Consultancy, because learners remember a comparison they ran themselves much better than one they read. For background on each tool, see what Google Gemini is and our guide to using ChatGPT well.

FAQ

Is Gemini better than GPT-4?

At launch they were roughly level, with each ahead on different benchmarks. Later versions of both have leapfrogged each other. Test them on your own tasks.

Was Gemini Ultra really better than human experts?

Google reported 90.0% on MMLU, above the 89.8% human-expert figure, but using a prompting method that differed from how GPT-4's score was measured. It is a narrow benchmark result, not proof of general expert ability.

Do Gemini and GPT-4 use the same technology?

Both are transformer-based large language models trained on very large datasets and then tuned for helpful behaviour. Their exact architectures and training data are not public.

Which one is cheaper for developers?

Both offer several model sizes at different prices per token, and prices drop regularly. Compare current pricing pages from Google AI and OpenAI for the specific models you need.

Can I use both in the same project?

Yes. Many teams route simple tasks to a cheaper model and harder ones to a larger model, sometimes from different vendors.