OpenAI Launches Slimmer, Cheaper GPT-4o Mini

• By John K. Waters
• 07/18/24

OpenAI has announced GPT-4o Mini, a slimmed down, more affordable version of its flagship multimodal GPT-4o model. The "mini" version, which replaces the GPT-3.5 model, is designed to "significantly expand the range of applications built with AI by making intelligence much more affordable," the company said in a statement.

Free and paid users of ChatGPT, including those on the Teams plan, will have access to GPT-4o mini today, and the company plans to roll it out to enterprise customers next week.

This new model was designed to enable a broad range of tasks with its low cost and latency, the company said, including applications that require multiple model calls, large volumes of context, and real-time customer interactions. The company is supporting is "faster, cheaper" claim with some impressive benchmark test scores. The GPT-4o Mini scores 82% on the MMLU (Massive Multitask Language Understanding) benchmark for evaluating the capabilities of language models, and outperforms GPT-4 on chat preferences in the LMSYS Chatbot Arena leaderboard.

Key benchmark results for GPT-4o Mini include:

• Reasoning Tasks: 82.0% on MMLU, compared to 77.9% for Gemini Flash and 73.8% for Claude Haiku.
• Math and Coding Proficiency: 87.0% on MGSM (math reasoning) and 87.2% on HumanEval (coding performance), outperforming Gemini Flash and Claude Haiku.
• Multimodal Reasoning: 59.4% on MMMU, leading Gemini Flash and Claude Haiku.

Currently, GPT-4o Mini supports text and vision inputs through the API, and the company plans to include text, image, video, and audio inputs and outputs in the future. The model features a 128K token context window and up-to-date knowledge to October 2023. The improved tokenizer shared with GPT-4o is meant to enhance cost-effectiveness for handling non-English text.

OpenAI has built comprehensive safety measures into GPT-4o Mini that align with its Preparedness Framework and voluntary commitments. More than 70 external experts evaluated GPT-4o to identify potential risks, and their insights have improved the safety of both GPT-4o and GPT-4o Mini, the company says.

GPT-4o Mini is the first model to apply OpenAI's instruction hierarchy method, which enhances its ability to resist jailbreaks, prompt injections, and system prompt extractions. This innovation makes the model's responses more reliable and safer for large-scale applications, the company says.

OpenAI isn't the first vendor to offer a lightweight versions of its main product offering. Google has Gemini Nano and Anthropic has Claude Haiku.

GPT-4o Mini is available now to ChatGPT Free, Plus, and Team users starting today, and will be accessible to enterprise users next week. Fine-tuning options for GPT-4o Mini will be available in the coming days.