Overview
MiniMax M3 represents an attempt to combine several capabilities that are usually developed separately: coding, agentic execution, million-token context, and native understanding of images and video. Olive explains that the model has 428 billion total parameters but activates only 23 billion, while its MiniMax Sparse Attention architecture selects relevant context blocks before performing detailed attention calculations. This design reflects MiniMax's belief that long context becomes especially valuable when agents must retain tool responses and multi-round interactions. M3's multimodality is also unusual: instead of adding vision adapters after text pre-training, MiniMax trains text and visual understanding together from the first step. The team says it prevented the instability associated with that approach through work on the vision transformer, interleaved multimodal data, cleaning, masking, and reward modeling. Organizationally, MiniMax gives researchers—including interns—access to models and infrastructure so they can identify weaknesses, propose projects, and contribute improvements to releases. The company also uses its own models and research harnesses to automate data creation, kernel optimization, and other workflows. Although M3 still has multimodal limitations, MiniMax intends to continue open-sourcing models, scale beyond one trillion parameters, and explore model routing and multi-agent systems for more complex tasks.
Sections
Core Concepts
Terms used to explain M3's architecture, training method, and intended applications.
- MiniMax Sparse Attention is an architecture in which an index branch identifies important regions of the context and a sparse attention branch computes attention over the selected blocks.
- Native multimodality means training text and visual understanding together from the first step rather than attaching vision components after text pre-training.
- Interleaved data is naturally occurring training data in which images and videos are retained alongside text instead of being masked out.
- A research harness is an internal system that combines model capabilities with automated workflows for tasks such as data generation and kernel optimization.
Training and Capability Comparisons
Explicit contrasts between MiniMax's approach and common alternatives.
- Training multimodality from the first step is presented as more scalable and balanced than adding vision adapters after text pre-training, which MiniMax found could harm text performance and produce weaker visual convergence.
- Starting multimodal training from the beginning is contrasted with introducing it halfway through continued pre-training, whose results are described as highly sensitive to architecture, data mixture, and learning rate.
- Earlier MiniMax systems demonstrated 10-million-token document processing without agentic behavior, whereas M3 provides a one-million-token context designed for multi-round tool-using agents.
- APIs provide direct model access for technical users, while consumer applications give a much broader audience practical interfaces and scenarios through which to experience model capabilities.
Architecture, Scale, and Training Details
Specific technical claims and quantitative details stated during the interview.
- M3 contains 428 billion total parameters and activates 23 billion parameters per inference.
- M3 supports a one-million-token context window; earlier MiniMax models reportedly handled tasks involving 10 million tokens.
- Sparse attention separates high-level context selection from detailed computation over selected blocks.
- The native multimodal training recipe includes vision-transformer development, naturally interleaved text and visual data, data cleaning, masking, and reward modeling.
- MiniMax expects future model generations to exceed one trillion total parameters.
- Internal research harnesses automate workflows including kernel optimization, model bootstrapping, and data generation.
- MiniMax applications reportedly reach more than 300 million people across approximately 200 countries and are used by over one million companies.