I'm Nitin Kumar, Lead Project Coordinator at Debugged Pro. I help software and IT agencies in India and the USA turn research releases into delivery plans clients can approve. Olmo-core 3 landed on October 1, 2026, and it is already showing up in MoE training conversations.

Ai2 (Allen Institute for AI) released Olmo-core 3 as a redesigned open mixture-of-experts training system. The goal is clear. Scale MoE training toward the trillion-parameter range without giving up efficiency.

SiliconANGLE covered the same release the next day. This brief sticks to those public facts plus the open GitHub stack. No invented quotes. No fake customer wins.

Olmo-core 3 open MoE training infrastructure visual from Ai2 blog — Feature image: visual from Ai2's Introducing Olmo-core 3 blog post. Credit: Allen Institute for AI / Ai2.

Key takeaways on Olmo-core 3 for agency teams

Olmo-core 3 is Ai2's open MoE training upgrade, announced October 1, 2026.
Expert pool grew 8 to 128 while still selecting 4 experts per token; active params stayed near 3.2B; total capacity rose 4.6B to 47B; throughput fell under 5%.
Same stack benchmarked at over 1 trillion total parameters.
Switched from earlier FSDP MoE path to a DDP-based system that keeps experts resident and routes data to them.
Preliminary test: 8 NVIDIA B300 GPUs, 47B MoE: 52,000 tok/s/GPU vs 19,400 earlier (~2.7×).
MXFP8 on 4 B300s: ~21% higher throughput vs BF16; peak active memory 103→95 GiB.
Stack is open on GitHub allenai/OLMo-core; technical report at allenai.org/papers/olmocore3.

What Olmo-core 3 actually is

Olmo-core is Ai2's framework for developing large language models. Olmo-core 3 is a major upgrade focused on how MoE models train at scale.

MoE models hold many more parameters than they activate for each token. That can cut compute per token. The full model still lives across GPU memory. Routing tokens to the right experts adds communication cost.

Ai2 built Olmo-core 3 to close that gap. In one published benchmark, the expert pool grew from 8 to 128. The run still selected only four experts per token. Active parameters stayed near 3.2B. Total capacity rose from 4.6B to 47B. Training throughput fell by less than 5%.

The same infrastructure has been benchmarked at over one trillion total parameters. That is the headline agencies need when clients ask if open MoE stacks can even reach that class of scale.

How Olmo-core 3 changed the training design

From FSDP gathering to resident experts

Earlier MoE work in Olmo-core used fully sharded data parallelism (FSDP). That path gathered and resharded weights for each small batch.

Olmo-core 3 switches to a distributed data parallelism (DDP) based system. It keeps experts resident on GPUs. It routes the relevant data to them. That avoids repeated weight gathering.

Ai2 notes that NVIDIA Megatron-Core is an established option for large MoEs. Olmo-core 3 brings an integrated MoE stack into the framework behind Olmo. The redesign improved throughput over Ai2's earlier FSDP-based implementation.

Abstract GPU cluster visualization representing distributed MoE training — Body image: abstract GPU cluster illustration for MoE training context. Credit: Debugged Pro generated visual.

Parallelism and routing optimizations

Three techniques decide how the model and training state split across hardware:

Expert parallelism spreads experts across GPUs so each card stores only part of the pool.
Pipeline parallelism splits layers across GPU groups to cut memory per card.
Distributed optimizer spreads optimizer state across GPUs instead of full copies everywhere.

Olmo-core 3 also cuts routing and compute overhead with:

Rowwise expert parallelism places routed data into expert input buffers.
GPU-resident routing keeps routing metadata on GPUs so the CPU can queue work without waiting for copies.
Grouped GEMM combines many small expert computations for better GPU utilization.
MXFP8 support for lower-precision values where conversion cost does not wipe out the savings.

Benchmark numbers agencies can put in a brief

Use only published figures. Do not round into marketing fluff. Here are the numbers from Ai2's announcement and technical framing.

47B MoE on 8 B300 GPUs: 52,000 tokens/sec/GPU with the new stack vs 19,400 earlier (~2.7×).
MXFP8 vs BF16 on 4 B300s: about 21% higher end-to-end throughput; peak active memory 103 GiB to 95 GiB.
1.2T param model: 58.36B active per token across 512 GPUs; peak 858 TFLOP/s/GPU (random routing for systems performance).
DeepEP v2 experiment: 2.38T total parameters in a short capacity test, not a full training run.

SiliconANGLE also places the release in context for Seattle-based Ai2 and notes GitHub availability. Treat SiliconANGLE as secondary coverage. Prefer Ai2's own blog and technical report for exact system claims.

Diagram-style illustration of tokens routing to selected MoE experts — Body image: simplified MoE routing concept illustration. Credit: Debugged Pro generated visual.

Documented findings that matter for delivery risk

Ai2's technical report does not only celebrate wins. It documents training and measurement findings agencies should mention in discovery:

Token gerrymandering: a score meant to encourage balanced routing could improve even as the real workload became less balanced.
Lowering expert learning rates because experts see fewer tokens did not help in the tested model family.
GPU timing can depend on values, not only matrix shapes. Fair comparisons need matching inputs.
Overlapping communication and compute on separate streams did not always speed training. In some tests it slowed end-to-end runs.

Those caveats are useful client language. They show the stack is open about trade-offs, not just peak charts.

History: OlmoE, dense Olmo 3, then Olmo-core 3

Ai2's sparse-model work goes back to OlmoE, which used an MoE architecture with 64 routed experts. Olmo 3, by contrast, used a dense architecture. Nearly all of the model was active for every token. Its training stack followed that dense design.

Olmo-core 3 extends the framework for much larger MoE models. Ai2 says the next-generation Olmo will use an MoE architecture. The team aims for its most capable Olmo yet, on its largest dataset and longest context window.

When India and USA agencies should evaluate Olmo-core 3

Not every client needs to train a trillion-parameter MoE. Many do need a clear answer when research or platform teams ask about open MoE infrastructure.

Good fit signals

The client already runs or funds LLM training research and wants an open stack to inspect.
Cost models depend on active parameters per token, not only total parameter count.
The team can access multi-GPU clusters and cares about expert parallelism and memory layout.
Procurement wants GitHub-visible code plus a public technical report, not only closed vendor decks.

Caution signals

The SOW is inference-only product work with no training budget.
The client expects production MoE quality guarantees from a systems benchmark that used random routing.
Hardware plans do not match B300-class assumptions used in published tests.
Compliance teams need a named support SLA that an open research stack does not provide by itself.

If you are shaping a broader LLM delivery roadmap, start from our software development consulting page. Also scan related open-stack notes on the Debugged Pro blog, including the Strands Decider agency guide for agent routing decisions.

What to tell clients in India and the USA

Separate systems scale from model quality. Ai2's 1.2T run used random routing to measure systems performance, not trained quality.
Quote published throughput with hardware context. Say eight B300 GPUs and 47B MoE when you cite 52,000 tok/s/GPU.
Call out MXFP8 as optional. The ~21% gain and memory drop are controlled-benchmark results, not a universal promise.
Point to GitHub and the technical report. Let technical stakeholders read the ablations themselves.
Plan a narrow pilot. Validate expert parallelism, checkpointing, and monitoring on the client's actual cluster before promising a training timeline.

Read the primary source on the Ai2 Olmo-core 3 blog. For implementation details, open the allenai/OLMo-core repository and the technical report.

Practical delivery checklist

Confirm whether the client wants research experimentation, full pretraining, or continued pretraining of an existing MoE.
Map GPU count, interconnect, and storage bandwidth against Ai2's published configurations.
Decide early if MXFP8 is in scope for the pilot cluster.
Add acceptance criteria for tokens/sec/GPU and peak memory, with matching input values for fair timing.
Document routing balance checks so token gerrymandering cannot hide behind a green score.
Keep change control for parallelism settings. Overlap tweaks can slow end-to-end runs.
Assign an owner for open-source updates from the OLMo-core repo.

Cost and throughput tradeoffs in plain terms

MoE training can look cheaper because only a few experts run per token. Memory and networking still scale with the full expert pool. Olmo-core 3's published story is that expert count can rise sharply while active compute stays roughly fixed, with modest throughput loss.

For India delivery teams, that often means better utilization of limited GPU hours if the cluster supports expert and pipeline parallelism. For USA teams with larger clusters, the same facts help justify multi-node MoE pilots without jumping straight to closed stacks.

Either way, cost models should include communication overhead, optimizer state sharding, and precision conversion cost. Ai2 is explicit that speeding one part of training can raise costs elsewhere.

Key Takeaways

Olmo-core 3 is open MoE training infrastructure from Ai2, released October 1, 2026.
Published scaling kept active params near 3.2B while total capacity rose to 47B with under 5% throughput loss.
DDP-based resident experts delivered about 2.7× throughput vs Ai2's earlier FSDP MoE path on an 8×B300 47B test.
MXFP8, expert parallelism, pipeline parallelism, and grouped GEMM are core optimization levers.
Agencies should cite hardware context, treat random-routing TFLOP numbers as systems metrics, and pilot before promising production MoE quality.

FAQ

What is Olmo-core 3?

Olmo-core 3 is Ai2's October 1, 2026 upgrade to its open training framework for large language models. It features a redesigned mixture-of-experts (MoE) training system meant to scale toward trillion-parameter models while keeping computational efficiency. The stack is open on GitHub under allenai/OLMo-core.

How does Olmo-core 3 improve MoE training throughput?

Ai2 switched from an earlier FSDP-based MoE path to a DDP-based system that keeps experts resident on GPUs and routes data to them. In a preliminary test on eight NVIDIA B300 GPUs, a 47B MoE reached about 52,000 tokens per second per GPU versus about 19,400 with the earlier stack, or roughly 2.7 times the throughput.

What scale has Olmo-core 3 been benchmarked at?

Ai2 reports a benchmark that grew the expert pool from 8 to 128 while still selecting 4 experts per token, with active parameters near 3.2B and total capacity from 4.6B to 47B, while throughput fell by less than 5%. The same infrastructure was benchmarked above one trillion total parameters, including a 1.2T configuration on 512 GPUs. A short DeepEP v2 capacity test reached 2.38T total parameters.

Does Olmo-core 3 support MXFP8?

Yes. On four B300 GPUs in a controlled benchmark, MXFP8 raised end-to-end training throughput by about 21% versus BF16, and peak active memory fell from 103 GiB to 95 GiB. Most of the gain came from feed-forward compute and moving data between experts rather than attention alone.

Why should India and USA agencies care about Olmo-core 3?

Clients in both markets ask when open MoE training is cheaper than dense training, and what stack risks belong in an SOW. Olmo-core 3 gives a public, open reference with published throughput and memory numbers. Agencies can use those facts to set realistic pilots, GPU budgets, and research partnership language without inventing vendor claims.

About the author

Nitin Kumar is Lead Project Coordinator at Debugged Pro. He works with software and IT agencies across India and the USA on delivery planning, vendor evaluation, and client-ready technical briefs.