Meta Andromeda: ads retrieval’s bottleneck moved from FLOPs to memory/network co-design

On 28 October 2026 Meta Engineering detailed the next Meta Andromeda ads-retrieval stack: near ranking-stage model quality at retrieval scale, with claimed throughput above one billion ads per second on a single GPU server—roughly 100× the 2024 system.

The real thesis is not “another bigger tower.” As generative creative tools swell the eligible pool, the bottleneck shifts from model FLOPs to memory bandwidth and cross-host networking. Near-ranking quality at retrieval is a fused model–system–GPU problem.

Retrieval is the first stage of ads recommendation: narrow a vast eligible pool to strong candidates in milliseconds before a costlier ranker. The 2024 Andromeda launch on NVIDIA GH200 already grew model capacity by four orders of magnitude with hierarchical indexing and GPU-native operators, but still evaluated tens of millions of ads per second per card. Business demand now points at multi-billion ads/sec per card.

The new Retrieval Mixture-of-Experts uses ensembles of user-expert towers (sequential engagement, cohorts, LLM semantics, cross-surface co-engagement), a context-fused ads tower, and an MoE attention gating overarch. Meta says quality approaches ranking-level performance in a typical retrieval-then-ranking system at ~300× fewer inference-time FLOPs than the 2024 retrieval model.

Two system moves: fuse embedding gather, interaction and scoring into one kernel so intermediates stay in registers/shared memory instead of HBM; co-locate index matching, scoring and top-K on one GPU box so only a compact user/targeting payload crosses the network and only top-K returns—~100× less traffic. The fused kernel exceeds 10B ads/sec on a single GH200; the co-located production pipeline exceeds 1B ads/sec while scoring every eligible ad.

A Meta–NVIDIA collaboration early in 2026 pushed the gating kernel from 2.4B to >10B ads/sec (~5.3×, ~85% of peak HBM bandwidth); INT8 adds up to ~2× over optimized FP16; index matching more than doubled to >100B ads/sec. Advertiser outcome cited: ~2.4% conversion lift on Facebook since 2025.

Steelman: the 100× and 300× figures are vs Meta’s own 2024 baseline, not a cross-vendor leaderboard; 2.4% conversion is a compound business result hard to pin on one kernel; co-location may not port cleanly to smaller ad platforms.

Still thin: public end-to-end latency distributions, the quality-alignment protocol vs the ranker, quantified Grace Blackwell CPU/GPU ratio results, and the growth rate of the generative ad pool. Confidence in the engineering claims is high; confidence that every recommender must adopt a fused GPU pipeline is medium.

For advertisers, a larger candidate pool fits the same latency budget and conversion gains land in retrieval. For infra buyers, memory/network co-design reframes the next purchase metric beyond TFLOPS toward HBM utilization and whether targeting logic can live on GPU. For other recommenders, when candidate scale explodes, a wider dense tower may already be the wrong Pareto move.

Watch whether INT8 and extra retrieval pathways reach production without hurting conversion; whether Grace Blackwell CPU–GPU ratios show a knee for Andromeda-like loads; and whether rivals keep pouring FLOPs into ranking—or start competing on “near-ranking quality at retrieval + billion ads/sec.”