At the bottom of the Atlantic Ocean, nestled between tectonic plates and silent abyssal plains, lies a series of high-capacity fiber optic threads no thicker than a garden hose. To a casual observer, they are just infrastructure. To Meta’s engineering teams, they are the specialized vascular system of a global brain.
As we pivot from a company built on social graphs to one built on generative intelligence, the fundamental constraints of our architecture have shifted. We are no longer just optimizing for “Time to First Byte” for a static JPEG on a newsfeed. We are now battling the Latency Tax—the unavoidable physical cost of moving massive model weights, KV (Key-Value) caches, and synchronized state across a planetary-scale compute fabric.
To solve this, Meta isn’t just building faster servers; we are building our own oceanside infrastructure. This is the story of how our subsea cable investments—like Anaximander and 2Africa—are fundamentally reshaping global zonal replication and making real-time, cross-continent AI inference possible.
The Physics of the Tax: Why Light Isn’t Fast Enough
In the world of high-performance distributed systems, the speed of light is a frustratingly slow constant. Light in a vacuum travels at roughly 300,000 km/s. In a silica fiber optic core, due to the refractive index, that speed drops by about 30%, landing at roughly 200,000 km/s.
For a round-trip from a data center in Ashburn, Virginia, to one in Dublin, Ireland, you’re looking at a theoretical physical minimum (the “great circle” distance) of about 60–70ms. But in the real world of legacy “hop-heavy” subsea infrastructure, that latency often balloons to 100ms or more due to switching, amplification, and circuitous routing.
The AI Inference Problem
Why does this matter for AI? Because inference is no longer a localized event. When a user in London interacts with a Llama-powered agent, the request doesn’t just hit a local cache. To provide a personalized, context-aware experience, the system may need to:
- Retrieve User Context: Pull vector embeddings from a global database.
- Synchronize State: Ensure the inference node has the latest model fine-tuning or RAG (Retrieval-Augmented Generation) updates.
- Manage KV Caches: In multi-turn conversations, the “memory” of the chat (the KV cache) needs to be accessible. If the user’s session migrates from one region to another (due to load balancing or failure), moving that cache across a high-latency link creates a “stutter” that destroys the user experience.
This is the Latency Tax. If our cross-region replication is too slow, we are forced to choose between Consistency (waiting for the data) or Availability (serving a stale response). For Meta’s next-gen AI, neither is acceptable.
Rewiring the Abyssal Plain: SDM and the Anaximander Project
Most people think of “the cloud” as a wireless entity, but it is deeply grounded in the mud of the ocean floor. Traditionally, subsea cables were built by consortia of telecom giants. These cables were designed for voice and general internet traffic, prioritizing total capacity over the specific low-jitter, high-burst needs of a synchronized compute cluster.
Meta’s shift toward owning and designing its own cables—most notably the Anaximander system (connecting the US to Europe)—represents a leap in optical engineering.
1. Space Division Multiplexing (SDM)
Traditional cables hit a “power wall.” You can only pump so much light (and thus data) through a single fiber pair before you run into non-linear effects that distort the signal.
Meta’s new cables utilize Space Division Multiplexing (SDM). Instead of trying to squeeze more bits through a few fibers using massive amounts of power, we increase the number of fiber pairs.
- Legacy Cables: 8 to 12 fiber pairs.
- Meta’s SDM Cables: 24+ fiber pairs.
By spreading the data across more physical “lanes,” we can operate at lower optical power per fiber. This reduces the heat in the repeaters (the underwater amplifiers) and allows us to achieve a massive jump in total system capacity—reaching half a Petabit per second (Pbps) on a single cable.
2. Eliminating the “Middleman” Latency
By owning the “wet plant” (the actual cable), Meta’s Network Infrastructure team can bypass traditional carrier exchange points. We terminate these cables directly into our Edge PoPs (Points of Presence) which are linked via dedicated terrestrial backhaul to our massive Hyper-scale Data Centers (HDCs).
This reduces the “Logical Hop Count.” In networking, every router is a queue. Every queue is a source of jitter. By flattening the topology from the ocean floor to the GPU cluster, we bring the P99 latency closer to the theoretical limit of the speed of light in glass.
Global Zonal Replication: Moving Beyond “Eventually Consistent”
In a standard web app, “eventual consistency” is fine. If your “Like” count takes 500ms to update across the globe, no one dies. But AI inference clusters are different. They are state-heavy.
The Challenges of Global Sharding
When we deploy a model like Llama 3 across multiple global regions (zones), we shard the data. However, the Control Plane that manages these shards must be highly synchronized.
Imagine a “Zonal Replication” scenario:
- Zone A (Oregon, USA): Holds the master weights and the latest fine-tuned gradients.
- Zone B (Denmark): Is currently serving 100,000 concurrent inference streams.
If a new safety filter or a “system prompt” update is pushed, it must be replicated to Zone B instantly. If the subsea link has high jitter, the replication lag (the “Replication Gap”) increases. In an AI context, a 200ms gap could mean the difference between a model responding with outdated information or a blocked hallucination.
Syncing the “KV Cache”
The most technical hurdle in global AI is the KV Cache Migration. For long-form conversations, the GPU stores the intermediate states of the transformer layers (the Key and Value tensors) to avoid re-calculating the entire chat history for every new token generated.
If a user moves from a mobile connection to a Wi-Fi connection and their request gets routed to a different geographic data center, we have two choices:
- Re-compute: Waste thousands of GPU cycles calculating the history again.
- Migrate: Send the KV cache over the subsea cable.
With Meta’s new high-capacity, low-latency subsea links, we can treat global regions as a loosely coupled memory fabric. We use custom protocols built on top of RDMA (Remote Direct Memory Access)—translated over long-haul WDM (Wavelength Division Multiplexing)—to “shoot” these tensors across the ocean in milliseconds.
The Software Stack: From BGP to Intelligent Routing
Building the physical cable is only half the battle. The software layer—how we route the bits—is where the magic happens. Meta uses a combination of Express Backbone (EBB) and custom traffic engineering to ensure AI traffic gets the “HOV lane.”
BGP is Not Enough
Standard Border Gateway Protocol (BGP) is “path vector” based; it cares about the number of network hops, not the quality of the fiber. Meta’s internal backbone uses an SDN (Software Defined Networking) controller that monitors the optical layer in real-time.
# Conceptual logic for Meta's AI-aware routing
def get_best_subsea_path(request_type, user_location):
paths = global_topology.get_available_paths(user_location, "datacenter_cluster")
if request_type == "AI_INFERENCE_SYNC":
# Prioritize path with lowest jitter and highest OSNR (Optical Signal-to-Noise Ratio)
return min(paths, key=lambda x: x.jitter_p99 + x.propagation_delay)
else:
# Standard traffic can take the high-throughput, higher-latency path
return max(paths, key=lambda x: x.available_bandwidth)By differentiating between “Bulk Replication” (moving training logs) and “Synchronous AI State” (moving inference weights), we can saturate our subsea cables without impacting the user-facing latency.
The “AI Cluster” as a Continental Entity
With the deployment of these new cables, the definition of a “Data Center” is changing at Meta. We are moving away from the idea of isolated buildings and toward “Continental Availability Zones.”
MTIA and the Subsea Link
Meta’s custom AI chip, MTIA (Meta Training and Inference Accelerator), is designed for high-efficiency inference. But even the best silicon is throttled by the “I/O bottleneck.”
By integrating our subsea fiber terminations closer to our MTIA clusters, we enable a concept called Global Model Parallelism. In this setup:
- The Embeddings might live in a high-memory cluster in the US East.
- The Attention Layers might be processed on an MTIA cluster in Europe.
- The Output Logits are streamed back to the user via a local Edge PoP.
This sounds like a recipe for a latency nightmare, but with sub-80ms round-trip times and zero-packet-loss fiber, it becomes a viable way to balance the global GPU load. If California is at peak usage during the day, we can “offload” the compute to Europe (where it’s night) without the user ever knowing their tokens are being generated 5,000 miles away.
Engineering for Resilience: The “Shark and Trawler” Problem
The ocean is a hostile environment. Fishing trawlers, anchors, and even the occasional curious shark can sever a cable. In the past, a cable cut meant a massive spike in latency as traffic failed over to congested, third-party routes.
Mesh Networking at Sea
Meta’s subsea strategy involves a “mesh” architecture. We don’t just lay one cable; we lay multiple redundant paths (e.g., Havfrue, Amitiê, and Anaximander).
- Optical Switching: Using ROADMs (Reconfigurable Optical Add-Drop Multiplexers), we can reroute light at the photonic layer in less than 50ms.
- Predictive Maintenance: We use specialized sensors on the fiber to detect “micro-strains.” If we see a cable is being dragged by an anchor before it actually snaps, our SDN controller can preemptively shift the AI inference state to another path.
Why the “Hype” is Actually Infrastructure
The tech world is currently obsessed with “GPU Moats”—the idea that the company with the most H100s wins. But the real moat is increasingly becoming the Interconnect.
You can buy GPUs. You can (theoretically) rent power. But you cannot easily build a 10,000km, 24-pair SDM subsea cable system. It requires years of diplomatic permits, specialized “cable-laying” ships, and deep expertise in optical physics.
The “hype” surrounding Meta’s AI capabilities often focuses on the Llama models themselves. But the technical substance—the “secret sauce”—is the fact that Meta is the only AI company that also operates a global-scale tier-1 telecommunications network.
The Impact on the End User
What does this mean for you?
- Lower Per-Token Latency: When you talk to an AI, the “stream” feels like a human conversation because the backend isn’t fighting for bandwidth with a Netflix stream on a congested public cable.
- Global Consistency: Your AI assistant remembers what you said 10 minutes ago, even if you’ve traveled across a border or your session has been handed off to a different continent’s data center.
- Reliability: AI-powered features (like real-time translation in glasses) require “always-on” connectivity. Meta’s subsea mesh ensures that a single fiber cut doesn’t break the illusion of intelligence.
The Next Frontier: Beyond the Cable
As we look toward the future, even 24-pair SDM cables might not be enough. Meta is already researching:
- Hollow Core Fiber: Where light travels through air-filled channels in the fiber, potentially reducing latency by another 30% (since light travels faster in air than in glass).
- Satellite-Subsea Integration: Using LEO (Low Earth Orbit) satellites to handle the “control plane” signals while the “data plane” stays on the massive subsea pipes.
The Latency Tax is a law of nature, but laws are meant to be optimized. By building the infrastructure from the ocean floor up to the GPU kernel, Meta is ensuring that the global AI engine isn’t just smart—it’s fast.
We are rewiring the world, one kilometer of deep-sea fiber at a time, to ensure that the bottleneck for AI isn’t the distance between us, but the speed of our own ideas.
Technical Specs for the Curious:
- Cable System: Anaximander / 2Africa
- Capacity: Up to 500 Tbps per system
- Technology: SDM (Space Division Multiplexing) with 24+ fiber pairs
- Target Latency (NY to London): < 65ms RTD (Round Trip Delay)
- Hardware Integration: Direct-to-Fabric (HDC to Subsea) via custom Edge PoPs