A deep dive into Nvidia's Vera CPU and the Olympus cores that power it

DEEP DIVE For the first time, Nvidia has directly challenged Intel and AMD's CPU dominance. With the launch of Vera, the AI arms dealer aims to flog its standalone CPUs to as many hyperscalers and other cloud providers as it can.

Alibaba, ByteDance, Meta, Oracle, CoreWeave, Lambda, Nebius, and NScale have already signed up to deploy the chips in their respective clouds.

The follow-on to Nvidia's Grace CPU promises 88 custom Armv9.2 cores, 176 threads, support for up to 1.5 TB of LPDDR5X memory, and, critically, availability as a standalone platform independent of Nvidia's GPUs.

But beyond that, and a mountain of marketing about how it'll be the best CPU for everything AI, Vera's inner workings have largely remained a mystery until recently.

That changed late last month, when Nvidia released a whitepaper spilling the beans on its first fully-custom CPU, which is far weirder than anyone could have anticipated.

Ostensibly, Nvidia is aiming Vera at two key workloads: the first and least surprising is as the AI head node responsible for managing the GPUs in its upcoming Vera Rubin systems. The second, and more contentious, is as a host for AI agents, which, unlike the large language models (LLMs) that power them, don't actually run on GPUs.

From what we gather, much of Vera's core architecture is predicated on quashing pipeline and execution bottlenecks in order to make it more effective in these roles. But before we dive into Nvidia's Olympus core, let's revisit the chip itself.

Monolithic compute, multi-die memory and I/O

Here's a quick refresher on Vera's construction. Image credit Nvidia

Peel back Vera's rather substantial heat spreader and you'll find an assortment of chiplets responsible for I/O, memory, and compute. However, this isn't another rehash of the chiplet architecture popularized by AMD.

Unlike x86 processor makers, which spread dozens of cores across multiple dies, Vera's compute die is monolithic. All 88 cores are housed in one big chunk of silicon that from what we understand is fabbed on TSMC's 3nm process. Nvidia argues its monolithic compute architecture has benefits for both core-to-core bandwidth and latency compared to competing designs.

Surrounding the compute die is an assortment of chiplets, including eight LPDDR5x controllers, and what appears to be two distinct I/O dies, one responsible for PCIe 6.4 and CXL 3.1 connectivity and another dedicated to the chip's NVLink Chip-to-Chip interface.

In terms of package design, Vera is somewhat reminiscent of Amazon's Graviton 4 CPUs, which also combine a monolithic compute die while disaggregating the I/O and memory functionality to dedicated silicon.

The Vera CPU Superchip

Nvidia’s Vera CPU superchip packs 176 cores and 3 TB of LPDDR5x memory. Image credit Nvidia

Like most modern datacenter CPUs, Vera supports both single- and dual-socket configurations, the latter of which Nvidia calls the Vera CPU Superchip.

Unveiled at GTC in March, the Superchip features two Vera CPUs connected over NVLink-C2C at a blisteringly fast 1.8 TB/s of bidirectional bandwidth, for a total of 176 cores and 352 threads.

The two chips are fed by 16 SOCAMM2 LPDDR5x memory modules, which deliver 2.4 TB/s of aggregate memory bandwidth (1.2 TB/s each) — roughly twice the bandwidth of AMD's Turin Epycs, which were launched in 2024.

Nvidia's agentic AI reference designs call for cramming as many as 128 of these superchips (256 CPUs) totaling 22,528 cores and 384 TB of memory into a single liquid-cooled rack.

Summiting Olympus

Previously, Nvidia had relied on off-the-shelf CPU cores from Arm. For example, depending on the iteration, Nvidia's Grace CPUs used Arm's Neoverse V2 (GB200/300), Neoverse V3 (AGX Thor), or Cortex X925 and A725 cores (GB10), depending on which was most convenient.

With Vera, Nvidia has moved on to designing its own ARMv9.2-compatible core, called Olympus. In fact, it's up for debate how custom the chip really is. We've been told that the chip builds heavily on existing Arm IP, and looking at Olympus' architectural block diagram, we can see why.

Here's a block diagram of Nvidia's Olympus core architecture. Image credit Nvidia

Olympus features a 10-wide decoder and dispatch, eight integer arithmetic logic units (ALUs), six vector/FP pipelines (SVE 128), four load units, and 2x store units. So it's got a fat front and back end compared to the Zen 5 cores found in AMD's Turin Epycs or the Redwood Cove cores used by Intel's Granite Rapids Xeons.

But compared to off-the-shelf Arm cores, Olympus' front and back end aren't breaking any records. In fact, Olympus looks a lot like a heavily modified Cortex X925.

With that said, Nvidia claims Olympus features an entirely custom neural branch predictor, which is capable of exploring two branches simultaneously.

Branch prediction is key to performance in modern CPUs, and involves anticipating future code paths and executing down them before they're needed. By predicting two paths per cycle, Vera decreases the likelihood of a mispredict, theoretically boosting its performance in the process.

And from what we can tell, a lot of the customization that went into Vera was made explicitly to stamp out pipeline and execution bubbles in order to maximize the number of instructions per clock (IPC).

In particular, Nvidia claims its neural branch predictor delivers higher hit rates than history-based approaches, making it ideal for running branch-heavy workloads, like the Python scripts generated on the fly by AI code assistants.

Nvidia also claims to have made several innovations to the mid-core, again aimed at mitigating pipeline stalls. Memory renaming speeds up store-to-load dependency chains by allowing dependent instructions to execute before a load completes if the relationship to the data can be divined.

"This is particularly beneficial for pointer-heavy software, graph traversal, runtime frameworks, and complex object-oriented workloads," the company explained.

Nvidia also implemented a value prediction scheme which "identifies stable dependency chains and predicts future values before they are produced, allowing dependent instructions to execute speculatively while correctness is verified later."

This functionality, the company claims, helps to speed up repetitive software patterns and sequential data processing.

On the back end, Nvidia Olympus features a similar complement of ALUs and FP units as the X925, but they've been implemented differently. Eight of the ALUs are simple integer units which handle basic mathematics, while the company says two complex ALUs handle multiplication and division, CRC, and shift intensive workloads.

The back end also features four dedicated branch units that work with the chip's branch predictor to resolve control flow decisions that would otherwise cause pipeline stalls.

Meanwhile, for floating point operations, Olympus features six 128-bit Arm scalable vector engines (SVE2) extensions which also support FP8, along with a pair of vector units dedicated to speeding up crypto operations.

Each core is supported by a 64 KB L1 instruction cache, 96 KB L1 data cache, and 2MB per core of L2. The chip also features 164 MB of system level cache (SLC) that's sharded across the chip.

SMT makes a comeback on Arm

Perhaps Olympus' most interesting party trick is support for simultaneous multithreading (SMT).

SMT has been a mainstay of x86 chips since 2002. The technology boosts utilization by enabling two threads to harness idle execution units during a single cycle. While SMT doesn’t double throughput, for certain applications it can deliver double-digit percentage gains.

While Arm has supported SMT in the past — you may recall Cavium ThunderX2's four-way SMT implementation prior to its acquisition by Marvell — the company has downplayed its value with its own agentic CPU.

Arm's dislike for the tech didn't stop Nvidia from implementing SMT-like functionality in Olympus. We say SMT-like because it's not SMT in the conventional sense. Rather than simultaneous multi-threading, Nvidia markets it as spatial multithreading, which is an apt name given how it works.

From what we understand, Nvidia's SMT implementation is a bit more like core bifurcation than what Intel would call Hyperthreading. Image credit Nvidia

Rather than two threads that share the resources of one core, Olympus' SMT implementation looks more like core bifurcation with a shared cache line. You can either have one fat core or two skinny ones. To our eyes, it looks a bit like multi-instance GPU (MIG) for the CPU.

The white paper makes a big to-do about how spatial multithreading avoids the thread contention you can see with x86 parts, which makes sense, because Nvidia's threads behave more like individual cores. We wouldn't read too much into these claims, though. There have always been applications that benefit from SMT and those that don't, and it's easy enough to turn it off if you don't want it.

With that, Nvidia's SMT implementation could be beneficial to agent sandboxes for agentic workloads. With containers — that's all these sandboxes really are – there are two workloads. There's the job you're running inside the container – let's say a Python script and interpreter  – and then there's the container runtime, which is responsible for orchestration of container workloads and managing networking and storage overlays.

These are very different jobs. By bifurcating its cores, Nvidia can keep the workload and runtime separate, which in theory should make for more deterministic performance.

Fat pipes galore

Before we move on to Vera's memory subsystem, we need to spend a few minutes discussing Nvidia's second-gen scalable coherency fabric (SCF), which is used to stitch all 88 cores together.

As we mentioned earlier, Vera features a monolithic compute die, which means all of its cores are packed on the same silicon. The SCF is essentially a cache-coherent superhighway designed to connect cores together at a whopping 3.4 TB/s of bisectional bandwidth.

Here's a quick overview of how Nvidia's 2nd-gen SCF connects Vera's cores and memory subsystem together. Image via Nvidia

As Nvidia explains it: "SCF is organized as a distributed coherent interconnect connecting multiple cores through a collection of Coherency Switch Nodes (CSNs). Each CSN connects up to two cores and serves as a routing point within the fabric, enabling efficient movement of requests between cores, caches, memory controllers, and external interfaces."

The SCF is also where the L3 cache lives, which explains why Nvidia opted to shard its SLC across the CPU rather than having one big, centrally located pool of it.

The SCF is critical, as without it, Vera wouldn't actually be able to utilize its memory bandwidth effectively.

Vera features eight LPDDR5x memory controllers with support for SOCAMM2 modules up to 9600 MT/s. At 1.2 TB/s and roughly 14 GB/s per core, Vera has one of the fastest memory subsystems in the world today, but it's worth pointing out that the competition's chips launched nearly two years ago.

Both Intel and AMD's next-gen CPUs will make the jump to 16 memory channels. We're still waiting on details on Intel's 192-core Diamond Rapids platform, but we learned last week that AMD's Venice CPUs will boast up to 256 cores and support for 8,000 MT/s RDIMMs or 12,800 MT/s MRDIMMs, which translates to between 1 TB/s and 1.6 TB/s.

Assuming AMD's GMI-links aren't a bottleneck this time around, the House of Zen's 96-core Epycs should hit around 17 GB/s per core, and there's no reason to think Intel's Xeon 7 processors won't do the same.

With that said, Vera's memory subsystem should suck a bit less power thanks to the use of LPDDR5X rather than standard RDIMMs. The company claims under a sustained load, its memory subsystem contributes just 30 to 40 watts of consumption versus 100 to 200 watts depending on channel and DIMM count on a traditional server platform. For reference, Vera's TDP is configurable between 250W and 450W.

Power consumption might explain why AMD is working on a new Epyc, codenamed Verano, which will also use LPDDR5X.

An I/O trifecta

Vera isn't bursting just with memory bandwidth, but with I/O bandwidth as well. The chip is among the first to support PCIe 6.4 and CXL 3.1, beating out Intel, and possibly AMD depending on the first systems to actually make it into customers' hands.

As with past generational PCIe bumps, 6.0 sees per lane bandwidth double to 64 Gbps, which translates to 128 GB/s (256 GB/s bidirectional) for a single x16 slot. A single Vera socket can support 6 such slots (96 lanes in total) for connectivity to NICs, storage, and other peripherals. In the dual-socket Vera CPU superchip config, the lane count does drop slightly to 176 lanes or 11 x16 slots.

Along with PCIe, those lanes also support Compute Express Link 3.1. CXL has been around for a while now, but the 3.1 iteration is notable for its support for fabrics and memory sharing.

As we've previously discussed, this capability opens the door to network attached memory appliances — which we affectionately call memory godboxes — where multiple CPUs can share the memory.

Finally, we need to talk a bit about Vera's NVLink chip-to-chip connectivity. First introduced back in its Grace CPUs, the high-speed interconnect's bandwidth has now been doubled to 1.8 TB/s of bidirectional bandwidth (900 GB/s up and down).

This high-speed interconnect allows for coherent memory access across connected CPUs or GPUs. In the Vera Rubin superchip, NVLink is how the CPU talks to the two GPUs. Meanwhile for Vera CPU superchip, the NVLink-C2C interface provides a blazing fast interconnect that mitigates many of the bandwidth bottlenecks associated with multi-socket systems.

Performance

In agentic workloads, Nvidia estimates Vera will deliver 1.8x higher per-core performance than AMD's 128-core Epyc 9755 in benchmarks representative of agentic workloads, and up to 2.6x faster performance in graph traversal jobs.

Take these performance claims with the usual grain of salt, but Nvidia claims that in workloads representative of agentic AI, Vera delivers 1.8x higher per core performance. Image credit Nvidia

While that sounds impressive, we're not reading too deeply into Nvidia's performance claims just yet. Turin launched nearly two years ago, which makes it an advantageous comparison point. CPU benchmarks are notoriously easy to cherry pick, and microbenchmarks, while useful for quantifying hardware's capabilities, aren't always representative of real-world performance.

A classic example of this is core-to-core latency on early AMD Epycs versus Intel Xeons. As we've already established, AMD employs a multi-chiplet architecture that, for Turin, used up to eight frequency-optimized cores or 16 density-optimized cores per core-complex die (CCD).

For workloads that can fit within a single CCD, the core-to-core latency is extremely small. But if a core on one CCD needs to talk to a core on another CCD, it has to walk the I/O die, which incurs a stiff performance penalty.

Intel, until recently, has favored higher core-count compute dies, which historically have fared better for latency-sensitive applications, like databases, for the same reason Nvidia opted for a big monolithic compute die on Vera.

Here Nvidia makes a case for its monolithic compute architecture by demonstrating core-to-core latency versus AMD's chiplet-based Epycs, which incur a latency penalty for CCD-to-CCD comms. Whether this actually makes a difference varies from application to application. Image credit Nvidia

Whether this actually confers an advantage varies from application to application, and it's not a clear cut victory for one approach or the other. Both have their merits. AMD's chiplet architecture lets them use smaller, better yielding dies, which translates to better margins.

We should also mention that Venice makes the jump to 12- and 32-core CCDs, which means the workload would need to be even more talkative for you to notice the latency hit.

We've said it before and we'll say it again: there has never been and never will be one CPU to rule them all. Vera doesn't change that.

After getting a peek at Vera's performance in the Phoronix Test Suite in a selected, Nvidia-dictated set of tests earlier this year, we're optimistic about Vera's performance in its target workloads, but look forward to seeing fully independent runs on production hardware, especially with AMD's Venice just a few months away.

We fully anticipate Vera to sell extremely well, if not for its performance characteristics, then because if you want more than eight Rubins, it's probably coming with a Vera CPU attached. ®