# NVIDIA's Moat Is More Than a CUDA Monopoly — Breaking Its Advantage into Layers, and Reading Competitors and Future Scenarios from Primary Sources

> NVIDIA's advantage layer by layer — CUDA, libraries, NVLink, supply — checked against ROCm, TPU and Trainium docs and SEC filings, with bull/base/bear scenarios.

- Published: 2026-11-17
- Author: 友田 陽大
- Tags: AI, インフラ, 技術選定, コスト最適化, アーキテクチャ設計
- URL: https://tomodahinata.com/en/blog/nvidia-competitive-advantage-cuda-ecosystem-future-scenarios
- Category: NVIDIA, GPUs & AI chips
- Pillar guide: https://tomodahinata.com/en/blog/nvidia-gpu-ai-tensor-core-fp8-hbm-nvlink-explained

## Key points

- NVIDIA's strength cannot be summed up as a 'CUDA monopoly'. It is a stack: chips, secured supply of memory and packaging, the interconnect that binds GPUs together (NVLink), networking, libraries, accumulated developer know-how, and the ability to ship complete systems.
- The most eroded layer is the framework. PyTorch's AMD build reuses the torch.cuda API as is, and Google TPU (TorchTPU) and AWS Trainium (Neuron) both run as PyTorch devices once you install each vendor's separate package ('native' is the vendors' word). The center of gravity of the moat is shifting toward supply, interconnect and system integration.
- Per SEC filings, two direct customers accounted for 22% and 14% of revenue in fiscal 2026 (ended January 25, 2026). In Q2 of fiscal 2027 (ended July 26, 2026), the Hyperscale category was about 51% of revenue (author's calculation). The category is not defined in the filing and does not precisely measure dependence on large cloud providers, but many of those providers are also competitors with their own chips.
- Gross margin was 71.1% in fiscal 2026 and 75.0% in Q2 fiscal 2027. This reads as pricing power that lasts only as long as NVIDIA can show customers better value in cost per token.
- This article reflects information as of November 17, 2026 (JST) and was written before the Q3 results scheduled for the same day (U.S. time). The scenarios are the author's analysis, not investment advice.

---

"NVIDIA wins because it has a CUDA monopoly" is half an explanation.

CUDA and the libraries around it are a big asset. But that explanation alone cannot account for two facts. First, AMD, Google and AWS have each built out their own software, and from PyTorch you can now use them by changing little more than the device name. Second, NVIDIA's revenue and gross margin have stayed high anyway.

**The short answer:** NVIDIA's advantage is a stack of layers with very different properties. Surprisingly, the layer rivals find easiest to catch up on is "running code written for CUDA". The hard layers to copy are the interconnect that makes dozens of GPUs behave like one computer, advance claims on supply of HBM (stacked high-bandwidth memory placed right next to the GPU) and leading-edge packaging (the technology that combines chips and memory into one package), and the ability to design, ship and even help finance entire racks.

This article first checks where NVIDIA stands using reported numbers. It then splits the moat into layers and asks how much it would cost, and how long it would take, for someone else to replicate each one. It ends with bull, base and bear scenarios, each with assumptions and leading indicators. In brief: bull is "the system-level advantage holds", base is "still the leader, but with a smaller share of the growth", and bear is "an AI investment slowdown coincides with insourcing by large cloud providers".

**Ground rules for this article.**

- Financial figures come only from NVIDIA's 10-K and 10-Q filings with the SEC and its own earnings releases. NVIDIA's fiscal year ends in late January, so "fiscal 2026 (FY2026)" is the year that ended on January 25, 2026. That is one year off from the calendar year.
- This was written on November 17, 2026 (JST). NVIDIA is scheduled to report Q3 of fiscal 2027 on November 17, U.S. time, and those results were not out at the time of writing. The latest quarterly data used here is **Q2 (ended July 26, 2026)**.
- Jensen Huang's resignation as CEO on November 11, 2026 is covered in detail in [Jensen Huang's resignation and NVIDIA's governance](/blog/jensen-huang-resignation-nvidia-ceo-transition-governance). Here it appears only as one source of uncertainty in the scenarios.
- Performance figures are the theoretical peaks each vendor publishes. Precision, dense versus sparse, and what counts as one chip are not aligned, so the numbers in the tables cannot be compared directly.
- The scenarios and ratings are the author's analysis. They are not investment advice.

## 1. The short answer: what "CUDA monopoly" explains, and what it doesn't

Let's start by sorting out the familiar story.

**What "CUDA monopoly" does explain.** AI research and development ran on NVIDIA GPUs and CUDA for a long time. NVIDIA's 10-K says it introduced CUDA in 2006 and that "over 7.5 million developers" use CUDA and its other software tools (a company figure; the counting method is not disclosed). Libraries such as cuBLAS for matrix multiplication, cuDNN for neural-network operations and NCCL for GPU-to-GPU communication have been tuned across many GPU generations. Existing code, know-how and people all assume CUDA, and that becomes a switching cost.

**What "CUDA monopoly" does not explain.**

- PyTorch's AMD build uses the `torch.cuda` API as is. User code keeps saying `cuda` and runs on AMD GPUs (Section 4).
- Google TPU and AWS Trainium both officially offer ways to run as PyTorch devices, shipped as each vendor's separate package (Section 4).
- Even so, NVIDIA's gross margin was 75.0% in Q2 of fiscal 2027 (Section 8). If the software barrier is falling but prices hold, the advantage must come from more than that barrier.

NVIDIA itself lists the principal competitive factors in the "Competition" section of its 10-K:

> performance, breadth of product offerings, access to customers and partners and distribution channels, software support, conformity to industry standard APIs, manufacturing capabilities, processor pricing, and total system costs

Software is one item out of eight. This article rearranges those eight items into layers and looks at which are thick and which are thin.

## 2. Where NVIDIA stands, in numbers (through Q2 fiscal 2027)

Here is the size and shape of the business, using figures from the filings.

| Item | Figure | Period / source |
|---|---|---|
| Revenue | USD 96.2 billion (+106% year over year) | Q2 fiscal 2027 (ended July 26, 2026), 10-Q |
| of which Data Center | USD 89.0 billion (+117%) | Same |
| of which Hyperscale | USD 48.7 billion | Same |
| of which AI Clouds, Industrial & Enterprise | USD 40.3 billion | Same |
| Gross margin (GAAP) | 75.0% | Same |
| Company outlook for Q3 | Revenue USD 108.0 billion ±2%, gross margin 74.0% ±0.5 points | Earnings release of August 26, 2026 |
| Revenue (full year) | USD 215.9 billion | Fiscal 2026 (ended January 25, 2026), 10-K |
| of which Data Center | USD 193.7 billion (compute USD 162.4 billion, networking USD 31.4 billion) | Same |

Data Center is about 92.5% of revenue (89.0 ÷ 96.2, author's calculation). NVIDIA's future is, for practical purposes, the future of data-center AI infrastructure.

The Q3 outlook was issued in August by the management team that still included Huang. Whether results matched it will only be known after the November 17 (U.S. time) report.

Starting in Q1 of fiscal 2027, NVIDIA changed how it breaks down revenue. The Q2 10-Q no longer shows the old "Compute" and "Networking" split. It uses customer-type categories instead: "Hyperscale" and "AI Clouds, Industrial, & Enterprise (ACIE)". The 10-Q text does not define "Hyperscale", so this article uses the category name as is.

## 3. What the CUDA ecosystem actually is: three layers

People use "CUDA" to mean at least three different things. If you don't separate them, you misjudge where the moat is.

### 3.1 CUDA as a programming model

This is the language extension, compiler and runtime for writing programs (kernels) that run on the GPU. You write something close to C++ and have thousands of threads execute the same code. AMD's HIP offers a very similar model at this layer. AMD's conversion tool, HIPIFY, automatically rewrites CUDA runtime API calls, kernel launch syntax and keywords such as `__global__` into HIP.

### 3.2 CUDA as libraries (CUDA-X)

Users rarely write this layer directly, yet it decides real-world speed.

- **cuBLAS**: NVIDIA's documentation calls it "an implementation of BLAS (Basic Linear Algebra Subprograms) on top of the NVIDIA CUDA runtime." It picks a matrix-multiply implementation for each matrix shape and GPU generation.
- **cuDNN**: a library of optimized training and inference operations, covering attention, matrix multiplication, convolution and multi-operation fusion.
- **NCCL**: the GPU-to-GPU communication library. Its documentation says it provides communication primitives that are "topology-aware", and it implements collectives such as AllReduce by handling communication and computation in a single kernel. It supports PCIe, NVLink, InfiniBand and IP sockets.

The strength of this layer lies less in the number of functions than in **accumulated optimization**. With each new GPU, the fastest implementation for each matrix shape and the choice of communication paths were updated. A rival can offer functions with the same names, but matching that level of optimization across every combination takes time.

### 3.3 CUDA as people and habits

This is the 7.5 million developers, university courses, profilers and other tools, search results for error messages, and in-house knowledge of the "write this kernel this way and it's fast" kind. It is hard to document, and porting tools cannot carry it.

### 3.4 Where switching costs come from

AMD is candid about the limits in its HIPIFY documentation. In summary:

- HIPIFY does not automatically convert all CUDA code to HIP. HIP is not a complete replacement for CUDA.
- CUDA libraries or third-party libraries that have no HIP equivalent cannot be translated.
- Code optimized for performance on NVIDIA GPUs may need additional work to perform well on AMD GPUs.

So switching costs arrive in two stages: the cost of making the code run, and the cost of making it run as fast. The second is usually larger, and much harder to estimate in advance.

## 4. PyTorch makes the moat both shallower and deeper

Most AI developers do not write CUDA directly. They use it indirectly through frameworks such as PyTorch. How well the framework supports other vendors' hardware therefore decides how easy it really is to switch.

### 4.1 Shallower: the main rivals became devices you can use from PyTorch

Here is the state of play as of November 2026, as confirmed in official documentation.

| Hardware | How you use it from PyTorch | What the official docs say |
|---|---|---|
| AMD GPUs (ROCm) | Keep `torch.device('cuda')` | PyTorch "intentionally reuses the existing torch.cuda interfaces". The device names `'rocm'` and `'hip'` are not valid; use `'cuda'` |
| Apple Silicon (MPS) | `torch.device("mps")` | A backend for GPU training on Metal |
| Google TPU (TorchTPU) | `torch.device("tpu")` | A PyTorch-native backend co-developed with Meta. Eager mode by default; supports `torch.compile` |
| AWS Trainium (Neuron) | `device='neuron'` | Native PyTorch (TorchNeuron) with eager mode, `torch.compile`, FSDP and DDP |

Note, however, that TorchTPU (`torch_tpu`) and Neuron's PyTorch support (`torch-neuronx` and related packages) are not part of PyTorch itself; Google and AWS each distribute them separately. "Native" is the vendors' word, and you need to check each vendor's documentation for which operations and features are covered.

The AMD case is telling. NVIDIA's own **API name**, CUDA, has become the common entry point for porting. From the user's point of view, the string `cuda` in their code is less and less NVIDIA's property.

On the TPU side, there was a big shift. Running PyTorch on TPUs used to require PyTorch/XLA. Google Cloud's documentation explains that PyTorch/XLA relied on "lazy-tensor graph recording", which often meant adding code to mark compilable regions. On October 19, 2026, the PyTorch/XLA repository was marked deprecated and began directing users to its successor, TorchTPU. The README says the repository will be archived on November 15, 2026, but when I checked through the GitHub API on November 17, it had not yet been archived.

Inference servers and kernel languages are multi-hardware too.

- **vLLM**: the official docs list NVIDIA CUDA, AMD ROCm, Intel XPU and Apple Silicon (via vLLM-Metal), among others, and describe a mechanism for external hardware plugins. AWS names its "vLLM Neuron" plugin as the recommended path on Trainium.
- **Triton**: a language and compiler for writing kernels in Python. Its README lists supported hardware as "NVIDIA GPUs (Compute Capability 8.0+)" and "AMD GPUs (ROCm 6.2+)".
- **XLA (OpenXLA)**: an open-source compiler that optimizes PyTorch, TensorFlow and JAX models for GPUs, CPUs and dedicated accelerators. It is developed jointly by AMD, AWS, Google, Intel, Meta, NVIDIA and others.

### 4.2 Deeper: defaults and version lag

NVIDIA still holds advantages at the framework layer.

**Defaults.** On PyTorch 2.15.0's install page, typing `pip3 install torch torchvision` on Linux installs the CUDA 13.2 build. To get the ROCm build, you have to point at a separate index (`whl/rocm10.0`). Anyone who starts without thinking about it starts on CUDA.

**Version lag.** The ROCm 10.1.0 compatibility matrix validates PyTorch only up to 2.14.0, and lists vLLM 0.29.0 and SGLang 0.5.18 as "requires PyTorch 2.13.0". PyTorch's stable release is 2.15.0. PyTorch itself also ships ROCm builds, so this is not the only combination that works on ROCm. Still, a vendor-validated stack that trails upstream means a little friction every time you want a new feature.

**The same API does not guarantee the same numbers.** PyTorch's HIP documentation says TF32 (a reduced-precision format used inside matrix multiplication) works on AMD's MI300 via hipBLASLt, but notes hardware-level numerical differences from NVIDIA's implementation. Even with the same API, you need to re-validate accuracy.

### 4.3 What PyTorch changed is who owns the moat

NVIDIA does not own the PyTorch layer. Every vendor competes on the same ground there. As a result, switching costs now differ sharply by **type of user**.

| User type | Switching cost | Why |
|---|---|---|
| Runs standard models through PyTorch or vLLM | Low to medium | Likely to run after changing the device. Performance checks, accuracy validation and swapping operational tooling are still needed |
| Depends on custom CUDA kernels or CUDA-only libraries | High | HIPIFY can't convert everything, and performance has to be re-tuned |
| Large cloud provider with its own chips and compiler team | Lowest per accelerator | The fixed porting cost is spread over a huge fleet (Section 12) |

The last row matters most for NVIDIA. The biggest customers are the ones best placed to recover the cost of switching.

## 5. The power to bind GPUs together: NVLink, InfiniBand and Spectrum-X

In large-scale AI training and inference, how fast you can connect many GPUs matters as much as how fast one GPU is. The basics of GPU-to-GPU communication, with the math, are in [Section 8 of the pillar article](/blog/nvidia-gpu-ai-tensor-core-fp8-hbm-nvlink-explained). Here we look only at the competitive angle.

### 5.1 Two kinds of "connecting"

- **Scale-up**: connecting dozens of GPUs within a rack so they act like one big computer. At NVIDIA, NVLink and NVLink Switch do this. GB200 NVL72 puts 72 GPUs in one NVLink domain and quotes 130 TB/s in total (the sum of each GPU's bidirectional bandwidth, not bisection bandwidth). For Vera Rubin NVL72, NVIDIA's own documents disagree: the datasheet and the NVLink product page say 216 TB/s, while the January 2026 announcement and the developer blog say 3.6 TB/s per GPU and 260 TB/s per rack.
- **Scale-out**: connecting rack to rack, thousands to hundreds of thousands of accelerators. At NVIDIA, InfiniBand (Quantum) and Ethernet (Spectrum-X) do this.

### 5.2 What the Mellanox acquisition meant

NVIDIA completed its acquisition of Mellanox Technologies on April 27, 2020, for a transaction value of USD 7 billion. The 10-K says the deal added networking to NVIDIA's offerings and let it build platforms at "data center scale".

In fiscal 2026, networking was USD 31.4 billion of Data Center revenue, about 16.2% (author's calculation; about 11.3% in fiscal 2025 and about 18.0% in fiscal 2024). Selling networking alongside GPUs is one clue that customers are choosing NVIDIA as a "system", not as a parts supplier.

### 5.3 Catching up through open standards

At the network layer, industry standards are pushing back.

- **UALink**: an open standard for scale-up links between accelerators. The UALink 200G 1.0 specification is described as connecting up to 1,024 accelerators in an AI computing pod at 200G per lane.
- **Ultra Ethernet**: an Ethernet standard for AI and HPC. Specification 1.0 was published on June 11, 2025, and 1.1 on October 12, 2026.

NVIDIA describes Spectrum-X itself as standards-based Ethernet with support for the open SONiC network OS. The scale-out layer appears to be moving from proprietary technology toward performance competition on top of standards.

### 5.4 NVLink Fusion: from lock-in to plug-in

What stands out is that NVIDIA has begun opening NVLink to other companies' chips. According to the 10-K, NVIDIA introduced "NVLink Fusion" in fiscal 2026 so that large cloud providers and designers of custom ASICs (chips designed for a specific purpose) can integrate their own CPUs and dedicated chips (XPUs) into NVIDIA's platform. The product page says Amazon's Annapurna Labs (AWS's chip design arm) will be the first to work with NVIDIA on NVHBM, NVIDIA's memory technology.

This move assumes custom ASICs will keep growing. Even if a customer uses its own chip, the rack, interconnect, networking, and power and cooling design ride on NVIDIA's specifications. My reading is a strategy of giving ground at the chip layer while staying in at the interconnect and system layers.

## 6. Where competitors stand: hardware and software

NVIDIA's 10-K names AMD, Huawei and Intel as competitors, and also names Alibaba, Alphabet, Amazon, Baidu, Huawei and Microsoft as "large cloud services companies with internal teams designing hardware and software". This section looks at AMD, Google and AWS, which I could check against official sources.

### 6.1 Vendor specs side by side (conditions are not aligned)

| Product | Memory capacity | Memory bandwidth | Peak FP8 (vendor) | Caveats |
|---|---|---|---|---|
| NVIDIA Blackwell Ultra (one GPU in GB300) | 279 GB HBM3E | 8 TB/s | 10 PFLOPS (sparse; dense 5) | NVIDIA datasheet |
| NVIDIA Rubin GPU (one GPU) | 288 GB HBM4 | (omitted; sources disagree) | FP8/FP6 training 17.5 PFLOPS (dense) | HGX page table |
| AMD Instinct MI355X (MI350 Series) | 288 GB HBM3E | 8 TB/s | 5 PFLOPS (10.1 with structured sparsity) | AMD product page |
| AMD Instinct MI455X (MI400 Series) | 432 GB HBM4 | 23.3 TB/s | 20.1 PFLOPS (OCP FP8) | AMD product page. "Launch Date 7/23/2026" is AMD's term and not necessarily a shipment date. The Helios page also says 19.6 TB/s per GPU, so AMD's documents disagree |
| Google TPU7x (Ironwood, one chip) | 192 GiB HBM (about 206 GB) | 7,380 GBps | 4,614 TFLOPS | Dense vs sparse not stated. Two chiplets per chip |

All this table supports is this: on vendor memory capacity and bandwidth, AMD's next generation (MI455X) lists figures equal to or above NVIDIA's next generation (Rubin), and the current generations (MI355X and Blackwell Ultra) are close. It does not say which is faster. Theoretical peaks are not effective performance, and results swing widely with software maturity, communication and workload shape. How to read these numbers is covered in detail in the [pillar article](/blog/nvidia-gpu-ai-tensor-core-fp8-hbm-nvlink-explained).

For condition-matched comparisons, there is MLCommons' MLPerf. In MLPerf Training v6.1, published on November 16, 2026, 24 organizations submitted results. MLCommons says systems "utilized NVIDIA, AMD and Google accelerator hardware", including NVIDIA's Rubin generation and Google's "TPUv7 (Ironwood)", as MLCommons calls it, for the first time.

### 6.2 AMD: the challenger that chose the path closest to CUDA

AMD describes ROCm as its "open software stack". Its library lineup maps closely onto NVIDIA's.

| Role | NVIDIA | AMD (ROCm) |
|---|---|---|
| Matrix multiplication | cuBLAS | rocBLAS, hipBLAS, hipBLASLt |
| Neural-network operations | cuDNN | MIOpen, Composable Kernel |
| GPU-to-GPU communication | NCCL | RCCL |
| Programming model | CUDA | HIP (with the HIPIFY converter) |

On hardware, AMD has also moved to rack-scale products. According to AMD's product page, "Helios" holds 72 MI455X or MI450X GPUs connected with UALink over Ethernet (UALoE), for 260 TB/s of aggregate scale-up bandwidth. It is the same "72 accelerators tied together as one" shape as NVL72.

AMD's strategy is to lower the switching barrier by staying close to how CUDA is written. PyTorch reusing the `torch.cuda` API is a result of that. Its weak spots are the version lag and optimization depth covered in the previous section.

### 6.3 Google TPU: from its own path toward PyTorch

Google TPU grew up on its own software, centered on JAX and XLA. The TPU7x documentation states that JAX and PyTorch are the frameworks you can use on TPU7x and that TensorFlow is not supported. One pod has 9,216 chips, linked by ICI (Inter-Chip Interconnect, TPU's dedicated chip-to-chip link).

The big change in 2026 is TorchTPU, mentioned above. Google worked with the PyTorch team so that TPUs can be used with ordinary PyTorch code. That lowers the biggest barrier, "you have to learn JAX to use TPUs".

### 6.4 AWS Trainium and Inferentia: optimizing inside its own cloud

AWS's Neuron SDK is the software for running AI on Trainium and Inferentia. Per its documentation, the recommended training path is native PyTorch (TorchNeuron) with TorchTitan, and the recommended inference path is the vLLM Neuron plugin. For engineers who want to push performance, it offers NKI (Neuron Kernel Interface), which exposes Neuron's ISA (instruction set) directly. In CUDA terms, this is the layer where you write custom kernels.

Supported hardware includes Inf2, Trn1, Trn2 and Trn3; the first-generation Inf1 is no longer supported. Because AWS's chips are only available inside AWS's cloud, the question is not merchant market share but how much of its own demand AWS can serve with something other than NVIDIA.

### 6.5 The broader custom-ASIC trend

Google's TPU and AWS's Trainium are dedicated chips (ASICs) that large cloud providers designed for themselves. That is what the 10-K means when it names "large cloud services companies with internal teams designing hardware and software" as competitors.

The key point is the structure: **the largest customers are also the largest potential competitors**. Section 9 looks at this with numbers.

## 7. The low-precision race and the shifting inference market

### 7.1 Number formats: from proprietary to standard

AI arithmetic has moved to formats with fewer bits: FP16, FP8, FP4. Smaller formats let the same circuitry do more operations and cut memory traffic (see [Section 3 of the pillar article](/blog/nvidia-gpu-ai-tensor-core-fp8-hbm-nvlink-explained)).

Here, NVIDIA's own format (NVFP4) sits alongside open formats defined by the Open Compute Project (MXFP8, MXFP6, MXFP4 and others). AMD's MI355X highlights MXFP6 and MXFP4 support, and AWS's vLLM Neuron lists FP8 and MXFP8 quantization. The more number formats are standardized, the more any company can build "circuits that compute that format fast", and the difference moves to software and systems.

### 7.2 As inference grows, the terms of competition change

Training links thousands to tens of thousands of accelerators and runs for weeks, so communication and reliability dominate. Inference generates text one token at a time in response to user requests, so memory bandwidth, latency and **cost per token** dominate.

Inference is easier than training for purpose-built chips to break into. Once a model is fixed, you can favor efficiency over generality.

NVIDIA is responding to this shift too. According to its 10-K, in December 2025 NVIDIA entered into a **non-exclusive license agreement** with Groq, a maker of inference chips, for its LPU (language processing unit) technology, and hired certain Groq employees. No customer contracts, products or equity were acquired. The consideration was USD 13.0 billion paid at closing plus USD 4 billion (including imputed interest) payable within one year. NVIDIA's August 2026 earnings release says "NVIDIA Groq 3 LPX" is in full production. My reading: a GPU company paying that much for a different inference-chip design signals that NVIDIA sees purpose-built designs as a real threat in inference.

## 8. TCO and cost per token: the source of pricing power

### 8.1 Cost per token

The economics of inference ultimately come down to this formula:

$$
C_{\mathrm{token}} = \frac{C_{\mathrm{hour}}}{3600 \cdot R}
$$

- $C_{\mathrm{token}}$: cost per token
- $C_{\mathrm{hour}}$: total cost of running one accelerator for one hour (TCO), including amortized purchase price, power, cooling, facilities, operations staff and software
- $R$: tokens generated per second

The point of the formula: **even if the numerator (price) is high, the customer pays less if the denominator (throughput) is larger still**. And the denominator depends heavily not only on hardware but on software: kernels, quantization, batching and the inference server.

### 8.2 How far does pricing power reach?

When a customer compares NVIDIA's product with a rival's, the point at which cost per token is equal is the ceiling on the total cost the customer will accept for the NVIDIA option. Let NVIDIA's hourly total cost be $C_N$ and its throughput $R_N$, and the rival's be $C_A$ and $R_A$:

$$
C_{N}^{\max} = C_{A} \cdot \frac{R_{N}}{R_{A}}
$$

$C_{N}^{\max}$ is a ceiling on the **total cost** of the NVIDIA option, not on NVIDIA's price. NVIDIA's price ceiling is what remains after subtracting power, facilities, operations and other costs. The formula also assumes both options run at the same utilization.

The larger the throughput ratio $R_N / R_A$, the higher the price at which NVIDIA still gets chosen. If a rival's software matures and the ratio approaches 1, the ceiling falls to the rival's total cost. In a real comparison, porting cost and risk, supply certainty and operational familiarity also enter total cost.

In its 10-K, NVIDIA says Blackwell Ultra plus Dynamo (its inference software) cut cost per token versus the Hopper generation, and that Rubin delivers "up to a 10x reduction" versus Blackwell. These are company claims without stated conditions, and this article does not verify them. Still, the fact that NVIDIA frames its value in "cost per token" rather than "GPU price" is consistent with the formula above.

### 8.3 Pricing power, seen through gross margin

The result of pricing power shows up in gross margin.

| Period | Gross margin (GAAP) | Notes |
|---|---|---|
| Fiscal 2024 (ended January 28, 2024) | 72.7% | Author's calculation from 10-K revenue and gross profit |
| Fiscal 2025 (ended January 26, 2025) | 75.0% | 10-K |
| Fiscal 2026 (ended January 25, 2026) | 71.1% | 10-K. Includes a USD 4.5 billion charge for H20 inventory and related items |
| Q2 fiscal 2027 (ended July 26, 2026) | 75.0% | 10-Q |
| Q3 fiscal 2027 (company outlook) | 74.0% ±0.5 points | Earnings release of August 26, 2026. Actual not yet reported |

For the fiscal 2026 decline, the 10-K cites the "business model transitioned from offering Hopper HGX systems to Blackwell full-scale datacenter solutions" and a USD 4.5 billion charge related to H20 (a China-market product) inventory and purchase obligations. The more you sell by the rack, the larger the share of components bought from others, which plausibly pulls gross margin down (the author's interpretation). Staying in the 70s despite that shows strong pricing power.

On the other hand, gross margin moves with product transitions and inventory valuation. Don't judge from a single quarter; track the gap between the company's outlook and its results.

## 9. Customer bargaining power: concentration among the top customers

### 9.1 Concentration as disclosed in filings

NVIDIA discloses customers that account for 10% or more of revenue, without naming them.

| Period | Disclosure (figures verbatim from the 10-K and 10-Q) |
|---|---|
| Fiscal 2026 | One direct customer at 22% of revenue, another at 14% |
| Q2 fiscal 2026 | Two direct customers at 23% and 16% |
| First half of fiscal 2027 | Three direct customers at 16%, 15% and 13% |
| Q2 fiscal 2027 | One direct customer at 16% |

"Direct customers" here include AIBs (add-in board makers), OEMs, ODMs (original design manufacturers), system integrators, distributors, cloud providers and AI model makers. A direct customer may build servers and ultimately deliver them to another company, so "top customer" does not necessarily mean "end user". The 10-Q also says that some indirect customers individually represent 10% or more of revenue, and that NVIDIA estimates "one AI research and deployment company" contributed "a meaningful amount" of revenue by buying cloud services from NVIDIA's customers.

In Q2 of fiscal 2027, Hyperscale revenue was USD 48.7 billion, about 50.6% of total revenue and about 54.7% of Data Center revenue (author's calculation).

The 10-Q, however, does not define this category. It also says ACIE growth includes demand from "hyperscalers utilizing AI clouds", and in Q2 one company was moved from ACIE to Hyperscale because its business model changed. The Hyperscale share therefore does not precisely measure dependence on large cloud providers. This article uses it only as a rough proxy (the author's judgment).

### 9.2 A buyer-strong structure

Concentrated customers each carry more bargaining power. In NVIDIA's case, the largest customer group, especially large cloud providers, has three levers on top of that.

1. **Switching to their own chips**: they already have chips such as TPU and Trainium.
2. **Adopting other vendors' GPUs**: they can run AMD and others in parallel and use that in price negotiations.
3. **Amortizing porting cost**: their fleets are large enough to recover fixed porting costs (Section 12).

### 9.3 NVIDIA's countermeasures, and the risk behind them

NVIDIA is also widening its customer base. In Q2 of fiscal 2027, ACIE revenue grew 138% year over year, faster than Hyperscale (up 102%). As noted above, though, ACIE includes large cloud providers using AI clouds, so the gap does not simply mean diversification away from them.

Behind that, though, NVIDIA is increasingly supporting its customers' funding and facilities itself. The 10-Q discloses:

- As of July 26, 2026, USD 99 billion in equity investments and USD 25 billion in equity investment commitments.
- A new business model with select AI cloud providers, under which NVIDIA's commitments to buy cloud services (typically six years) total USD 36 billion.
- Guarantees with SB Energy (August 2026): for a campus leased to OpenAI under 20-year leases, NVIDIA's obligation is capped at USD 105 billion in aggregate. The guaranteed amount rises as each of nine construction phases is completed, the first expected in fiscal 2029.
- Guarantees of select AI cloud partners' data center lease obligations, with maximum gross exposure of USD 3.5 billion. Including SB Energy, maximum gross exposure under all guarantees is USD 108.5 billion.
- Memorandums of understanding with large capital providers to mobilize more than USD 500 billion of third-party capital, which may not lead to definitive agreements.

The 10-Q also states NVIDIA's view that AI clouds and AI model makers "currently lack the ability to secure long-term infrastructure contracts and investment-grade financing capacity". Supporting demand directly widens the customer base, but it is also a risk that bounces back onto NVIDIA if those customers' finances deteriorate. It is an important variable in the scenarios.

## 10. Supply chain: TSMC, CoWoS and HBM

### 10.1 NVIDIA owns no fabs

NVIDIA is fabless. Per its 10-K, it uses foundries such as TSMC and Samsung for wafers, buys memory from SK Hynix, Micron and Samsung, and uses CoWoS technology for packaging. Assembly and test are handled by Hon Hai, Wistron, Fabrinet and others.

CoWoS is TSMC's advanced packaging technology, which mounts a GPU die and HBM (stacked high-bandwidth memory) side by side on one base (an interposer). Large AI GPUs tend to be constrained in volume by this step and by HBM supply.

### 10.2 Supply commitments are both a moat and a burden

According to the 10-Q, NVIDIA's supply and capacity purchase commitments grew from USD 119 billion the previous quarter to USD 279 billion as of July 26, 2026 (about 2.3 times), mainly for memory and manufacturing capacity. The 10-K says that during shortages, lead times for some supply have extended beyond twelve months.

That cuts both ways.

- **As a moat**: competitors buy from the same TSMC and the same HBM makers. While supply is tight, whoever locked in large allocations first can ship. The structure favors the biggest player with the most capital.
- **As a burden**: if demand comes in weaker than expected, the portion of purchase commitments that cannot be cancelled or changed turns into inventory write-downs and provisions. The 10-Q says some agreements may be cancelable, rescheduled or adjustable before firm orders are placed, and that changes may add costs. The USD 4.5 billion H20 charge in fiscal 2026, though caused by export controls, is an example.

This article does not use third-party estimates such as what share of TSMC's CoWoS capacity NVIDIA has secured, because they cannot be confirmed from primary sources.

## 11. Original analysis 1: replication cost and time, layer by layer

Based on the facts above, here are NVIDIA's strengths split into eight layers, with an assessment of how hard each would be for others to replicate. **The ratings are my judgment, not facts.** The evidence is the primary sources cited in each section.

| Layer | What NVIDIA has | Where challengers are | Difficulty to replicate | Degree of erosion |
|---|---|---|---|---|
| ① Compute chips | Tensor Cores, FP8/FP4 | MI355X, MI455X, TPU7x and others match or exceed on some vendor specs | Medium (catchable in 1–2 generations) | Under way |
| ② Memory and packaging supply | USD 279 billion in purchase commitments | Competing for the same suppliers | High (needs capital and advance claims) | Small while supply is tight |
| ③ Scale-up interconnect | NVLink, NVL72 (130 TB/s for GB200; 216–260 TB/s for Vera Rubin, depending on the document) | UALink 1.0, TPU ICI, AMD Helios (72 GPUs) | High (chip, switch and rack designed together) | Small to medium |
| ④ Scale-out network | InfiniBand, Spectrum-X | Ultra Ethernet, incumbent Ethernet vendors | Medium (converging on standards) | Under way |
| ⑤ Libraries | cuBLAS, cuDNN, NCCL and more | ROCm, NKI Library, XLA | Medium to high (accumulated optimization) | Starting with the main operations |
| ⑥ Frameworks | Not owned by NVIDIA | PyTorch, vLLM, Triton, JAX support each vendor | Low (already common ground) | Large |
| ⑦ Developer base | 7.5 million developers, existing CUDA code, tools | Each vendor is building one | High (grows only with time) | Small |
| ⑧ System integration and shipping | Rack-scale design, ODM network, financial support | Large cloud providers can do it themselves | High (but replicable by the largest) | Depends on the customer |

Three things stand out.

**First, the "CUDA monopoly" view focuses on the layer that has eroded most (⑥).** At the framework layer, the main rivals already have support (some through separate packages). For users who only run standard models, that barrier is now low.

**Second, the center of gravity of the moat is moving to ②, ③ and ⑧.** Advance claims on supply, interconnect that binds GPUs in groups of 72, and the ability to ship and bring up whole racks do not come from designing one chip. NVLink Fusion shows NVIDIA itself is aware of this shift.

**Third, ②, ③ and ⑧ are also the layers "the largest customers can replicate".** Google designs and runs TPU pods and ICI, and AWS designs and runs Trainium UltraServers, in house. NVIDIA's moat is shallower the larger the customer. Combined with the customer concentration in Section 9, I think that is NVIDIA's biggest structural risk.

## 12. Original analysis 2: the break-even point for switching

Let's check the claim that "the largest customers find it easiest to switch" with a simple formula.

Suppose switching cuts total cost by a fraction $s$, the payback horizon is $T$ years, and the port succeeds with probability $p$. Express the porting cost $K$ as "how many accelerator-years of total cost it equals". The minimum fleet size at which the porting cost is recovered is:

$$
N^{*} = \frac{K}{s \cdot T \cdot p}
$$

Expressing $K$ in accelerator-years avoids having to assume a GPU price. The fleet size $N^{*}$ is proportional to $K$ and inversely proportional to the savings rate $s$. Because porting cost is close to a fixed cost that does not depend on fleet size, the burden per accelerator shrinks as the fleet grows.

The code below computes the break-even fleet size with this formula. All inputs are illustrative placeholder values, not market data.

- Language: Python 3.9 or later (standard library only)
- Run: `python3 switching_breakeven.py`
- Environment: the author's Mac (Apple M4 Pro), Python 3.9.6
- The program prints Japanese labels. "移植コスト(台年)" means porting cost in accelerator-years, "削減" means savings rate, "成功確率" means success probability, and "台" means accelerators.

```python
"""Switching break-even: above how many accelerators is the porting cost recovered (theoretical).

Express the porting cost as "how many accelerator-years of total cost (TCO) it equals".
That avoids assuming any price, isolating only how scale matters.
"""


def breakeven_fleet(port_cost_accel_years: float, saving_ratio: float,
                    horizon_years: float, success_prob: float = 1.0) -> float:
    """Return the minimum fleet size N that recovers the porting cost.

    Condition: N * saving_ratio * horizon_years * success_prob >= port_cost_accel_years
    """
    if not (0 < saving_ratio < 1 and horizon_years > 0 and 0 < success_prob <= 1):
        raise ValueError("saving_ratio は (0,1)、horizon_years > 0、success_prob は (0,1]")
    return port_cost_accel_years / (saving_ratio * horizon_years * success_prob)


def main() -> None:
    horizon = 3.0  # payback horizon (years)
    savings = [0.05, 0.10, 0.20, 0.30]  # TCO savings rate after switching
    port_costs = [10, 100, 1_000]  # porting cost (accelerator-years)

    print(f"回収期間 {horizon:.0f} 年・成功確率 100% のときの損益分岐台数")
    print("移植コスト(台年) | " + " | ".join(f"削減{s:.0%}" for s in savings))
    for k in port_costs:
        row = [breakeven_fleet(k, s, horizon) for s in savings]
        print(f"{k:>15,} | " + " | ".join(f"{n:>7,.0f}" for n in row))

    print()
    print("移植コスト100台年・削減20%・回収3年で、成功確率を変えた場合")
    for p in [1.0, 0.7, 0.5]:
        n = breakeven_fleet(100, 0.20, horizon, p)
        print(f"成功確率 {p:.0%}: {n:,.0f} 台")


if __name__ == "__main__":
    main()
```

Output:

```text
回収期間 3 年・成功確率 100% のときの損益分岐台数
移植コスト(台年) | 削減5% | 削減10% | 削減20% | 削減30%
             10 |      67 |      33 |      17 |      11
            100 |     667 |     333 |     167 |     111
          1,000 |   6,667 |   3,333 |   1,667 |   1,111

移植コスト100台年・削減20%・回収3年で、成功確率を変えた場合
成功確率 100%: 167 台
成功確率 70%: 238 台
成功確率 50%: 333 台
```

How to read it: if porting costs "100 accelerator-years" and switching cuts total cost by 20%, you need at least 167 accelerators to recover it within 3 years. If the port only has a 50/50 chance of succeeding, that rises to 333.

This simple model suggests:

- **For users with dozens of accelerators, switching rarely pays.** Only when porting is cheap and savings are large.
- **For large cloud providers with tens to hundreds of thousands of accelerators, it nearly always pays.** Even at a porting cost of "1,000 accelerator-years", savings of a few percent recover it. That is the economic reason large cloud providers build their own chips.
- **What NVIDIA must defend is keeping the savings rate $s$ small.** If software improvements keep driving down cost per token, the savings from switching to a rival shrink, and the break-even fleet size grows.

**Limits of the model.** It ignores discounting (the time value of money). The fleet size $N$ is counted in NVIDIA-GPU-equivalent units; if one chip on the other platform performs differently, the required count changes too. Also, in reality, porting is not a one-time job; each new model brings more work. Securing supply, operations staff and incident response all enter total cost. Conversely, when NVIDIA products are unavailable, switching happens regardless of savings. The formula is a tool to show how scale matters, not something to apply directly to an individual decision.

## 13. Medium- to long-term scenarios: bull, base and bear

From here on, this is my analysis. Rather than predict a single future, I lay out three scenarios with different assumptions and map which indicators move in each. **I do not assign probabilities**, because there isn't enough evidence to justify them. **This is not investment advice.**

### 13.1 Three scenarios

**Bull: the system-level advantage holds**

- Assumptions: Rubin ramps on plan and the one-year product cadence continues. Software improvements preserve the cost-per-token lead, keeping the switching savings rate $s$ small. Customers beyond the large cloud providers (AI clouds, enterprises, governments) grow.
- What happens: even as large cloud providers add their own chips, total demand grows faster. The share of revenue from networking and systems rises.
- Observable signs: the gross margin outlook stays in the mid-70s. ACIE keeps outgrowing Hyperscale (though ACIE includes large cloud providers using AI clouds). The share of direct customers above 10% falls.

**Base: still the leader, but a smaller share of the growth**

- Assumptions: NVIDIA stays the leader in training and frontier inference. Meanwhile, large cloud providers move part of their inference to their own chips such as TPU and Trainium, and AMD wins more inference deployments. Framework support keeps improving, and switching costs for standard models keep falling.
- What happens: total AI infrastructure investment grows, but NVIDIA's share gradually declines. Price negotiations get tougher and gross margin drifts down slowly.
- Observable signs: the gross margin outlook steps down. Hyperscale grows slower than Data Center overall (only a rough sign, since the category is not defined). Competitors' MLPerf submissions increase, and vendor-validated framework versions catch up with upstream.

**Bear: an investment slowdown coincides with insourcing**

- Assumptions: AI investment slows because of funding constraints or delayed monetization. At the same time, large cloud providers move most of their inference to their own chips. Export controls tighten further.
- What happens: demand falls short of supply commitments, and inventory write-downs and purchase-obligation provisions increase. Customer-support investments and guarantees turn into losses. NVIDIA cuts prices to defend volume.
- Observable signs: revenue misses the company's outlook. Inventory provisions rise. Supply purchase commitments start to shrink. Customer financing frameworks fail to reach definitive agreements.

### 13.2 Sensitivity: which variables move the scenarios

| Variable | Moves toward bull when | Moves toward bear when | Indicator to watch (source) |
|---|---|---|---|
| Insourcing by large cloud providers | Demand grows faster than their own chips | Most of their inference moves to their own chips | 10%+ customer disclosures and Hyperscale share of revenue (10-Q; the category does not precisely measure dependence) |
| Cost-per-token gap | NVIDIA's improvements outpace rivals | Rivals' software matures and the gap narrows | MLPerf results; verification of vendor claims |
| Supply (HBM, packaging) | Stays tight, so advance claims pay off | Loosens, so commitments become a burden | Purchase commitment balance, inventory provisions (10-Q) |
| Framework portability | Version lag and numerical differences persist | Major rivals support new releases in step with upstream | ROCm compatibility matrix; TorchTPU and Neuron coverage |
| Customer finances | AI cloud financing comes together | Financing stalls | Notes on guarantees, investments and cloud commitments (10-Q) |
| Regulation | Rules ease or stabilize | Covered products or regions expand | Risk factors in the 10-K and 10-Q |
| Leadership | Strategy continues under a new CEO | A prolonged CEO search slows decisions | Official company announcements |

The two rows I think matter most are the top two: **insourcing by large cloud providers** and **the cost-per-token gap**. As Sections 11 and 12 showed, NVIDIA's moat is shallower for larger customers, and the category centered on large cloud providers (Hyperscale) is about half of revenue. Because the category is not defined, that "about half" involves some inference on my part.

## 14. Leading indicators: what to watch to update your view

Scenarios should be updated every time new numbers arrive. Everything below can be tracked in public materials.

1. **Q3 fiscal 2027 results (scheduled for November 17, 2026, U.S. time)**: did revenue land within the USD 108.0 billion ±2% outlook? Did gross margin land within 74.0% ±0.5 points? What is the Q4 outlook?
2. **Disclosure of 10%+ customers**: is the share of the top direct customers rising or falling?
3. **Revenue by category**: the gap between Hyperscale and ACIE growth. The categories are not defined, and large cloud providers also buy through AI clouds counted in ACIE, so this does not precisely measure dependence on them.
4. **Supply purchase commitments and inventory provisions**: do commitments keep growing? Are provisions rising?
5. **Customer-support programs**: balances of equity investments, guarantees and cloud purchase commitments, and their effect on earnings.
6. **Competitors' software**: which PyTorch versions the ROCm compatibility matrix validates; which models and features TorchTPU and Neuron support.
7. **Condition-matched performance comparisons**: which hardware submits, at what scale, in MLPerf training and inference rounds.
8. **Roadmap execution**: after Rubin, NVIDIA has announced Vera Rubin Ultra NVL576, a new rack design called Kyber, and the following Feynman generation. The 2026 official materials I checked do not give dates for these. Tracking the gap between announced plans and actual shipments is worthwhile.

## 15. Limits of this analysis

- **No market-share figures are used.** Third-party share estimates vary in method and timing and cannot be confirmed from primary sources. So this article does not answer "what is NVIDIA's market share".
- **Customers' internal costs are unknown.** How much large cloud providers save with their own chips is not public. The numbers in Section 12 are placeholders.
- **Vendor specs are not measurements.** The comparison table in Section 6 does not show which product is faster.
- **Company claims are not verified.** Claims such as "up to one-tenth the cost per token" are company announcements without stated conditions.
- **The author's hands-on scope.** My production experience is an inference platform that runs vLLM with AWQ-quantized Qwen3-8B on a single Tesla T4. I have not operated large GPU clusters, TPUs or Trainium. Even at that scale, which quantization methods and number formats were usable was set by the GPU generation and software support, and that shaped the design. Large-scale users should treat this article's framework as a starting point and measure on their own workloads.

## Summary

- NVIDIA's strength is not a "CUDA monopoly" in a word, but a stack of chips, supply, interconnect, networking, libraries, developers and system integration.
- The framework layer is the most eroded. PyTorch's AMD build reuses the `torch.cuda` API, and TPU and Trainium can now be used as PyTorch devices through each vendor's separate package. Friction remains, though: the default install is the CUDA build, vendor-validated versions lag, and HIPIFY has limits.
- The center of gravity of the moat is moving to advance claims on supply, NVLink scale-up and rack-scale system integration. NVLink Fusion reads as a strategy for staying in the interconnect and system layers on the assumption that custom ASICs will grow.
- The break-even point for switching depends strongly on fleet size. The largest customers find it easiest to switch, and the Hyperscale category, centered on large cloud providers, was about half of revenue in Q2 fiscal 2027 (the category is undefined, so this is not a precise measure of dependence). I think that is NVIDIA's biggest structural risk.
- Gross margin was 71.1% in fiscal 2026 and 75.0% in Q2 fiscal 2027, so pricing power is strong for now. Whether it lasts depends on whether NVIDIA can keep the cost-per-token gap.

This article was written before the Q3 results. When the numbers come out, plug them into the indicators in Section 14 and see which scenario they point toward. The technical foundation of why GPUs are so strong at AI is covered in [Why NVIDIA GPUs are so strong at AI](/blog/nvidia-gpu-ai-tensor-core-fp8-hbm-nvlink-explained).

How to compare cost per token across machines is covered in [CPU vs GPU vs TPU](/blog/cpu-gpu-tpu-comparison-ai-workload-tco-selection-guide), the export controls that shape the China market in [The US–China AI chip war and export controls](/blog/us-china-ai-chip-export-controls-bis-ear-explained), and the road to here in [A history of NVIDIA](/blog/nvidia-history-from-gpu-to-ai-infrastructure).
