Skip to main content
NVIDIA, GPUs & AI chips
AI
インフラ
技術選定
コスト最適化
アーキテクチャ設計

NVIDIA's Moat Is More Than a CUDA Monopoly — Breaking Its Advantage into Layers, and Reading Competitors and Future Scenarios from Primary Sources

NVIDIA's advantage layer by layer — CUDA, libraries, NVLink, supply — checked against ROCm, TPU and Trainium docs and SEC filings, with bull/base/bear scenarios.

Published
Reading time
36 min read
Author
友田 陽大
Share
Contents

"NVIDIA wins because it has a CUDA monopoly" is half an explanation.

CUDA and the libraries around it are a big asset. But that explanation alone cannot account for two facts. First, AMD, Google and AWS have each built out their own software, and from PyTorch you can now use them by changing little more than the device name. Second, NVIDIA's revenue and gross margin have stayed high anyway.

The short answer: NVIDIA's advantage is a stack of layers with very different properties. Surprisingly, the layer rivals find easiest to catch up on is "running code written for CUDA". The hard layers to copy are the interconnect that makes dozens of GPUs behave like one computer, advance claims on supply of HBM (stacked high-bandwidth memory placed right next to the GPU) and leading-edge packaging (the technology that combines chips and memory into one package), and the ability to design, ship and even help finance entire racks.

This article first checks where NVIDIA stands using reported numbers. It then splits the moat into layers and asks how much it would cost, and how long it would take, for someone else to replicate each one. It ends with bull, base and bear scenarios, each with assumptions and leading indicators. In brief: bull is "the system-level advantage holds", base is "still the leader, but with a smaller share of the growth", and bear is "an AI investment slowdown coincides with insourcing by large cloud providers".

Ground rules for this article.

  • Financial figures come only from NVIDIA's 10-K and 10-Q filings with the SEC and its own earnings releases. NVIDIA's fiscal year ends in late January, so "fiscal 2026 (FY2026)" is the year that ended on January 25, 2026. That is one year off from the calendar year.
  • This was written on November 17, 2026 (JST). NVIDIA is scheduled to report Q3 of fiscal 2027 on November 17, U.S. time, and those results were not out at the time of writing. The latest quarterly data used here is Q2 (ended July 26, 2026).
  • Jensen Huang's resignation as CEO on November 11, 2026 is covered in detail in Jensen Huang's resignation and NVIDIA's governance. Here it appears only as one source of uncertainty in the scenarios.
  • Performance figures are the theoretical peaks each vendor publishes. Precision, dense versus sparse, and what counts as one chip are not aligned, so the numbers in the tables cannot be compared directly.
  • The scenarios and ratings are the author's analysis. They are not investment advice.

1. The short answer: what "CUDA monopoly" explains, and what it doesn't

Let's start by sorting out the familiar story.

What "CUDA monopoly" does explain. AI research and development ran on NVIDIA GPUs and CUDA for a long time. NVIDIA's 10-K says it introduced CUDA in 2006 and that "over 7.5 million developers" use CUDA and its other software tools (a company figure; the counting method is not disclosed). Libraries such as cuBLAS for matrix multiplication, cuDNN for neural-network operations and NCCL for GPU-to-GPU communication have been tuned across many GPU generations. Existing code, know-how and people all assume CUDA, and that becomes a switching cost.

What "CUDA monopoly" does not explain.

  • PyTorch's AMD build uses the torch.cuda API as is. User code keeps saying cuda and runs on AMD GPUs (Section 4).
  • Google TPU and AWS Trainium both officially offer ways to run as PyTorch devices, shipped as each vendor's separate package (Section 4).
  • Even so, NVIDIA's gross margin was 75.0% in Q2 of fiscal 2027 (Section 8). If the software barrier is falling but prices hold, the advantage must come from more than that barrier.

NVIDIA itself lists the principal competitive factors in the "Competition" section of its 10-K:

performance, breadth of product offerings, access to customers and partners and distribution channels, software support, conformity to industry standard APIs, manufacturing capabilities, processor pricing, and total system costs

Software is one item out of eight. This article rearranges those eight items into layers and looks at which are thick and which are thin.

Do you actually know whether self-hosting beats the API?

Self-hosted LLM technology selection — quantization, GPU budget, commercial licensing

2. Where NVIDIA stands, in numbers (through Q2 fiscal 2027)

Here is the size and shape of the business, using figures from the filings.

ItemFigurePeriod / source
RevenueUSD 96.2 billion (+106% year over year)Q2 fiscal 2027 (ended July 26, 2026), 10-Q
of which Data CenterUSD 89.0 billion (+117%)Same
of which HyperscaleUSD 48.7 billionSame
of which AI Clouds, Industrial & EnterpriseUSD 40.3 billionSame
Gross margin (GAAP)75.0%Same
Company outlook for Q3Revenue USD 108.0 billion ±2%, gross margin 74.0% ±0.5 pointsEarnings release of August 26, 2026
Revenue (full year)USD 215.9 billionFiscal 2026 (ended January 25, 2026), 10-K
of which Data CenterUSD 193.7 billion (compute USD 162.4 billion, networking USD 31.4 billion)Same

Data Center is about 92.5% of revenue (89.0 ÷ 96.2, author's calculation). NVIDIA's future is, for practical purposes, the future of data-center AI infrastructure.

The Q3 outlook was issued in August by the management team that still included Huang. Whether results matched it will only be known after the November 17 (U.S. time) report.

Starting in Q1 of fiscal 2027, NVIDIA changed how it breaks down revenue. The Q2 10-Q no longer shows the old "Compute" and "Networking" split. It uses customer-type categories instead: "Hyperscale" and "AI Clouds, Industrial, & Enterprise (ACIE)". The 10-Q text does not define "Hyperscale", so this article uses the category name as is.

3. What the CUDA ecosystem actually is: three layers

People use "CUDA" to mean at least three different things. If you don't separate them, you misjudge where the moat is.

3.1 CUDA as a programming model

This is the language extension, compiler and runtime for writing programs (kernels) that run on the GPU. You write something close to C++ and have thousands of threads execute the same code. AMD's HIP offers a very similar model at this layer. AMD's conversion tool, HIPIFY, automatically rewrites CUDA runtime API calls, kernel launch syntax and keywords such as __global__ into HIP.

3.2 CUDA as libraries (CUDA-X)

Users rarely write this layer directly, yet it decides real-world speed.

  • cuBLAS: NVIDIA's documentation calls it "an implementation of BLAS (Basic Linear Algebra Subprograms) on top of the NVIDIA CUDA runtime." It picks a matrix-multiply implementation for each matrix shape and GPU generation.
  • cuDNN: a library of optimized training and inference operations, covering attention, matrix multiplication, convolution and multi-operation fusion.
  • NCCL: the GPU-to-GPU communication library. Its documentation says it provides communication primitives that are "topology-aware", and it implements collectives such as AllReduce by handling communication and computation in a single kernel. It supports PCIe, NVLink, InfiniBand and IP sockets.

The strength of this layer lies less in the number of functions than in accumulated optimization. With each new GPU, the fastest implementation for each matrix shape and the choice of communication paths were updated. A rival can offer functions with the same names, but matching that level of optimization across every combination takes time.

3.3 CUDA as people and habits

This is the 7.5 million developers, university courses, profilers and other tools, search results for error messages, and in-house knowledge of the "write this kernel this way and it's fast" kind. It is hard to document, and porting tools cannot carry it.

3.4 Where switching costs come from

AMD is candid about the limits in its HIPIFY documentation. In summary:

  • HIPIFY does not automatically convert all CUDA code to HIP. HIP is not a complete replacement for CUDA.
  • CUDA libraries or third-party libraries that have no HIP equivalent cannot be translated.
  • Code optimized for performance on NVIDIA GPUs may need additional work to perform well on AMD GPUs.

So switching costs arrive in two stages: the cost of making the code run, and the cost of making it run as fast. The second is usually larger, and much harder to estimate in advance.

4. PyTorch makes the moat both shallower and deeper

Most AI developers do not write CUDA directly. They use it indirectly through frameworks such as PyTorch. How well the framework supports other vendors' hardware therefore decides how easy it really is to switch.

4.1 Shallower: the main rivals became devices you can use from PyTorch

Here is the state of play as of November 2026, as confirmed in official documentation.

HardwareHow you use it from PyTorchWhat the official docs say
AMD GPUs (ROCm)Keep torch.device('cuda')PyTorch "intentionally reuses the existing torch.cuda interfaces". The device names 'rocm' and 'hip' are not valid; use 'cuda'
Apple Silicon (MPS)torch.device("mps")A backend for GPU training on Metal
Google TPU (TorchTPU)torch.device("tpu")A PyTorch-native backend co-developed with Meta. Eager mode by default; supports torch.compile
AWS Trainium (Neuron)device='neuron'Native PyTorch (TorchNeuron) with eager mode, torch.compile, FSDP and DDP

Note, however, that TorchTPU (torch_tpu) and Neuron's PyTorch support (torch-neuronx and related packages) are not part of PyTorch itself; Google and AWS each distribute them separately. "Native" is the vendors' word, and you need to check each vendor's documentation for which operations and features are covered.

The AMD case is telling. NVIDIA's own API name, CUDA, has become the common entry point for porting. From the user's point of view, the string cuda in their code is less and less NVIDIA's property.

On the TPU side, there was a big shift. Running PyTorch on TPUs used to require PyTorch/XLA. Google Cloud's documentation explains that PyTorch/XLA relied on "lazy-tensor graph recording", which often meant adding code to mark compilable regions. On October 19, 2026, the PyTorch/XLA repository was marked deprecated and began directing users to its successor, TorchTPU. The README says the repository will be archived on November 15, 2026, but when I checked through the GitHub API on November 17, it had not yet been archived.

Inference servers and kernel languages are multi-hardware too.

  • vLLM: the official docs list NVIDIA CUDA, AMD ROCm, Intel XPU and Apple Silicon (via vLLM-Metal), among others, and describe a mechanism for external hardware plugins. AWS names its "vLLM Neuron" plugin as the recommended path on Trainium.
  • Triton: a language and compiler for writing kernels in Python. Its README lists supported hardware as "NVIDIA GPUs (Compute Capability 8.0+)" and "AMD GPUs (ROCm 6.2+)".
  • XLA (OpenXLA): an open-source compiler that optimizes PyTorch, TensorFlow and JAX models for GPUs, CPUs and dedicated accelerators. It is developed jointly by AMD, AWS, Google, Intel, Meta, NVIDIA and others.

4.2 Deeper: defaults and version lag

NVIDIA still holds advantages at the framework layer.

Defaults. On PyTorch 2.15.0's install page, typing pip3 install torch torchvision on Linux installs the CUDA 13.2 build. To get the ROCm build, you have to point at a separate index (whl/rocm10.0). Anyone who starts without thinking about it starts on CUDA.

Version lag. The ROCm 10.1.0 compatibility matrix validates PyTorch only up to 2.14.0, and lists vLLM 0.29.0 and SGLang 0.5.18 as "requires PyTorch 2.13.0". PyTorch's stable release is 2.15.0. PyTorch itself also ships ROCm builds, so this is not the only combination that works on ROCm. Still, a vendor-validated stack that trails upstream means a little friction every time you want a new feature.

The same API does not guarantee the same numbers. PyTorch's HIP documentation says TF32 (a reduced-precision format used inside matrix multiplication) works on AMD's MI300 via hipBLASLt, but notes hardware-level numerical differences from NVIDIA's implementation. Even with the same API, you need to re-validate accuracy.

4.3 What PyTorch changed is who owns the moat

NVIDIA does not own the PyTorch layer. Every vendor competes on the same ground there. As a result, switching costs now differ sharply by type of user.

User typeSwitching costWhy
Runs standard models through PyTorch or vLLMLow to mediumLikely to run after changing the device. Performance checks, accuracy validation and swapping operational tooling are still needed
Depends on custom CUDA kernels or CUDA-only librariesHighHIPIFY can't convert everything, and performance has to be re-tuned
Large cloud provider with its own chips and compiler teamLowest per acceleratorThe fixed porting cost is spread over a huge fleet (Section 12)

The last row matters most for NVIDIA. The biggest customers are the ones best placed to recover the cost of switching.

In large-scale AI training and inference, how fast you can connect many GPUs matters as much as how fast one GPU is. The basics of GPU-to-GPU communication, with the math, are in Section 8 of the pillar article. Here we look only at the competitive angle.

5.1 Two kinds of "connecting"

  • Scale-up: connecting dozens of GPUs within a rack so they act like one big computer. At NVIDIA, NVLink and NVLink Switch do this. GB200 NVL72 puts 72 GPUs in one NVLink domain and quotes 130 TB/s in total (the sum of each GPU's bidirectional bandwidth, not bisection bandwidth). For Vera Rubin NVL72, NVIDIA's own documents disagree: the datasheet and the NVLink product page say 216 TB/s, while the January 2026 announcement and the developer blog say 3.6 TB/s per GPU and 260 TB/s per rack.
  • Scale-out: connecting rack to rack, thousands to hundreds of thousands of accelerators. At NVIDIA, InfiniBand (Quantum) and Ethernet (Spectrum-X) do this.

5.2 What the Mellanox acquisition meant

NVIDIA completed its acquisition of Mellanox Technologies on April 27, 2020, for a transaction value of USD 7 billion. The 10-K says the deal added networking to NVIDIA's offerings and let it build platforms at "data center scale".

In fiscal 2026, networking was USD 31.4 billion of Data Center revenue, about 16.2% (author's calculation; about 11.3% in fiscal 2025 and about 18.0% in fiscal 2024). Selling networking alongside GPUs is one clue that customers are choosing NVIDIA as a "system", not as a parts supplier.

5.3 Catching up through open standards

At the network layer, industry standards are pushing back.

  • UALink: an open standard for scale-up links between accelerators. The UALink 200G 1.0 specification is described as connecting up to 1,024 accelerators in an AI computing pod at 200G per lane.
  • Ultra Ethernet: an Ethernet standard for AI and HPC. Specification 1.0 was published on June 11, 2025, and 1.1 on October 12, 2026.

NVIDIA describes Spectrum-X itself as standards-based Ethernet with support for the open SONiC network OS. The scale-out layer appears to be moving from proprietary technology toward performance competition on top of standards.

What stands out is that NVIDIA has begun opening NVLink to other companies' chips. According to the 10-K, NVIDIA introduced "NVLink Fusion" in fiscal 2026 so that large cloud providers and designers of custom ASICs (chips designed for a specific purpose) can integrate their own CPUs and dedicated chips (XPUs) into NVIDIA's platform. The product page says Amazon's Annapurna Labs (AWS's chip design arm) will be the first to work with NVIDIA on NVHBM, NVIDIA's memory technology.

This move assumes custom ASICs will keep growing. Even if a customer uses its own chip, the rack, interconnect, networking, and power and cooling design ride on NVIDIA's specifications. My reading is a strategy of giving ground at the chip layer while staying in at the interconnect and system layers.

6. Where competitors stand: hardware and software

NVIDIA's 10-K names AMD, Huawei and Intel as competitors, and also names Alibaba, Alphabet, Amazon, Baidu, Huawei and Microsoft as "large cloud services companies with internal teams designing hardware and software". This section looks at AMD, Google and AWS, which I could check against official sources.

6.1 Vendor specs side by side (conditions are not aligned)

ProductMemory capacityMemory bandwidthPeak FP8 (vendor)Caveats
NVIDIA Blackwell Ultra (one GPU in GB300)279 GB HBM3E8 TB/s10 PFLOPS (sparse; dense 5)NVIDIA datasheet
NVIDIA Rubin GPU (one GPU)288 GB HBM4(omitted; sources disagree)FP8/FP6 training 17.5 PFLOPS (dense)HGX page table
AMD Instinct MI355X (MI350 Series)288 GB HBM3E8 TB/s5 PFLOPS (10.1 with structured sparsity)AMD product page
AMD Instinct MI455X (MI400 Series)432 GB HBM423.3 TB/s20.1 PFLOPS (OCP FP8)AMD product page. "Launch Date 7/23/2026" is AMD's term and not necessarily a shipment date. The Helios page also says 19.6 TB/s per GPU, so AMD's documents disagree
Google TPU7x (Ironwood, one chip)192 GiB HBM (about 206 GB)7,380 GBps4,614 TFLOPSDense vs sparse not stated. Two chiplets per chip

All this table supports is this: on vendor memory capacity and bandwidth, AMD's next generation (MI455X) lists figures equal to or above NVIDIA's next generation (Rubin), and the current generations (MI355X and Blackwell Ultra) are close. It does not say which is faster. Theoretical peaks are not effective performance, and results swing widely with software maturity, communication and workload shape. How to read these numbers is covered in detail in the pillar article.

For condition-matched comparisons, there is MLCommons' MLPerf. In MLPerf Training v6.1, published on November 16, 2026, 24 organizations submitted results. MLCommons says systems "utilized NVIDIA, AMD and Google accelerator hardware", including NVIDIA's Rubin generation and Google's "TPUv7 (Ironwood)", as MLCommons calls it, for the first time.

6.2 AMD: the challenger that chose the path closest to CUDA

AMD describes ROCm as its "open software stack". Its library lineup maps closely onto NVIDIA's.

RoleNVIDIAAMD (ROCm)
Matrix multiplicationcuBLASrocBLAS, hipBLAS, hipBLASLt
Neural-network operationscuDNNMIOpen, Composable Kernel
GPU-to-GPU communicationNCCLRCCL
Programming modelCUDAHIP (with the HIPIFY converter)

On hardware, AMD has also moved to rack-scale products. According to AMD's product page, "Helios" holds 72 MI455X or MI450X GPUs connected with UALink over Ethernet (UALoE), for 260 TB/s of aggregate scale-up bandwidth. It is the same "72 accelerators tied together as one" shape as NVL72.

AMD's strategy is to lower the switching barrier by staying close to how CUDA is written. PyTorch reusing the torch.cuda API is a result of that. Its weak spots are the version lag and optimization depth covered in the previous section.

6.3 Google TPU: from its own path toward PyTorch

Google TPU grew up on its own software, centered on JAX and XLA. The TPU7x documentation states that JAX and PyTorch are the frameworks you can use on TPU7x and that TensorFlow is not supported. One pod has 9,216 chips, linked by ICI (Inter-Chip Interconnect, TPU's dedicated chip-to-chip link).

The big change in 2026 is TorchTPU, mentioned above. Google worked with the PyTorch team so that TPUs can be used with ordinary PyTorch code. That lowers the biggest barrier, "you have to learn JAX to use TPUs".

6.4 AWS Trainium and Inferentia: optimizing inside its own cloud

AWS's Neuron SDK is the software for running AI on Trainium and Inferentia. Per its documentation, the recommended training path is native PyTorch (TorchNeuron) with TorchTitan, and the recommended inference path is the vLLM Neuron plugin. For engineers who want to push performance, it offers NKI (Neuron Kernel Interface), which exposes Neuron's ISA (instruction set) directly. In CUDA terms, this is the layer where you write custom kernels.

Supported hardware includes Inf2, Trn1, Trn2 and Trn3; the first-generation Inf1 is no longer supported. Because AWS's chips are only available inside AWS's cloud, the question is not merchant market share but how much of its own demand AWS can serve with something other than NVIDIA.

6.5 The broader custom-ASIC trend

Google's TPU and AWS's Trainium are dedicated chips (ASICs) that large cloud providers designed for themselves. That is what the 10-K means when it names "large cloud services companies with internal teams designing hardware and software" as competitors.

The key point is the structure: the largest customers are also the largest potential competitors. Section 9 looks at this with numbers.

7. The low-precision race and the shifting inference market

7.1 Number formats: from proprietary to standard

AI arithmetic has moved to formats with fewer bits: FP16, FP8, FP4. Smaller formats let the same circuitry do more operations and cut memory traffic (see Section 3 of the pillar article).

Here, NVIDIA's own format (NVFP4) sits alongside open formats defined by the Open Compute Project (MXFP8, MXFP6, MXFP4 and others). AMD's MI355X highlights MXFP6 and MXFP4 support, and AWS's vLLM Neuron lists FP8 and MXFP8 quantization. The more number formats are standardized, the more any company can build "circuits that compute that format fast", and the difference moves to software and systems.

7.2 As inference grows, the terms of competition change

Training links thousands to tens of thousands of accelerators and runs for weeks, so communication and reliability dominate. Inference generates text one token at a time in response to user requests, so memory bandwidth, latency and cost per token dominate.

Inference is easier than training for purpose-built chips to break into. Once a model is fixed, you can favor efficiency over generality.

NVIDIA is responding to this shift too. According to its 10-K, in December 2025 NVIDIA entered into a non-exclusive license agreement with Groq, a maker of inference chips, for its LPU (language processing unit) technology, and hired certain Groq employees. No customer contracts, products or equity were acquired. The consideration was USD 13.0 billion paid at closing plus USD 4 billion (including imputed interest) payable within one year. NVIDIA's August 2026 earnings release says "NVIDIA Groq 3 LPX" is in full production. My reading: a GPU company paying that much for a different inference-chip design signals that NVIDIA sees purpose-built designs as a real threat in inference.

8. TCO and cost per token: the source of pricing power

8.1 Cost per token

The economics of inference ultimately come down to this formula:

Ctoken=Chour3600⋅RC_{\mathrm{token}} = \frac{C_{\mathrm{hour}}}{3600 \cdot R}
  • CtokenC_{\mathrm{token}}: cost per token
  • ChourC_{\mathrm{hour}}: total cost of running one accelerator for one hour (TCO), including amortized purchase price, power, cooling, facilities, operations staff and software
  • RR: tokens generated per second

The point of the formula: even if the numerator (price) is high, the customer pays less if the denominator (throughput) is larger still. And the denominator depends heavily not only on hardware but on software: kernels, quantization, batching and the inference server.

8.2 How far does pricing power reach?

When a customer compares NVIDIA's product with a rival's, the point at which cost per token is equal is the ceiling on the total cost the customer will accept for the NVIDIA option. Let NVIDIA's hourly total cost be CNC_N and its throughput RNR_N, and the rival's be CAC_A and RAR_A:

CNmax⁡=CA⋅RNRAC_{N}^{\max} = C_{A} \cdot \frac{R_{N}}{R_{A}}

CNmax⁡C_{N}^{\max} is a ceiling on the total cost of the NVIDIA option, not on NVIDIA's price. NVIDIA's price ceiling is what remains after subtracting power, facilities, operations and other costs. The formula also assumes both options run at the same utilization.

The larger the throughput ratio RN/RAR_N / R_A, the higher the price at which NVIDIA still gets chosen. If a rival's software matures and the ratio approaches 1, the ceiling falls to the rival's total cost. In a real comparison, porting cost and risk, supply certainty and operational familiarity also enter total cost.

In its 10-K, NVIDIA says Blackwell Ultra plus Dynamo (its inference software) cut cost per token versus the Hopper generation, and that Rubin delivers "up to a 10x reduction" versus Blackwell. These are company claims without stated conditions, and this article does not verify them. Still, the fact that NVIDIA frames its value in "cost per token" rather than "GPU price" is consistent with the formula above.

8.3 Pricing power, seen through gross margin

The result of pricing power shows up in gross margin.

PeriodGross margin (GAAP)Notes
Fiscal 2024 (ended January 28, 2024)72.7%Author's calculation from 10-K revenue and gross profit
Fiscal 2025 (ended January 26, 2025)75.0%10-K
Fiscal 2026 (ended January 25, 2026)71.1%10-K. Includes a USD 4.5 billion charge for H20 inventory and related items
Q2 fiscal 2027 (ended July 26, 2026)75.0%10-Q
Q3 fiscal 2027 (company outlook)74.0% ±0.5 pointsEarnings release of August 26, 2026. Actual not yet reported

For the fiscal 2026 decline, the 10-K cites the "business model transitioned from offering Hopper HGX systems to Blackwell full-scale datacenter solutions" and a USD 4.5 billion charge related to H20 (a China-market product) inventory and purchase obligations. The more you sell by the rack, the larger the share of components bought from others, which plausibly pulls gross margin down (the author's interpretation). Staying in the 70s despite that shows strong pricing power.

On the other hand, gross margin moves with product transitions and inventory valuation. Don't judge from a single quarter; track the gap between the company's outlook and its results.

9. Customer bargaining power: concentration among the top customers

9.1 Concentration as disclosed in filings

NVIDIA discloses customers that account for 10% or more of revenue, without naming them.

PeriodDisclosure (figures verbatim from the 10-K and 10-Q)
Fiscal 2026One direct customer at 22% of revenue, another at 14%
Q2 fiscal 2026Two direct customers at 23% and 16%
First half of fiscal 2027Three direct customers at 16%, 15% and 13%
Q2 fiscal 2027One direct customer at 16%

"Direct customers" here include AIBs (add-in board makers), OEMs, ODMs (original design manufacturers), system integrators, distributors, cloud providers and AI model makers. A direct customer may build servers and ultimately deliver them to another company, so "top customer" does not necessarily mean "end user". The 10-Q also says that some indirect customers individually represent 10% or more of revenue, and that NVIDIA estimates "one AI research and deployment company" contributed "a meaningful amount" of revenue by buying cloud services from NVIDIA's customers.

In Q2 of fiscal 2027, Hyperscale revenue was USD 48.7 billion, about 50.6% of total revenue and about 54.7% of Data Center revenue (author's calculation).

The 10-Q, however, does not define this category. It also says ACIE growth includes demand from "hyperscalers utilizing AI clouds", and in Q2 one company was moved from ACIE to Hyperscale because its business model changed. The Hyperscale share therefore does not precisely measure dependence on large cloud providers. This article uses it only as a rough proxy (the author's judgment).

9.2 A buyer-strong structure

Concentrated customers each carry more bargaining power. In NVIDIA's case, the largest customer group, especially large cloud providers, has three levers on top of that.

  1. Switching to their own chips: they already have chips such as TPU and Trainium.
  2. Adopting other vendors' GPUs: they can run AMD and others in parallel and use that in price negotiations.
  3. Amortizing porting cost: their fleets are large enough to recover fixed porting costs (Section 12).

9.3 NVIDIA's countermeasures, and the risk behind them

NVIDIA is also widening its customer base. In Q2 of fiscal 2027, ACIE revenue grew 138% year over year, faster than Hyperscale (up 102%). As noted above, though, ACIE includes large cloud providers using AI clouds, so the gap does not simply mean diversification away from them.

Behind that, though, NVIDIA is increasingly supporting its customers' funding and facilities itself. The 10-Q discloses:

  • As of July 26, 2026, USD 99 billion in equity investments and USD 25 billion in equity investment commitments.
  • A new business model with select AI cloud providers, under which NVIDIA's commitments to buy cloud services (typically six years) total USD 36 billion.
  • Guarantees with SB Energy (August 2026): for a campus leased to OpenAI under 20-year leases, NVIDIA's obligation is capped at USD 105 billion in aggregate. The guaranteed amount rises as each of nine construction phases is completed, the first expected in fiscal 2029.
  • Guarantees of select AI cloud partners' data center lease obligations, with maximum gross exposure of USD 3.5 billion. Including SB Energy, maximum gross exposure under all guarantees is USD 108.5 billion.
  • Memorandums of understanding with large capital providers to mobilize more than USD 500 billion of third-party capital, which may not lead to definitive agreements.

The 10-Q also states NVIDIA's view that AI clouds and AI model makers "currently lack the ability to secure long-term infrastructure contracts and investment-grade financing capacity". Supporting demand directly widens the customer base, but it is also a risk that bounces back onto NVIDIA if those customers' finances deteriorate. It is an important variable in the scenarios.

10. Supply chain: TSMC, CoWoS and HBM

10.1 NVIDIA owns no fabs

NVIDIA is fabless. Per its 10-K, it uses foundries such as TSMC and Samsung for wafers, buys memory from SK Hynix, Micron and Samsung, and uses CoWoS technology for packaging. Assembly and test are handled by Hon Hai, Wistron, Fabrinet and others.

CoWoS is TSMC's advanced packaging technology, which mounts a GPU die and HBM (stacked high-bandwidth memory) side by side on one base (an interposer). Large AI GPUs tend to be constrained in volume by this step and by HBM supply.

10.2 Supply commitments are both a moat and a burden

According to the 10-Q, NVIDIA's supply and capacity purchase commitments grew from USD 119 billion the previous quarter to USD 279 billion as of July 26, 2026 (about 2.3 times), mainly for memory and manufacturing capacity. The 10-K says that during shortages, lead times for some supply have extended beyond twelve months.

That cuts both ways.

  • As a moat: competitors buy from the same TSMC and the same HBM makers. While supply is tight, whoever locked in large allocations first can ship. The structure favors the biggest player with the most capital.
  • As a burden: if demand comes in weaker than expected, the portion of purchase commitments that cannot be cancelled or changed turns into inventory write-downs and provisions. The 10-Q says some agreements may be cancelable, rescheduled or adjustable before firm orders are placed, and that changes may add costs. The USD 4.5 billion H20 charge in fiscal 2026, though caused by export controls, is an example.

This article does not use third-party estimates such as what share of TSMC's CoWoS capacity NVIDIA has secured, because they cannot be confirmed from primary sources.

11. Original analysis 1: replication cost and time, layer by layer

Based on the facts above, here are NVIDIA's strengths split into eight layers, with an assessment of how hard each would be for others to replicate. The ratings are my judgment, not facts. The evidence is the primary sources cited in each section.

LayerWhat NVIDIA hasWhere challengers areDifficulty to replicateDegree of erosion
① Compute chipsTensor Cores, FP8/FP4MI355X, MI455X, TPU7x and others match or exceed on some vendor specsMedium (catchable in 1–2 generations)Under way
② Memory and packaging supplyUSD 279 billion in purchase commitmentsCompeting for the same suppliersHigh (needs capital and advance claims)Small while supply is tight
③ Scale-up interconnectNVLink, NVL72 (130 TB/s for GB200; 216–260 TB/s for Vera Rubin, depending on the document)UALink 1.0, TPU ICI, AMD Helios (72 GPUs)High (chip, switch and rack designed together)Small to medium
④ Scale-out networkInfiniBand, Spectrum-XUltra Ethernet, incumbent Ethernet vendorsMedium (converging on standards)Under way
⑤ LibrariescuBLAS, cuDNN, NCCL and moreROCm, NKI Library, XLAMedium to high (accumulated optimization)Starting with the main operations
⑥ FrameworksNot owned by NVIDIAPyTorch, vLLM, Triton, JAX support each vendorLow (already common ground)Large
⑦ Developer base7.5 million developers, existing CUDA code, toolsEach vendor is building oneHigh (grows only with time)Small
⑧ System integration and shippingRack-scale design, ODM network, financial supportLarge cloud providers can do it themselvesHigh (but replicable by the largest)Depends on the customer

Three things stand out.

First, the "CUDA monopoly" view focuses on the layer that has eroded most (⑥). At the framework layer, the main rivals already have support (some through separate packages). For users who only run standard models, that barrier is now low.

Second, the center of gravity of the moat is moving to ②, ③ and ⑧. Advance claims on supply, interconnect that binds GPUs in groups of 72, and the ability to ship and bring up whole racks do not come from designing one chip. NVLink Fusion shows NVIDIA itself is aware of this shift.

Third, ②, ③ and ⑧ are also the layers "the largest customers can replicate". Google designs and runs TPU pods and ICI, and AWS designs and runs Trainium UltraServers, in house. NVIDIA's moat is shallower the larger the customer. Combined with the customer concentration in Section 9, I think that is NVIDIA's biggest structural risk.

12. Original analysis 2: the break-even point for switching

Let's check the claim that "the largest customers find it easiest to switch" with a simple formula.

Suppose switching cuts total cost by a fraction ss, the payback horizon is TT years, and the port succeeds with probability pp. Express the porting cost KK as "how many accelerator-years of total cost it equals". The minimum fleet size at which the porting cost is recovered is:

N∗=Ks⋅T⋅pN^{*} = \frac{K}{s \cdot T \cdot p}

Expressing KK in accelerator-years avoids having to assume a GPU price. The fleet size N∗N^{*} is proportional to KK and inversely proportional to the savings rate ss. Because porting cost is close to a fixed cost that does not depend on fleet size, the burden per accelerator shrinks as the fleet grows.

The code below computes the break-even fleet size with this formula. All inputs are illustrative placeholder values, not market data.

  • Language: Python 3.9 or later (standard library only)
  • Run: python3 switching_breakeven.py
  • Environment: the author's Mac (Apple M4 Pro), Python 3.9.6
  • The program prints Japanese labels. "移植コスト(台年)" means porting cost in accelerator-years, "削減" means savings rate, "成功確率" means success probability, and "台" means accelerators.
"""Switching break-even: above how many accelerators is the porting cost recovered (theoretical).

Express the porting cost as "how many accelerator-years of total cost (TCO) it equals".
That avoids assuming any price, isolating only how scale matters.
"""


def breakeven_fleet(port_cost_accel_years: float, saving_ratio: float,
                    horizon_years: float, success_prob: float = 1.0) -> float:
    """Return the minimum fleet size N that recovers the porting cost.

    Condition: N * saving_ratio * horizon_years * success_prob >= port_cost_accel_years
    """
    if not (0 < saving_ratio < 1 and horizon_years > 0 and 0 < success_prob <= 1):
        raise ValueError("saving_ratio は (0,1)、horizon_years > 0、success_prob は (0,1]")
    return port_cost_accel_years / (saving_ratio * horizon_years * success_prob)


def main() -> None:
    horizon = 3.0  # payback horizon (years)
    savings = [0.05, 0.10, 0.20, 0.30]  # TCO savings rate after switching
    port_costs = [10, 100, 1_000]  # porting cost (accelerator-years)

    print(f"回収期間 {horizon:.0f} 年・成功確率 100% のときの損益分岐台数")
    print("移植コスト(台年) | " + " | ".join(f"削減{s:.0%}" for s in savings))
    for k in port_costs:
        row = [breakeven_fleet(k, s, horizon) for s in savings]
        print(f"{k:>15,} | " + " | ".join(f"{n:>7,.0f}" for n in row))

    print()
    print("移植コスト100台年・削減20%・回収3年で、成功確率を変えた場合")
    for p in [1.0, 0.7, 0.5]:
        n = breakeven_fleet(100, 0.20, horizon, p)
        print(f"成功確率 {p:.0%}: {n:,.0f} 台")


if __name__ == "__main__":
    main()

Output:

回収期間 3 年・成功確率 100% のときの損益分岐台数
移植コスト(台年) | 削減5% | 削減10% | 削減20% | 削減30%
             10 |      67 |      33 |      17 |      11
            100 |     667 |     333 |     167 |     111
          1,000 |   6,667 |   3,333 |   1,667 |   1,111

移植コスト100台年・削減20%・回収3年で、成功確率を変えた場合
成功確率 100%: 167 台
成功確率 70%: 238 台
成功確率 50%: 333 台

How to read it: if porting costs "100 accelerator-years" and switching cuts total cost by 20%, you need at least 167 accelerators to recover it within 3 years. If the port only has a 50/50 chance of succeeding, that rises to 333.

This simple model suggests:

  • For users with dozens of accelerators, switching rarely pays. Only when porting is cheap and savings are large.
  • For large cloud providers with tens to hundreds of thousands of accelerators, it nearly always pays. Even at a porting cost of "1,000 accelerator-years", savings of a few percent recover it. That is the economic reason large cloud providers build their own chips.
  • What NVIDIA must defend is keeping the savings rate ss small. If software improvements keep driving down cost per token, the savings from switching to a rival shrink, and the break-even fleet size grows.

Limits of the model. It ignores discounting (the time value of money). The fleet size NN is counted in NVIDIA-GPU-equivalent units; if one chip on the other platform performs differently, the required count changes too. Also, in reality, porting is not a one-time job; each new model brings more work. Securing supply, operations staff and incident response all enter total cost. Conversely, when NVIDIA products are unavailable, switching happens regardless of savings. The formula is a tool to show how scale matters, not something to apply directly to an individual decision.

13. Medium- to long-term scenarios: bull, base and bear

From here on, this is my analysis. Rather than predict a single future, I lay out three scenarios with different assumptions and map which indicators move in each. I do not assign probabilities, because there isn't enough evidence to justify them. This is not investment advice.

13.1 Three scenarios

Bull: the system-level advantage holds

  • Assumptions: Rubin ramps on plan and the one-year product cadence continues. Software improvements preserve the cost-per-token lead, keeping the switching savings rate ss small. Customers beyond the large cloud providers (AI clouds, enterprises, governments) grow.
  • What happens: even as large cloud providers add their own chips, total demand grows faster. The share of revenue from networking and systems rises.
  • Observable signs: the gross margin outlook stays in the mid-70s. ACIE keeps outgrowing Hyperscale (though ACIE includes large cloud providers using AI clouds). The share of direct customers above 10% falls.

Base: still the leader, but a smaller share of the growth

  • Assumptions: NVIDIA stays the leader in training and frontier inference. Meanwhile, large cloud providers move part of their inference to their own chips such as TPU and Trainium, and AMD wins more inference deployments. Framework support keeps improving, and switching costs for standard models keep falling.
  • What happens: total AI infrastructure investment grows, but NVIDIA's share gradually declines. Price negotiations get tougher and gross margin drifts down slowly.
  • Observable signs: the gross margin outlook steps down. Hyperscale grows slower than Data Center overall (only a rough sign, since the category is not defined). Competitors' MLPerf submissions increase, and vendor-validated framework versions catch up with upstream.

Bear: an investment slowdown coincides with insourcing

  • Assumptions: AI investment slows because of funding constraints or delayed monetization. At the same time, large cloud providers move most of their inference to their own chips. Export controls tighten further.
  • What happens: demand falls short of supply commitments, and inventory write-downs and purchase-obligation provisions increase. Customer-support investments and guarantees turn into losses. NVIDIA cuts prices to defend volume.
  • Observable signs: revenue misses the company's outlook. Inventory provisions rise. Supply purchase commitments start to shrink. Customer financing frameworks fail to reach definitive agreements.

13.2 Sensitivity: which variables move the scenarios

VariableMoves toward bull whenMoves toward bear whenIndicator to watch (source)
Insourcing by large cloud providersDemand grows faster than their own chipsMost of their inference moves to their own chips10%+ customer disclosures and Hyperscale share of revenue (10-Q; the category does not precisely measure dependence)
Cost-per-token gapNVIDIA's improvements outpace rivalsRivals' software matures and the gap narrowsMLPerf results; verification of vendor claims
Supply (HBM, packaging)Stays tight, so advance claims pay offLoosens, so commitments become a burdenPurchase commitment balance, inventory provisions (10-Q)
Framework portabilityVersion lag and numerical differences persistMajor rivals support new releases in step with upstreamROCm compatibility matrix; TorchTPU and Neuron coverage
Customer financesAI cloud financing comes togetherFinancing stallsNotes on guarantees, investments and cloud commitments (10-Q)
RegulationRules ease or stabilizeCovered products or regions expandRisk factors in the 10-K and 10-Q
LeadershipStrategy continues under a new CEOA prolonged CEO search slows decisionsOfficial company announcements

The two rows I think matter most are the top two: insourcing by large cloud providers and the cost-per-token gap. As Sections 11 and 12 showed, NVIDIA's moat is shallower for larger customers, and the category centered on large cloud providers (Hyperscale) is about half of revenue. Because the category is not defined, that "about half" involves some inference on my part.

14. Leading indicators: what to watch to update your view

Scenarios should be updated every time new numbers arrive. Everything below can be tracked in public materials.

  1. Q3 fiscal 2027 results (scheduled for November 17, 2026, U.S. time): did revenue land within the USD 108.0 billion ±2% outlook? Did gross margin land within 74.0% ±0.5 points? What is the Q4 outlook?
  2. Disclosure of 10%+ customers: is the share of the top direct customers rising or falling?
  3. Revenue by category: the gap between Hyperscale and ACIE growth. The categories are not defined, and large cloud providers also buy through AI clouds counted in ACIE, so this does not precisely measure dependence on them.
  4. Supply purchase commitments and inventory provisions: do commitments keep growing? Are provisions rising?
  5. Customer-support programs: balances of equity investments, guarantees and cloud purchase commitments, and their effect on earnings.
  6. Competitors' software: which PyTorch versions the ROCm compatibility matrix validates; which models and features TorchTPU and Neuron support.
  7. Condition-matched performance comparisons: which hardware submits, at what scale, in MLPerf training and inference rounds.
  8. Roadmap execution: after Rubin, NVIDIA has announced Vera Rubin Ultra NVL576, a new rack design called Kyber, and the following Feynman generation. The 2026 official materials I checked do not give dates for these. Tracking the gap between announced plans and actual shipments is worthwhile.

15. Limits of this analysis

  • No market-share figures are used. Third-party share estimates vary in method and timing and cannot be confirmed from primary sources. So this article does not answer "what is NVIDIA's market share".
  • Customers' internal costs are unknown. How much large cloud providers save with their own chips is not public. The numbers in Section 12 are placeholders.
  • Vendor specs are not measurements. The comparison table in Section 6 does not show which product is faster.
  • Company claims are not verified. Claims such as "up to one-tenth the cost per token" are company announcements without stated conditions.
  • The author's hands-on scope. My production experience is an inference platform that runs vLLM with AWQ-quantized Qwen3-8B on a single Tesla T4. I have not operated large GPU clusters, TPUs or Trainium. Even at that scale, which quantization methods and number formats were usable was set by the GPU generation and software support, and that shaped the design. Large-scale users should treat this article's framework as a starting point and measure on their own workloads.

Summary

  • NVIDIA's strength is not a "CUDA monopoly" in a word, but a stack of chips, supply, interconnect, networking, libraries, developers and system integration.
  • The framework layer is the most eroded. PyTorch's AMD build reuses the torch.cuda API, and TPU and Trainium can now be used as PyTorch devices through each vendor's separate package. Friction remains, though: the default install is the CUDA build, vendor-validated versions lag, and HIPIFY has limits.
  • The center of gravity of the moat is moving to advance claims on supply, NVLink scale-up and rack-scale system integration. NVLink Fusion reads as a strategy for staying in the interconnect and system layers on the assumption that custom ASICs will grow.
  • The break-even point for switching depends strongly on fleet size. The largest customers find it easiest to switch, and the Hyperscale category, centered on large cloud providers, was about half of revenue in Q2 fiscal 2027 (the category is undefined, so this is not a precise measure of dependence). I think that is NVIDIA's biggest structural risk.
  • Gross margin was 71.1% in fiscal 2026 and 75.0% in Q2 fiscal 2027, so pricing power is strong for now. Whether it lasts depends on whether NVIDIA can keep the cost-per-token gap.

This article was written before the Q3 results. When the numbers come out, plug them into the indicators in Section 14 and see which scenario they point toward. The technical foundation of why GPUs are so strong at AI is covered in Why NVIDIA GPUs are so strong at AI.

How to compare cost per token across machines is covered in CPU vs GPU vs TPU, the export controls that shape the China market in The US–China AI chip war and export controls, and the road to here in A history of NVIDIA.

Frequently asked questions

What is NVIDIA's advantage in one sentence?
This article's conclusion is that it can't be said in one sentence. CUDA and its libraries are real strengths, but if that were all, rivals' software work would close the gap. In practice, NVIDIA combines the GPU itself, secured HBM and packaging supply, NVLink that links GPUs in groups of 72, InfiniBand and Spectrum-X networking, libraries such as cuBLAS, cuDNN and NCCL, and the ability to ship whole racks. A rival can catch up on one layer, but assembling all of them at once takes time and capital.
Does CUDA make it impossible to switch to other AI chips?
It depends heavily on who you are. If you only run standard models through PyTorch or vLLM, AMD ROCm, Google's TorchTPU and AWS Neuron (the latter two are separate vendor packages) now let PyTorch use their hardware as devices, so the barrier is lower than it used to be. If you depend on custom CUDA kernels or CUDA-only libraries, you face rewriting and re-tuning: AMD itself says HIPIFY cannot convert everything automatically. The more accelerators you run, the easier it is to recover the porting cost.
How close are AMD, Google TPU and AWS Trainium?
On some vendor specifications they match or exceed NVIDIA's latest GPUs: AMD's next-generation MI455X lists 432GB of HBM4 at 23.3TB/s (NVIDIA's Rubin GPU: 288GB), and Google's TPU7x lists 192GiB of HBM per chip. Each vendor is also building out its software. But vendor figures are theoretical peaks, and precision, dense versus sparse, and what counts as one chip are not aligned. Compare using condition-matched results like MLPerf, or measurements on your own workload.
How should I think about NVIDIA's future? Should I buy the stock?
This article is not investment advice and does not recommend buying or selling any stock. Instead of a single forecast, it lays out bull, base and bear scenarios with explicit assumptions, and maps which indicators would move you toward each. The ones to watch most closely are the gross margin outlook, the revenue share of customers above 10%, revenue by category (Hyperscale and ACIE), the Rubin ramp, and supply commitments alongside inventory provisions. Use the numbers in SEC filings and earnings materials, and make your own decisions.

References

友田

友田 陽大

Developer of a METI Minister's Award–winning product. With TypeScript + Python + AWS, I deliver SaaS, industry DX, and production-grade generative AI (RAG) end to end — from requirements to infrastructure and operations — single-handedly.

Do you actually know whether self-hosting beats the API?

Self-hosted LLM technology selection — quantization, GPU budget, commercial licensing

When to use AWQ / GPTQ / FP8 / GGUF, VRAM estimates that include the KV cache, the break-even point against per-token API pricing, and the license terms of Apache-2.0 / Llama / Qwen / Gemma. From hands-on experience adopting commercially usable open quantized models stage by stage against license, quality, and speed trade-offs, I help you assemble the decision criteria before you implement.

Available for both project-based (contract) and advisory engagements. Start with a free 30-minute consult.

Also worth reading