Tensor Core

The specialised silicon at the heart of the AI revolution — and the reason your AI budget keeps growing.

NVIDIA's Tensor Cores generate over $65 billion in annual revenue for the company and underpin virtually every large language model training run on the planet. These specialised processing units are the engine room of the AI economy: without them, training a model like GPT-4 would take years instead of months, and inference costs would be 5-10x higher. For enterprises, understanding Tensor Cores is essential for making informed decisions about AI infrastructure investment, cloud computing costs, and the build-versus-buy calculus for AI capabilities. With AI infrastructure spending projected to exceed $300 billion globally by 2027, Tensor Core performance directly determines the cost and feasibility of your organisation's AI ambitions.

What It Is

A Tensor Core is a specialised processing unit within NVIDIA's GPU chips designed specifically to accelerate matrix multiplication operations — the mathematical foundation of artificial intelligence. While traditional processors (CPUs) and even standard GPU cores handle a wide variety of computing tasks, Tensor Cores are purpose-built to perform the specific calculations that neural networks require, doing so with dramatically higher throughput and energy efficiency. Think of it as the difference between a general-purpose utility vehicle and a Formula 1 car: both have engines, but one is optimised for a very specific type of performance.

Matrix multiplication is the mathematical operation at the core of neural networks. Every prediction, classification, or generation an AI model makes involves multiplying vast matrices of numbers — billions of these operations for a single inference, trillions during training. Tensor Cores perform these operations in a single clock cycle what would take multiple cycles on standard processing units. The latest Blackwell architecture Tensor Cores can perform over 2,000 trillion floating-point operations per second (2 petaflops) per chip, a level of performance that makes modern AI practical and affordable.

Tensor Cores have evolved significantly across NVIDIA's GPU generations. Volta (2017) introduced the first Tensor Cores with FP16 precision. Ampere (2020) added support for lower precisions (INT8, FP8, and even INT4), dramatically increasing throughput for inference workloads. Hopper (2022) introduced the Transformer Engine, which dynamically adjusts precision during computation, and Blackwell (2024) pushed performance even further with next-generation Tensor Cores optimised for the largest AI models. Each generation has delivered 2-4x improvements in AI-specific performance, driving down the cost of AI computation.

Why It Matters for Business

For enterprises, Tensor Core performance translates directly to AI economics. Training a large language model from scratch on previous-generation hardware might cost $50-100 million in compute resources. The latest Tensor Core-equipped GPUs reduce this by 50-75%, making previously prohibitive AI projects feasible. For inference — the ongoing cost of running AI models in production — Tensor Core efficiency determines your cost per query, cost per user, and ultimately whether your AI-powered product or service is economically viable at scale.

The Tensor Core monopoly has significant strategic implications. NVIDIA controls over 85% of the AI accelerator market, and Tensor Core performance is the primary differentiator that maintains this dominance. This concentration creates supply chain risk — during the 2023-2024 AI boom, NVIDIA H100 GPUs were selling at 300% premiums with 12-month wait times. For enterprises planning AI infrastructure investments, understanding Tensor Core capabilities helps evaluate whether to invest in NVIDIA hardware, wait for alternatives from AMD and Intel, or rely on cloud providers who absorb the hardware management complexity.

Energy consumption is an increasingly important consideration. AI workloads are among the most energy-intensive computing tasks, and Tensor Cores, while more efficient than general-purpose processors, still consume significant power. A single NVIDIA H100 GPU draws 700 watts, and a typical training cluster might contain thousands of these chips. The energy cost of AI training and inference is becoming a material line item for enterprises and a sustainability concern that boards are increasingly required to address.

Enterprise Use Cases

Meta (Facebook) operates one of the world's largest AI training clusters, with over 350,000 NVIDIA H100 GPUs dedicated to training its Llama series of large language models and recommendation systems. The company has publicly stated that Tensor Core performance improvements are the primary enabler of its open-source AI strategy, making it economically feasible to train and release increasingly capable models. Without the efficiency gains from each new Tensor Core generation, Meta's ambitious AI roadmap would be financially impractical.

Tesla uses Tensor Core-equipped GPUs extensively for training its autonomous driving neural networks. The company processes millions of video clips from its vehicle fleet, training vision models that must make split-second driving decisions. Tesla's AI infrastructure, powered by thousands of NVIDIA GPUs with Tensor Cores, represents one of the most demanding AI workloads in the world, and the company has invested billions in this capability. The Tensor Core throughput directly determines how quickly Tesla can iterate on its self-driving technology.

Financial institutions like JP Morgan and Goldman Sachs deploy Tensor Core GPUs for quantitative modelling, risk assessment, and algorithmic trading. These institutions have found that Tensor Core-accelerated models can evaluate risk scenarios in seconds that previously took hours, enabling real-time risk management during market volatility. Goldman Sachs reported that GPU-accelerated Monte Carlo simulations run 40x faster than CPU-based implementations, fundamentally changing how the firm approaches risk modelling.

Vendor Landscape

NVIDIA dominates the AI accelerator market with over 85% share, and Tensor Cores are the primary technical differentiator maintaining this position. The H100 (Hopper architecture) and B200 (Blackwell architecture) are the current flagship products for enterprise AI. AMD competes with its Instinct MI300X accelerator, which offers competitive performance for inference workloads but lacks the mature software ecosystem (CUDA) that surrounds NVIDIA's Tensor Cores. Intel's Gaudi3 accelerator targets cost-sensitive deployments but holds minimal market share.

Cloud providers offer access to Tensor Core GPUs through their platforms: AWS (P5 instances with H100), Google Cloud (A3 instances), Microsoft Azure (ND H100 v5), and Oracle Cloud. Pricing typically ranges from $3-5 per GPU hour for H100-class instances, with significant volume discounts available through committed spend agreements. For most enterprises, cloud-based access to Tensor Core GPUs is more cost-effective than on-premises deployment unless utilisation exceeds 60-70%.

Strategic Considerations

When evaluating AI infrastructure, focus on total cost of ownership rather than raw GPU prices. Factor in power consumption, cooling requirements, software licensing, and engineering talent costs. For most enterprises, starting with cloud-based GPU access and migrating to dedicated hardware only when utilisation justifies it is the prudent approach. Monitor the competitive landscape: AMD and Intel are investing billions to challenge NVIDIA's dominance, and increased competition will benefit buyers. Consider the geopolitical dimension — US export restrictions on advanced AI chips to China and other markets may affect global supply chains and pricing. Finally, plan for rapid obsolescence: each new GPU generation delivers 2-4x improvements, meaning hardware purchased today will be significantly less competitive within 18-24 months. Structure procurement to maintain flexibility.