SambaNova Systems

SambaNova Systems is an American artificial-intelligence hardware and cloud company, founded in 2017 in Palo Alto, California, that designs its own Reconfigurable Dataflow Unit (RDU) accelerators, complete rack-scale systems, and the compiler and orchestration software to run them as a full-stack platform for enterprises and cloud operators.¹ The company was started by Stanford professors Kunle Olukotun and Christopher Ré together with semiconductor executive Rodrigo Liang, on the premise that the hardware the world was using to run AI was never designed for it.¹ Its chips are the SN40L, introduced in September 2023, and the SN50, announced in February 2026, built around a dataflow architecture with a three-tier memory system of on-chip SRAM, co-packaged HBM, and DDR DRAM, and the company has shifted its business from selling training systems to selling AI inference, both as hosted tokens and as on-premises racks.^{18 • 3 • 2 • 4}

History and founding

SambaNova began operations in 2017 in Palo Alto, California.¹ Its founding combined Olukotun's work on chip multiprocessors and Ré's on machine learning systems with Liang's semiconductor industry experience.¹ The company's early years were defined by very large private funding: in April 2021 it raised $676 million in a round led by SoftBank Group's Vision Fund 2, valuing the company at more than $5 billion.⁵

Pivot to inference. In 2025 the company laid off 77 employees, roughly 15 percent of its approximately 500-person workforce, while shifting its focus from training to inference design.⁷ SambaNova later said 2025 was its strongest, record-breaking year and that it had shifted its focus to inference, the computation required to answer a user's query in a chatbot.⁴

Architecture: the reconfigurable dataflow unit

The RDU is SambaNova's answer to the GPU. SambaNova argues that GPUs make multiple redundant calls back to memory across separate kernel launches, while the RDU combines many operations into a single kernel call, so data can flow between computation steps without repeatedly returning to memory.⁶ The streaming dataflow architecture fuses hundreds of operations into one kernel call without hand-written kernels, exploiting pipeline, data, and tensor parallelism on-chip.² According to the company's technical paper, this fusion lets weight loads overlap with computation and achieves over 85 percent HBM bandwidth utilization for weights and KV cache, whereas optimized H100 GPU implementations rarely exceed 50 percent.²

Three-tier memory. The SN40L's defining feature is its memory hierarchy: 520 MiB of on-chip SRAM in the Pattern Memory Units, 64 GiB of co-packaged HBM, and up to 1.5 TiB of DDR DRAM on pluggable DIMMs, with HBM bandwidth of 1.8 TB/s and DDR bandwidth of 200 GB/s per socket.² The SambaFlow compiler stages data across the three tiers, with DRAM sitting on the accelerator card rather than in the host server.⁹ This capacity is the basis of the company's large-memory inference bet: the SN40L provides about 2.5 times higher aggregate memory capacity per socket than the NVIDIA GH200, letting a single system hold very large models and more simultaneous models in memory.^{2 • 10}

The SN40L itself is fabricated in TSMC 5 nm as a 2.5D Chip-on-Wafer-on-Substrate chiplet design containing two Reconfigurable Dataflow Dies and co-packaged HBM, and delivers 638 BF16 TFLOPS of peak compute using 1040 Pattern Compute Units and 1040 Pattern Memory Units.² It is the company's fourth-generation RDU and is based on the "Cerulean" architecture.^{7 • 11}

Products and systems

SambaNova sells through three channels built around the same rack platform: SambaRack systems with 16 SN40L RDUs, SambaCloud, which supports inference models including DeepSeek, Llama, and Qwen, and SambaManaged, a fully managed inference cloud that can be built in a customer's own data center.⁷ In February 2024 the company announced Samba-1, a one trillion parameter composition of more than 50 open-source generative AI models, including Llama2 7B/13B/70B, Mistral 7B, DeepSeek Coder, Falcon 40B, CLIP, and Llava, aimed at regulated enterprises.¹²

SN50. Announced in February 2026, the SN50 delivers five times more compute per accelerator and four times more network bandwidth than the previous generation, links up to 256 accelerators over a multi-terabyte-per-second interconnect, and supports models above 10 trillion parameters and context lengths above 10 million tokens through the three-tier memory.³ Its SRAM cache decreased to 216 MB per chiplet from 260 MB on the SN40L, while HBM bandwidth stayed at roughly 1.84 TB/s and HBM capacity at 64 GB; the SN50 uses HBM2E rather than the SN40L's HBM3, which CEO Rodrigo Liang attributes to the dataflow architecture's ability to overlap computation and communication, avoiding supply-constrained HBM3E.⁹ A rack of SN50s draws a maximum of 30 kW, power-cappable to 15-20 kW for standard air-cooled enterprise data centers, and the chips and racks ship in the second half of 2026.⁹

By the numbers

• Compute and memory per SN40L socket: 638 BF16 TFLOPS; 520 MiB SRAM, 64 GiB HBM at 1.8 TB/s, up to 1.5 TiB DDR at 200 GB/s.²
• Throughput claims: on 16 SN40L sockets, token generation reaches 129 tokens/s/user for Llama 3.1 405B, 457 for 70B, and 1042 for 8B, with speculative decoding on the larger models.²
• Power: a SambaRack SN40L averages 10 kW, air cooled, against as much as 140 kW and liquid cooling for GPU-based systems, per SambaNova's white paper; an independent comparison puts SN40L rack power at 8-15 kW.^{6 • 13}
• Funding: $676 million (April 2021, over $5 billion valuation), $350 million-plus (February 2026), and $1 billion (July 2026, $11 billion post-money, led by General Atlantic, partly earmarked to expand the SN50).^{5 • 3}
• Revenue: analysts pegged 2025 revenue at roughly $75-100 million; Reuters reports more than $1 billion raised from investors since 2017, while one analysis puts total funding at roughly $1.5 billion.^{14 • 5}
• Pricing: SambaNova Cloud lists Llama 3.1 8B at $0.10/$0.20 and Llama 3.1 405B at $5.00/$10.00 per million input/output tokens; SambaCloud charges $6 per million output tokens for Llama 4 Maverick, against $0.47 per million tokens on spot H200 instances.^{15 • 13}

Performance claims and independent benchmarks

Most of the performance record comes from SambaNova's own papers and company-cited benchmarks. The company's arXiv paper states that the SN40L was, at the time of writing, the fastest platform for Llama 3.1 405B inference in full BF16 precision, while the compared GPU implementations use 8-bit quantization, and reports a 3.7x to 6.6x overall speedup over DGX H100 and DGX A100 for its Samba-CoE trillion-parameter composition of 150 8B expert models, with up to 19 times reduced machine footprint and 15x to 31x faster model switching.² At DAC 2025 the company claimed the SN40L produces tokens over 3 times faster, consumes 3 times less energy, and lowers model hosting costs by up to 19 times versus a DGX H100.¹⁶

Independent data is limited but partly supportive. Artificial Analysis measured SambaNova's DeepSeek-R1 671B performance at 198-255 output tokens per second on 16 RDU chips, versus roughly 250 tokens per second on eight Nvidia GPUs using FP4 quantization, so the comparison involves different chip counts and precision.¹⁰ For the SN50, SambaNova cites SemiAnalysis InferenceX benchmarks showing Llama 3.3 70B at FP8 with 1K input/1K output reaching 895 tokens per second per user versus 184 on an Nvidia B200.³ The company also claims its pricing undercuts major API providers by 5-7 times on average for comparable models, a company-cited figure, and claims 5 times the speed and 8 times the cost savings of the Blackwell B200 on its own benchmarks; Nvidia holds an estimated 85 percent share of the AI accelerator market.¹⁵ Readers should treat vendor performance and cost claims as not independently verified except where Artificial Analysis figures are cited.

How it compares with Groq, Cerebras, and Nvidia

SambaNova's distinguishing feature against its closest rivals is memory. Groq's LPU carries 230 MiB of SRAM per chip and Cerebras's wafer-scale parts carry about 40 GB of SRAM, while the SN40L pairs 520 MiB of SRAM with 64 GiB of HBM, and 1.5 TiB of DDR, so RDUs can hold larger models and more simultaneous models in memory than either competitor.^{10 • 13} SambaNova's own paper names Graphcore, Cerebras, Groq, and its own earlier RDU generations as rivals lacking the three-tier memory system.²

The CUDA moat. Nvidia's CUDA programming framework has about four million developers, and major AI frameworks, including PyTorch and TensorFlow, have often been optimized first for Nvidia hardware, which makes switching to SambaNova costly for procurement and operations teams.¹⁴ SambaNova's response has been partly cooperative: at Computex 2026 it demonstrated SN40 chips running decode acceleration alongside Nvidia Blackwell GPUs for prefill and Intel Xeon 6 CPUs, a setup it says boosted performance 2 to 3 times per chip over a GPU-only system.¹⁷ Liang has also claimed the SN50 delivers higher performance than the Blackwell B200 and more computing power for the same price.⁸ Among rivals' finances, Cerebras raised a $1.1 billion private round at an $8.1 billion valuation led by Fidelity and Atreides before withdrawing its 2024 IPO filing, and Groq won a $1.5 billion Saudi sovereign AI infrastructure deal.¹⁰

Customers, partnerships, and deployments

SambaNova's customer base spans US national laboratories, banks, and enterprises: Oak Ridge, Los Alamos, Argonne, and Lawrence Livermore National Laboratories, OTP Bank, Accenture, Aramco, SoftBank, TACC, and RIKEN.¹⁰ JPMorganChase will deploy SN40 and SN50 systems for secure on-premises AI inference.¹⁷ Samba-1 is used by customers and partners including Accenture and NetApp.¹² Argonne National Laboratory uses the systems for applications from weather forecasting to predicting how tumors respond to drug combinations, and has reported a tripling in performance speed.¹⁸ A US Department of Energy study found the RDA with the SambaFlow stack attractive for AI-for-science workloads, noting that its very large memory capacity allows training of models that typically do not fit in a GPU, and positioned it for deeper integration with supercomputers at the Argonne Leadership Computing Facility.¹⁹ SoftBank Corp. will be the first customer to deploy SN50 in next-generation AI data centers in Japan for low-latency sovereign and enterprise inference across Asia-Pacific.³

Ownership and corporate recent developments

The SN40L was introduced in September 2023, and the years since have reshaped the company. The 2025 layoffs of 77 employees accompanied the shift from training to inference.⁷ In February 2026 Intel partnered with SambaNova after reported buyout talks failed; the talks had centered on inference, as Intel's earlier cost-effective alternative to Nvidia GPUs had not succeeded.^{8 • 7} Intel CEO Lip-Bu Tan chairs SambaNova, and the collaboration pairs GPUs for prefill, SambaNova RDUs for decode, and Intel Xeon 6 CPUs for agentic tools.^{4 • 3} The company raised $350 million or more alongside the SN50 announcement, with a claimed 3 times lower total cost of ownership for agentic AI, and in July 2026 completed a $1 billion round at an $11 billion post-money valuation led by General Atlantic.^{3 • 5} Liang has said the company keeps a relatively low burn rate because it builds and ships racks rather than operating data centers.¹⁷

References

1. SambaNova, About Us
2. SambaNova SN40L: Scaling the AI Memory Wall with Dataflow and Composition of Experts, arXiv
3. SambaNova Unveils Fastest Chip for Agentic AI, Collaborates with Intel, and Raises $350M+, Business Wire
4. EXCLUSIVE: Intel looks to put millions more into SambaNova startup chaired by CEO Tan, Reuters
5. AI chip startup SambaNova valued at $11 billion in $1 billion funding round, Reuters
6. SambaNova SN40L Reconfigurable Dataflow Unit, white paper
7. Intel Eyeing AI Catchup in Inference with SambaNova Acquisition, EE Times
8. Intel partners with SambaNova after buyout talks reportedly failed, CNBC
9. SambaNova Pits Its Engineering Against Nvidia For Agentic AI, The Next Platform
10. Cerebras, Groq, and SambaNova Line Up to Compete with Nvidia, Cambrian AI Research
11. SambaNova SN40L RDU, Hot Chips 2024 presentation
12. SambaNova is First to Market with Secure, One Trillion Parameter Generative AI Model for the Enterprise, Business Wire
13. Groq LPU vs SambaNova SN40L: Llama 4 inference comparison, Airtrain
14. SambaNova Raises $350M, Partners With Intel on AI Chips, Implicator
15. SambaNova, Benched.ai
16. Invited: SambaNova SN40L: Unleashing Agentic AI with Dataflow, DAC 2025, ACM
17. SambaNova Raises $1B, Signs JPMorganChase As A Customer, EE Times
18. How SambaNova Systems is Navigating Shifts in the AI Industry, SoftBank Vision Fund
19. Accelerating Scientific Applications With SambaNova Reconfigurable Dataflow Architecture, OSTI

Topic: Encyclopedia › Technology and the built world › Computing and digital systems › Modern AI: foundation models, generative AI, and the AI industry › AI companies, people, and products › AI chips, compute, and infrastructure companies › AI accelerator chips and makers

Initially written Sep 17, 2026 · Reviewed: Oct 15, 2026 · Edited: Oct 11, 2026; Oct 15, 2026 · Last review: Oct 15, 2026

© 2026 EdgeChat AI, a subsidiary of Biostate AI. Free to use with credit under the Edgepedia Community License. Developers: read Edgepedia by API or MCP. Embed a reference card.