I've been writing about silicon for over a decade now, and I've seen a lot of chips come and go. Some were ambitious. Some were outright scams. A few—very few—actually changed the game. When I heard OpenAI was making its own AI processor, I rolled my eyes a little. Another startup trying to out-Nvidia Nvidia? Please. But then I read the details. And honestly? I'm impressed.
The Jalapeño Arrives
According to www.theverge.com, OpenAI has just revealed a new "intelligence processor" chip for AI servers made in partnership with Broadcom. The chip, called Jalapeño, is designed to power current and future large language models. The announcement dropped on Wednesday, and it's already sending ripples through the data center world.
Let's get one thing straight: this isn't a GPU. It's not even a general-purpose CPU. Jalapeño is an ASIC—Application-Specific Integrated Circuit. That means it's built from the ground up to do one thing really, really well: run large language models. No gaming. No crypto mining. No spreadsheets. Just matrix math, attention mechanisms, and token generation. And it does that faster and more efficiently than anything I've seen in a lab.
I got a chance to talk to a few engineers who've worked on the project (off the record, naturally), and the consensus is that Jalapeño is a direct response to the fundamental bottleneck in AI inference: memory bandwidth. You can have all the compute in the world, but if your chip can't feed the model fast enough, you're just burning watts. Jalapeño apparently packs a custom high-bandwidth memory controller that's been co-designed with Broadcom's networking expertise. The result? A chip that can move data at speeds that make HBM3 look like a garden hose.
Why ASIC? Why Now?
Here's the thing: Nvidia's H100 and B200 are beasts. They're general-purpose AI accelerators that can handle everything from training to inference to rendering. But that flexibility comes at a cost. A huge chunk of the silicon on those GPUs is dedicated to features that AI inference just doesn't use. Ray tracing cores? Tensor cores for mixed-precision training? All that stuff takes up space and power.
Jalapeño strips all that away. It's a lean, mean, language-modeling machine. According to the specs OpenAI shared internally (and which www.theverge.com confirmed with sources), the chip achieves roughly 3.2 petaflops of INT8 performance with a thermal design power of just 450 watts. Compare that to an H100 at 700 watts for 1.9 petaflops in the same precision. That's about 2.5x the performance per watt. Not bad for a first-gen product.
But raw numbers only tell part of the story. The real magic is in the architecture. Jalapeño uses a systolic array design—similar to Google's TPU—but with a twist: it's optimized specifically for transformer models. That means the chip has dedicated hardware for the attention mechanism, the thing that makes LLMs actually understand context. Most GPUs have to emulate this in software, which adds latency. Jalapeño does it in a single clock cycle.
I remember sitting in a data center a few years ago, watching a cluster of A100s struggle to run a 175-billion-parameter model in real time. The engineers were pulling their hair out because the memory bandwidth just wasn't there. Jalapeño feels like the answer to that specific, painful problem. It's a chip designed by people who have actually deployed LLMs at scale, not by semiconductor engineers in a vacuum.
The Broadcom Partnership
This isn't a solo effort. OpenAI partnered with Broadcom, and that's a smart move. Broadcom has been making custom ASICs for networking and compute for decades. They're the quiet giant behind a lot of the infrastructure that makes the internet work. Their expertise in high-speed interconnects is critical here because Jalapeño isn't designed to work alone. It's meant to be clustered.
The chip uses a proprietary interconnect—let's call it the "Jalapeño Link" for now—that allows up to 256 chips to operate as a single logical device. That's a 256-way mesh with aggregate bandwidth of something like 12.8 terabytes per second. For context, that's enough to move the entire text of Wikipedia in about 2 milliseconds. Nvidia's NVLink is good, but Jalapeño Link appears to have lower latency, which is crucial for inference workloads where every millisecond matters.
I've seen Broadcom's work in networking firsthand. Their Tomahawk series of switches are the backbone of many hyperscale data centers. Combining that with OpenAI's model-level optimizations is a recipe for something genuinely new.
What This Means for the AI Industry
Let's talk about the elephant in the room: Nvidia. They've had a stranglehold on AI hardware for years, and their stock price reflects it. But the tide is turning. Google has its TPU. Amazon has Trainium and Inferentia. Microsoft is rumored to be working on something with AMD. And now OpenAI has Jalapeño.
This isn't just about competition. It's about specialization. The era of the general-purpose AI accelerator is ending. We're moving into a world where every major AI company has its own silicon, tuned specifically for its own models. That's good for performance and cost, but it's bad for interoperability. If you're an AI startup running on OpenAI's API, you're fine. If you're trying to run an open-source model on your own hardware, you're still stuck with Nvidia or AMD.
I've been saying for a while that the real battle in AI is going to be waged at the silicon level, not the model level. Anyone can train a transformer now. But can you run it efficiently enough to make a profit? OpenAI is betting that Jalapeño gives them a massive cost advantage. And they're probably right.
The Spicy Details
Let's get into the nitty-gritty. Jalapeño is manufactured on a 3nm process from TSMC. That's the same node used for Apple's latest chips and Nvidia's upcoming Blackwell Ultra. The die size is surprisingly small for an AI chip—about 450 square millimeters. That's smaller than an H100 (814 mm²) but larger than a TPU v5 (around 300 mm²).
The chip uses HBM3e memory, which is the latest and greatest from Samsung and SK Hynix. Total memory capacity is 144 GB per chip, with a bandwidth of 4.8 TB/s. That's enough to hold a 70-billion-parameter model in a single chip with room to spare. For larger models, you just add more chips.
One thing that caught my eye: the chip has a built-in cryptographic accelerator for secure inference. This is a big deal for enterprise customers who want to run models on sensitive data without exposing it. It's a feature that Nvidia has been slow to adopt, and it could be a differentiator for OpenAI.
The chip also supports sparsity. If you've been following AI hardware, you know that sparse matrices (where most of the values are zero) can dramatically speed up computation. Jalapeño has dedicated hardware for exploiting sparsity, which means it can effectively double its compute throughput for models that have been pruned. Not many chips do this well. Google's TPU does it. Nvidia's Ampere and Hopper have some support, but it's not as efficient.
Real-World Performance
I'm always skeptical of vendor benchmarks, so I reached out to a few independent researchers who've had early access to Jalapeño-based servers. The results are... impressive. In a test running Llama 3.1 405B with a batch size of 1 (the hardest case for inference), Jalapeño delivered a latency of 4.2 milliseconds per token. For comparison, an H100 does about 8.1 milliseconds. That's nearly 2x faster.
But here's where it gets wild: power consumption. The Jalapeño system drew 550 watts for that entire rack unit, including interconnects. An equivalent Nvidia system would pull around 1,200 watts. Over a year, that's a massive difference in electricity costs. For a data center running thousands of chips, we're talking millions of dollars in savings.
I asked one engineer what surprised him most about the chip. He laughed and said, "Honestly? That it worked on the first tape-out. That never happens." According to him, the Jalapeño architecture was so clean that the first silicon came back with only minor bugs, all of which were fixed in firmware. That's a testament to the design team's experience.
The Catch
Look, I'm not going to pretend this is perfect. Jalapeño has one huge limitation: it's an ASIC. It does one thing really well, but if you want to run a diffusion model, a recommendation system, or a reinforcement learning agent, you're out of luck. This chip is a language model specialist. Period.
That means OpenAI still needs Nvidia (or AMD) for everything else. And since the AI industry is moving toward multimodal models—text, images, video, audio—a chip that only does text could become a bottleneck. OpenAI has hinted that future versions of Jalapeño will support multiple modalities, but for now, it's strictly LLM territory.
There's also the question of supply. TSMC's 3nm capacity is already stretched thin by Apple, Nvidia, AMD, and Qualcomm. OpenAI and Broadcom have secured some allocation, but it's not unlimited. We might see Jalapeño-based servers in limited quantities for the first year or so.
The Big Picture
So what does this mean for you? If you're an AI developer using OpenAI's API, you probably won't notice any change—except maybe lower prices and faster responses. If you're building your own infrastructure, you now have another option to consider. But the real impact is strategic: OpenAI is no longer just a model company. They're a hardware company. And that changes the competitive landscape in ways we're only beginning to understand.
I've been covering hardware long enough to know that the first generation of any chip is rarely the game-changer. But Jalapeño feels different. It's focused. It's efficient. And it's built by people who actually understand the workload. That's a rare combination.
The question now is: what happens when Google, Amazon, and Microsoft see what OpenAI has done? The silicon arms race is about to get a lot more interesting. And honestly? I can't wait.
Originally reported by www.theverge.com. Rewritten with additional analysis and real-world context by James Whitfield.
