OpenAI & Broadcom’s Custom Chip Set to Rival GPUs!
Published On: October 30, 2024
OpenAI is collaborating with Broadcom to develop a custom chip tailored to power AI inference processes, marking a significant shift in how AI-driven systems operate. This new chip is expected to handle the "inference" phase, where AI models respond to user queries, distinct from the "training" phase, which typically requires GPUs like Nvidia's for building model frameworks. This partnership allows OpenAI to accelerate custom chip development, aiming for a release by 2026.
Inference chips have seen rising demand as AI technology permeates various industries, requiring efficient and large-scale data processing. Current GPUs are geared toward training AI models, a process that consumes considerable resources but doesn’t align with inference’s operational efficiency requirements. Broadcom’s specialization in application-specific integrated circuits (ASICs) provides OpenAI with a promising partner. ASICs are designed for specific tasks, enabling energy-efficient and high-speed data processing that’s crucial for AI inference.
For Broadcom, this partnership is a notable extension of their client base, already featuring industry giants like Google and Meta. Broadcom's Chief Executive Officer Hock Tan recently confirmed the company's selective approach to onboarding new clients, focusing on those with significant volume demand. OpenAI’s significant computing needs make them an ideal fit, particularly as their model-based services, like ChatGPT, expand and require robust, efficient computing infrastructure.
To facilitate the project, OpenAI has assembled a specialized team of around 20 engineers, many of whom previously worked on Google’s Tensor Processing Units (TPUs). These experts are likely designing a chip that balances processing power and energy efficiency, tailored for inference demands. This chip will be manufactured by TSMC, whose capabilities align well with high-tech, custom-designed ASIC production. Additionally, OpenAI’s shift to custom chip designs also allows them to mitigate dependency on Nvidia, whose GPUs currently dominate AI processing.
OpenAI’s new chip could also address some broader industry challenges. Data centers, where AI inference tasks are conducted, face increasing power consumption and heat generation, which can be mitigated with efficient, custom chip designs. The custom chip may also pave the way for more modular and scalable AI systems, as OpenAI pushes forward in building large-scale data infrastructure. This ambition aligns with OpenAI’s broader vision to enhance infrastructure, particularly through initiatives that seek support from the U.S. government and international investors to expand AI-optimized data centers.