OpenAI has grown dissatisfied with some of Nvidia’s latest artificial intelligence chips and has been quietly seeking alternatives since last year, according to eight sources familiar with the matter, a shift first reported by Reuters that could complicate the relationship between two of the most prominent companies driving the global AI surge.
The move reflects OpenAI’s increasing focus on inference chips, the hardware used when AI systems like ChatGPT generate responses to user queries. While Nvidia continues to dominate chips used for training large AI models, inference has emerged as a new and rapidly intensifying battleground, with speed and efficiency becoming critical competitive factors.
According to Reuters, OpenAI’s reassessment of its chip strategy represents a significant test of Nvidia’s dominance at a moment when the two companies have also been in talks over a major investment. In September, Nvidia said it intended to invest up to $100 billion in OpenAI in a deal that would give the chipmaker a stake in the startup and provide OpenAI with funding to purchase advanced hardware. The transaction was expected to close within weeks, Reuters previously reported, but negotiations have instead stretched on for months.
During that period, OpenAI has struck deals with AMD and other suppliers for graphics processing units designed to rival Nvidia’s offerings. Sources told Reuters that OpenAI’s evolving product roadmap has also changed the type of computing resources it needs, further slowing talks with Nvidia.
Nvidia has publicly played down any friction. Chief Executive Jensen Huang dismissed reports of tension as “nonsense” over the weekend and reiterated plans for a large investment in OpenAI. Nvidia said customers continue to choose its products for inference due to performance and total cost advantages at scale. OpenAI, in a separate statement, said Nvidia still powers the vast majority of its inference fleet and offers the best performance per dollar.
Behind the scenes, however, seven sources told Reuters that OpenAI is unhappy with the speed at which Nvidia’s hardware can generate answers for certain use cases, particularly software development and AI systems interacting with other software. OpenAI is seeking new hardware that could eventually handle about 10% of its inference computing needs, one source said.
Reuters reported that OpenAI has discussed working with startups such as Cerebras and Groq, which design chips optimized for faster inference. Those talks were disrupted after Nvidia struck a roughly $20 billion licensing deal with Groq, according to one source. Nvidia said Groq’s intellectual property complements its own roadmap, while industry executives described the move as an effort to shore up Nvidia’s position in a rapidly shifting AI landscape.
The technical dispute centers on memory architecture. Nvidia’s GPUs excel at the massive calculations required to train AI models, but inference increasingly demands chips with large amounts of memory embedded directly on the silicon, known as SRAM. This design can speed up response times by reducing delays caused by fetching data from external memory, a limitation of most Nvidia and AMD GPUs.
Inside OpenAI, the issue became especially apparent in Codex, its code-generation product, which the company has been aggressively promoting. Sources told Reuters that some staff attributed Codex’s performance weaknesses to Nvidia’s GPU-based hardware. In a January 30 call with reporters, OpenAI CEO Sam Altman emphasized that customers place a premium on speed for coding applications, adding that OpenAI’s recent deal with Cerebras would help meet that demand.
Competitors such as Anthropic and Google have gained advantages by relying more heavily on custom chips, including Google’s in-house tensor processing units, which are tailored for inference workloads and can outperform general-purpose GPUs in certain tasks.
As OpenAI signaled its reservations, Reuters reported that Nvidia approached several SRAM-focused chipmakers, including Cerebras and Groq, about potential acquisitions. Cerebras declined and instead announced a commercial partnership with OpenAI last month. Groq, which had held talks with OpenAI and attracted investor interest at a valuation of about $14 billion, later agreed to license its technology to Nvidia in a non-exclusive, all-cash deal.
Although the licensing agreement allows other companies to use Groq’s technology, sources told Reuters that Groq is now shifting its focus toward cloud-based software after Nvidia hired away some of its chip designers. The episode underscores how competition over inference hardware is reshaping alliances and power dynamics at the heart of the AI industry.
