Jul 8, 2025

OpenAI had to put temporarily on the feature's usage highlighted a key bottleneck in the mass adoption of AI: the infrastructure isn't ready. Let's call it “Altman's Law”: The cost of running the AI infrastructure is proportional to the square of the number of connected users. How do you fight the burden of Altman's Law? To serve the rapidly growing number of users employing the ever-evolving AI models, the AI infrastructure must be constantly upgraded, not only with new GPU versions but also with new tools increasing the efficiency and resilience of the chips at its core. Several startups and established companies are “hiding in Nvidia's [nearly $4 trillion] shadow,” to use the Wall Street Journal' s apt phrase. They pursue innovative approaches to optimize the performance, power, and reliability of AI chips, making sure they stay cool, efficient, fast, and responsive. “They're not really melting, but with so many chips running together, they're getting very hot, possibly corrupting the AI training models,” says Evelyn Landman, co-founder and CTO at proteanTecs. Monitoring the chip's performance in real-time, the Israeli startup helps some of the world's largest data centers reduce the power consumption of AI servers by up to 14%. ProteanTecs addresses a significant global challenge today and in the future: the demand for electricity by data centers worldwide is expected to more than double by 2030, according to the International Energy Agency. By 2030, data centers are projected to consume as much electricity as the whole of Japan does today. Nvidia, providing most of the electricity-devouring AI chips installed worldwide, is rapidly improving the efficiency of its hardware. The 2024 Nvidia Blackwell GPU uses 105,000 times less energy to generate tokens than its 2014 Kepler predecessor, notes Mary Meeker in her recent comprehensive presentation on the state of AI. Meeker also states that these improvements are not doing enough “to offset the strain of increasing AI and internet usage on our grid.” AI follows the pattern we have observed with previous technology breakthroughs: “costs fall, performance rises, and usage grows, all in tandem.” Meeker provides data on the rapid growth of AI usage by consumers (5.5 billion connected to the internet) and businesses (that don't want to be left behind by competitors) and new applications (e.g., medical devices), summarizing the demand side as “unprecedented.” MORE FOR YOU Amazon Prime Day 2025: Our Editors Found The 58 Best Deals This Morning Today's NYT Mini Crossword Clues And Answers For Tuesday, July 8th WWE Raw Results, Winners And Grades On July 7, 2025 AI—the creation, collection, and analysis of data-is also enlisted in ensuring a smooth functioning of the AI infrastructure. “We make the measurements that measure something never calculated before,” says proteanTecs' Landman. ProteanTecs on-chip monitoring agents detect timing issues, operational and environmental effects, aging, and application stress. After getting a B.Sc. in Electrical Engineering from the Israel Institute of Technology, the Technion, Landman worked for eleven years at Intel before co-founding Mellanox in 1999 and proteanTecs in 2017. She has been a core member of a growing network of engineers cum entrepreneurs that have established what Amir Mizroch, a veteran observer of the Israeli tech scene, called recently the “ West's semiconductor bulwark .” While proteanTecs monitors chips after they are installed in the data center, Mizroch notes Israel's “outsized presence” in the development of tools that ensure semiconductor manufacturing precision. This segment employs just 16% of Israel's chip workforce but handles an estimated third of global chip inspection processes. As “Western tech giants are urgently reconfiguring their supply chains,” writes Mizroch, “Israel offers a perfect combination: world-class engineering talent, geopolitical alignment with Washington, and geographic distance from China's sphere of influence.” Several startups in the Israeli semiconductor ecosystem, with45,000 people across 200 companies, are emerging as direct or indirect competitors to Nvidia. These include Element Labs, which raised $50 million in April, and Speedata, which announced a $44 million Series B funding round in March. Still in stealth mode, but generating a lot of buzz, reports Mizroch, is Majestic Labs. Founded by Ofer Shacham, a semiconductor engineer who previously worked at Google and Meta (and interned at Nvidia while pursuing a Stanford PhD), Majestic Labs secured initial funding from Pipeline Capital Partners and a Series A round from Grove Ventures, with Grove's Dov Moran (inventor of the USB memory stick) joining the board of directors. The business significance of providing an alternative to Nvidia was already realized by Amazon in 2015, when it acquired Annapurna Labs for $370 million. The company's entire AI strategy is built on the chips designed by the Israeli startup. “If and when they go back and tell the story of AWS, our acquisition of Annapurna was one of the most important moments,” Andy Jassy, Amazon CEO, told the Wall Street Journal Other Israeli startups are busy providing hardware and software to enhance the work of Nvidia's AI chips. Retym emerged from stealth in March with over $180 million raised across multiple rounds to enable faster and more efficient transmission within and between AI data centers; Weka.io, “the world's only storage system purpose-built to meet the demands of accelerated compute workloads,” which raised $140 million in Series E funding round in May 2024; and Vast Data, which Nvidia's Jensen Huang devoted a significant portion of his keynote at the recent Computex conference to explaining how its flash-based technology complements Nvidia's AI chips. Nvidia not only promotes complementary solutions but also buys them. Most recently, it acquired Canadian startup CentML, a developer of software for inference optimization, which reduces the cost of deploying AI models. In Israel, it acquired Mellanox in 2019, and in 2024, AI startups Deci and Run:AI, for a combined $1 billion. Nvidia must be preparing to tap even more into the Israeli talent pool, as it was recently revealed that the company is searching for land in Israel's north to build a massive campus , with an investment expected to reach billions of dollars for the land and construction.