Why Humans Struggle to Estimate AI Progress, Antitrust Woes, OpenAI Hiring Robotics, and, Datasets (6.6.24)

This week's edition focuses on the training and progression of AI towards AGI. We also see activity by regulators for antitrust investigations, some new roles in Robotics advertised by the OpenAI team, and a new dataset for model training with detailed commentary. Short, sweet, and enlightening. Enjoy!

-- Sasha Krecinic

Why do we struggle to estimate AI Advancements?

Humans inherently struggle with understanding exponential relationships and growth. The wheat and chessboard problem illustrates this. The story goes that a king agrees to place one grain of rice/wheat on the first square of a chessboard, doubling it on each subsequent square. By the 64th square, the amount of wheat is huge, enough to bankrupt the kingdom. Similarly, during COVID-19, many couldn't grasp how quickly the virus could spread, leading to delayed responses and widespread impacts. We all had that friend who warned, “It’s coming, start preparing,” but most didn’t listen. 

It may not be obvious, but AI development is experiencing similar exponential growth and the information asymmetry is also increasing. To understand how it will evolve, you need to look at the sub-components, roadmaps, and rate-limiting steps in the hardware, software, data, people, and capital. There are those in the know who quietly acknowledge this exponential growth, aware of its potential, often keeping their ‘extreme views’ to themselves, once again, analogous to early COVID-19. 

Leopold Aschenbrenner's extensive analysis, "Situational Awareness," highlights the accelerating pace of AI capabilities across the various layers that drive innovation. From massive compute clusters to evolving AI models, the trajectory toward Artificial General Intelligence (AGI) is still a function of the sum of its parts. Leopold posits that AI advancements will continue to be exponential, driven by computing power, algorithmic efficiency, and new methodologies. Despite potential bottlenecks like data scarcity and unknowing challenges, the path to AGI is becoming clearer. Improvements in hardware, software, data, skills, and capital are compounding and driving transformative impacts that are often hiding in plain sight. 

While mainstream views often downplay AI ‘end game’ scenarios, a small group of experts is busily preparing for and predicting an ‘imminent’ (read this as within the next 3-5 years) AGI breakthrough. As an investor in this space, you see a range of people's reactions to this; unfortunately, some are still a little too skeptical, in my opinion. Naivety on this scale hasn’t served societies well historically. If we account for the consistent revision of AGI forecasts, it also points to a trend that AGI may be much closer than society thinks. Check out the full text if you'd like to see the detailed breakdown, I think it's worth a read! [Situational awareness] Share this story by email

Regulators Target AI Giants for Antitrust Probes

Federal regulators have agreed to initiate antitrust investigations into Microsoft, OpenAI, and Nvidia to examine their dominant positions in the AI industry, according to the New York Times, referencing two individuals familiar with the confidential discussions. The Justice Department will investigate Nvidia, while the FTC will focus on OpenAI and Microsoft, reflecting a broader initiative to address potential monopolistic practices in the AI sector. [U.S. Clears Way for Antitrust Inquiries of Nvidia, Microsoft and OpenAI] Share this story by email

OpenAI Recruits for Robotics

OpenAI is seeking a Research Engineer for its Robotics team in San Francisco to focus on training and fine-tuning large multimodal LLMs, as detailed in their job posting. The role involves collaborating with industry partners to enhance robotics applications and signals the first time they have made these sorts of moves since 2020. [OpenAI recruits for robotics talent] Share this story by email

FineWeb-Edu Sets New Dataset Standard

Guilherme Penedo from Hugging Face shared a report on the release of FineWeb and its educational subset, FineWeb-Edu, which incorporates 1.3 trillion tokens from a high-quality filtered Common Crawl dataset. According to the report, FineWeb-Edu outperforms all other publicly available web-scale datasets on benchmarks such as MMLU, ARC, and OpenBookQA, prompting a reassessment of the perceived quality of internet data. The post also received a lot of praise from industry leaders like Andrej Karpathy and Thomas Wolf, who noted it as potentially the best 45 minutes of reading you could do in the space if you want to understand how high-performing models work! Check out the full text if you're in this space! [via @Thom_Wolf] Share this story by email

—