Gemini Leads LLM Leaderboard, Character.AI Founders Return to Google, Groq Series D and More (8.8.24)

Google Gemini 1.5 Pro is making waves as it outperforms GPT-4o and Claude-3.5 in the LMSYS Chatbot Arena. Many predicted it was only a matter of time before Google caught up, and it's the first time we've seen them take the lead, somewhat reversing sentiments that Google was getting 'left behind'. We also see hints of a broader strategy towards higher personalization with Google's partnership with Character.AI, known for its expertise in creating advanced conversational models and personalized AI interactions. In other news, Groq has announced new funding, and Argentina is pursuing AI applications reminiscent of sci-fi, drawing comparisons to the 2002 movie Minority Report.

Google Gemini 1.5 Pro tops Chatbot Arena

Google's Gemini 1.5 Pro (Experimental Version 0801) has claimed the top spot in Chatbot Arena, surpassing GPT-4o and Claude-3.5 with a score of 1300. The score of 1300 for Google's Gemini 1.5 Pro in Chatbot Arena reflects its ELO rating, a ranking model used in Chess, and indicates a win percentage of 54% against GPT-4o and 59% against Claude-3.5 Sonnet, showcasing its superior performance in head-to-head comparisons. It has shown that it excels in multilingual tasks and technical areas like Math, Instruction-Following, and Coding, though it trails in domains like Coding and Hard Prompts compared to Claude 3.5 Sonnet and GPT-4o. Google Cloud says this experimental version is now available for early testing and feedback in Google AI Studio and the Gemini API. [For the first time, Google Gemini has claimed the #1 spot, surpassing GPT-4o/Claude-3.5 with an impressive score of 1300 (!), and also achieving #1 on our Vision Leaderboard.]

Character.AI Gets Funding from Google and Co-Founders Return to Google

Noam Shazeer and Daniel De Freitas, co-founders of Character.AI, are returning to Google, with Shazeer joining the DeepMind research team, while Character.AI’s general counsel Dominic Perella will serve as interim CEO. Character.ai is a platform where users can create and interact with AI-driven virtual characters that adapt to individual inputs, tailoring their responses and behaviors for personalized, engaging interactions across various applications. According to a TechCrunch interview, Google has signed a non-exclusive agreement to use Character.AI’s technology, which Shazeer says will provide funding for Character.AI’s continued growth and focus on building personalized AI products. [Title: Exclusive: Character.AI CEO Noam Shazeer returns to Google as the tech giant invests in the AI company]

Groq raises $640 million, valuation hits $2.8 billion

Groq has raised $640 million in Series D funding, led by Cisco Investments, Samsung Catalyst Fund, and BlackRock Private Equity Partners, bringing its valuation to $2.8 billion. Groq's chips are designed with a unique architecture called Tensor Streaming Processor (TSP). This architecture processes data in a highly parallel manner, allowing it to handle multiple tasks simultaneously with the benefit of both speed and efficiency. Unlike traditional GPUs that are designed for a wide range of tasks, Groq's TSP is specifically optimized for AI workloads, making it exceptionally fast and energy-efficient for inference and tasks like running large language models. The company says its chips, which now run Meta Platforms' LLaMA, are four times faster, five times cheaper, and three times more energy-efficient than Nvidia's GPUs for AI inference tasks. [AI chip startup Groq valued at $2.8 bln after latest funding round]

Argentina's President Launches AI Security Unit

Argentina's President Javier Milei has launched the AI Applied to Security Unit, raising concerns about potential human rights violations. The Ministry of Security says the unit will use AI to predict crimes and monitor social media, while human rights groups fear it could lead to over-surveillance and profiling. [Argentina will use AI to ‘predict future crimes’ but experts worry for citizens’ rights]