A New Era for Personal Computing

Agents, AI Privacy, Latency, OpenAI, Google, Microsoft

Author

Sasha Krecinic
May 22, 2024

Microsoft has launched its AI-Optimized Copilot, along with a new line of PCs, marking a new chapter in personal and business computing. We break down why this is such a big deal and also share other players in the space working on agents and computer vision to improve contextual awareness. In case you missed it, other headlines from earlier in the week were about Google's I/O event and Gemini 1.5 results. Since then, we've seen their search results incorporate some of the new functionality, which has been a step up in the search experience. We also see headlines in the world of 'agents,' and we suspect this will continue to be the buzzword that garners attention for a while!

First impacted: Everyone

Microsoft has launched Copilot+ and a new line of Windows PCs designed specifically for AI applications. These devices feature powerful new silicon, including ARM processors with Neural Processing Units (NPUs) capable of a whopping 40+ TOPS, all-day battery life, and access to advanced AI models. Starting at $999, these PCs are within reach of average consumers and set an interesting stage for a world that can accommodate 'agent'-centric workflows, or what Microsoft is calling their 'Copilot'.

Hardware > Local Models > Privacy > HAL9000?
The new Copilot+ PCs introduce AI-driven experiences like Recall and Cocreator. This innovation, along with advancements in multimodal AI and privacy, is transforming our interactions with computers. Users will be able to share significantly more data with a locally hosted model than they otherwise might feel comfortable with. Instead of the usual "Don't insert any personal data," soon you might find yourself comfortable enough to share your social security or credit card details. The core mechanics of locally hosted models, combined with enhanced privacy and computer vision, could change most human-to-computer interactions.

Enhancements to Microsoft Copilot for Business
Microsoft has also announced significant updates to its Copilot feature, now integrating with Microsoft Teams and Planner to improve team collaboration by managing agendas and tracking action items. Additionally, the company introduced Copilot Studio, which allows users to create custom AI agents that automate business processes and integrate with business data systems. The ramifications for workflow management software are quite interesting, and it is impressive how quickly the landscape is changing.

Why Is This Important?
In the context of highly capable large language models (LLMs) with low latency, computer vision, and hardware optimized to host local LLMs/Agents, advancements are occurring faster than most anticipated (even those of us who were saying "it's coming sooner than you think"). We are quickly approaching a future where your local computer will be able to perform tasks autonomously and assist you proactively. What the local LLM cannot do on the local machine, the bigger, more powerful version in the cloud helps with, potentially guiding the local LLM as required. For instance, imagine needing to change your desktop background. Instead of searching for instructions online, you could simply ask, "Hey, find how to change the desktop background to X," and your computer would handle it for you. Or if you are a software vendor, instead of having customer support, you could provide agent instructions that prompt users for help if they are clicking around for more than X seconds, offering guidance on the relevant workflow. This could potentially change most workflows significantly. [Introducing Copilot+ PCs - The Official Microsoft Blog] Share this story by email

First impacted: Everyone

In a significant shakeup at OpenAI, key team members Ilya Sutskever and Jan Leike resigned, with Leike highlighting that “safety culture and processes have taken a backseat to shiny products” at the company. Their departure follows the disbanding of OpenAI’s Superalignment team, which aimed to address long-term AI risks. Leike emphasized the need for serious preparations for AGI to benefit humanity. Meanwhile, AI leader Yann LeCun criticized the urgency around controlling superintelligent AI, arguing that we need to focus on developing systems smarter than basic animals first. He likened the current urgency to trying to ensure the safety of advanced aircraft before the fundamental technology even exists, emphasizing a gradual, iterative approach to AI development and safety. [via @gdb] Share this story by email

First impacted: mobile developers, smartphone users

Developers at Wafer.System are creating AI at the OS level, enabling AI agents to use the same device interfaces as users, such as virtual keyboards and touchscreens, to enhance efficiency and user experience. They say this integration allows AI agents to access extensive app data and user interactions, potentially tripling efficiency by predicting and automating user actions without third-party app integrations. Check out the video, it's super interesting! [Wafer] Share this story by email

First impacted: mobile developers, smartphone users

Awni Hannun says AmbientGPT, a new MacOS app that integrates GPT-4o for enhanced contextual understanding directly from your screen, is set to launch soon. The app, which operates entirely on-device to ensure data privacy, requires an ARM64 MacBook and a specific OpenAI API key, and is pending Apple certification. [GitHub - siddrrsh/ambientGPT] Share this story by email

First impacted: AI researchers, software developers

A report from Google's Gemini team highlights that the Gemini 1.5 Pro model demonstrates improved performance over the previous 1.0 Ultra, particularly in text and vision benchmarks, achieving a 91.7% score in the MMLU benchmark. The model has enhanced in-context learning capabilities, especially in low-resource language translation and mixed-modal learning, as detailed in the updated 153-page technical report. Check it out if you'd like to get into the details. [goo.gle] Share this story by email

First impacted: AI developers, investors, tech industry developers

Scale AI, led by CEO Alexandr Wang, has raised $1 billion in a financing round with Accel and existing investors, boosting its valuation to $13.8 billion. The company says the funds will be used to further develop its frontier data and enhance its Data Engine, which supports advanced LLMs and computer vision models. [Careers | Scale AI] Share this story by email