Microsoft Build, started in 2011, is Microsoft’s annual developer conference in the US, which hosts keynote speeches by company executives and various educational sessions tailored for software engineers and developers. The event, which transitioned to the Seattle Convention Center in 2023 after being digital-only from 2020 to 2022, draws most of its attention during its keynote speeches, which typically feature significant product announcements.

Microsoft Build 2024 took place last week, starting on Tuesday, May 21, and spanned three days. Let’s see what were the main announcements from this conference.

Microsoft CEO Satya Nadella opened Build 2024 by discussing the new era of artificial intelligence. He says Microsoft has had two dreams for decades:

1) Can computers understand us, instead of us having to understand computers?

2) In a world where we have more and more information, can computers help us reason, plan and act more effectively based on all that information?

Nadella positions this wave of AI as the answer to Microsoft’s dreams. Here are the main highlights from Satya Nadella’s keynote:

 

Copilot+ PCs – the fastest, most intelligent Windows PCs ever built.

Copilot+ PC is designed to bring AI features to the computer, offering new ways to interact with the operating system and software, but not much new information has been released beyond the June 18 release date.

Copilot: Major upgrade with GPT-4o and new language models

Copilot has received a GPT-4o upgrade, bringing numerous updates including Teams integration, development of extensions for GitHub, and new Phi-3 language models such as Phi-3-vision and variants for local or cloud use.

Surface Pro 11 & Surface Laptop 7: Surprising news with OLED

Instead of the expected Surface Pro 10 and Surface Laptop 6 for consumers, Microsoft unveiled the Surface Laptop 7 and the new Surface Pro 11 with OLED display, but details about these devices were limited during the main presentation.

Build Smarter AI Assistants using Copilot Studio

Microsoft has added new features to Copilot Studio, making it easier to create advanced AI copilots. These copilots can automate various business tasks and processes. By integrating with Microsoft Graph, Power Platform, AI capabilities in Microsoft Fabric, and Dataverse, developers can now build copilots that interact smoothly with both company data and external sources.

Improvements in Azure AI Search

Azure AI Search received updates to enhance enterprise-level search capabilities. The key improvements include:

• Scalable Retrieval-Augmented Generation (RAG)
• Better search relevance with vector and hybrid search enhancements
• New data integrations and vectorization improvements
• OneLake connector for easy archival integration

Introducing GPT-4o on Azure OpenAI

Microsoft announced the general availability of GPT-4o, a powerful model that combines text, vision, and audio capabilities. Available in Azure AI Studio and API, GPT-4o offers organizations a high level of customization for AI outputs. Additionally, the new Assistants API supports creating advanced virtual assistants.

Microsoft Fabric introduces Real-Time Intelligence

Microsoft Fabric now includes Real-Time Intelligence, a solution for managing large volumes of time-sensitive data. Features include:

• A Real-Time Hub for event management
• Stream connectors for data from multiple cloud sources
• Improved dashboards for data exploration
• AI-powered insights with integrated Microsoft Copilot for query generation and anomaly detection

Customizable Extensions for GitHub Copilot:

GitHub Copilot has new extensions that allow it to be customized with services like Azure, Docker, and Sentry. These enhancements make workflows more efficient by enabling natural language interactions that can be deployed in the cloud.

New AI-Powered PCs

The latest AI-equipped PCs come with powerful NPUs, offering 45 TOPS of computing power. These devices are designed to help developers create AI applications with low latency, cost efficiency, and improved data privacy.

Introducing Phi-3 Vision Model: AI that understands visual data

Microsoft announced the early version of Phi-3 Vision, a model for multimodal visual reasoning. This model can process images and text to provide textual responses, making it easier to understand visual data such as charts and graphs.

Power Automate gets smarter with Generative AI

Power Automate now includes generative AI features for process automation. Key updates are:

• AI-driven, outcome-based automation flows
• Enhanced desktop flows with a new recorder and natural language-to-flow conversion capabilities

New Azure Virtual Machines

Azure introduced new virtual machines: ND MI300X v5, powered by AMD’s Instinct GPU for high performance, and Cobalt 100 Arm VMs, optimized for AI and cloud-native workloads.

Expanded Azure Infrastructure

Microsoft has expanded its AI infrastructure with more than 60 global data centers and introduced Azure Compute Fleet for easier provisioning across virtual machines and availability zones.

Enhanced Security in Microsoft Edge

Microsoft Edge now has advanced security features, including blocking screenshots on sensitive pages and streamlined updates for IT administrators.

Responsible AI Tools

New tools like Prompt Shields and Groundedness Detection ensure the safe use of AI by protecting against prompt injection attacks and hallucinations in model outputs.

Collaboration with OpenAI

Sam Altman, CEO of OpenAI, joined Microsoft CTO Kevin Scott to discuss future plans and developments for OpenAI, including new supercomputing initiatives. Sam Altman also drops little hints about GPT-5, saying: GPT-5 is coming, and it’s going to be amazing.

 

📹 You can see the full recording of the first day below.

Day 2 of Microsoft Build 2024 continued with exciting announcements, focusing on AI innovations and enhanced capabilities across various platforms. Here are the main highlights:

 

AI Innovations and Multimodal Capabilities

Microsoft is advancing its AI efforts with new mini-multimodal models that can process multiple types of data simultaneously. This development aims to enhance AI capabilities across different applications, making AI agents more versatile and effective.

Real-Time Youtube Translations in Microsoft Edge

Microsoft Edge now includes a real-time translation feature for YouTube videos, making online content more accessible and user-friendly for a global audience.

Microsoft Teams: Custom Emojis for Expressive Chats

Microsoft Teams has introduced custom emojis, allowing users to add personalized touches to their messages. This feature aims to enhance communication by making interactions more expressive and engaging.

Hardware and Software Development:

• Tiny Snapdragon PC:
  Microsoft unveiled a new compact PC powered by Snapdragon, highlighting progress in the development of portable computing devices.
• Microsoft File Explorer:
  File Explorer now functions as a Git repository, simplifying version control and collaboration for developers.
• AI-Powered Clipboard in PowerToys:
  Windows has introduced AI-powered clipboard features in PowerToys, offering smarter clipboard management to improve user efficiency.
• Gaming on Arm-Based Devices:
  Microsoft and Qualcomm launched WorksOnWoA.com, a site that tracks the performance of games on Arm-based devices like the Surface Laptop. The site categorizes games as “Perfect,” “Playable,” “Runs,” or “Unplayable” to guide gamers on compatibility.

AI-Powered Call Management

Truecaller, in collaboration with Microsoft’s Azure AI Speech, will enable users to create AI-generated versions of their voices for answering calls. This feature aims to improve spam-blocking capabilities by providing customized call responses.

📹 You can see the full recording of the second day below.

Increase developer speed with SQL

JSON becomes a primary data type, and the introduction of Data API Builder automates the creation of a REST API or Graph API for data access, eliminating the hard work of creating a data access layer.

Enhancing solutions with new Azure AI products and features

AI content safety is essential for analyzing AI inputs and outputs to ensure the right level of safety, including detecting violence, suicide, sexual or hate content.
PromptFlow, a visual designer, allows integration of plug-ins and Azure Open AI.

Scott and Mark are learning to code

The presentation demonstrates how generative AI works by watching two programmers use GitHub Copilot to create a multiplayer version of Asteroids. They explain how temperature affects AI output and show concrete examples.

 

📹 You can see the full recording of the third day below.