What's new in artificial intelligence in Windows 11?
Windows 11 introduces its third feature update, which will be unlike any other release, as it introduces a range of new AI innovations that we have not seen before.
As part of the 24H2 release, Windows 11 introduces the new Copilot Runtime, which makes it possible to run many AI features.
These features include Windows Recall, Live Captions, Auto Super Resolution, Voice Clarity, Studio Effects, and other AI features for some inbox apps.
In total, Microsoft is adding about nine AI-powered features, most of which work locally and offline.
Copilot Runtime Feature
This feature is essentially the heart of AI on Windows 11 and integrates over 40 different AI models capable of running natively on the device.
While there are some AI features in Windows 11 that rely on the cloud, such as Copilot, the Copilot Runtime enables many features to run locally without sending any data to the cloud and without an internet connection.
Windows Recall feature
Recall is perhaps the most anticipated and controversial feature coming to Windows 11.
This feature constantly takes screenshots of whatever you do.
The feature then analyzes the content and context and stores everything on the device within the new Windows Semantic Index.
And whenever you need to find something, you can turn on Recall to search in natural language or use the timeline slider to scroll back in time to the screenshot.
You can also use the available controls to share, delete, or open the snapshot using the Snipping Tool.
It's important to note that Recall won't track websites when you use InPrivate or Incognito, and you can use filters to exclude certain websites that you don't want the feature to track.
Although the feature can be useful in many cases, it remains controversial because it fails to gain the trust of users.
There are a number of questionable security and privacy concerns, which Microsoft has moved quickly to address.
Windows Studio Effects
Windows Studio Effects includes a set of features and special effects on your webcam or microphone.
This feature isn't entirely new, but now Microsoft will start making it widely available to compatible devices.
Effects include a blurred background feature, while Eye Contact links your eyes to make it look like you're looking into the camera.
Auto frames zoom in and out to keep the person focused on the screen.
Portrait Lighting improves lighting conditions to improve visibility in low-light environments.
Creative filters add fun animations and glows, and Voice Focus filters out background noise during video calls.
Live translation
Live translation is another feature that has been around for some time on Windows 11 with the release of Update 2024, but it is now powered by artificial intelligence and includes live translations that aim to help anyone understand the audio by automatically transcribing and translating from any audio source.
The feature will also translate a YouTube video or a voice call.
Sound clarity
Microsoft is also adding Voice Clarity, a new feature that uses AI to analyze and remove background noise in real time during conversations or recordings.
The feature works automatically, so there is no setup to manage.
It is worth noting that this feature was originally reserved for Surface devices, but it will be available as part of the 2024 update without a neural processing unit.
Auto High Resolution
It is a feature that will use on-device artificial intelligence to upscale games and improve frame rates and image quality.
The feature works by automatically upgrading games without the need for developers to update their games.
As a result, you will not only have a better gaming experience, but you will also enjoy energy efficiency.
If you launch a game that is compatible with Auto SR, the system will display a notification letting you know that support is available, with the option to turn it on from the Settings app.
Image redesign feature
The Photos app now includes a new feature called Photo Retouch that uses on-device AI to help you change the style of any photo using different photography techniques, similar to the filters you're used to on phones.
You can also use the text prompt to describe and guide images on how to change the background and other parts of the scene.
Cocreator feature
The Paint app adds Cocreator, a new feature that scans your drawing and helps you create your artwork.
You can use this feature with a text prompt and use the slider option to adjust the intensity of the creativity.
Copilot App
In the Windows 11 2024 update, Microsoft is changing the Copilot AI integration with a new design similar to the ChatGPT web interface with a left side panel for prompt history and other options, instead of the side panel that slides out from the right side.
As a result of these changes, you can no longer use Copilot to change settings, and the Copilot button is no longer part of the taskbar.
Microsoft has also removed the "Copilot" setting from the Settings app.
Microsoft also plans to update Copilot's intelligence with OpenAI GPT-4o, which can process text, images, audio, and video.