Microsoft has introduced a new feature called Copilot Vision. This tool allows the AI to ‘see’ what users are viewing on their screens, offering insights and assistance as they browse the internet.
While it may remind some of Windows Recall, Copilot Vision is more akin to Google’s Gemini Live sharing feature, which has recently become free for all Android users. Additionally, Microsoft has announced that Copilot Vision is now available at no cost for Edge users, although it is currently limited to those in the United States.
Originally, Copilot Vision was only accessible to subscribers of the Copilot Pro plan, but it has since been extended to a wider audience. The tool aims to facilitate hands-free interaction, enabling users to engage with Copilot Vision in a more intuitive way. For instance, users can visit a web page and ask questions while the AI processes the information shown on the screen to provide answers. In addition to verbal queries, users have the option to type their questions as needed.
Interested users can try the new feature on the official website. At this stage, Copilot Vision has limited capabilities, with compatibility currently restricted to nine specific sites: Wikipedia, Amazon, Food & Wine, Tripadvisor, Target, OpenTable, Williams Sonoma, Wayfair, and Geoguessr. Microsoft plans to increase the range of compatible sites in the future, which should enhance the feature’s utility.
Despite its potential benefits, the launch of Copilot Vision has led to discussions regarding privacy and data security. There are worries about AI technologies that analyze on-screen content. Microsoft is proceeding cautiously with the rollout of this feature, particularly following recent feedback concerning Windows Recall, which had a function that took screenshots every five seconds. In light of the criticism, Microsoft paused this feature but has since resumed its implementation after making significant security enhancements.
Microsoft has stated that it only keeps records of the responses produced by Copilot for the user and does not collect any input, images, or content from web pages during a Copilot Vision session.
In contrast, Google claims that users value the Gemini Live video and screen sharing features, suggesting that Microsoft is not only looking to compete but aims to excel in the AI realm.
Image Source: Ken Wolter / Shutterstock
