Google Lens has introduced a new feature that allows users to search using videos and voice commands. This update enables users to capture a video of what they want to search for and ask questions about it using their voice. The AI Overview and search results will be based on the video’s content and the user’s query. This feature is currently being rolled out in Search Labs on both Android and iOS.
During Google’s I/O event in May, the company gave a preview of this video search feature. For example, if someone is curious about the fish they see at an aquarium, they can use Google Lens to record a video of the exhibit and ask a question like, “Why are they swimming together?” The Gemini AI model in Google Lens will then provide a response based on the video and question asked.
Rajan Patel, Google’s vice president of engineering, explained that the video is captured as a series of image frames and analyzed using computer vision techniques. The information is then passed to a custom Gemini model that understands multiple frames in sequence to provide a response rooted in web data.
While the current feature does not support identifying sounds in a video, Google is exploring this capability for future updates. Additionally, users can now ask questions about a photo using their voice in Google Lens. This new voice search feature is available globally on Android and iOS, with support for English language queries.
Overall, Google Lens continues to enhance its search capabilities by introducing video and voice search options, making it easier for users to search for information using multimedia content.
