Apple Introduces Audio Intelligence Features in iOS 27.3 and watchOS 27.3
On Tuesday, Apple rolled out a potentially controversial set of Audio Intelligence features into testing with the first developer beta versions of iOS 27.3 and watchOS 27.3. The betas bring four new AI features that allow Apple Watch Series 12 wearers to detect important sounds, identify songs, catch up on conversations, and take notes.
Innovative Features and Their Impact
Apple’s new features aim to enhance user experience by integrating sophisticated AI capabilities into everyday scenarios. While the sound and song detection features are straightforward and non-intrusive, aimed at aiding those with hearing impairments and leveraging Shazam’s song identification technology, the real conversation revolves around the Live Rewind and Siri Recaps features.
Understanding the Siri Recaps Feature
After just a day with Apple’s Siri Recaps feature on my Apple Watch, I can already see how its value may ultimately outweigh any creepiness factor that comes from the AI-powered audio recording and summarization it provides. However, I also see areas where AI-assisted note-taking can go too far, capturing things that aren’t worth remembering, and where there’s room for improvement.
Apple has taken care to ensure the privacy of these AI features by processing the audio and running AI models on-device, when possible, and by not identifying speakers, storing raw audio files, and redacting private information, like government ID numbers or personal account information.
Live Rewind and Its Privacy Concerns
It’s the two other features, Live Rewind and Siri Recaps, that have raised concerns. The features record and then transcribe or summarize the conversations happening with and around the Apple Watch’s owner. Live Rewind lets you go back to see what was said in the past 15 seconds, while Siri Recaps summarize audio throughout your day, saving notes of the important things that were said.
Image Credits:iPhone screenshot by TechCrunch
Practical Applications and User Experience
While some users express concerns about privacy and the notion of omnipresent technology, others, like myself, find the utility of these features compelling. For instance, during a recent visit to a car repair shop, the Siri Recaps feature efficiently captured detailed verbal information about time frames, prices, and policies, which I found extremely helpful for future reference.
Image Credits:iPhone screenshot by TechCrunch
For instance, it noted the deductible, the repair time frame, the amount of gas we had in the rental at the start, and other things I wanted to remember.
Challenges and Areas for Improvement
One complaint is that Siri Recaps had mixed this information with other topics that were later discussed in the home, like an errand we needed to run and what time babysitting was this week, something we were discussing in order to create an appointment on the family calendar with an appropriate alert time frame.
Image Credits:iPhone screenshot by TechCrunch
Future Enhancements and User Control
While I appreciated that the Recap picked up a reminder about an errand I needed to run from one of the family conversations, other tidbits it saved — like what we were watching on TV — would only be valuable if you were exporting a Siri Recap note to Apple’s Journal app, because it was something you wanted to record for personal reasons.
I’d like to see these sorts of extraneous details saved as separate notes — or at least as dedicated sections that could be exported or deleted on their own at the end of the day.
I’d also love to be able to swipe left on an individual bullet point to remove it from the Siri Recap entirely, rather than having to enter edit mode, place the cursor, and then start deleting the text I don’t need.
Another option I’d like to have would be a setting that would allow you to automatically export any saved Siri Recap (the ones you click the bookmark icon to save) to Apple’s Notes app, instead of that being an extra step.
These Recaps could go into their own folder in the Notes app for easy reference. I can understand why this option is not the default, given how many different ways people use a general-purpose app like Notes. However, it would be useful for those of us who want to keep Notes as the central starting point for all our jotted-down reminders and to-dos.
Though I wasn’t immediately warmed up to the idea that my Watch was always listening, so to speak, I see the value of this sort of automated note-taking in daily life.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
For more details, you can read the full article Here.
“`
