Apple is facing our face-masked future. This week, the company started testing some new software for the iPhone that will let device owners unlock the handset while wearing a face covering. There’s a catch though, one that lines up with Apple’s strategy of locking people in to different Apple products and highlights how challenging it can be to develop accurate facial recognition technology: The new face-unlock feature requires an Apple Watch.
The first developer beta of iOS 14.5 includes updates to app tracking controls and Siri alongside the face-mask function. App-makers typically get early access to the newest version of iOS in order to launch or retool their apps well in advance of the formal software release. (Brave souls who don’t mind the risk of potentially bricking their iPhones can also enroll in public beta releases.) The fully-baked version of the software is expected to be made available to the general public this spring.
That means by the time most people install the newest version of iOS on their iPhones, we’ll have been wearing masks for a year or more to help prevent the spread of coronavirus. Compared with all the other ways the pandemic has altered our lives, having to use a method other than Apple’s Face ID to unlock your iPhone isn’t a huge inconvenience. Still, it’s frustrating to hold your phone up to your face only to remember Face ID won’t work on account of your mask. The promise of facial recognition technology—which coexists with very valid concerns about its misuse and its error rates for people with darker skin—is that it’s supposed to get smarter and better over time.
With the coming software update, Apple is more or less handing off the authentication load to the Apple Watch. If you’re using a newer iPhone model (one with Face ID), have installed the iOS 14.5 beta software, and you’re wearing an Apple Watch with watchOS 7.4, raising the locked phone to your face will trigger a bit of communication between the phone and watch. The phone will unlock. The watch will also display a notification that the phone has been unlocked. One iOS developer described it to WIRED as a similar experience to unlocking a Mac with an Apple Watch.
As 9to5 Mac notes, this is the second change Apple has made to its Face ID authentication system to accommodate face masks. Last spring the company released software that made it easier to avoid using Face ID while wearing a mask, by showing the iPhone’s passcode screen after the first time Face ID fails. Still, these updates have their limitations. The new Face ID-with-face-mask feature will only work with phone unlocking. So if you use Face ID for Apple Pay transactions, or to log into third-party apps, you’ll still have to authenticate in some other way.
But the bigger question is why Apple is relying on the Apple Watch to unlock iPhones with face masks, instead of releasing software that simply recognizes the uncovered portion of a person’s face. At the time of publication, Apple hadn’t yet answered WIRED’s queries about this. Experts say there are plenty of ethical and technical considerations when deploying any facial recognition tech, but when it comes to performing facial recognition on faces that are partially covered, it’s particularly challenging.
Anil K. Jain, who researches computer vision, machine learning, and biometrics recognition at Michigan State University, says that despite facial recognition’s progress over the past five to 10 years, it’s still “susceptible to occlusion—that is, what part of the face is not visible. In most cases, the technology assumes the person will remove their glasses and facial coverings, the illumination would be homogenous, and the expression would be neutral, like a passport photo.”