Google Pixel 3’s selfie camera now detects kissing and facial expressions including duck face and surprise. Google detailed the machine learning techniques underlying the camera app (Pixel Camera) that ships on its Pixel 3 smartphone and announced a new feature — kiss detection — heading to the Photobooth mode in the latest release.
A new shutter-free mode Photobooth mode in the Google Pixel 3 Camera app should now make it easier to click selfies, be it just your own selfie, couple selfies or even a group of people. With Photobooth Google wants users to get the perfect shutter-free selfies. Taking selfies can be difficult with the phone often being shaky resulting in a bad photo. Photobooth is a new shutter-free mode available in the Google Pixel 3 camera app. Once you tap the shutter button, Pixel 3 will detect when the phone is steady and take a selfie.
According to Google senior software engineer Navid Shiee and Google AI staff research scientist Aseem Agarwala. The team had to solve for a number of computational challenges, chiefly how to identify objectively “good” content and how to optimally time the camera shutter. They tackled the first with twin AI models: one for facial expressions and another that detects kissing. Shiee Agarwala, and colleagues worked with professional photographers to figure out five expressions that should trigger capture — smiles, stuck-out tongues, kissy faces, puffed-out cheeks, and reactions of surprise — and trained the aforementioned models (both of which use MobileNets, a family of low-overhead computer vision models designed to maximize accuracy) to detect them. Interestingly, the kiss detection model is a variation of AI system trained for Google’s eponymous Google Clips camera, fine-tuned specifically on kissing.
“Photobooth applies an attention model using the detected expressions to iteratively compute an expression quality representation and weight for each face,” says Google.