Shiseido Develops Digital Application “Beauty AR Navigation” to Support Appropriate Beauty Treatment Regimens

Shiseido has developed the “Beauty AR Navigation,” a digital application system to support consumers, even in places where in-person instruction is inaccessible, in mastering and practicing appropriate beauty regimens through interactive communication using smartphones and tablet terminals. The application was developed by utilizing Shiseido’s expertise in beauty regimens and the Kansei*1 science field cultivated over 150 years of history in addition to AI-driven motion recognition technology that captures detailed hand movements. The Beauty AR Navigation provides consumers with easy-to-understand skincare steps via video and narration, as well as with quantitative evaluation that will show them if their skincare routines are appropriate—feedback that is expected to help increase their motivation to master a proper beauty regimen over the long term.
The Beauty AR Navigation was announced at CES 2024*2, one of the world’s largest digital technology trade shows in Las Vegas, US, held this year from January 9 to 12, 2024. With this application, which was developed in quick response to changes in society where all sorts of experiences are being digitally driven, Shiseido will provide consumers with a skincare experience that will meet both their satisfaction and sense of happiness from the act of beautifying oneself, while also addressing the needs of diverse consumers and the environment, anytime and anywhere.
*1 Kansei science is an approach to product innovation that focuses on emotional engineering. Fusing aesthetics, sensory psychology and design, it seeks to create objects and experiences that appeal to the senses (visual, olfactory, auditory and tactile) in order to arouse an emotional response in the user.
*2 CES 2024： https://www.ces.tech/about-ces.aspx

Fig1. Screen images of the Beauty AR Navigation, a digital application newly developed by Shiseido.

Download Small Image[147KB] Download Large Image[632KB]

The importance of beauty regimens and the challenges in the digital society

Since its founding in 1872, Shiseido has been engaged in the development of beauty regimens and the provision of beauty techniques through its beauty experts (currently called Personal Beauty Partners (PBPs)). In fact, it has been made scientifically clear that proper beauty regimens pay an important role in maximizing the effects of skincare treatment. When users apply lotion and emulsion following the “Effective Skincare Application Method” developed by Shiseido, the feeling of the moisture, firmness, and elasticity of the skin is enhanced and the psychological state of the users improve such that they experience a positive uplifting feeling, compared to the time when they follow their own self-care routine*3. Study results have also shown that skin moisture levels improve after one month of repeating the proper beauty regimen*4.
Meanwhile, types of beauty experiences, from information gathering, counseling, and purchasing to mastering application steps and sharing post-use evaluations and impressions, have changed from conventional, in-store and in-person communication to a digital-based, non-contact experience using smartphones and other devices. This has resulted in a significant increase in user convenience. At the same time, Shiseido’s interview surveys revealed the concerns and needs of users; their worries about their beauty routines, wondering whether they were using the cosmetics correctly, or if they were failing to get the full benefits due to inadequate application, while also expressing their desire to learn the proper steps of cosmetics application to become more beautiful. Thus, this time, with the aim of providing services that address such concerns and needs more thoroughly through advanced technology, with the convenience of the digital experiences currently offered, Shiseido began the development of a digital application that will help consumers to master and practice appropriate beauty regimens using their smartphones and other devices.
*3 Watanabe et al. Presented at the 22nd Annual Meeting of Japan Society of Kansei Engineering (2000)
*4 A skincare routine that provides a comfortable feel on the skin and brings smiles to faces (2020)https://www.shiseido.co.jp/sw/beautyinfo/DB009657/ (Japanese only)

Features of the Beauty AR Navigation application

The newly developed application has two key functions to support users in mastering an appropriate beauty regimen.

(1) Appropriate beauty regimen instructions
The application captures the user’s daily skincare routine via smartphone or other device camera and provides the best movements of the hands, including the hand direction and moving speed, etc., according to the “ideal” movements analyzed based on the techniques of Shiseido’s specialists. The hand movements will be explained in real-time and in an intuitive and easy-to-understand manner with the movements of the virtual hands superimposed on the image of the user’s own hands using AR technology*5 and voiceover narration.

(2) User hand movement evaluation
After the instruction, the hand movements of the user are quantitatively evaluated through the comparative analysis of hand direction and moving speed between the ideal movements and the user’s movements, followed then by the results being presented with scores.

In addition to conventional services such as online counseling, by repeating the above (1) and (2) steps via the application during daily skincare routines, users are expected to be able to master the appropriate beauty regimen more easily even in a non-contact environment, all the while keeping their motivation up.

*5 AR technology: Augmented reality technology. A technology that augments reality by overlaying digital information created by computer graphics, etc.

Future prospects

Since its founding in 1872, Shiseido has demonstrated its respect for society, the environment and nature and its approach has been to create social value across our business activities, through the development, production, and the sales of cosmetics. Now, with our corporate mission, BEAUTY INNOVATIONS FOR A BETTER WORLD, we aim to solve social issues through an approach unique to a beauty company, while working towards greater sustainability, and enriching people’s lives.
　With digital technology, the newly developed digital application, “Beauty AR Navigation,” provides inclusive opportunities for everyone to enjoy the power of beauty, anytime and anywhere, and acts as an important technology to accelerate the realization of a sustainable society. Going forward, Shiseido will apply this technology to its own businesses as soon as it can and bring it to people around the world. The development and social implementation of other digital technologies will also be accelerated at the same time.

Reference: Shiseido’s news release on digital technology

・Shiseido successfully predicts future skin concerns by facial images (2023)
https://corp.shiseido.com/en/news/detail.html?n=00000000003728
・Shiseido’s skin analysis function “Hada Pasha” becomes even more convenient with enhanced functionality (2021)
https://corp.shiseido.com/jp/news/detail.html?n=00000000003244 (Japanese only)
・Shiseido successfully develops an application that analyzes facial expressions based on 3D data of facial shape (2021)
https://corp.shiseido.com/jp/news/detail.html?n=00000000003174 (Japanese only)
・Shiseido launches “Hada Pasha” (application for skin measurement and advice) on Thursday, September 28, 2017 (2017)
https://corp.shiseido.com/jp/news/detail.html?n=00000000002276 (Japanese only)
・Shiseido Releases “Smile App” Based on Its Research into Facial Expressions
https://corp.shiseido.com/en/news/detail.html?n=00000000002323




Shiseido’s R＆D strategy：
This study was conducted under one of the three pillars of Shiseido’s R＆D strategy, Future Beauty INNOVATION, with the aim of achieving beauty in harmony with the skin, body, and mind, and by taking advantage of Shiseido’s expertise in the Kansei science field that objectively explains subjective matters such as human sensation and psychology through a scientific approach.
・Integrated Report 2022 (Beauty Innovation)
https://corp.shiseido.com/report/en/2022/value_creation/innovation/
・Key words:
Future Beauty INNOVATION, Kansei science, digital, beauty regimen