Facial Recognition and Cosmetics:
The Future Seen from the Pinnacle of That World

Are you familiar with the term "Fellow"? In a corporate context, Fellow refers to an individual who has mastered a specialized field and has made significant contributions to the company, industry, and society. This edition of Shiseido Talks is a special feature. Shiseido’s Tomonobu Ezure, pioneer in research on facial sagging, and NEC’s Hitoshi Imaoka, an authority on facial recognition, will engage in a dialogue. What thoughts drive these professionals at the forefront of research on the “face” as they conduct their studies, and what perspectives do they have on the world? They will discuss various aspects of the research activities within their respective companies, as well as prospects for the future.

NEC Fellow: Hitoshi Imaoka

Hitoshi Imaoka joined NEC in 1997 and became a Fellow in 2019. After joining the company, he first engaged in research and development of brain visual information processing. He began researching facial recognition technology in 2002, and has since contributed to the commercialization of biometric products in over 70 countries and achieved the world No.1 rating multiple times in the facial recognition benchmark tests conducted by NIST (National Institute of Standards and Technology). He received the Minister of Education, Culture, Sports, Science and Technology Award for Science and Technology (Development Division) in the 2022 fiscal year, and the “Medal with Purple Ribbon” in the spring of 2023. He is a specially appointed professor (on a visiting basis) at Tohoku University and a visiting professor at Tsukuba University.

Shiseido’s Mirai Development Research Institute Fellow: Tomonobu Ezure

Since joining Shiseido, Tomonobu Ezure has consistently engaged in research and development in the skincare field and is a pioneer in the study of facial sagging. His groundbreaking research on the mechanisms of facial aging, especially facial sagging, has provided a scientific basis for developing new solutions, including many of Shiseido’s flagship products. In addition to his publications in books and top academic journals, he is the first person to have received four consecutive top awards at meetings of the International Federation of Societies of Cosmetic Chemists (IFSCC), an unprecedented achievement. Indeed, in a keynote speech at IFSCC, he was referred to as "the most famous cosmetic researcher in the world." He is the first researcher to be honored with the title of "Fellow" in Shiseido's 150-year history.

Ezure:　Dr. Imaoka, the facial recognition system you created is now used worldwide. Can you tell me something about it?

Imaoka:　Facial recognition is a means of identifying and verifying individuals. There are various methods of personal identification, such as passwords, but the advantage of facial recognition is that it is something you can’t forget. For example, you might forget your keys or passwords, or they could be stolen, but with facial recognition, you don't have to worry about that. Using facial recognition instead of entering a password makes logging into smartphones easier, right?

Ezure:　But faces change as we age. Can the system still accurately recognize someone as the same person when they get old?

Imaoka:　That's what is key. In our facial recognition system, we select and recognize the "unchanging parts" among the various elements present in a face. We eliminate all parts that are prone to change, such as the skin and hair, and focus on the more stable parts, like the underlying bone structures. While we both study faces, I guess you can say that our focuses are opposite; you concentrate on the changing parts, while I focus on the unchanging parts. What led you to research faces, particularly sagging?

Imaoka:　Starting research from scratch must have been quite challenging.

Ezure:　It was. There was no clear definition of what sagging was or how to measure it and all that, and so I had to build everything from the ground up, starting with basic definitions. While other researchers in our research center were engaged in cutting-edge genetic analysis, I was the only one looking at photos of faces every day. People apparently wondered what I was doing (laughs).

Imaoka:　It's interesting that your research intersects with physics and other fields. I believe corporate researchers should be like that. Researchers generally tend to focus only on their specialized fields, but to have your research bear fruit, it's better to incorporate knowledge from physics, chemistry, and other fields for that matter.

Ezure:　I agree, that will definitely broaden one’s world. What prompted you to start researching facial recognition?

Imaoka:　I started out researching brain visual information processing but began working on facial recognition after I turned 30. Just around that time, the September 11 attacks in the US happened and when I considered that all passports worldwide had facial photos, I thought that if we could develop facial recognition, it would be beneficial to society. Because, as human beings, we will always have our faces. This will not change even in a thousand or ten thousand years. That's why I believed it was important to study faces. It was a daunting challenge, but I thought, "Well, let's try climbing that mountain"…and started my research. And here I find myself after many years of accumulating knowledge.

Ezure:　Your research and development work has spanned a wide range, from technology development to social implementation. Where does your passion for this come from?

Imaoka:　My driving force is simple. Research is like climbing a mountain; climbing mountains is fun. If you keep climbing, you'll eventually reach the top and see a view you've never seen before. I have continued to give the unknown world a try with that in mind. Of course, there are times when things don't go well, but there's no point in dwelling on that. I try to discard all negativity and focus solely on deepening the technology.

Ezure:　Did you not experience difficulties in advancing your research and development?

Imaoka:　Yes, in fact there was a time when NEC considered stopping facial recognition research. Globally, the accuracy of facial recognition was not improving, and we were far from practical application. When the research team was reduced to just me and a newcomer, I saw it as an opportunity to take on the challenge and thought, "Let's give it our all." For about two years, I focused continually on facial recognition. In the mornings, I would check the previous day's experimental results and continue research while modifying programs during the day. And at night, I would read papers to get new ideas. I thought we had to create something overwhelmingly superior to win in the world and gain market recognition.

Ezure:　It's truly impressive that you have achieved so much and continue to lead the world. You received the world No.1 rating multiple times in the facial recognition benchmark tests conducted by the National Institute of Standards and Technology (NIST).

Imaoka:　The first time I received an award was through sheer effort and momentum; it wasn’t easy after that. The pressure increased, and honestly, I was scared to try pursuing it again. But like in major global sports, it’s not the athletes who win once who are considered legends; it's those who continue to win multiple times. I thought that by consistently staying at the top in facial recognition technology, I could bring about a change in myself, considering it a form of personal discipline. I feel that continuing to win has led to increased recognition and trust, which in turn benefits the company. You have also won four consecutive top awards at the International Federation of Societies of Cosmetic Chemists (IFSCC). What drives you?

Ezure:　I wanted to create a new trend in the industry. That's why I participated in the IFSCC. At that time, sagging skin was considered outside the scope of cosmetics. I thought that if I could change that common perception—in other words, the direction of the industry—cosmetics could contribute more to society. With that in mind, I decided to continue presenting my work at these globally influential conferences.

Imaoka:　You must experience many difficulties in advancing your research and development work. So, when do find it enjoyable?

Ezure:　I find it enjoyable when I imagine, "Tomorrow, I will see a world that no one has seen before." At the beginning, we had only looked at the inside of the skin two dimensionally through a microscope. This has many limitations. So, to understand the skin structure better, I developed a method by which the skin can be viewed in three dimensions, and then even in four dimensions, by which I mean the skin dynamics. The new technology I developed opens up a world I never imagined within the skin. Thinking, "What will I see tomorrow?" or "What method should I develop next?" makes me look forward to the next day.

Imaoka:　I completely understand. That feeling of joy is really a big one for researchers. Exploring and pioneering the unknown world leads to new discoveries, which expand our view of the world. But finding a foothold in research can be challenging at times.

Ezure:　When advancing research, I try to be conscious of my perspective. I switch between a bird's-eye view in which I look for other methods and a tighter perspective that I use to ensure I take the next step securely. Additionally, when I’m looking for a cosmetic solution, since I can't predict what the responses of consumers will be, I can’t be certain what the right answer will be. In such cases, I think it's important to try everything one can think of. By doing so, I believe I can find answers that might be overlooked through conventional approaches. How do you handle it when you hit a wall in your research?

Imaoka:　Dr. Ezure, have you ever encountered difficulties in your working environment?

Ezure:　I don’t think I ever thought that there was a problem with the working environment. I think the relationship between researchers and corporations is important, and it's crucial that researchers maintain an independent spirit. A collection of independent researchers with distinct personalities can really benefit a company. As an analogy, cosmetics contain various ingredients, and each ingredient must stand out. But because there are many ingredients that do not mix well, like oil and water, the art of cosmetics is to blend them effectively to create higher value. I feel that the relationship between corporations and researchers is similar to that.

Imaoka:　I tend to think in a slightly more straightforward way, as my aim is to work where individual and corporate interests align. By doing my best, I can improve the company, and as the company improves, I also benefit. My wish is to grow together in that way.

Ezure:　In Japan, many people pursue research positions in companies after completing graduate studies in science and engineering. However, I feel the actual situation of being a corporate researcher isn’t widely understood. Through my position as a Fellow, I want to convey to society and young people that being a corporate researcher is a fulfilling, stimulating job that contributes to society. I believe this can also contribute to the bottom-up development of a science-oriented Japan.

Imaoka:　Seeing the technology that one develops actually benefiting society is the real thrill of being a corporate researcher. On the other hand, research also involves necessary down-to-earth tasks. I find it fascinating to identify problems from a state of uncertainty and use various methods to find solutions and create something. One time, when I saw NEC's facial recognition system—the technology being utilized worldwide that I was involved in developing—at an airport on my way back from a business trip to the US, I felt a strange joy and wonder.

Ezure:　How do you plan to advance your research and development in facial recognition in the future?

Imaoka:　I find the study of expressions, particularly "how humans move their facial muscles," to be interesting. The fleeting expressions that reveal true emotions are called micro-expressions, but there is still much we don't understand about them.

Ezure:　So, it's not just about recognizing individuals through facial recognition, but also capturing emotions.

Imaoka:　My ultimate goal is to investigate the relationship between the face and the brain comprehensively. Research on facial recognition is the first step. I’m currently also tackling　 new fields, such as measuring vital signs like heart rate. In terms of climbing a mountain, I feel like I'm only at the second stop.

Ezure:　How do you expect your work to give back to society?

Imaoka:　Right now, I am working on developing systems that assess people’s health status through facial recognition. Our faces reflect health conditions in all sorts of ways. As researchers studying faces, it would be great if we could look forward to such a future together.