How Will Retailers Use Facial Recognition Technology?

Remember when checking out at a retail store with a wave of a smartphone seemed cutting edge? That’s so 2014. Now, customers can check out just by showing their faces.

Diners at a KFC in the eastern Chinese city of Hangzhou can pay for their meals simply by smiling, using a facial recognition system dubbed “Smile to Pay,” according to Reuters. Meanwhile, reports recently surfaced of a Walmart patent application that would use facial recognition to try to identify varying levels of customer dissatisfaction.

These innovations are the latest indications that retailers are seriously exploring facial recognition technology. The systems have the potential to make it more convenient for customers to check out and also could provide retailers with a wealth of data about their customers. However, some retail experts also question the utility of the technology and whether it could violate customers’ privacy, depending on how it is implemented.

A New Wave of Facial Recognition Tools in Retail

In China, Yum China Holdings, which spun off from its U.S. parent Yum Brands last year, is piloting the new technology at an outlet called KPRO, targeting a younger generation of Chinese consumers, Reuters reports.

Ant Financial, an affiliate of Alibaba Group, is powering the software behind the tech and says this is the first commercial application of it worldwide, according to Reuters. The news service notes that diners pay by scanning their faces at a kiosk and then entering their phone numbers, which is designed to prevent people from cheating the system.

“Combined with a 3D camera and liveness detection algorithm, Smile to Pay can effectively block spoofing attempts using other people’s photos or video recordings and ensure account safety,” Jidong Chen, Ant’s director of biometric identification technology, says in a statement.

"We hope one day in the future people can go out without their cell phones or wallets," Dong Liyun, a product manager at Ant Financial, tells CNN.

Meanwhile, closer to home, The Wall Street Journal first reported this past summer on a patent application Walmart filed in 2012 that “describes a system that uses facial recognition to detect customer dissatisfaction and adjust staffing accordingly.”

The system uses video cameras at store checkout lanes. If the cameras detect an unhappy customer, the system notifies employees in other parts of the store to come to the checkout in the hopes of resolving any issue. The idea is that store employees will be able to more proactively respond to customer service issues, perhaps before the customer even complains.

“It is easier to retain existing customers than acquire new ones through advertising,” the patent filing reads, according to Business Insider. “Often, if customer service is inadequate, this fact will not appear in data available to management until many customers have been lost. With so much competition, a customer will often simply go elsewhere rather than take the time to make a complaint.”

However, Business Insider adds, the system could also be used to analyze trends in shoppers’ purchase behavior over time, according to the patent filing. To do so, the publication says, the system matches customers’ “biometric data” (their facial expressions) to their transaction data (what they are buying).

Walmart says this will help stores detect changes in a customer’s purchase habits due to dissatisfaction. “Significant drops or complete absence of customers spending ... may be identified,” the patent filing states. The company has not yet deployed such technology.

This is not the first time Walmart has dabbled with facial recognition software though. In 2015, Fortune reported that the retailer trialed the technology to identify suspected shoplifters. However, the company abandoned the idea because it did not deliver a strong enough return on investment.

Questions About Customer Privacy

Walmart says in the patent application that the system can improve customer service and potentially reduce staffing costs. According to Forbes, the company notes in the application, it can “be very expensive to maintain sufficient staff to provide great customer service. It can also be difficult to establish an appropriate staffing level that will provide proper customer service without excess staffing.”

However, Forbes points out that several retail industry experts are not so enthusiastic about the potential of Walmart’s envisioned technology. They expressed misgivings about the system’s impact on customer privacy as well as how useful it would ultimately be.

Lee Kent, principal at Your Retail Authority, said in an online RetailWire discussion, “There are many good uses for facial recognition, but this is not one of them.”

“Facial recognition and biometric data is highly sensitive and personal and Walmart is significantly over-reaching to use it for improving customer experience and reducing staff costs,” said Mark Ryski, CEO of HeadCount, which helps retailers analyze store traffic and conversion data. “While I do believe there is great potential to use and apply biometric data (especially for security purposes), I am very concerned about how some retailers might use this data under the guise of ‘improving customer service.’”

Others said the system is not really necessary. “Why don’t cashiers ask if the customer found everything they were looking for — not having found it being the major cause of unhappiness?” asked Stephen Needel, managing partner at Advanced Simulations, a virtual reality marketing research firm. “You don’t need facial recognition for that. And give the cashier an action option if the customer says no.”