AI Trends Weekly Brief: Facial Recognition with AI Gaining


Facial Recognition Combined with AI
Making Strides in Authentication, Augmented Reality

Facial recognition software is being combined with artificial intelligence and computer vision to offer new ways to authenticate users, help law enforcement identify criminals, augment reality to help a user navigate unfamiliar surroundings.

The power of these systems naturally is raising concerns among privacy advocates about more government intrusion into everyday life.

Here we review a few companies combining facial recognition and AI to get a snapshot of how this market is evolving.

SenseTime Gaining in China with Facial Recognition

SenseTime, a Chinese company specializing in computer vision and deep learning, recently made headlines with its $410 million round of funding, said to be the most capital received in a single round within the field of AI.

In a recent interview with AI Trends June Jin, chief marketing officer of SenseTime, said the core team of scientists and engineers who founded the company in October 2014 came from the Chinese University of Hong Kong. They had performed research in computer vision, succeeding in early work to surpass the accuracy of human eyes.

Computer vision is the core of SenseTime offerings now extended to facial, text and image recognition. Jin emphasized that SenseTime developed its own deep learning framework, not based on any open source project, but developed internally.

Today SenseTime’s clients are in the fields of security, financial services, mobile Internet, smartphones and autonomous driving. Its facial recognition technology is being applied to payment and picture analysis, such as bank card verification services and security systems. Its facial recognition software is said to be in use by over 300 companies, including China Mobile Communication Co., China UnionPay, Huawei Technologies Co., Ltd, Xiaomi, Inc. and, Inc.

SenseTime says its facial recognition software has an error rate of less than one in 100,000.

Mobile payment applications are also helping SenseTime’s business in China, where few consumers today carry cash or credit cards. “Here there are many options for mobile payments,” Jin said. The SenseTime product that helps validate mobile payments is primarily used by banks.

SenseTime is also selling its facial recognition software to Chinese police departments, who are deploying advanced surveillance systems. The Chinese authorities combine the country’s photo ID database with extensive footage taken from public surveillance cameras, to analyze billions of faces in order to identify people. China has an estimated 176 million surveillance camera, according to IHS Markit consultants, compared to 50 million in the US.

Lenient privacy laws in China enable enormous amounts of data to be available to companies at a low cost, enabling further development of facial recognition software. Wang Shengjin, a professor at Tsinghua University’s Department of Electrical Engineering, was quoted in an interview in Next Biometrics as saying that compared to Western technology companies, Chinese companies are “far ahead when it comes to deploying facial recognition technology commercially”.

The smartphone market in China has been helpful to SenseTime’s business. “We have many more brands in China than in the US,” Jin said, noting that China Mobile is a large customer. In the past to register a phone, the user had to for to a shop and present a photo ID to be validated. With SenseTime’s SenseID product, a consumer who goes to the store puts their ID card on a pad, stands in front of a camera, and it automatically validates, Jin said.

Cloud Walk Tries to Predict Crime Before it Happens

Another Chinese facial recognition firm, Cloud Walk, is developing an AI system designed to predict the probability of a crime occurring, to potentially allow law enforcement to stop the crime from ever happening. An example conveyed in a company video, showed the software was able to distinguish thieves from normal passengers at a train station. Chinese law does not allow people to be charged with a crime they have yet to commit, but citizens can be charged for attempting to commit crimes.

Large Chinese Internet companies are also exploiting the facial recognition market. Ant Financial, the payment affiliate of Chinese e-commerce giant Alibaba, now allows users to conduction transactions by scanning their faces, according to a report in Forbes. Chinese search giant Baidu is using facial recognition technology to verify customer identities for insurance firm Taikang.

Also, many Chinese hotels, schools and kindergartens are installing cameras to scan faces before allowing entry. Some colleges are scanning students to spot “ghost writers” trying to sit for exams for students. And a KFC in Beijing is scanning customer faces in order to recommend menu items based on age, gender and mood.

As part of a national campaign to promote “civilized” behavior, the Forbes report states, regulators have deployed facial recognition to name and shame jaywalkers in dozens of Chinese cities. In Jinan, face-reading cameras can take short videos of pedestrians crossing roads on a red light. The personal information of offenders, including names and home addresses, are then displayed on screens at the side of the road as a warning, according to the state-run Xinhua News Agency.  This is not a breach of privacy under Chinese law.

Biao Leng, an associate professor at the School of Computer Science and Engineering at China’s Beihang University, stated, “China will remain ahead of western countries in using facial recognition. From a government strategy perspective, the technology will move much faster than in the US and Europe.”

Providing smart surveillance systems for national security has given Chinese startups a huge leg up, stated Cool Zhang, Research Director for Enterprise System and Software Research at IDC China, in an account in the Wall Street Journal. China’s annual security budget is huge, making it easier for companies to generate revenue, he stated. “You’ll likely see more and more attention and investment into this area,” Zhang stated. “The use cases in various industries and businesses are only increasing.”

Privacy advocates express concerns that the technology can be used by governments to conduct random surveillance without cause. Clare Garvie, a researcher for the Center on Privacy & Technology at Georgetown University’s Law Center, stated, “This is a force multiplier that takes the identification of people in public to a completely different level.”

US Not Standing Still

While progress in facial recognition is rapid in China, it is also happening in western markets.

An early player was Identix, founded in Minnesota in 1982, starting out in user authentication using fingerprints. In 2007, Identix announced FaceIt, which can pick out someone’s face in a crowd and compare it to databases worldwide to recognize and put a name to it. The software is written to detect multiple features on the human face. It can detect the distance between the eyes, width of the nose, shape of cheekbones, length of jawlines and many more facial features. The software does this by putting the image of the face on a faceprint, a numerical code that represents the human face.

Identix was eventually acquired by MorphoTrust, which today markets a “government-class biometric and authentication portfolio.”  Morpho recently announced a partnership with, which offers consumer-friendly mobile authentication services. Companies are offering mobile software development kits and APIs for developers who require advanced user verification and mobile facial recognition in their applications. Examples of how the applications could be used include verification of identity documents in mobile transactions, and the use of biometrics for user authentication in high-trust transactions.

In a press release, President and CEO of MorphoTrust Bob Eckel stated, “We have a shared vision of simplifying and securing lives, while enabling new business models. We are interested in shaping the future of identity for mobile, online and in-person transactions where identity matters.”

Bob Geiman, chairman and co-founder of the Boston-based mobile identity authentication company, stated, “With the rise of peer-to-peer business models and more and more transactions moving to online, companies have an unmet need to make sure people are who they claim to be.”

Blippar offers mobile facial recognition app

Facial recognition technology has been extended by AI company Blippar to smartphones. The app lets users point a mobile camera at a person and instantly pull any information about them available on the internet.

CEO Amarish Mitra stated, in a piece posted on CNBC, “A very large part of who we are an an identifiable individual uniquely is our face. It could be one of the biggest ways of how we transact day to day, so that there are no more handbags or credit cards.”

Blippar has compiled the face prints of some 250,000 celebrities and politicians. By the end of the year, Mitra sated, any of its users will be able to upload their facial biometrics so the app “recognizes” them.

Blippar is in talks to license its technology so that others can build their own products and services that use facial recognition. Mitra said the database will not include anyone under the age of 16, that private individuals must register, and the company will delete anyone who asks to be scrubbed from memory.

Blippar is also extending into augmented reality. The Pokemon Go augmented reality app because popular, with its display of virtual images through the smartphone display, against the real world in the background. That app depends on a combination of GPS and compass to know where the user is located, subject to virtual images appearing jittery and out-of-place in certain locations, such as big cities.

Blippar employs computer vision to figure out where the user is and what direction he or she is facing relative to the busy surrounding urban space. Omar Tayeb, a Blippar cofounder and its CTO, is quoted in a piece in MIT Technology Review, as saying the company has licensed image data for major cities, another version of Google Street View he declined to identify. The app indexes the pictures then matches them up with what the smartphone user seems through the camera. So far, the company has tested it in San Francisco, Mountain View and London.

A video on Blippar’s website demonstrates the potential. Images appear, such as of a restaurant menu board, floating above the ground, near the restaurant being promoted. Tayeb said the location estimation is accurate within a range of three to eight meters. Blippar plans a wider release in coming months.

In summary, privacy a concern in western countries.

Debate will continue in western countries on the tug of war between maintaining privacy and giving up personal information for some gain. A Pew Research study published in 2015 found that 91 percent of adults believed they have lost control of how personal information is collected and used by companies. At the same time, 74 percent said is “very important” for them to have control of who can get information about them. Citizens of China may not have much choice about what information the government collects and discloses. In the US and other western countries with a high value on personal privacy, the formula for success for facial recognition and computer vision products will need to include options for protecting personal privacy. The best solutions will be permission-based.

  • Written and compiled by John P. Desmond, AI Trends editor