What is face recognition software software & how does it work?

Face recognition is a technique for identifying or verifying an individual’s identity by using their face. Face recognition software can be used to identify people in photos, videos, or in real time. During police stops, law enforcement may also use mobile devices to identify people.

However, face recognition data is prone to error, which can lead to people being accused of crimes they did not commit. Facial recognition software is particularly bad at recognising African Americans and other ethnic minorities, women, and young people, frequently misidentifying or failing to identify them, affecting certain groups disproportionately.

Face recognition has also been used to target people engaging in protected speech. The software is expected to become more popular in the near future. It may be used to detect people’s activities in the real world, similar to how automatic licence plate readers track cars based on plate numbers. In other nations, and also at athletic events, real-time facial recognition is now in operation.

How does Face Recognition software work?

Face recognition software employs computer algorithms to identify specific, distinguishing features on a person’s face. These details, such as the distance between the eyes or the shape of the chin, are then mathematically represented and compared to data on other faces collected in a face recognition database. A face template is data about a specific face that differs from a photograph in that it is designed to only include details that can be used to distinguish one face from another.

Instead of positively identifying an unknown person, some face recognition is designed to calculate a probability match score between the unknown person and specific face templates stored in the database. Instead of returning a single result, these systems will present several potential matches, ranked in order of likelihood of correct identification.

Face recognition software
Face Recognition software work

Face recognition software differs in their ability to recognize individuals in difficult situations such as inadequate lighting, low image quality, and a suboptimal angle of view (such as in a photograph taken from above looking down on an unknown person). 

When it comes to mistakes, there are two main principles to grasp:
  • When a face recognition fails to match a person’s face to a picture that is, in reality, in a database, this is referred to as a “false negative.” To put it another way, the machine would return zero results in response to a question. 
  • A “false positive” occurs when a face recognition matches a person’s face to a picture in a database but the match is wrong. When a police officer submits a picture of “Joe,” the machine incorrectly informs the officer that the photo is of “Jack.”

When looking at a face recognition system, pay close attention to the “false positive” and “false negative” rates, as there is almost always a trade-off. For example, if you use face recognition to unlock your phone, it is preferable if the system misidentifies you a few times (false negative) than it is for the system to misidentify other people as you and allow those people to unlock your phone (false positive). If the consequence of a misidentification is that an innocent person is imprisoned (as in a mugshot database misidentification), then the system should be configured to have as few false positives as possible.

How Law Enforcement Uses Face Recognition software

Face recognition software is being used more and more commonly in routine policing by law enforcement agencies. Mugshots are collected from arrestees and compared to local, state, and federal facial recognition databases. When an arrestee’s picture is taken, it is stored in one or more databases and is scanned every time the police conduct another criminal investigation. 

  • The massive mugshot databases can then be queried by law enforcement to recognise suspects in pictures captured from social media, CCTV, traffic cameras, or even photos they’ve taken themselves in the field. Faces can also be linked in real time to “hot lists” of individuals accused of criminal activity.
  • Adaptability of mobile face recognition allows police officers  to use smartphones, tablets or other portable devices to take  a snapshot of a driver or pedestrian in the field and instantly link the photo to one or more facial recognition databases to try to identify them.
  • Face recognition has been used in airports, at border crossings, and at sporting events like the Olympics. Face recognition may be used in private spaces such as stores and sports stadiums, but private sector face recognition may be subject to different rules.

Numerous libraries at the city, national, and federal levels support these applications of face recognition . According to estimates, 25% or more of all state and local law enforcement authorities in the United States may conduct facial recognition searches on their own databases or those of other agencies.

As of 2015, at least 39 states were using facial recognition software in their Department of Motor Vehicles (DMV) databases to prevent fraud, according to Governing magazine. According to the Washington Post, 26 of these states encourage law enforcement to search or order scans of driver licence records, but this figure is likely to have grown over time.

Local databases can also be used, and these databases can be very broad. The Pinellas County Sheriff’s Office in Florida, for example, may have one of the most extensive state face recognition databases. According to Georgetown University study, more than 240 organisations scan the database approximately 8,000 times each month.

The federal government has many facial recognition software, but the FBI’s Next Generation Identification database, which holds over 30-million face recognition documents, is the most important for law enforcement. The FBI grants state and local governments “lights out” access to this website, implying that no one at the federal level monitors the individual searches. Facial Analysis, Comparison, and Evaluation (“FACE”) Services is a team of FBI employees devoted solely to face recognition searches. The FBI has access to over 400 million non-criminal images from local DMVs and the State Department, and FACE has access to driver’s licence and ID photos in 16 states in the United States.

Related Article : How to choose an AI development company that’s a perfect fit for you

Threats Posed By Face Recognition software

  • Face recognition data is easy to obtain for law enforcement and difficult to prevent for members of the public. Faces are constantly shown to the press, so unlike passwords, they are difficult to modify. We’re seeing an uptick in agency knowledge exchange. Cameras are becoming more efficient, and technology is evolving at a rapid pace.
  • Face recognition data is often obtained from mugshot photographs, which are taken immediately after an arrest and before a judge can decide guilt or innocence. Even if the arrestee is never charged, mugshot images are always kept on file.
  • Face recognition data is vulnerable to distortion, despite its pervasiveness and technological advancement. Indeed, the FBI acknowledged in its privacy impact evaluation that its method “might not be adequately effective to reliably identify other images of the same name, resulting in an increased percentage of misidentifications.”

While the FBI claims that its method will detect the true candidate in the top 50 profiles 85 percent of the time, this is only true when the true candidate is present in the gallery. If the applicant is not in the gallery, the machine can nevertheless generate one or more potential matches, resulting in false positive outcomes. These people, who aren’t candidates, could become suspects in crimes they didn’t commit. A faulty system like this shifts the traditional burden of proof away from the government and forces people to try to prove their innocence.

Face recognition becomes worse as the number of users in the database grows. This is due to the fact that so many beings on the planet resemble one another. Matching accuracy declines as the probability of identical faces grows. Face recognition is especially bad at recognising African Americans.

According to a 2012 study co-authored by the FBI, African Americans have lower accuracy rates than other demographics.

Face recognition also frequently misidentifies other ethnic minorities, young people, and women. Due to racially biassed police practises, a disproportionate number of African Americans, Latinos, and immigrants are included in criminal databases. As a result, the use of face recognition technology has an uneven impact on people of colour.

Some argue that having a human backup identification (a person who verifies the computer’s identification) can help to reduce false positives. However, research shows that when people lack specialised training, they make incorrect decisions about whether a candidate photo is a match roughly half of the time. Regrettably, few systems have specialised personnel who review and narrow down potential matches. 

Face recognition software can be used to identify people who are engaging in protected speech. For example, during the protests following the death of Freddie Gray, the Baltimore Police Department used face recognition on social media photos to identify and arrest protesters.

There are few safeguards in place to protect ordinary Americans from the abuse of facial recognition software . In general, departments do not need warrants, and often do not even require law enforcement to suspect anyone of committing a crime until using facial recognition to recognise them.