Impact case study database
Deep face recognition accurately identifies suspects in international criminal investigations
1. Summary of the impact
We have developed a state of the art of machine learning model for face recognition. Our deep learning-based facial recognition model is facial feature specific and age integrated. As a result, facial images under investigation can be matched with greater accuracy. Notable cases where the model has been applied include revealing the real identities of the suspects in the Salisbury poisoning and the suspects involved in the disappearance of the journalist Jamal Khashoggi at the Saudi Embassy in Turkey. Both these cases are of international significance and were two of the biggest news stories of 2018.
2. Underpinning research
Since 2014, the use of deep learning-based face recognition has emerged to be a key piece of technology for facial identity verification and matching. Such systems are now widely utilised; for example, at border control points, in passport applications and for surveillance. Criminal investigations are also one major area which benefit from the use of deep face recognition.
Inspired by the many practical applications of face recognition and also being mindful of the many challenges involved in developing efficient, accurate and scalable computer algorithms, we have developed a state-of-the-art deep learning-based face recognition model to overcome the many challenges in face recognition [1,2]. Our model has feature specific modules in which we have trained machine learning models to match between specific parts of the face such as the eyes, nose and the forehead [3]. Additionally, we have successfully incorporated face ageing algorithms within our face recognition model so that the effect of age on facial images (taken many years apart) can be taken into account. Thus, the incorporation of feature specific models and age progression/regression models [4] into face recognition means our model can provide very accurate results for practical face matching and verification.
Feature Specific Deep Face Model: Traditional deep face recognition models use full frontal faces for training face recognition algorithms and for face matching. However, in many practical applications, full face information may not be available. In fact, it is often the case that facial images which are of interest in criminal investigations are not full-frontal faces. To overcome the issue of recognition from partial images, we have developed a deep learning-based face recognition model which is trained on specific parts of the face. We have found that, when it comes to face processing, the deep features in the nose and eyes are far more important than the features of cheek or mouth. As a result, our model is accurate, even when partial faces are presented as the visual cues for identification or verification [1].
Incorporation of Age Progression/Regression: Often, in investigations, photos of an individual taken several years apart may be gathered for comparison. Traditional face recognition systems may present severe challenges in matching photos of faces taken years apart. To overcome this issue, we incorporated face ageing algorithms we have independently developed into our feature specific deep face model. Our face age progression/regression model is based on deep neural network features based on hundreds of thousand diverse faces – making it gender and ethnicity specific [5]. The age progression/regression model itself has been used to generate images of missing people such as Ben Needham, Mary Boyle and Lee Boxell [6].
The development and deployment of our face recognition system is based on many years of grant funded research and that of a number of researchers and PhD students at the Centre for Visual Computing at University of Bradford. The work was led by Prof Ugail (2002 – present) with key collaborators, Dr Moi Hoon Yap (2008 - 2011), Dr A. Bukar (former PhD student) and A Elmahmudi (PhD student).
3. References to the research
Elmahmudi, A. and Ugail, H. (2019) Deep face recognition using imperfect facial data, Future Generation Computer Systems, 19, 213-225. DOI: 10.1016/j.future.2019.04.025
Snapshot Paper - Facial recognition technology, Centre for Data Ethics and Innovation, Part of UK Department for Digital, Culture, Media & Sport. (2020). https://www.gov.uk/government/publications/cdei-publishes-briefing-paper-on-facial-recognition-technology/snapshot-paper-facial-recognition-technology
Yap, M., Ugail, H. and Zwiggelaar, R. (2014) Facial behavioural analysis: a case study in deception detection, British Journal of Applied Science and Technology, 4(10): 14855-1496. DOI: 10.9734/BJAST/2014/6369
Bukar, A. and Ugail, H. (2017) On automatic age estimation from facial profile view, IET Computer Vision, 11(8), 650–655. DOI: 10.1049/iet-cvi.2016.0486
Elmahmudi, A. and Ugail, H. (2020) A framework for facial age progression and regression using exemplar face templates, The Visual Computer. DOI: 10.1007/s00371-020-01960-z
Bukar, A. and Ugail, H. (2017) Facial age synthesis using sparse partial least squares (the case of Ben Needham). Journal of Forensic Sciences, 62(5): 1205-1212. DOI: 10.1111/1556-4029.13523
Key Grants
Zwiggelaar, R. & Ugail, H. Facial Analysis for Real-Time Profiling. EPSRC (EP/G004137/1). October 2008 – March 2011. GBP552,679.
PDE-based geometric modelling, image processing, and shape reconstruction (Grant agreement ID: 778035), H2020-EU.1.3.3. - Stimulating innovation by means of cross-fertilisation of knowledge. January 2018 – December 2021. EUR535,500, institutional share EUR76,500.
4. Details of the impact
Independent investigative journalists who work on high profile criminal cases have particularly benefited from our deep face recognition model. Examples include Bellingcat and the New York Times. Bellingcat is an investigative journalism website that specialises in fact-checking and open-source intelligence. Bellingcat’s work on the cases relating to the War in Ukraine, the MH17 air crash, Syrian Civil War, Yemeni Civil War and the Skripal Poisoning are well known, and their investigations have won numerous international prizes. The New York Times is an international newspaper with worldwide influence and readership (currently 3rd in the US by circulation, and 18th in the world) and is well respected for the quality and the thorough nature of their investigative journalism (recognised by 130 Pulitzer Prizes since 1918). We have used our deep face model to identify criminal perpetrators in investigations of international significance.
For Bellingcat, we provided significant assistance in identifying the suspected Russian military intelligence officers associated with the 2018 poisoning of Sergei and Yulia Skripal using the nerve agent Novichok in Salisbury, UK [1, 2, 3]. We used our model to compare photographs of the suspects taken at different ages. To compensate for the significant differences in age between the photos, we ran our automatic ageing model to bring the two individuals in the picture to the point where they were directly comparable. Then, we ran the same face recognition algorithms to compare between the two aged photos. As a result, we found that the percentage probability of a match ranged from 97.7 up to 98.4 percent. This meant that we could say with confidence that the images of Anatoliy Chepiga and that of Ruslan Boshirov were of the same man. Having confirmed the real identity of Boshirov as Colonel Chepiga, Bellingcat asked us to run the same analysis on the man known as Alexander Petrov, who had visited Salisbury with Chepiga, but was suspected to be Alexander Mishkin, also a member of the Russian military intelligence. We ran the same algorithms over an earlier passport picture of Mishkin and a cover passport in the name of Petrov. Again, we could say with confidence that they were the same man. Bellingcat, using our analysis, delivered the findings in a final report on the case and a briefing to the Houses of Parliament, widely reported in the international media. Bellingcat won the European Investigative Reporting Award 2019 for their investigation of this case [4]. Bellingcat acknowledged that the applications of our models were an invaluable part of this investigation [1] " The collaboration has been particularly invaluable when we have had to
compare and validate the identity of subject faces on photographs that were taken decades apart, or when photographic quality has been too poor to use in conventional tools." - Bellingcat.
In 2018, the New York Times reported that the US-based journalist and critic of Saudi Arabia’s government Jamal Khashoggi was killed by a team of 15 people from Saudi Arabia who were sent to the Saudi Embassy in Turkey. The Times has cited senior officials of the Saudi government and a forensic expert was involved in the killing of Khashoggi, who is believed to have used a handsaw to dismember Khashoggi’s body. The journalists from the New York Times identified some of the suspects based on the CCTV footage received from inside Istanbul airport, and they needed to verify the identities of the individuals. They trawled through a number of social media outlets looking for the profiles of the suspects and their photos online. We were requested to look at the similarities between the CCTV images and those from social media [5]. Our deep face model was able to successfully match the CCTV images with images from social media. In one case, a suspect was wearing a fake beard to disguise his appearance was also accurately identified by our deep face recognition model. Based on the findings reported by the New York Times, the Turkish authorities used these data when investigating the disappearance of Khashoggi.
In addition to the above two key examples, we also have helped both Bellingcat and New York Times to carry out another significant piece of joint investigative journalism. In that case, we were able to successfully identify an individual responsible for the assassination of a former Chechen separatist in Germany in 2019 [6, 7]. This piece of work was carried out for Bellingcat and New York Times independently. The reporting led to an investigation for finding the real individual responsible for this murder in Germany. " Given the importance of face recognition technology for our visual investigations specifically and separating fact from fiction more generally, the work of Professor Ugail is of great value for the journalism community and beyond." - The New York Times.
Aside from the application of our face recognition in helping with the investigation of high-profile criminal cases, the work is being applied in other areas. For example, recently, results from our face recognition system have been utilised as a crucial source for providing expert evidence in face identification at the first-tier tribunal in a UK court of law [8].
5. Sources to corroborate the impact
Testimonial letter from Bellingcat.
Full report: Skripal Poisoning Suspect Dr. Alexander Mishkin, Hero of Russia, https://www.bellingcat.com/news/uk-and-europe/2018/10/09/full-report-skripal-poisoning-suspect-dr-alexander-mishkin-hero-russia/ (The section on "A photo worth a thousand words" specifically discusses our findings).
Salisbury poisoning suspect Alexander Petrov is really military doctor Alexander Mishkin, investigative website claims - ITV News. https://www.itv.com/news/2018-10-08/salisbury-poisoning-suspect-alexander-petrov-is-really-military-doctor-alexander-mishkin-investigative-website-claims/
The Investigative Reporting Award 2019 Winner. https://www.europeanpressprize.com/article/unmasking-salisbury-poisoning-suspects-four-part-investigation/
Testimonial letter from The New York Times. (Global readership for the New York Times in 2020 stands at 5.8 million.)
A coalition of investigative journalists says ‘The New York Times’ is wrong about a Berlin murder, but the killer is still likely a Russian state assassin, https://meduza.io/en/feature/2019/09/28/a-coalition-of-investigative-journalists-says-the-new-york-times-is-wrong-about-a-berlin-murder-but-the-killer-is-still-likely-a-russian-state-assassin
New Evidence Links Russian State to Berlin Assassination. https://www.bellingcat.com/news/uk-and-europe/2019/09/27/new-evidence-links-russian-state-to-berlin-assassination/
Corroborator: Advocate, Themis Advocates, Edinburgh.
Additional contextual information
Grant funding
Grant number | Value of grant |
---|---|
EP/G004137/1 | £544,499 |
778035 | £485,000 |