Impact case study database
Advanced Algorithm Development for use in X-ray and Computed Tomography Security Scanners used for Transport and Border Security
1. Summary of the impact
Our development of novel algorithms for automatic threat item detection and associated capabilities for use within X-ray and Computed Tomography (CT) security scanners informed government policy, commercial product development and airport technology planning in this area. This research, carried out between 2014 and 2020 at Durham University, directly:
informed UK/US government policy for aviation and border security screening
provided new enhanced software capabilities within the design of security scanners by Gilardoni S.p.A. , Kromek PLC, Micro-X Ltd, Rapiscan Ltd and Smiths Detection
informed the design, development and evaluation of new software capabilities for security scanners with Cosmonio BV, Battelle, L3Harris Technologies and VisionMetric Ltd
informed the procurement of airport security screening products at Gatwick Airport
These impacts directly contribute to the security of over 500 million passenger journeys per annum with additional potential reach to 2-3 billion passenger journeys across 30+ countries globally.
2. Underpinning research
This research relates to the use of automated image understanding techniques to provide enhanced automatic screening for the detection of threat or contraband items within X-ray and CT security imagery. This addresses the question “What is in the bag/parcel/package?” (Figure 1) . Conventional X-ray screening offers multiple-view (2D) images of a scanned item (Figure 1, left) whilst recent advances in high-throughput CT imaging offer a full 3D volumetric image (Figure 2, left). Across both domains, the key challenge is to be able to address this issue with a high probability of detection (true positive) whilst minimising the false alarm rate (false positive) within highly cluttered scanner imagery under real-time processing bounds (as the item passes through the scanner). Furthermore, the aim must be achieved whilst working from limited example data availability (compared to other image understanding domains).
Fig1: Exemplar X-ray image (single view, 2D – left) and CT image (3D volume, rotatable – middle) imagery of a baggage and the on-site X-ray testing facilities at Durham University (right).
The Durham research team pioneered the use of object recognition algorithms for this task: firstly, by extending existing object recognition paradigms for use with 3D CT security imagery [R1, R2, R5] and subsequently by introducing the use of deep convolutional neural network (CNN) based approaches to the domain of X-ray based security screening [R3, R4]. Furthermore, we were able to extend this research to support the related research question of how to perform threat image projection (TIP) within CT imagery i.e. “How do we realistically insert a threat item into an otherwise benign 3D bag image”? [R5, R6]. TIP inserts such “fake” threat items into an otherwise benign baggage item in order to monitor security screening operator performance. This is a legally mandated process already in place for all X-ray based aviation security screening processes (UK/EU and beyond). In addition, it will now be increasingly used for 3D CT-based aviation security screening processes, due to the move to new mandatory UK/EU security screening standards for both hold baggage (ECAC EDS standard 3 - EU Regulation 1087/2011, September 2020) and cabin baggage (ECAC EDSCB C1-C3, December 2022) – both of which require the 3D CT security screening. The key findings of this research at Durham were that:
object detection can be performed using an extended 3D bag-of-visual words architecture in cluttered CT imagery to achieve ~98% true positive detection with low false positive <1% [R1].
plausible TIP within 3D CT imagery can be performed, by using materials-based segmentation for void space determination [R2] with lesser concern for noise and artefact filtering [R5], within an optimisation-based emplacement framework [R6].
transfer learning, with a CNN architecture pre-trained on visible-band imagery and refined on X-ray imagery, enables object detection in cluttered X-ray imagery with a high true positive (98%+) and low false positive <1% despite limited data set availability and inherent differences in image characteristics (projection, spectral-band, colour/texture) [R3, R4] (Figure 2).
the levels of performance achieved by these CNN approaches are repeatable over multiple datasets including those independently provided by the UK government [R4].
deep neural networks enable object detection in cluttered X-ray imagery [R3, R4] significantly outperforming earlier feature-based approaches (83% true positive / 3% false positive, [R4]).
3. References to the research
3D Object Classification in Baggage Computed Tomography Imagery using Randomised Clustering Forests (A. Mouton, T.P. Breckon, G.T. Flitton, N. Megherbi), In Proc. International Conf. on Image Processing, IEEE, pp. 5202-5206, 2014. [ DOI]
Materials-Based 3D Segmentation of Unknown Objects from Dual-Energy Computed Tomography Imagery in Baggage Security Screening (A. Mouton, T.P. Breckon), In Pattern Recognition, Elsevier, Volume 48, No. 6, pp. 1961–1978, 2015. [ DOI]
Transfer Learning Using Convolutional Neural Networks For Object Classification Within X-Ray Baggage Security Imagery (S. Akcay, M.E. Kundegorski, M. Devereux, T.P. Breckon), In Proc. Int. Conf. on Image Processing, IEEE, pp. 1057 -1061, 2016. [ DOI]
Using Deep Convolutional Neural Network Architectures for Automated Object Detection and Classification within X-ray Baggage Security Imagery (S. Akcay, M.E Kundegorski, C.G. Willcocks, T.P. Breckon), In IEEE Transactions on Information Forensics & Security, IEEE, Volume 13, No. 9, pp. 2203-2215, 2018. [ DOI]
On the Relevance of Denoising and Artefact Reduction in 3D Segmentation and Classification within Complex Computed Tomography Imagery (A. Mouton, T.P. Breckon), In J. of X-Ray Science and Technology, IOS Press, Volume 27, No. 1, pp. 51-72, 2019. [ DOI]
A Reference Architecture for Plausible Threat Image Projection (TIP) Within 3D X-ray Computed Tomography Volumes (Wang, Q., N. Megherbi, T.P. Breckon), In J. of X-Ray Science and Technology, IOS Press, Volume 28, No. 3, pp. 507-526, 2020 [ DOI]
Papers were peer reviewed as part of the publication process and show clear originality / rigour.
4. Details of the impact
X-ray, and more recently CT-based, security screening is used in both transport and border security for the detection of explosives, weapons and contraband for the purposes of both law enforcement and counter terrorism. Whilst the regulatory mandate for this technology in aviation and border security is a policy matter for government, the technical development of security scanners to meet these regulatory requirements is carried out by commercial suppliers. The research work at Durham [R1-R6] has had both policy and commercial impact.
Impact on Government Policy: Based on this research at Durham, Professor Breckon has:
advised the UK government in his role as a member of the Cabinet Office Cyber Experts Group, under the direction of the Civil Contingencies Secretariat reporting to the Chief Scientific Advisor for National Security, on aspects of computer technology within transport and border security (2015-present) [E1]. [REDACTED]
advised the UK Government Office of Science (2016-2017), reporting directly to the Chief Scientific Adviser (CSA) to HM Government, [REDACTED]
enabled the test and evaluation of reference detection algorithms from [R4] on classified X-ray imagery by scientists at the UK Home Office [REDACTED]
informed US Department of Homeland Security (US DHS) – Transportation Security Administration (TSA) [REDACTED]
Fig2: Exemplar object detection in X-ray image for threat detection (single-view, 2D–left) [R4].
Commercial Impact: Based on this research at Durham:
- Rapiscan Systems Ltd (UK) [REDACTED]
Conservative estimates put Rapiscan at a 40% market share of the global CT security scanner market (source: European Commission, DG Competition - Case M.8087). With the move to CT to meet the new security standards, this translates as a global reach of 1.64 billion passenger journeys per annum (based on: IATA statistics, 2018).
- Smiths Detection GmBH (Germany) [REDACTED] Conservative estimates put Smiths Detection at a 30% market share of the global CT security scanner market (source: European Commission, DG Competition - Case M.8087) – with the move to CT to meet the new security standards this translates as a global reach of 1.23 billion passenger journeys per annum (based on: IATA statistics, 2018).
Cosmonio Imaging BV (Netherlands) [REDACTED] “in collaboration with X-ray scanner manufacturer L3 Harris Technologies, [REDACTED] (resulting company income - GBP260k) [E7]. [REDACTED] (contributing to the acquisition of Cosmonio by Intel in 2020). Conservative estimates put L3 Harris as having at a 9% market share of the global X-ray security scanner market (source: X-Ray Screening Report, 360 Market Updates, 2019) - 369 million passenger journeys per annum (IATA statistics, 2018).
Gilardoni S.p.A. (Italy) has integrated an algorithm from the research of [R4] within their X-ray scanner product line resulting in an industry showcase at the UK Security Expo (2017) [REDACTED] “Today, detection is done by measuring the atomic number [of an object]. Now we want to include shape recognition [to be commercialised by Gilardoni] and integrated into its products ... [This] will give us much more precise detection ... [and] increase our market share by having better equipment than our competitors ... The market wants to see these ... upgrades as quickly as possible. .. Whilst all new Gilardoni equipment should carry the latest algorithm [from Durham], there is also a strong possibility that it will be retrofitable on legacy systems.” Andrea Rotta, product specialist Gilardoni [E8].
Kromek Ltd (UK) [REDACTED]
Micro-X Ltd (Australia) [REDACTED]
VisionMetric Ltd (UK) [REDACTED]
Gatwick Airport (UK) [REDACTED]
Battelle (USA) [REDACTED]
Summary: Research from the Durham team [R1-R6] has significantly impacted government regulatory policy in the US and UK (representing the 1st and 4th largest national passenger volumes - source: World Bank, 2018), directly helps secure 500+ million passenger journeys annually and enjoys global commercial impact across nine businesses that includes commercial reach to over 70% of the market – securing up to 2-3 billion passenger journeys annually.
5. Sources to corroborate the impact
Letter - Chief Scientific Adviser for National Security, HM Government, 9 Feb. 2015.
Letter - [REDACTED]
Testimonial – [REDACTED] DSTL, HM Government, Feb. 2020.
Testimonial – [REDACTED] US DHS, Dec. 2020 & TSA - Report to US Congress, 2019.
Testimonial – Rapiscan Systems Ltd, [REDACTED] 2020
Testimonial – Smiths Detection, [REDACTED]
Testimonial – Cosmonio Ltd, [REDACTED] (Feb. 2020) and CEO + L3Harris presentation [REDACTED] (2018).
“Automation accelerates the checkpoint - Electronics screening research blends with machine learning for improved accuracy and efficiency”, Jane’s Airport Review (2018, 30(1)).
Testimonial – Kromek Ltd, [REDACTED] Feb. 2020
Testimonial – Micro-X Ltd, [REDACTED], Feb. 2020
Testimonial – VisionMetric Ltd, [REDACTED] Feb. 2020
Testimonial – Gatwick Airport, [REDACTED] Sept. 2020
Testimonial – Battelle, [REDACTED] Dec. 2020
Additional contextual information
Grant funding
| Grant number | Value of grant |
|---|---|
| DSTL DSTLX1000123869 / ACC106998 | £1,200,000 |
| Rapiscan Systems Ltd PO 109792 | £0 |
| Centre for Defence Enterprise DSTLX1000133302 | £0 |
| Department for Transport ACC6007893 | £0 |
| U.S. Department of Homeland Security 505118 – 78052 | £0 |
| Home Office 77302-507152 | £0 |
| Centre for Defence Enterprise ACC500173 | £0 |
| Home Office CST-HOS/15/051 | £0 |
| Department for Transport T-TRIG 2016 | £0 |
| Home Office SST/1617/005-09 / SBRI_HO-297_007 | £0 |
| DSTL – 1000085877 | £0 |
| Home Office IRC2013 – 38119-271233 | £0 |
| Micro X 18-01071 | £0 |