Results and submissions : REF 2021

Enabling Better Mobile Learning for Digital Skills Development

Submitting institution: The University of Huddersfield
Unit of assessment: 11 - Computer Science and Informatics
Summary impact type: Societal
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

Challenges in access to education, and the well-publicized digital skills gap, call for innovative methods and supporting technology to promote education. Research at the University of Huddersfield (UoH) has led to two innovations: WRS, a communication device to give rapid feedback to learners, and iDEA, a learning platform based on sound technology acceptance principles. Together, they have influenced changes in best practice for government and professional bodies, revolutionized professional standards and training, changed educational practice in primary, secondary and higher education, and encouraged continuing personal and professional development and lifelong learning. This impact has been felt by millions of users covering over 95% of the world’s countries, delivering worldwide impact in educational attainment.

2. Underpinning research

The success of individuals and the societies in which they live, depends on education. This is particularly true in the field of IT, where there is a well-publicized digital skills gap. This is exacerbated by the fact that at Key Stage 4 only 4% of girls rank IT first in terms of an enjoyable subject, compared to 17% of boys. This has resulted in women being only 20% of the STEM workforce, even though, in the UK, digital skills are required for 82% of online job vacancies [S1].

Digital skills gaps are often discussed in terms of a set of 21^st-Century skills that describe the knowledge, skills and behaviours required for success in today’s multi-cultural society. A better link is required between 21^st century skills development and employment needs, especially in Computer Science [S2] and, globally, millions lose out on learning because they cannot access formal education.

Mobile learning, whereby learners access the curriculum from their phone or tablet, can offer a potential solution to these challenges by mixing formal and informal teaching methods. However, the learning materials must be structured such that they provide the skills needed by industry, satisfy the requirements of exam boards and effectively engage students to learn and retain what they have learned. This requires learning tools that are multilingual and culturally sensitive.

In 2020, more than 1.3bn children globally had their education impacted by Covid-19, due to school closures. A mature mobile learning infrastructure would have reduced this impact. Research at the University of Huddersfield (UoH) has focused on identifying and testing ways to solve the challenges of developing such systems for mobile learning and increasing their adoption.

The research in this case study was performed by the Technology Acceptance and Mobile Learning (TAML) research group at UoH. It was led by Prof. Rupert Ward (at UoH since 2005), and Prof. Joan Lu (at UoH since 2000). TAML’s research is pitched at two levels:

Work Surrounding the Underlying Mobile Learning Platform

Professor Lu started her research over 15 years ago, creating a Wireless Response System (WRS) which provided rapid feedback to schoolroom pupils, communicating answers during lessons. WRS are affordable, efficient and accessible and can be adapted to diverse learning contexts. However, they suffered from performance and user acceptance issues and in 2015, Lu undertook research to improve the operational efficiency of wi-fi using fast data streaming technology. The initial application was in smart homes [R1], where they demonstrated its use anytime and anywhere across multi-session and multi-user configurations. This was applied to speed up the data-acquisition element of WRS.

To maximize access to mobile learning globally, language translation needed to be optimized. In 2016, the researchers in TAML investigated the key multilingual WRS access issues, which were found to be character sets, operating systems, user interfaces, formatting, and culturally appropriate content [R2]. The WRS was applied within English primary education [R3] (2018). Two schools in West Yorkshire, were provided with WRS technology (small tablet computers) and teachers were trained to use them. Feedback from heads, teachers and pupils was used to analyse the level of pupil interest, engagement and understanding.

Work Surrounding the Technology Acceptance Model

Professor Ward’s work on mobile learning adoption began in 2011, when he investigated how to make mobile learning easier for the student, making reflective learning tasks undertaken by university students more effective and engaging, as they built their personal development portfolios. He developed a mobile application (app) that organized student learning activities within an e-portfolio and could be accessed at any time.

Ahmed and Ward (2015) explored a range of technology acceptance models, to identify ways that the app could be improved. The research highlighted that the current models were too generic and inadequate when applied to e-learning. In 2016, their investigations focused on improving the adoption of e-learning. A meta-analysis of the existing technology acceptance models identified over 150 possible acceptance factors. From these, five key factors were synthesized and codified in the General Extended Technology Acceptance Model for E-Learning (GETAMEL) [R4]. The factors were: self-efficacy (confidence), subjective norm (socially conforming), enjoyment, computer anxiety and experience. The model was then tested with students using the e-portfolio solution [R5].

The individual factors in GETAMEL enabled mobile-learning solutions to be customized so that technology acceptance barriers to learning were lowered and intention to use mobile-learning solutions increased. An initial e-portfolio test showed that the five factors identified in this model accounted for 58% of behavioural intention to use [R5]. The factors therefore have become key determinants of effectiveness and adoption online when there is a lack of direct human engagement.

Between 2015 and 2017, Ward was seconded to the Royal Household, where he applied the GETAMEL factors to the development of a mobile learning platform for a charitable foundation (iDEA idea.org.uk/). The platform hosted the learning materials needed to develop a 21^st century skills e-portfolio. Interactive digital badges (validated indicators of accomplishment or skill) and an associated awards structure were developed. iDEA was designed to promote confidence, social learning, enjoyment and experiential learning, and to reduce anxiety [R6].

3. References to the research

Case for quality of supporting publications: [R1], [R4] & [R5] are published in Scimago-ranked Q1 journals, and two ([R1,R4]) are being submitted as REF outputs. MOBILITY is a peer reviewed, international conference for mobile learning [R2]. IJEEE is an open access international journal [R3]. Ward’s book was commissioned by Emerald Publishing [R6].

[NB Joan Lu is the anglicised name of Zhongyu Lu]

[R1] Meng, Z., and Lu, J. (2015). A Rule-based Service Customization Strategy for Smart Home Context-Aware Automation, 28 Apr 2015, In : IEEE Transactions on Mobile Computing. 15, 3, p. 558–571 14 p., 7097069. https://doi.org/10.1109/TMC.2015.2424427

[R2] Ali, A. and Lu, Z., Internationalization Testing for a Multilingual Based Wireless Response System: A Case Study. MOBILITY 2016 The Sixth International Conference on Mobile Services, Resources and Users (pp. 7–12). International Academy, Research, and Industry Association (IARIA), (MOBILITY Conference on Mobile Services, Resources, and Users). https://www.thinkmind.org/index.php?view=article&articleid=mobility_2016_1_20_70014

[R3] Joan Lu, Mike Joy, Gail Newton, James Robert, Mohammed Yousef, Simon McKenna, Use of a Student Response System in Primary Schools — An Empirical Study, International Journal of e-Education, e-Business, e-Management and e-Learning, IJEEEE 2019 Vol.9(4): 324–330 ISSN: 2010-3654, http://www.ijeeee.org/show-77-915-1.html#

[ R4] Abdullah, F., and Ward, R. (2016). Developing a General Extended Technology Acceptance Model for E-Learning (GETAMEL) by analysing commonly used external factors. Computers in Human Behavior, 56, 238–256. https://doi.org/10.1016/j.chb.2015.11.036

[R5] Abdullah, F., Ward, R., and Ahmed, E. (2016). Investigating the influence of the most commonly used external variables of TAM on students’ Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) of e-portfolios. Computers in Human Behavior, 63, 75–90. https://doi.org/10.1016/j.chb.2016.05.014

[R6] Ward, R. (8^th June, 2020). Personalised Learning for the Learning Person. Emerald Publishing Limited. ISBN 978-1-789731507 (available on request)

4. Details of the impact

The research findings in section 2 were translated into the two technologies:

(i) The Wireless Response System (WRS), developed by Professor Lu, is a mobile learning tool to measure learning behaviour and quantify understanding. It improves on the state of the art, in that it does not use specialist equipment (it can use a wide range of existing devices), it is multi-lingual and multi-discipline, and its online implementation uses a range of web services to ensure secure local or remote connection between tutor and learner.

(ii) Professor Ward led the iDEA.org.uk project, a digital skills portfolio learning portal whose design is based on the five GETAMEL factors [R4], with the objective of optimizing adoption and engagement through self-directed mobile learning. Ward carried out most of the development during his Royal Household secondment (2015–17), creating an online environment where learners could gain digital badges and awards through bite-sized personalized learning. The initial focus of iDEA, was to enable the development of skills necessary for employment, especially in the digital and computer science sectors.

To summarise the impact, these technologies have benefited educational institutions, learners (Key Stage 2 to adult), local and national governments, employers and employees, via practitioners, professional services delivery and performance enhancement. The detailed impacts are grouped under four headings:

Encouraging Continuing Personal and Professional Development and Lifelong Learning

The use of WRS [R2] has increased intercultural awareness through its 300,000 users worldwide, from more than 100 countries. It supports five languages (Russian, Polish, Arabic, Chinese and English) and six disciplines (engineering, languages, law, maths, cyber-security and database design). The system means students are prepared to learn by making mistakes. A lecturer in Shanghai said: “The students can freely express their individual opinion because the system is designed anonymously [..] they do not worry in case they have made mistakes” [S6].

iDEA has been widely adopted by the personal and professional development programmes for UK job centres, prisons, local councils, schools and universities. Since its launch in 2017, over 7 million iDEA digital badges have been completed in over 190 countries [S7]. Talking about the breadth of the iDEA user base, the Special Adviser to the City & Guilds Group stated: “in coordination with the Armed Forces, the Department for Work and Pensions (DWP) and Jobcentre Plus (JCP) Youth Obligation programme, iDEA training is [enabling] learners to move from the benefit system into employment, [..] with the job centre initiative alone providing free and highly accessible training for 1.6M learners across the UK”. He continued: “iDEA’s uptake has been phenomenal, with more than six million badges completed to date. This success, and iDEA’s impact more generally across the world, have been because it [..] is so fit for purpose” [S7]. The philosophy of iDEA is discussed in detail in a book by Ward, which has a foreword by the OECD’s Director for Education and Skills: “Learning systems need to better recognise that individuals learn differently, and in different ways at different stages of their lives. They need to create new ways of providing education that take learning to the learner and that are most conducive to students’ progress. Learning is not a place, but an activity. [..] Future learning systems need to use the potential of technologies to [..] connect learners in new and powerful ways, with sources of knowledge, with innovative applications and with one another” [R6].

Changing Educational Practice in Primary, Secondary and Higher Education

Testing of the WRS [R2] demonstrated improved learning progress and subsequent academic achievement. The immediate feedback on student understanding provided by the WRS enabled teachers to monitor progress in real-time and more easily analyse learning barriers, especially in bilingual contexts. Using the information, they adjusted what they taught and how they optimized the overall learning experience. For instance, in Wuhan, China, its use has improved student performance and maintained their employability during the 2020 pandemic. The Course Leader, School of Computer Science and Engineering, Wuhan Institute of Technology said: “The use of WRS improves student performance and the grade average of our courses. This year, despite the Covid-19 period, through the use of the WRS system, students still maintain a high degree of enthusiasm for learning. Compared with previous years, we have performed well, and the employment rate of graduates still reached 85%. WRS has made a good contribution to our success and achievements” [S8].

iDEA has improved education in UK primary and secondary schools, by influencing and enabling changes to the curriculum. It was used by Computing at Schools, a charity supported by the professional body for computing, known as BCS the Chartered Institute for IT, that works to improve the quality of computing education. iDEA has also enabled gender equality; the scheme Code: First Girls has supported 25,000 female coders since 2017. Internationally it was adopted by the Council of British International Schools, iamtheCODE (an African-led scheme to encourage marginalized girls to study STEM subjects) and the British Council (2017 onwards) [S9]. A Programme Manager at iDEA wrote: “iDEA has changed learning. In formal education, it is used both to teach the curriculum in Key Stage 2 and 3, as well as to inspire and motivate digital skills development outside of the classroom.[..] iDEA’s overall audience is 52% female this year, reflecting its cross-gender appeal and helping to break down gender barriers in the tech sector. Internationally, it is used in more than 95% of all countries”. Commenting on the unusual circumstances of 2020, she added: “at the start of the UK lockdown, more than 1M additional iDEA badges were completed in less than five weeks as schools transitioned rapidly to mobile learning” [S9].

Defining Best Practice for Government and Professional Bodies

iDEA was cited as best practice by the UK Government in a DCMS report (2016), which stated: “Third sector organisations have particular roles to play at various levels: for example, support for digital skills development in young people is being provided through iDEA” [S3]. The Royal Society, in a 2017 report on Computing Education in UK Schools, commented: “iDEA is an innovative Badge Store concept that helps people develop skills for free [..] Badges have been mapped against the National Curriculum and the Skills Framework for the Information Age. This helps support teachers across a range of curriculum subjects including the three core areas of formal computing education: digital literacy, computer science and IT” [S4].

The Greater Manchester (GM) Digital Strategy (2018) identified iDEA as a means of increasing digital inclusion. It decided to “Roll out the iDEA digital enterprise award across GM so that we have an easily accessible free way for any young person or adult across GM to develop digital skills for life and work”. [S5] This factor was underlined by the description of iDEA as “the digital and enterprise equivalent of the Duke of Edinburgh Award” by the Royal Society in 2017 [S4].

Revolutionizing Professional Standards and Training

The application of iDEA within the Computing at Schools scheme and the Key Stage 3 National Curriculum in the UK, demonstrated how digital badging can underpin existing qualifications by providing a granular skills profile to an employer, rather than an opaque traditional qualification syllabus. The work was applied to higher education through the development of a 21^st Century digital skills taxonomy and framework for the mini-qualifications (microcredentials) that constitute a full, traditional qualification. The framework informed the review of the new BCS computing accreditation standard for universities. It was also used in developing a global microcredentialing standard for the cross-professional body, International Council on Badges and Credentials (ICoBC). Its impact on the BCS standard is in increasing transparency within university degrees by better communicating digital skills acquisition to employers, lecturers and learners. It has also been adopted by Manchester Metropolitan University for their international computing degrees. Wider international adoption has increased skills transferability globally, making it easier to communicate capabilities across subjects, sectors and economies. This has enabled governments, the OECD and other inter-governmental organizations to better analyse and address global digital skills gaps [S10]. The President of the BCS (2017–2018), stated: “Rupert’s research, [..] has highlighted to the BCS review group an additional important strand for consideration within its accreditation review. Rupert is representing the BCS in discussions with the ICoBC, in parallel with the accreditation review, to develop an international recognised standard through a digital skills taxonomy (an agreed categorisation of skills) and a microcredentialing framework (a way of linking acquired skills to qualifications) [..] Rupert’s work is significant input to the inclusion of informal learning, something which has profound implications for skills development both within the BCS and within other professional bodies” [S10]. Without the novelty demonstrated in identifying and testing the five factors in the GETAMEL model and applying these to iDEA.org.uk, the technology would not have been accepted, the intention to use would not have been as high as it has been, and the global ongoing reach and significance both in numbers and breadth would not have occurred.

Enabling Social and Commercial Impact from Innovations in Visual Computing

Submitting institution: The University of Huddersfield
Unit of assessment: 11 - Computer Science and Informatics
Summary impact type: Technological
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

Computer vision, visualization and machine learning are state of the art techniques that require enormous computing power. Researchers at the University of Huddersfield have accelerated their practical application by developing techniques that enhanced data processing capability, thus enabling more data sources to be used, the data to be processed more rapidly and more knowledge to be derived.

Key impacts resulting from the underpinning research were as follows:

The ShenZhen Public Security Department in China installed software that removed the need for human operators for 9,712 CCTV cameras, thus saving £8m.
The global brand, Phoenix Bicycle was able to update its factories in China to be greener and more efficient, which also opened up new markets, using new computer vision techniques.
A Chinese supplier of augmented reality-based wearable devices for the visually impaired exploited UoH patents and grew their customer base to 80,000 whilst greatly improving the quality of life for users.

2. Underpinning research

Advances in the overlapping spheres of engineering and computing have been increasingly driven by developments in highly specialised fields such as computer vision (where computers replace the human eye), visualization (computer graphics, animation, and virtual reality), machine learning (where systems improve automatically) and the software and systems strategies associated with them. Practical applications have multiple input streams containing data that must be analysed, combined, and interpreted. A key challenge is ensuring that the data processing burden is not so high that the system cannot be utilized in real-time. Research at the University of Huddersfield (UoH) developed techniques that improved the speed of data processing, which were applied practically in field of visual and immersive computing.

The research described in this case study was carried out at the Centre for Visual and Immersive Computing (CVIC) at UoH. It was led by Professor Zhijie Xu (Director of the CVIC, at UOH since 1999). Other members of the team who aided the impact building included, in surveillance and security (Dr Jing Wang), automation and machine vision (Professor Paul Scott, an associate member of CVIC), and health technology (Dr Duke Gledhill). The work focused on three interlinked strands of expertise. New algorithms and techniques for real-time CCTV analysis; smart factory automation based on machine vision; and improvements in data handling techniques in Augmented Reality (AR) product design used in health products for the visually impaired.

Devising New Algorithms and Techniques for Real-Time CCTV Analysis

Globally, huge amounts of CCTV data are acquired daily by sources such as security cameras (in the public realm), private enterprises such as Tube and Dashcams for insurance firms, and private homes. The efficient processing of this data is now critical in numerous aspects of daily life, not least due to the heightened threat of terrorism and the consequent need for effective surveillance and crowd control.

The research focused on how to use machine learning to enable data from a video camera, to replicate the decisions of a human operator. A foundational study developed a system that used high dimensional information (additional data streams) to identify individual behaviors by constructing compact spatial and temporal signal representations. The techniques learned were further developed and applied to enable real-time intelligent monitoring and early warning of emergencies in large-scale networks monitoring crowds.

The research team devised a representational model of the environment and a corresponding scheme to code the scenario, based on category theory (which enabled disparate data sources to be compared and interpreted) [3.1]. In the model, ‘higher’ dimensional information such as individual pedestrian ‘actions’, crowd movement and the nature of the surrounding environment, were used to construct a ‘global’ model in which a subject’s movements in three-dimensional space were inferred by combining multiple 2-D images from CCTV cameras, and then geometrically represented as a whole. This enabled the individuals to be monitored as they moved, including from camera to camera.

When implemented, the model automatically updated and recognized changes in a subject’s behavior, motion, and context (environment) by applying a feature map fast vector coding scheme (which ensures features in images can rapidly be identified). The new model used advancements in Artificial Neural Networks (created via so-called deep learning) combined with UoH-developed heterogeneous scalable neural networks (again designed to accelerate data analysis) called the Treble Stream Semantic Network [3.2]. These were used to resolve the long-standing challenge of accurate delicate (subtle) feature (individual item of interest) selection. It was used to process and classify complex crowd scenarios and enabled the recognition of changes in the behavior of an individual relative to their surroundings.

Developing Smart Factory Automation Based on Information Visualization and Machine Vision

Virtual Reality is an important platform for simulating and designing manufacturing systems. Its real-world adoption was slow because complex manufacturing scenarios had to be treated as unique, meaning that each one was individually modelled in the VR environment. This made it particularly expensive to model large scale and complex manufacturing environments, such as assembly lines. The research solved the problem with a novel approach, a top-down virtual environment construction model [3.3], which enabled, for the first time, a modular approach to building complex VR simulations. A software system which separated the “look” (models) and “feel” (GUI and programme functions) of the system, was designed and implemented. A general case was created, which was then applied to multiple manufacturing scenarios, resulting in reduced build time. The model incorporated practical processes that were leading-edge in automated manufacturing at the time e.g., flexible manufacturing concepts such as cells, in a practical manner that led to optimised scheduling, reduced lead time and logistical cost saving.

The research [3.4] explored the whole manufacturing process from design to implementation on the shop floor. It used visualization techniques such as single view and stereo vision to improve the performance of robotics equipment, such as visual alignment for robot guidance (which improved accuracy in part placement), 3D detection (which enabled a robot arm to pick up parts) and parts measurement (used for quality control). Parallel computing hardware acceleration strategies (based on adapting commercially available ‘gaming’ graphics cards) were devised. These transformed algorithm-acceleration approaches, originally developed for the software arena, to the computer hardware. Consequently, the algorithms ran faster. The faster online processing delivered, meant it was possible to incorporate deep learning capability (i.e., an artificial neural network trained model that runs in real-time).

Deploying Intellectual Property in Augmented Reality (AR) Product Design for Global Visually Impaired Population

Scene recognition establishes the context of the objects in a ‘view’ (a practical example of which is an ever-changing streetscape). It plays a crucial role in robotic navigation and localization, which is used in assistive technology, especially for the visually impaired. Current scene recognition techniques were prone to error, due to issues with their inability to distinguish individual features and a weakness in template matching (which is used to identify specific elements of ill-defined objects). The research team developed algorithms described in international patents [3.5], which improved i). the integrity of data supplied from multiple sensors (e.g. IR, UV, ultra sound etc.) and provided better signal quality for signal fusion (2016), and ii). an open-source scene recognition programming API [3.6, 5.10], that was made available for programmers to exploit on the IOS and Android platforms. It has transformed practice in the challenging domain of real-time scene recognition by synthesizing data from multiple-input sources in real-time.

New developments included improved differentiation between passable areas and new obstacles (for instance between a wall and a staircase), and recognition of traffic lights and zebra crossings. The approach improved performance versus state-of-the-art benchmarks in the acquisition of close-range depth data. This used laser pulses in a similar fashion to sonar, to create a depth ‘map’ which was fused with other data sources, such as polarized infra-red (which allowed differentiation between a wall and a glass door). The closest detectable range for the sensors was reduced from 650mm to 165mm with an accuracy of 95%.

3. References to the research

The main outputs in Section 2 have been disseminated through multiple channels, including publishing in top research journals, workshops for industrial and societal stakeholders, invited speeches in prestigious forums and conferences, and leading bids and projects with international partners. References below include high-impact papers, international patents, in-house developed open source toolkits for industry, and Apps for health product users:

Wang, J., & Xu, Z. (2016). Spatio-temporal texture modelling for real-time crowd anomaly detection. Computer Vision and Image Understanding, 144, 177-187. https://doi.org/10.1016/j.cviu.2015.08.010
Xu, Y., Lu, L., Xu, Z., Wang, J., Huang, J., & Lu, J. (2018). Towards Intelligent Crowd Behaviour Understanding through the STFD Descriptor Exploration. Sensing and Imaging, 19(17). https://doi.org/10.1007/s11220-018-0201-3
Xu, Z., Zhao, Z., and Baines, R. W. (2000) Constructing virtual environments for manufacturing simulation. International Journal of Production Research, 38 (17). pp. 4171-4191. https://doi.org/10.1080/00207540050205000 [can be supplied on request]
Xu, Y., Xu, Z., Jiang, X., and, Scott, P. (2011). Developing a knowledge-based system for complex geometrical product specification (GPS) data manipulation. Knowledge-Based Systems, Elsevier. 24(1), https://doi.org/10.1016/j.knosys.2010.05.002
Patent 1: 201811436184.0 A glass detection device and corresponding methodology based on fusion of RGB-D and ultrasonic sensors; and, Patent 2: 201811436208.2 A scene representation device and methodology based on semantic stixel. https://bit.ly/3qUERYQ [Patent applications available on request]
Development API and Apps https://a.app.qq.com/o/simple.jsp?pkgname=cn.krvision.litekrnavi; and, Software API: OpenMPR - the open-sourced software for place recognition using multiple descriptors derived from multi-modal images: https://github.com/chengricky/OpenMultiPR

4. Details of the impact

The research findings led to commercial impacts in three organisations in China. The impacts can be summarized under three headings: 1. New forensic imaging technologies for law enforcement; 2. Enabling automation in a bicycle factory; 3. Commercialization of wearable devices for visually impaired individuals.

New Forensic Imaging Technologies for Law Enforcement

CCTV is a pervasive tool for monitoring crowd behaviour and has had proven success in enabling the detection of suspicious individuals, such as terrorists and alerting the authorities to risks to individuals in large crowds. Each camera, however, needs to be monitored by a human operator, which introduces a non-scalable cost factor.

Forensic imaging scientists at the Shaanxi Electronic Information Scene Investigation (EISI) in China collaborated with the UoH CVIC since 2010. One project [5.1] resulted in the creation of a real-time crowd behaviour analysis capability [3.1] (2016). It was trialled as a live system in the southern Chinese city of ShenZhen and significantly reduced the number of people required for continuous (24-hour) real-time monitoring. It also improved the emergency response speed for public emergencies and reduced loss of life and damage to property [5.3].

The technology [3.2] and IPs [5.2] were adopted by policing and public security bureaus in eight provinces in China [5.4]. In one example, the system has been installed in more than 640 sites (such as streets, railway stations and parks) in Shenzhen City, Guangdong Province, with 9,712 video cameras in the network. Since 2016 more than 22,000 abnormal emergencies have been identified. Multiple police, traffic control and crime prevention agencies provided positive feedback on the pilot system, which delivered an estimated cost-saving to the public purse of 70 million RMB dollar (£8m) since 2016. Significantly, it is estimated that in the Sichuan Province alone, the technology saved over 3,000 security monitoring personnel. Based on the average cost of a police officer per year, the technology has reduced costs by up to 240 million RMB dollar (£27.5m) annually for the provincial public security and police departments in Sichuan. The Police Superintendent stated “… the crowd monitoring technology significantly reduced the cost of police resources including time, workload, and public grant” [5.3].

Enabling Automation in a Bicycle Factory

The China-based Shanghai Phoenix Company Ltd. produced 4.89 million bicycles in 2019 and exported to more than 50 countries. In 2000 the manufacturing practices of the company were similar to those seen in Europe in the 1920s. Since then, the company has collaborated with various global research institutions to accelerate the modernisation of its production lines. The CVIC at UoH provided research and consultancy-based support since 2010, focused on the design and development of an efficient shop floor by utilizing virtual manufacturing-based planning and simulation [3.3]. This enabled Phoenix to modernize its production operations to match current global best practice in just a decade, becoming more energy efficient and cost effective [5.5].

The CEO of Phoenix stated “Prof. Xu’s expertise in industrial design and automation has greatly reduced the lead time to modernise and operate complex manufacturing and production environments. Substantial cost saving from labour and energy of up to 15 percent have been recorded in the past 3 years. The increased production capacity has also enabled the strategic redirection of the product type towards more green lines. For example, the partnership with the Shared Bicycle scheme (OfO Ltd.) exported Phoenix products to countries including France, United States and United Kingdom” [5.6].

Visual guidance systems were jointly developed with the sector leading Second2None Machine Vision Systems Co. Ltd [3.4]. These included the ‘Long-Rui’ series of smart vision sensors, which were applied to a series of vision appliances used for robot guidance, 3-D detection, visual alignment, and welding control in two of Phoenix’s plants. The company has been able to sell the new product to Epsom and Yamaha. It has grown for five years in a row and was named a Star Company in Shenzhen City [5.7].

Commercialization of Wearable Devices for Visually Impaired Individuals

The research facilitated ShiKe (KrVision) Ltd to commercialize wearables and headsets for the visually impaired in 2016. It exploited patented technologies [3.5] [5.8] and the open source API [3.6]. The product featured cutting-edge functionality in scene localization, video augmentation and sign and text recognition, plus the capability to render the information synthesised from the data in audio form. It could remember a regular route followed by the wearer, which was used as a template that enabled greatly improved identification of new obstacles. In addition, the device expanded the ‘visual’ range of a totally blind person from the ‘point in space’ defined by a ‘white stick’, to a three-dimensional hemisphere that radiated from the front face of the wearable device. The founder of the company stated, “… the collaboration with the CVIC at UK has greatly accelerated the release of our new AR kit” for the visually impaired community [5.9]. He added “The improved functions and new features on audio coding and feedback, semantic segmentation, street water pool and sinkhole identification have greatly benefitted the users and could not have been implemented so successfully without the collaboration with our international partners”. As a result of the joint effort, the company has grown from an SME in 2014 into a medium sized enterprise with a customer base of over 80,000 users across world [5.9].

A visually impaired user of the wearable device said “My life has been utterly transformed by my new headset. I can now move about more easily outside my home, confident that I will have a better warning of any obstacles I need to avoid” [5.8, 5.10].

New AI Technologies for Enriching and Recovering Value from Urban Data with Applications ...

Submitting institution: The University of Huddersfield
Unit of assessment: 11 - Computer Science and Informatics
Summary impact type: Technological
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

The Internet of Things (IoT) offers great potential because of the tremendous volume of data that can be harvested from the instruments and sensors connected to it. However, because the data collected is in different formats it needs to be normalized before it can be combined and put to useful purposes, such as driving AI-enabled decision engines. Researchers have developed tools to normalize the data rapidly enough for information from multiple sources to be used in real-time; and have discovered new methods that input this data and scale up to a city region on-the-fly generation of goal-directed traffic-management strategies. IoT technology generated from this research has been used by BT in 20 collaborative projects involving industrial and public organisations. Development of software to automate the new methods of generating goal-directed traffic-management strategies has led to the creation of a spin-out company and enabled a change in professional practice by urban planners.

2. Underpinning research

Urban administrators of the future will be charged with enabling a thriving economy, while delivering a good quality of life to citizens and reducing their environmental footprint. There is an opportunity for Artificial Intelligence (AI) techniques to help in this respect, fed by data covering a large variety of the variables that exist in the urban environment (e.g. pollution, weather, traffic concentration). The data sources largely exist already, in the form of sensors that have already been installed for various functions across towns and cities. But the data produced is in a variety of proprietary formats which mean it is not simple to use the outputs for other purposes. Data hubs have been developed to act as “clearing houses”, within which the data can be normalized and combined for analysis. However, there are significant challenges associated with the extract, normalize and combine process, especially if the environmental data is to be utilized in real-time.

Traffic management is one element of urban planning that could benefit from an AI-based approach. It provides a good example of the sorts of challenges to be overcome and the solutions that can be delivered. Transport operators working in urban environments need flexibility within their traffic management systems to cope with infrequent but disruptive, unusual events. Some of these are planned (such as roadworks) and some are unexpected (such as traffic accidents). They also have to achieve increasingly challenging environmental and economic goals, such as reducing pollution and journey times, which are difficult to attain if traffic management approaches are not holistic across a whole city.

Historically, traffic management could only be carried out at a very local level (e.g. at a single or small group of junctions). Modern Internet of Things (IoT) technologies offer the prospect of collecting data from the multiple sensors found in the urban environment and using them to feed AI-enabled decision-making algorithms. Researchers from the University of Huddersfield (UoH) have developed techniques and tools that enable the data from urban sensors to be normalized, combined, and interpreted within the IoT. This enabled improvements to a data hub from which the data fed AI tools that produced actionable recommendations. An example that was explored in depth, is real-time traffic management planning.

The research was undertaken at University of Huddersfield (UoH) by Prof Lee McCluskey (at UoH since 1993) alongside Prof G Antoniou (at UoH since 2011), with Dr I Tachmazidis (Research Fellow since 2012, then lecturer) and Dr M. Vallati (Research Fellow since 2012, then senior lecturer and reader). In 2014, McCluskey and Antoniou joined their research groups to form the PARK (Planning, Autonomy and Representation of Knowledge) Centre.

In 2014, British Telecom’s (BT) IoT Data Exchange, a software tool used for collecting urban data from multiple sensors, could only store data in a low-level format. This meant that the data could only be used by the device for which it was collected. Antoniou, Tachmazidis and Batsakis, semantic web experts, obtained support from two research contracts with BT [Sem-Spine 1 £57k (2015-2016), Sem-Spine 2 £30k (2017-2018)] to design a way to store the data in a format that rendered it more accessible and easier to combine. They developed a technical solution that linked the data held in the Data Exchange with the Web of Data (a global location for storing meta-data) that enabled them to enrich the pieces of information with their contextual meaning. This is called semantic enrichment and it transformed the data stored on the BT system to be ubiquitous and interoperable.

At the same time McCluskey was leading a European network that explored the application of autonomic (self-managing) computing to transport in a COST (European Cooperation in Science and Technology) project with 24 country members [Network Action TUD1102]. The project led to the development of a new flow model of traffic movement. The traffic flow was modelled as if it were a liquid, rather than as discrete vehicles. This reduced the computing power needed and hence, increased the number of vehicles that could be modelled. The model was used with goal-directed automated planning software, in which specific targets were programmed-in as end points to be reached [1]. This meant that, for the first time, data on extremely large volumes of traffic (e.g. a complete city) could be collected, interrogated, and analysed to achieve strategic management goals.

In 2014 a transport consultant who had been a member of the COST Network, led a consortium which included the UoH PARK Centre, BT, and Transport for Greater Manchester (TfGM), which won funding [NERC ref NE/N007239/1; Innovate project ref 132029] from the Innovate Competition “Solving Urban Challenges with Data” (2015 - 2016). The PARK research team developed new high-level semantic languages and tools for the BT Data Exchange [2,6] and applied the resulting semantically enriched data in the traffic flow model. They then developed a method to apply existing AI planning techniques in a trial of an application to generate real time, road traffic signal strategies in the Manchester area [3,4].

KamFutures Ltd, a microbusiness, led another consortium which included TfGM and UoH’s PARK to make further successful bids [project ref 971481, project ref 971549] to the Government innovation agency, Innovate First of a Kind, scheme. The PARK research team, supported by research fellows Franco and Lindsay, discovered new ways to scale-up the generation of goal-directed traffic strategies. They used the semantically enriched data, and trials showed that strategies could be generated in real time. Traffic-signal timings which solved strategic goals such as a reduction in congestion, an optimized flow through roadworks and lower traffic pollution within large urban areas [3,5] were produced and shown to work.

3. References to the research

Quality of outputs: 1.2,4,5,6 are papers from CORE-ranked A or A* outlets, leading conferences in their fields, and strictly peer-reviewed. 3 is published in a Scimago Q1 international journal with an H index of 100.

Vallati, M., Magazzeni, D., De Schutter, B., Chrpa, L. and McCluskey, T.L. (2016) “Efficient Macroscopic Urban Traffic Models for Reducing Congestion: a PDDL+ Planning Approach”. In: Proceedings of the Thirtieth AAAI, Phoenix, Arizona USA: AAAI Press. pp. 3188-3194. https://dl.acm.org/doi/10.5555/3016100.3016349
Ilias Tachmazidis, Sotiris Batsakis, John Davies, Alistair Duke, Mauro Vallati, Grigoris Antoniou, Sandra Stincic Clarke: “A Hypercat-Enabled Semantic Internet of Things Data Hub”. The Semantic Web - 14th International Conference, ESWC 2017, Portorož, Slovenia, May 28 - June 1, 2017, Proceedings, Part II. Lecture Notes in Computer Science 10250: 125-137. https://doi.org/10.1007/978-3-319-58451-5_9
Antoniou, G., Batsakis, S., Davies, J., Duke, A., McCluskey, T., Peytchev, E., Tachmazidis, I. & Vallati, M. “Enabling the use of a planning agent for urban traffic management via enriched and integrated urban data” Transportation Research Part C: Emerging Technologies. 98, p. 284-297, January 2019. https://doi.org/10.1016/j.trc.2018.12.005
McCluskey, T. and Vallati, M. (2017) ‘Embedding Automated Planning within Urban Traffic Management Operations’. In: Proceedings of the Twenty-Seventh International Conference on Automated Planning and Scheduling (ICAPS-17), AAAI Press. pp. 391-399 https://www.aaai.org/ocs/index.php/ICAPS/ICAPS17/paper/view/15645/15129
S Franco, A Lindsay, M Vallati, TL McCluskey – “An Innovative Heuristic for Planning-Based Urban Traffic Control”, International Conference on Computational Science, Springer, 2018. https://doi.org/10.1007/978-3-319-93698-7_14
Ilias Tachmazidis, Sotiris Batsakis, John Davies, Alistair Duke, Grigoris Antoniou, Sandra Stincic Clarke: “Optimizing a Semantically Enriched Hypercat-enabled Internet of Things Data Hub”. SSN@ISWC 2018: 64-71. http://ceur-ws.org/Vol-2213/paper6.pdf

4. Details of the impact

The research findings benefitted companies that wanted to exploit the Internet of Things (IoT) for urban planning purposes, including British Telecom (BT), urban planners in Manchester, a micro-business and a spin-out company.

The impact can be summarized under three headings:

Delivering a new technology
Creating a new business
Improving existing technologies and changing attitudes in Urban Traffic Management

Delivering a New Technology

The PARK team worked with BT (2015-18) to integrate semantic tools into its state-of-the-art MK (Milton Keynes) Data Exchange, which were designed to make the data collected from the IoT available to be combined with other sources and, thus, create added value.

Application of the UoH research made the Data Exchange open-data enabled. A BT Research Manager stated that it solved the problem of making semantic queries over many different data sources scalable, “with sufficient performance to address the needs of our smart city development” [B2]. He added, “The solution is now embedded in the BT Data Exchange which is helping to drive the commercial direction of BT to assist our customers” (2020).

The semantic component, alongside the AI-planning platform, was deployed in a smart city project called CityVerve, which set out to transform the city of Manchester into a demonstrator for IoT technologies. As the Head of IoT Research at BT wrote in 2020,

“Huddersfield’s research technology has helped us at BT make our Data Hub open data enabled; this allows interoperation with other IoT data collections as well the semantic enrichment of data held in our exchange. Their pioneering research led to important enhancements to our IoT data exchange” [B1].

The BT Research Manager confirmed that the BT Data Exchange has been used “in more than 20 collaborative projects involving industrial and public organisations” [B2]. The data is now hosted within public data exchanges such as the one for Milton Keynes [B6], which incorporates the UoH technology.

Creating a New Business

A Joint Venture (JV) company called Simplifai Systems was established between KamFutures Ltd (a micro-company that was a partner in the European projects) and UoH [B10] as a result of the IP generated from the series of grants (Innovate project refs 132029, 971481 and 971549, 2015-2018). McCluskey has acted as Research Lead since the company was founded. An international patent No: PCT/EP2020/050815, owned by the JV, was published in July 2020 that captured the intellectual property underpinning the AI strategy-generating tool.

The company produced advanced Urban Traffic Management (UTM) software that incorporated the strategy-generating tool and the data adaptors that acted as an interface for moving information in and out of the tool. It enabled users to generate urban region-wide traffic strategies that improved traffic distribution relative to an operator-supplied strategic goal, such as reducing congestion.

In December 2020, SimplifAI employed two full time software engineers to develop its UTM software. It had benefitted from £400,000 venture capital investment and £1.3m research grant investment. In 2018 the company obtained a £50,000 grant from TUSforge, the first Chinese business accelerator scheme in the UK, to explore Chinese markets for the company to access. Other recognition, for instance SimplifAI being a finalist at Pitch@Palace 9.0, representing the North of England technology sector at the World Economic Forum, recognition by computer weekly [B7], being shortlisted for Engage-Invest-Exploit 2019 and being recognised as a top innovator in the Leeds list [B9], helped the company to attract investors.

Improving Existing Technologies and Changing Attitudes in Urban Traffic Management

The semantic enrichment processes for diverse sources of data, and the novel hybrid automated planning representations, have led to a change in attitudes and understanding of what is possible in Urban Traffic Management (UTM). The CEO of SimplifAI Systems said, “Opportunities for providing transformative tool support in UTM have been provided to the Transport Engineering sector via the outcomes of the AI research carried out at the University of Huddersfield’s PARK Centre” [B5](2020). The AI goal-directed strategy-generating tool [3,5] was trialled with Transport for Greater Manchester (TfGM) in 2017. The Head of UTC at TfGM stated, “The research tools … demonstrated a unique advance in UTM tools, and changed our attitudes towards what was possible in future developments in operational procedures” [B3].

In March 2020, a new joint traffic control centre, set up by Kirklees Council, SimplifAI Systems and UoH, applied the AI tools to planning for the Kirklees region. The results transformed ideas of what was achievable in UTM, showing it was possible to create plans targeted at strategic goals, such as reductions in pollution levels and traffic congestion. The Group Engineer at Kirklees commented, “These plans can work towards certain strategic goals, whereas in the past this was not possible to do” [B4] (2020). He also confirmed how the research has changed the understanding of the planners on his team, in terms of what is possible, commenting “Before the research, we were not aware that goal-directed region wide strategies could be generated in real time to cope with emergencies or rapidly changing circumstances” [B4].

The CEO of SimplifAI [B5] and the Group Engineer of Kirklees Council [B4] both confirmed that the product had global reach, with key markets being China and the USA.

Impact case study database

Search and filter

Filter by

Enabling Better Mobile Learning for Digital Skills Development

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Enabling Social and Commercial Impact from Innovations in Visual Computing

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

New AI Technologies for Enriching and Recovering Value from Urban Data with Applications ...

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Filter by higher education institution

Filter by unit of assessment

Filter by continued case study

Filter by summary impact type

Filter by impact UK location

Filter by impact global location

Filter by underpinning research subject