Impact case study database

Novel image capture, processing and analysis tools and methodologies developed and optimised through ECS research have had a far-reaching impact on the way multimedia content is published and developed on the Web, the development of innovative ‘big data’ products and services for commercial exploitation, and the management of cultural heritage.

Impact of libVIPS/OpenIMAJ libraries on web publishing, software development and teaching

The libVIPS library is now a standard open source Linux package, which has been refined and optimised by an international web developer community and the ECS group itself over the impact period. Recognised as the fastest available image processing library, it is widely used to develop websites and systems that can handle very large numbers of images [ 5.1]. Use of libVIPS minimises cloud resource usage; a saving of >50% of CPU/RAM for each burst of image activity reduces server loads, delivers a more efficient service and negates the need for costly upgrades. The libVIPS source code repository on github has an average of 800 regular users and 3,000 unique visitors each month. But the full scale of its impact is realised through the use of libVIPS as a core driver of user-facing web applications and software tools. In 2014, libVIPS was used to create Sharp, a high-performance module for resizing and formatting large images to smaller, web-friendly JPEG/PNG images of varying dimensions [ 5.1]. Sharp acts as a ‘web wrapper’ in that it provides access to libVIPS for web developers by making it accessible to javascript. By the end of 2020, Sharp had reached 1,800,000 downloads per week [ 5.1], while 640 websites are known to use libVIPS via Gatsby, a content management system that helps developers build ultra-fast websites [ 5.1]. While it is not possible to quantify the full reach of libVIPS, the founder of Sharp wrote [ 5.1]: ‘ Over 23,000 open source projects rely on sharp, including most JavaScript-based content management systems. This means you’ve almost certainly visited a website in the last week that used libvips.’

The widespread use of libVIPS is demonstrated through end user case studies. In September 2013, VipsScaler, a wrapper to libVIPS, was rolled out across all Wikimedia and Wikipedia sites [ 5.2]. This allowed, for the first time, the creation of thumbnails of very large PNG files due to significant reductions in required memory [ 5.2]. Amazon Web Services (AWS) uses libVIPS for efficient image resizing within one its main applications, Lambda@Edge – a tool that allows developers to run code in the AWS code without provisioning or managing servers [ 5.3]. This has a significant knock-on impact. For example, libVIPS was used, via AWS services, by the New York Times to overhaul its digital archive and create, store and serve billions of small images for subscribers interested in back issues of the newspaper dating back to 1851 [ 5.4]. Other high-profile users of the faster, more comprehensive image handling offered by libVIPS include Booking.com, AirBnB Engineering & Data Science, the US Food and Drug Administration, the bank Capital One, Braun, Pantene and US and Spanish property listing sites HomeFinder and Idealista [ 5.1, 5.5].

HomeFinder reports that libvips ‘ is fast enough that we can handle all the photo (real estate listings) manipulation on-demand, giving us the flexibility to use any photo size we want on the site, while only processing photos that are actually seen by our visitors’ [ 5.1]. Swiss digital agency Liip uses libVIPS to generate 100,000 images per day for clients that include Migros, Switzerland’s largest retailer and the country’s largest private employer; libVIPS has cut Liip’s response time for rendering images by three to four times and has saved them money by removing the need to scale up their cloud servers [ 5.1]. Slovenian company Hooray Studios uses libVIPS for their personalised children’s books service to process orders faster; the company doubled its revenues in 2019 to EUR27,000,000 and now has subsidiaries in the US, UK, Italy and Germany [ 5.1]. As project partners on the European grants [ G1, G2], the The National Gallery London and The Louvre uses libVIPS. Others including National Gallery of Washington use it to convert their collection to multi-resolution files [ 5.1].

The OpenIMAJ libraries are used by lecturers and web developers across the world, including large companies like Yahoo and Comcast (the former CTO of Comcast contributed to the development of OpenIMAJ through G2) [ 5.6]. They allow developers to quickly create applications that extract information from multimedia data. Around 31,000 code files mention OpenIMAJ on GitHub and it features in 1,461 ‘Maven pom files’, which represent individual software projects by users of the library [ 5.6]. It is a key component of very widely used software, for example the Webcam Capture library that allows webcams to be accessed directly from Java [ 5.6]. OpenIMAJ is used in university teaching internationally; for example, the University of Missouri uses OpenIMAJ in a module on big data analytics [ 5.6]. It is also used as the basis for the code in a number of textbooks on data visualisation techniques, Raspberry Pi and feature extraction [ 5.6].

Impact of new commercial data products on businesses

The collaboration with Ordnance Survey [ 3.5] on new ways to exploit its aerial imagery archives using machine learning has facilitated business innovation in the government-owned company. Before the partnership, methods for creating OS’s data products were largely manual and therefore limited in scope due to the high costs involved. Noting that ‘ commercial success’ would not have occurred without the ECS research contribution, OS cites the key impacts as [ 5.7]:

• Enabling the OS to ‘ efficiently meet some of the challenging new data needs’ of a multi-million-pound contract with the UK Government to deliver the Public Sector Geospatial Agreement – a commitment to opening up access to geospatial data to support the delivery of critical infrastructure and services.

• Opening up new multi-million-pound markets.

• Identifying opportunities for making up to GBP5,000,000 in annual savings.

• Granting of several patents in the UK, Europe and US.

The document analysis system created for Evolution AI [ G4], which intelligently extracts data from corporate and financial documents, is the foundation of all of the company’s commercial products [ 5.8]. It allowed Evolution AI, for the first time, to automate processes that were previously undertaken manually, bettering human accuracy and reducing the time taken (and therefore cost) ‘ from days to seconds’ [ 5.8]. The impact is demonstrated through client case studies [ 5.8]:

• Saved US analytics firm Dun & Bradstreet 50,000 hours of manual work a year (28 FTEs).

• Saved RBS (NatWest Group) between 100,000 and 200,000 hours of manual work a year and reduced their invoice processing times from 24 hours to three seconds.

• UK asset management firm Unigestion achieved 85% cost savings, increased data accuracy from 30% to 99.5% and halved their data extraction time.

• Made data extraction processes for US analytics firm Fitch Solutions 15 times faster.

Impact of imaging hardware systems on the cultural heritage sector

Fifteen custom-built RTI systems, developed through ECS research [ 3.6, 3.7, G2, G3], were supplied to 12 leading museums and universities over the impact period [ 5.9]. Institutions use these systems on a regular basis to capture images of their artefacts and better understand their collections, which in turn improves the quality of their public engagement. The systems have been used extensively to image cuneiform script; the oldest form of writing in the world (first used in around 3400 BC) distinguished by wedge-shaped marks on clay tablets [ 5.9]. Thousands of tablets have been captured with the RTI domes and logged with the global repository at UCLA: the Cuneiform Digital Library Initiative [ 5.9]. Historic England featured Southampton’s RTI dome imaging techniques as best practice in its guidelines for Multi-light Imaging (2018) [ 5.10].

Figure 1: A Cuneiform tablet (left) and an RTI Dome system (right)

Examples of impact on cultural heritage institutions during the impact period include [ 5.9]:

• Since 2014 the British Museum has used their Dome to image objects of stone, paper, jade and metal, including 15th-17th Century metalpoint drawings for a 2015 exhibition.

• The Louvre (from 2015) is using their Dome to image its 20,000 cuneiform tablets.

• The Ashmolean Museum (from 2014) in Oxford is using the Dome to image a wide range of objects, including its collection of 5,000-6,000 cuneiform tablets.

• The Direction de l'Archéologie du Pas-de-Calais (from 2017) uses it to photograph coins, lithics and ceramics; it enables improved technical drawings of the objects.

• The National Museum of Antiquities, Netherlands (2020) uses theirs to image objects.