Impact case study database

In the HPC environment, fast network interconnects (referred to as fabrics) are a key factor in determining the overall efficiency of a supercomputer under the highest workloads, which provide the most relevant usage scenario of BSMBench. In recent years, the two leading vendors of fabrics have been Intel and NVIDIA Networking (formerly Mellanox Technologies which was acquired by NVIDIA in 2020). Fabrics are complex and performance-critical hardware components. Assessing their robustness and performance needs a dedicated software tool that is flexible enough to emulate different workload scenarios. BSMBench has been the preferred tool of the HPC-AI Advisory Council (known as the HPC Advisory Council before 2018), the Dell/Intel Competence Centre for Cloud and HPC, Intel and NVIDIA Networking for assessing fabric performance in an independent, vendor-neutral way.

Since 2017, BSMBench has been used as a benchmarker by the HPC-AI Advisory Council, the leading and most influential independent advisory organisation in HPC, with over 400 member companies, including AMD, Intel, IBM, NVIDIA Networking and Microsoft. Its central mission is the promotion of best practice in the field. They have used BSMBench to perform an unbiased comparison of the performance of Mellanox InfiniBand (now NVIDIA Networking) adapters with respect to Intel Omni-Path [ C1], IBM POWER8 processors versus x86 processor technology [ C2] and of the use of a novel networking technology called adaptive routing versus more traditional methods [ C3]. BSMBench was chosen for these performance measurements because it “includes the ability to tune the ratio of communication over computations” in contexts that “make up a significant fraction of supercomputing cycles worldwide”. The tests were done “to provide best practice” and demonstrate “the scalability of the compute environment” [ C1-C3]. In all cases, BSMBench has provided clear results that have steered market adoption of the best available product or feature. As an example of the information provided by BSMBench, Fig. 1 [from C3] shows the clear advantage in terms of performance when adaptive routing is used.

Fig. 1: Tests performed by the HPC-AI Advisory Council with BSMBench to contrast the performance of the fabric obtained with the use of adaptive routing with the case in which this recently developed feature is turned off, for variable size of the supercomputing partition [ C3].

BSMBench is also widely used in three international supercomputing centres, namely the Dell/Intel Competence Centre for Cloud and HPC (Pisa, Italy), Supercomputing Wales (distributed across Cardiff and Swansea) and the Texas Advanced Computing Center (Austin, Texas), as a monitoring and performance tool. The Dell/Intel Competence Centre for Cloud and HPC is a joint venture between the University of Pisa and Dell EMC fully supported by Intel. One of the main objectives of the Centre is to design and test the next generation CPU and HPC technologies and solutions both in respect to performance and scalability using a suite of benchmarking platforms, of which BSMBench has been a key component since 2017. The CTO of the Centre says:

“BSMBench is very flexible by design, since it offers the possibility to test compute dominated, communication dominated, and balanced workloads. Since it is an independent benchmark, BSMBench offers a genuine measurement of the capabilities of CPUs and interconnects, since the performance results it provides are not informed by hidden features of commercial compilers that could limit the validity of the information one can extract. Scalable, adaptable to several environments and easy to use, BSMBench provides robust quantitative metrics and at the same time an immediate feeling of the capabilities of the HPC solution it is running on. Since adoption in February 2017, it has become our preferred suite for the early evaluation of new platforms” [ C4].

Among major HPC suppliers, BSMBench has gained wide adoption by Intel and NVIDIA Networking, who use it to develop, test and promote to consumers their fabric and high-performance computing hardware offers. Through improved fault diagnostics, better testing processes and mitigation of losses, these two companies have been the main beneficiaries of the economic impact resulting from BSMBench in the REF period.

From 2018-2020, BSMBench was used at Intel as a performance metric to demonstrate how Intel® Omni-Path Architecture (Intel® OPA) helps maximize cluster performance across applications [ C5] and to market the corresponding OPA fabric product line. BSMBench was deployed by Intel to compare latency, bandwidth, and message rate between it and a competitor [see C5, “application performance” section, OPA performance vs. InfiniBand* Enhanced Data Rate (EDR)]. This analysis enabled Intel to conclude that OPA offers to the customer better total cost of ownership, allowing for more compute or storage hardware to be procured within a fixed HPC budget. In September 2020, the Omni-Path Architecture was licensed by Intel to the newly formed company Cornelius Networks in a deal including a USD20,000,000 investment.

The ability to rely on robust metrics has been particularly important following the discovery of the Spectre and Meltdown security vulnerabilities in 2018. These vulnerabilities, which are caused by hardware design flaws and affect a wide class of modern processors commonly in use, including Intel processors, would allow attackers to execute malicious code on target systems, leaving the latter highly exposed to cyberattacks. These vulnerabilities can be bypassed by current processors only through the use of software fixes that degrade the overall performance of the system. Since the immediate workaround for those vulnerabilities resulted in a 10% reduction in the processing power, the associated cost resulting from the performance loss is estimated to be around USD2,400,000,000. This figure is calculated using market analysis performed by specialised companies such as Hyperion, which has found that the total HPC market, largely dominated by Intel, in 2017, i.e. immediately before the discovery of the vulnerabilities, had a volume of revenue of USD24,300,000,000. Using BSMBench as a measurement tool, Intel was able to demonstrate that even in the aftermath of Spectre and Meltdown the performance of their OPA product had increased [ C6 see first 2 figures for BSMBench application], preventing loss of reputation that could have resulted in decreased sales.

The High Performance Computing Account Manager at Intel UK notes:

[text removed for publication] [ C7].

BSMBench is also a central part of the development processes at NVIDIA Networking. The SVP of Marketing at NVIDIA notes it “fills a hole in the market”, thanks to its advanced design, and:

“ Since 2017, we have used BSMBench on all our key systems to benchmark and develop our InfiniBand technology and to demonstrate to our customers the superiority of our HPC products in the field. We chose to use BSMBench because of the robust metrics it provides around the relative importance of communication and computation. In this respect, BSMBench is a unique High-Performance Computing benchmarking tool that provides essential information on the performance of a supercomputing system. Therefore, when we present our systems to potential clients, we reference the BSMBench metrics to help prove the worth of our devices/systems/interconnects” [SVP of Marketing at NVIDIA, C8].

Oriel Science ( www.orielscience.co.uk) is an innovative and unique public engagement and outreach project which showcases our University’s research in the community. Our impact is significant - from 2016 to December 2020, Oriel Science reached 274,000 ( see Table 1) people in over 300 free events primarily in Wales, but also in England, Europe and Indonesia. We delivered societal impact by:

(1) enhancing the Future Generations’ educational and career journeys, and

(2) targeting under-represented groups by enriching their engagement with science.

We do this by taking Swansea University’s amazing research, packaging it into inspiring and interactive exhibits, and putting these in places people go. “Oriel” is Welsh for “gallery”.

Importantly, our approach is based on the Archer et al. “Science Capital” concept which measures people’s familiarity and connectivity with science. This “ASPIRES” longitudinal study of 19,000 young people famously proved that educational choices in the under-represented are enriched by building Science Capital.

Oriel Science was founded by Swansea physics academics in 2016. Our project is led by three physicists: Allton, a particle physicist, Bryan, a laser physicist and Roberts, an astronomer. Since 2016 our project has employed 50 demonstrator-ambassadors and five administrative staff members. As Oriel Science grew, we enlisted academics from across Swansea University, establishing a significant, pan-University engagement operation with broad appeal which has showcased work from around 100 research groups. At least 31 Swansea University research grant applications have included Oriel Science as a vehicle for impact via public engagement, bringing GBP10,500,000 funding to Swansea [ C1]. Oriel Science provides a platform for public engagement and elevates our Swansea colleagues’ public engagement skillset. Oriel Science has been awarded GBP368,000 from 15 competitive public engagement grants: 3 from Welsh Government, 5 STFC (the largest being a GBP119,000 Leadership Fellow in Public Engagement [ C2]), 2 EPSRC, 1 UKRI, 1 Prince’s Trust and 2 from other charities. Oriel Science has a large media footprint with 96 media engagements since 2016 appearing on TV, radio and a newspaper front page [ C1].

A. Establishing our Impact Model

- Our Pop-up Science Centre

Inside the Pop-Up: DeLorean & LHC mock-up

Our first project was a 400m² Pop-Up Science Centre in the heart of Swansea launched in September 2016 at 34 Princess Way. Amongst its 20 exhibits was the STFC Large Hadron Collider mock-up [R3], a “Back-to-the-Future” DeLorean sports car (the theme was “Time”!), a cloud chamber and other particle detectors, an animation of the history of the Universe since the Big Bang, and many non-physics interactive displays [R2].

Impact 1 Future Generations: Our Pop-Up welcomed 15,924 public visitors over 100 days, 34% of whom were children [ C1]. Our venue’s location and longevity encouraged repeat visits, crucial to building Science Capital. 83% of children (N=280) and 93% of adults (N=464) said they'd learnt more about Science from their visit. The word “interesting” was used by 57% of children to describe their Oriel Science experience (N=570) [ C1].

We also ran school visits for 892 students from 18 schools with 96% of the 25 teachers surveyed saying our exhibit-focussed workshops were “relevant to the curriculum”. In these school visits, 74% of primary students (N=592) and 100% of secondary students (N=55) said they “learnt more about Science” from their visit [ C1].

Impact 2 Under-representations: The venue was deliberately placed in the city centre adjacent to areas of social deprivation. Incredibly, a full 6,000 visitors (38%) said their first ever University experience was Oriel Science, objectively proving our impact on under-represented communities. A postcode analysis proved our visitors were fully representative of the Swansea population (N=197). The percentage of Oriel Science visitors matched the Swansea region, in each of the Welsh Index of Multiple Deprivation 20% bins [ C1]:

| | ⟵ Most deprived Least Deprived ⟶ | | --- | --- | --- | --- | --- | --- | | Oriel Science Visitors | 28% | 16% | 16% | 12% | 28% | | Swansea population | 26% | 13% | 17% | 12% | 32% |

These hugely significant results greatly contrast with visitor demographics to UK museums (e.g. the 2014/5 UK Department for Digital, Culture, Media and Sport ‘Taking part’ survey found that visit rates from the 20% most deprived were 48% lower than the 20% least deprived).

Amongst our visitors were A-level students who became UCAS science degree applicants, including to Swansea University [ C2]. One visitor said her Oriel Science experience “helped her choose a science career” leading to a 1st class Biology degree from Oxford – and becoming the first in her family to attend University [ C3]. A visitor commented

“A wonderful place to teach children about science, and very hands on” [ C2]

The Leader of Swansea Council has noted:

"Swansea Council is delighted with the impact that Oriel Science has had in the community, particularly in bringing footfall into the city centre." [ C4]

We tested our approach by running 10 separate surveys of 827 people, including members of the public, students, social media users, students and parents/carers. 90% indicated that they wanted Oriel Science to establish a permanent Science Centre. Four “Focus Groups” in 2020 with 50 contributors drawn from the public praised the design features we implemented in our Pop-Up [ C1]. All of this evidenced our Pop-Up’s impact.

We used our Pop-Up’s success to establish a new long-term 500m² venue, 21-22 Castle St, Swansea. The planned launch was March 2020, but this was Covid-delayed. We were ready to open in December 2020, but Welsh regulations still forbid us opening as of February 2021.

Table 1: Numbers of People Engaged

New Oriel Science venue, Swansea

B. Building on our Impact Success

Since the Pop-Up, we have engaged with 257,000 people at over 300 events, see Table. These have included co-curated major exhibitions in the Swansea Museum and Glynn Vivian Art Gallery, exhibits in student careers fairs, community events and the London Science Museum, school and public tours of CERN’s antimatter experiment, talks in targeted schools, and presentations in the Green Man Festival and the British, Cardiff, Merthyr and Swansea Science Festivals [ C1].

Impact 1 Future Generations: Oriel Science has a significant schools’ outreach programme reaching 200 schools across South Wales from years 3 to 13 (2016-2020), 130 schools in CERN visits (2014-2020), and a further 5 schools in Borneo (2018). 10,000 young visitors interacted with our antimatter exhibit in the Welsh language in the 2017 Urdd Eisteddfod, one of Europe’s largest Youth Festivals, with a further 18,000 in 2018 and 13,000 in 2019 [ C1].

We used an indicative talk to 30 A-level students from Gower College Swansea (19/11/19) to measure the impact of our schools’ programme. Before the talk, 52% were likely to apply for a Physics degree increasing to 68% afterwards. Using before and after questionnaires, students’ understanding of the Large Hadron Collider increased by 39% [ C1].

School visits to the CERN antimatter experiment were led by Swansea physicists responsible for the underpinning research, with before/after surveys showing a doubling of students’ knowledge of antihydrogen [ C1].

“Such visits are very important in their career choice” - Finnish school co-ordinator. [ C1]

We run annual Particle Physics Masterclasses which are spread over two days due to school demand. These typically have 200 attendees, but in our 2020 online event, this increased to 366 students from 98 schools [ C1]. The Head of Science, Bryn Tawe School, Swansea said:

“The workshops and lectures help enthuse our pupils and contribute in no small way to the numbers wishing to take Physics further.” [ C5]

We also deliver annual high school “Physics Christmas Lectures” to around 500 students each year, with speakers including the former Project Lead of CERN’s Large Hadron Collider. In a 2018 survey (N=169), the before/after percentage of students who correctly stated that the Higgs boson is responsible for the mass of fundamental particles grew from 45% to 83%.

Impact 2 Under-representations: Oriel Science targets schools from under-represented communities. For example, in a 2018 school visit programme designed around particle physics demonstrations, 7 of the 16 schools visited were in the 20% most deprived areas of Wales [ C1].

We sought out other under-served communities – a 2019 talk on the Physics of gravity, relativity and GPS was delivered to 20 Swansea Prison inmates [ C1].

C. Reaching the public in National Waterfront Museum events

We have partnered with Swansea’s National Waterfront Museum in March 2018, 2019 and 2020 to run our one-day Super Science Swansea Festival. These are timed to coincide with British Science Week. Footfalls for each year were 1987, 3588 and 3232 respectively, amongst the largest daily footfalls in the museum’s history [ C1]. These events showcased our research with exhibits such as *“Would I die if I fell into a black hole?*” [R1]. In addition, our CERN antimatter simulator [R4-R6] was a very well-received part of the Innovation Exhibition (12/10/19 – 18/03/20) and had a footfall of 90,000 [ C6]. Finally, the Waterfront hosts our monthly “Oriel Science Café” public lectures which have engaged with 2,440 people (2014-2020) [ C1].

"Oriel Science has been a wonderful partner for the Waterfront, enabling us to reach and engage with new audiences." – Head, National Waterfront Museum [ C6]

Impact 1 Future Generations: 70% of our annual Science Festivals’ visitors were in family groups and 29% were under 16 (2018, N=82). The impact of these festivals with interactive exhibits is clear. For example, the “Love a Maggot” demonstration in the 2019 Festival explained the use of maggots in wound healing. Surveys showed 46% of children would agree to having this therapy before their engagement with this activity, rising to 90% after (N=180).

“It's the best science day in the world” – child, aged 6 [ C1]

“My oldest two have thoroughly enjoyed all the science they have discovered” – parent [ C1]

Impact 2 Under-representations: A postcode analysis of our 2020 event showed that 18% of visitors came from the most deprived 20% of Wales (N=35) greatly reducing the demographic underreach that traditional museums suffer from. Participants’ awareness of Swansea University research increased by 87% (N=84, 2018), 61% (N=113, 2019) and 77% (N=91, 2020). A survey one month after the 2019 event proved this impact survived with 75% confirming that their awareness of our research had increased (N=16). We use these Festivals to celebrate International Women’s Day and partner with Women’s Equality Network who arrange a number of female scientists to give inspirational talks. Our Festivals were very well received, e.g., in 2018, 94% said they would come again [ C1].

D. Welsh Teacher CPD – the significance of education provision

Impact 1 Future Generations: Alarmingly, only 44% of Welsh A-level physics teachers have a physics degree. As Director of Oriel Science, Allton worked with the Welsh Government, the STEM Learning Centre and the former Project Lead of CERN’s Large Hadron Collider to address this poor Physics provision in schools. A CPD programme was devised, based around a week-long visit by teachers to CERN, with pre- and post-visit workshops in the Swansea Physics Department. This CPD is designed to enhance the confidence of non-specialist Physics teachers, so they go on to inspire their students, raising their Science Capital.

“This course was excellent providing me with examples to illustrate my lessons to show how physics is at the cutting edge of modern science.” – teacher, Cardiff [ C7]

48 teachers took part in this programme from 2016 to 2017, and some were inspired to take their own students to visit CERN [ C7]. An estimated 1,000 students are taught by these teachers annually. The benefit of this CPD was highlighted by the Welsh Education Minister:

“Opportunities like this are vital if our teaching workforce is to better understand and communicate the application of real-world science and technology… The feedback we have received from last year’s group is that the knowledge and experience gained from this trip has made the teaching of physics much more enjoyable and meaningful for the students.” [ C8]

Introduction

We have founded an award-winning spin-out company, CanSense Ltd, that takes Raman spectroscopy as the base technology for application to rapid biomarker testing for early cancer detection via a blood test. Its primary product is based around colorectal cancer (CRC) detection and relies on award winning research [ C1] by Harris and Dunstan in 2016.

Clinical impact

Our pilot work developing a CRC blood test has overcome previous limitations affecting Raman spectroscopy in clinical diagnostics using new sampling innovations on a modified research Raman microscope (currently at TRL4). The test has a clinical proof of concept with expanded patient training sets producing the latest results, showing sensitivity of 84% and specificity of 91% for CRC [ C2]. The cancer AI model also detects pre-malignant polyps (adenomas) down to 2 mm size with 84.2% sensitivity and 89.5% specificity [ C2, C3].

A high negative predictive value (NPV) >98% means that there is high confidence in the test ruling out cancer (c.f. colonoscopy NPV 99.5%). Additionally, good patient acceptability was found when the test was employed within an academic study of primary care patients (n=595). [ C4, preprint]. The study introduced a mechanism for the introduction of the test directly into primary care as a triage tool for GP referrals and evidenced that it would be well received. The positive predictive value ( PPV) (> 11.8%) offers GPs the opportunity to triage patients more effectively for referrals (c.f. only 3% under current NICE guidelines for suspected cancer pathway referrals), potentially reducing colonoscopies by 49.8% representing significant NHS savings ~GBP418 per patient and ~GBP265,000,000 annually.

Our test's clinical potential has been guided by the company’s co-founder Dean Harris, a practicing colorectal cancer surgeon and researcher. Trial patient testimonials include “It has changed my life, to know to have a blood test which can prove very very quickly if the cancer has come back. This is incredible." [Patient testimonial , C5]

Our blood test has been endorsed by the National Cancer Clinical Director for Wales, Clinical Lead Transforming Cancer Services, Professor Tom Crosby OBE who said, “Thousands of bowel cancer deaths could be prevented every year through earlier diagnosis of this disease. This is a potentially game changing innovation in terms of population-based access to a cheap, convenient and acceptable test. This innovative test will mean subjecting less patients to invasive endoscopic examinations. This is an important step in the development of precision medicine undertaking the right test for the right patient at the right time.” [ C6, page 10].

CanSense Ltd now has access to more than 1300 patients which have been recruited through this network and current on-going trials will considerably increase this number. NHS services and trained personnel including GPs are also impacted as they help the trial recruit patients, and then process, store and record all samples prior to transfer into our laboratory.

“You can say to (the patient) there’s a 98% chance it’s not going to be cancer, how relieving is that. It would be a huge weight off his mind and reduce a lot of patient anxiety” [GP testimonial , C5] and “Being Timely, if the blood test is normal, you can get on with your rest of your life!” [GP testimonial, C5].

We have already witnessed greater patient acceptance of a blood test rather than the usual unpleasant, poor compliance screening techniques (faecal sample/ colonoscopy), as clearly identified in the first mixed methods clinical evaluation study which evaluated test performance and patient acceptability via focus group interviews [ C4]. Further trials which continue to expand evidence on this are underway.

A new national clinical trial called Combined Raman and FIT (CRaFT) commenced in 2018 with the goal of recruiting 800 participants. The trial now has participating GPs from at least 4 Welsh health boards across South Wales and one English health board. This trial will inform on Raman performance compared to current faecal testing (FIT) and reports in the next 12 months.

Formation in 2018 of an Award-Winning spin-out company - CanSense Ltd

As a vehicle to achieve wider immediate translation of the diagnostic technology a spin-out company, CanSense Ltd [ C5], was registered in 2018. CanSense is working in partnership with SU and Swansea Bay University Health Board to clinically develop the technique to detect cancer early. Along with the researchers (Dunstan, Harris, Jenkins), a share-holding CEO with over 20 years’ finance experience was appointed in 2018, along with 5 advisory board members (ranging from specialised healthcare consultants to a patient involvement consultant). CanSense leveraged ongoing clinical trials in 2020 (>35 primary care practices participating) and has recruited 4 secondary care champions to influence stakeholders in health boards and ensure wide scale clinical interest, adoption and procurement in various Welsh regions. Since 2019, 3 Pharmaceutical companies (under NDA) have expressed an interest for their trials of oncology therapeutics as a test for minimal residual disease detection. CanSense is a business-to-business service provider with a direct fee-for-service model through a centralised processing laboratory. Given the need for timely primary care decision-making in line with the Single Cancer Pathway we have modelled returning the result to the GP within 48 hours.

The company collaborates with the University and the charity funders Cancer Research Wales (CRW). CRW’s strong endorsement of the activities of CanSense are provided in a letter of support [ C7], which includes the following quote:

‘Importantly, the greatest accolades have come from the end-users themselves, patients, GPs and the Wales Cancer Network. Those patients who were involved in the trials for the Raman blood test are unanimous in their agreement that this blood test, even at the early pre-clinical stage is a welcomed and a much-needed novel intervention that can transform how, and the speed at which, bowel cancers are diagnosed”.

The company and its founders support bowel cancer awareness and the CRW charity champions to promote fund raising and awareness of the impact of the charity’s funding of research [ C8]. This has led to a one-hour BBC Radio Wales programme which featured the blood test [ C9] and wider dissemination through public talks and workshops.

Innovation Funding

At present the company has had innovation funding in excess of GBP250,000 since 2019. A proportion of this funding came from winning competitive catalyst/accelerator programs including:

• BioCity Oxford Launch program 2020, GBP50,000 in-kind funding and Venture Capital potential, GBP50,000 convertible note [ C10]

• ICURE Midlands participant funding, GBP32,114 [ C10]

• BioCity/Oxford AHSN Accelerator program, 2019, in-kind funding GBP25,000 [ C10]

• University of Southampton Science park Catalyst and Catalyst + programs 2019/2020, in-kind funding GBP15,000 [ C10]

Awards

CanSense has won a number of key awards since its inception, namely:

South Coast tech awards, best start up and best innovative tech company, winner 2020 [ C11]

Empact Ventures-Super Connect for Good: Regional winner, 3^rd place nationally, 2020 [ C11]

MediWales: Best Innovation winner 2020 & Best Start-up winner 2019 [ C11]

MediLink – Start-up award - CanSense Highly Commended 2020 [ C11]

CanSense has interviewed over 200 customers (clinical commissioning groups, GPs, pharmaceuticals, diagnostic companies) thereby validating our business plan and identifying our early adopters as pharmaceutical clinical trials and private/public primary healthcare providers. CanSense is focussed on the product’s clinical development for CE/UKCA marking and NICE approval.

The Coronavirus pandemic has undoubtedly affected our company journey in the past 12 months, as trial recruitment was suspended, and diagnostic clinical development impacted. This period has however highlighted the urgent need for rapid and early cancer diagnostics based around non-invasive procedures, and how diagnostic triage tools can reduce unnecessary referrals into secondary care. Due diligence is being conducted by several investors at time of writing (December 2020) and we await the outcomes of two impactful NIHR awards (AI: GBP1,400,000 and i4i PDA: GBP1,100,000).