Impact case study database
Statistical methods for improving efficiency of online advertising
1. Summary of the impact
Real-time bidding has transformed online advertising, as companies compete over website advertising space in the milliseconds it takes to load a webpage. Cardiff developed statistical algorithms integrating big datasets on consumer online behaviour. These enabled companies to create an overall adaptive bidding strategy for advertising campaigns, informing rapid decisions on deploying adverts to specific online users. Cardiff’s algorithms were adopted by leading digital marketing company Crimtan, resulting in a 20% increase in their annual turnover, annual savings of £370K, and generation of £2M of new business with clients including Swiss Air and Springer Nature. The research benefited Crimtan’s global client base, resulting in increased sales estimated at over £3.5M per annum.
2. Underpinning research
The global digital advertising market has been particularly driven by the growth of programmatic buying, specifically the process of real-time bidding where companies compete for advertising space on websites in a ‘virtual auction’ that occurs in the milliseconds it takes for a webpage to load. This process is repeated billions of times each day: the Interactive Advertising Bureau Programmatic Market Sizing Report reported 62% of digital display adverts in European territories were traded programmatically during 2017.
Zhigljavsky and Pepelyshev of Cardiff University’s School of Mathematics developed mathematical and statistical algorithms [3.1] to enable adaptive targeting within website advertisements [3.2]. This allows companies to make faster and more accurate decisions on whether to show a given advert to a particular user using automated real-time bidding. The research was funded and developed in collaboration with digital marketing agency Crimtan, through proprietary data for development and testing of the algorithms.
The automated real-time bidding is based on algorithms and big data sets [3.3] typically informed by previous requests from auctions, impressions, clicks, conversions, and the likelihood of users clicking through to a website that is being advertised. Cardiff’s focus on machine learning techniques produced algorithms that identify suitable customers for adverts based on prior online behaviour, and then subsequently devise a bidding strategy when the purchase of an advert is deemed worthwhile.
The Cardiff team also explored the relative influence of factors on clickthrough and conversion rates. The resultant algorithms are computationally light yet able to achieve the same accuracy as computationally demanding machine learning algorithms such as Gradient Boosting – which produces prediction models based on collections of weaker models – or Field-Aware Factorisation Machines (FFM) – a variant of factorisation machines, a general predictor capable of modelling interactions between variables even in problems with huge sparsity. Cardiff’s new algorithms used stochastic global optimisation techniques (methods using probabilistic data in the problem data, algorithm, or both) [3.4] and are directly built on previous underpinning research and methods by Zhigljavsky and Pepelyshev, including stopping rules [3.5] and the rate of convergence [3.6] in general global random search algorithms.
Cardiff’s new computationally light algorithms were well-placed to operate in the split-seconds in which target-advertisement bidding takes place and, via the collaboration with Crimtan, the research was able to quickly impact on the company’s product development and sector competitiveness.
3. References to the research
[3.1] Pepelyshev A., Staroselskiy Y., Zhigljavsky, A. and Guchenko, R. (2016) Adaptive targeting in online advertisement: models based on relative influence of factors. Published in: Pardalos, P., Conca, P., Giuffrida, G. and Nicosia, G. (eds.) Machine Learning, Optimization, and Big Data. MOD 2016. Lecture Notes in Computer Science Springer, pp. 159-169. http://dx.doi.org/10.1007/978\-3\-319\-51469\-7\_13
[3.2] Pepelyshev A., Staroselskiy Y. and Zhigljavsky, A. (2016) Adaptive designs for optimizing online advertisement campaigns. MODA 11 - Advances in Model-Oriented Design and Analysis. Contributions to Statistics, Springer-Verlag, pp.199-208. https://doi.org/10.1007/978\-3\-319\-31266\-8\_23
[3.3] Pepelyshev A., Staroselskiy Y. and Zhigljavsky A. (2015) Adaptive Targeting for Online Advertisement, Machine Learning, Optimization, and Big Data. Springer Lecture Notes in Computer Science, Vol. 9432, pp. 240-251. http://dx.doi.org/10.1007/978\-3\-319\-27926\-8\_21
[3.4] Zhigljavsky A.., Žilinskas A.G. (2008) Stochastic Global Optimization, Springer-Verlag US.
[3.5] Zhigljavsky A., Hamilton E. (2010) Stopping rules in k-adaptive global random search algorithms. DOI: 10.1007/s10898-010-9528-6 Journal of Global Optimization, v. 48, No. 1, 87–97.
[3.6] Pepelyshev, A., Zhigljavsky A., and Žilinskas A. Performance of global random search algorithms for large dimensions. Journal of Global Optimization, 71 (2018): 57-71. https://doi.org/10.1007/s10898\-017\-0535\-8
4. Details of the impact
Through a long-standing collaboration with digital advertising company Crimtan, Cardiff-developed statistical algorithms enabled the company to significantly escalate its programmatic buying strategy and increase the success of its digital marketing campaigns. The research resulted in 20% increased turnover and £2M of new business for the agency. The research enhanced real-time programmatic bidding across Crimtan’s extensive global client base. This increased advert-response rate by 10% and is estimated to have added £3.5M a year in sales to these clients.
4.1 Enhancing Crimtan’s real-time advertising strategy
By incorporating the Cardiff algorithms into their decision-support products, Crimtan improved the timeliness, efficiency, and accuracy behind online advertisement bidding, and directly reduced their staffing costs. In particular, the automation of the real-time bidding process enabled Crimtan to save £370K annually in associated staff costs, allowing them to rationalise their staffing strategy and reposition some employees into new business areas within the company [5.1].
Dr Yuri Staraselski, Chief Technology Officer at Crimtan, stated the improved performance of their real-time bidding services “ has been highly influential in enhancing our global reputation and effectiveness of our services that we advertise to prospective clients”. As a result, Crimtan significantly increased their turnover and acquired new business. Staraselski stated that working with Cardiff was “ by far, our most successful academic collaboration to-date and has led to significant benefits and impact both to Crimtan and the clients we serve globally” [5.1].
Crimtam attributed Cardiff’s involvement to a range of outcomes, including the use of automation of optimisation methods, parameter selection, and adaptive bidding strategies developed by the Cardiff researchers. Staraselski continued: “ As a result of our collaboration with Professor Zhigljavsky and Dr Pepelyshev, and use of their improved and innovative algorithms, we have enhanced our reputation and increased customer sales for our clients”. Subsequently, Crimtan’s clients are able to intelligently target users using Crimtan’s more focused and data-driven services [5.1].
Drawing in additional international clients was also directly credited to the Cardiff collaboration. Staraselski confirmed that the collaboration “led to the acquisition of £2 million of new business and has increased our turnover by 20% since working with the Cardiff researchers, with new clients including Moss Bros, Camelot, Boohoo, KFH, AX Paris, Swiss Air, and Springer Nature” [5.1].
In both 2017 and 2018, Cardiff University and Crimtan organised workshops for the latter’s clients and users [5.2]. These workshops, held at the Royal Institution, focused on key topics surrounding statistical analytics and machine learning in digital marketing, and explained how the Cardiff research could help companies identify, and therefore retain, their most valuable customers, as presented by Zhigljavsky and Pepelyshev. Speakers from Crimtan included their Analytics Lead, Product Director and Technology Innovation Director. Several senior staff from Crimtan’s international client portfolio sat on a panel Q&A, including representatives from McLaren, Nationwide, O2, Amplyfi, and Camelot Group; this panel focused on discussion around the statistical techniques from Cardiff’s research studies.
4.2 Increasing benefits for global clients
Crimtan’s use of the Cardiff research improved highly targeted programmatic advertisement campaigns for their global clients [5.1], with client board members able to access comprehensive and reliable data for evidenced decision-making around their advertising campaigns. With the Cardiff algorithms embedded within these decision support tools, Crimtan’s clients were also able to identify and target non-traditional prospective customers, attracting new users to their websites and leading to increased sales [5.1].
Commenting on the global impact for their clients, Staraselski stated: “ Our evaluation suggests that the growth in new customers directly arising from the use of Cardiff’s algorithms is 10% i.e. a 10% growth in customers clicking through to our client’s promotions and website” [5.1]. Although client confidentiality agreements mean it is not possible to supply an exact value from increased sales, Crimtan was able to state the annual value of this growth in customer base across its clients “ would very likely exceed £3.5 million” [5.1].
Crimtan regularly receive feedback from clients who “ have unanimously been impressed by the improved performance and efficiency of the decision support tool since incorporating Cardiff’s research” [5.1]. Brief examples from the wide range of successful client impacts linked to Cardiff’s research include:
Red Carnation Hotels: Crimtan used the decision support tool to run a programmatic ad campaign for Red Carnation Hotels, targeting affluent travellers who were exhibiting a clear desire to stay in a high-end hotel in London. “ Crimtan’s campaign far exceeded our expectations. It had a significant impact on both our brand awareness in key markets and in reaching our quarterly revenue goals. We generated 22 times more revenue than any other previous online advertising campaign as a result” [5.1]. Red Carnation Hotels ran the programmatic campaign in the UK and USA and achieved a return on investment of £18 for every £1 spent on the campaign [5.3].
Miss Selfridge: In their campaign for Miss Selfridge, Crimtan were able to deliver a new customer rate of 47% for the company, generating £307K of revenue over six weeks. Insights developed during this period showed that conversions of new customers was particularly strong between 8pm-11pm, so Crimtan’s custom bidding strategy became more aggressive during these hours through application of the algorithms developed by the Cardiff team [5.4].
Moss Bros: “ Crimtan have become a key component in our marketing strategy. We have discovered completely new customers to our business resulting in £240K of incremental revenue. The overall return on investment was 9:1. We delivered a new customer rate of 43%, checked against their five-year customer file” [5.1]. Via the Cardiff-supported tool, Crimtan gathered new information for Moss Bros around peak times for engagement with prospective customers as well as critical information regarding the types of devices their customers used to purchase their products [5.5].
In summary, the implementation of Cardiff’s research by Crimtan resulted in a 20% increase in turnover for the company, and 10% increase in new customers for their global client base. For these achievements, Zhigljavsky received the 2019 Constantin Carathéodory Prize of the International Society of Global Optimization. This prize is awarded bi-annually for fundamental contributions to theory, algorithms, and applications of global optimisation. The award criteria includes scientific excellence, innovation, significance, depth, and impact [5.6].
5. Sources to corroborate the impact
[5.1] Testimony from Dr Yuri Staraselski, CO, Technical Director, Crimtan
[5.2] Cardiff-Crimtan workshops event itinerary
[5.3] Crimtan’s case study on their work with Red Carnation Hotels
[5.4] Crimtan’s case study on their work with Miss Selfridge
[5.5] Crimtan’s case study on their work with Moss Bros
[5.6] International Society of Global Optimization webpage corroborating award of the Constantin Carathéodory prize to Professor Zhigljavsky