Impact case study database

To engage and benefit fully from academic study, tertiary students studying through the medium of English need to have appropriate and sufficient English language skills. If under-qualified applicants are admitted, the economic, social, and personal costs can be enormous. Academic English tests serve a key role in providing universities with information they need to make admissions decisions and to determine whether any remedial language instruction is required ( 5.1). CRELLA’s research has contributed to a deeper understanding of the nature of Academic English and provided a theoretically sound and practically efficient framework for its measurement, giving test providers the basis for high-quality assessment. CRELLA’s work plays an integral role in the validation and further improvement of existing EAP tests and guides the development of new tests. It generates significant commercial impact for test providers and considerable educational impact for receiving HE institutions, English language teachers and students globally.

Among a range of academic English tests, CRELLA’s research substantially informed the design and validation of all four UK-based tests that have been officially (re)approved as meeting the criteria for secure English language tests (SELT) by UK Visas and Immigration (UKVI): IELTS, Trinity ISE, PTE Academic, and LanguageCert. CRELLA has thus contributed to the implementation of international student recruitment policy and to more stringent standards for selecting appropriately qualified students for HE in the UK.

Examples of Commercial Impact

• IELTS

CRELLA’s research has informed the agenda for the IELTS Partnership Research and Development group responsible for major revisions to the test. Shared with the IELTS Partners through strategic meetings, presentations and over 20 collaborative publications with their staff, the research has helped to distinguish IELTS from prominent international competitors, assisting them to standardise the content of the test through improved test writing and scoring procedures across 86 versions in concurrent operation. It has been empirically demonstrated that IELTS is a better predictor of academic success than alternative HE entry pathways ( 5.2). As stated by the Head of Research Strategy at Cambridge Assessment English (on behalf of the IELTS Partners), CRELLA’s research has also “contribute[d] to the test’s state-of-the-art standing, international reputation, and commercial success” over the recent years ( 5.3). Global recognition has grown 40% from 7,000 to over 10,000 educational institutions, government agencies and professional organisations. During the current REF period, annual test-taker numbers have risen from 2 million to 3.75 million ( 5.4), making IELTS the industry leader and generating an estimated GBP650 million p.a. (GBP170-195 per test) for the IELTS partners. Furthermore, the new online format of IELTS ( IELTS Indicator), launched across 50 countries in April 2020, features a video-conferencing ‘Speaking’ test and a computer-based ‘Writing’ test. These were introduced as a direct result of CRELLA’s validation research ( 3.5, 3.6), enabling IELTS to remain the preferred option for international student admissions during the global COVID-19 pandemic ( 5.3).

• Password

CRELLA’s research underpinned the design and development of a new EAP test: Password ( 5.5). In 2008, CRELLA’s original commercial proposal led to a joint venture business (English Language Testing Ltd). The company now employs 8 permanent staff members in its London-based office and 6 part-time item writers and 25 examiners who work from test specifications and scoring rubrics developed by CRELLA (Headcount: 39, FTEs: 15). Following the success of Password Language Knowledge, CRELLA’s research led to the commissioning from CRELLA of a further 6 test modules, targeting different student populations and a broader range of language skills: Password Intro, Password Pupil, Password Reading, Password Writing, Password Speaking, Password Listening. Since 2013, the Password tests have been taken by approximately 1 million learners and used by about 400 universities around the world, as an efficient means of assessing students’ language levels and identifying those needing language support. This represents a tenfold increase in candidature and an over 40% increase of institutional recognition (cf. 120 institutions and 100,000 candidates during the REF2014 period) ( 5.5).

Examples of Educational Impact

• Trinity Integrated Skills in English (ISE)

In 2013, CRELLA was commissioned to redesign a suite of the ISE tests ( 5.6) to reflect the integrated nature of academic English skills based on CRELLA’s research ( 3.1, 3.2, 3.4; 5.6). Both the Reading & Writing and Listening & Speaking components were fully re-developed and validated across four proficiency levels. As a result of CRELLA’s contribution, the new ISE launched in 2015 was one of the only two UKVI SELT tests approved at that time. The test was awarded the Association of Language Testers in Europe Q-mark, having met all 17 of their stringent quality standards ( 5.6). ISE is recognised by 98% of UK universities and regarded as “a truly state-of-the-art exam” (Director, University-wide Language Programmes, University of Manchester; 5.6). Commenting on the success of the test, the Academic Manager at St Edmund’s College Summer School underlined CRELLA’s accomplishment, noting that:

“the teachers…have been very enthusiastic about the exam and preparing students for it. Feedback about the methodology of the exam has been very positive. I was impressed by how they were able to integrate the exam skills into their normal class teaching — the exams fit well into our style of lessons” ( 5.6).

• The Test of English for Academic Purposes (TEAP) in Japan

Until the Eiken Foundation (Japan’s leading English language testing organisation) and Sophia University (Japan) commissioned CRELLA to develop a university entrance examination, four-skills language testing had been rarely employed for university entrance in Japan. TEAP, launched in 2014, has over 30,000 candidates annually and is recognised by 184 universities and colleges. In 2020, TEAP was delivered in 26 prefectures nationally. As the Chief Editor (Test Production Division) at the Eiken Foundation states:

“[t]he long-term impact of the TEAP project is not only limited to those universities that recognize the results: the test has played a significant role in improving EFL teaching and learning practices in Japan by overcoming the barrier to the reform imposed by the established university entrance examinations” ( 5.7).

As one journal author put it, The Test of English for Academic Purposes has “led to an increase in the amount of time students spent studying speaking” ( 5.8). This is a further demonstration of how TEAP (based on CRELLA research) has brought major benefits for high school English language education.

• General English Proficiency Tests (GEPT) in Taiwan (‘Advanced’ level used for university admission purposes)

CRELLA collaborated with the Language Training & Testing Centre (LTTC) in Taiwan on quality assurance for its locally developed tests ( 3.4). Improved quality increased acceptance of GEPT for HE admissions within Taiwan and among prestigious institutions in the UK, US, Europe, and elsewhere, facilitating access to international HE for local students, especially those in remote locations ( 5.9). As the most widely taken English test in Taiwan, GEPT has had positive effects on English language education in the country, promoting academic language use and the teaching of speaking and listening ( 5.10). As well as noting that CRELLA’s research was “instrumental” in ensuring that the test would “reflect the real-life tasks that students are expected to encounter in the academic context and to elicit the same underlying cognitive processes that real-life tasks do” ( 5.10), the R & D Program Director at LTTC also acknowledges that:

“CRELLA has been LTTC’s long-term partner in research, and partly as a result of the quality assurance and quality enhancement measures [that] their research has informed, the GEPT has seen a considerable growth in the past 6 years, marking 2 million candidates since 2014” ( 5.10).