Ad description
A Tweet for The University of Leicester, dated 13 May 2022, stated “We’re number 1! Our Arts and Humanities research is number 1 in the UK for overall research quality (GPA) in #REF2021”.
Issue
The complainant, a former academic, who understood that specific rankings for an “Arts and Humanities” research category were not included in the Research Effectiveness Framework (REF), and that the University of Leicester’s submissions of data in arts and humanities subjects in the REF 2021 were less extensive than those from other universities, challenged whether the claim “Our Arts and Humanities research is number 1 in the UK for overall research quality (GPA) in #REF2021” was misleading and could be substantiated.Response
The University of Leicester said they undertook a rigorous process to ascertain the rankings quoted in their ad. They said that after the Research Excellence Framework (REF) 2021 results were released in May 2022, the Times Higher Education (THE) released a further analysis of those results in which universities were given an overall institution ranking for research based on Grade Point Average (GPA). The University of Leicester’s ranking by the THE was confirmed as joint 30 out of 129 entries. They said the THE methodology was considered the most accepted and reputable across the Higher Education Institution sector for calculating a ranked position in the REF.
The University said they had internally self-calculated their number one ranking position for arts and humanities research, as referenced in the ad, by applying an additional layer of analysis to the THE results. The REF 2021 results were split into four main groups of subject areas called ‘panels’. They had applied the THE methodology (normally used to calculate overall institutional rankings) to panel ‘D’, which included a subset of ten arts and humanities subjects, and determined they were the top ranking university for research within that group.
The University said their ranked position for Panel D took into account all submissions from every higher education institute and, as per the THE methodology, the GPA was weighted by staff full-time equivalent (FTE) across the units of assessment that each higher education institute submitted to.
The University also said the REF methodology was complex and many institutions listed their results in different ways, in part because there was not a standard ‘ranking system’. They said that the THE methodology was the best guide, although internal methodology could also be used to help shape the position. They believed the methodology they had used was sound, and noted that after the 2021 REF results were published, other institutions had published similar posts that also lacked references to the methodology used.
The University additionally highlighted that their ad was an organic social media post that was not paid-for. Following the complaint being raised, the University had reviewed the tweet and recognised that it did not reference the THE methodology or a University of Leicester press release containing the full institutional results. It had taken action to remove the tweet and said it would ensure that any future marketing messaging strictly adhered to the Code and demonstrated the methodology used.
Assessment
Upheld
The ASA considered that the average consumer would interpret the claim “We’re number 1! Our Arts and Humanities research is number 1 in the UK for overall research quality (GPA) in #REF2021” to mean that the University of Leicester had been ranked by the REF 2021 as the top performing university for research in arts and humanities in the UK.
We understood that the REF 2021 assessed universities’ performance in research by Unit of Assessments, which were generally specific subjects, such as philosophy, but did not formally rank institutions either as a whole or by wider subject groups, such as ‘arts and humanities’. The THE had applied a methodology to analyse the REF 2021 results and published a league table ranking the institutions in research by order of GPA, and the University of Leicester were positioned joint 30 of 129 entries. The THE league table was based on an overall score for each institution and did not further rank universities by subject groups, including arts and humanities. Additionally, the University of Leicester explained that they had applied the THE methodology, usually used to determine overall rankings for institutions, to a subset of arts and humanities subjects (Panel D) as defined by the REF, and calculated their “number 1” position within that group.
We noted the REF 2021 results did not formally rank universities, and therefore there were different ways that institutions could be ranked depending on the methodology chosen. The basis of the comparative “number 1” claim had not been clarified in the ad, nor was there a means of verification provided. It did not state that the ranking had been generated by the University itself using the THE’s methodology rather than a finding explicitly stated in the REF 2021 report, nor otherwise explain how the result was deduced.
We had neither seen the University’s calculations, nor a full explanation of how the methodology accounted for the University submitting data across fewer arts and humanities subjects in the REF 2021 than other institutions, and therefore we were unable to comment on those findings. However, given the likely interpretation by consumers of the “number 1” claim, and in the absence of evidence to support that interpretation or a clear and prominent qualification to explain its basis, we concluded the ad was likely to mislead. We welcomed the University’s decision to withdraw it.
The ad breached CAP Code (Edition 12) rules 3.1 3.1 Marketing communications must not materially mislead or be likely to do so. (Misleading advertising), 3.7 3.7 Before distributing or submitting a marketing communication for publication, marketers must hold documentary evidence to prove claims that consumers are likely to regard as objective and that are capable of objective substantiation. The ASA may regard claims as misleading in the absence of adequate substantiation. (Substantiation), 3.9 3.9 Marketing communications must state significant limitations and qualifications. Qualifications may clarify but must not contradict the claims that they qualify. (Qualifications), 3.33 3.33 Marketing communications that include a comparison with an identifiable competitor must not mislead, or be likely to mislead, the consumer about either the advertised product or the competing product. and 3.35 3.35 They must objectively compare one or more material, relevant, verifiable and representative feature of those products, which may include price. (Comparisons with identifiable competitors).
Action
The ad must not appear again in its current form. We told the University of Leicester to ensure that future ads did not suggest they had been ranked directly by the REF 2021, and to ensure the basis of comparative claims was made clear and was verifiable, particularly when citing their own analysis or citing reports or methodologies by other bodies.
