Background
Summary of Council decision:
Three issues were investigated, all of which were Upheld.
Ad description
Three TV ads for BT broadband, seen in October 2015:
a. The first ad featured a man catching flies with chopsticks while an actress stated "Check out this guy - he's super responsive. But, he's not a patch on BT Infinity's super responsiveness when you're online gaming." During the latter part of the statement a BT hub was shown underneath a banner stating "SUPER RESPONSIVE GAMING". On-screen text stated “BT Infinity 2 average performance vs Virgin 50, 100 & 152. Includes wireless and fixed-line performance”. At the end of the ad, text stated "BT Infinity offers more responsive gaming than Virgin Media" while a voice-over stated "BT Infinity fibre optic broadband offers more responsive gaming than Virgin Media".
b. The second ad featured a topless man using a hula-hoop. As the actress stated "See this guy? He's super smooth. Oh, but he's nothing compared to the super smoothness of video-calling with BT Infinity." a BT hub was shown, underneath a banner stating "SUPER SMOOTH VIDEO CALLING". On-screen text stated “Virgin Media Comparison: SamKnows report June ’15, BT Infinity 2 average broadband performance vs Virgin broadband.” At the end of the ad, text stated "BT Infinity offers smoother video calling than Virgin Media" while a voice-over stated "BT Infinity fibre optic broadband offers smoother video calling than Virgin Media".
c. The third ad featured a cheetah, about which the actress stated "Wow, look - it's a live cheetah. She's fast and dangerous. But, do you know what's faster? When you download with BT Infinity it's so fast." During this statement a BT hub was shown underneath a banner stating "FASTER DOWNLOAD SPEEDS" and small on-screen text stated "Sky Comparison BT Infinity 2 average download speeds vs equivalent Sky fibre broadband ... Includes wireless and fixed-line performance" and gave a reference to the source of this information. At the end of the ad, text stated "BT Infinity offers faster download speeds than Sky" while a voice-over stated "BT Infinity fibre optic broadband offers faster download speeds than Sky."
Issue
The ASA received two complaints.
1. One complainant, who had used Virgin Media and BT Infinity and reported no difference in their user experience, challenged whether the claims in ads (a) and (b) that BT Infinity had more responsive online gaming and smoother video calling than Virgin Media were misleading and could be substantiated.
Sky UK challenged whether:
2. the claim "BT Infinity offers faster download speeds than Sky" in ad (c) was misleading and could be substantiated; and
3. sufficient information was provided for the comparison in ad (c) to be verified.
Response
1. BT said online gaming required download speeds between 1 and 4 Mb/s. Both Virgin Media and BT therefore provided speeds capable of supporting online gaming, so they focused on the quality of the experience rather than the download speed alone, and noted that upload speed was particularly important for online gaming. They said a customer could have a line speed fast enough to support online gaming, but low performance on various quality metrics would lead to less responsive gaming. They said the key metrics impacting on performance were: jitter (the variation and stability of data transfer); latency (how long data packets took to transmit); upload speed; and packet loss (which would affect the quality of the transmission).
BT said their verifiability page set out independent testing by SamKnows, which showed that BT were the best or unbeatable on all core network metrics, leading to clear superiority in terms of customer experience. They said BT Infinity 2 outperformed all Virgin Media packages for upstream jitter by a factor of about five, meaning that Virgin Media customers would see less responsive gaming and less stability. They also said BT Infinity 2 provided faster upload speeds than Virgin Media and that BT Infinity 2 was unbeatable against Virgin Media for latency, downstream jitter and packet loss. They provided a table showing their results.
BT said the above points related to network superiority, so they had also carried out tests relating to the performance of the hub (as 95% of consumers connect wirelessly via their router). They said they had had expert testing carried out on the BT HomeHub 5 and the Virgin Media Super Hub 2ac (the routers provided to fibre and cable customers) in a test house. The test measured the speed of a wireless connection from the routers to devices. BT provided a table of results showing that the HomeHub outperformed the Virgin Media hub in a scenario with three walls dividing the routers from the test devices. They stated the objective of the testing was to show that for gaming, which required a maximum of 4 Mb/s download speed, there was no significant difference in performance between the BT and Virgin routers.
BT provided an academic paper that they stated showed the impact of jitter on gaming, and a graph showing that players' opinion of their experience playing Quake IV dropped continuously for jitter of more than 20 milliseconds. They also provided a table showing the proportion of Virgin Media and BT Infinity 2 customers experiencing jitter of more than 20, 40, 50 and 100 milliseconds, with BT outperforming all Virgin Media line speed options. They also provided a link to a 2011 newspaper article where a Virgin Media representative had said that "millisecond variations in jitter can make a difference in online gaming".
With regard to video calling, BT noted that even HD calls required only 1.5 Mb/s line speeds, and that both BT and Virgin Media therefore offered sufficient speeds to make video calls. They said that upload speed, packet loss and jitter were again the key metrics impacting on video calling, and referred to the data provided in support of the claim regarding online gaming performance.
Clearcast stated that they had been given an Ofcom report from November 2014 outlining measures of jitter, latency, upload speed and packet loss, which they understood measured responsiveness. They said this focused on the quality of user experience, since both BT and Virgin Media offered sufficient speeds for online gaming. They also said that the Ofcom report data provided by SamKnows showed testing carried out on their Wi-Fi network, which 95% of their customers used. They felt that the on-screen text, which made the Wi-Fi testing and basis of the comparison clear, was acceptable. This was implemented as "Comparison: BT Infinity 2 average broadband performance vs Virgin 50, 100 & 152 broadband. Includes wireless and fixed-line performance. To verify see bt.com/more". Clearcast also stated that BT confirmed latency and jitter measured responsiveness. When asked whether there was industry agreement on this measure in the context of gaming, BT said that the official Ofcom report confirmed that they measure responsiveness through those factors and that these are important to gamers. They provided the same data as above.
Clearcast stated that they had requested BT demonstrate that the differences referred to in the report were noticeable. In response, BT had stated that they relied on the SamKnows data showing that they outperformed Virgin Media on jitter and latency, which were critical factors in delivering a responsive gaming experience. They had also said they could break down their data to show average performance data, and referred to the academic paper and press article noted above. Clearcast therefore felt that there was enough evidence to demonstrate that there was a jitter difference in favour of BT to approve the claim alongside the on-screen text clarifying the basis of the comparison.
2. BT said they were aware that claims relating to broadband superiority could mislead if they did not take into account wireless performance, and that in this instance they had included Wi-Fi performance in their testing. They said that to get consistent, repeatable, accurate results when completing testing it was important to have a consistent line performance so that any variation in the line performance did not incorrectly impact the results. They said if the test were to be completed when connected to a broadband line, the speed was more likely to flex up and down throughout the day dependent on the competing demands on the network – a test run in the morning when the network was less busy could deliver a higher speed (e.g. 70 Mb), but in the evening could deliver a lower speed (e.g. 68 Mb), and that the router tested in the afternoon would be unfavourably compared with the result from the morning. As such, they had tested the wireless router connection separately to the fixed-line network speed. They said Ofcom had applied this principle in a recent app launched to measure the quality of home wireless connections.
BT provided SamKnows data for fixed-line broadband speeds. They provided a table stating that BT Infinity 2 had better peak and 24-hour average download speeds than Sky's 38 Mb service and equal speeds to Sky's 76 Mb service. The table also stated that a higher proportion of BT customers achieved at least 80% and 90% of their maximum downloads at peak time (stated to be 8–10 pm) compared to both Sky services.
BT provided the results of a test comparing the BT Home Hub and the Sky SR102 routers, carried out in the same way as outlined above. The results given were for devices in the same room as the router and in rooms with two and three walls dividing the router from the devices. These results showed connection speeds that were all between 344.6 Mb/s and 0.5 Mb/s faster for BT than Sky. BT said that while their fibre access speeds were comparable, the BT Home Hub 5 had a faster average speed than the Sky SR102 router in every room test, as well as a faster average per room across the devices tested. They said that these differences would be noticed by consumers – for example, an HD film downloaded to a tablet on a 76 Mb/s line in a room with a router would take about 3.46 minutes with BT but 6 minutes with Sky. With a division of three walls BT customers could comfortably stream HD TV on a tablet but have a poor SD performance with Sky and that in this room it would also be difficult to do any online activities on more than one device simultaneously with the Sky router. BT stated further that the ad featured the HomeHub prominently, as it formed the centre point between the actress and the cheetah, which made explicitly clear that the claim included wireless performance. They said that unbeatable download speeds married to superior hub performance provided faster download speeds and that the claim was therefore substantiated.
BT said the ad contained on-screen text to clarify that the basis of the "faster downloads" claim was BT Infinity 2. They said the word "offers" was included in the final claims to make clear that BT Infinity had a product that provided faster speeds than Sky, and so avoid the impression that all Infinity products guarantee faster download speeds than all Sky fibre products. Combined with the legal, BT believed this made clear to viewers that they were comparing BT Infinity 2 against the equivalent Sky fibre product.
Clearcast noted that BT had included wireless performance in their claim and provided results for the Wi-Fi test mentioned above. The information from BT had noted that when the Wi-Fi routers were in the same room as the test devices the BT hub average speed was 212 Mb/s, which allowed users to achieve their headline fixed-line speed of 76 Mb/s, whereas the Sky router delivered an average of 59.4 Mb/s, which would impact a user's download speed over a 76 Mb/s fixed line. Clearcast had sought, and received, confirmation that the comparison was based on BT Infinity 2 against Sky's equivalent fibre product and that it was a like for like comparison. They understood that testing had been carried out throughout the day and repeated three times to ensure consistency at longer ranges, and that it had been carried out on devices with a range of wireless specifications. BT's summary of this testing was that more of their customers achieved greater than 90% of their maximum speed at peak time than on Sky's fibre products. BT customers were therefore likely to see less speed degradation at peak times than Sky customers. Clearcast had agreed that there was an advantage from BT's results and advised that the script should not exaggerate this advantage. They felt the data provided was sufficient and acceptable for the claim that BT offered faster download speeds.
3. BT said they made verification information available on their website, the URL of which was included in the on-screen text. This included information about the company that had carried out their fixed-line network texts and outlined how the wireless testing had been done. It stated that BT were superior to Sky regarding the proportion of customers achieving 80% and 90% of maximum speed from 8 pm to10 pm, and reported the wireless router testing results for the laptop and the smartphone. With regard to the statistical robustness of the comparative data, which they understood Sky had concerns about, BT stated that they had used the same approach as Ofcom (in their report about network speed and stability). They said that their claim was built on parity of fixed-line network speed and the superiority of the HomeHub, meaning that confidence intervals were obsolete because their wireless performance was better. They said confidence intervals were only useful in determining a difference if a result were lower (i.e. to determine if it is lower to a significant degree), and that as they hadn’t claimed superiority on their fixed-line network speed – for which they only claimed parity – then there was no need for confidence intervals.
Clearcast stated that they had asked BT to quote the source of their data testing in the on-screen text, along with a reference to where viewers could verify the data.
Assessment
1. Upheld
Ad (a) stated "BT Infinity offers more responsive gaming than Virgin Media" and ad (b) stated "BT Infinity offers smoother video calling than Virgin Media". The ASA considered that viewers would interpret these statements as claims that user experience of online gaming would be more responsive and video calling smoother with BT Infinity services than with Virgin Media services. As both ads prominently featured the BT HomeHub, we considered that consumers would expect the claim to be substantiated in relation to wireless performance, as well as fixed line. We also considered that consumers would expect the difference between the two services to be measurable and noticeable, and reflective of user experience.
We understood that it was only the Infinity 2 service that was reported to have top parity and superiority across the metrics. We noted that the ads stated "BT Infinity offers" and referred to BT Infinity in general in the voice-overs. While on-screen text stated that the comparison was specifically with BT Infinity 2, we considered that this contradicted, rather than clarified, the headline claims because it stated that the claims were only valid for a specific type of service, rather than the general service referred to in the ad. We acknowledged BT's view that use of the word "offers" implied that superior online gaming and video calling would not be available on all Infinity packages, but considered that consumers were unlikely to make this inference and also noted that the claims spoken by the actress did not include conditional language.
The above notwithstanding, we understood that a variety of factors affected the responsiveness of online gaming, with some of these also relating to smooth video calling, and considered that the evidence provided, BT had largely addressed these. We noted that BT Infinity 2 was stated to be superior to, or share top parity with, Virgin Media for the key indicators of online gaming and video performance, and that such results would be understood as superior performance. However, we were concerned that for some measures they had just stated that BT Infinity 2 and Virgin Media shared top parity but had not provided the actual test results. For those metrics where the results were given, BT had not stated the unit measurements and it was therefore not clear what the difference was between BT and Virgin Media's performance. We had also not been provided with details of the testing methodology and therefore could not determine whether the data regarding network performance was sufficiently robust to substantiate the claims.
With regard to the wireless hub connection speed data, we noted that the results showed a faster connection from the devices to the BT HomeHub than to the Virgin Media Super Hub. However, we were concerned that we had only been given the results relating to a room with three walls between the router and the device, and therefore could not determine whether the BT HomeHub was faster in other situations. We considered that it was not unlikely that consumers would use their devices with less separation than three walls between them and the router, and that this information was therefore of particular note. BT had also not demonstrated that the two devices tested (a high specification laptop and a smartphone) were representative of those typically used by consumers in terms of their Wi-Fi specifications, or that the arrangement of the rooms in the test house (including how they were furnished, and whether with items that may affect Wi-Fi signals) was typical of a UK household environment.
Nevertheless, we understood that BT had tested their service in two sections; first by testing the properties of the fixed-line network, and then by testing the speed of wireless connections to the router. We considered that consumers would understand the claims to relate to an internet connection from the telephone network to an internet enabled device, including when the device was connected to the router via Wi-Fi. We noted that BT had not tested the actual internet connection of devices connected via Wi-Fi, and had instead only tested the speed of the local network connection. We understood that use of Wi-Fi could affect the amount of jitter and packet loss experienced, apart from the upload and download speed of the connection, and that these metrics should therefore have been measured for a Wi-Fi service as well as fixe line. We also understood that the connection between the network cabinet and the router could introduce factors such as delayed upload and download speeds and increased problems with jitter and packet loss. Because we had not seen the testing methodology for the line speed test, it was not possible to determine whether this had been accounted for in BT’s evidence. The speed and quality of the entire connection from the network to the device would be positively correlated with the individual speed and quality of these elements, but BT had not demonstrated that the entirety of their connection was faster and of higher quality for gaming or video calling than provided by Virgin Media.
Finally, we were also concerned that BT had not demonstrated the effect of these differences on end users of the service during online gaming and video calling. We understood that when some metrics (such as upload speed) were sufficiently high-quality, further improvement would not convey a noticeable difference to users of the service. We noted that BT reported higher upload speeds and less upstream jitter than those offered by three Virgin Media packages (with the other metrics being matched), but they had not demonstrated that this was within a range where the differences reported would make a noticeable improvement. In relation to upstream jitter, BT's data did not show the unit measurements, although we understood that it was commonly measured in milliseconds and assessed the evidence on this basis. We noted the academic paper stating that perception of gaming experience declined rapidly when jitter was more than 20 milliseconds (ms), and that one table provided by BT showed a higher proportion (between 7% and 53%, depending on package type and amount of jitter) of Virgin Media customers experiencing 20, 40, 50 and 100 ms jitter than BT. However, we understood from another of BT’s data tables that all the Virgin Media packages reportedly featured upstream jitter of 4.41ms or less. As such, we did not consider that this data in itself demonstrated poorer online gaming or video calling performance on the Virgin Media service. We also noted the 2011 press quote from Virgin Media, but did not consider that this constituted documentary evidence about the actual performance of the current service. In light of the above points, we considered that the claims "BT Infinity offers more responsive gaming than Virgin Media" and "BT Infinity offers smoother video calling than Virgin Media" had not been adequately substantiated and therefore concluded that ads (a) and (b) were misleading.
On this point, ads (a) and (b) breached BCAP Code rules
3.1
3.1
Advertisements must not materially mislead or be likely to do so.
and
3.2
3.2
Advertisements must not mislead consumers by omitting material information. They must not mislead by hiding material information or presenting it in an unclear, unintelligible, ambiguous or untimely manner.
Material information is information that consumers need in context to make informed decisions about whether or how to buy a product or service. Whether the omission or presentation of material information is likely to mislead consumers depends on the context, the medium and, if the medium of the advertisement is constrained by time or space, the measures that the advertiser takes to make that information available to consumers by other means.
(Misleading advertising),
3.9
3.9
Broadcasters must hold documentary evidence to prove claims that the audience is likely to regard as objective and that are capable of objective substantiation. The ASA may regard claims as misleading in the absence of adequate substantiation.
(Substantiation), and
3.33
3.33
Advertisements that include a comparison with an identifiable competitor must not mislead, or be likely to mislead, consumers about either the advertised product or service or the competing product or service.
(Comparisons with identifiable competitors)
2. Upheld
The ASA considered that, as the BT HomeHub was shown prominently in the ad, consumers would understand the claim “faster download speeds” to relate to an internet connection from the telephone network to an internet enabled device, including when the device was connected to the router via Wi-Fi. We understood that BT had compared their 76 Mb/s Infinity 2 service with Sky’s 76 Mb/s service and that the claim was made on this basis. As before, we noted that the ad stated "BT Infinity offers" and referred to BT Infinity in general in the voice-overs, and considered that consumers were unlikely to understand the word “offers” as meaning that superior download speeds would not be available on all BT Infinity packages. We considered that the on-screen text referring to Infinity 2 contradicted, rather than clarified, the headline claims by referring to a specific type of Infinity package, rather than the general services under this title.
This point notwithstanding, we considered the evidence provided by BT. As previously, we noted that some data for the Wi-Fi connection test was missing (in this instance, relating to a one-wall division between device and router) and that a full testing methodology had not been provided for either the Wi-Fi test or the fixed-line network test. We also noted that BT had not provided information to demonstrate that the four devices used for the Wi-Fi test (a high-specification laptop, a smartphone, a tablet computer and a receiver dongle) were representative of those generally used by consumers. As such, we considered that it was therefore not possible to determine whether the results were sufficiently robust to substantiate the claim.
We understood that BT had tested the speed of their fixed-line network, and that of Sky’s 76 Mb/s service, and concluded that Infinity 2 was as fast as Sky. They therefore believed that the faster connection speeds to their router meant that they had demonstrated that BT Infinity could offer faster download speeds to the consumer because this faster Wi-Fi connection would lead to faster downloads overall. We did not consider that consumers would understand from the ad that the comparison was based on parity of fixed-line performance in conjunction with superior router connection speed. We also understood that BT had not tested the whole connection from the network to the device (including via Wi-Fi) and that factors such as cabling between the cabinet and the device, and the ability of the router to account for environmental interference and obstacles had not been addressed. We also understood that for some devices and situations (such as a same-room connection to a high-specification laptop) the connection speeds for both routers were considerably higher than the bandwidth that the service would provide, and that there may therefore be no perceptible difference to consumers. Our other concerns regarding the missing data and methodologies notwithstanding, and although we acknowledged that a faster router connection could confer a positive benefit on the user, we considered that combining the results of a fixed-line network test with the speed at which devices connected to a Wi-Fi router was not a robust method to determine which service provided faster download speeds to the end user. We noted BT’s concerns that testing the whole connection at once would be difficult because of fluctuating bandwidth use at peak times, but considered that this did not negate the need for an alternative testing methodology to be robust in itself.
We also noted BT’s assertion that a greater percentage of their customers achieved more than 90% of the maximum line speed at peak time, compared with Sky. We considered that, although this data related to the speed achieved by consumers, the degradation (or lack thereof) of line speed during peak hours was a fundamentally different issue to the matter of which provider offered faster download speeds in general. We were concerned both that this data related to speed degradation and not to speed capability, and that it referred to peak hours only, rather than the overall performance of the service. Taking the above elements into account, we considered that the claim "BT Infinity offers faster download speeds than Sky" had not been adequately substantiated and therefore concluded that ad (c) was misleading.
On this point, ad (c) breached BCAP Code rules
3.1
3.1
Advertisements must not materially mislead or be likely to do so.
and
3.2
3.2
Advertisements must not mislead consumers by omitting material information. They must not mislead by hiding material information or presenting it in an unclear, unintelligible, ambiguous or untimely manner.
Material information is information that consumers need in context to make informed decisions about whether or how to buy a product or service. Whether the omission or presentation of material information is likely to mislead consumers depends on the context, the medium and, if the medium of the advertisement is constrained by time or space, the measures that the advertiser takes to make that information available to consumers by other means.
(Misleading advertising),
3.9
3.9
Broadcasters must hold documentary evidence to prove claims that the audience is likely to regard as objective and that are capable of objective substantiation. The ASA may regard claims as misleading in the absence of adequate substantiation.
(Substantiation), and
3.33
3.33
Advertisements that include a comparison with an identifiable competitor must not mislead, or be likely to mislead, consumers about either the advertised product or service or the competing product or service.
(Comparisons with identifiable competitors).
3. Upheld
Advertisers should provide the means for consumers and competitors to be able to verify comparative claims by checking the information for themselves, including by replicating the tests if they wished and if appropriate. We noted that the page referred to in the ad included details of the methodologies used and provided tables of results for some metrics, and we therefore acknowledged that BT had taken steps to make the information supporting their comparisons available to viewers for verification. However, the page provided only basic details of how the Wi-Fi router test was set up, and did not include information such as the layout of the test house or details of potential interference (such as furniture). Although the page stated that SamKnows had compiled a report of line speeds, no information was given as to how this third party obtained the information including, for example, the sample size used. As such, we were not satisfied that sufficient information was given as to the methodologies used to allow viewers to verify the accuracy of the claims.
The above notwithstanding, we considered that for some claims, information about methodology alone would be insufficient for viewers to verify the comparison, such as when data taken into account by the advertiser when making the comparison was not readily available or was (at least in part) historic and might be subject to fluctuation, and that in these instances data should be made available. We considered that technical data about fixed-line networks (including speed) would not be generally available to those viewing the ad and that, given that such testing could be subject to external interfering factors, BT had not demonstrated that the Wi-Fi router test was readily replicable. As such, sufficient results data should have been made available for viewers to verify the accuracy of the comparative claim. Results were given for some aspects of the Wi-Fi router test, but not for all room scenarios, and no results data was given for the comparison of the fixed-line network speeds.
We understood Sky were particularly concerned that the verification information available did not provide the appropriate statistical confidence intervals for BT’s results data. We noted BT’s assertion that these were irrelevant because they were claiming parity with Sky on line speed, and superiority for router connectivity. We understood that information about confidence intervals or margins of error would not be applicable to the Wi-Fi test, which assessed the performance of devices with largely consistent performance. However, it was relevant for comparing the performance of two ISPs when the data in question was formed of many individual data points (such as the line speed or quality for thousands of individual customers), regardless of whether the performance of one party was higher, lower, or equal to the other. We understood that it could provide information about where true comparative values were likely to lie and that without this data it may not be possible to accurately verify a comparison between two data sets, particularly when the results were close. We understood that the fixed-line comparison used data of this type and that, even if the mean line speeds had been provided by BT, without information about the confidence interval or margin of error the comparison would not have been verifiable. As such, in this case the data required for viewers to adequately verify the comparison should have included confidence intervals for the relevant data sets (or sufficient means to calculate them). We considered that, because not all the necessary data and results were available, the information provided was not sufficient to ensure the details of the comparison could be verified by consumers and competitors, and concluded that the ad breached the Code.
On this point, ad (c) breached BCAP Code rule 3.35 3.35 Advertisements must objectively compare one or more material, relevant, verifiable and representative feature of those products or services, which may include price. (Comparisons with identifiable competitors).
Action
The ads must not appear again in the form complained about. We told British Telecommunications plc not to repeat the claims in the ads unless they held documentary evidence to substantiate them and made available sufficient information about their methodology and data for verification.
BCAP Code
3.1 3.10 3.11 3.12 3.2 3.33 3.35 3.9
