Are mass media campaigns effective in reducing drinking and driving? Systematic review – an update

INTRODUCTION

Road-traffic injuries are becoming a growing issue globally, particularly in low- and middle-income countries [1]. The leading cause of road-traffic injuries worldwide is alcohol-impaired driving (AID) [2]. According to the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) transport-related deaths attributed to alcohol use numbered amounted to 5,000 in the year 2010 in the USA alone [3]. Data from the USA shows that of all the persons killed in traffic crashes approximately one-third die due to AID [4]. The World Health Organisation estimates that in some countries the proportion of road traffic deaths involving alcohol consumption is as high as 50% [5].
In an attempt to tackle this mounting death toll, in the past several decades policy-makers have increasingly relied on mass media campaigns [6]. These campaigns involve delivery of educational messages through one or more media channels, such as television, radio, or newspapers, but also billboards, websites, or electronic social media [7]. The recurring message themes in mass media campaigns aiming to reduce AID included (a) fear-evoking approaches (e.g. descriptions of, and imagery displaying bodily harm, death, disability, or arrest), (b) stigmatisation of AID behaviours, and (c) positive approaches (promotion of positive social norms) [6].
However, despite over six decades of research, little consensus exists regarding the efficacy of such interventions [8]. First of all, the optimal amount of anxiety that such campaigns should produce in order to maximise their impact is not clear [6]. In addition, the mass media campaigns against AID have been theorised to be most successful when accompanied by additional reinforcing efforts, such as law enforcement or grassroots activities, and they also tend to be implemented alongside such interventions [9]. The variety of approaches taken by these accompanying interventions, and the difficulty of controlling for their individual effects, makes it difficult to isolate and evaluate the importance of the mass media campaigns at addressing AID. A further challenge for the synthesis of the literature is the diversity of outcomes that can be used to assess such interventions, including self-reported AID, or objective measures, such as hospitalisation rates [7].
Several systematic reviews of effectiveness of mass media campaigns to tackle AID have been conducted (see Appendix A for details of search strategy used to identify previous systematic reviews). The scope of these reviews has varied, with some focused either on alcohol-involved crashes in particular [6, 10], while others on road safety in general, but including a discussion of drink-driving [7, 9]. These reviews have not reached a unanimous agreement on the effectiveness of mass media campaigns for preventing AID. Instead, they have pointed at low volume and poor quality of the existing research and recommended more high-quality studies.
A recent comprehensive meta-analysis of 67 studies assessed the effect of road safety campaigns on accidents, including campaigns against AID [7]. Despite this broad focus, a large part of the review was devoted to investigating how message delivery via mass media channels impacts the effectiveness of interventions against AID. It has, however, only included studies published up until 2007. To the authors’ knowledge no systematic review exists on the studies published in the period between 2008 and 2014. The negative health consequences of AID remain a growing problem. Therefore, the principal motivators behind conducting the present systematic review was to, first, address the gap in literature and synthesis of more recent findings, and second, to provide further recommendations to policy makers regarding mass media campaigns targeting AID.

MATERIAL AND METHODS

The aim of this systematic review was to update the review conducted by Phillips and colleagues (published in 2011) [7], with a focus on assessing whether mass media campaigns are helpful in preventing AID.

INCLUSION CRITERIA

Figure 1 shows the flow chart of study inclusion. In order to be included in the present review, studies had to fulfil the following conditions:
1. Report on mass-media campaigns promoting road safety behaviours, necessarily including messages targeted at reducing AID.
2. Provide data measuring outcomes related to AID (alcohol-related traffic crashes, alcohol-related traffic fatalities, single-vehicle night-time crashes, or self-reports of AID).
3. Be articles reporting on original research published in peer-reviewed journals between 31.12.2007 and 13.03.2014.
Self-reported AID was ultimately chosen as the primary measure as it was the only measure clearly related to the outcome of interest – AID – recurring in all studies identified. It was supplemented by more objective secondary measures such as drink-driving accidents or single vehicle night time collisions whenever these data were available.
Due to the heterogeneous nature of mass-media and road safety campaigns no limits were set on the type of design of studies included. In order to overcome language bias no limit was set on the language in which the studies were published. Finally, due to the fact the problem of AID affects populations of a broad spectrum of ages and backgrounds, the inclusion criteria were not limited to any particular age groups, socio-economic groups, ethnic groups, or to a single gender.
While most mass media campaigns were tied to AID prevention programmes, an effort was made to avoid further confounding their effect by only including studies which maintained that no additional AID prevention activities took place in the geographical region they targeted throughout the duration of the assessed mass media campaign. This excluded mass media campaigns that coincided with changes in laws or unrelated changes in enforcement levels.
It has been suggested in previous systematic reviews that mass media campaigns in order to be effective need to ensure that the target audience is exposed to the messages with enough frequency [6]. Only paid or commissioned campaigns can allow for the optimal timing and placement of media messages that permit to control this frequency. Therefore this review excluded studies looking solely at the impact of ‘earned media’, media reports of celebrity drink-driving accidents, or news stories on high-profile AID crashes, not accompanied by paid media campaigns [2].

SEARCH STRATEGY

A systematic search was conducted for peer-reviewed journal articles in Medline and Embase databases. The search period covered dates between 31.12.2007 and the date of the search – 13.03.2014. Subject headings, keywords and phrases in titles and abstracts that reflected the inclusion criteria were used, including “media”, “television”, “news”, “road injur*” and “alcohol*” (see Appendix B, for search strategy details). The same terms were used for both Medline and Embase. This was supplemented with a manual search through the reference sections of included studies. The studies whose abstracts fulfilled the eligibility criteria were further screened by a detailed assessment of their full text. Grey literature was not included in the search strategy.

CRITICAL APPRAISAL OF INCLUDED STUDIES

A study-by-study quality assessment of included papers was conducted using a check-list developed by MZ and adapted from a set of criteria from the Guide to Community Preventive Services (Community Guide) [11]. Some of these criteria were used in previous systematic reviews of the effectiveness of mass media for reducing AID [6]. The quality checklist included nine items against which each study was assessed. Each item could be scored from 0 to 2 points, resulting in a quality scale ranging from 0 to 18 per study (see Appendix C for details). Criteria of evaluation included:
1. Quality of reporting – were clear and adequate descriptions provided of the study population, the campaign delivery method and content, the cost of the campaign, the response rate and outcome variables?
2. Quality of campaign – was pretesting used to assess if campaign messages would be relevant to the target audience? Were adequate levels of audience exposure ensured?
For studies reporting multiple outcomes, all measures relating to AID were reported, but in each case self-reported AID was treated as the primary outcome.

DATA EXTRACTION METHODS

Data was extracted using an adapted version of the form used by Elder and colleagues [6]. The information collected included author and date, participants (sample size, age), study design, study setting, details of the intervention (scope, theme, cost, delivery method, duration), and findings on relevant outcomes. The extracted information is presented in Table 1.

DATA SYNTHESIS

Due to the heterogeneity in the methodology of the different mass-media campaigns assessed and the interventions accompanying them the descriptive information had to be grouped under broad headings such as “message theme” or “delivery method”. Similarly, due to a lack of standard measures regarding AID, the primary outcome – self reported AID – synthesised such diverse answers of study participants as ‘driven within 2 hours of drinking’ [12], ‘driven after having two or more drinks’ [13] or ‘driven after drinking too much’ [14]. For secondary outcomes, proxy measures for AID crashes such as single night-time motor accidents were combined with more direct measures, such as traffic crashes and traffic fatalities, where the drivers were positively tested as driving under the influence of alcohol. A meta-analysis was not attempted due to the lack of standardised effect sizes and confidence intervals, but the benefit achieved by various campaigns was illustrated using an effect direction plot. See Table 1 for details.

RESULTS

The systematic search yielded 319 studies. Three hundred and two papers were excluded based on a screening of titles. Of the remaining 17 studies, 10 were excluded after screening their abstracts. Full texts of the remaining 7 studies were retrieved to assess their eligibility, and a further 3 were eliminated. The four remaining studies meeting all the inclusion criteria were included in the present review [12-15]. See Figure 1 for details.

QUALITY ASSESSMENT

The identified studies were generally of a moderate quality, scoring between 8 and 14 out of 18 on the quality checklist used in the review (see Appendix C). All studies reported their study populations adequately, and most described the mass media campaign content in detail. On the whole the studies scored poorest on reporting campaign cost. Only two out of four assured that the mass media campaign had adequate audience exposure. Perkins et al. (2010) was the study with the highest summative score, but it did not pre-test the mass media campaign before launch [13].

SETTING AND POPULATIONS

All selected studies assessed media campaigns conducted in different parts of the United States, thus limiting how representative and generalisable their results can be to other countries. The number of participants surveyed on their AID behaviours ranged from 218 [13] to 1,725 [12] (see Table 1). The samples included college undergraduate students [15], a predominantly educated urban population [14], and heterogeneous populations of two different states [12, 13].

CHARACTERISTICS OF MASS MEDIA CAMPAIGNS EVALUATED

All of the mass media campaigns, apart from Perkins et al. [13], were used as reinforcement for various road safety interventions they accompanied. These interventions included state-wide sobriety checkpoints [12], environmental management at a single university campus [15], or location of taxi stands outside bars throughout a city [14]. The motivational themes of the analysed mass media campaigns included fear of arrest and legal consequences of arrest [12, 15], promotion of positive social norms [13], and encouraging specific behavioural patterns, such as use of taxi services [14]. The media campaigns differed in the level of pre-planning involved, quality of execution, and the audience exposure they attained. The campaigns focused predominantly on delivering their message using traditional mass media channels – television, radio, press, and billboards (see Table 1). Only Rivara et al. [14] attempted to also use the internet and electronic social media, however it reported little success in doing so (low number of ‘clicks’).

OUTCOMES

The studies assessed a number of measures related to AID (see Table 1). All four included self-reported AID as their measure, and this was selected as the primary outcome for the present review. Three studies measured their primary outcome of interest through random-sample, before-and-after telephone surveys [12-14]. However, the follow-up rates for the surveys were generally low and only Rivara et al. [14] managed to attain a follow-up rate of over 50%. Wood et al. [15] compared results from a telephone surveys conducted among students from a campus that was exposed to alcohol prevention intervention with those from a control campus. In addition, Beck [12] also measured alcohol-related crashes and traffic fatalities in its state of interest, Perkins et al. [13] measured alcohol-related crashes in its state of interest, and Rivara et al. [14] measured single vehicle night-time crashes in its city of interest. These data were all based on National Highway Traffic Safety Administration statistics and were included in the review as secondary outcomes. Only Wood et al. [15] did not measure a secondary objective outcome directly relevant to AID.
Across the four studies, the mean decrease in self-reported AID was about 2%. The studies with the highest scores achieved in the quality appraisal, and with the highest mean age of respondents, were also the ones which witnessed the highest percentage decreases [13, 14]. The campaigns conducted as part of those two studies were based on positive messages, while Beck [12] and Wood et al. [15] attempted to change behaviours through fear of enforcement.

DISCUSSION

The present systematic review has supported the findings of the recent reviews conducted by Phillips et al. [7] and Elder et al. [6]. The reviewed studies indicate that under some conditions, and with careful design and good execution, and preferably with a focus on positive messages, mass-media campaigns can successfully contribute to the reduction in AID.
Nonetheless, the main challenges in evaluating the effectiveness of these campaigns is the heterogeneity of these interventions, as well as the lack of success in isolating their effects from the AID prevention activities that they accompany. In order to ascertain the impact of specific types of mass media campaigns future studies will need to evaluate them in the context of the same road safety interventions. Moreover, with the changing technological landscape more attention should also be paid to novel mass media, such as websites and social media, which are currently under-researched. More research is also needed on whether the impact of mass media campaigns accompanying road safety programmes can have lasting effects on AID.
Finally, the studies included in this review have been conducted in the USA, thus research in other countries would be desirable. The present review also pointed to methodological shortcomings in research on mass media campaigns, and especially the over-reliance on self-reported outcomes assessed through telephone follow-up. Where possible, self-reports should be supplemented by more objective measures, for example the rates of AID-related accidents in the areas where the mass media campaigns were conducted. Finally, it must be remembered that the influence of publication bias in reporting might mean that the positive effect of the small studies is overestimated. In this review the only study showing a negative result was the one that surveyed the largest amount of participants [12].

LIMITATIONS

The present manuscript has several limitations. A rigorous and comprehensive search strategy was used, but it only included studies published between 01.01.2008 and 13.03.2014, to supplement the review published by Phillips et al. [7]. Grey literature was not included, and only two databases were searched. Additionally, after much deliberation and due to the lack of a different measure shared by all identified studies, self-reported AID, a subjective measure, was chosen as the primary outcome – a more objective measures should be striven for in the future. Furthermore, only MZ has reviewed, selected, and assessed the reported studies and extracted the data. However, the author used transparent and standardised procedures.

CONCLUSIONS

Alcohol-impaired driving remains an important public health concern, and mass media campaigns have the biggest potential to reach and influence behaviour of the wider population. Nevertheless, some of the methodological shortcomings and challenges that characterise research on mass media campaigns and AID, and their focus on US-based populations, indicate that further rigorous studies in this area are needed.

DISCLOSURE

Authors report no conflict of interest.

References

1. Peden M, Scurfield R, Sleet D, et al. (eds.). World report on road traffic injury prevention. World Health Organization, Geneva 2004.
2. Nakahara S, Ichikawa M. Effects of high-profile collisions on drink-driving penalties and alcohol-related crashes in Japan. Injury Prevention 2011; 17: 182-188.
3. Lim SS, Vos T, Flaxman AD, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012; 380: 2224-2260.
4. Department of Transportation (US). Traffic Safety Facts. National Highway Traffic Safety Administration, 2010.
5. World Health Organization. Global status report on road traffic safety: time for action. World Heart Organization, Geneva 2009.
6. Elder RW, Shults RA, Sleet DA, et al. Effectiveness of mass media campaigns for reducing drinking and driving and alcohol-involved crashes: a systematic review. Am J Prev Med 2004; 27: 57-65.
7. Phillips RO, Ulleberg P, Vaa T. Meta-analysis of the effect of road safety campaigns on accidents. Accid Anal Prev 2011; 43: 1204-1218.
8. Delaney A, Lough B. Whelan M, et al. A review of mass media campaigns in road safety. Monash University Accident Research Centre, Victoria 2004.
9. Elliott B. Road safety mass media campaigns: A meta analysis. Federal Office of Road Safety, Canberra 1993.
10. Haskins JB. The role of mass media in alcohol and highway safety campaigns. J Stud Alcohol Suppl 1985; 10: 184-191.
11. Zaza S, Wright-De Agüero LK, Briss PA, et al. Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med 2000; 18 (1 Suppl): 44-74.
12. Beck KH. Lessons learned from evaluating Maryland’s anti-drunk driving campaign: assessing the evidence for cognitive, behavioral, and public health impact. Health Promot Pract 2009; 10: 370-377.
13. Perkins HW, Linkenbach JW, Lewis MA, Neighbors C. Effectiveness of social norms media marketing in reducing drinking and driving: A statewide campaign. Addict Behav 2010; 35: 866-874.
14. Rivara FP, Boisvert D, Relyea-Chew A, Gomez T. Last Call: decreasing drunk driving among 21-34-year-old bar patrons. Int J Inj Contr Saf Promot 2012; 19: 53-61.
15. Wood MD, Dejong W, Fairlie AM, et al. Common ground: an investigation of environmental management alcohol prevention initiatives in a college community. J Stud Alcohol Drugs Suppl 2009; 16: 96-105.