Campaign pretesting

Campaign pretesting: a new approach

Over Checon’s 25 years in the market, one of the qualitative studies we have run most often is the advertising campaign pretest, whether for films, concepts or print ads

Judging by the volume of work and the many agencies and advertisers who have come back to us for other projects over the years, Checon’s analyses and results seem to be on the right track

Like most other firms, we have generally used focus groups, usually segmented by age group, socioeconomic class and, most of the time, by region

For smaller evaluations, we typically run around 9 groups, and for many larger national campaigns it’s not unusual to need more than 20 focus groups

Using edited audio clips of participants’ own statements and opinions was an innovation Checon developed, and it added a great deal of information and understanding to the comments and suggestions we present as recommendations

Besides being highly persuasive, the audio clips always made the testimonials feel real and more credible, adding emotion and humor and making the presentations far richer

So much so that, time and again, agencies’ and advertisers’ attention focused almost entirely on the audio clips themselves, given how powerful the messages were, coming straight from the target audience

But in many of these pretests, it still wasn’t always possible to get a completely clear, unambiguous answer to some of the subjective or emotional questions about the campaign concepts

The overall sense was that a more suitable, reliable metric was still missing, since focus groups alone couldn’t always clear up every question advertisers and agencies had

Questions that didn’t always get satisfactory answers included the campaign’s impact on viewers, how far certain positioning was actually perceived as “safe” or “trustworthy,” and even which message was really getting through

On an exploratory basis, we tried complementing focus group evaluations with individual video recordings of participants to assess their reactions through changes in facial expression, but without much success, since the changes were barely noticeable

We also tried other biometric measurement techniques such as GSR (galvanic skin response), used abroad to predict box office results for films and plays, but the results fell short of expectations and were hard for non-specialists to understand

At international conferences, both ESOMAR’s and the NMSBA’s, we have constantly looked for new options that could add further evidence on these questions in a way that is transparent and, above all, clear to agencies and advertisers

This determination at Checon Pesquisa to innovate and broaden our toolkit, to better understand how people react to advertising campaigns, led us to implicit association tests (implicit tests)

Implicit association tests were originally developed to explore the unconscious roots of human thoughts and emotions, since people often find it hard to express their emotions or don’t even know exactly what’s going on in their own heads

These problems get worse the lower people’s level of education, or when they’re shy, and political correctness also tends to creep in fairly often, which can compromise the evaluation

The case for using implicit association tests in pretests lies in their ability to measure responses, ideas and beliefs at a subconscious level, free from rationalization that isn’t always helpful

The results of implicit association tests are also easy to interpret, with no knowledge of psychology required, and flexible enough to dig into the specific aspects of each campaign

Under Checon’s methodology, the tests take place after the focus groups, which serve as the basis for building the test protocol by identifying which questions remain unresolved

Checon recommends running implicit association tests with the same participant profiles: a control group of people who haven’t seen the campaign and a second group who take the test right after seeing the material (film, spot or print ads)

This creates a very clear basis for comparison to assess the impact of the aspects the focus groups didn’t fully clarify

Each implicit association test typically takes 3 to 5 minutes and can evaluate up to 5 different attributes, which is usually more than enough for the projects we handle

Checon recommends a minimum of 30 tests, usually 15 with the control group and 15 with participants who watch the same material shown and tested in the focus groups

This number can change in specific situations that call for more detailed analysis (such as results by class or age group), or when comparisons with competitors are also needed.

One advantage is that far fewer focus groups are needed, avoiding the repetition that usually sets in after the 10th group and focusing effort on measuring very specific aspects

Another is having a tool that can assess beyond the conscious and rational, with metrics that, while not claiming to be quantitative research, complement the focus group results

The result is savings in both processing and analysis time and in cost, since implicit association test results are available immediately