Before making product claims… 5 lessons learned from the NAD

---

Most personal care product companies know that a study used to substantiate advertising claims must possess certain controls.  For example, the study must test products that are substantially equivalent to the products in the marketplace, include product use instructions that are identical to the label instructions, and focus on a study population that will produce results applicable to the target population for the claims used in advertising.[1]

A review of recent advertising challenges before the National Advertising Division of the Better Business Bureau (NAD), however, indicates that fewer companies understand how to design a questionnaire that will ensure there is adequate support for claims regarding consumer opinion of products.

Examples of claims for which a study would necessary include “Women prefer [x],” “[y] product makes my hair feel softer,” and use of consumer testimonials regarding a consumer’s personal experience with a product pulled from social media that a company would like to disseminate in a company-sponsored advertising campaign.In 2014, many claims regarding consumers’ opinion of their sensory experiences with products that were reviewed by the NAD were found to be insufficiently substantiated for use in advertising.

Often, the advertiser provided a sensory study as support for the claims, but the NAD determined the survey instrument contained methodological flaws that compromised its probity.

This article identifies common mistakes and provided recommendations for companies seeking to support sensory claims in testimonials. The National Advertising Division
The National Advertising Division is an arm of the Advertising Self-Regulatory Council (ASRC) of the Better Business Bureau.  NAD administers an industry-sponsored self-regulatory program that reviews national advertising for truthfulness and accuracy, typically applying the same standards employed by the Federal Trade Commission and courts when reviewing advertising claims.

Responsible companies often look to NAD decisions as benchmarks to help the company to assess the business and  legal risks associated with marketing practices.Sharing Consumer Opinion In Advertising
Federal law prohibits advertisers from making claims about consumer opinion, including through consumer testimonials, that could not be substantiated if made directly by the advertiser.[2]

As recently explained by NAD:
[i]t is not enough that a [claim or] testimonial represent the honest opinion of the endorser.

Advertisers must also have appropriate scientific evidence to back up the underlying claim. Indeed, it is a basic principle of advertising law that in the absence of competent and reliable scientific evidence, product performance claims cannot be substantiated by anecdotal evidence such as consumer testimonials.[3]NAD recognizes questionnaires that ask subjects for their opinions regarding a test product (“self-assessment questionnaires”) as a valid method to assess consumers’ subjective evaluations of a product, provided the questionnaire is appropriately designed to provide a good fit between its results and the scope and nature of the claims in advertising.[4]

Thus, companies often rely on subject responses to self-assessment questionnaires provided through sensory studies to substantiate sensory claims in advertising.Five Lessons Learned From NAD in 2014
Based on 2014 NAD decisions involving sensory studies, advertisers should be sure to follow these five basic guidelines:

1. Read the ASTM Guide on Sensory Claim Substantiation (ASTM Guide).[5] The ASTM Guide is an industry standardized protocol that provides helpful guidelines for designing sensory studies, including the questionnaires. NAD defers to the ASTM guidelines and, along with many other authoritative bodies such as the FTC, often considers compliance with ASTM recommendations to provide a safe harbor.2. Place the questions of most interest to the company early in the questionnaire, while limiting the number of questions unrelated to the desired advertising claims. The ASTM Guide advises that key questions relating to advertising claims should be positioned early in a questionnaire to ensure they are “free of influences of other questions and most defensible under scrutiny.”[6] It also recommends that questions that are not needed to substantiate desired advertising claims be excluded from the questionnaire because exclusion “will preclude any potential biasing effect of any one question on any other question.”[7] NAD applied these guidelines consistently in 2014.[8]

3. Refrain from asking substantially similar questions regarding the same product attribute. NAD has rejected studies that contain “multiple and similarly-worded questions” regarding the product attribute touted in advertising,[9] especially where the questions precede the one upon which the advertiser ultimately relies for its adverting claim. NAD’s primary concern is that the preceding questions regarding the attribute may “improperly influence the answers” to the question relied upon by the advertiser.   The ASTM Guide discusses similar concerns, while also warning that “asking about an attribute in more than one way increases the risk of results that could be inconsistent, for example a difference in preference without a difference in liking.”[10]

4.     Exclude “double-barreled” questions. A double-barreled question is a compound question asking a respondent about more than one attribute, simultaneously (e.g.,  “does this product leave your skin soft and smooth” in lieu of separately asking “does this product leave your skin soft” and “does this product leave your skin smooth”). While double-barreled questions may shorten a questionnaire, they are insufficient to substantiate advertising claims. NAD reiterated this principle last spring, when it issued a decision noting that an advertiser’s consumer study was “unreliable” because, among other things, “the participants were asked a double-barreled question for their rating, combining two attributes into one question.”[11]

5.     Collect participant responses at each time period that marketing desires to  reference in advertising.  Questionnaires that ask consumers to recall product experiences at time points earlier than when they’re provided the questionnaire can lead NAD to question the test’s ability to ensure that participants accurately recalled their opinions.  For example, in one decision, NAD determined that the advertiser’s questionnaire was insufficient to substantiate several of the advertiser’s claims because the participants’ answers could have been affected over time by “faded memory.”[12]  The same decision offered ways to mitigate the risk of faded recall in future studies noting that the advertiser could have (1) provided participants with something to record their thoughts throughout the study, (2) encouraged them to record their thoughts contemporaneously, and (3) collected any written notes to confirm that participants’ answers matched their contemporaneous sentiments.  According to NAD, “such actions would have helped the subjects in accurately answering their surveys, allowed the advertiser to verify the accuracy of its survey, and avoided the potential bias [from]… exposing the subject to the ultimate question they will be answering prior to completing the product usage.”[13]

Failure to adhere to these lessons could prevent NAD from giving much weight to a study offered as support for use of consumer testimonials in an advertising campaign.  Embracing them can help protect a company’s investment in a sensory study, while enabling it to capitalize on the benefits of effectively (and legally) sharing consumer testimonials with prospective consumers.