Standards for Claim Substantiation Surveys

“What are the standards for a claim substantiation survey?” Clients often ask this question when they are contemplating a claim substantiation survey in general. The question comes up during discussions about advertising claim substantiation or about claims used in packaging, on point of sale materials, or during sales presentations. I am also asked this question when discussing litigation matters involving issues such as false or deceptive advertising.

Often, the question is asked in search of a clear standard that will describe exactly how to conduct a claim substantiation survey or a litigation survey. This article presents some of the sources I consult more commonly for matters involving claim substantiation, substantiation surveys, and litigation surveys on topics such as deceptive advertising.[1] After listing the sources, I will provide some general thoughts regarding the standards for designing and executing claim substantiation surveys and litigation surveys.

The Federal Trade Commission
The FTC has issued a number of policy statements relating to claim substantiation. For example, see the “FTC Policy Statement Regarding Advertising Substantiation.” This statement, among other things, states that advertisers must “…have a reasonable basis for advertising claims before they are disseminated.” The “reasonable basis standard” is well-known and depends on a variety of context specific factors, such as “the type of claim, the product, the consequences of a false claim, the benefits of a truthful claim, the cost of developing substantiation for the claim, and the amount of substantiation experts in the field believe is reasonable.”

The Federal Judicial Center
The Federal Judicial Center has published the Reference Manual on Scientific Evidence. The manual includes the “Reference Guide on Survey Research.” Written by Professor Shari Seidman Diamond, the Reference Guide outlines a series of criteria for evaluating survey research that is used in court. The document raises a series of questions to consider relating to topics such as the purpose and design of the survey, population definition and sampling, and survey questions and structure. The document provides helpful guidance on survey-specific issues, including broad questions such as the purpose of the survey, and detailed issues of survey execution, such as validation or the rotation of question and response choices.

McCarthy on Trademarks
Published by Westlaw, McCarthy on Trademarks and Unfair Competition analyzes trademark law, including issues relating to topics such as deceptive advertising and infringement. This is a multi-volume sourcebook, frequently cited and well-respected. Sections present citations of articles and precedents on specific topics, including how the surveys were conducted and how the results were interpreted or accepted by courts.

American Society for Testing and Measurement
The ASTM publishes the “Standard Guide for Sensory Claim Substantiation,” which is available for purchase. This is a technical guide that describes specific practices to substantiate sensory or perceptual claims. The guide addresses both comparative and non-comparative claims, and is particularly helpful in describing the types of statistical tests that should be used in claims. The guide is written with claim substantiation in mind. As the ASTM’s description states, “The objective of this guide is to disseminate good sensory and consumer testing practices. Validation of claims should be made more defendable if the essence of this guide is followed.”

The Trademark Reporter
The Trademark Reporter is a peer-reviewed journal that frequently publishes articles relevant to substantiation surveys and litigation surveys. Some of the articles describe when certain kinds of surveys should be used. TMJ is published by INTA, the International Trademark Association. As INTA states on its website, The Trademark Reporter is “…the world’s only scholarly journal dedicated solely to trademark law and related topics.”

There are many other sources that could be listed here, as I have presented only a small sample of the many sources relevant to substantiation surveys and litigation surveys. There are also a variety of sources maintained or published by private sources; for example, many large marketing companies have policies that govern the use of words such as “free”, “most”, “better”, or “leading”.

As I look across these sources and others that I often consult, I’d offer these general thoughts:

1. Claim substantiation surveys: Claim substantiation surveys are different from the kind of marketing surveys that are used for purposes such as tracking brand progress or measuring brand image. Substantiation surveys are typically more tightly focused on a smaller number of key issues, and are designed with certain elements specifically needed for claim substantiation.
2. Research standards: None of these sources tells you exactly how many surveys to conduct, but they do present standards and guidelines that may be relevant in setting sample size, selecting respondents, using controls, or other issues. Often, these guidelines must be interpreted by a qualified expert or experienced attorney.
3. Statistical testing: One thing all sourcebooks agree upon is that statistical testing for claim substantiation should be done at the 95% level of confidence.