PR firm exeucutives are either optimistic, pessimistic or undecided according to “survey”

Posted by Dan Keeney

 Please don’t try to foool people by suggesting your data is representative if you survey 57 people
I am not a member of the Council of Public Relations Firms but I pay close attention to what the organization says and does. PR firms are the engine of the public relations industry and as their voice, the CPRF has the ability to be influential and to help guide the rest of us. All of this buttering up is meant to soften the blow as I dissect all that is wrong with the CPRF’s latest research initiative, the Q1 ’09 Firm Executive Survey.

Conducted by Kelton Research, the survey only had 57 respondents and has a margin of error of +/- 13 percentage points. Holy cow! That basically makes all the data presented in the survey unreliable.

What bothers me most about it is that the survey results are presented in charts and graphs, which gives the data the feeling of being more certain and statistically relevant than it really is. For instance, Kelton states that “more than six in ten (61%) PR firms report that their U.S. revenue grew from 2007 tio 2008.” What they don’t make clear is that just 35 PR firm executives responded that way. Fewer than THREE DOZEN public relations firm executives say that they saw revenue growth last year and suddenly the Council of PR Firms is reporting that 61% reported growth!

I am not suggesting that their data is wrong. I have no idea. I am simply suggesting that they shouldn’t so confidently present data that they know could easily be wrong without some very clear caveats.

The fact is that with a 13 percentage point margin of error, the real percentage of U.S. firms reporting growth could be as high as 74% or as low as 48%. Even though Kelton reports that 39% reported a revenue decline last year, the margin of error could push that as high as 52% or as low as 26%. 

The data is completely unreliable.

When they suggest “PR firms are split on what to expect for the coming year,” they fail to make note of the fact that their pie chart could just as easily show overwhelming optimism as overwhelming pessimism. The margin of error could push the percentage projecting a decline up to 41% and those projecting growth as low as 20%. Or it could nearly be the opposite. The data says absolutely nothing in terms of what PR firm leaders in general expect. The sample is too small.

So the real question becomes, what is the purpose of releasing this data? The results can’t be reliably extrapolated over the broader population of America’s PR firms. At most, it is a terrific gauge of what these 57 people believe. It should be presented as such. Instead of charts, graphs and percentages, use whole numbers.

Be Sociable, Share!

This entry was posted on Thursday, March 5th, 2009 at 12:26 am and is filed under Future of PR. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

One Response to “PR firm exeucutives are either optimistic, pessimistic or undecided according to “survey””

  1. Linda Ld Jacobson APR Says:

    Your comments are exactly on point. I’m surprised that the CPRF didn’t make their numbers more prominent but more surprised that they would report survey results that represent less than 100 firms. These numbers are already being reported as indicative of PR firms across the nation ( Thanks for the illumination.

Leave a Comment