http://www.nevillehobson.com/2006/07/12/the-twisted-jupiter-tale/

I’d say the story is far from over, though - take a look at Toby’s latest post.

As a highly trained qualitative researcher (PhD, OSU, 2000) I’m deeply disturbed by this whole affair. I’d like to get into more commercially relevant research myself so seeing a major research firm undermine the credibility of such research is quite disturbing.

It’s been interesting to read the comments about the agency and I’ve gotten much more interested in the relationship of research and pr. But doesn’t any company have to take responsibility for the services they outsource? I don’t see how Jupiter Research could get off the hook at this point in time.

Because most of my blogging is industry oriented I am regularly faced with the call or not question. In this case, that’s not an important issue based on the extensive contact with the appointed representatives of Jupiter Research.

On bloggers and echo chambers, the reality is that the web communication is simply heightening the dynamics that already existed. If you talk to researchers who deal with the press or study what happens in the process of research findings going mass media, you’ll find that the nuances disappear very quickly no matter how well you state things.

Deep research always produces a lot of maybes and the media, politicians, etc. don’t handle that very well and remove the nuances that they don’t understand or can’t fit into a soundbite. In the process they typically misrepresent the research.

It’s a mess and it’s sad to see a research firm contributing to the situation in this manner.

Peace

les blogues laissent ils indifférents les entreprises ? non si on en croient une étude (mise en cause d’ailleurs ici ) du célèbre cabinet jupiterResearch À les croire 34% des grandes entreprises américaines disposeraient d’un blogue corporatif,…

First of all, thanks for the great response with lots of good points. It’s exactly this kind of dialog that the blogosphere is good at when it’s at its best.

I’m not going to respond at great length, because I’d like to let you points have time to sink in a little. But a couple of things pop into mind.

First of all, I don’t think that what Toby herself did was irresponsible. As I said above, I think Toby did the right thing up to a point, and apart from the headline what she wrote wasn’t inflammatory. However, the effect of the echo chamber is such that what she said was bound to be spun in directions that she couldn’t justify: that Jupiter doesn’t get bloggers, that it does lousy research, and so on - none of which is justified from her original post.

Does Toby bare some responsibility for what others have interpreted her story to mean? It seems harsh to say yes. After all, “you own your own words” (to borrow the old WELL slogan) but you don’t own others. The problem with that view is simply that others will put you in roles that you’re not actually taking. Toby was posting as a blogger, yet the first comment on her story thanks her for “investigative journalism”. I’d suggest that once you start to be put in that role, you start to have to do what investigative journalists do - and to her credit, I believe that Toby is taking that responsibility seriously.

And from your comments, so are you - it’ll be interesting to see what you come up with.

Marianne, I understand your “maybe” The problem is that in the echo chamber, “maybes” are often the first thing that are dropped as a meme moves on. I’m actually going to do some proper work when I get chance tracking how stories evolve as they are passed around the blogosphere - it’ll be interesting to see how it works.

I think that I am justified in saying that there is really no sense in quibbling over every word that I used in my comment…of course they released a press release ABOUT the report; and within that press release there was information announcing the FINDINGS from the report. A press release that was merely announcing the report would have done just that…perhaps “New Report Available From Jupiter Research on Corporate Blogging” period. So by releasing the information that was at odds with other available data points without any supporting methodology, Toby began inquiring about the methodology; she was not seeking the report. So, with all due respect, I believe I am justified in my statement that the press relesase was problematic in the information that it provided and in the information that it did not provide; I don’t believe I have made a “serious accusation” about Jupiter’s report; I am quite aware that I have not seen it which is why I chose the word “maybe.”

Fard,

I agree with you regarding the release of the methodology both in the press release and the report that you purchased…it is standard operating procedure to provide this information; context provides meaning. To paraphrase Mark Twain, the difference between the right word and the almost right word is like the difference between lightening and a lightening bug. The difference between 6.8% and 70% could be the difference between a Fortune 500 corporation and a large corporation. It helps to know the right word.

Marianne

Thank you for your close attention to this issue and for your many comments over the past few days. The fact that the JupiterResearch report is receiving so much attention within the blogosphere indicates that bloggers take research in this area seriously and are more than willing to debate it.

In your comments, you make a number of points. I’d like to respond to two of your major assertions below.

1. The JupiterResearch Incident Is Another Example Of Bloggers Acting Irresponsibly

You argue that bloggers have a responsibility to exercise all due diligence before posting negative commentary about a company. You have asserted that Toby did not do enough to get JupiterResearch’s side of the story before posting her original article on this subject. If she had gone around the PR agency, she may have received answers to her questions.

You make a good point about the responsibility of bloggers to check their facts before going public with a story. However, as someone who has been involved in numerous public relations campaigns on a number of contentious issues, I am well aware that companies hire public relations counsel to interface with the media and direct inquires to the appropriate company spokesperson(s). The media have come to expect this and nine times out of ten, if the company wants to comment on an issue they will direct their PR counsel to schedule an interview.

My point is this: By going to the PR agency, Toby was interfacing with the company directly. In fact, based on the content of their e-mail, Peter Arnold Associates contacted JupiterResearch about Toby’s questions. The PR firm relayed the company’s response to Toby.

When I contacted the PR firm, I was told the same thing: JupiterResearch (not the PR agency) is not releasing any more information about the study methodology. Anyone else in this position would have made the conclusion that Peter Arnold’s response was that of the company. Simply put, JupiterResearch did not want to respond to her inquiry.

As you well know, this happens to mainstream media all of the time.

Overall, I believe Toby acted very responsibly. She did not post before contacting JupiterResearch (through their PR agency) about the research. She could have simply posted her opinion on the research without contacting the company. She did not, which raises her level of professionalism in my eyes.

Remember: Companies set up information sharing channels for a reason. Toby followed the channel established for the media and received a corporate response. I don’t think Toby would have received a different one if she had gone around the PR firm.

2. I Asked The Wrong Questions When Reviewing The Report

As I mentioned in my original post on this issue, I decided to purchase the report because of my intense interest in this issue. I also purchased it because the PR agency directed me to the report, which they implied would provide me with answers to the questions I was asking about the study’s methodology.

Overall, I am very confident that I asked the right questions about the report. For example, the National Council on Public Polls (NCPP), has a very good article, “20 Questions Every Journalist Should Ask About A Poll.” Key questions they suggest journalists ask include:

- How many people were interviewed for the survey?

- How were these people chosen?

- When was the poll conducted?

- How were the interviews conducted?

These questions get at the heart of how a study is conducted and whether the results can be accepted. It is standard operating procedure to provide this information. For example, Harris Interactive, which produces a number of online and client-directed surveys, releases detailed information about the survey’s methodology in every press release it issues about its polls. For example, see this press release.

In Conclusion

As someone who is trained in statistical methods and has interpreted and reported a number of surveys, I find it odd that JupiterResearch did not reveal their survey methodology in the press release they issued about the report. I find it even odder that they did not provide detailed information about the research methodology in the report I purchased. I expected to see this information and still wonder why it was not provided. I don’t think I should have to speak with an analyst from the firm to answer basic questions regarding how the research was conducted.

Every survey has limitations. I believe that they should be highlighted so that people will view the research as credible and reliable. (In fact, I am currently planning to launch an opinion poll in partnership with another firm that — while it has a number of limitations — I believe will provide some useful information. I plan to ensure that anyone reading the research report will be fully aware of these limitations and know that we are planning follow-up data to attempt to bolster the data and identify trends.)

Thank you again Ian for your comments. As an aside, I am currently speaking with someone at JupiterResearch who is providing me with information about the survey methodology. As I have mentioned a number of times, I plan to provide JupiterResearch with the opportunity to respond to my criticism of the report and set the record straight about their research practices.

Overall, I think we acted very responsibly.