Fixing a polling system that’s out of sync

Latest Voting Intention

Reading the various reports, narratives and commentaries this weekend about the results of the UK general election that took place on May 7, the overall perspective I’ve formed on all of that is how could the expert commentators, opinion-formers and outcome-predictors have got it all so wrong?

The election result produced a clear win for the Conservatives with a slender majority in the House of Commons (12 seats), and the virtual annihilation of the primary opposition political parties – the leaders of Labour, the Liberal Democrats and Ukip have all quit – that confounded every single opinion poll in the months, weeks and days leading up to May 7, which had all predicted a hung Parliament as the best outcome anyone could expect.

So another coalition government looked a likely election outcome according to those polls – followed perhaps by another election in six months or so – and many column inches and pixels have been spent in offering what-if? scenarios of who might be able to form a government with whom, etc (the BBC’s interactive tool was especially good), much of it based on those opinion poll results.

About the only thing the pollsters did get right was the surging Scottish Nationalist Party which triumphed in Scotland in almost a clean sweep, winning 56 of 59 Scottish seats at Westminster.

Having been in America since May 3 with hardly a moment spare to look at the TV never mind online news, I had been shielded from any mainstream reporting and commentary back home in the run-up to election day last Thursday (our election was unquestionably not a big news item in the US mainstream media). What I did see, though, was plenty of comment and opinion on social media channels, notably Twitter, that presented a view of Labour being well ahead as the likely voting preference of a majority, and reinforced much of the mainstream feeling about a close-run election and a hung Parliament.

Socialist Media - Economist.com

And so I flew back to the UK on Thursday night US time arriving here on Friday morning UK time to the news that took me by surprise as much as it apparently did all those experts I mentioned – not a close-run thing at all but a pretty decisive Conservative victory, nothing like a hung Parliament, and a political landscape that no longer looked familiar with the downfall of the traditional political opposition.

With the nationalists rampant in Scotland and the Conservatives resurgent just about everywhere else outside the large urban centres in England, the former looks alarmingly like a one-party state with the latter arguably close to that territory. Indeed, it doesn’t look like a very United Kingdom at the moment.

But analysis on comment like that is for more knowledgeable subject-matter experts to ponder over.

What interests me mostly now is those opinion polls I mentioned earlier – how could they have got it so wrong?

You can choose from a great deal of opinion on that question, to which I add my two-pence-worth to suggest a combination of factors such as:

1. Reliance on an opinion-polling system that, largely, behaves the same as 50 years ago when few-to-many was the only communication model: the few controlled the news and methods of communication (the mainstream media companies); the many (the great British public) formed opinion based on what they read in the newspapers or heard on the radio (TV was still in its infancy) – their only reliable sources of news and information; and the pollsters formed their predictions based on what the public told them in answer to narrow questions where you read what the newspapers said to help you form opinions.

That’s totally not the picture today where the mainstream media is but one element in an immersive crowded information and communication landscape that enables anyone with an opinion and an internet connection to become a content-creator, news broadcaster and opinion-former.

Anyone with an opinion...

2. Lack of trust in, and engagement by, the political process and politicians themselves: let’s start with the Edelman Trust Barometer 2015 published in January that shows a continuing trend line for lack of trust in governments and politicians on a worldwide level, not only in the UK.

3. Public tiredness and disenchantment with politics in general and this election process in particular: so much partisan opinion and commentary – yes, I do call it propaganda – where it has been tough to filter signal from relentless noise and focus on what you think is credible and trustworthy to warrant your attention and your willingness to believe.

A case in point for me was the Leaders’ Debate on BBC’s Question Time programme on April 30. Debate? Hardly. Prepared sound-bite responses by each leader individually to questions from a carefully-controlled audience. The inauthenticity of it was breath-taking.

(Of course, I should point out that some analysts are saying that this TV event was instrumental in helping many voters decide who to vote for. If that’s true, then I’ll stick to my day job.)

4. The remoteness of much of it: so much stuff by people you don’t know, with hashtags on social media like #GE2015 that are tsunamis of opinions you don’t trust because much of it is so clearly partisan; and politicians who sound so patronising with their so-sincere-sounding and constant over-use of phrases like “hard-working families” and “working people” that you eventually tune it all out.

Some or all of this probably contributed to the huge number of “Don’t know” responses when people were asked by pollsters for their voting intentions – 25 percent of voters said they didn’t know who they’d vote for on the day, according to one report I saw.

That meant that the polling organizations, pundits and others were left to predict outcomes based on incomplete data from which to glean credible insights, along with that imperfect methodology for a contemporary society – are those the major factors that let it all be so wrong?

I read of one poll where the organizers predicted the actual election outcome with some clarity (and accuracy as it turned out) but who said they didn’t publish it for fear of being ridiculed: their poll was so totally different to all the others that were predicting a neck-and-neck close race, hung Parliament, etc.

And what was their methodology? Actually talking to voters: ringing them up on the phone and directly asking them relevant questions that they would want to answer.

YouGov’s Antony Wells summarized what he thought of the polling debacle:

[…] there is something genuinely wrong here. For several months before the election the polls were consistently showing Labour and Conservative roughly neck-and-neck. Individual polls exist that showed larger Conservative or Labour leads and some companies tended to show a small Labour lead or small Conservative lead, but no company consistently showed anything even approaching a seven point Conservative lead. The difference between the polls and the result was not just random sample error, something was wrong.

It’s worth taking a look at the 700+ comments to Well’s blog post.

So the current polling system used in this kind of significant national event has suffered a severe setback in how it is regarded from accuracy, trust and credibility perspectives. This has clearly rung a loud alarm bell as the British Polling Council, the trade body for the polling industry, has announced with some understatement that it’s setting up a public enquiry into what went wrong:

The final opinion polls before the election were clearly not as accurate as we would like, and the fact that all the pollsters underestimated the Conservative lead over Labour suggests that the methods that were used should be subject to careful, independent investigation.

The British Polling Council, supported by the Market Research Society, is therefore setting up an independent enquiry to look into the possible causes of this apparent bias, and to make recommendations for future polling.

The focus of the enquiry will be on polling methodology, according to the announcement.

Looking forward to learning what those recommendations are.

Astounding Psy

PSYWhen it looked like it would exceed one billion views on YouTube by the summer of 2012, Korean singer Psy’s Gangnam Style headed into the record books for the sheer number of people who have watched the video and heard the music.

Today, it’s become the most watched video on YouTube of all time.

Now, says YouTube, it exceeds two billion views and, as a result, Google has devoted more servers to handle the traffic.

Reshared post from YouTube on Google+:

We never thought a video would be watched in numbers greater than a 32-bit integer (=2,147,483,647 views), but that was before we met PSY. “Gangnam Style” has been viewed so many times we have to upgrade!

Hover over the counter in PSY’s video to see a little math magic and stay tuned for bigger and bigger numbers on YouTube.

Why not add your view to the counter. Even if you’ve already watched it, it’s catchy!

The local newspaper is dead, long live the local newspaper

The decline in print and the rise in digitalThe closure of printed newspapers around the UK counts new casualties in the battle to stem the tide of declining circulations and the ever-diminishing number of titles in print with news this past week that Trinity Mirror is shutting down seven regional newspapers in southern England.

The news has particular interest to me as my local paper, The Wokingham Times, is one of those casualties.

Founded in 1903, the Times has gone through many evolutions especially during the past decade or so as it changed ownership a few times; and as alternative sources for local news emerged as the internet and the world wide web evolved, more online choices appeared and the ability for anyone and everyone to get online becomes almost ubiquitous and continues to be ever easier, cheaper and faster.

The closure is a picture you could paint in communities up and down the country.

Trinity Mirror, current owner of the title and its siblings in Berkshire (and Surrey), said in its announcement that it intends to develop and grow its digital business around the getreading.co.uk website which offers digital versions of its Berkshire titles – Reading Post, The Bracknell Times and The Wokingham Times – and also delivers content to mobile devices via an app.

It’s not hard to see why Trinity Mirror is making this move. As its statement says:

[…the getreading.co.uk website] has achieved unrivalled market leading penetration in the area – in the last year monthly unique users have grown by 68% (Jan-Oct 213 to Jan-Oct 2014) and the site continues to show phenomenal audience growth.

In its report, Press Gazette quotes Simon Edgley, managing director of Trinity Mirror Southern, from the company’s announcement:

This is a bold digital-only publishing transformation that will re-establish us as a growing media business that delivers the best quality journalism to our digital-savvy audience. We wholeheartedly believe that the future of our business here in Berkshire is online and this is an important and pioneering step that might, in time, be applicable to other existing markets or indeed new ones.

Bold indeed, with the inevitable human cost – 26 job losses in Berkshire (50 in total if you include the other closures, according to reports). The flip side of that is “the creation of around 10 new digital editorial roles and two digital commercial roles,” says Trinity Mirror in its announcement.

The type of hard commercial decisions made that will lead to the closure of seven print newspapers are confronting media companies across the UK and elsewhere – at all levels, nationally, regionally and locally – as trends continue to show the inexorable decline in print and the increasing growth in digital content that meets the preferences and needs of contemporary consumers who want to consume content wherever and whenever they want, with whatever device they wish, comment on and share that content with their networks, repurpose it, create additional insights from it, and more.

The move to digital is indeed inevitable as is the consequent human cost in lost jobs where current skills clearly aren’t what the media companies need as they evolve in the new digital-only environment to survive and grow.

Does it mean there is no place for print any more? Not necessarily – looking at it purely in commercial terms, if your market analysis, business plan and the numbers add up, you may have a workable proposition.

And The Guardian’s report on the Berkshire closures includes this:

The Reading Chronicle, which has been published since 1825, will become the town’s only print title. Editor Lesley Potter said it was a sad day for those losing their jobs and for the people of Reading.

“We have been fierce rivals over the years, but we have always had a healthy respect for one another. We at the Reading Chronicle have absolutely no intention of abandoning print.”

You have to feel a touch of sadness at developments like this even as they mark another milestone in the transition of news and information, how it’s produced and presented to readers, and what they do with it.

So print newspapers gradually vanish but they continue online in name and purpose, mirroring the look, feel and presentation of their analogue forbears.

It’s called progress.

Scotland referendum results via WhatsApp and more

Yes / No

Tomorrow, the United Kingdom will not be the same no matter what happens in Scotland today as citizens there cast their votes in a referendum to decide whether Scotland will separate from the UK and become an independent country, or not.

The campaigning is done; now it’s up to the voters of Scotland to decide what they want for their country and the union with England that’s been in place since 1707.

Obviously media of all types – mainstream, social – and from all over the world are devoting huge time and resources to coverage of an event that has got the world’s attention especially in countries where the flames of separatism may be further fanned on the outcome in Scotland.

I’ll be following events as time permits during the day on TV and online. It’s once the polls close at 10pm tonight that interest will be most strong as the votes are counted with the first results to be declared expected sometime around 3am on Friday morning.

What appeals to me is the idea of key news as it breaks coming to me in a way that lets me focus just on that and gives me just the facts. I can choose whether to look for more detail, if I want, whether that’s online or via more traditional news channels.

So an idea from Channel 4 News in the UK is most interesting – broadcast breaking news on the results as it happens, directly to your smartphone via WhatsApp and Snapchat:

[…] We’re going to publish all of our best content, as well as live updates, via Snapchat and Whatsapp, from the moment the polls close on Thursday night right up to when the results are announced on Friday morning – ahead of publishing it anywhere else.

That last sentence is most interesting: “ahead of publishing it anywhere else.” Before TV?

My interest is WhatsApp; here’s how to set it up:

WhatsApp the message INDYREF to 07768555671 and add us to your contacts list to sign up for all of our best overnight news and analysis, pictures and video, delivered to you ahead of all the other social networks.
If you change your mind, WhatsApp STOP to the same number.

I’d added C4News to my WhatsApp and can’t wait to see how this plays out.

C4News

It’s great to see such innovation from mainstream broadcasters, especially communication methods that clearly show the broadcaster not only gets audience preferences by demographic according to social medium but also is able to execute an idea well.

Channel 4 is not alone in this. BBC News, for instance, announced this week that its content will be available on smartphone instant messaging platform LINE. Earlier this year, the BBC experimented with WhatsApp and WeChat in English and Hindi.

And Sky News launched its Stand Up Be Counted initiative, described as “a place for 16 – 25 year olds to safely upload and share the videos, pictures or blogs they make on the issues that matter most to them.” It’s been a very active place in relation to the Scottish referendum.

Innovation really is thriving.

(Via Journalism.co.uk; picture at top via The Guardian.)

Weighing up the worth of sharing AP content or not

Retweet to your followers?

A news item on Techmeme caught my eye, so I clicked to read it.

Oregon sues Oracle over failed health care website,” the headline said, linking to a report by the Associated Press about a lawsuit against Oracle filed by the US state of Oregon alleging some pretty serious malfeasance on Oracle’s part over a health care website.

It’s the kind of business story that interests me, and one I tend to share on Twitter as some of my community there might also be interested in it. It’s also the kind of thing I might share in my Flipboard magazine – which, if I choose, can also re-share that share across Twitter, Google+, LinkedIn and Facebook – to bring it to a wider audience. It might even become a news item or discussion topic for the weekly business podcast I co-host.

Much depends on the topic, who it’s about, which publication it’s in, how credible and timely it is, how well presented the story is, etc.

I don’t especially seek out stories or reports by the AP. Yet I encounter AP reports a lot, either direct reports filed by an AP journalist like this one, or as a newswire story reported in another online publication.

(AP) Orgeon sues Oracle...

In whatever case, as with all sharing of content published online by others, I’m mindful of copyright.

But get a load of the AP’s copyright statement at the foot of this story (and in every story on their website).

AP copyright text

The yellow highlight in the screenshot is my emphasis of the off-putting wording:

© 2014 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.

I’m not a lawyer, but that looks to me like the AP won’t allow the kind of sharing I do across social networks, eg, retweeting a link to their story, never mind any content from it. Wouldn’t that be regarded as “broadcasting”?

That’s not what they intend, surely?

Well, take a look at the terms of use referenced in the full footer statement, in particular numbers 5 and 6:

AP Terms of Use

(Number 6 even mentions ‘fax’ which makes me think this wording was written in the command-and-control heyday of the mid- to late-90s and unchanged since.)

I’d say number 5 makes it clear that this is what they intend. Even retweeting a link on Twitter isn’t something they’d like you to do by the looks of it:

5. Except as provided in this agreement, you may not copy, reproduce, publish, transmit, transfer, sell, rent, modify, create derivative works from, distribute, repost, perform, display, or in any way commercially exploit the Materials carried on this site, nor may you infringe upon any of the copyrights or other intellectual property rights contained in the Materials. You may not remove or alter, nor cause to be removed or altered, any copyright, trademark, or other proprietary notices or visual marks and logos from the Materials.

I suppose the key words here are “commercially exploit” which I guess means making money from the AP’s intellectual property without permission, recognizing their rights or paying them for usage.

Yet surely there are better ways in communicating such intent that don’t leave you feeling that whatever you do to amplify their story under the fair use or fair dealing aspects of copyright laws, you should probably look over your shoulder just in case you see a lawyer bearing down on you.

I contrast this unfriendly attitude with that of an arch-competitor of the AP – Reuters.

Reuters actively encourages you to share its content!

Look at this same story, for instance, as reported by Reuters on its website – with social share buttons arrayed at the top:

(Reuters) Oregon sues Oracle...

Not only that, the footer in the story repeats those social share buttons and also tells you how many of your friends have recommended the story on Facebook and/or urges you to be the first to do so, as it does in every news story on the Reuters website.

Reuters encouraging sharing...

And not a copyright notice or terms of use link anywhere except among general site links in a specific area at the very bottom of the website, each of which is written in far less draconian language. Much more concise and contemporary, too.

Comparing these two different approaches to creating and publishing copyrighted content that others inevitably would wish to share, which one gives you confidence in sharing with your social online communities? Which one behaves like trusting you is the default rather than the other way around? At a time of continuing evolution of mainstream media and how people use online to get, consume and share their news, which one appears equally confident in making content available online that will be shared and so actively encourages it?

In essence, which one is the publisher who gets it about content-sharing, trends, behaviours and the social web?

I know which one gives me that confidence.

PS: As it happens, I shared the AP story on Google+ as I wanted to highlight some of the text that I couldn’t do in Twitter (more than 140 characters). Plus my community there is, broadly, more tech-oriented and so I thought I might get some interesting comments back. None yet though…

Valuable insights in 2014 #InternetTrends report by Mary Meeker

netflix-chromecast.jpg

Last week, US venture capitalist and former Wall Street securities analyst Mary Meeker published her 2014 Internet Trends report that offers a deep-dive look into the trends, possibilities, probabilities, scope and scale of what the global connected world will look like in the coming few years.

It highlights trends to pay attention to, offering keen insights into what’s shaping this connected world:

  1. Key internet trends showing slowing internet user growth but strong smartphone, tablet and mobile data traffic growth as well as rapid growth in mobile advertising.
  2. Emerging positive efficiency trends in education and healthcare.
  3. High-level trends in messaging, communications, apps and services.
  4. Data behind the rapid growth in sensors, uploadable / findable / shareable data, data mining tools and pattern recognition.
  5. Context on the evolution of online video.
  6. Observations about online innovation in China.

At 164 pages, the slide deck is huge in its scope, and a challenge to decipher detailed meaning from just a deck without the benefit of hearing its creator talk you through it (she did that at the event last week for which she had prepared the deck).

Many others are filling the vacuum to do that. I have some thoughts, too, on a few areas from the 164 slides. I expand on that below, but if you want to just feast on all of Meeker’s data right now, here’s the deck:

Last year’s 2013 Internet Trends report was 117 pages, a slim volume by comparison. Indeed, I found it it a relatively simple matter to quickly glean and absorb insights from her deck to come up with what I saw in May 2013 as fifteen big trends for the evolving digital age.

A year later, how does the landscape look?

Here are three elements from the 2014 report that caught my attention (and imagination).

1. The rise of the mobile internet and the mobile devices that people want to use on the web are irresistible

The first aspect is the steady increase in shipments of smartphones (Wikipedia definition) worldwide since 2009 …

mm2014slide06

…  and, in tandem, the rocketing growth in tablet (Wikipedia definition) shipments which overtook shipments of desktop and notebook PCs at the end of 2012/beginning of 2013.

And notice the massive uptick in tablet shipments that started at the end of the first quarter in 2013 …

mm2014slide07

… which makes it easy to understand in the context of the increasing numbers of people accessing content on the web via mobile devices like smartphones and tablets in May 2014 compared to the same time in 2013. While there isn’t a slide to show how connectivity – whether wired, wireless or cellular – is growing everywhere, these figures surely provide convincing evidence that that is what’s happening.

And global mobile usage average has almost doubled year on year, broadly reflecting the detail in each of the regions measured.

mm2014slide09

What these metrics say to me is this: if your presence on the web isn’t attuned to mobile – meaning, your site delivers the content people want and a great experience they expect when they come to you on their mobile devices – you’re in serious trouble.

2. The evolution of mobile apps

If using the web on a mobile device is increasing at a rapid pace as smartphones and tablets eclipse desktops and laptops, the requirement for mobile tools – apps – to let you do what you want on your mobile connected device is equally increasing at a rapid pace …

mm2014slide40

… where those apps are evolving into tools of genuine utility for the user, that let you do certain things very well.

So instead of being all things to all men, so to speak, many apps are shifting into specific use formats …

mm2014slide41

… that offer you context-aware interactions that, as TechCrunch notes, are purpose-built and informed by contextual signals like hardware sensors to interact with you in far more compelling ways than at present to maximize their usefulness to you.

3. Game changers for mobile TV and video consumption

Meeker’s slide deck has a great deal of content about the rise of personalized television where you the user define what the content is that you will watch and where you get it from (think of custom user preferencing in Netflix and Chromecast, as examples of this), and how you control it.

Consumers increasingly expect to watch TV content on their own terms.

I have a good example: watching a film that’s delivered from Netflix where I control its output with my smartphone or tablet to play on my digital smart television via wifi connection to the Chromecast dongle plugged in to the HDMI port on the TV. No traditional TV broadcaster in this transmission/consumption equation at all.

mm2014slide124

For me, this text slide summarizes very well the key aspects of all this, the “televisual game changers.”

mm2014slide126

And so, a small subset of the compelling content in Mary Meeker’s 164 pages of metrics and insights that make up her Internet Trends 2014 report. My focus has very much been on mobile. That’s by accident and by design – I didn’t plan this post to be like that, yet all the things that grabbed my attention that I’ve written about here are all to do with mobile.

Well, maybe not everything. Big data trends, for instance.

mm2014slide60

Do review the full deck and see what strikes you as compelling. And some of the other reporting on it is pretty good, adding to the ways in understanding what the report is about:

Download the PDF report here: 2014 Internet Trends By Mary Meeker or view the deck on Slideshare.