James.Cridland.net

James Cridland's blog

Where radio and new platforms collide. With beer.

« The BBC’s fifteen web principles | Blog index | 8,200 and counting… »

The quality pitfalls of user-generated content

Posted on Thursday, April 26th, 2007 at 10:08pm. #


Photo: Ian Grove-Stephensen | Creative Commons licenced

At a conference today in Cambridge, I was struck by the thoughts of one of the audience members (which consisted of mostly ‘creative types’ - designers or artists).

I was on the panel, discussing what Virgin Radio has been doing in the user-generated-content space. A lady probably in her late 40s was concerned at what she called the ‘fad’ of user-generated-content. She described herself as a creative, and was concerned that “we were allowing” people with no creative talent to post their own work online. “Surely you have a duty to sift through”, was her argument, “to make sure that the work you promote has some artistic merit?”

She warmed to the theme. “You’re conning us all,” she said. “Do you even care what goes onto your website? It’s a totally cynical ploy to earn money out of people.”

My efforts to persuade her that the quality rises to the top wasn’t totally successful - probably because I wasn’t instantly able to show her Flickr’s automated “best of” collection, which they call ‘interestingness’.

Someone else started talking about ‘amateurs’ on the web posting content. My hackles rose; but thankfully the excellent, considerate, handsome, intelligent, erudite, and man-who-reads-this-blog, John Naughton, called time.

Michael Mullane’s blog recently featured a similar thought, describing an editor, who he calls Prendergast:

Prendergast was losing sleep over moves to embrace user-generated content. “We are pouring more and more editorial resources into sifting through pictures of kittens playing with balls of wool.”

He said that it was turning his company into the editorial equivalent of a skinny model with a disproportionately large head. “Even if we come to our senses, I’m scared that we will have irrevocably damaged our health.”

After the panel session, while talking to her along with a chap from Sky, I was struck by a thought.

You see - in many cases, quality does rise to the top - whether from popularity or from the type of algorithm that Flickr uses. (Indeed, I once wrote a similar algorithm for Media UK, which used all manner of interesting things to work out whether a particular forum thread was ‘worth reading’ or not.)

But in many cases, quality doesn’t. BBC News produces a most popular page, showing news stories which most people are emailing to their friends or reading. As of now, “Stephen Hawking In Space” is the most popular, with a story about Bob Woolmer second, and a Hugh Grant arrest in third place. Meanwhile, the US Congress has just approved a pull-out of Iraq, which the BBC bills as second-most important, but appears as the tenth-most popular story. (There’s a website somewhere which compares these two weightings side-by-side, but I can’t find it). Indeed, the second-most emailed story right now on the BBC News website is Sudan man forced to ‘marry’ goat, a story (predictably involving human/goat relations that go rather too far) which is now well over 13 months old. There’s an algorithm that needs changing.

I’ve managed to post this drivel quite successfully without anyone else checking over it for any redeeming quality - and I’m guided, predominantly, by Google Analytics (having made a conscious decision not to splatter ads all over it, otherwise I’d find myself blogging about credit cards) - so perhaps I’ve also fallen for ‘popularity is good’.

We’re drowning in content. My Google Reader is permanently full of more stuff to read. I need someone, or something, to rate that content for me - in the same way that Flickr successfully does it.

Our questioner doesn’t use the internet much, so she’ll never find this (particularly since my email address was spelt wrong in the programme, not that it would take anyone two minutes to work out what it should be). But I wonder, and whisper this, whether she might have a point after all.

UPDATE: John Naughton speaks about the panel session (MP3)

6 comments

Martin Belam said at April 26th, 2007 at 10:31pm

The comparison between what the BBC “want” people to read, and what people are actually reading is at http://cgriley.com/bbctouch/

>> I wonder, and whisper this, whether she might have a point after all.

I have this nagging suspicion that most of the people saying things like the woman you talk about are people who have managed to get away with calling themselves “creative” by putting on airs and graces. They are not horrified that the masses now have publishing tools *per se*, they are horrified that they themselves are about to be found out.

>> I’ve managed to post this drivel quite successfully without anyone else checking over it for any redeeming quality

Me too, but to be honest I yearn for a sub-editor and someone who will write me a style guide so I can be consistent about whether I write things like web site or web-site or website.

Nicky said at April 26th, 2007 at 10:53pm

Interesting perspective. I was wondering about this myself a while ago, my thoughts here.

http://www.squeezypaws.com/?p=919

Olly said at April 28th, 2007 at 10:50am

Re the BBC “who’s reading what” … isn’t part of the problem though that there isn’t a single aggretor for all content delivered from whatever platform. For example, if I’m home in time I tend to watch Channel 4 News. Then I might check my email. My default page is the BBC homepage, so I might check the news and sport. Because I’ve watched Channel 4 News, and I’ve read one of the freesheets on the way home, I don’t generally check the same stories (unless it has a different angle, or it’s a story I’m particularly interested in).
The same is true on the way to work. I’ve generally caught the headlines on BBC Breakfast (or via the red button) and read the Metro. So once at work, after seeing the headlines haven’t changed that much I may be drawn towards querkies and “also in the news” type stories.
Taking the stories I actually click on the BBC News website would give a seriously skewed impression of what my news values are.
Secondly, and as I’ve always taught people: the page impression is a measure of the link to that page, not the quality of that page. Let’s face it, who isn’t going to click on a link that says “Man forced to marry goat”… it doesn’t mean they necessarily rank that as more important in terms of news values.

Adam Bowie said at April 29th, 2007 at 12:08am

It seems to me that it’s inordinately difficult to really determine where the best “content” (I really hate that word) is to be found.

The examples listed here are good ones. Flickr has its “interestingness” and very clever it is too. But that’s not to say that there aren’t other good pictures that it doesn’t pick up - stark differences between colours certainly seem to help your picture become more “interesting.”

And the BBC News example really reflects how we read news in an offline world. The broadsheets generally have the “better” news, but tabloids sell more. Even if everyone is reading about a man marrying a goat, I’d rather not have that as the lead story thanks.

We do need guides to help us wade through all that material constantly being produced, and in an older world that was the job of an editor. It’s a human being who determines what stories appear where in a newspaper, and indeed, on the BBC website. Google News, on the other hand, doesn’t use an editor - yet it’s effectively relying on lots of editors on lots of other news sites to determine what’s really important. Otherwise the lead story would always be Hugh Grant throwing baked beans at a photographer or similar.

Who those editors are is where it really gets difficult. In general I want small groups of editors, not the vast unwashed. Leaving things to the public at large means we end up with films like Norbit. A single editor might not be to your liking, but that’s not a problem. Go away and find someone who does agree with your sensibilities.

It’s the problem I have with Digg and its ilk - you simply end up with a cocktail of stories of a particular flavour. If that’s what you want then fine.

James, you have a sidebar of links to stories that you find interesting. If several of my friends were to have similar sets of links, then perhaps that could act as a quasi-Digg service - in this instance recommending internet and media related topics. Another set of friends might point me in the direction of sports, and another set science stories.

But someone needs to get out there and find those stories in the first place, like the sub-editor who reads monitors the stories coming in off the wire services in a newsroom.

And finally, I do want to be surprised from time to time. I tend not to personalise news sites too much to just give me what I know I’m interested in. If all I read are tech and media stories, then my life will be poorer for it. I may skim over some sections that I’m not so interested in, but once in a while you do read something you’d never have considered reading had it not been in front of you, and you might just be a little better for it.

Frankie Roberto said at April 29th, 2007 at 9:28am

The point of much user generated content is that, by being open for contributions from everyone, you gain something that would be impossible for even a few trained, professionally, ‘creative’ people to produce.

For instance, the BBC’s ‘WW2 Peoples War’ site (http://www.bbc.co.uk/ww2peopleswar/) gathered an incredible range of first person accounts of the war. This was a hugely valuable experience to the people who contributed, and also makes for valuable reading. The range means, for example, you can read stories from people living in your area.

Let’s take another example. User generated content in news. Whilst for the most part, much of news is predictable, or PRed, or researchable, all of which can be handled by a professional news team, there are some news events that just happen suddenly, unexpectedly. It’s impossible for a news team to be everywhere at once, so by opening up to contributions from the public, you can get a greater breadth of coverage.

For most established websites - cultural, news, media - ‘doing’ user generated content should mean giving careful consideration to what you’re doing it for, including why people would want to contribute, and why people would want to read it. YouTube, Flickr, etc don’t have any specific call-to-action, other than just ‘upload your stuff’. This makes the quality, and type of content hugely variable, but they benefit from the huge range and network effects.

Basically, most of us producing websites shouldn’t be trying to replicate what Flickr and YouTube, but should be linking to the best examples/groups/communities on those sites, and running our own UGC projects which are much more highly targeted.

Frankie Roberto said at April 29th, 2007 at 9:32am

Oh and the BBC News thing is skewed by being a) limited to what’s popular in the last half-hour/day and b) by the fact that quirky stories are contained within one story, whereas bigger stories (like Iraq) have several different ones. It’s also partly self-reinforcing. So whilst it’s quite useful, and interesting, I don’t think it should be taken as ‘what people are generally most interested in’ on the BBC News website.

Leave a comment

This website's Gravatar enabled (that's the pictures on the right)

To prove you're human, type the two words below into the box provided.