Rating system breakdown

5 March 2007

I wanna discuss rating systems for a bit. From what I gather there are three flavors: a simple “one vote” method, a thumbs up or down, and the scale. Each solutions goal is to find out what people think of your content. This is helpful when trying to understand your audience.

The “one vote” method

As simple as it gets. If someone likes a piece of content they simply click a button saying so. Nothing more nothing less. This is a visceral response, a “blink” reaction which requires minimal thought. The downside is you can’t infer that a non-vote means people dislike content. A non-vote most likely means they didn’t read the content enough to care which could suggest weak content but it’s a little grey.

Thumbs up, thumbs down

This one makes me think of Siskel and Ebert and their black and white critiques of film. Probably the most widely used style because of its simplicity. Like the “one vote” method, it relies on gut reactions but allows people to express their dislike for a piece of content. The style sometimes borrows the visualization side of the scale system below.

Scale

Frequently represented as stars, we commonly see this method used along with reviews and entertainment ratings. This system requires the most amount of thought from a person which often leads to poor return. It’s best used in situations where a person is investing time into writing a review.

Conclusion

Implementing a rating system isn’t for the faint of heart. You need to be comfortable with powers of online democracy. Oddly enough rating systems are why Google works. They’re Page Rank system is based almost entirely on the “one vote” method where a link from one site to another stands as a “vote.” “Democracy on the web works.” This can be seen on Newsvine, Digg, Delicious, Flickr, and more.

Remarks

Jeff Croft http://jeffcroft.com

The online “rating system” that fascinates me the most if Flickr’s “interestingness.” I love that thing, and here’s why: you don’t know you’re using it. I love that Flickr programatically determines whether I like a photo or not simply based on my normal interactions at the site. Interestingness is based upon some top-secret algorithm that considers all sorts of things, like number of views, number of comments, number of incoming links, number of times a photo has been favorited, and so forth. But even more interesting (pardon the pun) is the meta-level things they consider. For example, I heard George Oates from Flickr say once that views, comments, so forth from your contacts aren’t as powerful as the same action from people you don’t know. Which makes a great deal of sense, if you think about it.

I guess my main problem with most rating systems is that I don’t believe people will use them. It’s that one percent thing. You can assume about one percent of your user base will use a feature like that. Why so low? Because, in most cases, users don’t see the value they get out of it. But, at Flickr, actions you regularly perform in your usage of the site (which do have value to you) act as your votes. I love that.

Nathan Borror http://www.playgroundblues.com

I wouldn’t be surprised if Flickr took some cues from Google’s Page Rank system.

I guess my main problem with most rating systems is that I don’t believe people will use them. It’s that one percent thing

Or people are using the wrong method. Digg shows how successful rating can be and it has everything to do with its simplicity. They rely on gut responses which is the only way a rating system can be successful in my opinion.

Joshua Brewer http://brewerdesigngroup.com

Just a quick off-subject note… your comments color makes them almost unreadable… color: #757575; might look nice and be readable.

Nathan Borror http://www.playgroundblues.com

@Joshua, you might need to clear your cache. I made some CSS updates over the weekend.

Also, sorry for the issues this morning. Still working out some kinks.

Adii http://www.adii.co.za

Hi

As part of Lorelle’s Blog Design challenge (http://lorelle.wordpress.com/2007/03/07/blog-challenge-to…), I’ve posted your site design as one of my Top 10 designs.

View it here: http://www.adii.co.za/2007/03/08/top-10-blog-designs/

I would love it if you could comment on the post and maybe give some insight on your design as well as some input on my other choices.

Keep up the good work!

Martin Kool http://thenew.oddend.com

There may be a fourth system that you may want to consider, let’s call it the survey. This would help the reader focus by asking her one, very straightforward question and by giving her the answers. Say “Was this helpful?” with a selection of “Not at all”, “Partially”, “Yes”, “I’m even recommending it to others”. The options don’t actually have to scale, but they can depending on what you want people to confirm for you. The question and answers you choose will largely influence the amount of thinking involved.

(BTW, the text of comments on the dark page are still nearly illegible in Firefox 2.x on Windows XP.)

Remarks are closed.

Remarks have been close for this post.