Skip to content

Establishing rules for blog sentiment scoring

July 3, 2008

Agree upon the rules before you get into the content -decide how to decide up front.  I learned that once upon a time in a class about leading groups of people involved in decision making…

Now suppose you are monitoring and scoring customer comments for your brand, ACME Acorns.  You have devised a clever strategy to categorize the comments and to score them as positive, negative, nuetral, or mixed views,  with regard to your Acorns.

You need to establish some rules before you start looking at the data.  Why?   What tends to happen is that when you look at the data, you will start rationalizing the relative merit of some of the comments.  I’m going to throw this one out because this person doesn’t have an Acorn.   And this one, well it’s not that negative, maybe it’s really mixed.  This one seems kinda positive – they didn’t say anything bad.  Is that neutral or positive?  Do they really have to gush about our Acorns to be positive, or just say they are ok?  It goes on.  Pretty soon, one has colored the data with their own personal point of view – for better or worse.

Then you show the data to someone else and the issues compound.  They are concerned by the negatives and want to take action  to rectify them – you show the specific comments and many seem less credible, perhaps even suspect, so it’s tempting to toss them out, and to concentrate efforts on those that seem qualified in one regard or another.    And it goes on.  Pretty soon, one begins to wonder if any progress is being made, or if all the improvements are simply due to exclusions, rationalized re-scoring, or other biased and preferential treatments of the data.

I think fundamentally, one has to set some ground rules before they are tempted to rationalize in degrees.  Let the rules guide the activity.   Also, one needs to decide the purpose of the monitoring and scoring – is it to represent a fair sample of what is in the public domain, and  visible influence to all who see it, or to only reflect a smaller qualified, and credible subset of the broader material?  In either case, it is important to share the scoring rules and rationale along with the results when presented.

So, let’s hear from any practitioners out there who find me through search.  What’s in your rulebook?

 

 

 

Advertisements
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: