So Andrew Mueller (@andrewmueller) pointed me to this Social Media R.O.I. calculator thing yesterday, and I have to admit that I was pretty psyched at first that someone had figured out how to create what I thought was… well, impossible to create in a table/equation type format.
That was until I started actually looking at it and realized that it makes absolutely no sense whatsoever.
Watch the video for the full explanation (and the kind version of this post):
First of all, let me let you in on some truth about the “how” of R.O.I. measurement that will probably disappoint you: As far as I can tell, there really is no way to calculate Social Media R.O.I. through an equation or a cost-accounting type table. It just doesn’t work that way. I know it isn’t what you want to hear, but I am not one to blow smoke or sell you sunshine and daisies on a cloudy day.
In order to do it right, you have to take timelines into consideration: When a campaign or program started and ended in relation to deltas (changes) in sales and other metrics. You have to separate spikes from trends. You have to measure the impact of all media and all channels in order to isolate sticky elements from ineffective ones. You have to be able to look at the impact of special offers and promotions separately from the impact of your social media activities alone. It is a multi-phased and multi-layered process. Kind of like pealing the layers of an onion while looking at each layer’s tangential relationship to (overlaid) timelines for a variety of activities. Not difficult, (and certainly not as difficult as it sounds) but much more involved and complex than a cookie-cutter equation on a spreadsheet.
I know everyone wants that magic pill, that miracle R.O.I. calculator that fits on just one page (fill in the blanks and watch Excel do its magic), but it isn’t that simple. If it were, ad agencies and PR firms would have figured it out long ago. And while it is tempting to want to believe that such an equation or calculator exists, it doesn’t.
Not that it will stop masses of Social Media and measurement “gurus” from trying to sell you their own particular flavor of “proven, secret” measurement snake-oil. Be forewarned because they are starting to come out of the woodwork and this whole R.O.I. thing is about to become a monumental clusterf**k if we don’t start calling people out on their BS right away.
Getting into the R.O.I. Calculator: Section 1 – The setup
So anyway… since Social Media R.O.I. can’t be calculated in one single step, the very notion of this calculator is flawed from the start. But then it gets worse: All you have to do is take a look at some of the elements of this “calculator” equation to smell the pungent and all too familiar aroma of BS. Check out these screenshots:
(above) Not a bad start. We have some cost accounting there, some setup and training costs… Good wholesome stuff to start with. But it doesn’t take long for the calculator to start getting into cost per click and incoming traffic… Which might belong in the calculator somewhere… but front and center like this? Not likely… unless whomever crafted it happens to come from a web analytics background. (You tend to first want to measure what you know.)
The danger in this is that less than five lines into this calculator, we already find ourselves prey to an old-school web analytics mentality that has absolutely nothing to do with Social Media.
Mark my words: The tendency to turn solely to web analytics specialists to measure the impact of Social Media is going to be a monumental mistake made by most businesses that engage in Social Media in the coming year. And sadly, it is inevitable: Business folks making these decisions don’t understand that the social web is the Seine river, not all of Paris.
Just like IT managers shouldn’t be the de-facto bloggers for their companies, web analytics people shouldn’t be measuring R.O.I. just because the tools are digital.
But back to this post and the calculator: Is traffic to a site really the most important metric to focus on? And based on this calculator, are we to infer that our Social Media strategy to attract customers begins with pay-per-click banners and Google Ads? Seriously?
As an aside, “cost of incoming traffic” might be a better title for the bottom section than “value of incoming traffic.” Just a thought.
But that’s nothing. It gets much worse:
Getting into the R.O.I. Calculator – Section 2: The dollar value of stories and WOM (Huh?)
(Above): I don’t need to dive too deeply into the nonsense on display in the above screenshot. It kind of speaks for itself – with the help of my little red comments. Quantity of stories? Value of stories? What? Um… What stories? and how do you assign arbitrary value to a story? Even if this were relevant to Social Media and conversations in some way (which it isn’t), someone please explain to me how one calculates the value of a story.
Or the value of puppies?
Or the value of sock-flinging?
The worst are yet to come (my favorites): “Word of Mouth Value” and “Word of Mouth Value Gross”. I can’t even come up with anything smart to say about this. How do you assign a dollar value to WOM exactly? Oh, never mind.
And then we get into focus groups. Seriously. Focus groups? In a Social Media R.O.I. calculator?
I swear on my front yard’s emerald lushness, this thing is real. I did not make this up.
But hang on, it isn’t over:
Getting into the R.O.I. Calculator – Section 3: We’re already out of ideas so let’s just make **it up!
(Above): The third and final section of the Social Media R.O.I. calculator looks at the cost of employee engagement, right? Yet the three elements of this cost accounting fiasco ask for estimates. Not hard numbers, but estimates. As in a questionnaire to circulate around the office asking people to estimate how much time they spend using Social Media.
Not hard numbers, but guesses. To calculate R.O.I. Seriously.
Not only that, but this is it. Calculation over. Hit submit and you’re done.
Are you kidding me?
Are you kidding me.
But wait, there’s still more – The calculator comes with a digest of what you’ve actually calculated, in context:
(Above): The benefit analysis spreadsheet (priceless). The four elements listed are “Advertising value,” “PR value,” WOM Value” and focus group equivalence value.
Take a close look at that table. Read every word. Get into the thinking behind it (if you can). Now answer these simple questions:
1. What does any of this have to do with Social Media, engaging in conversations, building relationships with customers, building value, managing a brand’s online reputation, or increasing brand relevance – for starters? How is any actual engagement measured?
2. What is this actually measuring, aside from estimated labor costs, pay-per-clicks and… focus group substitutes? (I can’t figure it out.)
3. In what universe does this actually calculate R.O.I. of… anything?
4. Were transactions or sales ever mentioned? Social Mention, maybe? Conversions? Loyalty indicators?
5. Would you like that giant bag of bulls**t sent to you pre-paid or C.O.D.?
One final word about Social Media and Professional Responsibility:
Note that this calculator was developed by two pretty smart and well respected people in the New Media world, and I am sure they meant well, but this is the kind of thing that companies really need to watch out for: Well-packaged nonsense that just doesn’t work.
Not even in make-believe land.
Which leads me to my last point: This is the real world, people. Where the stakes are: If you screw companies out of hefty consulting fees and they follow your made-up advice, they lose money and people end up losing their jobs. Real people with real families who depend on them. Because of you. So if you lie about what you can and can’t do, shame on you for being part of the problem instead of being part of the solution, and for preying on open-minded execs who stick their necks out to try to do something cool for a change. Shame. Really.
So… Social Media “experts” and measurement “gurus” or whatever you like to call yourselves this week… Stop with the bulls**t. If you don’t know how to calculate Social Media’s impact or its R.O.I., don’t just make something up. Admit that you don’t know how to do it, and go talk to someone who does know so they can help you. If not for your own sake, for the sake of the companies who PAY you to be their trusted advisors and need your “expert” help. It is the professional (and the ethical) thing to do. Don’t just take the cash and make it up as you go. Come on.
Okay, rant over for now. Have a great weekend. We’ll continue this discussion next week, and my videos probably won’t be so very nice. 😉