Snake-oil 2

Part 1: Return on Incompetence

Here’s a flash of obvious: When most of us don’t know how to do something, we typically know that we don’t know how to do it.

Let me illustrate: As much as I would love to be an F-18 pilot, I don’t know how to fly an F-18 (or any aircraft, for that matter). As a result, you don’t see me walking around in a flight suit  pretending that I am an F-18 pilot.

More to the point, you don’t see me advertising my services as an F-18 flight instructor.

For the exact same reason, you don’t see me trying to sell services as an ice-carving, cat wrestling or underwear-modeling instructor. Why? because when most of us don’t know how to do something, we have a) enough sense and personal integrity not to pretend that we do, and b)  enough professional acumen not to pretend that we are qualified to teach it.

Yet this line of logic (and basic sense of professional ethics) seems to escape a disturbingly large group of people who evidently have latched-on to Social Media as an easy meal ticket  – one to be earned, in many cases, on the backs of people and companies who don’t know any better.

To add injury to insult, most of these would-be Social Media “experts” aren’t even good at masking the fact that they have no clue what they are talking about. You would think that they would at the very least grasp the most basic building blocks of social media… like the fact that a Twitter follower, a Facebook fan and a YouTube subscriber are essentially the same thing, but evidently, even that simple of a concept escapes some of these folks. In far too many instances, they don’t seem smart enough to realize how little they understand about a discipline they claim to be experts in.

Yet these are the people who increasingly find themselves advising companies around the US as to how to build, integrate, manage and measure social media programs.

Now, those of us who actually do this for a living and take it seriously can spot these posers a mile away, but how are unsuspecting executives supposed to? To the uninitiated, anyone with a neat sounding social media measurement formula and a whiteboard presentation can sound like they know what they’re talking about. (If you had never seen what gold looks like and a self-proclaimed expert came along with a bag of yellow dust and told you it was gold, why wouldn’t you believe them?)

More often than not, the otherwise innocent combination of inexperience, ambition and lack of professional accountability are to blame. But in far too many cases, an unhealthy blend of raw opportunism and… economic conditions  have given rise to a mob of social media snake oil salesmen. Whatever the causes may be, it worries me to see how quickly both dangerously inexperienced amateurs and deliberate hacks alike have taken over the “social” management business with their special brand of complete nonsense.

Perhaps more troubling: Very few among those of us working hard to build up this discipline are speaking out against bad practices and obnoxious BS from this unscrupulous crowd.

Look, leadership isn’t just about thought leadership. It isn’t just about being pioneers or evangelists or mavens. It certainly isn’t just about getting the accolades and the followers and the invitations to speak at conferences.  Leadership is also about responsibility. And we have a responsibility to keep our community and this field as ethical, professional, and free of BS as we can.

If leaders in this field don’t stand up for its integrity, who will?


Return on Inaction:

The reality is that the hijacking of Social Media by hack jobs isn’t happening in a vacuum: Every instance of an agency, firm or consultant promising results and delivering a goose egg makes it that much less likely that their  clients will put their trust in a social media adviser again, genuine or not. And rightly so.

To outsiders, there is no difference between a Chris Penn (who knows his shiznit) and a Joe Shmoe (who couldn’t find his way out of a paper bag if Twitter drew him a map), and that is a big problem: Anyone with a little bit of SEO savvy can have his/her bogus methodology pop up at the top of Google searches. And if no one calls him/her on the BS, there will be no indication anywhere that what s/he is selling is complete nonsense.

Case in point: How many companies have already fallen for misguided methodologies like these? (I use the term misguided since as far as I can tell, their authors seem to have developed these methods in good faith.)

Social Media ROI Calculator no.1. (Go here for the analysis.)

Social Media ROI Calculator no. 2.

Social Media ROI Calculator no. 3.

Bogus Social Media ROI measurement methodology no. 327.

And then there’s  this conversation on LinkedIn. Count how many of these folks are consultants. Tell me that level of widespread confusion and ignorance about the mechanics of Social Media doesn’t open the door for abuse and nonsense on a large scale.

We can do better.

If you believe that the hacks and misguided amateurs will eventually go away on their own, you’re wrong. Why would they? The money isn’t bad, the opportunities are growing and there is still virtually zero accountability in this line of work. Every other company is looking for a social media expert to teach them how to either develop, integrate, manage or measure social media, yet the vast majority of business execs couldn’t tell a true professional from an agency flunkie with a Facebook account. You do the math.

We’re going to be on the wrong side of that growth curve for while unless we start establishing standards for the industry. And I mean sooner rather than later.

How Not To Measure Social Media – Part 2

Here is Part 1.

Below is another example of social media nonsense passing for expertise. I don’t want to make any assumptions about anyone’s motives in this specific case, so to be fair, it’s probable that whomever put this together genuinely thought that the thinking behind the equation and methodology was sound. There is no reason to think that anyone at Digital Royalty was trying to make a quick buck off unsuspecting clients when they developed this. (My guess is that they mean well.) But the fact remains that the people behind this thing don’t understand either Social Media or program measurement well enough to teach either. And that’s the danger: Regardless of people’s intentions and motives, bad methodology is bad methodology no matter how you look at it. And bad methodology quickly turns into bad business for everyone involved.

But don’t take my word for it: Watch the video and make up your own mind. (Watch carefully because I’ll have questions for you afterwards.)

And here is the “equation” referenced in that video:


If the video doesn’t play for you, go watch it here.

Rather than listing out all of the flaws in this methodology and equation, let me ask a few questions that outline some of my key concerns. You can try to answer them yourselves or go straight to the answer/comment. You choice. (These were questions I asked the author on her blog post. They remain unanswered.)

1) Why do FB fans and Twitter followers fall into the volume column but YouTube subscribers fall into the engagement column?

Twitter followers, Facebook fans & friends and YouTube subscribers fall into the same category. They should be in the same column. (The “reach” column – not featured in this methodology.) Reach is neither hot nor cold. It’s a hard metric.

Demonstrating a lack of understanding about something as basic as this throws up a bright red flag right off the bat.

2) Why do frequency and reach apply to volume but not engagement or conversions? (Warning: This one may hurt your brain, so feel free to skip ahead.)

Assuming that Volume and Engagement are relevant categories/columns, (and that’s a very big if) Frequency and Reach would apply to both.

Let’s take a step back and look at the definitions of Frequency and Reach:

Frequency is a measure of how often something happens. (An activity, a transaction, etc.)

Reach is a measure of how many people you can touch. To use Amy’s own ecosystem terminology, reach is the number of people who live in your ecosystem.

A subset of reach is how many people within that ecosystem you actually touched or engaged with for a particular campaign.

Once you realize that, you start to see how the concept of a volume column is a bit shaky: You would have to include “reach” as an element of the “volume” column even though they are basically two words describing the same thing. See how this makes no sense?

But I digress.

Based on this model, Frequency (of interactions) should also show up in the engagement column: (How often do you engage?) Likewise, Reach would manifest itself in the engagement column in terms of how many people were touched/reached through… engagement.

If a Conversion column exists, then Frequency also applies to it: Frequency of conversions = how often conversions happen. (Or transactions, for that matter.)

Regardless of how flawed your method may be, this is simple, basic, common sense stuff that can be plugged in properly IF you understand it.

The lack of basic understanding of reach and frequency raises another red flag.

3) What does “actual activity and action” mean?

Transactions? Website visits? Who knows? Knowing what you’re talking about matters. Letting other people what you’re talking about matters too. “Actual activity and action” doesn’t mean anything.

Another red flag.

4) The size of your “ecosystem” = Reach. How can reach fall into both the cold and warm columns?

The size of an ecosystem is the definition of reach.  Calling something two different things doesn’t actually make it two different things. (This doesn’t happen when you know what you’re talking about.)

Having the same element appear twice in the same equation is bad math.

The red flags keep popping up.

5) Return on Influence = Adding 3 web analytics values to sentiment x reach (again)? Seriously? That’s the equation?

Before we dive deeper into this, the equation needs more work, starting with the fundamental flaws I already outlined.

Once you’ve revamped the basic elements of the method and equation, you have to then understand how the pieces fit mathematically (assuming they even do). Throwing a bunch of unrelated values together does not constitute serious social media measurement.

6) What unit of measure do you use? If it’s a “return,” what are we talking about? What does the number mean? What is it in reference to?

If the equation is supposed to create an influence index, then it doesn’t need a unit of measure. Just call it a Social Influence Index (SII) or whatever you want, and you’ll be fine. Whatever number the equation spits out is your “score”. Fair enough. There is nothing scientific about it, but you can probably get away with it.

But if you are going to call it “return on X,” you need a unit of measure. In real Return on Investment, you distill the investment in time, human capital and other resources to $$$. The “return” on that investment therefore is calculated in $$$ as well. Money is the common unit of measure.  The ROI equation gives you a ratio of money invested to money earned. The same rules apply here: If this is to be a “return on influence,” what unit of measure is used to calculate that ratio of return?

Without a clear unit of measure, you cannot calculate “return” on something. Basic 101 stuff. Learn it.

7) If “warm” data is “intangible and hard to measure,” why is it part of the equation to begin with?

Either it’s soft data and thus anecdotal or it is hard data and it can be used here. You have to pick one. You can’t claim that “warm” data is intangible/hard to measure and then add it to your “equation”. This is not a gray area.

Data isn’t cold, warm or hot. There’s only reliable data and unreliable data.

8) (Just added) Why does the equation on the board not match the equation in the graphic?

I knew something else about that equation was bugging me, but I couldn’t quite put my finger on it. Yet it was under my nose the whole time:

In the video, the “cold” half of the equation is interpreted as (Page Views x Visits) / Time Spent

In the graphic, the “cold” half of the equation is interpreted as (Page Views x Visits) + Time Spent

Which is it? Do we divide by Time Spent or do we add Time Spent?

(Thanks to @AnnaObrien for pointing it out.)

Incidentally, Frequency is not defined either in net page views or net unique visits either…

… And most of the KPIs explained early in the video seem to completely vanish…

… but by this point, who cares.

I could go on and on and on, but you get the picture: At first glance, the equation and the methodology look solid to the average person: Clean graphics, columns on a whiteboard, buzzwords we’ve all heard before, the promise of a digital agency backing it up, etc. Because the video and equation has already been posted on a variety of sites that fail to ask basic stink test questions by people who don’t know what flaws to look for, it makes its way to the top of search queries. Before you know it, complete nonsense goes mainstream and passes for real methodology because of three things: Decent packaging, Search, and confirmation of validity by other non-experts who barely skimmed the material.


This is how real Social Media expertise gets hijacked, and how Social Media management – as a discipline – loses credibility in the business world before it even gets a chance to prove its worth.

Last words:

This stuff isn’t just going to go away on its own. It’s great to see some folks in the Social Media management community speak out against this kind of BS, but they are the exception rather than the rule. At some point, the incessant retweeting and arse-kissing has to start making room for more relevant and responsible behavior: Call a cat a cat. When you see BS, don’t just look the other way. Call it out. Ask real questions. Put people to the test. That’s what leaders do. That’s what professionals do. We can’t keep allowing remedial BS to invalidate the real work being done in this space by those of us who actually care about it.

Rant over.