measurement – Page 5 – Andy writes!

Davide Savenije, assistant editor of the website Education Dive, recently posted his picks for The 10 best university Twitter accounts — and what they do right. The list has generated a lot of chatter on Twitter, especially among the fans of those schools who made the list. But there hasn’t been much looking into the methodology behind that ranking.

Enter moi.

The Dive’s list consists of big schools, many from “big” confrerences (Big 10, Big East, SEC) that have big followings — 208,000 for No. 1 selection @Harvard and 110,000 for No. 2 @Stanford — and impressive Klout scores. That is, if you find Klout scores impressive.

But Savenije knows bigger isn’t always better. He points out that “followers alone do not equal social media success,” which is why he included Klout scores and something he calls “our subjective appraisal” of the accounts. But take a look at the list, and it’s the numbers and Klout scores that jump out at you. Which might lead a reader to be skeptical of this compilation. Especially when “our subjective appraisal” is so ill-defined. It isn’t even ill-defined. It isn’t defined at all.

Maybe the descriptions of each account will give us some insight. Let’s pick on No. 6, The Ohio State University (@OhioState). Savenije writes:

The Ohio State University’s Twitter account keeps students, faculty and alumni apprised of the school’s goings-on. From student tips, daily photos and event promotions to alumni stories, faculty research and sports team notifications,@OhioState serves followers with the information they need and want.

Really. The school’s twitter account keeps not only students informed, but also faculty and alumni? (But apparently not staff.) That’s what sets this account apart?

Is that all it takes to attain Twitter greatness in the higher ed realm? That and a minimum following of 16,000 plus a Klout score of 83 or better?

I don’t buy it. I doubt you do, either.

So let’s come up with some metrics that make sense for higher ed Twitter accounts.

Please tell me in the comments below (or on Twitter): What makes a great university Twitter account? Feel free to share examples of schools that you think are doing it right (regardless of size of audience or Klout score) and suggest some ways to measure success. Your thoughts just might make their way into a future blog post on this subject.

P.S. – I’m not taking anything away from the 10 schools on this list. They’re all great and have robust social media efforts. I’m just questioning the validity of the metrics. (Something I picked up from administrators dissing the U.S. News & World Report rankings.)

P.S. to Education Dive: In your blog post, is the “S” in “University of Oklahoma” supposed to be larger than the other letters in that name?

A recent “social media ROI” report from the University of Massachusetts Dartmouth talks about how college and university admissions offices view and use social media. The report summary trumpets the findings as an indication that social media is significantly altering the way colleges and universities recruit students. The report (or at least the headline) even goes so far as to trumpet social media tools as “a Game Changer.” The headline and report also tout the return on investment (ROI) social media tools bring to higher education.

The report is filled with some good information, but I couldn’t really pinpoint any actual ROI indicators.

I’m no ROI expert, but it seems to me that the authors of this report have confused cost reductions with ROI. For instance, the report points out that institutions surveyed “report 33% less spent on printing, 24% less spent on newspaper ads and 17% less spent on radio and TV ads.”

But reducing costs, or reducing your media spend, isn’t the same thing as investing. Cutting costs does not equal increased investment.

One uncritical report about this research (posted on businessgrow.com) attracted the interest of several higher ed folks, including three very bright guys: Andrew Gossen of Cornell (@agossen), higher ed consultant Michael Stoner (@mStonerblog) and Mike Petroff, a digital strategist at Harvard (@mikepetroff). Gossen and Stoner took to Twitter to chat about the findings with a more skeptical view. But of course, there’s only so much you can discuss 140 characters at a time.

So Stoner visited businessgrow.com to share his thoughts on the subject, then expanded them in this blog post on his own site. (Tracy Playle of Pickle Jar Communications — @picklejar — also does a fine job of dissecting the Massachusetts Dartmouth study.) Both Stoner’s comment and his blog post are worth reading, as he eloquently discusses the nuances of higher education that the Massachusetts Dartmouth study seems to overlook.

The money quote, however, comes from Stoner’s comment on businessgrow.com blog:

[T]he story is more complicated than The University of Massachusetts Dartmouth Center for Marketing Research data indicate.

For example, Noel-Levitz, an enrollment and admissions consulting firm, does a yearly study of prospective students to see what they say about various channels (in 2012, 2000 students, 51% high school seniors). This year, 72% said they used “brochures/mail” to research colleges. When asked the best ways to learn about academic programs (the most important content to them), 71% said brochures were best; 38% cited social media as the best source. Only 10% said that social media was the best source of info for financial aid and 35% for information about the campus location and community. So while they do use Facebook (and other social channels), these aren’t the only, or even the primary, sources of information.

Indeed, this confirms my observation that adults continuously over-estimate the appetite of teens for whatever it is adults we think they should be interested in. We’ve seen from our own experience that teens love printed pieces — they just aren’t impressed with the kind of glossy viewbooks that most admission offices send out.

Interestingly, just a week after that Massachusetts Dartmouth report came out, businessgrow.com’s report about another study — the Social Habit Research Project — provided the perfect definition of ROI for businesses. It’ comes from social media marketer Rhonda Hurwitz, and it’s a definition that can be modified to fit the realm of higher education:

For me, it always comes down to figuring out how to connect social media usage or activity to revenue. I would ask “have you ever bought a product or service due to a social media interaction” … “have you ever recommended a product or service to others via social media?”

So, for higher education, perhaps the ROI questions should be:

Has a prospective student ever decided to attend a college due to a social media interaction?
Has a prospective donor ever given to his or her alma mater due to a social media interaction?
Has a current student ever recommended a college or university to a prospect via social media?

And perhaps we should be asking them of our customers (students, alumni, etc.) rather than those in charge of delivering the services, recruiting students or raising money. Maybe then we would gain a better sense of the true ROI of our social media activities.

* Disclaimer: I am no ROI guru. Far from it. But I agree with Stoner when he says, in his recent blog post on this topic, that “Measuring ROI is complicated, especially in higher ed.” It’s messy, and it’s a hassle. And that’s probably why we don’t do much of it.

Image: Clara Peller as the “Where’s the Beef?” lady from Wendy’s classic early 1980s ad campaign.