Perspectives on Consumers

Not that long ago, I conducted a study in multiple markets.  In one market, recruiting was done in the traditional way:  all by phone.  In another market, an email blast was sent out to filtered database members to see if they qualified based on category usage alone.  In the third market, I later found out that the entire screener had been sent to filtered database members, with follow-up personal validation.

Truth is, I couldn’t tell the difference in the quality of recruits.  They were all excellent.

A positive aspect of online recruiting is that you can screen someone all the way through without termination points and it doesn’t impact costs and/or you can prevent the use of the “back button” in the survey so that people can’t  “beat the system” by changing/anticipating answers.

However, if online recruiting is to be done, qualitative researchers and facilities need to consider several  things:

• Is the target online?  It might be faster, more accurate, and less expensive to administer the screener online, but if the target isn’t represented online as it might be in the general telephone population (or even if they don’t check their email as often – a major problem I’ve noticed before), other options should be employed.  Some stats:
  • Only 79% of adults are online, according to a May ’10 Pew Research report; this doesn’t vary by urbanicity, although African Americans and Hispanics (English and Spanish speaking) are less likely to have Internet access (70% and 64%, respectively, Pew Jan ’10).
  • As expected, Internet access is lower among older adults and those less well educated (only 38% of those age 65+ have Internet access,  coupled with only 39% of those with less than a high school education and 63% of high school grads,  Pew Jan ’10.)
• Security screening as it’s been done in the past needs to be re-worked for an online environment.  (It probably should be re-worked anyway.)  It’s fairly easy to focus on a potential recruit’s employment, but harder to get at immediate family “conflicts.”  But it’s doable.
• Just as online quantitative surveys required alternative question types and scales to increase engagement, online self-administered qualitative screening questionnaires need to be written differently.  This applies to questions about a participant’s aptitude for creativity, problem-solving, influencing others, etc., as well.
• For quality control:
  • Qualitative researchers should be approving skip patterns and the like for the online survey versions vs. just passing off the screener in print form.  (After all, the quality of the questionnaire creates an initial impression for the project, just as a personal recruiter does.)
  • Personal follow-ups with new types of validation questions are needed to ensure the person recruited is who they say they are.
  • Moderators should be hyper-vigilant in their own re-screening to ensure the newer methods are working as they’d like.
  • The facilities need to build systems that verify respondents are answering the same way on key demographic questions from study-to-study – something that would increase the integrity of participants, however they’re recruited, for all studies.

Recruiting, like much of qualitative research, is both an art and a science.  Technology and the Internet affords us other options for recruitment.  As time goes on, other issues are likely to emerge.

For now, these are my thoughts.  I’d love to hear what other moderators and clients have to add!

In qualitative research, we’ve long sought out not only representative participants, but those who would engage with us.  To that end, many of us have developed screening batteries for things like articulation and/or creativity.

With the introduction of online focus groups and bulletin boards, new mini-batteries were created to make sure that people were comfortable with the technology and/or comfortable communicating in written form.

In the Summer 2010 issue of QRCA’s Views magazine, there was an interesting piece entitled “Emergent Consumers and New Product Development” by Praveen Kopalle, Ph.D., Tuck School of Business Administration.  What caught my eye was the discussion of the value of Emergent (vs. Lead) users.

“Lead” users we’ve often heard described as early adopters.  They are also people who might know what their needs are going to be, often because of their high level of involvement in the category; as such, they’re often ahead of the curve of the mass marketplace in being able to articulate these needs.

“Emergent” consumers are described differently.  Dr. Kopalle’s studies lead his team to conclude that there were consumers who had an almost “instinctive” ability to understand new concepts – even if they weren’t highly involved in the category – and that findings using these types of consumers in new product development lead to products which were more likely to be adopted by the mainstream.  There are eight questions in this “emergent consumer” battery, items such as:

• “Even if I don’t see an immediate use for a new product or service, I like to imagine how people in general might use it in the future,” and
• “When I see a new product or service idea, it is easy to visualize how it might fit into the life of an average person in the future.”

While I find this battery very intriguing, I’m a bit concerned that the only people who could answer all these questions confidently/accurately are those with lots of new product/service experience.  Maybe this is a new variant of “Prosumers” that has been discussed over the past two years or so.   And maybe my concerns are unfounded.

Today I was watching a TED talk by Chris Anderson on “How web video powers global innovation.”  A very interesting talk about how “crowds accelerate innovation.”   And when he described the “crowd” aspect, he put up this image:

In a crowd, he said, all these various types of people exist.  You and I might be able to add to this list (e.g., wall-flowers, copy-cats), but to my way of thinking, this is spot-on and a great springboard for refining recruiting screening criteria.

• For social media/communications, I might want to focus on Commentators or Superspreaders.
• In website usability research, I might want a mix of Innovators, Trend-spotters and Skeptics.
• And Cheerleaders takes the idea of a loyal customer to another level.

For me, this adds another rich layer to a study design and recruit that is likely to lead to some very interesting sessions.  The questions aren’t written yet to get at these various groups, but conceptually I like the idea of where this, coupled with the Emergent consumer battery for new product/service concepts, could go.

Your thoughts?

Last month when I wrote about discourse/text analytics, I said that the field wasn’t fully developed yet and that for now I was leaning in the direction of market research online communities (MROCs) for two reasons:  they were a closed community of targeted consumers, and with a group of 200-500, it was manageable to read/review all their postings and interactions.

Within a couple of days, I attended a local QRCA meeting where social media was discussed at length.  It reminded me that I needed to clarify that my issues with discourse/text analytics were primarily on the quantitative level.  Clearly, any of us in marketing research, and specifically in qualitative research, have been following social media with great interest and using it where we can.

Today, my take is this:  social media review is another aspect of the due diligence I conduct when preparing to meet a client for the first time or when preparing a discussion guide for a study.  It’s always important to learn as much as we can and to have our fingers on the pulse of what’s being said.  (For the record, there are those who believe that qualitative social media mining is comparable to a qualitative research study.  I don’t agree with that point of view at all!)

The limitations I highlighted regarding quantitative text analytics apply to qualitative explorations as well.  Specifically, are you listening the “right” people online, are they a representative mix of your target?  And are the people that may have the most important things for us to hear saying it in an online public forum?

The advantage to qualitative exploration of social communications – a term I like better than social media – is that we are able to read the tweets and posts and do a better job of making sense of what we’re reading vs. machine classification (positive/neutral/negative or some other tagging structure).  More often than not , we still don’t know who the people are, but we can get an overview of what’s being said and the terms/tone being used.

The biggest hurdle we have is how to aggregate this information.  For the casual searches (those I do for myself, without client commission), I like Addictomatic.com and SocialMention.com.  (Images below.)  Addictomatic allows me to read more from their dashboard; SocialMention provides me with a directional overview and top keywords (often useful when I need to dig further).  For clients seeking more detailed, automated solutions, there is Radian6 and ObjectiveMarketer, among others.

I haven’t yet had a client give me their social-media report as a springboard for developing a qualitative study, but the time is coming.  Until then, I’ll remain proactive in including social-communications analysis as part of my process, in the best way I can.

What’s your take on including social media/communications analysis as a routine part of the qualitative process?

Addict-o-matic

SocialMention

For nearly two years, I’ve been researching alternative approaches to analyzing social media “chatter.”  Frankly, as the social media movement started gaining momentum more than five years ago, many of us knew the challenge ahead.  And I’ll be honest when I say that I’m still not sure there’s a good answer out there.

Discourse analysis, sentiment analysis, text analytics:  whatever you call it, you’re looking at a huge volume of information.  Unstructured information.

When CRM started in the 1990s, the challenge was to link customers to behaviors and make recommendations to keep them happy/buying more.  Think of the Amazon “recommendation” feature:  “people who looked at this, bought this” or “if you liked this, may we recommend…”  They were able to “mine” purchase data to make better shopper suggestions.

Text mining is infinitely more complicated.  The longer a blog post, Facebook comment, or product review, the more difficult it is to categorize.  Twitter posts and other microblogs are theoretically easiest because they’re so short.

How It’s Done.  Comments are filtered based on keywords (e.g., brand name).  Next, as I understand it, using proprietary algorithms (depending on the software), comments are grouped into three buckets:  positive, negative, and neutral.

The longer the post, the more likely the comment is to be classified as neutral.  This isn’t surprising.  Think of the complexity in nuance, sarcasm, slang, humor and differences in word usage across different cultures speaking the same language.  (Anyone watch the Emmy’s Sunday night and see Ricky Gervais?  He was laughing at Bucky Gunts’ name and it’s taken me 10 minutes to figure out why – so bad I can’t repeat it here!)

I’ve seen statistics showing that between 65-80% of all comments are tagged as neutral.  The number is somewhat lower among the microblogs – maybe only 40%.  Nevertheless, it’s obvious that huge biases can be introduced using this approach.

Elsewhere I’ve read that humans aren’t very accurate either.  Using Mechanical Turk (which isn’t something I’d personally recommend), humans agree only 79% of time on how items should be coded.

We should be moving forward in this discipline. Tom Anderson, of Anderson Analytics, agrees.  In a correspondence with him several months back, Tom said that he was using three different software packages and gained confidence based on the intersection their joint findings.  I’m not sure if more data is successfully coded or if this just provides more confidence in what he reports as findings to his clients.  I think this is very responsible, given the accuracy of single-solution approaches today.

Currently, I’m leaning in the direction of relying more on Market Research Online Communities (or Insight Communities), which straddle the space between pure discourse analysis and more structured qualitative or quantitative marketing research.  I view them as a hybrid, where I can get out of the way and follow the discussion.  With only 200-500 members, keeping up and staying accurate is more manageable.  Further, when custom research is needed, the recruit is a snap.

And sometimes we don’t need to build communities.  One may already exist because the product or person built their following online.  In this case, the fan group may be the “go to” community because it’s larger, more representative and more cost effective than creating a new community.  Think Justin Bieber, who was “discovered” by his YouTube videos.  Those subscribing to his stream, particularly in the early days, would have been a great insight-mining resource.

I’ll be keeping a close eye on both these topics as they appeal to both my quant and qual sides.  And if you have any thoughts, I’d love to hear and learn!

Some reading:

The best of the articles I read, BrandSavant:  http://brandsavant.com/the-hidden-bias-of-social-media-sentiment-analysis/

August 2010 Quirks has several articles on social media research

Mashable:  http://mashable.com/2010/04/19/sentiment-analysis/

I got a DVR five or six years ago.  I wasn’t exactly an early adopter, but I must say I love it.  I record everything, often watching a program a mere 20 minutes after it starts to avoid the commercials.  I know I’m not alone.

Don’t get me wrong.  Good advertising I really like.  But so much of it is bad – or stupid.  The DVR lets me watch commercial-free if I want to.

Among DVR owners in Nielsen markets (37%), Nielsen recently reported that 40-50% of commercials are watched during playback, up from earlier estimates of 30-40%.  (Source:  MediaDailyNews, 8/5/10)

I’m not sure what to believe about this:  Commercial viewership up among DVR owners?  Really?  Or did Nielsen change how they’re measuring things? One hypothesis I had was that more recent DVR adopters could be less sophisticated/less comfortable using the fast forward/jump feature on their remote controls.  But Nielsen says that DVR playback is happening more among younger and higher-income viewers, so I can’t imagine my hypothesis is true.

While DVR viewership isn’t universal, these “facts” got me asking a few questions:

• Who’s not watching ads – consistently?  Do they have any identifiable characteristics that we can plan around?
• What are the best, most effective ways to reach these ad-skippers in other media and/or via other ways on TV (e.g., product placement, sponsorships, etc.)?
• What defines “stopping power” in a TV ad today, and how is it measured, particularly among DVR viewers?   I would think that the first and last ads in a pod would have the highest viewership, yet is this reflected in the numbers agencies are getting to calculate impressions?  Related to this, for researchers/account planners, is there a method out there for testing ads in a program-embedded clutter and letting people skip as they would at home, to get at stopping power and ad effectiveness?  Plus, can ads be tested at fast-forward speed to assess their stopping power?
• What impact is all this having on brand awareness or brand image?  Have any tracking studies been done among ad-skippers vs. those who fast-forward through ads vs. ad-watchers to see what impact ad viewing behavior has on these critical brand measures?

As I write these questions from my logical left brain, I have to remind myself that how we take in information isn’t very well understood.  I might not “know” the latest musical phenom, but I always seem to have heard the tune before I learn the artist’s name or see their face – and this is without kids at home.  What goes on around us seems to sink in and have an impact.

I don’t know the answers to the questions above, and maybe I’ll dig around to learn a bit more.  Yet it’s clear that whether TV ads are skipped or seen, consumers continue to form impressions of brands.  And since no matter how hard we try, not everything that impacts them will be known or measurable, I feel more strongly than ever that having consistent brand values which shine in all we do – and being present where our target lives – is the best foundation for the relationships we want to have with them.