How We Conduct Social Listening Research—Part One

Gear up for some Campus Sonar behind-the-scenes! I’m covering how the Research team at Campus Sonar conducts social listening research, start to finish, to pull back the curtain on what we do from project kickoff to presentation of findings and insights to the client.

My 30-second elevator pitch for the work my team does usually goes something like this:

We deliver research (that’s conducted online) for colleges and universities to help campus professionals make informed strategic decisions. With the online data we collect, we can answer a variety of questions (like the following examples) and develop custom insights that are actionable for your campus.

    • What is the online reputation of my campus? 
    • What do certain key audiences think of my campus?
    • Is my online reputation aligned with my campus’s brand? How does the branded content we produce compare to the content others produce online about my campus?
    • Is there a market for recruiting for a specific program in a certain area of the U.S.?
    • What did people *really* think about the negative events that happened on campus last year?
    • Do you have another question you’d like answered about your campus’s perception, brand, event, or key audience? My team loves exploring what’s possible with online research.

Sometimes, we work with colleges and universities to identify key audience members that talk about them online. For example, prospective students who indicate interest in a particular campus—whether the mention is untagged on Twitter, deep in a forum, or a student-athlete in a local newspaper. Our Client Success team generally follows up with our clients when a key audience member mentions them online. This information helps our clients with yield, understanding issues important to audience members, and targeted, human-first communication.

So that’s clear as mud as to how we conduct social listening research, right? Blog post, done! 

Question or Key Audience → Social Listening Online Research → Custom Insights

Just kidding. There’s so much more I want to dig into and share for how the magic actually happens in and between those three phases.

This is closely tied to one of my favorite things to showcase at Campus Sonar—the expertise of the analysts on my team. When you work with Campus Sonar, you aren’t buying software and you don’t just work with people who click buttons and let artificial intelligence (AI) do the work. You get real human beings who make hundreds of decisions as they work on your project from developing the research question to insights based on our relationship with you. Analysts partner with our Client Success team members to ensure that they know—inside and out—your research questions, goals, brand, unique social media accounts and hashtags, points of brand confusion, and much more. This knowledge informs every step of our research process.

From left to right: Data Analysts Reed, Amanda, and Sarah looking at data

Campus Sonar Research Process Overview

  1. Develop research questions and define project goals.
  2. Write a custom query (search) to collect online data related to project goals.
  3. Segment the data collected into categories custom to the project.
  4. Validate the data collected and segmented.
  5. Analyze and build insights, in partnership with the Client Success team.

Part One: Planning and Query Writing

Why Are We Here, Anyway?

When we begin a new project, there’s a meeting of the minds: Business Development, Client Success, and Research. These experts come together to share project information and outcomes, and generally level set what they, as a project team, hope to accomplish. At this stage the Research team focuses on a few key things.

  • What type of project is being delivered?
  • Who on campus will use the research we execute and why?
  • What are the main outcomes to uncover or key research questions to answer?
  • What is the project timeline and important milestones?
  • Other information that needs to be clarified to begin the project, either internally or from the client.

You’re Not Just Known as Sampleton Northern University

Once the initial project details are squared away, the research tasks can start. For this post, I’m describing the tasks and decisions for a project related to assessing the online reputation of an entire campus. To do so, the analyst rolls up their sleeves, puts on some headphones, and conducts preliminary research on the client. To build a query that pulls in a robust dataset, the analyst meticulously finds key data points to use in building a custom query for each campus we work with.

References to Your Campus

You’re not just known as “Sampleton Northern University.” Sometimes you’re known by Sampleton or Sampleton Northern, your mascot, or “SNU.” The query we build should reflect all of the ways people refer to your campus to bring in data that reflects that variety. From information provided by campus professionals to a quick search of popular social sites, the analyst captures all the ways people refer to your campus.

When it comes to actually putting those terms in a social listening query, it’s part science (knowing what Boolean operators to use) and part art (figuring out how to put the key terms together in a way that makes sense). In the case of Sampleton Northern, what would happen if I simply put “SNU” in my query—telling my social listening software to capture all publicly-available instances of “SNU” on the web?

Well, I’d gather all instances of SNU for Sampleton Northern, but I’d also gather online data related to Southern Nazarene University, Shiv Nadar University, and Spiritualists’ National Union because people use the same acronym to refer to those institutions online.

Image of building on SNU campus

What’s an analyst to do? Project dependencies may dictate one (or a combination) of several options:

  1. Leave “SNU” in the query and pull in all mentions using the acronym. Once the data is collected, data that doesn’t refer to Sampleton Northern can be marked as irrelevant at the data validation stage. This might be a heavy lift depending on how much extra data is captured by leaving this term in the query.
  2. Leverage Boolean operators like AND or NEAR to narrow down which mentions containing “SNU” should pull in. For example, we could specify that we only want to collect “SNU” in online data when it appears on the same page with the name of the campus mascot. This approach is pretty focused, however, the tradeoff is that we exclude “SNU” mentions when the mascot isn’t mentioned.
  3. Specify what your query should exclude based on what you know isn’t relevant using the NOT operator. In this case, you’d want to exclude mentions of the names of organizations that use “SNU” but aren’t relevant: Southern Nazarene University, Shiv Nadar University, Spiritualists’ National Union … And as appropriate, exclude the names of their mascots, names of online campus-owned publishing accounts, or even the geographic locations. In this way, you exclude data you know is irrelevant that overlaps with the desired relevant “SNU” data. Here, the analyst considers that there is still a lot of data validation, but the volume is lower because of excluding names of known irrelevant data.
  4. Consider specifying where, geographically, you want to pull in online data that uses “SNU.” If Sampleton Northern is located in Minnesota (hey der!), specify that your query only captures mentions published within Minnesota. The analyst knows that because of limitations with social listening tools, this may affect the sites we pull data from as not all online data specifies its publishing location in metadata.

Deep breath … All of this happens in step one!? Yep—and it gets even more fun from here on out. (I really mean that—query building is an exercise in strategy and creativity. I’m glad we’re on this journey together). As the query is written, the analyst tests the strings of Boolean with different approaches (some listed above) to expand the social listening net, if you will, just broadly enough to bring in more relevant mentions, but narrow enough to exclude what is known as mentions irrelevant to the campus.

A Comprehensive Audit of Owned Publishing Online Accounts

Bear with me, just two more steps to query writing. The second aspect of query writing is ensuring that we capture online data from campus-affiliated accounts that publish content representing you. This is an important strategy to expand the social listening net related to a campus’ online reputation. If you agree that key audiences for campuses include prospective and admitted students, current students, parents, alumni, faculty and staff, I can tell you that these groups pay attention to more than your official, verified campus page.

  • Prospective student-athletes follow campus athletic accounts.
  • Current students follow and interact with department-level accounts, from English to Biochem.
  • Alumni cheer on their alma mater by interacting with content from your advancement team.

There are many more examples, but suffice to say that because we know audiences important to higher education institutions pay attention to a campus’s entire online presence, we know it’s important to capture as much data as we can from as many campus-affiliated accounts that publish content online.

To do that, the Sonarian analyst ferrets out all active, owned accounts across the interwebs to include in their custom query, including, Twitter, YouTube, Flickr, Tumblr, and others. They use the names of these owned accounts with specific operators in the query to ensure they capture content published by owned accounts, but also shares and retweets as well as comments and replies to owned content.

Campus-Specific Hashtags

By this point, the analyst has a solid start on their custom query between campus references and names of owned accounts that publish content. We must briefly turn our attention to social media and the power of the hashtag to expand our social listening net. Our goal at this step is to understand (using equal parts divining, intuition, and investigating hashtags on Twitter and Instagram) what hashtags key audiences use on social to refer to your campus.

Even if they don’t use any of the terms we uncovered as ways to refer to your campus, if they use a common hashtag, it helps us gather more relevant data. We start with hashtags referring to your campus (#Sampleton, #SampletonNorthern), but employ some of the same strategies used with general references to hashtags like #SNU.

Reed with social logos swirling around his head

For some campuses, we find that what they consider an “owned” hashtag unique to them, actually isn’t. That also adds some nuance to query building, but it’s a topic that Amanda covered extensively in Owning Higher Ed Hashtags.

Leveraging Unique Operators

As the query is built, it’s re-worked and tested until the analyst gets to a point where they do some fancy footwork to push the boundaries even further by creatively using Boolean operators. This ability comes with curiosity, experience, and an innovation orientation. At this step, the analyst considers how to write Boolean that pulls in campus-related mentions when their name is used:

  • In the URL of a website.
  • In a URL of a link that’s tweeted or retweeted.
  • In a Tumblr tag.

Including terms with unique operators pushes the bounds of our social listening net again. Those tweets where there’s no text, just a link to the campus’s athletic website? We’ll grab it. A Tumblr post that is simply a meme but tags them? We got it covered. This step is both art (figuring out the right terms to use with unique operators) and science (systematically testing with the tools at hand to ensure we’re using operators and terms in an appropriate way).

In Part Two, I’ll showcase the next step in our research process: coding and cleaning the data so that it’s segmented appropriately and squeaky clean before analysis. Watch for the next post!

Wavy line


Subscribe to the NewsletterThis post originally appeared on Campus Sonar's Brain Waves blog.