Choosing the Right Text Analytics and Sentiment Analysis Tool to Analyze Your Open-Ended Responses

Open-ended responses provide some of the most valuable insights for guiding business decisions, yet are often underutilized (or outright ignored) when analyzing market research surveys and other datasets with open-ended or textual responses. By leaving this key data on the table, customer loyalty metrics, employee engagement data, innovation research, and more go untold. Critical opportunities are lost or—at best—acted on with limited insight.

Market researchers often report feeling frustrated that the data they’re collecting, and know to hold key understandings, goes unused—its potential untapped because to analyze would be a long, arduous process that they don’t have the time or resources to execute.

Ascribe CX Inspector was designed to meet the need for an efficient text analytics tool. With CX Inspector, gaining insight from verbatim comments is not only possible… It’s simple. Manually analyzing the open-ended responses sitting on your desk could take days or even weeks, but with Ascribe CX Inspector, a leading tool for sentiment analysis, that data becomes usable knowledge in a matter of minutes.

Analyze Large Amounts of Open-Ended Responses from Datasets Quickly and Easily

Ascribe CX Inspector is a customizable text analytics software that allows users to gather and sift through open-ended data quickly and easily. CX Inspector integrates structured and unstructured data using topic and sentiment analysis to better understand the responses provided in customer satisfaction, employee engagement, concept idea tests, and other valuable datasets.

X-Score

CX Inspector uses a patented tool called X-Score to efficiently measure satisfaction and sentiment and identify the key drivers of positive and negative sentiment, or satisfaction and dissatisfaction. A score between 0-100 indicates positive sentiment or satisfaction, while -100 to 0 indicates negative sentiment or dissatisfaction. X-Score helps businesses uncover crucial next steps for improving the score and driving real business results, and can be used to benchmark results over time.

Driven by Sentiment and Topic Analysis

CX Inspector’s interactive and customizable dashboards empower market researchers to explore their data by filtering through key variables to better understand who or what is being affected—zero in on individual comments or focus on the top-level themes that matter most to you and your business. You can also analyze and create reports, like crosstabs and co-occurrences, to uncover knowledge to improve your understanding of the data.

Ease of Use

Ascribe CX Inspector is a leading text analytics tool for insight professionals and market researchers who have large datasets with open-ended comments that they would like to analyze, but haven’t found the time, money, nor right solution to do so. CX Inspector empowers you to analyze thousands—even hundreds of thousands—of open-ended responses in minutes (compared to hours, days, or weeks).

CX Inspector includes user-friendly features such as:

• Interactive, customizable dashboard
• ‘Drag and Drop’ grouping capabilities
• Ability to click through and see individual comments
• Translation from 100 languages
• Ability to filter by any variable
• PII removal and profanity clean-up
• Import and export of data is quick and easy

Ascribe CX Inspector uses advanced AI and NLP to smoothly integrate data and generate swift and accurate results. And if data integration is an issue, Ascribe offers custom API services to support data importing.

How to Use CX Inspector for Text Analytics and Sentiment Analysis

As any business owner knows, it costs more to gain a new customer or onboard a new employee than it does to retain an existing one. Text analytics and sentiment analysis from CX Inspector bridges the gaps between your business and your customers or employees by uncovering key sentiment, engagement, and loyalty insights.

But CX Inspector isn’t just limited to customer satisfaction and employee engagement datasets—in fact, it can be used to analyze open-ended responses from:

• NPS studies
• VOC research
• Advertising copy tests
• Various tracking studies
• Innovation research
• Concept idea tests
• And more

Whatever your data represents, CX Inspector empowers you to ask the valuable open-ended questions that previously felt off-limits due to time-consuming, costly, and complicated analysis. Already swimming in a sea of data? Finally make it work for you by analyzing it in a matter of minutes, and interpret it in a variety of meaningful ways that support your business.

CX Inspector is Your Solution to Analyzing Open-Ended Responses

With key insights that could inform critical business decisions on the line, analyzing and understanding open-ended survey results is more important than ever. Whether you need to gauge customer satisfaction or employee engagement, test advertising copy or concept ideas, Ascribe CX Inspector is the text analytics tool for the job. Analyze your datasets in minutes, visualize and export results in whichever way is most meaningful, and uncover key insights to help guide your business decisions.

Analyze open-ended responses and gain key business insights with CX Inspector

https://www.youtube.com/watch?v=ng2NmoOwgc4

‍

The Ascribe APIs provide documentation both as static html pages and an interactive Swagger page.  The static documentation is more convenient for looking over the API, but the Swagger documentation provides powerful features for interaction with the API.  We will use examples from the Ascribe Coder API, but the same techniques apply to the Ascribe CXI API.

To open the documentation page for the API, navigate to the root URL of the API in your browser.  For the Ascribe Coder API this is https://webservices.goascribe.com/coder.  The top of the page looks like this:

Click the Documentation link in the toolbar to open the static help pages, or the Swagger link to open the Swagger documentation.  The top of the Swagger documentation page looks like this:

The REST resources available are laid out on the page like this:

Two resources are shown here, Codebooks and Companies.  Each of these has one GET operation and one POST operation.  Click on the operation to show details.  For example, if we click the POST operation for the Companies resource we see:

The example response produced by Swagger is a bit confusing.  Swagger wraps the response example in an object that specifies the content type, in this case application/json.  The actual response body is the value of this property, or in this case:

{
 "key": 3,
 "id": "ACME"
}

This POST operation accepts parameters in the body of the request, as described in the Parameters section in the operation above.  To get details about the fields of the response or request bodies, click on the Model link above the example.  In this case for the Response body we see:

And for the Request body we find:

Experimenting with the API from the Swagger page

You can experiment with the API directly from the Swagger page.  To do so, the first step is to obtain a bearer token for authentication as detailed in this post.  To obtain the token, POST to the Session resource.  Expanding the POST operation of the Sessions resource we find:

Click on the request example to copy it to the request body:

Now edit the values for the request body, providing your credentials.  I’ll log in to an Ascribe account called “Development”:

Now click the Try it Out! button.  Scroll down to the Response Body section to find:

The API has provided two tokens, either of which can be used for client interactions with the API.  You can elect to use either token, but don’t use both.  The tokens have different semantics.  The authenticationToken should be passed in a header with key authentication.  The bearer token should be passed in a header with key authorization.  The word bearer and the space following it should be included in the value of the authorization header.

The authentication token will remain valid for thirty minutes after the last request to the API using this token.  In other words, it will remain valid indefinitely provided you make a request with this token at least once every thirty minutes.  The bearer token never expires, but will become invalid if the account, username, or password used to obtain the token change.

The bearer token (but not the authentication token) can be used for experimentation with the API from the Swagger page.  To use the bearer token, copy it from the response body.  It is long, so it is easiest to copy the response body and paste it into a text document.  Then get rid of everything except the value of the bearer token.  This means everything within the quotation marks, including the word bearer, but not including the quotation marks.  You can save this value for future use so that you don’t have to go through this each time you experiment with the API.

Now, armed with your bearer token, paste it into the api_key box at the top of the Swagger documentation page:

Now you are ready to work out with the API.  For example, you can expand the GET operation of the Companies resource and click Try it Out!  Assuming you have one or more companies in your account, you will get a response like:

{
 "companies": [
   {
     "key": 11288,
     "id": "Language Logic"
   },
   {
     "key": 11289,
     "id": "Saas-fee rentals"
   },
   {
     "key": 11291,
     "id": "ACME"
   }
 ],
 "errors": null
}

Using Postman

The Swagger documentation page is handy for simple experimentations, but you will probably want a better API development tool as you develop your own API client.  Postman is one such tool.  You can easily import the API from Ascribe into Postman.  To do so, open Postman and click the Import button in the toolbar:

In the Import dialog, select Import from link.  Paste in the URL from the toolbar of the Ascribe Swagger documentation page:

Click Import, and Postman will import the API like so:

Adding the bearer token

That did a lot of work for you, but you still must tell Postman about your bearer token.  Edit the new Ascribe Coder API collection:

Select the Authorization tab and set the type to Bearer Token.  In the Token box, paste the value of the bearer token without the leading word bearer and the space following it:

Completing setup in Postman

Now you are ready to work out with the API in Postman.  Most of the operations are set up for you, but there are a few details to be aware of.

First, those operations that require a request body will not have that body properly populated.  You can either edit these by hand or copy the example body from the Swagger documentation into Postman as a starting point.

Second, be aware that Postman will create variables for operations that accept parameters in the request path and query string.  An example is the GET /Studies operation.  This operation accepts three parameters in the query portion of the url: idFilter, statusMask, and clientCompanyKey.  If we try this operation in Postman using initial configuration after importing the Coder API we get the response:

{
 "studies": [],
 "errors": null
}

Where are our studies?  You might expect to see all the studies for your account listed.  To understand why they are not listed, open the Postman console.  In the View menu in Postman, select Show Postman Console:

Now, with the Postman console displayed, try the GET /Studies operation again.  In the Postman console we see:

This is the actual GET request URL sent to the API.  The request accepts three variables as parameters, and we have not supplied these variables.  We can tell Postman not to send these parameters by opening the Params section of the GET request in Postman:

Clear the checkbox to the left of each of these parameters, then send the request again.  The Postman console now displays:

and the response body contains a list of all studies on our account.  To query only for studies that are complete, set the statusMask parameter value to c:

The console shows that the request now contains the desired status mask:

and the response body contains only those studies that are complete.

Using Swagger to Generate the API Client

The Swagger documentation can also be used to generate the initial code for your API client automatically.  Visit https://swagger.io/tools/swaggerhub/ for more information.  The code generation tools on this site can create API client code in a wide variety of languages and frameworks, including Python, Ruby, Java, and C#.

Summary

The Swagger documentation for Ascribe APIs does far more than simply documenting the API.  It can be used to create a test platform for API client development using Postman.  It can also be used to generate stub code for your API client.

The Ascribe Coder API allows your company to send survey responses to Ascribe and retrieve back the codebooks and coded responses.  Using the Ascribe Coder API you can create a study, create questions in the study, load responses to be coded to the question, and get the results of the coding.Before you can interact with the Ascribe Coder API you must authenticate.  See this post for details.The API interactions for this workflow are:

1. POST /StudiesCreate a new study. Its key is returned.
2. POST /Questions/{StudyKey}Create a new question in the study. Its key is returned.
3. POST /Responses/{QuestionKey}Put responses to be coded into the question.
4. Humans code the responses, creating a codebook as they code.
5. GET /Studies/{StudyKey}Query for completion of the coding work. Repeat at intervals until coding is complete.
6. GET /Codebooks/Question/{QuestionKey}Get the codebook constructed by the coders.
7. GET /Responses/{QuestionKey}Get the coding results.

Let’s go through each of these operations a step at a time.

Create a Study with the Ascribe Coder API

Assuming that we want to create a new study in Ascribe we POST to the /Studies resource.  The body of the request looks like this:{
 "id": "S458-3"
}This creates a new study with the ID S458-3.  The ID can be any text.  The API responds with:{
 "key": 544,
 "id": "S458-3",
 "status": "p"
}The status of "p" means that coding is in progress.  This is an indication to the coding team that this job needs to be coded.  The key returned is important.  That is how we will refer to the study in subsequent interactions with the API.If we don’t want to create a study, but would rather add a new question to an existing study, we can query the /Studies resource to find the study key given the study ID.

Create a Question with the Ascribe Coder API

Armed with our study key, 544 in the example above, we can create a question in the study.  To create the question, we POST to the /Questions/{StudyKey} resource, so in our case we POST to /Questions/544.  The body of the request looks like this:{
 "id": "Q1",
 "type": "o",
 "text": "What did you like about the product?"
}This creates a new Open question ("type": "o"), with ID “Q1”.  The response has this form:{
 "key": 3487,
 "id": "Q1",
 "type": "o",
 "text": "What did you like about the product?"
}Now we have the key for the newly created question, and we are ready to load it with responses to be coded.

Load Responses to Code with the Ascribe Coder API

To load responses, we POST to the /Responses/{QuestionKey} resource, so for the example above we would POST to /Responses/3487.  The body of the request would look like this:{
 "responses": [
   {
     "rid": "34-22907",
     "verbatim": "I do not like the red color"
   },
   {
     "rid": "34-22988",
     "verbatim": "This is a fantastic product"
   }
 ]
}And the response:{
 "responsesAdded": 2,
 "existingResponsesUnchanged": 0
}Each response in the array of responses sent to the API must have the rid (respondent ID) and verbatim properties set to strings of one character or more.  The rid values must be unique within the array of responses.  If the rid value for a given response is already present in the question in Ascribe the response will be ignored.  If we do exactly the same POST again to the API, the response would be:{
 "responsesAdded": 0,
 "existingResponsesUnchanged": 2
}This is because the respondent ID’s are already present in the question.You should limit the number of responses sent in a single POST to 1000 or so.  If you have more responses to load into the question than that, send them in separate POST operations.

Human Coding

The coding team can now get to work on the question.  As the coders do their work they will add codes to the codebook for the question.  Once coding of all the questions in the study is complete the coding team will set the status of the study to Completed.When the coding team has set the status of the study to Completed it is the signal that the coding results are available for retrieval via the API.

Polling for Completion of Coding with the Ascribe Coder API

You can test for completion of the coding work with a GET to the /Studies/{StudyKey} resource.  Our study key is 544, so we need to GET from /Studies/544.  The response looks like this:{
 "key": 544,
 "id": "S458-3",
 "status": "p"
}Whoops!  The coding is not yet complete, because the status is "p", meaning coding in progress.  Wait for a while and try again, until the status is "c", meaning completed.  Now we can get the coding results.

Retrieving a Codebook with the Ascribe Coder API

We need two things to collect the results of coding.  The codebook and the codes applied to each response.  We can retrieve these in either order.  To get the codebook we GET from the /Codebooks/Question/{QuestionKey} resource.  In our example we would GET /Codebooks/Question/3487.  The response has this form:{
 "codebookKey": 1499,
 "codebook": [
   {
     "key": 528,
     "description": "Not like red",
     "outputId": "201"
   },
   {
     "key": 529,
     "description": "Fantastic product",
     "outputId": "202"
   }
 ]
}This codebook contains only two codes.  Each has a key, textual description as displayed in the codebook, and an outputId.  The output ID identifies the code for the tab department.  But to associate these codes with the coding results from the API we want the code keys, 528 and 529.

Retrieving Coding Results with the Ascribe Coder API

The final operation in our workflow is to get the results of the coding.  We GET from the /Responses/{QuestionKey} resource.  The response has this form:{
 "responses": [
   {
     "rid": "34-22907",
     "codeKeys": [
       528
     ]
   },
   {
     "rid": "34-22988",
     "codeKeys": [
       529
     ]
   }
 ]
}The coders have applied code 528, “Not like red”, to respondent "34-22907".  528 is the key of the code in the codebook.  The other respondent has received the “Fantastic product” code.  While each response in this example has only one code applied, in general any number of code keys can be returned in the codeKeys arrays.Note that the text of the responses is not included in the response to reduce the payload size.  If you want the response text you can set the includeText property in the request body to true.  The response would then look like this:{
 "responses": [
   {
     "rid": "34-22907",
     "text": "I do not like the red color",
     "codeKeys": [
       528
     ]
   },
   {
     "rid": "34-22988",
     "text": "This is a fantastic product",
     "codeKeys": [
       529
     ]
   }
 ]
}

Summary

The Ascribe Coder API can be used to automate the workflow in your coding department.  You can send responses to be coded to Ascribe and retrieve the results of the coding operation.See the Ascribe Coder API documentation for information on how to automate coding of responses as they are sent to Ascribe.

The various Ascribe™ APIs, such as the Ascribe Coder API, share a common authentication operation.  You authenticate with the API by a POST to the /Sessions resource.  To interact with any of the other resources in the APIs you must include a token returned in the POST response in an HTTP header.

Obtaining an Authentication Token

To obtain an authentication token, POST to the /Sessions resource.  The body of the POST request contains the login credentials for an Ascribe user: Account, Username, and Password.  The body of the POST request has this form:{
 "account": "MyAscribeAccount",
 "userName": "apiuser",
 "password": "--------"
}Here we are authenticating as with the user name apiuser, on the MyAscribeAccount.  We recommend that you create an Associate in Ascribe exclusively for the API.  This is because the time accounting features of Ascribe will attribute activity initiated via the API to this user.  This makes interpretation of the time accounting reports easier.The POST response has this form:{
 "authenticationToken": "0e2338c4-adb4-474a-a26a-b9dbe1b32ac8",
 "bearerToken": "bearer eyJhbGciOiJSUzI1NiIsInR5cC ... WiiVzXPhdgSWNvH1hNOIirNmg"
}The value of the bearerToken property is actually much longer than shown here.  You can use either of these tokens (but not both) in subsequent operations with the API.

Differences between the Token Types

The authenticationToken returned by the /Sessions resource expires after 30 minutes of inactivity with the API.  In other words, you must perform an operation with the API at least once every 30 minutes.  After that you must obtain another token from the /Sessions resource.  The token will remain valid indefinitely if you do make a request at least once every 30 minutes.The bearerToken does not expire.  Once you have obtained a bearerToken it will remain valid until the login credentials of the user are changed.

Using Authentication Tokens

To use the authenticationToken returned by the /Sessions resource, include it in an HTTP header.  The key of the header is authentication, and the value is the authenticationToken.  The raw HTTP request looks like this:GET /coder/studies
accept: application/json, text/json, application/xml, text/xml
authentication: 207e5be1-c3fa-4ff6-bbc7-475a367ec095
cache-control: no-cache
host: webservices.languagelogic.net
accept-encoding: gzip, deflateTo use the bearerToken, use an HTTP header whose key is “authorization” and value the bearerToken.  The raw HTTP request looks like this:GET /coder/Studies
accept: application/json, text/json, application/xml, text/xml
authorization: Bearer eyJhbGciOiJSUzI1NiI ... lJqc7nPxcFdU3s90BkOPlPF6wTxkzA
cache-control: no-cache
host: webservices.languagelogic.net
accept-encoding: gzip, deflate

Summary

The Ascribe APIs require the remote client to authenticate.  The remote client authenticates by providing user credentials, and uses one of the two tokens returned in subsequent operations with the API.See the Ascribe Coder API documentation, and the Ascribe CX Inspector API documentation for more information.

Coding Open-Ended Questions

Verbatim coding is used in market research to classify open-end responses for quantitative analysis. Often, verbatims are coded manually through software such as Excel, however, there are verbatim coding solutions and coding services available to streamline this process and easily categorize verbatim responses. 

Survey Research is an important branch of Market Research. Survey research poses questions to a constituency to gain insight into their thoughts and preferences through their responses.  Researchers use surveys and the data for many purposes: customer satisfaction, employee satisfaction, purchasing propensity, drug efficacy, and many more.

In market research, you will encounter terms and concepts in data specific to the industry. We will share some of those with you here, and the MRA Marketing Research Glossary can help you understand any unknown terms you encounter later.

Seeking Answers by Asking Questions

Every company in the world has the same goal: they want to increase their sales and make a good profit. For most companies, this means they need to make their customers happier — both the customers they have and the customers they want to have.

Companies work toward this goal in many ways, but for our purposes, the most important way is to ask questions and plan action based on the responses and data gathered.  By “ask questions,” we mean asking a customer or potential customer about what they care about and taking action based on the customer feedback.

One way to go about this is to simply ask your customers (or potential customers) to answer open ended questions and gather the responses:

Q: What do you think about the new package for our laundry detergent?
A: It is too slippery to open when my hands are wet.

This technique is the basis of survey research.  A company can conduct a questionnaire to get responses by asking open ended questions.

In other cases, there may be an implied question and response.  For example, a company may have a help desk for their product.  When a customer calls the help desk there is an implied question:

Q: What problem are you having with our offering?

The answers or responses to this implied question can be as valuable (or more!) as answers and responses to survey questions.

Thinking more broadly, the “customer” does not necessarily have to be the person who buys the company’s product or service. For example, if you are the manager of the Human Resources department, your “customers” are the employees of the company. Still, the goal is the same: based on the feedback or response from employees, you want to act to improve their satisfaction.

Open, Closed, and Other Specify

There are two basic types of data to gather responses in survey research: open and closed.  We also call these open-end and closed-end questions.

A closed-end question is one where the set of possible responses is known in advance.  These are typically presented to the survey respondent, who chooses among them.  For example:

Open-end questions ask for an “in your own words” response:

The response to this question will be whatever text the user types in her response.

We can also create a hybrid type of question that has a fixed set of possible responses, but lets the user make an answer or response that was not in the list:

We call these Other Specify questions (O/S for short).  If the user types a response to an O/S question it is typically short, often one or two words.

Just as we apply the terms Open, Closed, and O/S to questions, we can apply these terms to the answers or responses.  So, we can say Male is a closed response, and The barista was rude is an open response.

What is an Answer vs a Comment?

If you are conducting a survey, the meaning of the term answer is clear. It is the response given by the respondent to the question posed. But as we have said, we can also get “answers” to implied questions, such as responses to what a customer tells the help desk. For this reason, we will use the more generic term comment to refer to some text or responses that we want to make an examination for actionable insight.

In most cases, comments are electronic text, but they can also be images (handwriting) and voice recording responses.

You need to be aware of some terminology that varies by industry. In the marketing research industry, a response to a question is called either a response or a verbatim. So, when reading data in survey research we can call these responses, verbatims, or comments interchangeably.  They are responses to open-end questions. As we will see later, we don’t call the responses to an open-end question answers. We will find that these verbatims are effectively turned into answers by the process of verbatim coding.

Outside of survey research, the term verbatim is rarely used.  Here the term comment is much more prevalent.  In survey research the word verbatim is used as a noun, meaning the actual text given in response to a question.

Survey Data Collection 

In the survey research world, verbatims are collected by fielding the survey. Fielding a survey means putting it in front of a set of respondents and asking them to read it and fill it out.

Surveys can be fielded in all sorts of ways. Here are some of the different categories of surveys marketing research companies might be using:

• Paper surveys
  • Mailed to respondents
  • Distributed in a retail store
  • Given to a customer in a service department
• In-person interviews
  • In kiosks in shopping malls
  • Political exit polling
  • Door-to-door polling
• Telephone interviews
  • Outbound calling to households
  • Quality review questions after making an airline reservation
  • Survey by voice robot with either keypad or voice responses
• Mobile device surveys
  • Using an app that pays rewards for completed surveys
  • In-store surveys during the shopping experience
  • Asking shoppers to photograph their favorite items in a store
• Web surveys
  • Completed by respondents directed to the survey while visiting a site
  • Completed by customers directed to the survey on the sales receipt

There are many more categories of survey responses. The number of ways to field surveys the ingenious market research industry has come up with is almost endless.

As you can see, the form of the data collected can vary considerably.  It might be:

• Handwriting on paper
• Electronic text
• Voice recording responses
• Electronic data like telephone keyboard button presses
• Photographs or other images
• Video recording responses

And so on.  In the end, all surveys require:

1. A willing respondent
2. A way of capturing the responses

The way of capturing the responses is easy.  The first takes us to the area of sample we will consider soon.

Looping and Branch Logic

Data collection tools can be very sophisticated.  Many data collection tools have logic built in to change the way that the survey is presented to the respondent based on the data or responses given.

Suppose for example you want to get the political opinions of Republican voters.  The first question might make the respondent provide his political party affiliation.  If he responds with an answer other than “Republican,” the survey ends.  The survey has been terminated for the respondent, or the respondent is termed.  This is a simple example of branch logic.  A more sophisticated example would be to direct the respondent to question Q11 if she answers A, or to question Q32 if she answers B.

Another common bit of data collection logic is looping.  Suppose we make our respondents participate in an evaluation of five household cleaning products.  We might have four questions we want to ask the respondents about each product, the same four for each product.  We can set up a loop in our data collection tool.  It loops through the same four questions five times, once for each product.

There are many more logic features of data collection tools, such as randomization of the ordering of questions and responses to remove possible bias for the first question or answer presented.

The Survey Sample 

A sample can be described simply as a set of willing respondents. There is a sizable industry around providing samples to survey researchers. These sample providers organize collections of willing respondents and provide access to these respondents to survey researchers for a fee.

A panel is a set of willing respondents selected by some criteria.  We might have a panel of homeowners, a panel of airline travelers, or a panel of hematologists.  Panelists almost always receive a reward for completing a survey.  Often this is money, which may range from cents to hundreds of dollars, however, it can be another incentive, such as coupons or vouchers for consumer goods, credits for video purchases, or anything else that would attract the desired panelists. This reward is a major component of the cost per complete of a survey: the cost to get a completed survey.  

Sample providers spend a lot of time maintaining their panels.  The survey researcher wants assurance that the sample she purchases is truly representative of the market segment she is researching.  Sample providers build their reputation on the quality of sample they provide.  They use statistical tools, trial surveys, and other techniques to measure and document the sample quality.

Trackers and Waves

Many surveys are fielded only once, a one-off survey.  Some surveys are fielded repeatedly.  These are commonly used to examine the change in the attitude of the respondents over time.  Researching the change in attitude over time is called longitudinal analysis. A survey that is fielded repeatedly is called a tracker. A tracker might be fielded monthly, quarterly, yearly, or at other intervals. The intervals are normally evenly spaced in time. Each fielding of a tracker is called a wave.

Verbatim Coding 

In the survey research industry responses to open-end questions are called verbatims.  In a closed-end question the set of possible responses from the respondent is known in advance.  With an open-end question, the respondent can say anything.  For example, suppose a company that sells laundry detergent has designed a new bottle for their product.  The company sends a sample to 5,000 households and conducts a survey after the consumers have tried the product.  The survey will probably have some closed-end responses to get a profile of the consumer, but to get an honest assessment of what the consumer thinks of the new package the survey might have an open-end question:

What do you dislike about the new package?

So, what does the survey researcher do with the responses to this question?  Well, she could just read each verbatim.  While that could provide a general understanding of the consumers’ attitudes, it’s really not what the company that is testing the package wants.  The researcher would like to provide more specific and actionable advice to the company.  Things like:

22% of women over 60 thought the screw cap was too slippery.
8% of respondents said the bottle was too wide for their shelves.

This is where verbatim coding, or simply coding, comes in. Codes are answers, just like for closed-end questions. The difference is that the codes are typically created after the survey is conducted and responses are gathered. Coders are people trained in the art of verbatim coding and often on a coding platform, such as Ascribe Coder. Coders read the verbatims collected in the survey and invent a set of codes that capture the key points in the verbatims. The set of codes is called a codebook or code frame. For our question, the codebook might contain these codes:

• Screw cap too slippery
• Bottle too wide
• Not sufficiently child-proof
• Tends to drip after pouring

The coders read each verbatim and assign one or more codes to it. Once completed, the researcher can now easily read each one of the coded responses and see what percentage of respondents thought the cap was too slippery. You can see that armed with information from the closed-end responses the researcher could then make the statement:

22% of women over 60 thought the screw cap was too slippery.

Now you can see why the responses to open-end questions are called verbatims, not answers.  The answers are the codes, and the coding process turns verbatims into answers.  Put another way, coding turns qualitative information into quantitative information.

Codebooks, Codes, and Nets

Let’s look at a real codebook. The question posed to the respondent is:

In addition to the varieties already offered by this product, are there any other old-time Snapple favorites that you would want to see included as new varieties of this product?

And here is the codebook:

• VARIETY OF FLAVORS
  • like apple variety
  • like peach variety
  • like cherry variety
  • like peach tea variety (unspecified)
  • like peach iced tea variety
  • like raspberry tea variety
  • like lemon iced tea variety
  • other variety of flavors comments
• HEALTH/ NUTRITION
  • good for dieting/ weight management
  • natural/ not contain artificial ingredients
  • sugar free
  • other health/ nutrition comments
• MISCELLANEOUS
  • other miscellaneous comments
• NOTHING
• DON’T KNOW

Notice that the codebook is not a simple list.  It is indented and categorized by topics, called nets, and the other items are codes.  Nets are used to organize the codebook.  Here the codebook has two major categories, one for people whose responses are that they like specific flavors and the other for people mentioning health or nutrition.

In this example, there is only one level of nets, but nets can be nested in other nets.  You can think of it like a document in outline form, where the nets are the headers of the various sections.

Nets cannot be used to code responses.  They are not themselves answers or responses to questions and instead are used to organize the answers (codes).

Downstream Data Processing

Once the questions in a study are coded they are ready to be used by the downstream data processing department in the survey research company.  This department may be called data processing, tabulation, or simply tab.  In tab, the results of the survey are prepared for review by the market researcher and then to the end client.

The tab department uses software tools to analyze and organize the results of the study.  These tools include statistical analysis which can be very sophisticated.  Normally, this software is not interested in the text of the code.  For example, if a response is coded “like apple variety” the tab software is not interested in that text but wants a number like 002.  From the tab software point of view, the respondent said 002, not “like apple variety”.  The text “like apple variety” is used by the tab software only when it is printing a report for a human to read.  At that time, it will replace 002 with “like apple variety” to make it human-readable.  Before the data are sent to the tab department each code must be given a number.  The codebook then looks like this:

• 001   VARIETY OF FLAVORS
  • 002   like apple variety
  • 003   like peach variety
  • 004   like cherry variety
  • 021   like peach tea variety (unspecified)
  • 022   like peach iced tea variety
  • 023   like raspberry tea variety
  • 024   like lemon iced tea variety
  • 025   other variety of flavors comments
• 026   HEALTH/ NUTRITION
  • 027   good for dieting/ weight management
  • 028   natural/ not contain artificial ingredients
  • 029   sugar free
  • 030   other health/ nutrition comments
• 031   MISCELLANEOUS
  • 032   other miscellaneous comments
• 998   NOTHING
• 999   DON’T KNOW

The tab department may impose some rules on how codes are numbered.  In this example the code 999 always means “don’t know”.

Choose Ascribe For Your Verbatim Coding Software Needs

When it comes to verbatim coding for open-ended questions in market research surveys, Ascribe offers unparalleled solutions to meet your needs. Our sophisticated coding platform, Ascribe Coder, is designed to streamline the process of categorizing and analyzing open-ended responses, transforming qualitative data into quantitative results. Whether you are dealing with responses from customer satisfaction surveys, employee feedback, or product evaluations, Ascribe provides the tools necessary for efficient and accurate verbatim coding.

If you are short on time or need further assistance with your verbatim coding projects, Ascribe Services can complete the coding project for you. They also offer additional services like Verbatim Quaity review, AI Coding with human review, and translation with human review.  Many of the top market research firms and corporations trust Ascribe for their verbatim coding needs. Contact us to learn more about coding with Coder. 

You hear the term text analytics a lot these days.  It seems to be used in so many different ways it is hard to pin down just what it means.  I’ll try to clear it up a bit.

The basic idea is that a computer looks at some text to accomplish some goal or provide some service.  Just what those goals and services might be we will get to in a moment.  First note that text analytics works with text and analyzes it in some way.  That’s the analytics part.  When I say text, I mean electronic text, like this blog page or some comments from your customers stored in a spreadsheet.  Text analytics does not work with voice recordings, videos, or pictures.

Because text analytics examines text written in a natural language (like English or French), it uses techniques from the sciences of Natural Language Processing (NLP) and Linguistic Analysis.  The computer techniques used include machine learning, word databases (like a dictionary in a database), taxonomies, part of speech tagging, sentence parsing, and so on.  You don’t need to understand these techniques to understand what text analytics does, or how to use it.

Just what can we use text analytics for?  Some of the more common uses are:

• Document classification and indexing – Given a bunch of documents, the computer can figure out the key concepts and let you search a library of documents by these. A sophisticated example of this is E-Discovery used in legal practice, which seeks to use the computer to assist in discovery in legal proceedings.
• National security – Governments use computers to monitor web postings for information or discussions of interest to national security.
• Ad serving – We are all experienced with the uncanny ability of some web sites to show us ads that are relevant to our needs and interests. Text analytics applied to the pages we are viewing is a big part of this magic.
• Business intelligence – For most of us this is the big one! We can the computer to give us insights into how to improve our service, retain customers, and give us a competitive advantage.

Text analytics for business intelligence is a rapidly growing market.  Using the right tool, you can analyze thousands of customer comments in minutes.  If the tool does a good job of presenting the results it is amazing how quickly you can figure out what they care about, their pain points and plaudits, your weaknesses and strengths.

Choosing a Text Analytics Tool

How do you find the right software product for your needs?  Well, there are many providers of raw linguistic analysis capabilities.  Google, Microsoft, Amazon, SAP, IBM, and many others provide such services via an API.  But this takes an engineering effort on your part, and you still need to figure out how to navigate the results of the analysis.

There are several vendors of complete text analytics packages for customer experience management.  As you evaluate these consider:

• Does the vendor specialize in customer experience feedback?
• Are the results of the analysis clear and insightful?
• Is text analytics a core competency or a side product?
• Are multiple languages supported?
• Is automatic translation between languages supported?
• How easy is it to tailor the text analytics to your specific business?

Navigating the Linguistic Analysis Results

Suppose you have a database of feedback from your customers.  If you run 10,000 customer comments through a linguistic analysis engine it will produce a mountain of data.  To gain insight from this data it needs to be organized and navigable.  A good text analytics tool will organize the results and display them in a manner that helps you to find the actionable insights in the comments.  Optimally the tool will help you pinpoint the strengths and weaknesses of your company from the customer's viewpoint.

Reducing the Raw Linguistic Analysis Data

Let's look at a specific example to gain an understanding of what is involved in organizing and presenting the results of linguistic analysis.  As I described in Linguistic Analysis Explained, sentence parsing is one tool in the NLP analytics bag of tricks.  Google has a very well regarded sentence parser, and it is available via an API.  You can try it out at https://cloud.google.com/natural-language/.  It's a fun way to get some insight into the complexities of presenting the results of linguistic analysis.  Try running this comment through the API:

The support staff are helpful, but slow to respond.

Now and take a look at the Syntax results.  These are the results of the sentence parser.  You find:

Wow, that's a lot of data, but hardly actionable insight!  You can see that the sentence has been broken into tokens (words and punctuation).  Each token has been tagged with its part of speech (slow is an adjective).  Each token has also been assigned a parse label, indicating how it is used in the sentence (slow is used as a conjunction).  The green arrows show how the token are interrelated.  Imagine how much data would be generated by running 10,000 customer comments through the sentence parser!

The job of text analytics is to distill this pile of data down.  In this case the analysis might go something like this:

• "Support staff" is the nominal subject of the root verb.  That's a topic the customer has mentioned.
• "Helpful" is an adjectival complement (acomp) of the subject.  The customer has said the staff are helpful.
• "Support staff" is further described by the customer by the coordinating conjunction (cc) "but", as "slow" (conj).

So we find that the support staff are both helpful and slow.  We have extracted a topic of "support staff", with expressions of "helpful" and "slow" associated with it.  This reduction of the raw data from linguistic analysis has resulted in what we are interested in knowing.  Our customer thinks the support staff is helpful, but also slow!  This is a single finding from linguistic analysis.

Presentation of the Reduced Data

Now that we have extracted the desired nuggets from the raw linguistic analysis data we need to present it in a way that helps you find the insights you seek.  An actual analysis of 10,000 customer comments may well produce 50,000 findings.  To navigate these the findings need to be organized in a way that emphasises the important insights, and allows you to explore the findings quickly and intuitively.  A good text analytics tools will assist you in ways such as:

• Grouping similar topics, preferably automatically or with a custom taxonomy you control.
• Ranking topics by frequency of occurrence or other metrics such as sentiment.
• Allowing you to filter the results in various ways, such as by demographic data.
• Showing trends across time or other variables.

Summary

Text analytics for customer experience management reduces and organizes the results of linguistic analysis.  If done well, the results are presented to you, the user, such that you can find actionable insights quickly, and explore the data to help formulate an action plan for improvement.

For social or market researchers who have years or decades of experience with quantitative data analysis, AI qualitative data analysis can feel intimidating. Researchers often specialize in quant or qual because the skill sets are quite different. However, you’ll soon see that qualitative data analysis has much in common with quantitative data analysis, especially if your experience is with advanced statistics like factor analysis or cluster analysis. Let’s find out why!

The qualitative side of factor and cluster analysis

When conducting a factor analysis, statistical software determines which answers tend to co-occur. For example, correlations will show that someone who indicates their job title is CEO or President probably also indicates they have a high income. Correlations will also show that someone who indicates they love bargains probably seeks out coupons and BOGOs. Across 100 questions, it’s impossible for a researcher to identify all the meaningful correlations among two, three, or ten variables.  

That’s why we love when statistical software steps in with factor and cluster analysis. This software makes it much faster and easier to identify significant correlations and categorize 100 variables and 500 possible answers into several solutions, each one with 4, 5, 6, 7, 8, or 15 buckets. None of these solutions are inherently truth but as you review each set of buckets, you’ll generate a subjective opinion that one solution feels more valid and reliable than others. That solution will have:

• Fewer variables and answers that don’t seem to logically belong to the bucket they’ve been assigned to.

• Fewer buckets that look like a random collection of answers.

• More buckets that feel like cohesive, meaningful, nameable constructs.

After reviewing the various solutions, you’ll choose just one to name and claim. Despite the extensive amount of quantitative statistical analysis taking place, you can see that the final process is very qualitative.

The quantitative side of AI qualitative coding

Thirty years ago, before automated and generative AI tools simplified and sped up the process in a meaningful way, people did all the work manually. Over days and weeks, researchers would read paper questionnaires and transcripts to identify potential trends in the words and phrases and then determine whether there were broader connections and patterns among them. As technology improved, it became much easier and faster to search for concepts and assign qualitative codes (e.g., gender, anger, pricing) and quantitative codes (e.g., 1=Female, 7.2=Anger.High, 32=Pricing). Today, with the assistance of AI qualitative coding, creating a coding structure with multi-level netting and extracting descriptive themes from open end comments has turned months, weeks, and days into mere minutes.

A researcher with lots of qualitative experience might prefer to review and analyze the qualitative codes manually to generate possible theories and choose the best one. Similarly, someone with lots of quantitative experience might see envision the quantitative codes serving as a dataset for running a factor analysis. Whether you see the process from a qualitative or quantitative point of view, the better method of identifying a theory is the one that feels right to you and results in valid and actionable results.  

Putting a human into the coding

Whether generated by AI or by human, having a set of codes and themes, named or not, does not mean the qualitative analysis is done. The next step is to review and understand the codes to identify a theory or result that is the most meaningful, strategic, and actionable.  

• Which solution connects in a logical way with an existing consumer, personality, social, or economic theory?  

• What aspects of the existing theory are currently missing from the analysis? Are those aspects still in the data or were they never a part of the data - why?  

• What aspects of potential solution are different from the existing theory? Is the theory wrong? Is the solution wrong? What are the known or unknown caveats?

• Do the results warrant development of a new theory that will require additional research?

AI and statistical software don’t have the personal, social, emotional, psychological, and cultural understanding of the human experience - yet. What makes sense statistically will still need some tweaking to flush out the full depth of the human experience. That’s what expert market and social researchers bring to the table.

Summary

AI is no longer the wave of the future. It is today and it is in everything we do. From pattern recognition, theory development, logical reasoning, consumer behavior, and more, your skills as a quantitative researcher transfer in meaningful ways to the qualitative world. Consider AI qualitative coding as a slightly different way to conduct a factor or cluster analysis, a technique you already have fun with!

If you’d like to learn more about our AI tools, check out our AI qualitative coding tool, ascribe, and read Voxco’s answers to ESOMAR’S 20 Questions to Help Buyers of AI-Based Services. When you’re ready to speed up your qualitative data analysis with a one-stop tool, please get in touch with one of our survey experts.

‍

We may not like to admit it but making mistakes is an expected part of the scientific process. As market and social researchers, we regularly discover errors, learn from them, and improve our processes because of them. You might even say that errors are essential for innovating and generating better insights. From the initial stages of data collection to later stages of results presentation, embracing mistakes is a pathway to better research.

‍

Champion Methodological Mistakes

Plenty of errors are likely to occur during the research design and data collection phase. They reveal unexpected gaps that can be anticipated and prevented in future projects with better training, tools, and processes. How can we embrace these errors?

• Expect flawed data collection tools. Whether you’re working with discussion guides, questionnaires, diaries, or bulletin board outlines, errors such as leading questions, unclear scales, and missing response options are bound to appear. Plan for these problems by including time in the schedule for colleagues to review your tools, perhaps adding incentives for creative solutions. Further, be sure to pilot test the final tool with a few participants prior to the full launch.
• Train for neutrality. As hard as we try to avoid it, questionnaire authors, interviewers, and moderators have feelings and opinions that show up as biased questions, micro-expressions/body language, and tone of voice. This unintentional lack of neutrality can decrease participation rates and distort results. To address this problem, attend refresher training courses on a regular basis. You’ll not only be reminded of common mistakes you may have forgotten about but you’ll also learn new techniques that have gained prominence since your last training.
• Plan for sampling gaps. Every researcher knows how easy it is to recruit a large group of research participants. At the same time, every researcher also knows how difficult it is to recruit participants who are representative of the target population. When budget and field dates are pressuring you, avoid the temptation to overlook representativeness. Build extra time into the schedule and actively oversample difficult target audiences from the beginning to avoid the biases that will result from settling on non-representative samples.

‍

Embrace Analytical Errors

Once data has been collected, the potential for errors doesn’t stop. By embracing errors during the analytical phase, we can achieve more thorough and nuanced data analysis.

• Seek outliers. Sometimes, outliers are discovered to be mistakes like miscodes or shifted columns. These are easily corrected. However, other outliers are analytical puzzles that need to be deciphered. It’s easy to automatically dismiss statistical anomalies but first consider whether they signal an unexpected insight. Mark the anomaly so that once you have a better understanding of the entire research problem, you can go back to it and determine if it was more meaningful than first realized.
• Contradict yourself. As you’re reviewing the analysis and building theories, actively contradict your own ideas. Try to prove the alternative hypothesis. Ask yourself what is missing. Assume errors have been made. Consider if you’ve overgeneralized beyond what the data is telling you. This will help you to avoid the confirmation bias that can arise out of early discoveries.
• Encourage dissent. Throughout your analysis, invite other team members to independently interpret your data. Incentivize them to contradict your interpretations and conclusions with plausible alternatives. And be prepared to put your ideas aside when other ideas are more likely.
• Leverage technology. Rushing through every potentially important theoretical model or statistical analysis is a fast-track to making errors. Use AI tools to dramatically improve your productivity and accuracy. Read our case studies to learn how C+R Research and Frost & Sullivan use Ascribe and Voxco Online to build complex questionnaires and code qualitative data faster, more accurately, and at a lower cost.

‍

Capitalize on Communication Glitches

In many cases, stakeholders review research reports without guidance from the researcher. Consequently, it’s essential that reports are clear, engaging, and powerful.

• Test reports with stakeholders. After spending days and weeks writing a report, your headlines and summaries will seem clear and concise – to you. Schedule sufficient time into your plan so that non-researchers can review it for clarity. Invite them to interpret and mis-interpret charts and tables, and point out conclusions that don’t make sense. Incentives are a great idea here too as many people don’t feel comfortable sharing criticisms of their colleagues’ work.
• Use plain language. As much as you love talking about various statistical tests and study designs with your colleagues, research jargon is not plain language and will not help readers who are further down the chain. Yes, share the jargon as learning opportunities but also incorporate sufficient descriptions so that people who don’t know your jargon will still understand what you’re talking about.
• Highlight limitations. Limitations are not flaws or mistakes. They are acknowledgements that research can never uncover every possible insight regarding every possible scenario. Since no one knows the limitations of the research better than you do, share details about any study constraints during verbal presentations and in the report. Preventing misunderstandings is a key part of your role and your clients will appreciate the guardrails.

‍

Convert Errors into Enhancements

Even the best researchers can’t avoid mistakes. What they do, however, is implement processes to anticipate, detect, and learn from those mistakes. Leverage AI and automation to reduce errors arising out of tedious activities. Collaborate with colleagues and other teams to invite dissenting opinions and alternative insights. Formalize debriefing sessions to identify problems and implement processes to prevent them in future projects.

Remember, mistakes aren’t failures. They’re feedback. By normalizing the presence of errors and fostering a culture of feedback, researchers can improve research validity and methodologies, build trust with stakeholders, and produce more impactful results. Embrace imperfection and the continuous improvement that goes along with it!

If you’d like to work with an experienced team of researchers who also value continual improvement, please get in touch with one of our research experts.

‍

In the age of AI, analyzing open-ended survey responses is more efficient than ever. While general AI models like ChatGPT can process text, they lack the precision, structure, and market research (MR) expertise needed for accurate coding and categorization. Ascribe Coder stands out as the superior choice for researchers, offering a purpose-built solution designed specifically for open-end analysis. Here’s why: 

‍

Purpose-Built for Market Research & Survey Analysis 

Unlike ChatGPT, which is a general-purpose language model, Ascribe Coder is specifically designed to process, categorize, and analyze open-ended responses. It employs a proprietary method for extracting themes, refined over decades of market research experience. This specialization ensures that responses are coded meaningfully, aligning with industry expectations. 

‍

Consistent, Reliable, and Transparent Coding 

One of the major drawbacks of ChatGPT is the potential for inconsistent responses due to its generative nature. Ascribe Coder, on the other hand, provides: 

• Structured and reproducible coding, with clear parameters and guardrails to keep results focused on the topic. 

• Transparency, in the process ensures applied codes stay linked to the original context and meaning within responses. 

• Partially coded response reporting, making it easy to review and refine results. 

‍

Customizable Codebooks 

Ascribe Coder gives users complete control over the codebook or code frame. Researchers can: 

• Automatically create descriptive, thematic codebooks tailored to their projects and refine as needed. 

• Create code frames with hierarchical and multi-level netting for detailed categorization. 

• Save and reuse codebooks for ongoing or similar projects 

This level of customization is not possible with ChatGPT, which lacks structured taxonomy management. 

Scalability & Efficiency with AI Flexibility 

Market researchers often handle large volumes of text data. Sometimes they prefer to use AI assistance for coding, and sometimes they prefer, or are required, not to use AI. Ascribe Coder is built for scalability and flexibility. It: 

• Handles large volumes of text with fully automated, fully manual, or hybrid processing to fit user or project needs. 

• Offers both generative and non-generative processing modes to suit different analysis needs 

In contrast, ChatGPT processes text one query at a time, making it inefficient for large-scale analysis. 

Seamless Integration with Market Research Workflows 

Ascribe Coder fits seamlessly into research ecosystems, offering: 

• Direct integration with survey platforms for smoother data processing. 

• Access to Ascribe Coder’s project management system for improved workflow visibility and efficiency. 

• Access to MR deliverables when needed for further analysis and reporting. 

• Built-in visualization tools, delivering instant insights. 

• ASK Ascribe, a tool allowing users to ask questions of the analysis and immediately receive insights, summaries and reports. 

Human Oversight of Coding for AI Accuracy & Control 

AI should enhance, not replace, human expertise. Ascribe Coder embraces human oversight, enabling researchers to: 

• Supervise and customize AI-generated results and make adjustments as needed. 

• Use integrated power tools to oversee, refine, and ensure quality control of AI coding. 

ChatGPT, in contrast, operates autonomously, making it harder to systematically validate results. 

Language Agnostic 

Global research often requires analysis of multilingual responses. Ascribe Coder is language agnostic, meaning: 

• It can create codebooks and present results in any language, regardless of the input language. 

ChatGPT struggles with structured multilingual coding and lacks built-in tools for cross-language consistency. 

Exceptional Customer Training and Support 

Often times researchers need help with a complex project, training a new team member, or exporting results in a new format. Ascribe’s Customer Training and Support teams have worked for 20+ years in the business and are there to help when needed, ensuring customers receive the results they need from their data to deliver their business objectives. This makes Ascribe a true research companion, far beyond what ChatGPT can offer. 

‍

Conclusion 

While ChatGPT is a powerful AI model, it was never designed for market research and survey coding. Ascribe Coder delivers structured, transparent, scalable, and customizable text analysis tailored to researchers’ needs. Its proprietary AI, seamless integrations and interactive tools all with human oversight, make it the superior choice for analyzing open-ended responses. 

For researchers who demand accuracy, efficiency, and control, Ascribe Coder is the clear winner.

Editor’s note: This post was originally published on Ascribe in October 2020 and has been updated to reflect the latest data.

Sentiment analysis (or opinion mining) is used to understand the emotion or sentiment behind comments and text, allowing data analysts to gain actionable insights from verbatim comments. While measuring and understanding sentiment analysis scores is more involved than analyzing closed questions, it offers a valuable source of metric data.

What Does Sentiment Mean?

Audiences will have opinions on products, services, and more, that are either positive, negative, or neutral in tone. Companies can use this information to better understand the feedback given by audiences on products or how effective or ineffective messaging has been. Sentiment analysis provides your business with a way to quantify these emotions to discover the overall answer polarity and insights into your customer feedback.

Because customer sentiment is provided in the person’s voice, and not based on a set response or keywords, you need a way for your computers to understand it. Natural Language Processing (NLP), combined with machine learning, allows your sentiment analysis solution to look at a data set and pull more meaning from it. It does this by scoring each response based on whether the algorithm thinks that it’s positive, negative, or neutral.

While applying a sentiment score to the entire response can be useful, it does have problems.  For example, would you say that this comment is positive or negative?

The food was great but the service was awful.

More sophisticated sentiment analysis can apply sentiment scores to sections of a response:

Food-great: positive
Service-awful: negative

Analyzing versus Interpreting

While analysis and interpretation are often used interchangeably, they have two different meanings, especially within data science and sentiment analysis work. Interpreting sentiment in a series of responses is more of a qualitative assessment. If you are manually processing verbatim comments to determine the sentiment, your overall sentiment results could contain unique biases and possible errors. With sentiment analysis tools, this bias potential and possible interpretation errors are severely diminished in favor of a faster, automated, analysis program.

Sentiment analysis programs have a standardized approach that gets the same results regardless of the person running the process. It’s difficult to achieve this manually, but computer-aided methods make it possible.

Positive and Negative Keywords

Positive Words

These text analysis words are representative of positive sentiment. However, despite lists like this existing, these words are subject to change and machine learning models are incredibly sensitive to context. This makes the framing of the comment incredibly important. With these machine learning models, however, companies are able to find out what people like about products, and services while highlighting their experiences. This is a good way to see what you’re doing right and areas where you compare favorably to the competition. You can build on these successes as you move forward as a company.

Here are some of these words:

• Acclaim
• Brilliant
• Convenient
• Durable
• Enjoyable
• Ethical

Negative Words

These words are commonly associated with negative sentiment. These sentiments can indicate areas where you’re failing to deliver on expectations. It’s also a good way to see whether a product or service has a widespread problem during a rollout, to identify issues in the customer experience, and to find other areas of improvement that you can prioritize.

Here are some of these words:

• Dishonest
• Failure
• Gruesome
• Hazardous
• Imbalance
• Lackadaisical

Neutral Sentiment Words

Neutral sentiments are driven by context, so it’s important to look at the whole comment. Excelling in the customer experience means going beyond “okay” and moving in a positive direction. These middle-of-the-road sentiments are useful in determining whether your company is noteworthy in a product or service category.

Positive to Negative Comment Ratio

A ratio in sentiment analysis is a score that looks at how negative sentiment comments and positive sentiment comments are represented. Generally, this is represented on a scale of -1 to 1, with the low end of the scale indicating negative responses and the high end of the scale indicating positive responses. You may need to adjust how you evaluate the score to account for trends in your audience as some may be more negative than the standard population. For example, if you were conducting a survey that focused on dissatisfied customers, then you would be dealing with a tone that’s more negative than usual.

What is a Good Sentiment Score?

A good sentiment score depends on the scoring model that you’re using. Set minimum scores for your positive and negative threshold so you have a scoring system that works best for your use case.

How Accurate is Sentiment Analysis?

The accuracy of sentiment analysis depends on the method that you’re using to work with your verbatim comments, the quality of the data that you’ve given the computer, and the subjectivity of the sentiment. You want the most accurate results possible, which typically means that you’ll want to have a computer assisting your researchers with this process. The automated system can reduce the potential for bias and also use a standardized set of rules for going through the data. If there are any problems with accuracy, you can feed more data into the sentiment analysis solution to help it learn what you’re looking for.

What Algorithm is Best for Sentiment Analysis?

The algorithm that works best for sentiment analysis depends on your resources and your business needs. There are three major categories in algorithms: machine learning, lexicon-based, and those that combine both machine learning algorithms and lexicons.

Machine learning is one of the most popular Algorithms in both Data Science and Text Analytics and is an application of artificial intelligence. It allows your sentiment analysis solution to keep up with changing language in real-time. Because data scientists can’t predict when the next shift in colloquial slang and voice will occur and completely change what is negative and what is positive. They’ve begun to use machine learning with operational data provided to understand natural language and current vernacular. This is a core component of sentiment analysis and is an example of supervised learning, where you’re feeding it representative results so it can learn from them. Unsupervised learning refers to machine learning that is not based on data specifically designed to train it. Deep learning refers to the complexity of machine learning, with this moniker usually referring to complex neural networks.

A lexicon-based algorithm relies on a list of words and phrases and whether they’re positive or negative. It’s difficult to update the lexicon with the latest trends in language.

Ascribe’s dedication to Sentiment Analysis

If you are looking to leverage sentiment analysis when analyzing verbatim comments and open-ended text responses to uncover insights and empower decision-making, check out Ascribe’s text analytics offering, CX Inspector.

CX Inspector is a customizable and interactive text analytics tool with compatible APIs and unique machine learning techniques that provide topic and sentiment analysis from verbatim comments automatically. Analyze everything from the success of marketing campaigns, product feedback results, product reviews, social media platform comments, and more.

For a more comprehensive solution for sentiment analysis, use X-Score which is a feature within CX Inspector that provides a sentiment score from open-ended comments. X-Score is a great measure of customer satisfaction, and also identifies the largest drivers of positive and negative sentiment.

‍