Following a webinar presented in July 2021, Rowland’s Rob Lovegrove and Aginic’s Angel Chan and Danielle Simkus answer common questions on combining public sentiment with business data to manage brand, stakeholders and business performance.

Combining different data sets for insight

Q: We’ve heard about the benefits of combining different data sets, sometimes in unexpected ways. What’s the most unusual data set you’ve used in this space? 

A: Rowland has introduced a strategic campaign for the state government around the introduction of e-ticketing in public transport, and in that instance we were using human movement data. We were using data sets that were historically used in a town planning perspective, which leads to a good point that often a person’s skill set or profession dictates the lens in which they look at a dataset. This was a scenario where people were used to looking at the data with town planners to inform the build environment. I came in as a communications person looking at it from a perspective of wanting to understand where different demographics congregated and tied into other datasets from credit card companies, for example, and what their propensity to spend money on public transport was.

In this instance, the initial rollout was going to be on the Gold Coast, so we looked at a 1-2 kilometre buffer around the G-Link line to see what the human movement traffic looked like, where do demographics live at what stops and at what time of day are we likely to see most travellers – young females going to the beach, office workers travelling home from work.

This all contributed to a very precise plan on how, where and when to tailor campaign messaging and  advertising. Following this experience it has dawned on me that there will be many opportunities within organisations to share data with other colleagues who can look at it in a different perspective.

– Rob Lovegrove

 

Q: When dealing with so many different data sources, how do you bring them together?

A: Aginic is an analytics company. We love to transform the way users experience data, which means we often grapple with how to bring data together – both for the more technical people working “under the hood”, and the people using the data to make decisions.

We use a range of modern technologies and techniques to build data pipelines and combine data from both on-premise and cloud-based systems. These include SaaS products like Fivetran and Azure Data Factory, as well as open source products like Airflow and dbt. Wherever possible, we incorporate DevOps techniques and good engineering and modelling practices to improve robustness, performance, maintainability and observability.

On the front end, we use design thinking and visual design expertise to ensure that we take the end user needs and experiences into account from the start. We often build data solutions for end users who are not necessarily technical, such as teachers and nursing staff (or winemakers in the case of today’s demo), so it’s incredibly important for the front-end to be intuitive and tailored for users’ workflow and requirements. By combining analytics and design capabilities, we try to keep this front of mind.

– Angel Chan

 

Q: The Wine Australia work is both beautiful and has a very smooth user experience. What BI tool was used to put this together? Do you use a data warehouse behind the scenes or are you going directly to source systems? 

A: The dashboard itself was built in Power BI and was embedded into their website using Power BI embedded. We built the data model that was housed in Azure SQL Database. Data from the source systems were integrated previously to when we were engaged, but we pulled the different source data through Azure Data Factory pipelines into a transformation schema where we then implemented all the business logic transformations. This allowed us to move the complex queries to the SQL database as there is a significant amount of data underneath those dashboards and it made sense to leverage the compute power of databases rather than Power BI. Doing those transformations in the SQL layer allowed us to have a really fast and smooth experience on the surface for users. We also built it alongside designers at Aginic who are user experience experts and helped us understand what the stakeholders’ needs were.

– Danielle Simkus

 

Q: The examples today have focused on industry analysis. What approach do you use for organisations? 

A: Most of the work that Aginic does is with organisations, and in terms of how they analyse data, it’s often in relation to their own brands or products. For example, what we often do in a customer analytics context is to combine that organisation’s internal data and social data to understand the voice of the customer and potentially drivers behind purchase decisions or retention.

For example, we helped a theme park set up a survey tool to gather data from their parks and understand how people felt about the rides, food & beverage, and wait times, as well as how the public felt about them versus their competitor. We also had their customer sales and demographics data, so we were able to use those sources in combination to influence their strategy. This ranged from micro-strategies such as how to design, price and promote a particular event, from more macro-strategies such as investments in major capital assets (e.g. rides) or workforce management.

We also work with government agencies and other large organisations on data analysis projects, but the broad theme is similar, in that we try to identify insights from a combination of data points that paint a richer and more holistic picture.

– Angel Chan

Our work for organisations follows the same ethos – that is, understanding the environment in which their stakeholders live and engage with brands. A lot of our work with organisations is in helping them to continually evolve their proactive communication activity. One case of a council looks at how local people are perceiving issues of the day – for example the impacts of COVID;  what’s driving public sentiment at the moment? Is it personal health, community health, or is it jobs and economy concern? We help our clients refine their messaging to reflect public sentiment on a week-to-week or month-to-month basis.

– Rob Lovegrove

Sentiment analysis and market research

Q: We’ve used the words “next-generation”. How is your approach an evolution from current market research practices and traditional measures like NPS? 

A: Importantly we often suggest sentiment reporting doesn’t replace more traditional methods of market research. Each has their place to play but we find that sentiment provides cost effective, finger-on-the-pulse insights that other methods can’t match.

Focus groups and polling can be subject to participants ‘altering’ their views to fit a widely held belief, especially if they consider their views not to be ‘mainstream’. We’ve seen exit polling associated with political elections widely different to the outcome – this is because the interviewee knows that their answers are going to be used towards a specific purpose.

While many people know their comments on open digital channels are open to scrutiny, they’re unlikely to ever know their perceptions are being considered to help shape different brand experiences or narratives. 

And unlike net promoter scores, our insights go way beyond a single % score. With sentiment analysis, we’re putting context around behaviours and potential future actions.  

– Rob Lovegrove

 

Q: What baseline sentiment dictionary are you using to determine positivity / neutrality or is it custom made for this industry and gradually improved as time goes by?

A: Text and sentiment analysis is an area with a lot of R&D and innovation at the moment, and there’s a range of techniques out there for different use cases – which can be a whole webinar in its own right!

In the demo we saw, we used NetBase, which is a combined social listening and analysis tool that’s one of the global leaders in this space. NetBase pulls in data from a range of social and digital listening sources, from Twitter through to news and forum posts, and runs a proprietary Natural Language Processing engine over it. This NLP engine does a range of things, from text parsing through to entity, concept and sentiment analysis.

The engine offers several features that we look for as analysts. Firstly, it goes beyond just giving a sentiment score and identifies what the sentiment is about. Take an example of a review about a restaurant. “The burgers are great but the customer service was terrible. I won’t go back again.”

There’s positive and negative in that line, but it’s not exactly helpful for those to blur all of it into one sentiment, right? So to drive useful insight, you need the engine to understand your sentence structure – what the subjects are (“burgers”, “customer service”), what adjectives (“great”, “terrible”) or verb phrases (“won’t go back”) are used, and whether they’re good or bad. That’s why the NLP layer is important, and as you saw in the demo, NetBase went beyond just lumping everything into one score and let you see word clouds and other visualisations based on the sentiment drivers at a pretty granular level.

Secondly, NetBase is fairly transparent and offers some tuning options for non-technical users, which helps with adapting to different industry contexts and use cases. For example, you can configure which subjects you’re interested in knowing the sentiment for (e.g. you might care about customer service, but not burgers); you can manually change the sentiment for individual posts; and you can configure for certain words or phrases to be handled in particular ways to suit your context. A few years ago, I was running an analysis for an airline and “bag drop” was coming up as negative because in many cases, dropping things can be bad. In this case, however, “bag drop” could be reconfigured as neutral.

This isn’t the only approach to sentiment analysis, and it will never be a perfect science – after all, not even human beings understand each other all the time! But those features are helpful in letting the analyst know and improve what’s actually going on, and building trust with the audience of the analysis.

– Angel Chan

 

Q: How do Aginic and Rowland deal with uncontrolled/external data that can be “irrational” and try to deduce significant customer feedback? As opposed to feedback that doesn’t necessarily need to impact business decision making at the same level of significance (Expletives, post-truth commentary etc) without manually analysing all of this data?

A: Depending on the volume of commentary generated, manually analysing all commentary is unrealistic. At Rowland, when we undertake sentiment reporting we use the findings to guide a hypothesis. We basing our insights on trends in volume of comments and the subject matter. Whether it’s irrational commentary or otherwise, if it’s related to our client’s brand then it all has a part to play in helping us define different persona types. The fact that irrational behaviours exist may not represent the majority but they may impact on how conversations play out online and we have to help our clients prepare for these types of scenarios.

– Rob Lovegrove

 

Q: Is it possible to deliver any causal analysis, or link sentiment data to actual financial outcomes? 

A: As the saying goes, “Correlation doesn’t equal causation.” I would generally be cautious about attributing causation to sentiment. You will often find correlations between variables: for example, positive customer sentiment will often be correlated with other quantitative data points such as NPS scores, retention rates, and sales. Similarly, negative public or customer sentiment may be correlated to a decrease in share price or financial performance. But which is the cause and which is the effect, and what other variables are at play?

In other words, it may not be possible to identify specific causal factors in a scientifically rigorous way, at least not without a lot of thought into the design of the analysis and the extent to which you can control or isolate variables. My recommendation is to use sentiment to understand how people feel or think about certain things, and use that to design experiments or drive actions that may be more directly linkable to outcomes.

For example, you can use sentiment analysis to identify two possible messages for a marketing campaign, which can then be A/B tested for effectiveness based on metrics such as click-through rates, conversion rates, etc. – which may over time reduce customer acquisition costs or increase lifetime customer value compared to other customer cohorts. You can also use sentiment analysis from customer service interactions to identify high-value customers at higher risk of, design intervention processes or experiments, and compare retention rates to see whether this helps address churn. Again, sentiment data is much more useful when combined with other data sources and used in a targeted way.

– Angel Chan