Many platforms are now integrating AI, including some of our competitors, however, Engagement Hub remains resolute that currently, the risk is too great to trust the analysis of your community and stakeholder feedback to AI.
14 reasons not to use AI for sentiment analysis of your community and stakeholder feedback
1. Words
Qualitative analysis is the analysis of words used by your stakeholders, often in surveys. Analysing this has always been difficult, but the difficulty of programming technology to do this has not replaced human skills. Below are the key issues with automated sentiment and thematic analysis.
- Word Interpretation – A major downside is that words are used in different ways, either because of different definitions or because of colloquial uses or trends within different sub-populations. For example, in Australia, the word “deadly” could mean “a danger to life” or it could mean “excellent, amazing, awesome”.
- How would AI interpret feedback on a proposed new park if the response is “deadly”? A human conducting the coding would mark this as undecided until more information on the full response was gathered.
- The Definition of the word/ the list it is in may be wrong.
- Level of Analysis – The size/quantity of the text being analysed can make a difference in interpreting the word definition, however, programming AI to analyse larger pieces of text doesn’t fix the problem as not all text is of equal importance and automatically coding every sentence that has a word in the lexicon risks coding unimportant sentences that could unnecessarily skew the results.
- Subjectivity and Tone also may still be difficult to determine as the unit of analysis is not static across all responses, and a sentence may be full of compound ideas, colloquialisms, or punctuation problems.
- The Unit of Analysis may be too small to accurately assign a sentiment score or theme.
- There may be multiple themes within the same unit of analysis.
- The Unit of Analysis will differ throughout the same and different responses and it is difficult to program for this.
- Lack of punctuation causes the unit of analysis to be further complicated.
- Colloquial nature of text responses
- Context & Polarity – Imagine that the text being analysed for sentiment is the additional comment field from your survey. Without knowing how the person answered the question, there is no context to interpret their comment correctly, especially if they are using sarcasm. The risk of getting this wrong is high and thus making an incorrect decision based on incorrectly analysed feedback is high.
- How would AI code the following: “The colour is red! That is a controversial choice, one that is deadly!”. Knowing if they selected Liked or Disliked on the survey may help determine this and human coders can check this important contextual information.
- Irony & Sarcasm – It is difficult to interpret irony and sarcasm, even emoticons which were added to help share emotions, are often used ironically and people who are not “in the know” can miss the subtext. Different generations often use words and emoticons differently so it is possible that the same word needs to be on both negative and positive lists.
- For example, with a response of ‘Classy’. Does this mean that the idea is great or does it mean that the idea is tacky? Even adding additional contextual words – e.g. “The colour is red! Classy!” means it is still difficult to determine.
- Comparisons – A response may compare two items, but without additional information or reading their entire survey response, know if this is a positive or negative comparison. Here are some examples of comparison responses that are difficult to classify. “This proposal is better than what you usually propose”. “The proposal is better than what was done at the Baker Street park!” These are not necessarily positive responses. Additional context may assist but AI runs the risk of misclassifying the sentiment towards the proposal.
- Emojis – The double meaning or using a positive emoji to display a negative sentiment is very common and difficult to program for. The laughing-crying emoji was used by millennials to mean negative and is used by gGen Z to mean positive laughter. The thumbs-up used to mean good and is now used by younger generations to mean negative/ sarcasm.
- Defining Neutral – defining all sentiments is important including defining neutral, as otherwise you run the risk of adding unnecessary noise by coding something negative that doesn’t need to be counted. As AI assigns numbers to words and these are added up, including additional words/ numbers that are unnecessary increases the risk that the AI analysis is incorrect.
2. Time and Productivity Impediments
Far from offering a productivity improvement, AI in qualitative analysis is actually a substantial loss of productivity. All the work done by AI needs to be checked and often redone. This often takes significantly more time than a human doing it the first time. It takes more time and greater skill to double-check everything as there are numerous issues that need to be considered, including checking the quality of the analysis, checking for bias and checking for AI hallucinations. It often means that it is quicker for a human to do the work than to spend time checking and redoing the work. Additionally, the data needs to be cleaned before analysis can take place or greater errors will be in the AI analytical output. This additional step often isn’t taken into account. For example, lack of punctuation and typos can significantly impact automated qualitative analysis and there is a lot of time in cleaning the data before AI analysis can begin. Believing the results of AI without double checking has significant political and organisational risk, especially if it’s done by a Government agency or organisation working with the public.
3. Risk HR
AI will not replace hiring skilled staff or ongoing training of staff and will increase the training and skill level required. The cost of staff is more likely to increase from using AI than decrease. If you want the best from your AI you will need to have people to run it who understand the risks and can mitigate against them, at the very least, your stakeholder team will need to be able to double-check the sentiment and thematic analysis done by AI.
- Relying on AI to undertake analysis will require the work to be double-checked, and this takes longer or more staff than doing the work without AI.
- Double-checking work requires judgement and skill, which is a higher level of skill than may be required to just doing the coding.
- You will require new types of employees including Prompt Engineers, and specialists with the skill to develop the right questions to elicit the outcome you require from the AI. Developing prompts is a very iterative and time-consuming process.
- Other AIs require you to program or instruct them on what to do. This is called prompt and it is an iterative process that requires you to actively review the results before refining your prompts. It is often quicker to do this yourself using our inbuilt Category/ Tagging system on Engagement Hub – (Contact us for more information about this analysis tool included as part of your Engagement Hub licence.)
- One academic says it takes 14 times more carbon to ask ChatCGT to generate a knock-knock joke than to use Google to look one up.
- New Scientist has reported that 20 per cent of the energy required by AI will be produced by coal. “It is a sad coincidence that as humanity attempts to slash its carbon emissions, it is also rushing to develop a technology that could, in theory, consume an unlimited amount of energy!
- “If Google chose to shift to an entirely AI-powered search business, it would end up using 29.3 terawatt-hours per year, equivalent to the electricity consumption of Ireland and double the current energy consumption of 15.4 terawatt hours in 2020. There is also a limited supply of the powerful computer chips known as graphics processing units (GPUs) and would cost $100 billion to fund.”.
- “Social scoring: classifying people based on behaviour, socioeconomic status or personal characteristics” is considered an ‘Unacceptable Risk’. It could be argued that using AI to classify people using segmentation (sentiment or thematic codes) within an SRM system or for research, might be considered social scoring and thus potentially an ‘Unacceptable Risk’.
- “Access to and enjoyment of essential private services and public services and benefits” is considered High Risk. It could be argued that using AI to analyse the feedback from community consultations to inform the provision of public services and benefits, could be seen to be engaging in an ‘Unacceptable Risk.
- https://monkeylearn.com/sentiment-analysis/#context-and-polarity
- i https://www.zdnet.com/article/what-is-ai-heres-everything-you-need-to-know-about-artificial-intelligence/ 25/1/2024
- ii Sparks, M. “Should we worry about Ais energy use?” New Scientist, 14 October 2023, p11.
- iv https://www.servicedeskinstitute.com/five-ethical-issues-of-ai-in-the-modern-workplace/
- https://theconversation.com/ai-is-here-and-everywhere-3-ai-researchers-look-to-the-challenges-ahead-in-2024-218218
- https://theconversation.com/generative-ai-5-essential-reads-about-the-new-era-of-creativity-job-anxiety-misinformation-bias-and-plagiarism-203746
- https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence
4. Changes to your workforce
Using AI will require more staff to do the work, with a greater level of skill and at a commensurate higher cost and, as this is still early days, staff with these skills are rare. Your employment costs will increase with AI. For example, your organisation now needs to employ Prompt Engineers, AI Ethicists, AI risk managers and IT professionals who understand AI integration – hiring key staff will be costly and will require new skills in HR too. Similarly, your stakeholder engagement workforce will need to be fully trained in coding to be able to double-check the AI coding done to protect you from risks of poor analysis when it impacts the government or decisions impacting the community.
5. Sustainability/ Environmental issues
AI uses significantly more electricity than traditional IT and as such is a significant environmental concern and companies will need to include this in their environmental reporting. It’s been a dirty secret of IT companies, which the massive explosion of AI use in 2023 brought to the foreground. AI will have a negative impact on global warming and needs to be used appropriately. There are a lot of different numbers around about how much additional carbon is used by AI than by traditional IT, but the bottom line is that AI is a very heavy carbon user and if your organisation is monitoring this and concerned about it, it needs to be calculated.
6. Assumptions and Bias
Especially if the reason you use Engagement Software is for legislative reasons requiring you to consult with your community, you will need to be able to demonstrate that your methods and results are not biased and also to know that the conclusions you are drawing from the data are reflective of what the community said. Unfortunately, AI systems have been trained from an original bias, and even the creators of the AI systems cannot tell what biases are embedded. Newer AI is now using machine learning to expand on their original Large Language Model training and is being fed bias from social media comments. These are issues that are difficult to fix but do present a significant risk to organisations using AI for analysis. For example, if you ask for images of lawyers, scientists etc, it nearly always creates pictures of men. Take this to a community and if there are inbuilt biases about groups in your community, it will cease to be accessible.
7. Unreliable
“AI solutions rely heavily on training data, which, unfortunately, makes them vulnerable to deception. Currently, no AI technology can detect inaccuracies in data without the help of extensive example data that is free of mistruths, misinformation and disinformation. This means that the generated data could often be inaccurate or the data source unknown, which can be very problematic. AI-generated content may include falsehoods, inaccuracies, misleading information, and fabricated facts, known as “AI hallucinations.”
8. Consent
If the data is not de-identified, then having an AI use this data requires consent from the people who provided this feedback to you, This is particularly important because AI retains this data and it may be able to be compiled alongside other information on that same person and used by others. Consent would be required by your target population to indicate that you will be using AI to analyse and interpret their data, and this could have an adverse impact on the engagement as it may reduce engagement.
9. Legislative Risk
Any business that is using AI to categorise and assign sentiment or themes to individuals may be at legislative risk. The EU, often the leader in catch-up legislation, has now legislated around AI, and they have divided the rules into four risk groups: (1) Unacceptable risk; (2) High risk; (3) General purpose generative IA; (4) Limited risk. It could be argued that standard Community and Stakeholder Engagement work, are considered the two highest risk categories if they are undertaken by AI.
10. Relevance of dataset used to train AI to your stakeholder population
Despite its impressive capabilities, generative AI is not without limitations. The model is not capable of understanding the world in the way that humans do and has limited knowledge in some areas. All AI systems have been trained with large datasets, but, each community has very specific characteristics that may not be reflected within the original training material. An important consideration is: Does the AI represent your population enough to undertake a thematic and sentiment analysis? Now that AI has reached into many areas of our lives, attempts are being made to train AI on specific large language models so that it can make recommendations based on accurate data. It is not possible to train AI for your own population as training it requires terabytes of data, and it is also very expensive. If your particular community differs from the population that was used to train AI, then the risk of making incorrect assessments if using AI is high.
11. Copyright
Has the AI based its output on copyrighted material? There are a number of court cases starting, that the plaintiff is arguing, illegally and without permission used copyright material to train their AI machines, as the training did not constitute fair use. “If you’ve recently used any Generative AI tools, you must have been impressed by the quality of the output, especially given the speed at which it’s generated. But have you ever wondered how these stunning designs, poems, AI videos, voices and summaries are created? Also, the work your staff use to prompt AI is also going into training AI and questions remain as to who owns that material
12. Political Risk
If you are using AI to analyse the results of a community consultation, and the conclusions do not support the data when it’s independently assessed, this has real-world implications for all levels of government. Making a decision based on feedback that has been analysed incorrectly by AI, puts all government at significant political risk. Contact us for our report on comparing the same qualitative responses analysed by a local government and by an online AI and the conclusions on the differences between them
13. Technological Debt/ Ethical Debt
One of the arguments for being gung-ho about AI is the historical acceptance of new technology has been low, with many detractors and that all worked out fine. Those arguing this bring up the printing press; sewing machines, railways, aeroplanes, automobiles, and the internet – and while it is true that there are always slow adopters, it does not mean that risks should be ignored. We offer one word about the risks to any organisation or government thinking of using AI without significant oversight – Robodebt. Fixing this has cost the Government millions of dollars, but the long-term damage to the trust in Government has significantly declined as a result and it is reviewing the way AI is being used throughout Government as a result. The risks of AI must be thoroughly considered.
14. Australian legislation
Australia is considered well behind other countries in legislation relating to AI, however, the Robodebt disaster and the new EU and US legislation have increased the interest in Australia and we are fast catching up. At a minimum, familiarise yourself with the state of play regarding legislation.
Conclusion
We currently do not recommend AI for analysis of Stakeholder engagement/ community engagement and social research at this stage as we view the risks of using this are much greater than the benefits. It’s important to properly calculate the additional costs in time, the massive carbon costs, productivity losses, the risks of incorrect “hallucinations”, of assumptions about your population, internal skill development and costs and the reputational risk from relying on AI analysis without having appropriate methods and structures in place. Engagement Hub’s built-in content tagging system allows our users to quickly and easily ‘tag’ feedback from consultations e.g. Parking – not enough, too much, more street parking or Positive, Negative, Neutral. This humanisation of the data facilitates the correct interpretation of feedback being generated into reports. This enables our clients to still turn qualitative data into quantitative data but with the certainty that the interpretation of the data is accurate. To find out more about sentiment analysis using Engagement Hub, contact us for a demo.
