How Effective are Self-Explanations from Large Language Models like ChatGPT in Sentiment Analysis? A Deep Dive into Performance, Cost, and Interpretability

Language models like GPT-3 are designed to be neutral and generate text based on the patterns they’ve learned in the data. They don’t have inherent sentiments or emotions. If the data used for training contains biases, these biases can be reflected in the model’s outputs. However, their output can be interpreted as positive, negative, or neutral based on the context and input they receive. The context of the text is crucial when determining sentiment. A sentence might be negative when considered in isolation but positive when taken in the broader context of the text. Large language models consider surrounding text, but understanding the context can be challenging.

Sentiment analysis can be difficult for text with ambiguity, sarcasm, or mixed sentiments. Large language models may not correctly interpret such nuances. Misclassification or misuse of sentiment analysis can have real-world consequences. It’s important to consider these implications and use AI responsibly. Researchers at UC Santa Cruz analyzed the sentimental behavior of various models like ChatGPT and GPT-4. They assess the LLM’s capability to self-generate feature attributions. 

In the assessment, they studied two ways of generation. They compared generating the explanation before the prediction and generating the prediction and then explaining it. In both methods, they ask the model to develop a full list of feature attribution explanations containing the importance score of every word and ask the model to return the top-k most important words. They compare them with interpretability methods, occlusion, and Local Interpretable Model-agnostic Explanations. These two techniques are used in machine learning and deep learning to interpret and explain the predictions of complex models.

These models are also needed to be evaluated based on the input features. One must evaluate the model’s response to infinitesimal perturbation of the input feature value with representative methods such as gradient saliency, smooth gradient, and integrated gradient. The researchers followed a new method called occlusion saliency, where they evaluated the model’s response to various inputs with various features removed. To capture the non-linear interactions, they removed multiple features simultaneously, defined features’ importance as linear regression coefficients, and evaluated them.

According to the faithfulness evaluations, their results show that none of the self-generated explanations hold a distinct advantage over the rest. They are highly different according to the agreement evaluations. As a result, some explanations could be much better than the current ones, and novel techniques may be needed to reveal them.

This chain-of-thought generation can be considered as the model’s explanation. It is often helpful for the accuracy of the final answer, especially on complex reasoning tasks such as solving math problems. So, the team’s future work involves evaluating LLMs such as GPT-4, Bard, and Claude. They would run a comparative study to understand how these models understand themselves. They would also like to conduct studies on counterfactual explanations and concept-based explanations. 

Check out the Paper. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 32k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..

We are also on Telegram and WhatsApp.