Machine learning models could make mistakes and be difficult to make use of. Therefore, scientists have developed explanation methods to assist users understand when and the way they need to trust a model's predictions.
However, these explanations are sometimes complex and should contain details about tons of of model features. And they’re sometimes presented as multi-layered visualizations that may be difficult for users who lack machine learning skills to totally understand.
To help people understand AI explanations, MIT researchers used large language models (LLMs) to convert action-based explanations into easy language.
They developed a two-part system that converts a machine learning explanation right into a paragraph of human-readable text after which mechanically assesses the standard of the narrative so an end user knows whether or not they can trust it.
By prompting the system with some sample explanations, researchers can tailor its narrative descriptions to user preferences or the needs of specific applications.
In the long run, researchers hope to construct on this system by allowing users to ask a model follow-up questions on the way it made predictions in real-world environments.
“Our goal with this research was to take step one toward enabling users to have wealthy conversations with machine learning models in regards to the reasons they made certain predictions, so that they could make higher decisions about them “Whether they need to hearken to the model,” says Alexandra Zytek, a doctoral student in electrical engineering and computer science (EECS) and lead writer of a paper on the technique.
She is assisted on the paper by Sara Pido, an MIT postdoctoral fellow; Sarah Alnegheimish, an EECS doctoral student; Laure Berti-Équille, research director on the French National Research Institute for Sustainable Development; and senior writer Kalyan Veeramachaneni, a senior research scientist within the Information and Decision Systems Laboratory. The research will probably be presented on the IEEE Big Data Conference.
Insightful explanations
The researchers focused on a preferred kind of machine learning explanation called SHAP. In a SHAP statement, a price is assigned to every feature that the model uses to make a prediction. For example, if a model predicts house prices, one feature may be the placement of the home. The position is assigned a positive or negative value, indicating how much that feature modified the model's overall prediction.
SHAP statements are sometimes presented as bar charts showing which features are most or least vital. But for a model with greater than 100 features, this bar plot quickly becomes unwieldy.
“As researchers, we’ve got to make many selections about what we would like to present visually. If we only show the highest 10, people might wonder what happened to a different feature that isn't within the plot. Using natural language relieves us of the necessity to make these decisions,” says Veeramachaneni.
However, as a substitute of using a big language model to generate a natural language explanation, researchers use the LLM to convert an existing SHAP explanation right into a readable narrative.
By only handling the natural language portion of the method, the LLM limits the potential of introducing inaccuracies into the reason, Zytek explains.
Their system called EXPLINGO is split into two parts that work together.
The first component, called NARRATOR, uses an LLM to create narrative descriptions of SHAP explanations that match user preferences. By initially providing the NARRATOR with three to 5 written examples of narrative explanations, the LLM imitates this kind of text creation.
“Instead of creating the user attempt to define what kind of explanation they’re on the lookout for, it’s easier to simply allow them to write what they need to see,” says Zytek.
This allows NARRATOR to be easily adapted to latest use cases by presenting it with a distinct set of manually written examples.
After NARRATOR creates a plain-language explanation, the second component, GRADER, uses an LLM to judge the narrative based on 4 metrics: conciseness, accuracy, completeness, and fluency. GRADER mechanically sends the LLM the text of NARRATOR and the SHAP statement described therein.
“We find that even when an LLM makes a mistake in performing a task, they often don’t make a mistake in verifying or validating that task,” she says.
Users may customize GRADER to assign different weights to every metric.
“One could imagine that in a high-risk case, the weighting of accuracy and completeness is far higher than, for instance, language proficiency,” she adds.
Analyze narratives
For Zytek and her colleagues, certainly one of the most important challenges was adapting the LLM to generate natural-sounding narratives. The more guidelines they added to the control style, the more likely the LLM was to introduce errors into the reason.
“A whole lot of timely tuning was done to search out and fix each error individually,” she says.
To test their system, the researchers used nine machine learning datasets with explanations and had different users write narratives for every dataset. This allowed them to judge NARRATOR's ability to emulate unique styles. They used GRADER to attain each narrative explanation for all 4 metrics.
Ultimately, the researchers found that their system could generate high-quality narrative explanations and effectively mimic different writing styles.
Their results show that providing just a few manually written sample explanations significantly improves narrative style. However, these examples have to be written rigorously – the inclusion of comparison words corresponding to “larger” could cause GRADER to mark precise explanations as incorrect.
Building on these results, the researchers need to explore techniques that might help their system process comparison words higher. They also need to expand EXPLINGO by streamlining the reasons.
In the long run, they hope to make use of this work as a springboard for an interactive system through which the user can ask a model follow-up questions on a proof.
“That would assist in decision making in some ways. When people disagree with a model’s prediction, we would like them to give you the chance to quickly determine whether their intuition is correct or whether the model’s intuition is correct and where that difference comes from,” says Zytek.
