Generative AI has led to a profound and positive change inside City towards data-driven decision-making, however the country's three largest banks have decided against an external chatbot for now since the risks are still too high.
These comments got here from Promiti Dutta, head of analytics technology and innovation at Citi, during a chat she gave during VB's AI Impact Tour in New York on Friday.
“When I joined Citi 4 and a half years ago, data science or analytics was often an afterthought before I even talked about AI. We used to think, 'We're using analytics to prove a degree that the corporate already had in mind,'” she said during a conversation I moderated. “The emergence of genetic AI was a serious paradigm shift for us,” she said. “It has actually brought data and analytics to the forefront. Suddenly everyone wanted to unravel every thing with Gen AI. “
Citi’s “three buckets” of generative AI applications
She said this created a fun environment where employees across the corporate began suggesting AI projects. The bank's technology leaders were clear that not every thing needed to be solved with genetic AI, “but we didn't say no, we actually made it occur. We could at the least start having conversations about what data could do for them,” said Dutta. She welcomed the start of cultural curiosity around data. (See her full comments within the video below.)
The bank began sorting generative AI project priorities in accordance with “meaningful outcomes that may increase time value and have security related to them.”
Desirable projects may be divided into three primary categories. The first was “Agent Assist”, where large language models (LLMs) can provide call center agents with summarized notes about what Citi knows in regards to the customers, or more easily jot down the notes throughout the conversation and find information for the agent, in order that they’ve this information to have the opportunity to reply more easily to the shopper's needs. It's not about customer contact, but about providing information to the shopper, she said.
Second, LLMs could automate manual tasks, resembling reading through large compliance documents on topics resembling risk and control, by consolidating text and helping employees find the documents they’re on the lookout for.
Finally, Citi internally developed an internal search engine that centralizes data in a single location to make it easier for analysts and other Citi employees to generate data-driven insights. The bank is now integrating generative AI into the product so employees can use natural language to create evaluation on the fly, she said. The tool shall be available to 1000’s of employees later this yr, she said.
Outward-facing LLMs are still too dangerous
However, on the subject of using generative AI externally – for instance, to interact with customers via a support chatbot – the bank has decided it remains to be too dangerous for prime time, she said.
There has been loads of buzz over the past yr about how LLMs hallucinate, an inherent property of generative AI that may be useful in certain use cases, resembling when writers are on the lookout for creativity, but may be problematic when precision is the goal: “It can things can go flawed in a short time and there remains to be quite a bit to learn,” said Dutta.
“In an industry where each customer interaction truly counts and every thing we do is geared toward constructing trust with customers, we cannot afford for anything to go flawed with any interaction,” she said.
She said in some industries, LLMs are acceptable for external communications with customers, resembling in a shopping experience where an LLM might suggest the flawed pair of shoes. A customer probably won't be too upset about it, she said. “But once we inform you to get a loan product that you simply don't necessarily want or need, you lose interest in us slightly bit because you’re thinking that, 'Oh, my bank really doesn't understand who I’m.'
The bank is using elements of conversational AI that became standard before generative AI got here to market in late 2022, including pre-written natural language processing (NLP) responses, she said.
Citi is within the means of learning how much LLMs can do
She said the bank had not ruled out external use of LLMs in the longer term but would must “work towards it”. The bank needs to make sure there may be all the time a human within the loop in order that the bank can learn what the technology cannot do and “move on from there because the technology matures.” She identified that banks are also heavily regulated and must undergo many tests and audits before they’ll use latest technologies.
But the approach contrasts with Wells Fargo, a bank that uses generative AI in its virtual assistant Fargo, which supplies customers answers to on a regular basis banking questions via voice or text on their smartphones. “The bank says Fargo is heading in the right direction to achieve a run rate of 100 million interactions per yr,” said the bank’s CIO, Chintan Mehta, during one other talk I moderated in January. Fargo uses multiple LLMs in its process to perform different tasks, he said. Wells Fargo also integrates LLMs into its Livesync product that provides customers advice on goal setting and planning.
Another way generative AI is transforming the bank is by forcing it to rethink where to deploy cloud resources as an alternative of staying on-premises. The bank is considering leveraging OpenAI's GPT models through Azure's cloud services for this, although the bank has largely eschewed cloud tools up to now, preferring to maintain its infrastructure on-premises, Dutta said. The bank can be exploring open source models resembling Llama and others that can allow the bank to deploy models internally to be used on its on-premise GPUs, she said.
LLMs drive internal transformation at Citi
An internal task force on the bank reviews all generative AI projects in a process that goes all the way in which as much as Jane Fraser, the bank's chief executive, Dutta said. Fraser and the leadership team are hands-on because making these projects a reality requires financial and other resource investments. The task force ensures that each project is executed responsibly and that customers are secure each time they use generative AI, Dutta said. The task force asks questions like: “What does this mean for our risk management model, what does it mean for our data security, what does it mean for a way others access our data?”
Dutta said that generative AI has created a singular environment where there may be enthusiasm from the highest and bottom of the bank, to the purpose where there are too many hands within the pot and there could also be a have to curb the keenness.
Responding to Dutta's speech, Sarah Bird, global head of responsible AI engineering at Microsoft, said Citi's thorough approach to generative AI reflects best practice.
Microsoft is working to repair LLM errors
She said loads of work goes into resolving cases where LLMs can still make mistakes even after being updated with a real source. For example, many applications are built using Retrieval Augmented Generation (RAG), where the LLMs can query a knowledge store to get the precise information to reply questions in real time, but this process remains to be not perfect.
“It can add additional information that shouldn’t be there,” Bird said, acknowledging that this just isn’t acceptable in lots of applications.
“Microsoft has been on the lookout for ways to eliminate these kinds of ground faults,” Bird said in a chat that followed Dutta’s and that I also moderated. “This is an area where we've actually seen loads of progress, and there's still loads of work to be done, but there are a lot of techniques that may significantly improve effectiveness.” She said Microsoft is spending loads of time to check this and find other ways to detect ground faults. Microsoft is seeing “really rapid progress by way of what's possible, and I believe we will see quite a bit more in the subsequent yr.”
