Generative Ai chatbots have become increasingly popular over the last year with the launch of
OpenAI’s Chatgpt. However, the current trend for operationalizing foundational large language
models (LLMs) that are used in chatbots and fine-tuning them to specific domains have revealed
the fake responses or hallucinations specifically for mathematical operations. Thus, designing
chatbot-like products for finance-based use cases without controlling for hallucinations can have
the most detrimental impact on its decision-making users. In this talk we review the three major
stages in designing and delivering hallucination minimized chatbot solutions for use cases where
the input data is in the form of large tables. In the first prototyping stage, we control the language
data generation process and build a scoring engine for the chatbot responses to minimize
hallucinations. In the second scaling stage, we focus on the scalability, robustness, cost-per-
query, and trustworthiness of the overall LLM-based system architecture with the growing data
needs. The third evolution stage involves reinforcement learning with human feedback to learn to
execute complex financial instructions over time. A LLM system that is designed with these
stages under consideration will stay sustainable to LLM modifications over time and evolve in
functionality for domain specific use cases.
Dr. Sohini Roychowdhury is the Head of AI/ML and Generative AI Products at Accenture, USA.Prior to this, she formed the Founding team and served as Director of Curriculum and MachineLearning at an Ed-Tech Startup called FourthBrain that provides specialized hands-on courses inthe field of Machine Learning and AI. FourthBrain is incubated and funded by Dr. Andrew Ng’sAI Venture fund. Prior to her e...