Home Community AWS Researchers Propose Panda: A Latest Machine Learning Framework to Provide Context Grounding to Pre-Trained LLMs

AWS Researchers Propose Panda: A Latest Machine Learning Framework to Provide Context Grounding to Pre-Trained LLMs

0
AWS Researchers Propose Panda: A Latest Machine Learning Framework to Provide Context Grounding to Pre-Trained LLMs

Debugging performance issues in databases is difficult, and there may be a necessity for a tool that may provide useful and in-context troubleshooting recommendations. Large Language Models (LLMs) like ChatGPT can answer many questions but often provide vague or generic recommendations for database performance queries.

While LLMs are trained on vast amounts of web data, their generic recommendations lack context and the multi-modal evaluation required for debugging. Retrieval Augmented Generation (RAG) is proposed to reinforce prompts with relevant information, but applying LLM-generated recommendations in real databases raises concerns about trust, impact, feedback, and risk. Thus, What are the essential constructing blocks needed for safely deploying LLMs in production for accurate, verifiable, actionable, and useful recommendations? is an open and ambiguous query.

Researchers from AWS AI Labs and Amazon Web Services have proposed Panda, which goals to offer context grounding to pre-trained LLMs for generating more useful and in-context troubleshooting recommendations for database performance debugging. Panda has several key components: grounding, verification, affordability, and feedback.

The Panda system comprises five components: Query Verification Agent filters queries for relevance, the Grounding Mechanism extracts global and native contexts, the Verification Mechanism ensures answer correctness, the Feedback Mechanism incorporates user feedback, and the Affordance Mechanism estimates the impact of really useful fixes. Panda uses Retrieval Augmented Generation for contextual query handling, employing embeddings for similarity searches. Telemetry metrics and troubleshooting docs provide multi-modal data for higher understanding and more accurate recommendations, addressing the contextual challenges of database performance debugging.

In a small experimental study comparing Panda, utilizing GPT-3.5, with GPT-4 for real-world problematic database workloads, Panda demonstrated superior reliability and usefulness in keeping with Database Engineers’ evaluations. Intermediate and Advanced DBEs found Panda’s answers more trustworthy and useful because of source citations and correctness grounded in telemetry and troubleshooting documents. Beginner DBEs also favored Panda but highlighted concerns about specificity. Statistical evaluation using a two-sample T-Test showed the statistical superiority of Panda over GPT-4.

In conclusion, the researchers introduce Panda, an modern system for autonomous database debugging using NL agents. Panda excels in identifying and rejecting irrelevant queries, constructing meaningful multi-modal contexts, estimating impact, offering citations, and learning from feedback. It emphasizes the importance of addressing open research questions encountered during its development and invites collaboration from the database and systems communities to reshape the database debugging process collectively. The system goals to redefine and enhance the general approach to debugging databases.


Take a look at the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our 36k+ ML SubReddit, 41k+ Facebook Community, Discord Channel, and LinkedIn Group.

For those who like our work, you’ll love our newsletter..

Don’t Forget to hitch our Telegram Channel


Asjad is an intern consultant at Marktechpost. He’s persuing B.Tech in mechanical engineering on the Indian Institute of Technology, Kharagpur. Asjad is a Machine learning and deep learning enthusiast who’s all the time researching the applications of machine learning in healthcare.


[Free AI Event] 🐝 ‘Real-Time AI with Kafka and Streaming Data Analytics’ (Jan 15 2024, 10 am PST)

LEAVE A REPLY

Please enter your comment!
Please enter your name here