Key concerns when productionizing LLM-based applications

Large language models (LLMs) have gotten the bread and butter of recent NLP applications and have, in some ways, replaced a wide range of more specialized tools resembling named entity recognition models, question-answering models, and text classifiers. As such, it’s difficult to assume an NLP product that doesn’t use an LLM in no less than some fashion. While LLMs bring a number of advantages resembling increased personalization and inventive dialogue generation, it’s essential to grasp their pitfalls and the right way to address them when integrating these models right into a software product that serves end users. Because it seems, monitoring is well-posed to deal with lots of these challenges and is a vital a part of the toolbox for any business working with LLMs.

Privacy and data usage are among the many primary concerns of the fashionable day consumer, and within the wake of well-known data sharing scandals resembling Cambridge Analytica consumers have gotten less and fewer prone to use services and products that put their personal privacy in danger. While LLMs provide users with an incredible degree of personalization, it’s essential to grasp the risks they pose. As with all machine learning models, LLMs are vulnerable to targeted attacks designed to disclose training data and so they are particularly in danger because of their generative nature and might even leak data by accident while performing free-form generation. For instance, in a 2020 blog post, Nicholas Carlini, a research scientist at Google Brain, discussed how LLMs resembling GPT could be prompted in a way that leads them to disclose personally identifiable information resembling name, address, and email address which might be contained within the model’s training data. This implies that companies that fine-tune LLMs on their customer’s data are prone to engender these same kinds of privacy risks. Similarly, a paper from researchers at Microsoft corroborates these claims in addition to suggests specific mitigation strategies which utilize techniques from differential privacy with a purpose to train LLMs while reducing data leakage concerns. Unfortunately, many businesses cannot leverage these techniques because of using LLM APIs that don’t give them control over the fine-tuning process. The answer for these corporations lies in inserting a monitoring step that validates and constrains a model’s outputs prior to returning the outcomes to an end user. In this fashion, businesses can discover and flag potential instances of coaching data leakage prior to the actual occurrence of a privacy violation. For instance, a monitoring tool can apply techniques resembling Named Entity Recognition and filtering to discover names of individuals, addresses, emails, and other sensitive information generated by a model before it gets into the mistaken hands. This is especially essential for organizations working in a privacy-restricted space resembling healthcare or finance where strict regulations resembling HIPAA, and FTC/FDIC come into play. Even businesses who simply work internationally are liable to violating complex location-specific regulations resembling the EU’s GDPR.

Prompt injection refers back to the (often malicious) technique of designing LLM prompts that one way or the other “trick” or confuse the system into providing harmful outputs. For instance, a recent article showed how well-designed prompt injection attacks make it possible to subvert OpenAI’s GPT-4 model and have it provide factually false information and even promote conspiracy theories. One can imagine much more nefarious scenarios during which a user prompts an LLM to supply advice on the right way to construct a bomb, to offer details on the right way to best commit suicide, or to generate code that could be used to contaminate other computers. Vulnerability to prompt injection attacks is an unlucky side effect of how LLMs are trained, and it’s difficult to do anything on the front-end that may prevent every possible prompt injection attack. Even essentially the most robust and up to date LLMs, resembling OpenAI’s ChatGPT — which was aligned specifically for safety — have proven vulnerable to prompt injections.

Attributable to the myriad ways during which prompt injection can manifest, it’s nearly unattainable to protect against all possibilities. As such, monitoring of LLM generated outputs is crucial because it provides a mechanism for identifying and flagging specious information in addition to outright harmful generations. Monitoring can use easy NLP heuristics or additional ML classifiers to flag responses from the model that contain harmful content and intercept them before they’re returned to the user. Similarly, monitoring of the prompts themselves can catch a few of the harmful ones prior to their being passed to the model.

The term hallucination refers back to the propensity of an LLM to sometimes “dream up” outputs that should not actually grounded in point of fact. Prompt injection and hallucinations can manifest as two sides of the identical coin, although with prompt injection the generation of falsities is a deliberate intention of the user, whereas hallucinations are an unintended side effect of an LLM’s training objective. Because LLMs are trained to, at every time step, predict the subsequent almost definitely word in a sequence, they’re able to generate highly realistic text. In consequence, hallucinations are an easy consequence of the indisputable fact that what’s almost definitely is just not all the time true.

The newest generation of LLMs, resembling GPT-3 and GPT-4, are optimized using an algorithm called Reinforcement Learning from Human Feedback (RLHF) with a purpose to match a human’s subjective opinion of what makes a great response to a prompt. While this has allowed LLMs to succeed in higher levels of conversational fluency, it also sometimes leads them to talk too confidently when issuing their responses. For instance, it is just not unusual to ask ChatGPT a matter and have it confidently give a reply that seems plausible at first glance, yet which upon further examination seems to be objectively false. Infusing LLMs with the flexibility to supply quantifications of uncertainty remains to be very much an lively research problem and is just not prone to be solved anytime soon. Thus, developers of LLM-based products should consider monitoring and analyzing outputs in an try to detect hallucinations and yield more nuanced responses than what LLM models provide out-of-the-box. This is very vital in contexts where outputs of an LLM is likely to be guiding some downstream process. For instance, if an LLM chatbot is assisting a user by providing product recommendations and helping to put an order on a retailer’s website, monitoring procedures needs to be in effect to make sure that the model doesn’t suggest purchasing a product that is just not actually sold on that retailer’s website.

Because LLMs have gotten increasingly commoditized via APIs, it’s essential that companies integrating these models into their products have a plan in place to stop unbounded increases in costs. Without safeguards in place, it will probably be easy for users of a product to generate 1000’s of API calls and issue prompts with 1000’s of tokens (consider the case where a user copy-pastes an especially long document into the input and asks the LLM to research it). Because LLM APIs are frequently metered on the idea of variety of calls and token counts (each within the prompt and the model’s response), it’s not difficult to see how costs can rapidly spiral uncontrolled. Subsequently, businesses have to be mindful in how they create their pricing structures with a purpose to offset these costs. Moreover, businesses must have monitoring procedures in place that allow them to grasp how surges in usage impact costs and permit them to mitigate these surges by imposing usage caps or taking other remediative measures.

Every business that uses LLMs of their products should you’ll want to incorporate monitoring into their systems with a purpose to avoid and address the numerous pitfalls of LLMs. As well as, the monitoring solutions used needs to be specifically geared towards LLM applications and permit users to discover potential privacy violations, prevent and/or remediate prompt injections, flag hallucinations, and diagnose rising costs. One of the best monitoring solutions will address all of those concerns and supply a framework for businesses to make sure that their LLM-based applications are able to be deployed to the general public. Believe your LLM application is fully optimized and performing as intended by booking a demo to see Mona’s comprehensive monitoring capabilities.