Find out how to construct and run LLM evals — and why it’s best to use precision and recall when benchmarking your LLM prompt template

This piece is co-authored by Ilya Reznik

Large language models (LLMs) are an incredible tool for developers and business leaders to create latest value for consumers. They make personal recommendations, translate between unstructured and structured data, summarize large amounts of knowledge, and accomplish that far more.

Because the applications multiply, so does the importance of measuring the performance of LLM-based applications. This can be a nontrivial problem for several reasons: user feedback or some other “source of truth” is incredibly limited and infrequently nonexistent; even when possible, human labeling remains to be expensive; and it is straightforward to make these applications complex.

This complexity is commonly hidden by the abstraction layers of code and only becomes apparent when things go incorrect. One line of code can initiate a cascade of calls (spans). Different evaluations are required for every span, thus multiplying your problems. For instance, the easy code snippet below triggers multiple sub-LLM calls.

Fortunately, we will use the ability of LLMs to automate the evaluation. In this text, we are going to delve into learn how to set this up and be sure it’s reliable.

The core of LLM evals is AI evaluating AI.

While this may occasionally sound circular, we have now at all times had human intelligence evaluate human intelligence (for instance, at a job interview or your college finals). Now AI systems can finally do the identical for other AI systems.

The method here is for LLMs to generate synthetic ground truth that will be used to judge one other system. Which begs an issue: why not use human feedback directly? Put simply, since you won’t ever have enough of it.

Getting human feedback on even one percent of your input/output pairs is a huge feat. Most teams don’t even get that. But to ensure that this process to be truly useful, it can be crucial to have evals on every LLM sub-call, of which we have now already seen there will be many.

Let’s explore learn how to do that.

LLM_model_evals != LLM_System_evals

LLM Model Evals

You would possibly have heard of LLM evals. This term gets utilized in many various ways that each one sound very similar but actually are very different. Certainly one of the more common ways it gets used is in what we are going to call LLM model evals. LLM model evals are focused on the general performance of the foundational models. The businesses launching the unique customer-facing LLMs needed a option to quantify their effectiveness across an array of various tasks.

One popular library that has LLM model evals is the OpenAI Eval library, which was originally focused on the model evaluation use case. There are a lot of metrics on the market, like HellaSwag (which evaluates how well an LLM can complete a sentence), TruthfulQA (measuring truthfulness of model responses), and MMLU (which measures how well the LLM can multitask). There’s even a leaderboard that appears at how well the open-source LLMs stack up against one another.

LLM System Evals

Up thus far, we have now discussed LLM model evaluation. In contrast, LLM system evaluation is the entire evaluation of components that you’ve gotten control of in your system. An important of those components are the prompt (or prompt template) and context. LLM system evals assess how well your inputs can determine your outputs.

LLM system evals may, for instance, hold the LLM constant and alter the prompt template. Since prompts are more dynamic parts of your system, this evaluation makes a whole lot of sense throughout the lifetime of the project. For instance, an LLM can evaluate your chatbot responses for usefulness or politeness, and the identical eval can provide you with details about performance changes over time in production.

Which To Use? It Depends On Your Role

There are distinct personas who make use of LLM evals. One is the model developer or an engineer tasked with fine-tuning the core LLM, and the opposite is the practitioner assembling the user-facing system.

There are only a few LLM model developers, and they have a tendency to work for places like OpenAI, Anthropic, Google, Meta, and elsewhere. Model developers care about LLM model evals, as their job is to deliver a model that caters to a wide range of use cases.

For ML practitioners, the duty also starts with model evaluation. Certainly one of the primary steps in developing an LLM system is picking a model (i.e. GPT 3.5 vs 4 vs Palm, etc.). The LLM model eval for this group, nonetheless, is commonly a one-time step. Once the query of which model performs best in your use case is settled, the vast majority of the remaining of the applying’s lifecycle will probably be defined by LLM system evals. Thus, ML practitioners care about each LLM model evals and LLM system evals but likely spend far more time on the latter.

Having worked with other ML systems, your first query is probably going this: “What should the final result metric be?” The reply relies on what you are attempting to judge.

• Extracting structured information: You’ll be able to take a look at how well the LLM extracts information. For instance, you’ll be able to take a look at completeness (is there information within the input that will not be within the output?).
• Query answering: How well does the system answer the user’s query? You’ll be able to take a look at the accuracy, politeness, or brevity of the reply — or the entire above.
• Retrieval Augmented Generation (RAG): Are the retrieved documents and final answer relevant?

As a system designer, you’re ultimately answerable for system performance, and so it’s as much as you to know which features of the system must be evaluated. For instance, If you’ve gotten an LLM interacting with children, like a tutoring app, you’d wish to be sure that the responses are age-appropriate and are usually not toxic.

Some common evaluations being employed today are relevance, hallucinations, question-answering accuracy, and toxicity. Each one among these evals can have different templates based on what you are attempting to judge. Here is an example with relevance:

This instance uses the open-source Phoenix tool for simplicity (full disclosure: I’m on the team that developed Phoenix). Inside the Phoenix tool, there exist default templates for commonest use cases. Here is the one we are going to use for this instance:

You might be comparing a reference text to an issue and trying to find out if the reference text incorporates information relevant to answering the query. Here is the info:
[BEGIN DATA]
************
[Question]: {query}
************
[Reference text]: {reference}
[END DATA]
Compare the Query above to the Reference text. You could determine whether the Reference text
incorporates information that may answer the Query. Please give attention to whether the very specific
query will be answered by the data within the Reference text.
Your response should be single word, either "relevant" or "irrelevant",
and shouldn't contain any text or characters apart from that word.
"irrelevant" signifies that the reference text doesn't contain a solution to the Query.
"relevant" means the reference text incorporates a solution to the Query.

We will even use OpenAI’s GPT-4 model and scikitlearn’s precision/recall metrics.

First, we are going to import all essential dependencies:

from phoenix.experimental.evals import (
RAG_RELEVANCY_PROMPT_TEMPLATE_STR,
RAG_RELEVANCY_PROMPT_RAILS_MAP,
OpenAIModel,
download_benchmark_dataset,
llm_eval_binary,
)
from sklearn.metrics import precision_recall_fscore_support

Now, let’s usher in the dataset:

# Download a "golden dataset" built into Phoenix
benchmark_dataset = download_benchmark_dataset(
task="binary-relevance-classification", dataset_name="wiki_qa-train"
)
# For the sake of speed, we'll just sample 100 examples in a repeatable way
benchmark_dataset = benchmark_dataset.sample(100, random_state=2023)
benchmark_dataset = benchmark_dataset.rename(
columns={
"query_text": "query",
"document_text": "reference",
},
)
# Match the label between our dataset and what the eval will generate
y_true = benchmark_dataset["relevant"].map({True: "relevant", False: "irrelevant"})

Now let’s conduct our evaluation:

# Any general purpose LLM should work here, but it surely is best practice to maintain the temperature at 0
model = OpenAIModel(
model_name="gpt-4",
temperature=0.0,
)
# Rails will define our output classes
rails = list(RAG_RELEVANCY_PROMPT_RAILS_MAP.values())
benchmark_dataset["eval_relevance"] = 
llm_eval_binary(benchmark_dataset,
model,
RAG_RELEVANCY_PROMPT_TEMPLATE_STR,
rails)
y_pred = benchmark_dataset["eval_relevance"]
# Calculate evaluation metrics
precision, recall, f1, support = precision_recall_fscore_support(y_true, y_pred)

Evaluating LLM-Based Systems with LLMs

There are two distinct steps to the technique of evaluating your LLM-based system with an LLM. First, establish a benchmark in your LLM evaluation metric. To do that, you place together a dedicated LLM-based eval whose only task is to label data as effectively as a human labeled your “golden dataset.” You then benchmark your metric against that eval. Then, run this LLM evaluation metric against results of your LLM application (more on this below).

Step one, as we covered above, is to construct a benchmark in your evaluations.

To try this, you will need to begin with a metric best suited in your use case. Then, you wish the golden dataset. This ought to be representative of the kind of data you expect the LLM eval to see. The golden dataset must have the “ground truth” label in order that we will measure performance of the LLM eval template. Often such labels come from human feedback. Constructing such a dataset is laborious, but you’ll be able to often discover a standardized one for essentially the most common use cases (as we did within the code above).

Then you’ll want to resolve which LLM you should use for evaluation. This might be a special LLM from the one you’re using in your application. For instance, chances are you’ll be using Llama in your application and GPT-4 in your eval. Often this alternative is influenced by questions of cost and accuracy.

Now comes the core component that we are attempting to benchmark and improve: the eval template. If you happen to’re using an existing library like OpenAI or Phoenix, it’s best to start with an existing template and see how that prompt performs.

If there’s a particular nuance you should incorporate, adjust the template accordingly or construct your personal from scratch.

Be mindful that the template must have a transparent structure, just like the one we utilized in prior section. Be explicit in regards to the following:

• What’s the input? In our example, it’s the documents/context that was retrieved and the query from the user.
• What are we asking? In our example, we’re asking the LLM to inform us if the document was relevant to the query
• What are the possible output formats? In our example, it’s binary relevant/irrelevant, but it could possibly even be multi-class (e.g., fully relevant, partially relevant, not relevant).

You now must run the eval across your golden dataset. You then can generate metrics (overall accuracy, precision, recall, F1, etc.) to find out the benchmark. It is necessary to take a look at greater than just overall accuracy. We’ll discuss that below in additional detail.

If you happen to are usually not satisfied with the performance of your LLM evaluation template, you’ll want to change the prompt to make it perform higher. That is an iterative process informed by hard metrics. As is at all times the case, it can be crucial to avoid overfitting the template to the golden dataset. Be sure to have a representative holdout set or run a k-fold cross-validation.

Finally, you arrive at your benchmark. The optimized performance on the golden dataset represents how confident you’ll be able to be in your LLM eval. It should not be as accurate as your ground truth, but it should be accurate enough, and it should cost much lower than having a human labeler within the loop on every example.

Preparing and customizing your prompt templates permits you to arrange test cases.

The industry has not fully standardized best practices on LLM evals. Teams commonly have no idea learn how to establish the correct benchmark metrics.

Overall accuracy is used often, but it surely will not be enough.

That is some of the common problems in data science in motion: very significant class imbalance makes accuracy an impractical metric.

Interested by it when it comes to the relevance metric is useful. Say you undergo all the difficulty and expense of putting together essentially the most relevant chatbot you’ll be able to. You decide an LLM and a template which are right for the use case. This could mean that significantly more of your examples ought to be evaluated as “relevant.” Let’s pick an extreme number for instance the purpose: 99.99% of all queries return relevant results. Hooray!

Now take a look at it from the viewpoint of the LLM eval template. If the output was “relevant” in all cases, without even taking a look at the info, it could be right 99.99% of the time. Nevertheless it would concurrently miss the entire (arguably most) necessary cases — ones where the model returns irrelevant results, that are the very ones we must catch.

In this instance, accuracy could be high, but precision and recall (or a mix of the 2, just like the F1 rating) could be very low. Precision and recall are a greater measure of your model’s performance here.

The opposite useful visualization is the confusion matrix, which mainly helps you to see accurately and incorrectly predicted percentages of relevant and irrelevant examples.

At this point it’s best to have each your model and your tested LLM eval. You may have proven to yourself that the eval works and have a quantifiable understanding of its performance against the bottom truth. Time to construct more trust!

Now we will actually use our eval on our application. This may help us measure how well our LLM application is doing and determine learn how to improve it.

The LLM system eval runs your entire system with one extra step. For instance:

• You retrieve your input docs and add them to your prompt template, along with sample user input.
• You provide that prompt to the LLM and receive the reply.
• You provide the prompt and the reply to your eval, asking it if the reply is relevant to the prompt.

It’s a best practice to not do LLM evals with one-off code but fairly a library that has built-in prompt templates. This increases reproducibility and allows for more flexible evaluation where you’ll be able to swap out different pieces.

These evals must work in three different environments:

Pre-production

Whenever you’re doing the benchmarking.

Pre-production

Whenever you’re testing your application. That is somewhat just like the offline evaluation concept in traditional ML. The concept is to know the performance of your system before you ship it to customers.

Production

When it’s deployed. Life is messy. Data drifts, users drift, models drift, all in unpredictable ways. Simply because your system worked well once doesn’t mean it should accomplish that on Tuesday at 7 p.m. Evals make it easier to constantly understand your system’s performance after deployment.

What number of rows must you sample?

The LLM-evaluating-LLM paradigm will not be magic. You can not evaluate every example you’ve gotten ever run across — that will be prohibitively expensive. Nonetheless, you have already got to sample data during human labeling, and having more automation only makes this easier and cheaper. So you’ll be able to sample more rows than you’d with human labeling.

What evals must you use?

This relies largely in your use case. For search and retrieval, relevancy-type evals work best. Toxicity and hallucinations have specific eval patterns (more on that above).

A few of these evals are necessary within the troubleshooting flow. Query-answering accuracy is perhaps overall metric, but in the event you dig into why this metric is underperforming in your system, chances are you’ll discover it is due to bad retrieval, for instance. There are sometimes many possible reasons, and you would possibly need multiple metrics to resolve it.

What model must you use?

It’s not possible to say that one model works best for all cases. As a substitute, it’s best to run model evaluations to know which model is true in your application. You might also need to think about tradeoffs of recall vs. precision, depending on what is smart in your application. In other words, do some data science to know this in your particular case.

Having the ability to evaluate the performance of your application could be very necessary with regards to production code. Within the era of LLMs, the issues have gotten harder, but luckily we will use the very technology of LLMs to assist us in running evaluations. Such evaluation should test the entire system and not only the underlying LLM model — take into consideration how much a prompt template matters to user experience. Best practices, standardized tooling, and curated datasets simplify the job of developing LLM systems.