TLDR

  • Get 15-20 traces, more the merrier. Aim for 100. Write your comments about them, analyze errors.
  • Decide what a good output is and why it is a good output
    Have criteria like accuracy, no hallucinations, prompt adherence, etc
  • use OpenAI o3 for evals, give it the input prompt, input context and LLM output to give out a rating

I see evals as unit and integration tests combined. In the normal SWE world, functions, their inputs and outputs are deterministic and you can think of all paths the function can go through and write tests accordingly.

For LLMs, the outputs are not deterministic. You will have to keep looking at outputs – traces(input, LLM params and outputs). Make notes about them. You need to know what a good output is and what errors look like for your use case.

For simpler tasks like sentiment analysis, you would have a csv with text and corresponding correct output labels. Then you would run the LLM on each of the texts, get the corresponding output and compare to get an accuracy score. For such discrete outputs, we can use accuracy, precision, recall and f1-scores

The tricky part, and most real world use cases revolve around getting text output from the LLM. A summary may be good by your standards, but not by somebody else on the team. Traditional methods like Bleu or Rouge score don’t work as business context and nuance gets lost.

For complex tasks with varying text outputs, we need another LLM to judge the LLM output. Since text output quality is subjective, it is important to have distinct pillars/axes on which you will evaluate the output.

What makes a good output?

  • It is accurate
  • There are no hallucinations
  • Instructions in the original prompt are adhered to
  • The output covers everything from the inputs in a way that is agreeable to you (use case specific)
  • If you are supplying an image to the LLM, the output should capture the info correctly from it, etc

The judge LLM is like a teacher evaluating a student’s answer sheet. So it is imperative that the teacher has the same context as the LLM output it is judging.

A sample eval prompt would look like this:

'''
You are an expert evaluator of <TASK> --> set a persona, describe the task in detail

Inputs provided for the evaluation:

Original prompt given to the LLM
<llm_prompt>

Input context
<llm_input>

Output generated by the LLM
<llm_output>

Evaluation Criteria, Weights, & Scoring (Score 1-5):

Score the llm_output from 1 to 5 for each criterion. Note the weight assigned to each criterion. Assess how well the output accurately reflects the llm_input and adheres to the llm_prompt instructions and format.

1: Poor (Significant failure)
2: Fair (Major issues)
3: Good (Mostly meets basics)
4: Very Good (Minor issues)
5: Excellent (Flawless execution)

For each feedback, give a justification.

List of your axes and weightage --> this will be task specific
1. accuracy (weightage 5)
describe what accuracy means in your context

2. no hallucinations (weightage 3)
describe hallucinations

3. adhereance to inputs (weightage 3)
.
.
.

The final score is a weighted average of all the scores across all the criteria
Return a JSON
{
  {
    "criterion": ...,
    "accuracy" : <score>,
    "justification" : ...
  },
  .
  .
  .
  "overall_score": <weighted average>
  "overall_assessment": ...
}
'''

In my experience, OpenAI o3 is the best evaluator. It takes slightly more time but is really thorough and is strict on the scoring. I also made deliberate mistakes, even subtle 2-3 word changes in a 200 word summary and it caught it.

Aim to have 100 good samples you can run evals against, for complex tasks or lack of data you should have at least 15-20 good examples. Generate synthetic data if needed.

Based on tasks and bandwidth:

  • you could have different LLMs assess different axis mentioned in the prompt
  • or use different LLMs with the same evals and do a weighted average of all the judges’ scores personally, o3 alone is sufficient
contents
  1. TLDR