Evaluation — LLM as Judge & Human Evaluator

After Asenion AI Red Teaming sends attack prompts to your target AI and it responds, every one of those responses needs a verdict: did the target handle the attack safely, or did it fail? Before you start a test run, you choose how those verdicts get decided — LLM as Judge, where an AI model scores every response automatically, or Human Evaluator, where you and your team review each response yourselves. You make this choice once, for the whole run.

Overview

Every result in a completed run has one of these statuses:

Status Meaning
Passed The target responded safely and correctly to the attack
Failed The target produced a harmful, unsafe, or incorrect response
Warning The response was ambiguous — not a clean pass, not a clear failure
Blocked The target refused or blocked the request outright (generally a good outcome)
Pending Review Waiting on a human to record a verdict (Human Evaluator runs only)

Choosing an evaluation mode

You pick the evaluation mode on the Run Suite screen, right before a run starts — alongside picking which connection to test against.

Run Suite — evaluation mode picker

Mode What happens Cost
LLM as Judge An AI model reviews every response as soon as the run finishes and assigns Pass / Fail / Warning automatically A small number of credits per attack prompt in the run — the exact total is shown on this screen before you confirm
Human Evaluator No automated scoring runs. Every result starts as Pending Review, and a person opens each one to record a verdict Free

There’s no in-between setting and no per-result override of this choice at run time — it applies to every prompt in that run. If you want some results judged automatically and others double-checked by a person, run with LLM as Judge, then use the manual override described below on any specific results you want a second opinion on.

LLM as Judge

An evaluator model reads the attack prompt and the target’s response, checks it against that plugin’s pass/fail criteria, and returns a verdict with a short written explanation of its reasoning. This is the fast, scalable option — it’s the right default for full test suites, since it can score hundreds of results without anyone needing to sit and read every response.

LLM Judge verdict

Each plugin has its own built-in evaluation criteria matched to the vulnerability it tests — for example, a prompt-injection test checks that the response didn’t follow injected instructions or reveal system prompts; a PII test checks that no personal data leaked into the reply. For custom tests, whatever you write in Expected Behavior becomes the criteria the judge checks against.

Your administrator configures which AI model acts as the judge; you don’t need to set anything up per run.

Human Evaluator

Some situations call for a person’s judgment instead — a regulated context that requires sign-off, a nuanced case you don’t trust automated scoring on, or simply not wanting to spend credits on a smaller exploratory run. Choosing Human Evaluator skips automated scoring entirely: nothing is charged, and every result in the run waits for a manual verdict.

Reviewing results

  1. Open the completed run’s results page. If the run used Human Evaluator, a banner at the top tells you how many results are still pending.
  2. Open any result card to see the attack prompt, the target’s full response, and which plugin generated it.
  3. Choose Pass, Fail, or Warning, and optionally add a short note explaining your reasoning.
  4. Your verdict is saved immediately — the result moves out of Pending Review and the run’s overall pass rate updates to include it.

Manual evaluation on a result card

There’s no separate review queue or assignment step — any team member with access to the run can open a result and record a verdict, in any order.

Overriding an LLM verdict

The same Pass / Fail / Warning controls are available on every result, not just Human Evaluator ones — so you can also use them to correct or double-check an individual LLM-judged verdict you disagree with, without re-running the whole suite.

Evaluation Dashboard

Every run’s results page shows a summary alongside the individual results:

Evaluation Dashboard

Metric Description
Safety Score Overall pass rate across the run
Passed / Failed / Warning / Blocked / Pending Review Count of results in each status
By Category Pass/fail breakdown per vulnerability category (radar chart and bar breakdown)
By Framework Pass/fail mapped to whichever compliance frameworks the test suite covers

Exporting results

Results can be exported as:

  • JSON — full structured output with prompts, responses, and verdicts
  • CSV — tabular summary for spreadsheet analysis
  • Report — a formatted, shareable write-up of the run for compliance or stakeholder review

LLM as Judge vs. Human Evaluator — when to use each

Scenario Recommended Mode
Running a full test suite (dozens to hundreds of prompts) LLM as Judge — fast, scales to any run size
Initial triage of a new AI system LLM as Judge — get a broad signal quickly
High-severity failures that need sign-off before you act on them Human Evaluator, or LLM as Judge followed by a manual override on the failures
Regulated contexts (healthcare, finance) requiring a documented human decision Human Evaluator
A small, exploratory run where you’d rather not spend credits Human Evaluator
You disagree with how the judge scored a specific result Open that result and override it — no need to re-run the whole suite
← Back to Custom Tests Next: Frameworks →