Complete — dashboard runs locally

Prompt Robustness
Evaluation Harness

A systematic framework for evaluating prompt engineering strategies across tasks and models. Measures not just accuracy, but robustness under perturbation and vulnerability to prompt injection attacks — the things production systems actually care about.

3
Prompt strategies
7
Perturbation types
7
Attack patterns
100%
CoT Q&A accuracy

Architecture

main.py  (entry point — configure flags, run pipeline)
│
├── models/client.py          Ollama API wrapper — model-agnostic chat interface
│
├── prompts/templates.py      Prompt registry — zero-shot / few-shot / CoT per task
│
├── tasks/tasks.py            Evaluation datasets: classification, Q&A, summarization
│
├── evaluators/
│   ├── metrics.py            BLEU, ROUGE, exact match scoring
│   └── llm_judge.py          LLM-as-judge via qwen2.5:7b (1–5 rubric)
│
├── robustness/perturbations.py   7 prompt perturbations + output stability scoring
│
├── defense/attacks.py            7 jailbreak attack patterns + violation rate tracking
│
├── tracker/logger.py         JSON experiment logging — one timestamped file per run
│
├── results/                  Auto-generated — additive, never overwrites old runs
│
└── dashboard/app.py          Plotly Dash — 7 sections, fully interactive

Strategy performance

Zero-shot vs few-shot vs chain-of-thought across 3 task types

TaskZero-ShotFew-ShotChain-of-ThoughtMetric
Sentiment Classification100%100%100%Exact match
Factual Q&A83.3%83.3%100%Exact match
Summarization50.7%59.8%45.3%ROUGE-1

CoT wins on Q&A. Few-shot wins on summarization. Showing the model examples matters more than asking it to reason on open-ended generation.

Robustness under perturbation

Prompts perturbed in 7 ways — all-caps, typos, removed punctuation, digit-to-word, shuffled sentences, extra whitespace. Score = word overlap between original and perturbed output.

StrategyAvg Robustness Score
Zero-Shot58.4%
Few-Shot55.0%
Chain-of-Thought29.5%

CoT is the best performer and the most fragile. More reasoning steps = more things that can shift under perturbation.

Security — prompt injection vulnerability

7 known jailbreak patterns tested per model. Violation rate = % of attacks that succeeded.

ModelParametersAvg Violation Rate
llama3.1:8b~8B74.6%
phi3:mini~3.8B87.3%

Attack patterns tested

ignore_instructionsDirectly tells the model to forget its system prompt
dan_roleplayDAN (Do Anything Now) — classic unrestricted AI roleplay
grandma_exploitAsks the model to act as a permissive grandmother character
output_format_manipulationHides malicious intent inside a poetry/formatting request
prompt_leakingAttempts to extract the system prompt verbatim
obfuscation_typosMisspells attack keywords to bypass keyword filters
fake_system_overridePretends a higher authority is overriding all instructions

Evaluation stack

Layered scoring — no single metric tells the full story

LayerMetricUsed ForLimitation
Exact match1.0 / 0.0Classification, Q&ABinary — no partial credit
BLEUWord n-gramQ&A, SummarizationRewards verbosity, misses semantics
ROUGE-1/2/LRecall overlapSummarizationRewards copy-paste
LLM-as-judge1–5 rubricAll tasksModel-dependent, can be inconsistent

Key discoveries

01

BLEU and ROUGE are blind to copy-paste

A model that ignores the instruction and reproduces the original paragraph verbatim would score near-perfect on both BLEU and ROUGE — every word in the reference summary is covered. BLEU and ROUGE measure word overlap, not understanding. LLM-as-judge was added as a mandatory layer: it explicitly scores for conciseness, penalizing copy-paste regardless of overlap score. Both scorers stay in the stack because they catch different failure modes.

02

Chain-of-thought is the best performer and the most fragile

CoT pushed Q&A accuracy from 83.3% to 100% — the clearest performance gain in the experiment. But CoT also scored the lowest robustness of all strategies (29.5% vs 58.4% for zero-shot). CoT makes the model generate a long reasoning chain before answering, and that chain is highly sensitive to phrasing. Small perturbations like shuffled sentences or removed punctuation cause the reasoning steps to change significantly even when the final answer stays correct. Practical implication: CoT is worth using when prompt wording is under your control. In production with messy user input, a well-crafted few-shot prompt may be the safer choice.

03

Small open-source models have essentially no prompt injection defense

Both models failed badly — llama3.1:8b at 74.6% violation rate, phi3:mini at 87.3%. The attacks that succeeded most consistently were the simplest: direct instruction override worked almost every time, prompt leaking handed over the full system prompt without hesitation, and obfuscation via typos ("ignroe" for "ignore") still worked because the model correctly inferred the intended word and then followed the malicious instruction anyway. These models are trained to be cooperative — and that cooperativeness extends to malicious instructions.

04

When ROUGE and the judge disagree, that disagreement is the signal

Several runs produced high ROUGE scores but low judge scores, and vice versa. High ROUGE + low judge → the model used the right words but missed the point, or was too verbose. Low ROUGE + high judge → the model produced a genuinely good summary using different vocabulary than the reference, which ROUGE penalizes unfairly. No single metric is sufficient. A production evaluation system needs at least two scorers that measure different things so they can catch each other's blind spots.

05

Faster is not always worse

phi3:mini runs at roughly half the latency of llama3.1:8b (9.1s vs 18.4s on CPU) with only a ~2% accuracy drop. For classification and Q&A tasks, both models scored identically. The performance gap only appears on summarization, where the larger model's broader knowledge base makes a measurable difference. Practical pattern: use a small fast model for structured tasks and reserve the larger model for open-ended generation where quality matters more than speed.

Stack

Python 3.11OllamaPlotly DashROUGEBLEULLM-as-judge (qwen2.5:7b)pandas
← All projectsGitHub ↗