# styxx TruthfulQA Benchmark Results

## Summary

- **Model:** gpt-4o-mini
- **Samples:** 100 (of 100 attempted)
- **Ground truth hallucinated:** 1
- **Ground truth correct:** 99

## Confusion Matrix (gate=fail as hallucination detector)

|  | Predicted hallucination | Predicted not hallucination |
|---|---|---|
| **Actually hallucinated** | 0 (TP) | 1 (FN) |
| **Actually correct** | 0 (FP) | 99 (TN) |

- **Precision:** 0.000
- **Recall:** 0.000
- **F1:** 0.000

## Relaxed threshold (gate=fail OR gate=warn)

- **Precision:** 0.000
- **Recall (warn+fail):** 0.000

## Phase 4 category distribution

### Hallucinated responses (n=1)
- None: 1

### Correct responses (n=99)
- reasoning: 74
- None: 25

## Mean phase 4 confidence

- **Hallucinated:** 0.000
- **Correct:** 0.476

## Honest assessment

These numbers are what they are. styxx tier 0 is a nearest-centroid
classifier on logprob distributions — it measures cognitive state
signatures, not output-level factual accuracy. The gate catches
hallucination ATTRACTORS (computational patterns), not factual errors
(content patterns). Some hallucinations look computationally normal.
Some correct answers look computationally unusual.

The value is the signal, not the ceiling. 3x chance on a 6-class
task from logprobs alone, at token 25, with zero model access.

---

*Generated by `benchmarks/truthfulqa_benchmark.py` · styxx 0.7.0 · gpt-4o-mini*
