Interpretability · probe auditing

Does a Probe Mean What It Says? Auditing Probes with Natural Language Autoencoders

Probes that detect concepts in a model's activations are widely relied upon for both safety evaluation and deployment monitoring; however, what a probe actually detects may not exactly match what was intended. This work introduces a label-free method to audit what concepts a probe fires on, using NLAs. As a proof of concept, a demonstration is given discovering a confounding factor in a probe. Future work will evaluate its usefulness by analyzing published probes.

By Jared Baribeau · GitHub · LinkedIn

Published: May 25, 2026

Updated: May 26, 2026

Author's note: [show]

Introduction

Probes are an important tool for safety and interpretability, useful in both pre-release model evaluation and in deployment monitoring. They can be used to monitor a model's latent-space activations for the presence of specific concepts or behaviors. Probes can track potentially dangerous behaviors (such as deception, sandbagging, situational awareness) or topics (such as bio-weapons manufacturing, illegal activities).

Ideally, probes fire any time the target concept is represented in the model's activations, and not otherwise. In practice however, a probe learns the difference between the training data "positive" set and "negative" set, and verifying that the difference between labels accurately identifies and isolates the target concept can be difficult or impossible. It is common for related or unexpected (but different) topics to also be separated by the dataset labels, and therefore to be targeted by the probe.

Examples:

• A probe trained to detect "evaluation awareness" was shown to largely track multiple-choice formatting [1].
• A deception probe can trigger on deception related dialogues, even if the response is honest [2].
• Probes trained to detect whether statements are true were shown to fail on statements with "not" [3] or could not disambiguate "truth" from "commonly believed" or "verifiable" [4].

Recent work on Natural Language Autoencoders (NLAs)[6] introduced a tool for natural language readouts of a model's residual-stream activations. This new "LLM mind reading" tool opens up interesting new possibilities in cross-checking probe behavior: use an NLA to compare the model's latent-space representations to the probe's output. If successful, this could be used to improve the reliability of the probes relied upon for critical, deployed safety applications.

This work demonstrates the first step: using an NLA to discover a confounding factor in a probe.

Approach: Auditing a Probe

The probe audit answers the question: what concepts really cause the probe to fire? To do this, prompts are run through the audit pipeline, and concepts that are present/absent when the probe fires are tracked.

Probe audit pipeline

1. Prompt the target model. [A]
2. Probe the activations: above threshold? (yes/no). [B]
3. Verbalize the activations using an NLA. [C]
4. Extract concept tags from the verbalization (using a concept judge, with no set taxonomy). [D]
5. Rank the concepts most closely tracked by probe. [E]

To start, this is tested using a refusal probe, which is an easy probe to validate due to the very clear separation of "refusal" and "non-refusal" behavior.

Here are some refusal probe audit pipeline samples:

Next, the samples are collected and results ranked.

Audit Method Validation

To test the approach, two refusal probes are compared: a baseline, and one trained on format-confounded data. Harmful prompts are given in multiple-choice format, and harmless prompts in regular prose. This injects an obvious confound, which should be surfaced by the audit.

1. Train a refusal probe with an intentional confound (multiple-choice format)
2. Audit the confounded probe, and check if the confound is discovered.

Results

The NLA audit discovers the confound. The following chart shows the difference in concepts tracked by the baseline and confounded probes. The injected confound (multiple-choice format) appears, representing a significant portion of the confounded probe's tracked concepts.

This result validates the proof of concept: NLAs may indeed offer something useful for auditing and understanding probes.

Conclusion

The findings here are positive! This work intentionally used a simple, planted confound (prompt format) on a simple, verifiable probe (refusal) as a proof of concept. Under these conditions, the findings aren't particularly useful; however, they indicate that further work is worth pursuing.

The next step is to apply the presented audit methodology on existing, published probes and evaluate the output.

Limitations

Future work

References

1. ↑ return to bodyDevbunova (2026). Is Evaluation Awareness Just Format Sensitivity? arXiv:2603.19426.
2. ↑ return to bodyGoldowsky-Dill et al. (2025). Detecting Strategic Deception Using Linear Probes. arXiv:2502.03407.
3. ↑ return to bodyLevinstein & Herrmann (2024). Still No Lie Detector for Language Models. Philosophical Studies.
4. ↑ return to bodyMarks & Tegmark (2024). The Geometry of Truth. arXiv:2310.06824.
5. ↑ return to bodyArditi et al. (2024). Refusal in LLMs Is Mediated by a Single Direction. arXiv:2406.11717.
6. ↑ return to bodyFraser-Taliente et al. (2026). Natural Language Autoencoders. transformer-circuits.pub/2026/nla.

Code & data

Source on GitHub

Acknowledgements

This project was completed with the support from the BlueDot Impact rapid grant program, and as part of a BlueDot Impact Technical AI Safety Project Sprint.

Thanks for the support! 🚀

And to you 👀! Thanks for reading. Questions? Feedback? I'd love to hear from you, here.

Appendix

Experiment specifications and method details, referenced from the body by [letter].

Extract the concepts the description says are PRESENT — what the model is representing or about to do. Output 1–4 concise tags.

· short canonical noun phrases, lowercase (e.g. "multiple-choice format", "refusal", "illegal activity").
· name SPECIFIC content/structure, not vague words like "response" or "text".
· reuse the same tag for the same idea across explanations.
· include format/structure as a concept when present.

Respond with ONLY a JSON object: {"concepts": ["tag", ...]}