π Trier, Germany Β |Β π MSc AI @ Hochschule Trier (May 2026) Β |Β π’ ML Engineer @ One75 Labs, Berlin
I build evaluation infrastructure for language models β not dashboards that look good in demos, but pipelines that surface what metrics actually measure versus what they claim to measure.
My core thesis: confidence scores are lying to you. I proved it with a near-zero correlation (r = 0.009) between model confidence and internal reasoning faithfulness. That finding came from combining activation patching, causal circuit analysis, and a reproducible benchmarking framework I built from scratch.
Currently writing my MSc thesis on explainable AI for LLMs with causally grounded natural language explanations, while working as an ML Engineer at One75 Labs in Berlin on production LLM evaluation systems.
| What | Result |
|---|---|
| π¬ Causal circuit discovery speed | 1.2s on CPU vs 43.2s baseline β 37Γ faster than ACDC (Conmy et al. 2023) |
| π Confidence vs. faithfulness correlation | r = 0.009 β near-zero. Confidence-based eval signals are unreliable. |
| β LLM explanation quality | 99% quality via ERASER metrics vs. 60% template baseline |
| π§ͺ CI reliability | 12/12 passing tests β reproducible, auditable evaluation framework |
| π¦ Open-source reach | Published on arXiv, deployed on Hugging Face, packaged on PyPI with 76 automated tests |
| π Research output | Submitted to ICML 2026 Workshop on Mechanistic Interpretability |
Python PyTorch TransformerLens arXiv PyPI Hugging Face
The project that came out of a direct question: can we tell, causally, which parts of GPT-2 drove a specific prediction?
- Built a causal circuit discovery engine that answers that question in 1.2s on CPU using 3 forward passes β 37Γ faster than the ACDC baseline
- Quantified r = 0.009 correlation between model confidence and internal reasoning faithfulness, a result with direct implications for EU AI Act compliance
- Automated generation of all 9 required EU AI Act Annex IV sections from a single function call β structured JSON output ready for GRC system import
- Published on arXiv (2603.09988), deployed a live Hugging Face demo, and shipped to PyPI with a CLI + 76 automated tests
Compliance teams can audit any model in under a minute with zero infrastructure setup.
βοΈ Azure Cloud AI RAG System
Azure OpenAI Azure AI Search FastAPI Streamlit GPT-4o-mini
Document Q&A system with source citations built on Azure's full AI stack.
- Hybrid search combining vector embeddings + keyword matching for semantically-aware retrieval
- Document ingestion pipeline with 512-token chunking and
text-embedding-3-smallembeddings - FastAPI backend + Streamlit frontend with streaming responses for real-time answer generation
Azure Machine Learning MLflow scikit-learn Azure ML SDK v2
Automated 4-step ML pipeline: data prep β training β evaluation β model registration.
- 74% test accuracy, 80% F1, 87% AUC-ROC on heart disease prediction (200-record held-out test set)
- Auto-scaling compute with minimum zero nodes β clusters shut down automatically when idle
- MLflow tracking + Azure ML Model Registry for full experiment reproducibility and version rollback
Explainable AI for LLMs: A Causally Grounded Pipeline Submitted to ICML 2026 Workshop on Mechanistic Interpretability
The core finding: traditional attention-based metrics miss 39% of prediction behavior. Ground truth established via 100% sufficiency scoring using activation patching and causal circuit analysis. The pipeline converts technical circuit data into structured natural language explanations validated against ERASER metrics.
Languages: Python, SQL
ML / Research: PyTorch, TransformerLens, HuggingFace, scikit-learn, NumPy, Pandas
Cloud / Infra: Azure Machine Learning, MLflow, Docker, REST APIs, FastAPI, GitHub Actions, CI/CD
Core Expertise: Mechanistic Interpretability Β· Activation Patching Β· Transformer Architecture Β· LLM Evaluation Methodology Β· Causal Analysis Β· Python Package Development (PyPI) Β· Prompt Engineering
- π MSc Thesis β Mechanistic interpretability of LLMs with causally grounded explanations
- π’ ML Engineer @ One75 Labs β Production LLM evaluation infrastructure, Berlin
- π― Open to β ML Engineer / AI Researcher roles in the EU (post-graduation, May 2026)