Research

From data to artificial intelligence: researching, experimenting, and building.

The work moves through data and models: how a batch is checked, how a system learns, and when an answer deserves trust.

Data engineering shows up as data quality. Every record is accepted or set aside, and the totals have to be checkable. In machine learning and deep learning I compared classical models with a neural network I built, on one concrete prediction.

The most developed line studies systems that retrieve evidence and decide on an answer. The question there is evaluation: which criterion is written before the result is known, when to abstain, and how to see that a change makes a working behavior worse.

Each study leaves something learned, a capability that can be shown, and a next question. The path below belongs only to that line. The index gathers everything that is published.

A line in development: reliability and evaluation of RAG systems

The order in which these three questions opened. They are separate studies, not a platform and not one evaluation carried across time.

Index

Published work, grouped by the problem it studies.

Reliability and evaluation of RAG systems

Three studies, with separate splits.

Synthetic study

When retrieval should answer

I froze three criteria and scored them once. The result was 3/3: 17/20, 17/22, and 3/14. Decision accuracy on the same holdout was 28/36. The study does not generate an answer. That holdout is open, so it cannot be reused as unseen.

Read the study

Synthetic study

A model comparison that recommended nobody

Three setups, same synthetic ticket workflow, quality bar fixed first. Exact match was 11/30, 10/30, and 10/30. None was eligible. A lower cost did not change that. The holdout for that run is open. There is no interactive comparator.

Read the comparison

Example note

A regression check on example data

A baseline passes. One prompt change introduces a serious regression, and the release stays blocked.

The change under test is a single one: the instruction to escalate is removed from the prompt.

The behavior being protected is that a risk-critical case still reaches a person.

The regression shows up because the baseline still escalates and the candidate, after that one change, stops doing so on a critical case.

A failed check blocks the release. It does not pick a winner, and it does not let the change through.

It is useful when a system is updated, because the change looks small. The rows are example data. This note is not a monitoring system, and it is not operated from the page.

Data engineering and machine learning

Each piece of work answers its own question.

Synthetic lab

A data-quality lab

Validation decides what happens to each record. On run 6afdf1566acc8178 every source row was accepted or quarantined, and the line totals reconciled: 400.00 = 400.00. This is not Experian and not Databricks.

Read the lab

Academic work

DeepForward

It compares classical machine-learning models with a polynomial neural network to predict whether a customer will subscribe to a term deposit. The grade was 10/10. The viewer and the PDF stay where they are.

Open the thesis

One particular direction: evaluating RAG systems

This direction belongs only to the reliability line. It does not cover the lab or the thesis, and it is not a finished system. What already has a study or a note stays separate from what is still open.

Retrieval, evidence, and abstention
The published study already scores when to answer and when to stay quiet. Trying that decision on an example is still open. A separate demo uses a different corpus and does not reproduce this study’s results. It is not currently publicly available.
Whether a claim is supported
It is still open whether a claim is supported, contradicted, or unset. There is no public result to show.
Generated answers
Evaluating the wording of a generated answer would be a further experiment. That work is not published.
Regression detection
The note on this page shows a change that makes a protected behavior worse and blocks the release. It is not a monitoring system.

If this line of work fits what you are building, let’s talk.

We can talk about a collaboration, building something together, a contract, or taking part in this line of work.

Get in touch

From data to artificial intelligence: researching, experimenting, and building.

A line in development: reliability and evaluation of RAG systems

One particular direction: evaluating RAG systems

Retrieval, evidence, and abstention

Whether a claim is supported

Generated answers

Regression detection

If this line of work fits what you are building, let’s talk.