Integration and inference
ProblemA model that runs on one machine, in one person's environment, is not yet a model the team can run again.
ContributionI integrated, packaged, and ran models across different inference backends, in containers, in the cloud, and on GPU machines.
ResultThe same model could be packaged and run again, on more than one backend, without depending on a single workstation.
Compression
ProblemThe models were heavier than the deployment could comfortably carry, and compression changes what a model does.
ContributionI compressed them by pruning, by quantization, and with tensor networks. I used knowledge distillation so a smaller model could learn from a larger one, and I worked on recovering quality after the compression.
ResultThe compressed models could be served where the original weight did not fit, and brought back toward the behaviour the use actually needed.
Computer vision
ProblemA detector score can look healthy while a whole kind of object is missed.
ContributionI evaluated detectors with precision, recall, and mAP, the mean precision across detections, and I looked at the detection errors behind those scores.
ResultThe team could see which detections failed, not only the average.
Language-model evaluation
ProblemComparing language models on one automatic score leaves out whether an answer is actually acceptable.
ContributionI compared models and used another model to judge the answers.
ResultThe comparison could be read as a judgment, not only as a single number.
RAG evaluation
ProblemRetrieval plus a generated answer can look fluent when the retrieved context is thin, or when the system should have declined.
ContributionI evaluated retrieval and the answer with RAGAS, and with complementary measures: when the system should abstain, and whether the context is enough.
ResultThe evaluation showed the support for the answer, and the cases where answering was the wrong move.