# Evaluation Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/evaluation
> Markdown URL: https://aitinkerers.org/technologies/evaluation.md
> Technology record last updated: 2026-03-12T20:45:16Z
> Generated: 2026-09-21T06:43:49Z

DeepEval: The LLM Evaluation Framework that integrates unit testing directly into your CI/CD pipeline for production-grade AI applications.

Evaluation, specifically DeepEval, is the critical framework for rigorously testing and validating Large Language Models (LLMs) before deployment. It applies the familiar unit-testing paradigm (like Pytest) to AI, ensuring measurable, repeatable quality for your Generative AI applications. The platform leverages over 50 research-backed metrics, including advanced techniques like G-Eval, to score subjective criteria with objective, criteria-based reasoning. This integration allows engineering teams to embed robust model performance checks directly into their existing continuous integration workflows, ensuring every prompt tweak or model update maintains production-grade standards.

- Official technology site: https://deepeval.com
- Public AI Tinkerers demos and talks: 1
- Result page: 1 of 1

## Recent Public Talks and Demos

### [AI for Capital Markets, Agents, Evaluation](https://london.aitinkerers.org/talks/rsvp_k2jEXYltQMw)

The talk will discuss the reality of building applications with generative models in environments that demand high quality and actionable responses in production. We’ll be going into our particular challenge of creating an application within the very domain-driven space we are working in (financial). We’ll also discuss strategies we are building to evaluate the output of generated by leveraging LLMs, and challenges around LLM evaluations for agentic pipelines.

- Event context: AI Tinkerers London - October Meetup 🎃 — 2024-10-31 — London
- Public talk page: https://london.aitinkerers.org/talks/rsvp_k2jEXYltQMw

## Related Technologies

- [Agentic Pipelines](https://aitinkerers.org/technologies/agentic-pipelines) ([Markdown](https://aitinkerers.org/technologies/agentic-pipelines.md)) — 2 public demos
- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [Generative models](https://aitinkerers.org/technologies/generative-models) ([Markdown](https://aitinkerers.org/technologies/generative-models.md)) — 3 public demos
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) — 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [Keras](https://aitinkerers.org/technologies/keras) ([Markdown](https://aitinkerers.org/technologies/keras.md)) — 74 public demos
- [Llama-2](https://aitinkerers.org/technologies/llama-2) ([Markdown](https://aitinkerers.org/technologies/llama-2.md)) — 227 public demos
- [ONNX](https://aitinkerers.org/technologies/onnx) ([Markdown](https://aitinkerers.org/technologies/onnx.md)) — 83 public demos
- [PaLM 2](https://aitinkerers.org/technologies/palm-2) ([Markdown](https://aitinkerers.org/technologies/palm-2.md)) — 116 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos
- [RoBERTa](https://aitinkerers.org/technologies/roberta) ([Markdown](https://aitinkerers.org/technologies/roberta.md)) — 118 public demos
- [scikit-learn](https://aitinkerers.org/technologies/scikit-learn) ([Markdown](https://aitinkerers.org/technologies/scikit-learn.md)) — 84 public demos
- [TensorFlow](https://aitinkerers.org/technologies/tensorflow) ([Markdown](https://aitinkerers.org/technologies/tensorflow.md)) — 90 public demos