# RoBERTa Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/roberta
> Markdown URL: https://aitinkerers.org/technologies/roberta.md
> Technology record last updated: 2026-02-24T08:39:03Z
> Generated: 2026-09-20T13:42:03Z

RoBERTa (Robustly Optimized BERT Pretraining Approach) is a high-performance language model from Facebook AI that significantly outperforms BERT by optimizing the pretraining strategy, not the core architecture.

RoBERTa is a robustly optimized version of the BERT model, developed by researchers at Facebook AI in 2019. The team conducted a replication study, proving BERT was undertrained and could achieve state-of-the-art results with a refined recipe: they removed the Next Sentence Prediction (NSP) objective, implemented dynamic masking, and scaled up training dramatically. Specifically, RoBERTa trained for 500K steps (up from 100K) on a massive 160GB of text data (ten times BERT’s data) using much larger batch sizes (up to 8K). This optimized approach yielded superior performance on major benchmarks like GLUE, RACE, and SQuAD, establishing RoBERTa as a benchmark for subsequent language model development.

- Official technology site: https://arxiv.org/abs/1907.11692
- Public AI Tinkerers demos and talks: 118
- Result page: 1 of 5

## Recent Public Talks and Demos

### [Self-Improving Agents with Honcho](https://nyc.aitinkerers.org/talks/rsvp_u3mcI_u6ioA)

Learn how you can use Honcho to create a stateful agent that learns and improves itself over time. Remove context window and context engineering burdens, let your OpenClaw know what it needs when it needs it.

- Event context: 🦞Demo Night: OpenClaw ft Convex — 2026-02-17 — New York City
- Public talk page: https://nyc.aitinkerers.org/talks/rsvp_u3mcI_u6ioA

### [Stop Cloud Waste: AI Checks Costs Before Deploy in AWS](https://raleigh.aitinkerers.org/talks/rsvp_MX33E7LzJoM)

Ever had your IT team spin up cloud resources that cost way more than expected? I've developed an AI-powered assistant that reviews cloud infrastructure plans before deployment - like having a financial advisor check your shopping cart before checkout. Instead of finding out you spent $10,000 after the fact, teams get instant feedback: "This database is oversized for a dev environment - similar projects in your company use something 5x cheaper" or "You forgot the required cost tags that finance needs for billing." In this 5-minute demo, I'll show: Upload a cloud infrastructure plan (Terraform file) AI analyzes it against your company's cost policies written in plain English Get a breakdown: estimated monthly cost, policy violations, and smart suggestions See how it learns from your organization's past projects to give better recommendations The magic? It understands context - not just "this violates rule 247" but "this looks expensive for what you're building, here's what similar teams did."

- Event context: AI Tinkerers Raleigh Meetup — February 11, 2026 — 2026-02-11 — Raleigh
- Public talk page: https://raleigh.aitinkerers.org/talks/rsvp_MX33E7LzJoM

### [Should You Obey Your AI Reviewer?](https://brussels.aitinkerers.org/talks/rsvp_3tv20yxQ8mQ)

"Code Quality" is a system that posts feedback comments on your pull requests. We'll take a deep dive into how we hide non-determinism caused by LLMs and how we optimize the LLM's accuracy.

- Event context: AI Tinkerers Ghent Meetup - February 11 — 2026-02-11 — Brussels
- Public talk page: https://brussels.aitinkerers.org/talks/rsvp_3tv20yxQ8mQ

### [AI agents as interface layer for mobile apps](https://tiruchirappalli.aitinkerers.org/talks/rsvp_qrJiLeqgTOo)

Mobile interfaces are reaching their limit. More features mean more screens, more taps, and more friction. This talk explores a different direction. AI agents not as assistants bolted onto apps, but as the interface itself. Through a live demo of Kuralit, I show how user intent can bypass traditional UI and directly trigger real actions inside an app. No buttons. No navigation trees. Just intent to execution. This is not about voice for convenience. It is about interface evolution. From visual control to intent-driven software. The goal is to question a simple assumption. If software can understand what a user wants, why does it still wait for clicks?

- Event context: AI Tinkerers Trichy: January Meetup & Live Demos — 2026-01-31 — Tiruchirappalli
- Public talk page: https://tiruchirappalli.aitinkerers.org/talks/rsvp_qrJiLeqgTOo

### [Shifting Out: Using AI to Translate Across the Organization](https://atlanta.aitinkerers.org/talks/rsvp_ws8aA88TM4Y)

I will explores how AI can act as a translation layer across the organization. Not replacing people, but helping teams shift perspective without losing meaning. How to: Translate executive intent into actionable product and engineering scopes Normalize language between product, design, engineering, and marketing Preserve context while shifting between abstraction levels (strategy ↔ tactics ↔ implementation) Reduce “telephone game” loss across async, distributed teams Rather than focusing on prompt tricks or specific models, this talk centers on organizational leverage: where AI creates clarity, where it introduces risk, and how to design workflows that make translation reliable instead of fragile.

- Event context: Co-Co-Code & Cocoa: The AI Tinkerers Atlanta Holiday Meetup — 2025-12-16 — Atlanta
- Public talk page: https://atlanta.aitinkerers.org/talks/rsvp_ws8aA88TM4Y

### [Conversation Games with AI](https://chicago.aitinkerers.org/talks/rsvp_8_C1ufLf42g)

I made an app/game that gives fun little AI-powered voice challenges. I will demo it, talk about my approach to prompting and how the app works, and briefly get on a soapbox why I think you should "lean in to the AI medium" when doing creative design.

- Event context: AI Tinkerers Chicago December Meetup ft Turing — 2025-12-09 — Chicago
- Public talk page: https://chicago.aitinkerers.org/talks/rsvp_8_C1ufLf42g

### [Long running, workflow context aware conversational agent solutions on Temporal](https://bengaluru.aitinkerers.org/talks/rsvp_pAuEWcjhhvA)

We built a long running Temporal workflow based solution to build conversational agents, that allows you to have multi turn conversations, can live for 12 hours at a go, works on suspension/rerun models, has inbuilt retries, error handling and dependency management, and works to trigger complex workflows. We also built a DeepRAG model that allows agents to gain greater context across complex storage options

- Event context: AI Tinkerers - Bengaluru - November meetup — 2025-11-29 — Bengaluru
- Public talk page: https://bengaluru.aitinkerers.org/talks/rsvp_pAuEWcjhhvA

### [classifai.dev - simple, self-improving classification api](https://la.aitinkerers.org/talks/rsvp_1OwmludKh3w)

The simplest possible classification API that any developer can use. Simply provide data you'd like classified, and the classes/labels OR a description of the task. If you provide feedback, the same endpoint starts improving its responses over time.

- Event context: AI Tinkerers LA – October 2025: Ghosts in the Machine w/ Oxen.ai — 2025-10-21 — Los Angeles
- Public talk page: https://la.aitinkerers.org/talks/rsvp_1OwmludKh3w

### [Thinking in Systems with Juggernaut Labs](https://waterloo.aitinkerers.org/talks/rsvp_-kGdH_Qb-bg)

Showcasing how one can create sustainable complex AI systems. Viewers will learn how to leverage a modular framework that we created for builders who want to automate internal business processes or outsource their AI infrastructure for products they are building

- Event context: AI Tinkerers Waterloo — October Meetup — 2025-10-20 — Waterloo
- Public talk page: https://waterloo.aitinkerers.org/talks/rsvp_-kGdH_Qb-bg

### [Personalized Product Discovery with Agentic AI: Beyond Traditional Recommendation Engines](https://singapore.aitinkerers.org/talks/rsvp_TYi5dO1vY58)

E-commerce platforms have long used recommendation engines to boost sales. Yet, these systems often fall short when it comes to true personalization. This talk introduces how Agentic AI can revolutionize recommendations for mobiles, gadgets, accessories, and insurance plans—shifting from static algorithms to adaptive, goal-oriented agents. We’ll discuss how these agents combine customer intent, multi-modal data, and conversational interactions to deliver next-generation recommendations that drive engagement, increase trust, and create measurable business impact.

- Event context: AIT Singapore - AI Agents Showcase - 7th Oct 2025 — 2025-10-07 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_TYi5dO1vY58

### [From Boilerplate to Boardroom: Unlocking Complex Legal & Financial Docs with PaddlePaddle](https://hong-kong.aitinkerers.org/talks/rsvp_SdYj_Cprbfg)

This presentation is a technical deep dive into the engineering choices required to build a robust pipeline for parsing complex legal and financial documents. Using slides with detailed code snippets and comparative outputs, I will walk through our systematic approach. Layout Analysis: We'll begin by analyzing the initial output from Baidu's PP-DocLayout-L on a complex, bilingual document. I'll then walk through the Python code snippets we engineered to handle specific challenges like multi-column layouts and footnote separation. Comparative OCR Analysis: I will present a direct, side-by-side comparison of Baidu's PaddleOCR versus the Tesseract engine on identical legal text. We'll examine the specific scenarios where one outperforms the other, providing a clear-eyed view of their respective strengths and weaknesses. LLM-Augmented Structuring: This is where we go beyond standard tools. I will share our tips and tricks for leveraging Large Language Models (LLMs) such as Baidu's ERNIE to handle the final, most nuanced structuring tasks. You'll see how we used LLMs to interpret ambiguous contexts and transform unstructured phrases into clean, structured data—a task that pure OCR or regex struggles with. Final Structuring Pipeline: To conclude, I will showcase the Python script that integrates these components, transforming the processed text into a final, hierarchically correct JSON object, ready for analysis or downstream tasks.

- Event context: AI Tinkerers - Hong Kong Meetup (August) - Meetup with Baidu PaddlePaddle — 2025-08-22 — Hong Kong
- Public talk page: https://hong-kong.aitinkerers.org/talks/rsvp_SdYj_Cprbfg

### [A Mindless Dialogue: What if two LLMs start debating with eachother about random stuff.](https://hamburg.aitinkerers.org/talks/rsvp_Ip6qa79jbxw)

In this presentation, I will show what happens when two LLMs start descussin various topic with each other. The focus will be on showing my learnings about the memory functions of Large Language Models (LLMs). I will show the underlying code and techniques like offline live text-to-speech functionality. To showcase the conversational abilities of AI through various use cases, including discussing different topics with each other. To provide a detailed explanation of the memory functions present within Large Language Models (LLMs). To demonstrate the coding techniques used to implement memory in LLMs. To introduce live text-to-speech functionality and allow participants to experience AI-generated speech.

- Event context: AI Tinkerers Hamburg #3 - August 14 — 2025-08-14 — Hamburg
- Public talk page: https://hamburg.aitinkerers.org/talks/rsvp_Ip6qa79jbxw

### [Building an AI agent that can help with real world chores](https://seattle.aitinkerers.org/talks/rsvp_OyB4-0xioHE)

Timee is a practical AI concierge that automates real-world tasks by making phone calls on a user's behalf. In the demo, we will show 2 agents. 1st: We'll demo the scheduler agent. From the initial user prompt (e.g., "schedule a haircut for next week") to the call to the final call summary and automated calendar event. 2nd: We will demo the look up agent that can be used for retrieving information from your emails and reach out to businesses on your behalf whether that's for a product return or membership cancellation.

- Event context: Building AI Agents with Google Cloud AI — 2025-07-25 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_OyB4-0xioHE

### [‌AI for Cultural Preservation: Bridging Generative AI and Classical Methods to Decode East Asian Archives‌](https://hong-kong.aitinkerers.org/talks/rsvp_VSPIcIhGtU4)

This talk dives into the technical journey of transforming millions of unstructured historical East Asian records—from genealogies to Qing dynasty bureaucratic texts—into structured, machine-readable and human-readable formats required for modern AI applications. Faced with complex layouts (multi-column text, nested annotations) and degraded materials, we designed a pipeline that merges advanced layout extraction models, domain-tuned OCR for irregular scripts, and generative AI for context-aware text reconstruction. Statistical methods refine outputs to align with historical linguistics, mitigating AI hallucination risks, while custom NER models isolate key entities (names, dates, roles) to convert chaos into clean, searchable databases. By tackling the gap between unstructured archival content and the structured inputs LLMs demand, this work unlocks scalable analysis of cultural patterns—governance, lineage, migration—and offers a blueprint for turning fragile, analog archives into AI-ready datasets. Join us to explore how hybrid AI systems can bridge centuries-old texts with tomorrow’s generative tools, making humanity’s collective memory accessible in the age of machine intelligence. We aim to foster a collaborative dialogue—getting feedback on our technical approaches and challenges while exchanging insights.

- Event context: AI Tinkerers - Hong Kong Meetup (May) - Tipsy Thursday x AI Tinkerers at Hong Kong Science Park — 2025-05-29 — Hong Kong
- Public talk page: https://hong-kong.aitinkerers.org/talks/rsvp_VSPIcIhGtU4

### [End-to-end AI Autograding](https://singapore.aitinkerers.org/talks/rsvp_cecnUyJmsIc)

Many AI autograding tools don't work as expected because Large Language Models are probabilistic. If these tools are not good at grading at this point in time, how can they still be a useful tool for educators. Built by a teacher for teachers, this tool shows the end-to-end pipeline of how an autograder can be built and deployed in the educational setting.

- Event context: AI Tinkerers Singapore: May Meetup - May 21st, 2025 — 2025-05-21 — Singapore
- Public talk page: https://singapore.aitinkerers.org/talks/rsvp_cecnUyJmsIc

### [Deconstructing RAG: Building, Iterating, and Exploring Advanced Patterns](https://milan.aitinkerers.org/talks/rsvp_pNg9pDV0OHk)

This talk dives into the practical construction of Retrieval Augmented Generation (RAG) systems, starting from a foundational "barebones" pipeline. We'll walk through each core component: query rewriting for clarity, efficient vector storage and retrieval with Qdrant, the crucial role of reranking for relevance, and finally, the generation step. Beyond this core, I'll share insights and lessons learned from experimenting with more advanced RAG variations, including Agentic RAG for complex tasks, Hierarchical RAG for handling large document sets, image-based RAG for multimodal understanding, and Graph RAG for leveraging relationships in data. Attendees will gain a clear understanding of how to build their own RAG, make informed design choices, and explore pathways for enhancing its capabilities.

- Event context: AI Tinkerers Milan - May 8, 2025 — 2025-05-08 — Milan
- Public talk page: https://milan.aitinkerers.org/talks/rsvp_pNg9pDV0OHk

### [Agents for Data Engineers in the Customer Data Platform Space](https://seattle.aitinkerers.org/talks/rsvp_XNA6bMwaGgE)

A big opportunity for agents to provide a lot of value is for data engineers that use platforms like Databricks. Databricks is a general platform that you can do almost anything on, but it's a bit like a blank canvas. A major use case for enterprise companies is using it as the primary layer for managing all of their customer data. This typically means using other ETL frameworks or writing lots of scripts. Agents can offer a way to start from the value or use case and help data engineers plan how to implement it, and then do that implementation for you. Instead of generating code, it can generate notebooks, configures various applications, etc. I've been working on a project to bring that to life and create an agent focused on helping data engineers implement customer data use cases on Databricks and with SAAS applications.

- Event context: AI Tinkerers Seattle - April Meetup — 2025-04-25 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_XNA6bMwaGgE

### [AI oncall engineer that actually works](https://seattle.aitinkerers.org/talks/rsvp__NkvtdoBss4)

OncallNinja is an AI powered oncall engineer that can resolve software incidents for teams

- Event context: AI Tinkerers Seattle - April Meetup — 2025-04-25 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp__NkvtdoBss4

### [GRPO - Learning Rust From Cargo Feedback](https://la.aitinkerers.org/talks/rsvp_0xGNqb9udS4)

I am putting together a training pipeline to improve LLMs on low resource languages like rust

- Event context: Feb 24 - LA AI Tinkerers Meetup & Demo — 2025-02-25 — Los Angeles
- Public talk page: https://la.aitinkerers.org/talks/rsvp_0xGNqb9udS4

### [Leveraging Domain-Specific Languages through Large Language Models](https://dublin.aitinkerers.org/talks/rsvp_ajKMXywUIPk)

Can LLM's be used to help tame the complexity of Large Enterprise solutions? Do you want to evolve legacy products and add flexibility whilst reducing the load on product owners, developers and QA engineers? Do you want to allow Subject Matter Experts to apply their knowledge more enhance the product functionality? Eoin outlines an approach that combines Large Language Models with Domain-Specific Languages to take a step closer to this Nirvana of Enterprise Software

- Event context: AI Tinkerers - Dublin Event (February) — 2025-02-24 — Dublin
- Public talk page: https://dublin.aitinkerers.org/talks/rsvp_ajKMXywUIPk

### [Wanderheart](https://seattle.aitinkerers.org/talks/rsvp_RYKnUdavnIU)

"Dungeons & Dragons" with an AI Dungeon Master, built by former Wizards of the Coast game developers.

- Event context: AI Tinkerers Seattle - February 2025 Meetup — 2025-02-22 — Seattle
- Public talk page: https://seattle.aitinkerers.org/talks/rsvp_RYKnUdavnIU

### [CRM AI Agent to handle and respond to customer emails](https://hamburg.aitinkerers.org/talks/rsvp_fVm5EXTsiAY)

Businesses receive a high volume of customer emails daily, ranging from product inquiries to technical support requests. Managing and responding to these efficiently requires significant human interaction and effort. This AI agent automates email management by classifying inquiries, retrieving relevant knowledge, and generating personalized responses. It intelligently decides whether to reply instantly, escalate the issue by creating a ticket, or provide troubleshooting guidance—ensuring faster response times and improved customer satisfaction. By leveraging LLMs, vector databases, and automation, this AI agent enhances email handling, reducing manual workload while maintaining high-quality customer interactions. Furthermore, I conducted this project as a learning exercise and proof of concept for an AI-driven CRM automation application.

- Event context: AI Tinkerers Hamburg #2 - February 20 — 2025-02-20 — Hamburg
- Public talk page: https://hamburg.aitinkerers.org/talks/rsvp_fVm5EXTsiAY

### [STRIDE, an automated development framework that leverages AI](https://austin.aitinkerers.org/talks/rsvp_FZoH3jZAlfM)

Introducing STRIDE, an automated development framework that leverages Large Language Models (LLMs) to revolutionize modern software development. By automating the phases of Specification, Testing, Refinement, Integration, Deployment, and Evaluation, STRIDE accelerates the creation of robust and scalable applications. This framework empowers teams to efficiently develop projects of any size, from demos and Proofs of Concept (PoCs) to full-scale applications, ensuring rapid and reliable delivery.

- Event context: AI Tinkerers February Meetup: Community AI Demos - North Austin — 2025-02-14 — Austin
- Public talk page: https://austin.aitinkerers.org/talks/rsvp_FZoH3jZAlfM

### [Transformer Lab](https://waterloo.aitinkerers.org/talks/rsvp_0KXArJU6wf4)

I am planning to demo the steps to quickly train a model on your local computer using Transformer Lab. Transformer Lab is an open-source, cross-platform application that lets you run, train, tune, eval, RAG and quantize LLMs on your local machine, regardless of what your hardware setup is.

- Event context: AI Tinkerers - Waterloo February Meetup — 2025-02-11 — Waterloo
- Public talk page: https://waterloo.aitinkerers.org/talks/rsvp_0KXArJU6wf4

## Related Technologies

- [BERT](https://aitinkerers.org/technologies/bert) ([Markdown](https://aitinkerers.org/technologies/bert.md)) — 179 public demos
- [GPT-3](https://aitinkerers.org/technologies/gpt-3) ([Markdown](https://aitinkerers.org/technologies/gpt-3.md)) — 191 public demos
- [GPT-4](https://aitinkerers.org/technologies/gpt-4) ([Markdown](https://aitinkerers.org/technologies/gpt-4.md)) — 529 public demos
- [BLOOM](https://aitinkerers.org/technologies/bloom) ([Markdown](https://aitinkerers.org/technologies/bloom.md)) — 115 public demos
- [Llama-2](https://aitinkerers.org/technologies/llama-2) ([Markdown](https://aitinkerers.org/technologies/llama-2.md)) — 227 public demos
- [PaLM 2](https://aitinkerers.org/technologies/palm-2) ([Markdown](https://aitinkerers.org/technologies/palm-2.md)) — 116 public demos
- [RAG](https://aitinkerers.org/technologies/rag) ([Markdown](https://aitinkerers.org/technologies/rag.md)) — 147 public demos
- [scikit-learn](https://aitinkerers.org/technologies/scikit-learn) ([Markdown](https://aitinkerers.org/technologies/scikit-learn.md)) — 84 public demos
- [TensorFlow](https://aitinkerers.org/technologies/tensorflow) ([Markdown](https://aitinkerers.org/technologies/tensorflow.md)) — 90 public demos
- [Keras](https://aitinkerers.org/technologies/keras) ([Markdown](https://aitinkerers.org/technologies/keras.md)) — 74 public demos
- [ONNX](https://aitinkerers.org/technologies/onnx) ([Markdown](https://aitinkerers.org/technologies/onnx.md)) — 83 public demos
- [PyTorch](https://aitinkerers.org/technologies/pytorch) ([Markdown](https://aitinkerers.org/technologies/pytorch.md)) — 273 public demos
- [Python](https://aitinkerers.org/technologies/python) ([Markdown](https://aitinkerers.org/technologies/python.md)) — 662 public demos
- [Generative AI](https://aitinkerers.org/technologies/generative-ai) ([Markdown](https://aitinkerers.org/technologies/generative-ai.md)) — 45 public demos
- [Large Language Models](https://aitinkerers.org/technologies/large-language-models) ([Markdown](https://aitinkerers.org/technologies/large-language-models.md)) — 8 public demos
- [Prompt Engineering](https://aitinkerers.org/technologies/prompt-engineering) ([Markdown](https://aitinkerers.org/technologies/prompt-engineering.md)) — 28 public demos
- [Fine-tuning](https://aitinkerers.org/technologies/fine-tuning) ([Markdown](https://aitinkerers.org/technologies/fine-tuning.md)) — 20 public demos
- [AI agents](https://aitinkerers.org/technologies/ai-agents) ([Markdown](https://aitinkerers.org/technologies/ai-agents.md)) — 35 public demos

## More Results

- Next: https://aitinkerers.org/technologies/roberta.md?page=2