cat3rpillar/derkino-qa-prototype-v1

Project: Fine-tuning a Cinema QA Prototype

Executive Summary

This notebook documents the process of fine-tuning a google/flan-t5-small language model on a custom dataset of movie facts. The goal was to determine if a small, fine-tuned LLM could act as a reliable, fact-based "expert" on cinema.

Conclusion & Key Findings

The experiment was a success, concluding that this approach is not the correct one for this specific problem.

While the model successfully learned the style of answering questions, it failed to reliably memorize the specific factual data from the training set. It consistently produced "hallucinated" answers that were plausible in format but factually incorrect.

This demonstrates a core principle of applied AI: Language models are not databases. Fine-tuning excels at teaching a model a new skill or style, but it is not an effective method for memorizing a large corpus of specific facts.

The correct architectural solution for building a fact-based expert system is Retrieval-Augmented Generation (RAG), which uses a database for fact retrieval and an LLM for natural language generation. This experiment serves as the successful research spike that validates this strategic direction for future work on the derkino project.

---

Project Details

• Base Model: google/flan-t5-small
• Dataset Used: cat3rpillar/derkino-expert-data
• Model Architecture: Encoder-Decoder Transformer
• Training Architecture: Fine-Tuning

Intended Use

This model is an experimental prototype and a research artifact. It is not intended for production use as it is not factually reliable. Its primary purpose is to demonstrate the outcome of the fine-tuning process documented in the associated Colab notebook.