LLM Dynamic Structured Records

An approach that enables LLMs to dynamically create, manipulate, and retrieve structured records in a NoSQL database. This approach serves as an alternative to vector-based RAG, creating a data store that’s both human and model-readable. Demoed in a mobile app for tracking daily records (e.g., cat medication, exercise, expenses) to illustrate practical applications.
The core mechanism I’m focusing on is prompting for structured Tool/Function completions to do CRUD operations on structured records - to capture, remember, retrieve, and query against information the user wants to reference later. The schema / structure of the records themselves is also frictionlessly created / edited by the model. Both the table structures and the records themselves are stored in a NoSQL database. The “magic” is in actively managing the system prompt on the chat chain to provide full context and dynamically generate the functions available as tools for each completion. This “dynamic but structured” approach uses JSON Schema to define, communicate, and validate functions.
A technical challenge was dynamically generating and validating table / context-specific functions. I did this by embracing JSON Schema in the table definition as well as the functions presented to the LLM. I’ll focus my demo on the prompt, the Firestore database, and the function building. This is a Flutter app that runs on iOS and Android.
I’m very excited to share this approach with the other Tinkerers - shown as a fully working implementation that can be tried out, as well as access to all the code.