Turning Claude into Postgres so I can raise a Series A
- The Concept: To capitalize on AI database hype, the author created a experimental database proxy using
(a Python Postgres proxy) to convert PostgreSQL wire protocol queries into prompts for Claude.Buena Vista - Storage & Encoding: Because standard binary database formats don't work well with LLMs, data was persisted to disk using a JSON array format separated by pipe characters (
PAGE | ... | ITEM | ...). - RelCache & Catalogs: To avoid circular references when locating database catalogs on disk, a simplified RelCache was implemented in Python to inject core catalog locations directly into Claude's prompt context.
- Performance & Query Planning:
- Claude managed basic SQL operations,
CREATE TABLE,INSERT, and simple point queries on 5,000 auto-generated rows. - Although it created B-Tree indexes, Claude initially cheated by performing sequential table scans until prompted with explicit instructions (and polite encouragement/emojis) to actually traverse the B-tree files.
- Benchmarking & Cost:
- Tested on Benchbase's Twitter benchmark at a small 0.05 scale, the system processed 4 transactions in 120 seconds (~0.033 TPS), making it about 200,000x slower than a standard PostgreSQL instance.
- Each transaction cost roughly $0.26 in API tokens, with individual simple queries taking ~10 seconds and $0.03.
- Conclusion: While functional for basic queries, using an LLM directly as a database storage/query engine remains wildly impractical, expensive, and slow. The open-source code is published on GitHub as
.claudegres
Turning Claude into Postgres so I can raise a Series A" from ByteofDev, is indeed a brilliant piece of satire[1][2]. It pokes fun at the current tech zeitgeist where investors throw money at anything combining "AI" and "databases," even if the implementation (like using Claude's context window as a literal filesystem) is objectively terrible for performance[1][2].
1. Semantic Caching (The literal "AI Cache Layer")
How it works: If User A asks an AI API, "How do I reset my password?" and User B asks, "What are the steps to recover my account?", a semantic cache converts these queries into vector embeddings and recognizes they mean the same thing[5][6]. It intercepts the API call and instantly returns the cached answer for User B without ever hitting the expensive underlying LLM or database API[5][6]. Real-world Tools: Redis / Valkey (often used as the backend for this), GPTCache, and API gateways like the Kong AI Semantic Cache plugin all provide this exact capability[4][7][8].
2. AI Gateways and API Proxies
How it works: These tools act as a universal translation and routing layer. They handle semantic caching[5], enforce data schemas (making sure the AI formats the API request correctly), route requests to the fastest/cheapest models, and strip out sensitive PII before a query hits an external API[8]. Real-world Tools: LiteLLM, Bifrost, Portkey, and Helicone[5][8]. They proxy API access, monitor performance, and provide a buffer between raw LLMs and your production endpoints[5][8].
3. AI-Native Databases (Treating AI Models like Tables)
MindsDB: This is an open-source federated query engine for AI[9]. It connects to dozens of different standard databases (Postgres, MySQL, Snowflake) and APIs, but allows you to query them alongside AI models using standard SQL[10]. For example, you can write a single SQL query that JOINs a standard Postgres table of customer reviews with an OpenAI model that predicts the sentiment of each review in real-time. The AI is literally treated as a virtual database table.
4. AI Context Engines & Semantic Layers
How it works: An "AI Context Layer" sits directly on top of the database. When an AI agent needs data, it doesn't write raw SQL. Instead, it queries the semantic layer, which translates the AI's natural language request into safe, pre-approved, highly optimized database queries[11][12]. Real-world Tools: Wren AI (an open-source context engine for AI agents)[11], Cube (a semantic layer adding AI natural language capabilities)[12], and TiDB (which markets itself as an AI application data layer combining vectors and relational data)[13].
No comments:
Post a Comment