Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley - YouTube
While LLM-based agents excel at probabilistic language generation, they often struggle with exact domain rules, state management, and logical boundaries—leading to hallucinations, brittle tools, and invalid operations (e.g., issuing duplicate refunds or invalid payouts). Frank Coyle argues that pure prompt engineering or system instructions are insufficient to solve these edge cases. Instead, agentic architectures require a formal ontology layer—a structured, symbolic representation of a domain that acts as logical guardrails outside the model.
Key Takeaways
1. The Core Problem: Probabilistic Models vs. Deterministic Realities
LLM Limitations: Large language models predict the next likely token, making them inherently probabilistic. They lack a deterministic understanding of state, explicit constraints, and business logic.
Failure Modes: Common agent errors—such as refunding an order twice or returning ambiguous states like "probably shipped"—stem from attempting to enforce rigid domain logic solely through natural language prompts.
2. What is an Ontology in AI?
An ontology provides a explicit, formal framework defining:
Classes/Entities: The core concepts (e.g., Customer, Order, Product, Refund).
Properties: Attributes describing entities.
Relationships: How entities link together (e.g., Customer
PLACEDOrder).Constraints: Business rules that cannot be violated (e.g., An order cannot be refunded if it has already been refunded).
Connectionist Meets Symbolic: Combining LLMs (connectionist) with ontologies (symbolic) creates a hybrid AI architecture capable of both flexible language understanding and strict rule enforcement.
3. Building an Organizational Ontology
To construct a robust ontology, Coyle recommends a meet-in-the-middle approach:
Top-Down (Expert Systems 2.0): Domain experts explicitly define entities, relations, properties, and constraints.
Bottom-Up (Data Mining & Extraction): Automated tools (NER, LLM mining) extract terms and relationships directly from existing documentation, logs, and user reviews.
4. Why Ontologies Outperform Raw Prompts
Separation of Logic & Execution: Keeping domain knowledge in an external ontology keeps tool harnesses thin and modular, avoiding bloated, fragile prompts.
Predictable Handoffs: Ontologies standardize data structures and handoff schemas when multiple agents coordinate across complex workflows.
Deterministic Guardrails: The ontology acts as a logical validation layer prior to tool execution, catching illegal state transitions before an agent makes irreversible real-world API calls.
Frank Coyle (@coyle_frankp) / X
0:00 - Intro and an educator's philosophy 2:21 - Two lineages: agents and ontologies 4:04 - Neurosymbolic AI: guardrails around a probabilistic model 5:23 - What an ontology actually is 6:14 - Building one, and the expert systems era 7:55 - Reusing existing taxonomies 9:12 - RDFS and OWL: inference and constraints 12:12 - Agents, loops, and how they break 14:22 - A Claude tool use loop with an ontology validator 17:47 - Pydantic at the door, ontology at the ledger 18:52 - The errors an ontology catches that English cannot
No comments:
Post a Comment