DataBooks: Markdown as Semantic Infrastructure

DataBooks are a design pattern that uses Markdown as semantic infrastructure to create self-describing, portable, and machine-processable documents. Unlike standalone data files like raw Turtle or JSON-LD, which lack context and processing instructions, a DataBook combines graph data, prose context, and provenance metadata into a single artifact that humans can read and machines can process.

Core Structure of a DataBook

A DataBook follows a specific pattern within a Markdown document:

• YAML Frontmatter: Carries document metadata, provenance information, and processing instructions.
• Typed Fenced Blocks: Contain the data payloads, such as Turtle or JSON-LD graph data, SPARQL queries, or even AI prompts and manifests.
• Prose Sections: Provide the human-readable documentation and explanation for the data. 

Role in the Semantic Web

DataBooks address a gap in the Semantic Web ecosystem—handling "small data" that does not require a full heavyweight database or triple store. 

• Local Ground Truth: They allow semantic content to travel between different systems without losing meaning, acting as a "holon" where the boundary condition and context travel with the artifact itself.
• LLM Integration: In AI workflows, DataBooks invert the standard model. Instead of treating data as ephemeral context for an LLM, the DataBook becomes the persistent, auditable artifact, while the LLM acts as one of several "transformation engines" used to enrich or process it.
• Auditable Pipelines: They include "process stamps" that record which transformer (AI or human) operated on what inputs at what time, creating a forensic trail for auditing and re-running pipeline stages. 

Key Technologies and Tools

DataBooks leverage established W3C standards to ensure interoperability:

• RDF (Resource Description Framework): Used to represent the data and dependency graphs.
• SPARQL: Allows the contents and dependency manifests of DataBooks to be queried as first-class semantic artifacts.
• Encryption Profiles: Designed to support sensitive data through encrypted fenced blocks that parsers can either decrypt or gracefully skip. 

While not yet a formal specification, DataBooks are implementable today using a combination of Markdown, YAML, and standard RDF toolchains like Apache Jena or RDFLib. They are specifically suited for knowledge work that is currently fragmented, such as AI-assisted ontology development or cross-institutional data integration. 

https://ontologist.substack.com/

 

The Ontologist

DataBooks: Markdown as Semantic Infrastructure

"Something has been missing from the semantic web stack for a long time, and it’s been hiding in plain sight.

The RDF ecosystem has always known how to handle large, persistent, well-indexed knowledge graphs. Triple stores, SPARQL endpoints, federated query — these are mature, well-understood tools for managing graph data at scale. What the ecosystem has never handled well is everything else: the small, contextual, task-specific, ephemeral, or pipeline-stage graph content that makes up the majority of actual knowledge work. The data that doesn’t need a database. The graph that lives for the duration of a process and then needs to be archived, referenced, or passed downstream. The semantic content that a human needs to read and a machine needs to process."

DataBooks, Part II: The Semantic Execution Layer

In Part I of this series, we introduced the DataBook format — a Markdown document that functions simultaneously as human-readable text, a typed data container, and a self-describing semantic artifact. We argued that Markdown, far from being a lightweight presentational format, carries the structural DNA needed to become a genuine semantic infrastructure layer.