AI: tensors vs vectors

What is a Tensor? An Animated Introduction! - YouTube

3Blue1Brown - YouTube

Vespa AI and Surpassing the Limits of Vector Search - Software Engineering Daily

Vespa

RAG at Scale: Why Tensors Outperform Vectors in Real-World AI | Vespa Blog

The blog post from Vespa.ai argues that while vector databases are the current standard for AI retrieval, they are becoming insufficient for complex, real-world RAG (Retrieval-Augmented Generation) applications. The author proposes tensors as a more powerful alternative because they preserve data structure and relationships that flat vectors lose.

---

The Problem with Vector-Only Systems

The post identifies several bottlenecks when using standard vector databases at scale:

• Lack of Hybrid Support: Difficulty matching exact phrases, boolean logic, or keyword expressions alongside semantic search.

• Weak Logic Integration: Hard to combine unstructured content with structured metadata (like price, date, or category).

• External Latency: Relying on external re-rankers or ML inference services introduces complexity and "fragile" pipelines.

• Flattened Context: Vectors represent data as a single embedding, losing sequence, position, and spatial relationships.

---

Why Tensors are Superior

Tensors act as a generalization of vectors (a vector is a 1D tensor), allowing for multidimensional data representation.

Feature	Vector Representation	Tensor Representation
Data Structure	Flattened (single embedding)	Structured (multi-embedding)
Dimensions	Anonymous/Implicit	Named (e.g., "time", "patch", "batch")
Capability	Semantic similarity only	Reasoning, ranking, and spatial context
Use Case	Basic search	ColBERT, ColPali, and multimodal search

---

Vespa’s Tensor Framework Principles

Vespa uses a specific "tensor formalism" designed for high-performance production environments:

1. Unified Dense & Sparse Support: Allows mixing fixed-size numeric vectors (dense) with symbolic tags or categories (sparse) in a single model.

2. Named Dimensions: Replaces confusing axis indices (axis=0) with semantic labels, making code safer and easier to debug.

3. Minimal & Composable Ops: A concise mathematical foundation that can express complex logic without a bloated API.

The Bottom Line

The future of AI applications requires more than just finding "similar" data; it requires reasoning. If vectors help a system retrieve information, tensors help the system understand and process it natively within the database engine.

For those interested in implementation, the post points toward The RAG Blueprint for building systems using this framework.