Monday, September 09, 2024

Postgres Full Text Search vs Elasticsearch

 Full Text Search over Postgres: Elasticsearch vs. Alternatives - ParadeDB

What is Full Text Search (FTS)?

Full text search is a technique that finds entries in a collection of text based on the presence of specific keywords and phrases. Most search engines like Elasticsearch use the BM25 algorithm to rank search results. BM25 considers how often a term appears and how unique that term is across all documents.

Full text search is different from similarity search, also known as vector search, which searches for and ranks results by semantic meaning. Many modern applications use a combination of full text and similarity search. This practice is called hybrid search and can yield more accurate results.


Postgres FTS is a native functionality available to all Postgres databases. It leverages the tsvector data type, which stores text as searchable tokens, and the GIN index, which improves search speeds.


ParadeDB is a full text search engine built for Postgres. Powered by an extension called pg_search, ParadeDB embeds Tantivy, a Rust-based Lucene alternative, inside Postgres. Like native Postgres FTS, ParadeDB plugs into any existing, self-managed Postgres database with no additional infrastructure. Like Elasticsearch, ParadeDB provides the capabilities of an advanced full text search engine.


94X Faster than Postgres
ParadeDB brings column-oriented storage and vectorized query execution to Postgres tables. Users can choose between row and column-oriented storage at table creation time.


AI: RAG app with Postgres DB

 Building a RAG application with GitHub Models and Postgres FROM SCRATCH - YouTube
@Visual Studio Code

Retrieval Augmented Generation (RAG) is a powerful technique to make your AI models even smarter - but how can you leverage it in your own applications? In this video, Pamela Fox breaks down the basics of RAG and shows you how to build your very own RAG app from scratch.

saving embeddings with PGVector extension to BD, from Python
with Code Spaces VSCode in web browser

GitHub - pamelafox/pgvector-playground: A dev container for using PostgreSQL + pgvector in Python, with SQLAlchemy, SQLModel, psycopg2, and asyncpg examples. @GitHub

Comparing main...demochanges · pamelafox/pgvector-playground · GitHub

Pamela Fox | LinkedIn


Intro to GraphRAG - YouTube @ Microsoft Reactor


What Is Retrieval-Augmented Generation aka RAG | NVIDIA Blogs

What is RAG? - Retrieval-Augmented Generation AI Explained - AWS


PostgreSQL as a Vector Database: Create, Store, and Query OpenAI Embeddings With pgvector