Saturday, August 12, 2023

Postgres as vector db with pgvector extension

 Vectors are the new JSON in PostgreSQL | Jonathan Katz

"Generative AI and all the buzz around it has caused developers to look for convenient ways to store and run queries against the outputs of these systems, with PostgreSQL being a natural choice for a lot of reasons. But even with the hype around generative AI, this is not a new data pattern. Vectors, as a mathematical concept, have been around for hundreds of years. Machine learning has over a half-century worth of research. The array – the fundamental data structure for a vector – is taught in most introductory computer science classes. Even PostgreSQL has had support for vector operations for over 20 years

Vectors are not new, but they’re having a surge in popularity these days. As mentioned earlier, this is due to the newfound accessibility of AI/ML systems, and that the output of these systems are vectors. A common use-case is to build a model on stored data (text, sound, video), convert it to vector format, and then use it for “semantic search.”

pgvector: an open source extension for storing and searching vectors in PostgreSQL

No comments: