Vecs is a new Python library for managing embeddings in your Postgres database with the pgvector extension.
It handles:
- Creating and indexing tables
- Querying vectors by cosine distance, l2 distance, and max inner dot product
- Filtering based on user-defined metadata
Our goal for vecs is to provide an interface that lets Postgres + pgvector look and feel like a dedicated vector store. It works with any Postgres database (or platform) that supports pgvector.
It was designed with ease-of-use, interactivity, and exploratory data analysis in mind, but works equally well as a search workhorse.
If you're interested in the nuts and bolts of what's going on, it's trivial to drop into the SQL layer and see what's happening. Alternatively, folks who don't want to know what's happening in the database, don't need to care.
Vecs makes it easy to create a collection (table) and insert a few records - just 5 lines of code.
Connecting#
_10
import vecs
_10
_10
DB_CONNECTION = "postgresql://<user>:<password>@<host>:<port>/<db_name>"
_10
_10
# create vector store client
_10
vx = vecs.create_client(DB_CONNECTION)
_10
_10
# create a collection of vectors with 3 dimensions
_10
docs = vx.get_or_create_collection(name="docs", dimension=3)
The get_or_create_collection call sets up a table in the Postgres database specified by DB_CONNECTION in a schema named vecs with the user defined name docs.
Or, more specifically:
_10
create table vecs.docs (
_10
id text primary key,
_10
vec vector(3) not null,
_10
metadata jsonb not null default '{}'::jsonb
_10
);
Insert/Update#
We can insert a few records in that new SQL table/vecs collection using Collection.upsert.
_10
# add records to the collection
_10
docs.upsert(
_10
vectors=[
_10
(
_10
"vec0", # the records user defined identifier
_10
[0.1, 0.2, 0.3], # the vector. A list or np.array
_10
{"year": 1973} # associated metadata
_10
)
_10
]
_10
)
which will add the records to our table if the id "vec0" does not exist, or updates the existing record if it does exist.
Query#
You can query a vecs collection at any time without an index, but it's a best practices to create an index on your collection after inserting data.
Where index optionally takes an argument for the distance measure to index.
Finally, we can search the collection for similar vectors using the query method:
_10
docs.query(
_10
query_vector=[0.10,0.21,0.29], # required
_10
limit=1, # (optional) number of records to return
_10
filters={"year": {"$eq": 1973}}, # (optional) metadata filters
_10
measure="cosine_distance", # (optional) distance measure to use
_10
include_value=False, # (optional) should distance measure values be returned?
_10
include_metadata=False, # (optional) should record metadata be returned?
_10
)
Which returns:
_10
[("vec1", 0.000697, {"year": 1973})]
Since all metadata is stored in a jsonb column, there's a lightweight but flexible DSL wrapped around it for filtering.
When you're done, disconnect with:
And 90% of the time, that minimal interface is all you'll need to touch.
For more in-depth information about vecs, checkout the API Quickstart,
celebrity look-alike demo,
or OpenAI integration example
As usual, if you combine supabase/vecs with the rest of Supabase, you get more than the sum of the parts. Once you're happy with your vecs collection, you can make it accessible to your front-end through a supabase client library by exposing the collection as a view in your public schema.
For example, you could create a view
_10
create view public.docs as
_10
select
_10
id,
_10
embedding,
_10
metadata, # Expose the metadata as JSON
_10
(metadata->>'url')::text as url # Extract the URL as a string
_10
from
_10
vecs.docs
And then access it with the supabase-js client library within your applications:
_10
const { data, error } = await supabase
_10
.from('docs')
_10
.select('id, embedding, metadata')
_10
.eq('url', '/hello-world')
For more deployment options, including enterprise scalable architecture, check out the engineering for scale guide.
Currently, vecs is unopinionated about where vectors come from or how they're produced. While there will always be a need for generic vector
storage and querying, it's becoming clear that text and image vectorization make up +95% of usage. That gives us the opportunity to
streamline those workflows for users.
One option we're exploring is to optionally assign transformation pipelines to collections along the lines of:
_14
# This is mock code only, not currently functional
_14
_14
docs: Collection =vx.get_or_create_collection(
_14
docs='docs',
_14
dimension=512,
_14
transform = TextPreprocessor( # this is new
_14
model="sentence-transformers/all-Mini-L6-v2"
_14
)
_14
)
_14
_14
docs.upsert([
_14
("id_0", "# Some markdown", {}),
_14
("id_1", "# Some more markdown", {})
_14
])
so users can choose to work with their preferred media type without ever thinking about vectors.
Another direction we're considering is adding an async client to avoid blocking when waiting on the database or network i.e.
_10
# This is mock code only, not currently functional
_10
_10
await docs.upsert([
_10
("id_0", [0.1, 0.2, 0.3], {}),
_10
])
Both possibilities are still up for debate. If you have view on either, feel free to weigh in on the Feature Request: Preprocessing Transform and Feature Request: Async Client GitHub issues.
- Source code: github.com/supabase/vecs
- Vecs Docs: supabase.github.io/vecs/
- Supabase Vector Toolkit: supabase.com/docs/guides/ai