After a winter of focused development, we are excited to announce that PlanetScale's support for vector search and storage is GA. Since the open beta began, we have doubled query performance, improved memory efficiency eight times, and focused on robustness to ensure vector support is as solid as every other data type MySQL supports. We are especially proud of the scalability of vector indexes, which now perform well even when they are 6x larger than available memory.
It's a real relational index
We have made vectors a first-class part of MySQL, meaning that writes and queries work like a normal RDBMS. Store your vector data in a column in your existing keyspace. Build an index with an ALTER or CREATE VECTOR INDEX statement. Write SELECT statements with JOIN and WHERE clauses like any other data. Wrap your writes in transactions, and expect the data to be visible as soon as the transaction commits — or to roll back if you ask it to. Your index will remain performant as you add and remove rows, without any need for periodic rebuilds.
It's a PlanetScale database
All the PlanetScale features you rely on work with vectors, too. Create a branch. Add a vector index. Open a deploy request and merge the index into your production branch. Revert it if you change your mind. Query your vector index on other replicas, even in other regions. Sleep easy knowing that your vector data is included in scheduled backups.
It's a good vector index
We didn't just get the relational and operational aspects right. We also built advanced vector-index features to satisfy a variety of embeddings and use cases. An index can rank vectors by Euclidean (L2), inner product, or cosine distance. It can store any vector up to 16,383 dimensions. It supports both fixed and product quantization. Fixed quantization down to one bit per field is crazy fast, or just crazy, depending on your needs.
Indexes should be bigger than RAM
One of the biggest things that sets PlanetScale vector support apart is the ability to use indexes larger than RAM. We are the first to incorporate an index based on SPANN (Space-Partitioned Approximate Nearest Neighbors) into an RDBMS. SPANN is a hybrid index design that combines the best aspects of partition-based and tree-based indexes. In SPANN, vectors are assigned to small partitions, which are stored in hidden InnoDB tables. One vector from each partition, around 20% of the index, is stored in a tree structure in memory, enabling the index to quickly identify which partitions are relevant to a query. Only the tree and a small, fixed number of relevant partitions need to be in memory when building and querying the index, allowing efficient operation with vector data up to 6x bigger than available memory. Using vector indexes on PlanetScale Metal ensures that loading vector partitions from InnoDB to answer queries will be as fast as possible.
How to enable vector support
To get started with vectors, go to the "Branches" page for any database. Click on a branch you want to add vectors to, and click on the small gear icon underneath the "Connect" button on the right for that branch's settings. Click the toggle next to "Enable vectors." Follow the examples in the vectors documentation to create a vector column, add data, and create and query an index.
More resources
To learn more about vector embeddings, check out our YouTube video:
You can check out the following documentation:
- PlanetScale vectors overview
- Vector database terminology and concepts
- Common use cases for vector search
- PlanetScale vector usage with ORMs
- Vector type and index reference
Vector support is ready for use in production today. As you build with vectors in your database, we welcome feedback to help us continually enhance performance and usability. You can submit a support ticket to relay any feedback or issues. We also have a vectors channel in our Discord where you can ask questions, share feedback, or chat about use cases.