About the Author: Kalin Arsov and Jags Ramnarayan

By ,
Kalin Arsov
Jags Ramnarayan

We’re excited to announce that we’ve developed and open-sourced an integration with the LlamaIndex, a leading framework for building generative AI applications. Along with MariaDB Vector (introduced in version 11.6 Tech-preview), and, now available in SkySQL, this integration allows customers to seamlessly incorporate vector search capabilities into their AI solutions such as semantic search and recommendation systems.

In this post, we’ll walk through the MariaDB features that enable seamless vector support. We’ll then explore how you can leverage this to build an effective Retrieval-Augmented Generation (RAG) pipeline using LlamaIndex.

Semantic Search with MariaDB Vector

You’re probably familiar with traditional keyword-based full-text search available in MariaDB/MySQL, where finding results depends on matching exact keywords. But what if you need something more nuanced? Enter vector search. Rather than exact matches, vector stores use similarity to return items that are conceptually related to a query, even if they don’t share identical terms.

Imagine you have a database filled with product reviews, such as: “This laptop is a beast! The M2 chip is incredibly powerful, and the battery life is amazing.”

Now, instead of searching for exact words, you can use a vector store to find laptops that received positive sentiment — capturing not just the words but their meaning. This opens up new possibilities for semantic search, recommendation engines, and intelligent discovery in your applications.

Refining Results with LLMs

While the vector store can rapidly surface semantically similar results, the nature of vector searches means the results are approximate—you might still end up with a large set of results that need further refinement. Suppose you want to go deeper: “Show me the top 5 products with negative sentiment and the key reasons for those ratings.” Now you need more than just a vector search—you need an LLM capable of generating precise, context-aware answers.

Here’s how it works:

  1. Generate embeddings: Convert your product reviews into high-dimensional vectors, also called embeddings, using models from HuggingFace or OpenAI. Curious about what embeddings are and how they work? You can explore more about the underlying concepts here.
  2. Store in a vector database: Save these embeddings in a vector DB like MariaDB for fast semantic search.
  3. Combine search with LLMs: For user queries, perform a similarity search in the vector store and pass the results as “context” to the LLM. The LLM then generates a refined, targeted response, such as extracting the most relevant sentiments or insights from the reviews.

LlamaIndex simplifies this entire pipeline, orchestrating the process of embedding generation, vector search, and LLM-powered synthesis into one streamlined workflow.

Why MariaDB Vector? It is already raining Vector DBs

Why add complexity by integrating another vector store when you can manage vector embeddings right alongside your existing data? With your data and vector embeddings in MariaDB managed by SkySQL, you get seamless integration—your embeddings are stored, scaled, and secured within the same database as the rest of your data. This approach ensures data consistency, reduces operational overhead, and is often far more cost-effective than relying on external vector databases.

Plus, MariaDB Vector is engineered for performance— the ANN benchmark results highlight that it is not only fast but efficient with its resource usage. For more details, visit the full blog here.

By keeping everything within SkySQL, you gain the advantage of lower latency, auto-scaling, simplified security management, and the confidence that your data and embeddings will always stay in sync.

A peek at the SQL extensions to support Vector operations

Using these vector capabilities is simple but powerful. You define vectors using the VECTOR INDEX in your CREATE TABLE statement, letting you build indexes that capture the semantic meaning of data.

Here’s how to set it up:

CREATE TABLE product_reviews (
   product_name VARCHAR(255),
   product_id INT,
   review TEXT,
   review_embedding BLOB,
   VECTOR INDEX (review_embedding)  -- Define the vector index
   );

In this example, the review_embedding column stores the semantic meaning of the review column as high-dimensional vectors. These vectors hold much more than just words—they capture the intent, sentiment, and context behind each review.

But how do you generate these embeddings from your text data? Let’s take a practical approach using LlamaIndex with a model from HuggingFace executed locally in Python:

# Make sure to install the dependences first:
# pip install llama-index llama-index-embeddings-huggingface
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

# Load a pre-trained model for text embeddings
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
embeddings = embed_model.get_text_embedding("This laptop is a beast! ....")

# Insert the embeddings into MariaDB
cursor = conn.cursor()
cursor.execute("INSERT INTO product_reviews (product_name, product_id, review, review_embedding) VALUES (%s, %s, %s, Vec_FromText(%s))",
(product_name, product_id, review, embeddings))

This Python snippet generates a review’s embedding using HuggingFace and seamlessly inserts it into MariaDB. The embedding, now a BLOB, becomes part of your table’s structure, enriching the data with its semantic layer.

Once the embeddings are in place, querying them becomes straightforward with MariaDB’s built-in VEC_Distance function:

-- Example: Find top 5 products with positive sentiment
SELECT * FROM product_reviews
  ORDER BY VEC_Distance(review_embedding, ) 
  LIMIT 5;

MariaDB uses the Modified HNSW algorithm for vector searches, defaulting to Euclidean distance for fast and efficient similarity searches. With this, your database is now more than just a collection of rows—it’s a rich, semantically aware system that can surface insights traditional searches might miss.

Launch on Google Colab

In this demo, we’ll show how to integrate MariaDB Vector with OpenAI’s LLM using LlamaIndex to build a smart, context-driven AI application.

We begin by launching MariaDB via SkySQL, where you’ll get access to vector capabilities in a cloud environment.

The following snippets highlight key parts of the code, guiding you through how to set up and connect your services. For the full implementation, you can jump into the Google Colab notebook.

1) Launch MariaDB in SkySQL using the API

Alternatively, you can visit app.skysql.com and launch using the portal. 

# Headers for the API requests
headers = {
    'Content-Type': 'application/json',
    'X-API-Key': API_KEY
}

""" Launch a new SkySQL DB using API """
# You can change your cloud provider, instance type and storage if you like. 
def launch_skysql_db():
    url = "https://api.skysql.com/provisioning/v1/services"
    payload = {
        "service_type": "transactional",
        "topology": "es-single",
        "version": "11.6.0-1-1", # Vector search is available only in this version
        "provider": "gcp",
        "region": "us-central1",
        "architecture": "amd64",
        "size": "sky-2x8",
        "storage": 100,
        "nodes": 1,
        "name": f"{SKYDB_SERVICE_NAME}",
        "ssl_enabled": True,
        "allow_list": [
            {
                "comment": "The allowed IP address",
                "ip": f"{CLIENT_IP}/32"
            }
        ]
    }
    try:
        response = requests.post(url, headers=headers, data=json.dumps(payload))
        response.raise_for_status()
        return response
    except requests.exceptions.RequestException as e:
        print(f"Failed to launch SkySQL DB service: {e}")
        return None

2) Populate the DB with some ‘Product reviews’

""" Populate Product reviews in DB """

from llama_index.core.schema import Document

documents = []

def create_skyservice_and_populate():
  """
  Creates a schema named 'vector_test' and populates a table named 'product_reviews'
  with specific laptop reviews.
  """
  global documents

  conn = launch_and_get_dbconn() ## Code in colab notebook
  cursor = conn.cursor()

  try:
    cursor.execute("CREATE SCHEMA IF NOT EXISTS vector_test")
    cursor.execute("USE vector_test")
    cursor.execute("CREATE TABLE IF NOT EXISTS product_reviews" +  
      " (product_name VARCHAR(255), product_id INT, review TEXT)")

    # Specific laptop reviews
    reviews = [
      ("Apple MacBook Pro M2", 1, "This thing is a beast! The M2 chip is crazy fast, and the battery lasts forever. The screen is gorgeous, and the keyboard is super comfortable. It's a bit pricey, but totally worth it for the performance and build quality."),
      ("Dell XPS 13", 2, "I love my XPS 13. It's sleek and powerful, and the screen is amazing. The only downside is the battery life could be a bit better, especially when running demanding tasks."),
      ("HP Spectre x360", 3, "I'm so glad I got this 2-in-1 laptop. It's perfect for work and play. The design is sleek, and it's really versatile. The battery life could be better, though."),
      ("Microsoft Surface Laptop Studio", 4, "This is the ultimate laptop for creatives. The touchscreen and stylus make it so easy to work on design projects. It's a bit heavy, but the performance is top-notch."),
      ("Lenovo ThinkPad X1 Carbon", 5, "I'm a big fan of ThinkPads, and this one doesn't disappoint. It's super durable, the keyboard is amazing, and the battery lasts forever. It's a bit pricey, but it's worth it for the quality."),
      ("Acer Predator Helios 300", 6, "This is a great gaming laptop for the price. It runs all the latest games smoothly, and the keyboard is comfortable. The only downside is it can get really loud and hot when gaming."),
      ("Razer Blade 15", 7, "I love the sleek design of this laptop, and it's a powerhouse. The screen is amazing, and the keyboard is perfect for gaming. It's a bit expensive, but it's worth it for the performance."),
      ("Asus ZenBook 14 OLED", 8, "This laptop is a dream. The OLED screen is stunning, and it's really lightweight. The battery life could be better, but it's still a great laptop."),
      ("LG Gram", 9, "I was surprised by how light this laptop is! It's perfect for traveling. The performance is decent, but it's not the fastest laptop I've used."),
      ("Huawei MateBook X Pro", 10, "I love the design of this laptop, but it's a bit hard to find in the US. The screen is great, and the performance is good. The battery life could be better.")
    ]

    cursor.executemany("INSERT INTO product_reviews " + 
      "(product_name, product_id, review) VALUES (%s, %s, %s)", reviews)
    conn.commit()
    print("Schema and table created successfully.")

    cursor.execute("SELECT product_name, product_id, review FROM product_reviews")
    reviews = [f"Product: {product_name}, ID: {product_id}, Review: {review}" for (product_name, product_id, review) in curso

    ## Convert DB data into LlamaIndex Documents
    documents = [Document(text=review) for review in reviews]
  except mysql.connector.Error as err:
    print(f"Error: {err}")

  finally:
     ...

3) Populate the MariaDB Vector (Product reviews) using LlamaIndex

Unlike the previous SQL example, you will notice that we now isolate the Vector Index and manage it using our LlamaIndex MariaDB Vector integration.

The code snippet below creates a VectorStoreIndex object in LlamaIndex using our SkySQL managed MariaDB as the actual Vector store.

""" Create MariaDB Vector store in LlamaIndex """

from llama_index.core import StorageContext, VectorStoreIndex
from llama_index.core.schema import Document
from llama_index.vector_stores.mariadb import MariaDBVectorStore

vector_store = MariaDBVectorStore.from_params(
    database = "vector_test",
    host = connection_properties["host"],
    user = connection_properties["user"],
    password = connection_properties["password"],
    port = connection_properties["port"],
) ## Uses the OpenAI number of dimensions (1536) for embeddings and HNSW is the ANN algorithm

storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Creating the index generates embeddings for the documents in the background and saves them
# into MariaDB with the help of the MariaDBVectorStore
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context, show_progress=True
)

4) Time to do semantic searches and engage the LLM

question = "Show me top 2 laptops that got a positive review along with key reasons cited."

# Using a retriever does a vector search only and returns a list of the top k documents matching semantically the question
retriever = index.as_retriever(
    similarity_top_k=3
)
retriever_response = retriever.retrieve(question)
for i, r in enumerate(retriever_response):
    print(f"\n Vector Store Retriever result {i+1}: {r.text}")

# Now, let's use an LLM on top of the retriever for more meaningful and accurate answers

# Using a query engine does a vector search first and then feeds the results to the LLM
query_engine = index.as_query_engine()
response = query_engine.query(question)
print("\n\nQuery engine response: ", response)

With the MariaDB Vector integration for LlamaIndex, available now in SkySQL, you have a powerful toolkit to build AI applications that leverage cutting-edge semantic search and vector capabilities. By combining the strengths of vector search with LLM-powered insights, you can unlock faster, more meaningful results from your data—all without the usual performance or cost trade-offs. Ready to build smarter, context-driven applications? Dive in and see how easy it is to get started.

Next Steps