Dense embedding have continuous numeric values. i.e after decimal point values will be present. Chunk will be converted to embeddings, each embedding point will have number like [0.3455566 ,0.6777779, ...]. Generated vectors will be plotted in a space called latent space. Discrete values like 0 won't be present.

Sparse embedding will mostly have values like 0. Rather than semantic meaning, it considers frequency or importance of words in a text.
Ex: one hot encoding

Models for Dense embedding
1. LLM

• Embed only LLMs are also available. Sole purpose of these LLMs is to generate embedding. Ex: Nomic embed, BGE.
• We can also give a prompt to general purpose LLM to generate embedding. But this is costly operation.

2. Transformers (encoder)
Ex: Minilm, nomic transformers

These models are available in hugging face, ollama.It also hosts other models as well.

Generated vectors will

How can we evaluate the performance of RAG system ?
For a given user query, RAG system will return some set of matching documents. If the returned documents matches with our expectations, we can say it is yielding good results. Say, if our expectation from RAG is to return a, b, c, d, e documents for a user query and in reality it returns a, b, d docs alone. Out of 5, 3 is returned. It is meeting expectation to half right ? Like how we write unit test cases for a software code, we need to write test cases for user query for evaluating the RAG systems.