Milvus¶

In this example, you will set up by creating a simple Llama Index RAG application with a vector store using Milvus. You'll also set up evaluation and logging with TruLens.

Before running, you'll need to install the following

• Docker Compose (Instructions)
• Milvus Standalone (Instructions)

Setup¶

Install dependencies¶

Let's install some of the dependencies for this notebook if we don't have them already

In [ ]:

# !pip install trulens trulens-apps-llamaindex trulens-providers-openai llama_index==0.8.4 pymilvus==2.3.0 nltk==3.8.1 html2text==2020.1.16

# !pip install trulens trulens-apps-llamaindex trulens-providers-openai llama_index==0.8.4 pymilvus==2.3.0 nltk==3.8.1 html2text==2020.1.16

Add API keys¶

For this quickstart, you will need Open AI and Huggingface keys

In [ ]:

import os

os.environ["OPENAI_API_KEY"] = "..."

import os os.environ["OPENAI_API_KEY"] = "..."

Import from LlamaIndex and TruLens¶

In [ ]:

from llama_index import VectorStoreIndex
from llama_index.readers.web import SimpleWebPageReader
from llama_index.storage.storage_context import StorageContext
from llama_index.vector_stores import MilvusVectorStore
from trulens.core import Feedback
from trulens.core import TruSession
from trulens.feedback.v2.feedback import Groundedness
from trulens.apps.llamaindex import TruLlama
from trulens.providers.openai import OpenAI as fOpenAI

session = TruSession()

from llama_index import VectorStoreIndex from llama_index.readers.web import SimpleWebPageReader from llama_index.storage.storage_context import StorageContext from llama_index.vector_stores import MilvusVectorStore from trulens.core import Feedback from trulens.core import TruSession from trulens.feedback.v2.feedback import Groundedness from trulens.apps.llamaindex import TruLlama from trulens.providers.openai import OpenAI as fOpenAI session = TruSession()

First we need to load documents. We can use SimpleWebPageReader¶

In [ ]:

# load documents
documents = SimpleWebPageReader(html_to_text=True).load_data(
    ["http://paulgraham.com/worked.html"]
)

# load documents documents = SimpleWebPageReader(html_to_text=True).load_data( ["http://paulgraham.com/worked.html"] )

Next we want to create our vector store index¶

By default, LlamaIndex will do this in memory as follows:

In [ ]:

index = VectorStoreIndex.from_documents(documents)

index = VectorStoreIndex.from_documents(documents)

Alternatively, we can create the vector store in pinecone

In [ ]:

vector_store = MilvusVectorStore(overwrite=True)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)

vector_store = MilvusVectorStore(overwrite=True) storage_context = StorageContext.from_defaults(vector_store=vector_store) index = VectorStoreIndex.from_documents( documents, storage_context=storage_context )

In either case, we can create our query engine the same way¶

In [ ]:

query_engine = index.as_query_engine()

query_engine = index.as_query_engine()

Now we can set the engine up for evaluation and tracking¶

In [ ]:

import numpy as np

# Initialize OpenAI-based feedback function collection class:
openai = fOpenAI()

# Define groundedness
grounded = Groundedness(groundedness_provider=openai)
f_groundedness = (
    Feedback(grounded.groundedness_measure, name="Groundedness")
    .on(
        TruLlama.select_source_nodes().node.text.collect()  # context
    )
    .on_output()
    .aggregate(grounded.grounded_statements_aggregator)
)

# Question/answer relevance between overall question and answer.
f_qa_relevance = Feedback(
    openai.relevance, name="Answer Relevance"
).on_input_output()

# Question/statement relevance between question and each context chunk.
f_context_relevance = (
    Feedback(openai.context_relevance, name="Context Relevance")
    .on_input()
    .on(TruLlama.select_source_nodes().node.text)
    .aggregate(np.mean)
)

import numpy as np # Initialize OpenAI-based feedback function collection class: openai = fOpenAI() # Define groundedness grounded = Groundedness(groundedness_provider=openai) f_groundedness = ( Feedback(grounded.groundedness_measure, name="Groundedness") .on( TruLlama.select_source_nodes().node.text.collect() # context ) .on_output() .aggregate(grounded.grounded_statements_aggregator) ) # Question/answer relevance between overall question and answer. f_qa_relevance = Feedback( openai.relevance, name="Answer Relevance" ).on_input_output() # Question/statement relevance between question and each context chunk. f_context_relevance = ( Feedback(openai.context_relevance, name="Context Relevance") .on_input() .on(TruLlama.select_source_nodes().node.text) .aggregate(np.mean) )

Instrument query engine for logging with TruLens¶

In [ ]:

tru_query_engine_recorder = TruLlama(
    query_engine,
    app_name="LlamaIndex_App",
    app_version="1",
    feedbacks=[f_groundedness, f_qa_relevance, f_context_relevance],
)

tru_query_engine_recorder = TruLlama( query_engine, app_name="LlamaIndex_App", app_version="1", feedbacks=[f_groundedness, f_qa_relevance, f_context_relevance], )

In [ ]:

# Instrumented query engine can operate as a context manager
with tru_query_engine_recorder as recording:
    llm_response = query_engine.query("What did the author do growing up?")
    print(llm_response)

# Instrumented query engine can operate as a context manager with tru_query_engine_recorder as recording: llm_response = query_engine.query("What did the author do growing up?") print(llm_response)

Explore in a Dashboard¶

In [ ]:

from trulens.dashboard import run_dashboard

run_dashboard(session)  # open a local streamlit app to explore

# stop_dashboard(session) # stop if needed

from trulens.dashboard import run_dashboard run_dashboard(session) # open a local streamlit app to explore # stop_dashboard(session) # stop if needed

Or view results directly in your notebook¶

In [ ]:

session.get_records_and_feedback()[0]

session.get_records_and_feedback()[0]