Azure OpenAI LangChain Quickstart¶

In this quickstart you will create a simple LangChain App and learn how to log it and get feedback on an LLM response using both an embedding and chat completion model from Azure OpenAI.

Setup¶

Install dependencies¶

Let's install some of the dependencies for this notebook if we don't have them already

In [ ]:

# !pip install trulens trulens-apps-langchain trulens-providers-openai llama-index==0.10.17 langchain==0.1.11 chromadb==0.4.24 langchainhub bs4==0.0.2 langchain-openai==0.0.8 ipytree==0.2.2

# !pip install trulens trulens-apps-langchain trulens-providers-openai llama-index==0.10.17 langchain==0.1.11 chromadb==0.4.24 langchainhub bs4==0.0.2 langchain-openai==0.0.8 ipytree==0.2.2

Add API keys¶

For this quickstart, you will need a larger set of information from Azure OpenAI compared to typical OpenAI usage. These can be retrieved from https://oai.azure.com/ . Deployment name below is also found on the oai azure page.

In [ ]:

# Check your https://oai.azure.com dashboard to retrieve params:

import os

os.environ["AZURE_OPENAI_API_KEY"] = "..."  # azure
os.environ["AZURE_OPENAI_ENDPOINT"] = (
    "https://<your endpoint here>.openai.azure.com/"  # azure
)
os.environ["OPENAI_API_VERSION"] = "2023-07-01-preview"  # may need updating
os.environ["OPENAI_API_TYPE"] = "azure"

# Check your https://oai.azure.com dashboard to retrieve params: import os os.environ["AZURE_OPENAI_API_KEY"] = "..." # azure os.environ["AZURE_OPENAI_ENDPOINT"] = ( "https://.openai.azure.com/" # azure ) os.environ["OPENAI_API_VERSION"] = "2023-07-01-preview" # may need updating os.environ["OPENAI_API_TYPE"] = "azure"

Import from TruLens¶

In [ ]:

# Imports main tools:
from trulens.core import Feedback
from trulens.core import TruSession
from trulens.apps.langchain import TruChain

session = TruSession()
session.reset_database()

# Imports main tools: from trulens.core import Feedback from trulens.core import TruSession from trulens.apps.langchain import TruChain session = TruSession() session.reset_database()

Create Simple LLM Application¶

This example uses LangChain and is set to use Azure OpenAI LLM & Embedding Models

In [ ]:

import os

# LangChain imports
from langchain import hub
from langchain.document_loaders import WebBaseLoader
from langchain.schema import StrOutputParser
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain_core.runnables import RunnablePassthrough

# Imports Azure LLM & Embedding from LangChain
from langchain_openai import AzureChatOpenAI
from langchain_openai import AzureOpenAIEmbeddings

import os # LangChain imports from langchain import hub from langchain.document_loaders import WebBaseLoader from langchain.schema import StrOutputParser from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.vectorstores import Chroma from langchain_core.runnables import RunnablePassthrough # Imports Azure LLM & Embedding from LangChain from langchain_openai import AzureChatOpenAI from langchain_openai import AzureOpenAIEmbeddings

Define the LLM & Embedding Model¶

In [ ]:

# get model from Azure
llm = AzureChatOpenAI(
    model="gpt-35-turbo",
    deployment_name="<your azure deployment name>",  # Replace this with your azure deployment name
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version=os.environ["OPENAI_API_VERSION"],
)

# You need to deploy your own embedding model as well as your own chat completion model
embed_model = AzureOpenAIEmbeddings(
    azure_deployment="soc-text",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    api_version=os.environ["OPENAI_API_VERSION"],
)

# get model from Azure llm = AzureChatOpenAI( model="gpt-35-turbo", deployment_name="", # Replace this with your azure deployment name api_key=os.environ["AZURE_OPENAI_API_KEY"], azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], api_version=os.environ["OPENAI_API_VERSION"], ) # You need to deploy your own embedding model as well as your own chat completion model embed_model = AzureOpenAIEmbeddings( azure_deployment="soc-text", api_key=os.environ["AZURE_OPENAI_API_KEY"], azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"], api_version=os.environ["OPENAI_API_VERSION"], )

Load Doc & Split & Create Vectorstore¶

1. Load the Document¶

In [ ]:

# Load a sample document
loader = WebBaseLoader(
    web_paths=("http://paulgraham.com/worked.html",),
)
docs = loader.load()

# Load a sample document loader = WebBaseLoader( web_paths=("http://paulgraham.com/worked.html",), ) docs = loader.load()

2. Split the Document¶

In [ ]:

# Define a text splitter
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200
)

# Apply text splitter to docs
splits = text_splitter.split_documents(docs)

# Define a text splitter text_splitter = RecursiveCharacterTextSplitter( chunk_size=1000, chunk_overlap=200 ) # Apply text splitter to docs splits = text_splitter.split_documents(docs)

3. Create a Vectorstore¶

In [ ]:

# Create a vectorstore from splits
vectorstore = Chroma.from_documents(documents=splits, embedding=embed_model)

# Create a vectorstore from splits vectorstore = Chroma.from_documents(documents=splits, embedding=embed_model)

Create a RAG Chain¶

In [ ]:

retriever = vectorstore.as_retriever()

prompt = hub.pull("rlm/rag-prompt")
llm = llm


def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)


rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

retriever = vectorstore.as_retriever() prompt = hub.pull("rlm/rag-prompt") llm = llm def format_docs(docs): return "\n\n".join(doc.page_content for doc in docs) rag_chain = ( {"context": retriever | format_docs, "question": RunnablePassthrough()} | prompt | llm | StrOutputParser() )

Send your first request¶

In [ ]:

query = "What is most interesting about this essay?"
answer = rag_chain.invoke(query)

print("query was:", query)
print("answer was:", answer)

query = "What is most interesting about this essay?" answer = rag_chain.invoke(query) print("query was:", query) print("answer was:", answer)

Initialize Feedback Function(s)¶

In [ ]:

import numpy as np
from trulens.providers.openai import AzureOpenAI

# Initialize AzureOpenAI-based feedback function collection class:
provider = AzureOpenAI(
    # Replace this with your azure deployment name
    deployment_name="<your azure deployment name>"
)


# select context to be used in feedback. the location of context is app specific.
context = TruChain.select_context(rag_chain)

# Question/answer relevance between overall question and answer.
f_qa_relevance = Feedback(
    provider.relevance, name="Answer Relevance"
).on_input_output()

# Question/statement relevance between question and each context chunk.
f_context_relevance = (
    Feedback(
        provider.context_relevance_with_cot_reasons, name="Context Relevance"
    )
    .on_input()
    .on(context)
    .aggregate(np.mean)
)

# groundedness of output on the context
f_groundedness = (
    Feedback(
        provider.groundedness_measure_with_cot_reasons, name="Groundedness"
    )
    .on(context.collect())
    .on_output()
)

import numpy as np from trulens.providers.openai import AzureOpenAI # Initialize AzureOpenAI-based feedback function collection class: provider = AzureOpenAI( # Replace this with your azure deployment name deployment_name="" ) # select context to be used in feedback. the location of context is app specific. context = TruChain.select_context(rag_chain) # Question/answer relevance between overall question and answer. f_qa_relevance = Feedback( provider.relevance, name="Answer Relevance" ).on_input_output() # Question/statement relevance between question and each context chunk. f_context_relevance = ( Feedback( provider.context_relevance_with_cot_reasons, name="Context Relevance" ) .on_input() .on(context) .aggregate(np.mean) ) # groundedness of output on the context f_groundedness = ( Feedback( provider.groundedness_measure_with_cot_reasons, name="Groundedness" ) .on(context.collect()) .on_output() )

Custom functions can also use the Azure provider¶

In [ ]:

from typing import Dict, Tuple

from trulens.feedback import prompts


class Custom_AzureOpenAI(AzureOpenAI):
    def style_check_professional(self, response: str) -> float:
        """
        Custom feedback function to grade the professional style of the response, extending AzureOpenAI provider.

        Args:
            response (str): text to be graded for professional style.

        Returns:
            float: A value between 0 and 1. 0 being "not professional" and 1 being "professional".
        """
        professional_prompt = str.format(
            "Please rate the professionalism of the following text on a scale from 0 to 10, where 0 is not at all professional and 10 is extremely professional: \n\n{}",
            response,
        )
        return self.generate_score(system_prompt=professional_prompt)

    def context_relevance_with_cot_reasons_extreme(
        self, question: str, context: str
    ) -> Tuple[float, Dict]:
        """
        Tweaked version of context relevance, extending AzureOpenAI provider.
        A function that completes a template to check the relevance of the statement to the question.
        Scoring guidelines for scores 5-8 are removed to push the LLM to more extreme scores.
        Also uses chain of thought methodology and emits the reasons.

        Args:
            question (str): A question being asked.
            context (str): A statement to the question.

        Returns:
            float: A value between 0 and 1. 0 being "not relevant" and 1 being "relevant".
        """

        # remove scoring guidelines around middle scores
        system_prompt = prompts.CONTEXT_RELEVANCE_SYSTEM.replace(
            "- STATEMENT that is RELEVANT to most of the QUESTION should get a score of 5, 6, 7 or 8. Higher score indicates more RELEVANCE.\n\n",
            "",
        )

        user_prompt = str.format(
            prompts.CONTEXT_RELEVANCE_USER, question=question, context=context
        )
        user_prompt = user_prompt.replace(
            "RELEVANCE:", prompts.COT_REASONS_TEMPLATE
        )

        return self.generate_score_and_reasons(system_prompt, user_prompt)


# Add your Azure deployment name
custom_azopenai = Custom_AzureOpenAI(
    deployment_name="<your azure deployment name>"
)

# Question/statement relevance between question and each context chunk.
f_context_relevance_extreme = (
    Feedback(
        custom_azopenai.context_relevance_with_cot_reasons_extreme,
        name="Context Relevance - Extreme",
    )
    .on_input()
    .on(context)
    .aggregate(np.mean)
)

f_style_check = Feedback(
    custom_azopenai.style_check_professional, name="Professional Style"
).on_output()

from typing import Dict, Tuple from trulens.feedback import prompts class Custom_AzureOpenAI(AzureOpenAI): def style_check_professional(self, response: str) -> float: """ Custom feedback function to grade the professional style of the response, extending AzureOpenAI provider. Args: response (str): text to be graded for professional style. Returns: float: A value between 0 and 1. 0 being "not professional" and 1 being "professional". """ professional_prompt = str.format( "Please rate the professionalism of the following text on a scale from 0 to 10, where 0 is not at all professional and 10 is extremely professional: \n\n{}", response, ) return self.generate_score(system_prompt=professional_prompt) def context_relevance_with_cot_reasons_extreme( self, question: str, context: str ) -> Tuple[float, Dict]: """ Tweaked version of context relevance, extending AzureOpenAI provider. A function that completes a template to check the relevance of the statement to the question. Scoring guidelines for scores 5-8 are removed to push the LLM to more extreme scores. Also uses chain of thought methodology and emits the reasons. Args: question (str): A question being asked. context (str): A statement to the question. Returns: float: A value between 0 and 1. 0 being "not relevant" and 1 being "relevant". """ # remove scoring guidelines around middle scores system_prompt = prompts.CONTEXT_RELEVANCE_SYSTEM.replace( "- STATEMENT that is RELEVANT to most of the QUESTION should get a score of 5, 6, 7 or 8. Higher score indicates more RELEVANCE.\n\n", "", ) user_prompt = str.format( prompts.CONTEXT_RELEVANCE_USER, question=question, context=context ) user_prompt = user_prompt.replace( "RELEVANCE:", prompts.COT_REASONS_TEMPLATE ) return self.generate_score_and_reasons(system_prompt, user_prompt) # Add your Azure deployment name custom_azopenai = Custom_AzureOpenAI( deployment_name="" ) # Question/statement relevance between question and each context chunk. f_context_relevance_extreme = ( Feedback( custom_azopenai.context_relevance_with_cot_reasons_extreme, name="Context Relevance - Extreme", ) .on_input() .on(context) .aggregate(np.mean) ) f_style_check = Feedback( custom_azopenai.style_check_professional, name="Professional Style" ).on_output()

Instrument chain for logging with TruLens¶

In [ ]:

tru_query_engine_recorder = TruChain(
    rag_chain,
    llm=azopenai,
    app_name="LangChain_App",
    app_version="AzureOpenAI",
    feedbacks=[
        f_groundedness,
        f_qa_relevance,
        f_context_relevance,
        f_context_relevance_extreme,
        f_style_check,
    ],
)

tru_query_engine_recorder = TruChain( rag_chain, llm=azopenai, app_name="LangChain_App", app_version="AzureOpenAI", feedbacks=[ f_groundedness, f_qa_relevance, f_context_relevance, f_context_relevance_extreme, f_style_check, ], )

In [ ]:

query = "What is most interesting about this essay?"
with tru_query_engine_recorder as recording:
    answer = rag_chain.invoke(query)
    print("query was:", query)
    print("answer was:", answer)

query = "What is most interesting about this essay?" with tru_query_engine_recorder as recording: answer = rag_chain.invoke(query) print("query was:", query) print("answer was:", answer)

Explore in a Dashboard¶

In [ ]:

from trulens.dashboard import run_dashboard

run_dashboard(session)  # open a local streamlit app to explore

# stop_dashboard(session) # stop if needed

from trulens.dashboard import run_dashboard run_dashboard(session) # open a local streamlit app to explore # stop_dashboard(session) # stop if needed

Or view results directly in your notebook¶

In [ ]:

records, feedback = session.get_records_and_feedback(
    app_ids=["LangChain_App1_AzureOpenAI"]
)  # pass an empty list of app_ids to get all

records

records, feedback = session.get_records_and_feedback( app_ids=["LangChain_App1_AzureOpenAI"] ) # pass an empty list of app_ids to get all records

In [ ]:

session.get_leaderboard(app_ids=["LangChain_App1_AzureOpenAI"])

session.get_leaderboard(app_ids=["LangChain_App1_AzureOpenAI"])