Unlocking Document Intelligence: E2E Azure-Powered Chatbot with Vector-Based Search (Part 2 — Q&A)

Author(s): Shravankumar Hiregoudar

Originally published on Towards AI.

In the previous part, we embarked on a remarkable journey into document processing. We witnessed the development of a robust document embedding mechanism and the creation of a vector store, setting the stage for streamlined and optimized querying. Let’s continue our exploration by delving into the exciting realm of querying this vector store using the natural language questions that drive the heart of our document processing pipeline.

Table of contents:

1. Introduction
2. Solution Overview
   1.1 Architecture
   1.2 Chat Pipeline
3. Chat Implementation
   2.1 Project Files
4. Conclusion
5. Reference

Introduction

In the previous part, we covered document processing, including document embedding and vector store creation. Now, we’ll explore how to query this vector store using natural language questions.

This development is split into two blog parts: PART 1 U+007C PART 2

Solution Overview

Architecture

Chat Pipeline

The Chat Pipeline allows users to interact with the system using natural language queries. It combines Azure Cognitive Search for document retrieval and OpenAI’s GPT-3.5 Turbo for generating responses. Here’s how it works:

1. User Query: Users input their queries through a chat interface in a Streamlit application.
2. Retrieval: The code retrieves relevant documents from Azure Cognitive Search based on the user’s query. (After vectors are stored, we can perform operations like vector similarity search. This function looks for similar vectors in the vector store based on cosine similarity or other distance metrics. This is particularly useful for tasks like finding similar documents, words, or user queries.​) (Langchain Retriever)
3. Question Answering: It utilizes a predefined question-answering chain that combines a language model (GPT-3.5 Turbo) and the retrieved documents. The system generates a response based on the user’s query and the extracted information from the documents.
4. Display Response: The response is displayed in the chat interface, providing the user with answers to their queries.

Chat Implementation

Python code that demonstrates a query-answer system using a vector store.

Project Files

Let's take a look at the code and its functionalities.

• app.py (. . . continued)

# Project Layout
.
├── README.md
├── requirements.txt U+2714️
├── app.py 
├── .gitignore
├── .env U+2714️
├── db/ U+2714️
│ ├── metadata.db U+2714️
└── src/ U+2714️
 ├── database_manager.py U+2714️

U+2714️ = covered in Part 1

The ChatPipeline class resides in app.py and is a fundamental component for answering natural language questions using a pre-trained language model and a document retriever. Its constructor initializes the retriever (responsible for fetching relevant documents) and the language model (a pre-trained model for generating responses).

# app.py

class EmbeddingPipeline:
# ........ (refer to part 1)


class ChatPipeline:
 def __init__(self):
 load_dotenv()
 self.retriever = AzureCognitiveSearchRetriever(
 content_key="content",
 index_name=DEFAULT_SEARCH_INDEX,
 service_name=AZURE_SEARCH_NAME,
 api_key=AZURE_SEARCH_KEY,
 top_k=1
 )
 self.llm = AzureChatOpenAI(deployment_name=DEFAULT_CHAT_MODEL,
 openai_api_version=CHAT_API_VERSION,
 temperature=0,
 openai_api_base=CHAT_API_BASE,
 openai_api_key=CHAT_API_KEY
 )

 def get_query_answer(self, query, verbose=True):
 """
 Retrieve an answer from a question answering system that utilizes a pretrained language model (llm) 
 and a pre-defined retriever (Azure Cognitive Search) (retriever) to respond to questions following a predefined question answering chain (chain_type).

 Args:
 query (str): The query for which an answer is sought.
 retriever: The retriever used for retrieving relevant documents.
 verbose (bool, optional): Whether to log detailed information. Default is True.

 Returns:
 dict: A dictionary containing the query and the generated answer.
 """
 # Create a chain to answer questions
 qa = RetrievalQA.from_chain_type(
 llm=self.llm, 
 chain_type='stuff', 
 retriever=self.retriever, 
 return_source_documents=True
 )

 logger.info(f"Generating response ⏳")
 result = qa({"query": query})
 print(result)
 if verbose:
 logger.info(
 f"\n\nQ: {result['query']}\nA: {result['result']}\nSource/s: {set(json.loads(i.metadata['metadata']).get('source') for i in result['source_documents'])}")

 return result['result'], set(json.loads(i.metadata['metadata']).get('source') for i in result['source_documents'])

main function resides on app.py and serves as the entry point of the document processing pipeline. It executes the following steps:

• Processing documents in the Azure Storage container by initializing the EmbeddingPipeline based on the LOAD_VECTOR flag.
• Creating a user-friendly Streamlit interface for interactions.
• Managing a chat history to preserve user and assistant messages.
• Accepting user queries and presenting them within the chat interface.
• Utilizing the ChatPipeline to produce responses to user queries and showcasing these responses within the chat interface.

def main():

 if LOAD_VECTORS:
 EmbeddingPipeline().perform_embedding_pipeline()
 else:
 logger.info(f"Retrieving the stored vectors from an Azure Search index: '{DEFAULT_SEARCH_INDEX}'")

 ### STREAMLIT UI

 st.set_page_config(page_title="<name>")
 st.title("Simple Chat")

 # Initialize chat history
 if "messages" not in st.session_state:
 st.session_state.messages = []

 # Display chat messages from history on app rerun
 for message in st.session_state.messages:
 with st.chat_message(message["role"]):
 st.markdown(message["content"])

 # Accept user input
 if prompt := st.chat_input("Enter your query"):
 # Add user message to chat history
 st.session_state.messages.append({"role": "user", "content": prompt})
 # Display user message in chat message container
 with st.chat_message("user"):
 st.markdown(prompt)
 # Display assistant response in chat message container
 with st.chat_message("assistant"):
 message_placeholder = st.empty()
 full_response = ""

 answer , source = ChatPipeline().get_query_answer(prompt)
 full_response += f"{answer} *(source: {', '.join(source)})*" if source and None not in source else f"{answer}"
 message_placeholder.markdown(full_response + "▌")
 message_placeholder.markdown(full_response)
 st.session_state.messages.append({"role": "assistant", "content": full_response})

if __name__ == "__main__":
 main()

Run the project by streamlit run app.py, and the streamlit app should launch locally.

Conclusion

This Python code showcases a robust document processing pipeline that combines Azure services and OpenAI capabilities. It can be used for various applications, from content indexing to interactive chatbots. Understanding the code and its components is essential for building and customizing similar document processing pipelines in various domains.

Reference

• Langchain Documentation U+007C Git
• Azure Documentation U+007C Azure SDK Git
• Build conversational apps — Streamlit
• Add vector fields to a search index
• Blog: Azure OpenAI + Cognitive Search (vector search) + Azure Storage and a sprinkle of LangChain
• Git: azure-open-ai-embeddings-qna

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI