Building RAG Application using Gemma 7B LLM & Upstash Vector Database

Author(s): Youssef Hosni

Originally published on Towards AI.

Retrieval-Augmented Generation (RAG) is the concept of providing large language models (LLMs) with additional information from an external knowledge source. This allows them to generate more accurate and contextual answers while reducing hallucinations. In this article, we will provide a step-by-step guide to building a complete RAG application using the latest open-source LLM by Google Gemma 7B and Upstash serverless vector database.

Getting Started & Setting Up Working EnvironmentDownload & Split the Cosmopedia DatasetGenerating Embedding with Sentence Transformers ModelStore the Embeddings in the Upstash Vector DatabaseIntroduce & Use Gemma 7B LLMQuerying the RAG ApplicationDesigned for the serverless with per-request pricing and Redis API on durable storage.

console.upstash.com

The first step in building an RAG application is to prepare the working environment. We will start with downloading the packages we will use in building the application:

%pip install -q -U langchain torch transformers sentence-transformers datasets tiktoken upstash_vector

Next, we will import the packages and libraries that will be used:

import torchfrom upstash_vector import Vectorfrom datasets import load_datasetfrom tqdm import tqdm, trangefrom langchain_community.document_loaders.csv_loader import CSVLoaderfrom langchain.text_splitter import RecursiveCharacterTextSplitterfrom langchain.embeddings import HuggingFaceEmbeddingsfrom transformers import AutoTokenizer, AutoModelForCausalLMfrom transformers import AutoTokenizer, pipelinefrom langchain import HuggingFacePipelinefrom langchain.chains import RetrievalQA

Retrieval-Augmented Generation (RAG) is the concept to provide LLMs with additional information from… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI