Building Blocks of an LLM and How To Choose the Right Model and Framework To Build a LLM Based Application

Author(s): Hari Harasudhan Duraidhayalu

Originally published on Towards AI.

Evolution of LLM

Large Language Models have been the topic of discussion in every organization ever since OpenAI launched ChatGPT. Every organization is exploring its options around building applications that can utilize this new-age Generative AI and how to capitalize on it. Some organizations are building their own LLM while some are trying to explore how to take advantage of the existing ones. There are many alternatives to ChatGPT. Google has come up with its own LLM — Gemini and META has developed their LLM — LLAMA 3, there are many other open-source LLM’s that are available in the market.

It is truly mind blowing how artificial intelligence has evolved from being addressed as machine learning that simply reads numbers and builds statistical models to being able to generate its own content with LLM’s has truly been phenomenal.

The figure below shows the timeline of how LLM has evolved from ML over the course of a few decades.

LLM has its roots in Machine learning.

Machine Learning: The area of artificial intelligence that changed how we made use of the data available to us. Before this we used the data to mainly show the trend or just do some descriptive statistics. But with ML we used the data to make predictions and we no longer give instructions but only provide training data and test data and the computer takes care of the process to predict the outcomes.

Deep Learning: Advanced Neural network models are termed Deep learning. With the amount of data available kept increasing exponentially it was getting increasingly difficult to scale the models to huge data. Neural network model was not able to keep up with the huge data and thus came Deep learning models.

NLP & Transformers: NLPs are machine learning that are applied to text data. The end goal of NLP being predicting the next word of an sentence. While NLPs are most suited for small textual data it is with the new methodology called Transformers that made LLM and Generative AI possible.

LLM: NLPs that are trained with vast amount of data using transformers are LLMs. It is estimated that the ChatGPT has text data equivalent to 10 million books.

Building blocks of an LLM:

There are 4 basic terminologies that we need to understand that constitute a LLM.

Tokens: These are the building blocks of an LLM. In simple words, these are the words that supply the text data to a LLM. Long tokens form a sequence, and sequences in LLM form a vocabulary.

Context and Context Window: Context is the information that the user gives when asking a question to an LLM. For e.g: Act like a sales chatbot and sell this pen to me. Here, the context is that the user instructs the GPT to mimic a sales agent and tries to sell that pen to the user.

Context window is the maximum size of context that we can give to an LLM. In ChatGPT 4 the max context window is set to 8,192 tokens.

Prompts: Prompts are basically the query we pose to a LLM engine. For e.g. every questions that a user asks to a GPT is a prompt. The quality of the LLM response is based on the quality of the prompts that a user asks, that is why it is necessary that we build good prompts.

Prompt engineering: Prompt engineering may not be one of the building blocks of llm, but with the way LLM applications are built, it is increasingly necessary to understand how to build better prompts. Prompt engineering is the art of building better prompts in order to get the best response from an LLM.

Existing frameworks to build an LLM application

There are two main frameworks that are available today to build an application using LLM. LangChain and LlamaIndex.

LangChain: LangChain provides a very simple way of understanding LLM concepts and to build them. They come with many connectors which can let your application connect to many external databases or servers. Though they were initially focused on building simple applications they have made a lot more changes to their solutions that helps in building complex applications.

LlamaIndex: LlamaIndex is another solution which is more focused towards building better RAG applications. It is a less generalised framework when compared to LangChain. I will talk more about RAG in my next post.

How to select the right LLM and framework to build an application?

Choosing the right model that suits for your personal/ organization needs might be a tedious process. There are many models out there and we are only at the brink of this LLM era, soon there will be much more models that will be available to use. While it might be difficult to point out the best model without knowing the requirements i have pointed out few criteria's to consider when picking a model or the framework.

Precision: Precision is one of the main top criteria's when selecting a LLM. Like every other model the accuracy/precision of a LLM is very important. ChatGPT precision seem to be very high when compared to other models that is available in the market. The other models available are llama, mistral and falcon.

Cost: ChatGPT and Gemini are two closed source LLM models. They charge based on the usage of their API. The pricing information is fully listed in their website and it gets updated quite often. So i suggest you to check their site for the latest pricing info. If closed source LLM is not an option, there are other open source models that can be used. Some of the highly renowned open source models are LLAMA-3, Mistral and Flacon.

Latency/Speed: Speed is the other important criterion when selecting a LLM. This becomes very essential when implementing an LLM application in production. Latency is usually measured in tokens/second. With the existing models available ChatGPT seems to have high latency compared to others.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI