Author(s): Michalzarnecki

Originally published on Towards AI.

Welcome to the first part of this series. In this part I want to take a step back from LangChain, LangGraph and coding, and focus on the foundations. We will look at the main ideas behind large language models so that you build a solid intuition about how they work. This intuition is helpful later, when your application starts returning answers that are “almost right”. It also give you intuition why LLMs play such huge role in modern applications and systems.

If you already feel confident with the basics of LLMs and you prefer to jump straight into building things, you could technically skip first 3 parts. Although, I encourage you to stay with me here. A bit of theory about transformers and neural networks pays off later when we start debugging prompts, pipelines and agents.

At the beginning let’s answer a simple but fundamental question:
how do large language models actually work and where do they fit inside the broader AI landscape?

I like to think of AI as something layered. A bit like an onion, or, if you prefer pop culture, like ogres from Shrek. There are layers inside layers, and each inner layer is a more specific version of the outer one.

On the outside we have artificial intelligence as a whole, the general idea of building systems that behave intelligently. Inside that sits machine learning, which focuses on algorithms that improve by learning from examples instead of being fully programmed by hand. Within machine learning we find deep learning, where the main tools are neural networks with many hidden layers. And somewhere inside deep learning live the models that are most important for this course: large language models, LLMs, (and recently LRM — Large Reasoning Models).

In this series we will mostly work right there, in that inner part. But to understand why LLMs work so well, it is useful to quickly travel back to one of the moments when neural networks first appeared.

The perceptron: one neuron that started a long story

In 1957 Frank Rosenblatt introduced the perceptron. The name sounds mysterious, but the idea is quite simple. You can imagine it as a tiny machine that takes a few numbers as input and returns one decision: yes or no, class A or class B. It does this by multiplying the inputs by internal weights, adding them together and then checking whether the result crosses some threshold (it’s a simplified picture a modern neural networks approach gives more improvements and adjustments).

The breakthrough was not just this one neuron. The breakthrough was the idea that you can connect many such simple units and, by combining them, model much more complicated behavior. Out of many small decisions you can build a system that recognizes digits, classifies emails as spam or not spam, or later even understands and generates natural language.

To see how this scales up, let’s briefly look at a tiny network.

A small neural network: three layers, many connections

Imagine a neural network with three layers. The input layer has two neurons. The hidden layer has four neurons. The output layer has a single neuron that represents the final decision or prediction.

Each neuron in one layer is connected to every neuron in the next layer. Every connection has its own weight. These weights are numbers that say how important a given input is for a given neuron.

To compute the value of a neuron in the hidden layer, we take the outputs of all neurons from the previous layer, multiply them by the corresponding weights, add everything together and then add a bias term. The bias is another number that slightly shifts the result and gives the model more flexibility. After that, we usually pass the result through some activation function, but for our intuition we can think of it simply as “the transformed value of this neuron”.

For example, for the first neuron in the hidden layer we might take the first input multiplied by its weight, add the second input multiplied by another weight, add the bias, and obtain some final number. We repeat this process for all neurons in the hidden layer, and then again for the neuron in the output layer. This pass from inputs to outputs is called forward propagation.

In real systems we do not stop at one hidden layer and a couple of neurons. Modern networks have many layers and sometimes billions of connections. Yet the basic principle is still the same: multiply, add, transform, pass forward.

How a network learns instead of being programmed

So far this sounds like a fancy calculator. The interesting part starts when we let the network learn.

At the beginning the weights are essentially random. The network does not know anything. To turn it into something useful, we show it many training examples with correct answers. For each example we run forward propagation, compare the prediction with the true label, measure the error and then adjust the weights a little bit in the direction that reduces this error. This adjustment is done using techniques such as back propagation and gradient descent.

We repeat this process many times. Over time the network discovers internal configurations of weights that produce good predictions. It “learns” patterns in the data, although it does not understand them in the human sense.

If not everything here is crystal clear, do not worry. Building and analysing neural networks is a big field of its own. For our purposes it is enough to keep two key ideas in mind:

1. We can scale networks by adding more layers and neurons, so the capacity grows.
2. A large enough network can store a huge amount of knowledge and behavior in its weights.

This second point is exactly why large language models can be so capable. They are just neural networks taken to extreme scale, trained on massive text corpora. Here also it’s important than such networks can understand language, generate answers, but still have no desires, goals and initiatives, so it is actually fancy calculator + database + multiple other features — all created from neural network neurons structure.

Why it took decades to make this practical

The perceptron and early neural networks were important, but for a long time they were limited by the hardware and data available at the time. In the 1950s and for many years after that, there was no web, no cheap terabytes of storage and no GPUs or TPUs ready to crunch matrices in parallel. The theory was promising, but the computers were simply too weak to train large, deep networks on big datasets.

Because of that, progress in AI came in waves. There were times of great enthusiasm and then long “AI winters” when interest and funding cooled down. The ideas didn’t disappear, but they had to wait for better infrastructure.

Only in the last two decades did three important things align: much faster hardware, much larger datasets and solid improvements in algorithms. Once these pieces were in place, deep learning started to dominate many areas: computer vision, speech recognition, translation and, eventually, open-ended text generation. Seems the winter is no longer coming and we experience AI rocket launch. Let’s see if we reach the sky or new wall will be discovered in the process.

2019 and GPT-2: predicting the next word, changing the game

A very visible milestone in the modern LLM story came in 2019 with GPT-2.

OpenAI built a model with a deceptively simple training goal: given some text, predict the next word. Nothing more, nothing less. The power came from the scale and from the transformer architecture under the hood, which we will talk about later in this module.

GPT-2 was trained on around 40 gigabytes of internet text, roughly eight million documents. You can think of it as an agent that has “read” millions of books, articles and web pages and internalised the patterns of language, style and facts that appear there. It does not store them word by word, but it encodes them in the weights of the network.

An interesting detail is that at first OpenAI hesitated to release the full model. There were concerns about misuse, such as generating convincing spam, fake news or impersonations at scale. This was one of the first widely discussed cases where the AI community had to seriously face questions of safety and ethics around language models.

Since GPT-2, the field has moved very quickly. We now have far larger and more capable models, often with multimodal abilities and reasoning. For developers, however, one of the most important changes is not just “more parameters”, but the ecosystem around these models. Today we have tools and patterns that allow us to connect LLMs to our own data, orchestrate them using agents and graphs, and wrap them in real applications.

This is exactly where LangChain and LangGraph enter the picture. Starting from the second module of the course, we will build concrete pipelines and multi-agent systems around LLMs. In this first part, I wanted to make sure you see where these models come from and why they behave the way they do.

That’s all for now. In the next article we will move closer to the actual architecture of modern LLMs, especially transformers and attention paradigm. With that in place, you will be much better prepared to reason about the behavior of your applications when we start building them.

see previous chapter

see next chapter

see the GitHub repository with the code examples: https://github.com/mzarnecki/course_llm_agent_apps_with_langchain_and_langgraph

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI