Streaming with Pydantic AI
Last Updated on April 15, 2025 by Editorial Team
Author(s): Barrett Studdard
Originally published on Towards AI.
Building a pattern around LLM streaming
In my prior article on building a DAX LLM, I utilized base API calls to stream data from Anthropic. In this article, we’ll look to utilize Pydantic AI to further productionize and expand on this pattern to also incorporate tool calls.
Pydantic AI is a newer LLM agent framework. It’s built by the team behind Pydantic, which I’m a fan of through past use in FastAPI projects. It’s smaller in scope compared to something like LangChain and provides a bridge for creating LLM building blocks.
async def anthropic_stream_api_call(chat_input_list: list) -> AsyncGenerator[str, None]: """Streams anthropic response. Args: chat_input_list (list): List of chat inputs to send to the API. Yields: AsyncGenerator[str, None]: Stream of anthropic response. """ # Build message list message_input = build_anthropic_message_input(chat_input_list=chat_input_list) # Setup and make api call. client = AsyncAnthropic(api_key=ANTHROPIC_API_KEY) stream = await client.messages.create( model="claude-3-5-sonnet-20241022", max_tokens=4096, temperature=0.2, system=get_system_prompt(chat_input_list), messages=message_input, stream=True ) async for event in stream: if event.type in ['message_start', 'message_delta', 'message_stop', 'content_block_start', 'content_block_stop']: pass elif event.type == 'content_block_delta': yield event.delta.text else: yield event.type
The most relevant example with the base API call approach is the following function:
The API call is made in an async function and then events handled for streaming and displaying where needed via a… Read the full blog for free on Medium.
Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.
Published via Towards AI
Towards AI Academy
We Build Enterprise-Grade AI. We'll Teach You to Master It Too.
15 engineers. 100,000+ students. Towards AI Academy teaches what actually survives production.
Start free — no commitment:
→ 6-Day Agentic AI Engineering Email Guide — one practical lesson per day
→ Agents Architecture Cheatsheet — 3 years of architecture decisions in 6 pages
Our courses:
→ AI Engineering Certification — 90+ lessons from project selection to deployed product. The most comprehensive practical LLM course out there.
→ Agent Engineering Course — Hands on with production agent architectures, memory, routing, and eval frameworks — built from real enterprise engagements.
→ AI for Work — Understand, evaluate, and apply AI for complex work tasks.
Note: Article content contains the views of the contributing authors and not Towards AI.
Related posts
Recent Posts
