LAI #73: Vision-Language at Scale, o1’s Limits, RAG 2.0, and Multi-Agent Builders

Author(s): Towards AI Editorial Team

Originally published on Towards AI.

Good morning, AI enthusiasts,

This week’s issue covers deploying in-house vision-language models for large-scale document parsing, and whether OpenAI’s o1 models have actually advanced reasoning, or just scaled search.

We also cover Meta-Booster, a new ensemble method that dynamically blends deltas from multiple base models; a real-time RAG pipeline using LangGraph and web scraping; and a hands-on guide to building multi-agent systems with Smolagents. Finally, we look at DeepSeek R1’s approach to scaling with smarter training instead of bigger budgets.

Let’s dive in.

— Louis-François Bouchard, Towards AI Co-founder & Head of Community

Learn AI Together Community section!

AI poll of the week!

It’s not surprising that many are hesitant to declare a ceiling on Transformer LLMs just yet. After all, we’ve seen meaningful gains from scaling laws, better data curation, training on longer contexts, and smarter inference techniques like speculative decoding. What kinds of developments would move the needle for you? Do techniques like MoE, longer context windows, or retrieval-based augmentation feel like true progress, or are they just clever patches on an aging architecture? Let’s discuss in the thread!

Collaboration Opportunities

The Learn AI Together Discord community is flooded with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, join the collaboration channel! Keep an eye on this section, too — we share cool opportunities every week!

1. Uwaix. wants to do some research in AI and is looking for people who’d like to join. If you have some topic ideas or want to get into research, reach out to him in the thread!

2. .tsekis is working on a project with RAG and is looking for someone excited to build something impactful from the ground up. If you enjoy experimenting, solving problems, and creating things, connect in the thread!

3. Adventurous_flamingo_86116 recently launched Learn AI’s Emotionally Intelligent API, a solution designed to enrich user interactions and experiences. They are looking for collaborators, and if this sounds interesting to you, message them in the thread!

Meme of the week!

Meme shared by rucha8062

TAI Curated Section

Article of the week

Deploy an in-house Vision Language Model to parse millions of documents: say goodbye to Gemini and OpenAI. By Jeremy Arancio

This blog details the deployment of an in-house Vision Language Model (VLM), specifically Qwen-2.5-VL, for extracting structured data from documents. It outlines using vLLM for efficient inference and AWS Batch with EC2 orchestration for managing the processing pipeline, addressing cost, data security, and reliability concerns associated with third-party LLM APIs. The process covers containerizing the application with Docker/uv and managing the AWS infrastructure via Terraform. A cost analysis indicates this self-hosted method can be more economical than external services for large-scale document processing.

Our must-read articles

1. Have o1 Models Solved Human Reasoning? By Nehdiii

OpenAI’s o1 models sparked discussions on solved LLM reasoning. This article speculates that o1 uses Reinforcement Learning with Chain-of-Thought (CoT) and process reward models for training, focusing on step-by-step validation. Inference likely employs large-scale search, generating and scoring numerous reasoning paths via significant computing. However, the author questions if the reasoning is “solved,” citing research showing CoT models fail on complex, out-of-distribution tasks, suggesting pattern matching over robust understanding. The inference method is seen as effective but computationally intensive and unlike human cognition.

2. A Novel and Practical Meta‑Booster for Supervised Learning By Shenggang Li

This article introduced Meta-Booster, an ensemble framework for supervised learning tasks. It dynamically combines incremental updates (deltas) from multiple base learners (like XGBoost, LightGBM, and NN) at each boosting step. Weights for these deltas are determined using least-squares stacking on a validation set, and an optimal learning rate is found via line search. Experiments on classification and regression datasets showed improved metrics (AUC, LogLoss, MAPE, RMSE) compared to individual models. Meta-Booster offers a flexible way to dynamically leverage diverse model strengths for better predictions.

3. RAG 2.0: Supercharging LLMs with Real-Time Web Data and LangGraph By Samvardhan Singh

This article explained how Retrieval-Augmented Generation (RAG) can be enhanced with real-time web data to keep large language models current. It detailed the limitations of traditional RAG’s static datasets and introduced a dynamic approach using web scraping tools like Scrapy. LangGraph was highlighted as the framework to orchestrate this complex workflow, managing steps from data scraping and embedding to vector storage (using FAISS for efficiency), retrieval, and final response generation. Latency optimization techniques were also discussed to ensure timely answers.

4. Building a Multi-Agent System with Multiple MCP Servers using Smolagents By Murat Şimşek

This piece details building a multi-agent system using the Smolagents library and multiple MCP (Model Context Protocol) servers. It showed how to create a custom MCP server for Markdown memory tasks and integrate a pre-built PubMed server from Smithery, using Google’s Gemini 2.5 Flash Preview LLM. The process covered server setup, Smolagents configuration, and defining distinct agent roles for memory and task execution. Examples illustrated the system creating fitness plans, updating them via PubMed searches, and recalling stored information.

5. DeepSeek R1: Pioneering Research and Engineering as a Competitor to Pure Scaling Approaches By Nehdiii

Unlike many closed labs, DeepSeek R1’s release offered notable transparency regarding its algorithms and training. The model achieved strong performance, reportedly surpassing benchmarks set by larger models, while using significantly less computing and funding, around $6 million compared to estimates of hundreds of millions for others. This highlighted efficient research and engineering over brute-force scaling. The piece details the training of models like DeepSeek-R1-Zero (using RL directly on a base model) and the refined DeepSeek-R1 (alternating SFT and RL with GRPO), including novel reinforcement learning techniques.

If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI