#40 Build Your Own Llama, LLMs From Scratch, and Understanding Meta’s Transfusion Model.

Author(s): Towards AI Editorial Team

Originally published on Towards AI.

Good morning, AI enthusiasts! This week, expect lots of resources and lighter conversations, research-focused collaborations, and, of course, the funniest meme from the community.

What’s AI Weekly

Keeping up with LLMs is getting tougher; there’s so much happening every week. I have compiled a guide to help you start and improve your LLM skills in 2024 without an advanced background in the field and stay up-to-date with the latest news and state-of-the-art techniques! All resources listed in the guide are free, except some online courses and books, which are certainly recommended for a better understanding, but it is definitely possible to become an expert without them, with a little more time spent on online readings, videos, and practice. Read the complete LLM guide here!

— Louis-François Bouchard, Towards AI Co-founder & Head of Community

This issue is brought to you thanks to FT Live:

The FT Future of AI Summit returns to assess the current landscape for AI innovation and examine the real-world use cases for companies investing in AI while navigating security, workforce, and ethical concerns. Learn about the most exciting advancements in ML, NLP, and robotics and how they are being scaled for success and growth.

Taking place over two in-person days, the event gathers a cross-sector audience of strategy, innovation, technology, and business function leaders charged with creating, integrating, scaling, and commercializing AI.

Date & Location: 6–7 November 2024, In-Person & Digital | London

Register now and use code TOWARDSAI to save 20% on your pass!

Learn AI Together Community section!

Featured Community post from the Discord

Danieldanieldaniel1 wrote an article on building a local-first vector database using RxDB and transformers.js. It shows how to run machine learning models directly in the browser to enable semantic search and similarity-based queries without the need for a backend. This approach provides zero network latency, offline functionality, better privacy, and no server costs. If you’re into offline-first apps, vector databases, or running ML models on users’ devices, read the article here. Support a fellow community member and share your feedback in the thread!

AI poll of the week!

Wow! 47% of you can’t remember when you took a break. Yes, we live in a world where breaks are ‘anti-productivity,’ but there are so many studies proving that conscious breaks lead to a great boost in productivity. So, let this poll be a reminder to take a break! And tell us all about it in the thread!

Collaboration Opportunities

The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, join the collaboration channel! Keep an eye on this section, too — we share cool opportunities every week!

1. Puskchan needs help fixing the front end for an MLops project. If you can help, connect with him in the thread!

2. Samyog_dhital is researching and exploring ways to enhance reasoning capabilities in LLMs. The goal is to solve this challenge, enabling LLMs to solve complex problems with logical, step-by-step planning similar to human reasoning. They are looking for someone to work on this and a potential co-founder. If you are interested, connect with them in the thread!

3. Dykyi_vladk is working on reimplementing and enhancing the PaLM model. If you are interested in NLP, contact him in the thread!

4. Knytfury is looking to work with someone on a new research paper or an existing paper’s implementation. If you are working on something and need some human resources to work on the paper, reach out in the thread!

Meme of the week!

Meme shared by ghost_in_the_machine

TAI Curated section

Article of the week

Build Your Own Llama 3 Architecture from Scratch Using PyTorch by Milan Tamang

This is a step-by-step guide to building the complete architecture of the Llama 3 model from scratch using PyTorch and performing training and inferencing on a custom dataset. It explains how each component of the Llama 3 model works under the hood, guides you on how to write codes to build each component and assemble them all together to build a fully functional Llama 3 model. Additionally, you’ll also write codes to train your model with new custom datasets and perform inferencing.

Our must-read articles

1. Genetic Algorithms Simplified: A Step-by-Step Example for Beginners by Linh V Nguyen

Genetic Algorithm (GA) is an evolutionary computation inspired by Darwin’s theory of natural selection. Its basic principle is to mimic natural selection and reproduction while searching for optimal solutions. This article simplifies complex concepts by providing a step-by-step example. Whether you’re new to AI or looking to expand your knowledge, this resource is perfect for understanding the fundamentals of genetic algorithms and their practical applications.

2. Meta’s Transfusion — A Game Changer Model!! by Akash Goyal

Transfusion is a new model developed by Meta’s team for generating both text and images using a unified model. It is pretrained on an equal mix of text and image data, applying different objectives: next token prediction for text and diffusion for images. This article dives into its features and potential impact.

3. Local GraphRAG + Langchain + GPT+4o = Easy AI/Chat for your Docs by Gao Dalie

This tutorial shows how to create an AI for your PDF with local GraphRag, Langchain, and local LLM to make a powerful Agent Chatbot for your business or personal use. This innovative approach allows you to create an easy AI chat interface for your documents, making information retrieval seamless and efficient. Whether you’re managing extensive data or simply looking to enhance user interaction, this guide provides the tools you need to implement a local LLM solution.

4. How Does AI Work? Create a Neural Network from Scratch by Sean Jude Lyons

This step-by-step tutorial breaks down the fundamental concepts of neural networks, making it accessible for beginners and informative for seasoned developers. Learn about the architecture, training process, and practical applications of neural networks. By the end of this article, you’ll be able to build your own model and Machine Learning library to make predictions.

5. Revisiting Chunking in the RAG Pipeline by Florian June

This article revisits the importance of effective chunking strategies for improving information retrieval and enhancing model performance. By understanding how to optimize this process, you can significantly boost the efficiency of your AI applications. Whether you’re a researcher or a developer, this piece offers valuable perspectives on refining your approach to RAG.

If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI