Learn AI Together — Towards AI Community Newsletter #20

Author(s): Towards AI Editorial Team

Originally published on Towards AI.

Good morning, AI enthusiasts! This week, we dive into infinity and beyond; with Google’s most recent paper, Infini-attention, context windows are no longer a problem. And, of course, we also have some exciting projects, fun memes, and hands-on blogs. Enjoy the read!

Paper Walkthrough: RAG for Knowledge-Intensive NLP Tasks

The Learn AI Together Discord community hosts AI seminars to help the community learn from industry experts, ask questions, and get a deeper insight into the latest research in AI. Join us for free, interactive video sessions hosted live on Discord by attending our upcoming event.

This week, we have a paper walkthrough for the research paper on Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Join us for a deeper insight into the prominent research in AI.

Join us next Saturday, April 27, for the RAG paper walkthrough on the Learn AI Together Discord server!

What’s AI Weekly

In LLMs, the bigger the context window, the more words you can send; the more context it can grasp, the better the understanding of your question and, thus, a better answer. The problem is that the language models’ performances dramatically decrease along with the context increase. Often, the more words it sees, the worse the results. Thanks to their new paper and approach, Infini-attention, Google made the attention mechanism much more manageable for such large contexts of millions of words. I shared an overview of this exciting new research paper and approach in the article. Read the complete article or watch it on YouTube!

-Louis-François Bouchard, Towards AI Co-founder & Head of Community

Learn AI Together Community section!

Featured Community post from the Discord

Edoardo022 has created RicercaMente, a project dedicated to mapping the evolution of data science through significant scientific papers published over the years. It traces the history of significant research, starting with the paper ‘On the Theory of Games of Strategy’ released in 1928 to ‘An Image is worth 16×16 words: Transformers for Image Recognition at scale’ released in 2021. You can contribute to the project on GitHub and support a fellow community member. Share your feedback and questions in the thread!

AI poll of the week!

We love RAG and believe it has a promising future, but the Infini-attention approach seems like an interesting development in AI architecture. It transforms the computational cost from quadratic to linear with respect to sequence length through key modifications to the attention mechanism. Have you read the paper yet? And do you think it will replace RAG? Share your thoughts in the Discord thread!

Collaboration Opportunities

The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, join the collaboration channel! Keep an eye on this section, too — we share cool opportunities every week!

1. Uknowwhoab1r is looking for a team to build a Code Pattern Analyst. This tool, designed with GPT-4, dissects and recognizes design patterns within C/C++ and JavaScript code snippets. If you would like to collaborate on the project, reach out to him in the thread!

2. R.raviyuvaraj and his team are building a real-time AI project. Currently, they are welcoming individuals with all levels of expertise in AI/ML for collaborative practice. If this sounds interesting, connect with him in the thread!

Meme of the week!

Meme shared by ghost_in_the_machine

TAI Curated section

Article of the week

RAG in Production: Chunking Decisions by Mandar Karhade, MD. PhD.

In technical terms, “Chunking” refers to segmenting the large corpus of documents into smaller, more manageable pieces that can be efficiently retrieved and processed by the model. The strategy for chunking is critical for RAG. In this article, we will consider retrieval only from the point of view of its implementation in the generative model context.

Our must-read articles

1. From Development to Deployment of an AI Model Using Azure by Prashant Kalepu

This article discusses the often overlooked but incredibly crucial aspect of Building ML models, i.e., Deployment. Deploying your machine learning models with a front end opens up a world of possibilities. It allows you to share your creations, showcase your skills to potential employers, and even contribute to solving real-world problems.

2. Tuning Word2Vec with Bayesian Optimization: Applied to Music Recommendations by Jimmy Jarjoura

This article explores how to leverage Bayesian optimization for tuning Word2Vec in the context of music recommendation systems. It also shows the performance improvements achieved from offline and online experiments.

3. Mixture of Experts by Louis-François Bouchard

What you know about Mixture of Experts is wrong. We are not using this technique because each model is an expert on a specific topic. Each of these so-called experts is not an individual model but something much simpler. In this article, the author dives into a compelling MoE model, Mixtral 8x7B.

If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.

Think a friend would enjoy this too? Share the newsletter and let them join the conversation.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI