#48 Interpretability Might Not Be What Society Is Looking for in AI

Author(s): Towards AI Editorial Team

Originally published on Towards AI.

Good morning, AI enthusiasts! This week, we are diving into some very interesting resources on the AI ‘black box problem’, interpretability, and AI decision-making. Parallely, we also dive into Anthropic’s new framework for assessing the risk of AI models sabotaging human efforts to control and evaluate them. There’s more on RAG, Collaboration of Experts, and other interesting collaboration opportunities. Enjoy the read!

What’s AI Weekly

This week in High Learning Rate, my other newsletter, we go back to the basics and explore the popular retrieval-augmented generation (RAG) method, introduced by a Meta paper in 2020. In one line, RAG answers the known limitations of LLMs, such as non-access to up-to-date information and hallucinations. Let’s dive into what it really is (simpler than you think), how it works, and when to use it (or not)!

— Louis-François Bouchard, Towards AI Co-founder & Head of Community

If you missed last week’s big update, here’s a quick reminder that Building LLMs for Production (second edition) is now available as an e-book at an exclusive price on Towards AI Academy!

PLUS, if you already have the first edition, you’re eligible for an additional discount for this second edition of the book (post-September 2024) — just reach out to [email protected] to upgrade affordably!

P.S. We will soon release an extremely in-depth ~90-lesson practical full stack “LLM Developer” conversion course. This new course is already available for pre-order on our new Towards AI Academy course platform.

Learn AI Together Community section!

AI poll of the week!

Many of you seem excited about ChatGPT’s web search capabilities. Personally, I’m also excited about an ad-free search experience, but curious to know your thoughts on ChatGPT automatically choosing to search the web based on what you ask. Let’s chat in the Discord thread!

Collaboration Opportunities

The Learn AI Together Discord community is flooding with collaboration opportunities. If you are excited to dive into applied AI, want a study partner, or even want to find a partner for your passion project, join the collaboration channel! Keep an eye on this section, too — we share cool opportunities every week!

1. Shreesha1573 is looking for teammates for a Kaggle competition with an understanding of RAG. If you are available this month, connect in the thread!

2. Lazybutlearning_44405 is looking for a study partner who wants to learn through practical projects using the Python framework. If you prefer this learning approach, reach out in the thread!

Meme of the week!

A special shoutout to ghost_in_the_machine for almost single-handedly keeping the meme channel active!

TAI Curated section

Article of the week

AI-Driven Decision Making: Comparing Markov Decision Process and Reinforcement Learning By Shenggang Li

This article explores the application of Markov Decision Processes (MDP) and Reinforcement Learning (RL) in decision-making, specifically focusing on the 2024 US presidential election. It clarifies the differences between MDP and RL and shows how they can be used to optimize campaign strategies and provides a step-by-step breakdown of how to model a campaign decision using MDP, including code examples for value iteration and policy iteration. It then introduces Q-learning, a type of RL, and demonstrates its application in a simulated campaign scenario. It also highlights ways to improve decision-making strategies through techniques like dynamic transition matrices, multi-agent MDPs, and machine learning for prediction.

Our must-read articles

1. Building Trustworthy AI: Interpretability in Vision and Linguistic Models By Rohan Vij

This article explores the challenges of the AI black box problem and the need for interpretable machine learning in computer vision and large language models. It highlights the dangers of using black box AI systems in critical applications and discusses techniques like LIME and Grad-CAM for enhancing model transparency. The article argues that while interpretability is valuable, societal trust in AI may be better served by building systems that act more like humans, exhibiting consistency and source attribution like LLMs with Retrieval-Augmented Generation (RAG) and emphasizing human-like traits over pure explainability, suggesting a shift in our interactions with AI towards a more intuitive and trust-based relationship.

2. CCoE: Approach to Mastering Multiple Domains with LLMs By Manpreet Singh

This article explores a framework called Collaboration of Experts (CCoE) that addresses the limitations of current LLMs in specialized domains. CCoE combines a general-purpose LLM (backbone model) with smaller, specialized expert models trained for specific fields. This allows the backbone model to use expert models when needed, achieving higher accuracy without sacrificing general knowledge or requiring extensive retraining. This approach is more efficient, scalable, and flexible, making it promising for future AI development.

3. Anthropic New Research Shows that AI Models Can Sabotage Human Evaluations By Jesus Rodriguez

This article explores the Anthropic’s Sabotage Evaluations, a framework designed to measure the potential of AI models to undermine human oversight. It defines key sabotage risks, such as a model’s ability to act independently, hide harmful actions, and introduce vulnerabilities, with simulated scenarios testing sabotage in areas like skewing research, misleading decisions, and inserting covert security flaws. It also explains early tests on Claude models show initial sabotage abilities, pointing to the need for advanced oversight strategies as AI capabilities evolve and become more sophisticated.

4. Build a Multilingual OCR and Translation App Using Pytesseract and Gemini API By BelovedWriter

This article guides you through building a multilingual OCR and translation app using Pytesseract and the Gemini API. It covers installing Tesseract, Pillow, and Pytesseract for text extraction from images and using the Gemini API for translation with prompt engineering. It also demonstrates integrating these features into a Streamlit app, enabling users to upload images, choose languages, view extracted text, and download translations, creating an interactive, user-friendly experience for multilingual OCR and translation.

If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI