This AI newsletter is all you need #63
Last Updated on November 6, 2023 by Editorial Team
Author(s): Towards AI Editorial Team
Originally published on Towards AI.
What happened this week in AI by Louie
This week in AI, we highlight further evidence of the adoption of Large Language Models (LLMs) driving significant revenue growth in the Western market, as well as the introduction of new AI models in China.
OpenAI has reportedly achieved a revenue run-rate of $1 billion, marking a substantial and rapid increase from the $28 million recorded in 2022. All eyes are also on Nvidia’s new H100 GPU chips, where the revenue run rate is reportedly now in the tens of billions. LLMs are no longer confined to niche research areas and ChatGPT has triggered a huge step change in demand for LLM products and infrastructure which is generating substantial revenue. This wave of interest has also triggered a significant influx of venture capital investments into the AI sector. However, it remains to be seen how quickly this wave of investment in AI APIs and infrastructure will translate into consumer products with lasting adoption.
Chinese tech companies and universities have also made substantial investments in AI in recent years and have made significant contributions to machine learning. However, unlike other nations, China mandates that companies undergo security assessments and obtain clearance for AI models. This policy had previously delayed the public release of LLM chatbots in China. Nevertheless, this week marked an important development as China approved the release of new models from 11 firms. We are watching to see the adoption of these products in China and how they will stack up against models from Western competitors.
While AI developments in China are promising, the US government has escalated sanctions on the supply of leading AI chips to China. This raises questions about the potential impact on progress. It remains unclear how extensive these restrictions will go and how Chinese AI projects will be able to compensate with less efficient chips, in-house chip designs, or Chinese chip fabrication, but we expect it to become an increasing challenge to train the most compute-intensive models going forward.
– Louie Peters — Towards AI Co-founder and CEO
This issue is brought to you thanks to invideo AI:
Turn Any Idea Into Video With One AI Tool
Imagine having a co-pilot to create videos — that’s invideo AI.
Transform your ideas into stunning visuals, instantly. Instruct the AI with text commands and get a ready-to-use video, including script, stock media, voiceover, and captions.
Make edits and adjustments with simple text prompts. Creation just got 100x easier for content creators, YouTubers, and marketers.
Test it out now — at no charge!
Hottest News
X’s recently updated privacy policy informed its users it would now collect biometric data as well as users’ job and education history to help train AI and machine learning models. Some reports speculate that Elon Musk may have acquired Twitter partially for its vast amount of proprietary training data, which could be used to train systems for his new AI company.
2. Google Goes All In on the AI Cloud
Google recently showcased an expanded ecosystem that includes both global systems integrators and smaller cloud service providers, promoting the widespread adoption of Google’s offerings worldwide. Although Google remains a distant third in the infrastructure-as-a-service/platform-as-a-service race, Google is adopting a long-term strategy and heavily investing in AI as a key driver of its cloud computing future.
3. WeatherBench 2: A Benchmark for the Next Generation of Data-driven Weather Models
Google has released the WeatherBench 2 dataset, which is gaining attention in the AI weather forecasting field. Machine learning methods are proving to be just as effective as physics-based models but with the advantage of faster forecasts using affordable hardware. The WeatherBench 2 dataset aims to enhance ML research in weather forecasting.
4. Ideogram Launches AI Image Generator With Impressive Typography
Ideogram is an alternative AI tool that excels in generating images with strong typography capabilities. It offers a unique feature of generating text within images, effectively addressing the common challenge faced by popular AI image generators.
5. AI Could Choke on Its Own Exhaust As It Fills the Web
Experts are predicting that AI-generated content will soon dominate the web, but this poses risks such as information overload and degradation. One notable concern is “Model Collapse,” wherein AI models overly rely on predictable word choices, potentially leading to a loss of their ability to produce engaging and nuanced content.
Five 5-minute reads/videos to keep you learning
If modern artificial intelligence were to have a founding document, it would be Google’s 2017 research paper, “Attention Is All You Need.” This article delves into the concept of Transformers, outlines their limitations, and discusses potential alternatives to overcome these shortcomings. It presents compelling alternatives supported by sound logic.
2. ChatGPT Glossary: 41 AI Terms That Everyone Should Know
With the pace of AI developments, it can be hard to keep up with the latest terminology. This glossary serves as a resource for those who are just being introduced to AI and for those looking for a reference or a vocabulary refresher.
3. Choosing the Right GPU Strategy for Your AI Project
This guide delves into the decision of whether to rent or buy a GPU for your AI project. Owning a GPU provides stability, whereas renting offers adaptability. Freelancers may find renting beneficial for short-term projects, while long-term researchers may discover ownership to be more cost-effective.
4. Options for Solving Hallucinations in Generative AI
This article tackles the issue of AI hallucinations in generative models and introduces a solution known as Retrieval Augmented Generation (RAG). RAG effectively mitigates hallucinations by retrieving pertinent context during data generation, making it a preferred choice for AI companies.
5. Vertex AI LLM Reinforcement Learning From Human Feedback
The article offers a comprehensive guide for implementing Reinforcement Learning with Human Feedback (RLHF) on Google Cloud utilizing Vertex AI.
Papers & Repositories
New findings from Google suggest that Reinforcement Learning from AI Feedback (RLAIF) could represent a viable alternative to Reinforcement Learning with high-quality human preference labels (RLHF). RLAIF employs an off-the-shelf language model to label preferences and has the potential to address scalability limitations and enhance the efficiency of LLMs.
2. YaRN: Efficient Context Window Extension of Large Language Models
This paper introduces YaRN, a computationally efficient method to extend the context window of such models, necessitating 10x fewer tokens and 2.5x fewer training steps than previous methods. It has been successfully employed to fine-tune LLaMa 2 7B and 13B models with context windows of 64k and 128k.
3. Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities
Alibaba Cloud introduces Qwen-VL, a powerful vision-language model designed for chat applications. Qwen-VL combines image and text inputs to generate accurate text and bounding box outputs, enhancing tasks such as image captioning, question answering, localization, and text-image understanding.
4. MedAlign: A Clinician-Generated Dataset for Instruction Following Electronic Medical Records
MedAlign is a valuable dataset comprising 983 instructions for analyzing Electronic Health Record (EHR) data. Researchers utilized this dataset to evaluate the performance of six general domain language models. They identified significant error rates, with GPT-4 exhibiting a 35% error rate, and MPT-7B-Instruct having a 68% error rate.
ChatDev is a virtual software company that operates through multiple intelligent agents, each assigned different roles, including Chief Executive Officer, Chief Technology Officer, Programmer, Tester, and more. These agents within ChatDev collaborate by engaging in specialized functional seminars, encompassing tasks like design, coding, testing, and documentation.
Enjoy these papers and news summaries? Get a daily recap in your inbox!
The Learn AI Together Community section!
Meme of the week!
Meme shared by neuralink
Featured Community post from the Discord
Well.job has introduced Multilingual Semantic Search, offering semantic search capabilities with open source or OpenAI. This solution is crafted to function with any language and offers a straightforward and efficient means to conduct semantic search similarity, employing the FAISS library across various languages. Check it out on GitHub and support a fellow community member. Share your questions and feedback in the thread here.
AI poll of the week!
Join the discussion on Discord.
TAI Curated section
Article of the week
10 Cool Things You Can Do With Embeddings! [Part 1] by Zubia Mansoor
Taking a step away from classical machine learning (ML), embeddings are at the core of most deep learning (DL) use cases. Having a grasp of this concept enables you to do flexible tasks with the feature space, and reframe ML/DL problems differently, especially with high-dimensional data in Computer Vision and Natural Language Processing.
Our must-read articles
Evolution in Your Code: Understanding and Coding Genetic Algorithm From Scratch — Part 1 by Francis Adrian Viernes
Reinforcement Learning: Dynamic Programming and Monte Carlo — Part 2 by Tan Pengshi Alvin
Machine Learning at Scale: Model v/s Data Parallelism by Shubham Saboo
Steepest Descent and Newton’s Method in Python, from Scratch: A Comparison by Nicolo Cosimo Albanese
If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.
Job offers
Team Lead, Software Engineering @Persado (Remote)
QA Software Testing Consultant — USA @ReVisionz (Remote)
Principal Software Engineer — Vue.JS @Creative Chaos (Remote)
Data Engineer @Mercari (Remote)
Data Research Intern @Clarity AI (Madrid, Spain)
Data Engineer @Decathlon (Singapore)
Software Developer Internships — Summer 2023/24 @Phocas Software (Christchurch, New Zealand)
Interested in sharing a job opportunity here? Contact [email protected].
If you are preparing your next machine learning interview, don’t hesitate to check out our leading interview preparation website, confetti!
Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.
Published via Towards AI
Related posts
Popular posts
Updates
Recent Posts
