Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Read by thought-leaders and decision-makers around the world. Phone Number: +1-650-246-9381 Email: pub@towardsai.net
228 Park Avenue South New York, NY 10003 United States
Website: Publisher: https://towardsai.net/#publisher Diversity Policy: https://towardsai.net/about Ethics Policy: https://towardsai.net/about Masthead: https://towardsai.net/about
Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Founders: Roberto Iriondo, , Job Title: Co-founder and Advisor Works for: Towards AI, Inc. Follow Roberto: X, LinkedIn, GitHub, Google Scholar, Towards AI Profile, Medium, ML@CMU, FreeCodeCamp, Crunchbase, Bloomberg, Roberto Iriondo, Generative AI Lab, Generative AI Lab Denis Piffaretti, Job Title: Co-founder Works for: Towards AI, Inc. Louie Peters, Job Title: Co-founder Works for: Towards AI, Inc. Louis-François Bouchard, Job Title: Co-founder Works for: Towards AI, Inc. Cover:
Towards AI Cover
Logo:
Towards AI Logo
Areas Served: Worldwide Alternate Name: Towards AI, Inc. Alternate Name: Towards AI Co. Alternate Name: towards ai Alternate Name: towardsai Alternate Name: towards.ai Alternate Name: tai Alternate Name: toward ai Alternate Name: toward.ai Alternate Name: Towards AI, Inc. Alternate Name: towardsai.net Alternate Name: pub.towardsai.net
5 stars – based on 497 reviews

Frequently Used, Contextual References

TODO: Remember to copy unique IDs whenever it needs used. i.e., URL: 304b2e42315e

Resources

Take our 85+ lesson From Beginner to Advanced LLM Developer Certification: From choosing a project to deploying a working product this is the most comprehensive and practical LLM course out there!

Publication

LLM-Powered Product Discovery: A Leap Beyond Hybrid Search
Latest   Machine Learning

LLM-Powered Product Discovery: A Leap Beyond Hybrid Search

Last Updated on August 1, 2023 by Editorial Team

Author(s): Maithri Vm

Originally published on Towards AI.

Photo by Markus Winkler on Unsplash

In this rapidly evolving era of cutting-edge technology, the world is immersed in the wave of LLM, churning novel ideas that revolutionize our daily lives. As the internet teems with a plethora of solution designs and niche possibilities, we find ourselves on the cusp of a transformative digital landscape.

This blog series embarks on an exploration of Product/Content discovery, striving not only to offer valuable perspectives but also to shed light on practical limitations within this realm. As emphasized throughout the series, context plays a pivotal role in this domain, and we have delved into the efficacy and mechanics of both semantic search and hybrid search systems.

Context is the King — The Art and Science of Content Discovery

— Exploring the power of Transformers, GPT, and Hybrid Search engines to revolutionise content discovery

medium.com

Hybrid Search — Amalgamation of Sparse and Dense vector representations for Active Content…

— Uniting meaning of data with metadata to leverage deeper context

medium.com

Additionally, the series has briefly touched upon the redefinition of Content discovery and Product search through the potent capabilities of LLM (Language Model-based Learning). In this particular post, our focus will be on LLM powered discovery, specifically targeting active search/explicit discovery to bridge the gap between hybrid search and LLM search paradigms.

By now, it should be evident that the true value of LLM lies in its remarkable capacity for generalization and its ability to apply knowledge in a broader worldly context, bringing us closer to achieving impressive outcomes via more natural interactions. Context continues to be a significant factor in the discovery process, encompassing not only the deep semantic micro-context of the content but also the vital aspect of developing an awareness of the evolving user context over time.

From Content to Knowledge Discovery and From Search Filters to Natural Language — Unleashing the…

If you have been following along my blog series starting from “Context is the king!” all the way to “From keywords to…

medium.com

As I delve deeper into the subject, I uncover several crucial contributions of LLM to the realm of discovery, particularly in the context of active search:

Knowledge-driven Content discovery using LLM — LLMs with the RAG (Retrieval Augmented Generation) pipeline possesses the capability to leverage the rich underlying knowledge present in the repository of documents efficiently. Through this, one can effectively surface relevant knowledge alongside the source documents. This revolutionary approach of Knowledge-led Content Discovery holds immense applicability in document-driven applications such as learning platforms like Coursera, Duo lingo and online content management systems like Medium, YouTube, and WordPress. Moreover, it extends its benefits to Enterprise apps like Sharepoint and Confluence, which host repositories of essential business documents.

An example scenario could be :
User: “Can I carry forward all my pending earned leaves to the next year?”
Bot: “Yes, you can carry forward upto 10 earned leaves across next year. Refer these documents for more details:”
Oranization_Leave_Policy_2023.pdf

By tapping into the inherent micro-semantic context within the content, this approach opens up new horizons for enhanced information retrieval and more insightful content discovery.

Metadata-driven Product /People discovery with LLM — Whether it is rich product attributes in an e-commerce platform or meaningful user profile attributes in a social media app, the abundance of metadata can significantly enhance contextual relevance within the broader spectrum offered by LLM. This convergence of entity attributes with content resembles the capabilities of a Hybrid search system, harnessing the combined power of semantic and literal textual contexts. Check my earlier blogposts that explore this aspect.

To effectively cater to hybrid search with the augmented semantic context in query and search filters, the use of Vector databases like Pinecone, Weaviate, or Solr becomes essential.

The integration of LLM can elevate the search experience to newer heights, capitalizing on the advantages it brings to the table. Here are a few such ideas that could potentially manifest:

a. Improved semantic representation with LLM: GPT models offer superior contextual understanding compared to BERT. While GPT models, including OpenAI’s GPT-3, are based on transformer architecture and predict the next word in a sequence, BERT focuses on fixed-length word or sentence representations without context. This difference makes GPT models more coherent and contextually appropriate in generating text, considering the entire preceding context for predictions.

b. Extensive product metadata enriched by LLM: LLMs demonstrate an exceptional capability to generate comprehensive and natural language product descriptions, encompassing all relevant metadata. For example :

Sample description in the search index for a kitchen product:
Introducing the ToastMaster 3000 — an efficient and sleek toaster with advanced toasting technology, wide slots, and a powerful 1200-watt heating element. Available in Classic Stainless Steel, perfect kitchen appliance with easy maintenance. Upgrade your breakfast routine today!

This approach proves highly advantageous for mapping user queries, especially those that are ambiguous or expressed in plain English, without specific acceptable values for filtering. By adopting this approach and eliminating cumbersome advanced search filters, the search experience becomes seamless and user-friendly. Users can effortlessly find precisely what they are looking for, even in complex and uncertain search scenarios. The use of natural language representation empowers smooth user interaction, making product discovery a breeze and enhancing overall user satisfaction.

c. Query expansion with LLM: Indeed, LLMs can play a pivotal role in generating descriptive strings that expand upon a user’s query. When the objective is content discovery, LLMs can go a step further by mining the inherent user interests, topics, or concept ideas. These mined insights can then be concatenated to the user’s original query, leading to a more comprehensive and extensive search coverage.

The user’s original query could go like:
easy to clean, high capacity small toaster
LLM-enhanced query could look like this:
A small toaster that requires minimal maintenance while offering a high toasting capacity for efficient and convenient breakfast preparation.

By incorporating these additional elements, LLMs facilitate a wider exploration of relevant content, ensuring that users are presented with a diverse array of results that align closely with their preferences and interests. This approach significantly enhances the content discovery process, providing users with a more personalized and enriching search experience.

Reranking the search results with LLM— Indeed, there are active projects where LLMs can be employed to generate relevance rankings for surfaced candidate results. Leveraging the LLM’s powerful ability to holistically generalize user queries against candidate descriptions and reason their relevance, these projects aim to enhance the accuracy and precision of search results. By utilizing the knowledge captured by LLMs, the ranking process becomes more informed and contextually aware, resulting in a more refined list of relevant results tailored to match the user’s intent.

Fully Natural Interface enabled discovery with LLMs— This can be seen as a bold leap towards transcending the frictions caused by traditional search boxes, filters, and pagination controls. Embracing a natural language interface, the “Talk to your data” approach, empowered by LLMs, takes center stage. Leveraging LLMs’ incredible generative capabilities, the search agent generates queries to interact with databases and retrieves the most relevant results, matching user queries and context provided during the conversation. Say farewell to the constraints of conventional search interfaces, and welcome a new era of intuitive and conversational exploration!

Example CUI-based discovery could like this:
User: I am looking for toaster for my kitchen
Bot generated query: product_category : kitchen appliance & product_type: toaster
<retrieved results shown to user>
User: show me only smallest size
Bot generated query: product_category : kitchen appliance & product_type: toaster & slot_size:2
<retrieved matching results shown to user>
User: hope these come with low wattage?
Bot generated query: product_category : kitchen appliance & product_type: toaster & slot_size:2 & current_capacity: 800 watts
<retrieved matching results shown to user>

Note: Observe the user’s query context is building through the interaction!

Implementation details for LLM-powered product discovery in Conversational User interface (CUI):

Talk to the data in natural language:

Regardless of whether the data is stored in a vector database or any other structured database, LLMs have the remarkable capability to generate queries for it based on natural language context. The key advantage of this approach is the ability to incrementally expand the query context during a conversation. By employing simple prompt engineering with few-shot examples, high-quality queries can be generated, which can automatically retrieve relevant results from the database.

Additionally, incorporating historical conversation as a memory to the prompt allows for the synthesis of comprehensive queries, encompassing all query context augmented by the user’s query and expressed filters to fine-tune the result set. This flexible and dynamic query generation process facilitates a more intuitive and conversational interaction with the data, unlocking a new level of contextual understanding and relevance.

Let us look at the detailed steps involved with implementation:

  1. Data Preprocessing: Generate comprehensive product summary in natural text, which encompasses all the metadata. Using LLM for this step is also imperative, given its proven dominance.
  2. Generating semantic embeddings powered by the GPT model. Generating dense vector representation for the product data (description) as well as the auto-generated summary (as seen in step#1). Choice of OpenAI embedding endpoint (text-embedding-ada-002) or using Sentence transformer (all-MiniLM-L6-v2) by Huggingface are favorite options amongst developers.
  3. Indexing the generated semantic embeddings (from earlier step) in vector db of choice like Pinecone, ChromaDb, Apache Solr etc. This must also consider metadata for ex : product category, genre, locality, ratings, etc associated with the document vector.
  4. Building query synthesiser which accepts natural language input from the user that produces the vector store queries as guided by the prompt (containing schema as well as few shot template as examples) as well as contextual memory. Additionally, one can also append topics extracted from the user’s query to infer her intent.
  5. Query Re-evaluation : The generated queries can be re-validated for possible errors, as well run through auto-correction by having another LLMs task dedicated for this step.
  6. Retrieve the results from the vector database, matching the queries generated and pre-validated by LLM.
  7. Present the results to the user while also adding the user’s further inputs and feedback to the conversation memory.

And the cycle through steps 4 to 7 as the interaction progress.

Retaining user inputs, including queries, filters, and feedback, within the memory is of utmost importance to extend the context of the user’s session. As user interacts with the search results and refine his queries, the valuable long-range context stored in memory enables tailoring responses to evolving requirements more effectively.

Indeed, while the concept may appear straightforward, I can assure you that hands-on experience with this approach involves a significant learning curve. Despite the impressive and promising results, my personal experience has uncovered certain challenges and imperfections. Let me share my observations, detailing the pros and cons of this approach to provide a more comprehensive understanding of its practicality and potential pitfalls.

Advantages of LLM-powered conversation search:

  1. The primary offering of LLM-powered search lies in its exceptional ability to deeply understand the user’s intent through inherent context. This understanding goes beyond merely interpreting the user’s descriptive requirements; it extends to efficiently interlacing these requirements with diverse product contexts, spanning unstructured, semi-structured, and structured data. The versatility and contextual prowess of LLMs make them an indispensable tool for unlocking the true potential of data exploration and enriching the user search experience.
  2. User interaction is completely in a natural format, which eliminates the hassle of dealing with menus, filters and paginated scrolling. The user’s expectation /query context builds over time across multiple turns of conversation and feedback provided by the user in the CUI. That way user does not have to enter keywords & filter repeatedly (which often leads to frustrations); rather the LLM search agent can do the heavy lifting of developing contextual understanding over ongoing interactions.
  3. LLM powered search exhibits flexibility to adapt to evolving search requirements —either due to the product schema updates or due to the changes related to UX. CUI allows for a seamless and dynamic discovery process and more importantly, the responsibility of updates primarily lies with the LLM model employed by the search agent.

And are there any caveats with this approach?

  1. Scalability and Prompt maintenance overhead: Exposing schema as part of the prompt can lead to token leakage due to the limited context length handled by LLMs.
  2. Maintenance overhead: Continuous updates to the prompts to keep them in sync with evolving schema is another unavoidable chore. To mitigate these issues, alternative strategies for handling schema and prompt management may be explored, ensuring a more seamless and robust LLM-powered search experience.
  3. Query quality with complex schema: The quality of generated query is most efficient when the data is spread across only a limited number of tables /collections. While it is easy to convert schema to denormalise the structure and have all elements part of single /fewer number of tables or collections, ongoing updates to vector databases in still not performance efficient. Continuous maintenance of transactional data elements (like price, availability, etc) is still going to be ineffective in vector databases.
  4. Query inaccuracies: Despite explicit instructions in the prompt and elaborate samples in a few shot templates, however, LLM-produced queries are still not devoid of errors. As erroneous queries lead to unfavorable outcomes upon database interaction, it is highly important to re-check the LLM-generated queries. Re-evaluating with CoT (chain of thought) prompting and auto-correcting the generated queries with another round LLM call is an alternate approach that can be considered. SelfQueryRetriever from Langchain is one such method that employs this approach.
  5. Increased latency: LLMs are large and complex language models, and their extensive computations require significant processing time, leading to increased latency. Owing to model size, token limitations, and resource constraints can often lead to increased delays in retrieving the obvious results.
  6. Risks to data privacy: It is imperative to evaluate the choice of LLM services with data security concerns in mind. While using off-the-shelf LLM services seems most reliable, the potential risk of sensitive information exposure and data breaches become a growing concern. Hosting open-source LLMs in a Virtual Private Cloud (VPC) can be an effective measure to meet the data privacy and security requirements.
  7. Increased cost: Whether opting for LLMs as a paid service or hosting customized LLMs on their own infrastructure, it’s crucial to acknowledge their resource-intensive nature, which directly impacts costs.

Due to these challenges, I have been consistently adjusting the solution design to create a text-to-database interface that comes close to perfection. Indeed, this is an opportune moment to unleash creativity and explore the vast possibilities of LLM.

There is one such idea that I have evaluated, which eliminates the need for maintaining categorical metadata and risks the outcomes to quality of queries generated by LLM —

Converting the entire product attributes set into comprehensive natural language descriptions, specifically for search purposes, and treating search queries as purely unstructured document retrieval is a compelling option to consider. This approach can simplify the search process and offer more intuitive interactions for users, leading to enhanced discoverability and user satisfaction. Another alternative, albeit challenging, is to develop independent LLM fine-tuned models for each contextual signal.

The above approach not only removes the dependency on queries to retrieve product data, it also offers a better generalization of these very categorical attributes in a worldly context with the power of LLM. A possible approach for this mechanism (as I have also quoted in my earlier post) could employ an LLM-generated product summary in the search database. And this may look like this:

Introducing our collection of enchanting blue party skirts designed specifically for 13-year-olds. Crafted with care and attention to detail, these skirts are the epitome of youthful style and sophistication. The vibrant blue hue adds a touch of vibrancy to any ensemble, making it ideal for special occasions, celebrations, or simply dressing up for a fun day out. With their comfortable fit and flattering silhouette, these skirts will make your young fashionista feel confident and ready to shine. Let her showcase her unique sense of style with these charming blue party skirts.

End user querying ‘sea blue teen skirts for birthday’ can fetch not only the above product but also all the relevant items matching similar intent!

In the midst of this thrilling Generative AI revolution, striking a balance between creativity and a pragmatic gain-to-pain ratio evaluation becomes paramount. By adopting a well-balanced approach, businesses can harness the vast potential of LLMs and make informed decisions that pave the way for successful and cost-effective solutions.

As I immerse in the myriad aspects of product discovery, I explore and generate innovative ideas for harnessing the capabilities of LLM; I eagerly look forward to having your perspectives or counterarguments, as it enriches our understanding and fosters a more comprehensive exploration of this exciting field! Stay tuned and subscribe for more captivating ideas and approaches in my upcoming posts!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI

Feedback ↓

Sign Up for the Course
`; } else { console.error('Element with id="subscribe" not found within the page with class "home".'); } } }); // Remove duplicate text from articles /* Backup: 09/11/24 function removeDuplicateText() { const elements = document.querySelectorAll('h1, h2, h3, h4, h5, strong'); // Select the desired elements const seenTexts = new Set(); // A set to keep track of seen texts const tagCounters = {}; // Object to track instances of each tag elements.forEach(el => { const tagName = el.tagName.toLowerCase(); // Get the tag name (e.g., 'h1', 'h2', etc.) // Initialize a counter for each tag if not already done if (!tagCounters[tagName]) { tagCounters[tagName] = 0; } // Only process the first 10 elements of each tag type if (tagCounters[tagName] >= 2) { return; // Skip if the number of elements exceeds 10 } const text = el.textContent.trim(); // Get the text content const words = text.split(/\s+/); // Split the text into words if (words.length >= 4) { // Ensure at least 4 words const significantPart = words.slice(0, 5).join(' '); // Get first 5 words for matching // Check if the text (not the tag) has been seen before if (seenTexts.has(significantPart)) { // console.log('Duplicate found, removing:', el); // Log duplicate el.remove(); // Remove duplicate element } else { seenTexts.add(significantPart); // Add the text to the set } } tagCounters[tagName]++; // Increment the counter for this tag }); } removeDuplicateText(); */ // Remove duplicate text from articles function removeDuplicateText() { const elements = document.querySelectorAll('h1, h2, h3, h4, h5, strong'); // Select the desired elements const seenTexts = new Set(); // A set to keep track of seen texts const tagCounters = {}; // Object to track instances of each tag // List of classes to be excluded const excludedClasses = ['medium-author', 'post-widget-title']; elements.forEach(el => { // Skip elements with any of the excluded classes if (excludedClasses.some(cls => el.classList.contains(cls))) { return; // Skip this element if it has any of the excluded classes } const tagName = el.tagName.toLowerCase(); // Get the tag name (e.g., 'h1', 'h2', etc.) // Initialize a counter for each tag if not already done if (!tagCounters[tagName]) { tagCounters[tagName] = 0; } // Only process the first 10 elements of each tag type if (tagCounters[tagName] >= 10) { return; // Skip if the number of elements exceeds 10 } const text = el.textContent.trim(); // Get the text content const words = text.split(/\s+/); // Split the text into words if (words.length >= 4) { // Ensure at least 4 words const significantPart = words.slice(0, 5).join(' '); // Get first 5 words for matching // Check if the text (not the tag) has been seen before if (seenTexts.has(significantPart)) { // console.log('Duplicate found, removing:', el); // Log duplicate el.remove(); // Remove duplicate element } else { seenTexts.add(significantPart); // Add the text to the set } } tagCounters[tagName]++; // Increment the counter for this tag }); } removeDuplicateText(); //Remove unnecessary text in blog excerpts document.querySelectorAll('.blog p').forEach(function(paragraph) { // Replace the unwanted text pattern for each paragraph paragraph.innerHTML = paragraph.innerHTML .replace(/Author\(s\): [\w\s]+ Originally published on Towards AI\.?/g, '') // Removes 'Author(s): XYZ Originally published on Towards AI' .replace(/This member-only story is on us\. Upgrade to access all of Medium\./g, ''); // Removes 'This member-only story...' }); //Load ionic icons and cache them if ('localStorage' in window && window['localStorage'] !== null) { const cssLink = 'https://code.ionicframework.com/ionicons/2.0.1/css/ionicons.min.css'; const storedCss = localStorage.getItem('ionicons'); if (storedCss) { loadCSS(storedCss); } else { fetch(cssLink).then(response => response.text()).then(css => { localStorage.setItem('ionicons', css); loadCSS(css); }); } } function loadCSS(css) { const style = document.createElement('style'); style.innerHTML = css; document.head.appendChild(style); } //Remove elements from imported content automatically function removeStrongFromHeadings() { const elements = document.querySelectorAll('h1, h2, h3, h4, h5, h6, span'); elements.forEach(el => { const strongTags = el.querySelectorAll('strong'); strongTags.forEach(strongTag => { while (strongTag.firstChild) { strongTag.parentNode.insertBefore(strongTag.firstChild, strongTag); } strongTag.remove(); }); }); } removeStrongFromHeadings(); "use strict"; window.onload = () => { /* //This is an object for each category of subjects and in that there are kewords and link to the keywods let keywordsAndLinks = { //you can add more categories and define their keywords and add a link ds: { keywords: [ //you can add more keywords here they are detected and replaced with achor tag automatically 'data science', 'Data science', 'Data Science', 'data Science', 'DATA SCIENCE', ], //we will replace the linktext with the keyword later on in the code //you can easily change links for each category here //(include class="ml-link" and linktext) link: 'linktext', }, ml: { keywords: [ //Add more keywords 'machine learning', 'Machine learning', 'Machine Learning', 'machine Learning', 'MACHINE LEARNING', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, ai: { keywords: [ 'artificial intelligence', 'Artificial intelligence', 'Artificial Intelligence', 'artificial Intelligence', 'ARTIFICIAL INTELLIGENCE', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, nl: { keywords: [ 'NLP', 'nlp', 'natural language processing', 'Natural Language Processing', 'NATURAL LANGUAGE PROCESSING', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, des: { keywords: [ 'data engineering services', 'Data Engineering Services', 'DATA ENGINEERING SERVICES', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, td: { keywords: [ 'training data', 'Training Data', 'training Data', 'TRAINING DATA', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, ias: { keywords: [ 'image annotation services', 'Image annotation services', 'image Annotation services', 'image annotation Services', 'Image Annotation Services', 'IMAGE ANNOTATION SERVICES', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, l: { keywords: [ 'labeling', 'labelling', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, pbp: { keywords: [ 'previous blog posts', 'previous blog post', 'latest', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, mlc: { keywords: [ 'machine learning course', 'machine learning class', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, }; //Articles to skip let articleIdsToSkip = ['post-2651', 'post-3414', 'post-3540']; //keyword with its related achortag is recieved here along with article id function searchAndReplace(keyword, anchorTag, articleId) { //selects the h3 h4 and p tags that are inside of the article let content = document.querySelector(`#${articleId} .entry-content`); //replaces the "linktext" in achor tag with the keyword that will be searched and replaced let newLink = anchorTag.replace('linktext', keyword); //regular expression to search keyword var re = new RegExp('(' + keyword + ')', 'g'); //this replaces the keywords in h3 h4 and p tags content with achor tag content.innerHTML = content.innerHTML.replace(re, newLink); } function articleFilter(keyword, anchorTag) { //gets all the articles var articles = document.querySelectorAll('article'); //if its zero or less then there are no articles if (articles.length > 0) { for (let x = 0; x < articles.length; x++) { //articles to skip is an array in which there are ids of articles which should not get effected //if the current article's id is also in that array then do not call search and replace with its data if (!articleIdsToSkip.includes(articles[x].id)) { //search and replace is called on articles which should get effected searchAndReplace(keyword, anchorTag, articles[x].id, key); } else { console.log( `Cannot replace the keywords in article with id ${articles[x].id}` ); } } } else { console.log('No articles found.'); } } let key; //not part of script, added for (key in keywordsAndLinks) { //key is the object in keywords and links object i.e ds, ml, ai for (let i = 0; i < keywordsAndLinks[key].keywords.length; i++) { //keywordsAndLinks[key].keywords is the array of keywords for key (ds, ml, ai) //keywordsAndLinks[key].keywords[i] is the keyword and keywordsAndLinks[key].link is the link //keyword and link is sent to searchreplace where it is then replaced using regular expression and replace function articleFilter( keywordsAndLinks[key].keywords[i], keywordsAndLinks[key].link ); } } function cleanLinks() { // (making smal functions is for DRY) this function gets the links and only keeps the first 2 and from the rest removes the anchor tag and replaces it with its text function removeLinks(links) { if (links.length > 1) { for (let i = 2; i < links.length; i++) { links[i].outerHTML = links[i].textContent; } } } //arrays which will contain all the achor tags found with the class (ds-link, ml-link, ailink) in each article inserted using search and replace let dslinks; let mllinks; let ailinks; let nllinks; let deslinks; let tdlinks; let iaslinks; let llinks; let pbplinks; let mlclinks; const content = document.querySelectorAll('article'); //all articles content.forEach((c) => { //to skip the articles with specific ids if (!articleIdsToSkip.includes(c.id)) { //getting all the anchor tags in each article one by one dslinks = document.querySelectorAll(`#${c.id} .entry-content a.ds-link`); mllinks = document.querySelectorAll(`#${c.id} .entry-content a.ml-link`); ailinks = document.querySelectorAll(`#${c.id} .entry-content a.ai-link`); nllinks = document.querySelectorAll(`#${c.id} .entry-content a.ntrl-link`); deslinks = document.querySelectorAll(`#${c.id} .entry-content a.des-link`); tdlinks = document.querySelectorAll(`#${c.id} .entry-content a.td-link`); iaslinks = document.querySelectorAll(`#${c.id} .entry-content a.ias-link`); mlclinks = document.querySelectorAll(`#${c.id} .entry-content a.mlc-link`); llinks = document.querySelectorAll(`#${c.id} .entry-content a.l-link`); pbplinks = document.querySelectorAll(`#${c.id} .entry-content a.pbp-link`); //sending the anchor tags list of each article one by one to remove extra anchor tags removeLinks(dslinks); removeLinks(mllinks); removeLinks(ailinks); removeLinks(nllinks); removeLinks(deslinks); removeLinks(tdlinks); removeLinks(iaslinks); removeLinks(mlclinks); removeLinks(llinks); removeLinks(pbplinks); } }); } //To remove extra achor tags of each category (ds, ml, ai) and only have 2 of each category per article cleanLinks(); */ //Recommended Articles var ctaLinks = [ /* ' ' + '

Subscribe to our AI newsletter!

' + */ '

Take our 85+ lesson From Beginner to Advanced LLM Developer Certification: From choosing a project to deploying a working product this is the most comprehensive and practical LLM course out there!

'+ '

Towards AI has published Building LLMs for Production—our 470+ page guide to mastering LLMs with practical projects and expert insights!

' + '
' + '' + '' + '

Note: Content contains the views of the contributing authors and not Towards AI.
Disclosure: This website may contain sponsored content and affiliate links.

' + 'Discover Your Dream AI Career at Towards AI Jobs' + '

Towards AI has built a jobs board tailored specifically to Machine Learning and Data Science Jobs and Skills. Our software searches for live AI jobs each hour, labels and categorises them and makes them easily searchable. Explore over 10,000 live jobs today with Towards AI Jobs!

' + '
' + '

🔥 Recommended Articles 🔥

' + 'Why Become an LLM Developer? Launching Towards AI’s New One-Stop Conversion Course'+ 'Testing Launchpad.sh: A Container-based GPU Cloud for Inference and Fine-tuning'+ 'The Top 13 AI-Powered CRM Platforms
' + 'Top 11 AI Call Center Software for 2024
' + 'Learn Prompting 101—Prompt Engineering Course
' + 'Explore Leading Cloud Providers for GPU-Powered LLM Training
' + 'Best AI Communities for Artificial Intelligence Enthusiasts
' + 'Best Workstations for Deep Learning
' + 'Best Laptops for Deep Learning
' + 'Best Machine Learning Books
' + 'Machine Learning Algorithms
' + 'Neural Networks Tutorial
' + 'Best Public Datasets for Machine Learning
' + 'Neural Network Types
' + 'NLP Tutorial
' + 'Best Data Science Books
' + 'Monte Carlo Simulation Tutorial
' + 'Recommender System Tutorial
' + 'Linear Algebra for Deep Learning Tutorial
' + 'Google Colab Introduction
' + 'Decision Trees in Machine Learning
' + 'Principal Component Analysis (PCA) Tutorial
' + 'Linear Regression from Zero to Hero
'+ '

', /* + '

Join thousands of data leaders on the AI newsletter. It’s free, we don’t spam, and we never share your email address. Keep up to date with the latest work in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

',*/ ]; var replaceText = { '': '', '': '', '
': '
' + ctaLinks + '
', }; Object.keys(replaceText).forEach((txtorig) => { //txtorig is the key in replacetext object const txtnew = replaceText[txtorig]; //txtnew is the value of the key in replacetext object let entryFooter = document.querySelector('article .entry-footer'); if (document.querySelectorAll('.single-post').length > 0) { //console.log('Article found.'); const text = entryFooter.innerHTML; entryFooter.innerHTML = text.replace(txtorig, txtnew); } else { // console.log('Article not found.'); //removing comment 09/04/24 } }); var css = document.createElement('style'); css.type = 'text/css'; css.innerHTML = '.post-tags { display:none !important } .article-cta a { font-size: 18px; }'; document.body.appendChild(css); //Extra //This function adds some accessibility needs to the site. function addAlly() { // In this function JQuery is replaced with vanilla javascript functions const imgCont = document.querySelector('.uw-imgcont'); imgCont.setAttribute('aria-label', 'AI news, latest developments'); imgCont.title = 'AI news, latest developments'; imgCont.rel = 'noopener'; document.querySelector('.page-mobile-menu-logo a').title = 'Towards AI Home'; document.querySelector('a.social-link').rel = 'noopener'; document.querySelector('a.uw-text').rel = 'noopener'; document.querySelector('a.uw-w-branding').rel = 'noopener'; document.querySelector('.blog h2.heading').innerHTML = 'Publication'; const popupSearch = document.querySelector$('a.btn-open-popup-search'); popupSearch.setAttribute('role', 'button'); popupSearch.title = 'Search'; const searchClose = document.querySelector('a.popup-search-close'); searchClose.setAttribute('role', 'button'); searchClose.title = 'Close search page'; // document // .querySelector('a.btn-open-popup-search') // .setAttribute( // 'href', // 'https://medium.com/towards-artificial-intelligence/search' // ); } // Add external attributes to 302 sticky and editorial links function extLink() { // Sticky 302 links, this fuction opens the link we send to Medium on a new tab and adds a "noopener" rel to them var stickyLinks = document.querySelectorAll('.grid-item.sticky a'); for (var i = 0; i < stickyLinks.length; i++) { /* stickyLinks[i].setAttribute('target', '_blank'); stickyLinks[i].setAttribute('rel', 'noopener'); */ } // Editorial 302 links, same here var editLinks = document.querySelectorAll( '.grid-item.category-editorial a' ); for (var i = 0; i < editLinks.length; i++) { editLinks[i].setAttribute('target', '_blank'); editLinks[i].setAttribute('rel', 'noopener'); } } // Add current year to copyright notices document.getElementById( 'js-current-year' ).textContent = new Date().getFullYear(); // Call functions after page load extLink(); //addAlly(); setTimeout(function() { //addAlly(); //ideally we should only need to run it once ↑ }, 5000); }; function closeCookieDialog (){ document.getElementById("cookie-consent").style.display = "none"; return false; } setTimeout ( function () { closeCookieDialog(); }, 15000); console.log(`%c 🚀🚀🚀 ███ █████ ███████ █████████ ███████████ █████████████ ███████████████ ███████ ███████ ███████ ┌───────────────────────────────────────────────────────────────────┐ │ │ │ Towards AI is looking for contributors! │ │ Join us in creating awesome AI content. │ │ Let's build the future of AI together → │ │ https://towardsai.net/contribute │ │ │ └───────────────────────────────────────────────────────────────────┘ `, `background: ; color: #00adff; font-size: large`); //Remove latest category across site document.querySelectorAll('a[rel="category tag"]').forEach(function(el) { if (el.textContent.trim() === 'Latest') { // Remove the two consecutive spaces (  ) if (el.nextSibling && el.nextSibling.nodeValue.includes('\u00A0\u00A0')) { el.nextSibling.nodeValue = ''; // Remove the spaces } el.style.display = 'none'; // Hide the element } }); // Add cross-domain measurement, anonymize IPs 'use strict'; //var ga = gtag; ga('config', 'G-9D3HKKFV1Q', 'auto', { /*'allowLinker': true,*/ 'anonymize_ip': true/*, 'linker': { 'domains': [ 'medium.com/towards-artificial-intelligence', 'datasets.towardsai.net', 'rss.towardsai.net', 'feed.towardsai.net', 'contribute.towardsai.net', 'members.towardsai.net', 'pub.towardsai.net', 'news.towardsai.net' ] } */ }); ga('send', 'pageview'); -->