Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Read by thought-leaders and decision-makers around the world. Phone Number: +1-650-246-9381 Email: pub@towardsai.net
228 Park Avenue South New York, NY 10003 United States
Website: Publisher: https://towardsai.net/#publisher Diversity Policy: https://towardsai.net/about Ethics Policy: https://towardsai.net/about Masthead: https://towardsai.net/about
Name: Towards AI Legal Name: Towards AI, Inc. Description: Towards AI is the world's leading artificial intelligence (AI) and technology publication. Founders: Roberto Iriondo, , Job Title: Co-founder and Advisor Works for: Towards AI, Inc. Follow Roberto: X, LinkedIn, GitHub, Google Scholar, Towards AI Profile, Medium, ML@CMU, FreeCodeCamp, Crunchbase, Bloomberg, Roberto Iriondo, Generative AI Lab, Generative AI Lab Denis Piffaretti, Job Title: Co-founder Works for: Towards AI, Inc. Louie Peters, Job Title: Co-founder Works for: Towards AI, Inc. Louis-François Bouchard, Job Title: Co-founder Works for: Towards AI, Inc. Cover:
Towards AI Cover
Logo:
Towards AI Logo
Areas Served: Worldwide Alternate Name: Towards AI, Inc. Alternate Name: Towards AI Co. Alternate Name: towards ai Alternate Name: towardsai Alternate Name: towards.ai Alternate Name: tai Alternate Name: toward ai Alternate Name: toward.ai Alternate Name: Towards AI, Inc. Alternate Name: towardsai.net Alternate Name: pub.towardsai.net
5 stars – based on 497 reviews

Frequently Used, Contextual References

TODO: Remember to copy unique IDs whenever it needs used. i.e., URL: 304b2e42315e

Resources

Take our 85+ lesson From Beginner to Advanced LLM Developer Certification: From choosing a project to deploying a working product this is the most comprehensive and practical LLM course out there!

Publication

Methods, Challenges, and Hazards of Collecting Tweets
Natural Language Processing   Web Scraping

Methods, Challenges, and Hazards of Collecting Tweets

Last Updated on January 10, 2021 by Editorial Team

Author(s): Stephen DeFerrari

500 million tweets are sent a day, properly using them in your project goes beyond just writing an API request

Photo by Christian Lue on Unsplash

Disclaimer: This article is only for educational purposes. We do not encourage anyone to scrape websites, especially those web properties that may have terms and conditions against such actions.


After finishing a sentiment analysis project on Covid vaccine-related tweets, I was left feeling like I was only seeing a small part of the picture. I had built the project using tweets somebody else graciously collected and posted on Kaggle. The criteria for collection was having the hashtag “#CovidVaccine” with retweets filtered out.

Using somebody else’s data was convenient but a lot of tweets must have been missing with such strict criteria. Worse still, a lot of the tweets I did have seemed to be cut off; ending with “…” mid-sentence. I set out to do my own collecting to see if I could do any better. What I had expected to be a simple process of tweaking a few parameters and rewriting some code turned into an ordeal of figuring out 3 questions:

  • How am I going to actually collect any tweets?
  • How exactly does my query affect my project’s results?
  • How useful is Twitter for capturing “national conversations”?

This article is broken down into 3 sections, each one dedicated to one of these questions. Think of this article as part tutorial, part experiment, and part musing. All the code will be in Python but if that isn’t your language of choice, there are still lessons here for anyone interested in using tweets for their next project. Feel free to skip around to portions that interest you most; I won’t mind too much.

Section 1: Different Methods of Collecting Tweets

Tweets can be collected either with or without Twitter’s API. I am going to cover each method and give you their pros and cons.

Twitter API Collection via Tweepy

Tweepy is a Python library that allows for easy access to Twitter’s API. Before you write your first line of code you’ll need to apply for a developer account. Don’t be intimidated, it’s free (for basic access, more on that later), and you should be accepted within a few hours of applying.

After being accepted, you’ll need to create your own project to generate the keys, tokens, and secrets (passwords) needed to access the API. You can find them in the “keys and tokens” section of your project.

Image by author

Once you have them, start your code by importing Tweepy and Pandas and saving your keys, tokens, and secrets as variables. Then authenticate your account’s credentials so we can start accessing the API!

When we’re successfully authenticated, we can use Tweepy’s Cursor object to search and collect tweets based on specific criteria. For this example, we’re going to collect 100 English tweets between 2020–12–25 and 2020–12–26 that contain the hashtag “#vaccine”. Tweepy also supports more complex searches including using geocodes. For those interested, I highly recommend reading Tweepy’s documentation for a better idea of what’s possible along with Twitter’s own search operators documentation.

tweets = tw.Cursor(api.search,
              q="#vaccine",
              tweet_mode='extended',
              lang="en",
              since="2020-12-25",
              until="2020-12-28").items(100)

This will return a new object which contains all 100 tweets, but accessing the tweets’ texts isn’t as simple as iterating through and running “print.” Each tweet inside tweets contains a host of information including when it was posted, its text, location, etc. We need to ask the right questions to get what we want from them. Below is a function I’ve written to retrieve each tweet’s user name, location, full text, and tell me if it’s a retweet or not.

You may have noticed earlier that the Cursor object had tweet_mode set to “extended”. This is related to Twitter’s 2017 decision to up the character limit of tweets from 140 to 280. Without it, we’ll end up with tweets longer than 140 cut off with “…”, the issue I ran into with the original dataset.

Simply setting our search to extended and asking for the full_text attribute still isn’t perfect due to retweets getting cut off. This is where the “try”- statement at the bottom of the function comes in. Asking for the full text of the original tweet instead works. Tweepy’s documentation covers this issue in more detail.

With full_text_rt()set up, all we need to do now is a list comprehension on the tweets object. Run each tweet through the function and then convert the list of lists into a Pandas dataframe.

final_tweet_list = [full_text_rt(tweet) for tweet in tweets]
tweets_df = pd.DataFrame(data=final_tweet_list, 
                    columns=['user', "location", "text", "retweet"])

And there you have it, a dataframe with all of the information collected from Twitter’s API via Tweepy.

Image by author

Non-API Collection via Twint

As smooth as the Tweepy process is, we can only get so far using a free Twitter developer account. Our searches can only go back 7 days and we get 50 requests a month. For those who don’t want to shell out between $99 and $1,899 a month for premium access, there is Twint.

Twint is an OSINT (Open Source Intelligence) focused Twitter scraper that allows us to get around the API’s limitations by collecting tweets via Twitter’s own search feature.

We can use Twint via the command line but I’m going to show you how to do it from a notebook. Twint uses asyncio, which has a history of not playing nice with notebooks, so we need to make sure we have nest-asyncio installed, imported, and applied before we do anything.

import nest_asyncio
nest_asyncio.apply()

Actually importing Twint, running it, and converting its results into a dataframe can all be done in a single cell, thanks to Twint’s Pandas compatibility.

With far less struggle (but a little more processing time), we should now have our own dataframe with tweets scraped with Twint.

Image by author

Comparing Tweepy and Twint

As simple and as free as Twint is, it has its downsides. You may have noticed that we didn’t have locations in the Twint dataframe when we did have them in the Tweepy one. Due to Twint’s method of scraping via Twitter’s search function, collecting locations is actually a lot harder and more time consuming for it to do, and Twint is already slow compared to Tweepy.

You may have also noticed that while we requested 100 tweets, my dataframe only came back with 84. If you ran that code on your end you may very well have ended up with anywhere from 80 to 120 tweets despite using identical search criteria. Again, due to Twint’s scraping method, it has problems with consistency.

And then there is the issue of retweets. Despite setting c.Filter_Retweets to false, not a single retweet was returned. This is another problem inherited from using Twitter’s search function and one that Twint’s devs aren’t keen on fixing due to the library’s focus on OSINT collection.

Comparing documentation and tutorials, Tweepy and the API are a lot easier to understand and learn than Twint. Whereas I learned Tweepy by reading its documentation and guides found online, I learned Twint by navigating its “Issues” section on Github where the devs are very active in helping people troubleshoot problems.

Rather than choosing one over the other, it’s better to think of how they can be used together. With Tweepy and the API, you have the consistency and robustness you need to experiment and calibrate your queries to suit your needs. With Twint you have the ability to collect a large number of tweets without the API’s limitations.

One idea is to pull your initial bank of tweets from Tweepy, set up your processing pipeline, and create a minimum viable product. If you’re not satisfied go over your search criteria and try again. If the results work for you, then use Twint to pull a larger bank of tweets using the criteria you perfected on Tweepy and run them through the same pipeline to get your final product.

Section 2: The Challenges of Writing the “Right” Query

Now that we have established how to collect tweets, we need to see how our query changes our results. In the examples used in the last section I searched for tweets containing the “#vaccine” hashtag, but what if I instead dropped the hashtag and searched for tweets just containing “vaccine”? My Tweepy searches also included retweets while my Twint ones did not; how meaningful is that difference?

Rather than sit and wonder, let’s do an experiment where we run four different search results based on whether we use the hashtag or not, and whether we use retweets or not through the same VADER sentiment analysis pipeline and analyze the differences. The pipeline will result in a new column for each tweet with the classification of either “positive”, “neutral”, or “negative” based on the suggested thresholds from VADER’s documentation.

Here are the 3 functions we will be using:

With the functions in place, all we need to do is import the sentiment analyzer and run our search results through it. Here is my full code block for getting the sentiments for 500 tweets posted between 2020–12–25 and 2020–12–16 that contained the word “vaccine” with retweets filtered out. I’m using “vaccine” here rather than the longer “Covid vaccine” to include users who used “corona” instead of “Covid” in their tweets. Additionally, the term “vaccine” in conversation has become nearly ubiquitous with referring to the vaccine for Covid-19.

I ran this code 3 more times, adjusting only the “search_words” variable and dataframe names to cover hashtags vs no hashtags and retweets vs no retweets. Afterward, I took the resulting sentiment columns and imported them into Tableau, turning them into percentage pie charts.

Let’s start with comparing “vaccine” and “#vaccine” without retweets:

Image made with Tableau by author

These graphs paint two very different pictures. If we stuck with just using “#vaccine” for our search, we would conclude that a majority of people are feeling positive about the prospect of getting vaccinated, something the “vaccine” search directly disputes with results that instead tell us vaccine rollouts have been met with a lot of negativity and uncertainty on Twitter.

Why the gap in results? It’s important to understand how often people actually use hashtags in their tweets. A 2010 paper titled “Language Matters in Twitter: A Large Scale Study”[1] by researchers from Google and the Palo Alto Research Center found that only 14% of English language tweets contained hashtags. A little bit rosier, brand engagement company Mention in a 2018[2] report concluded that 40% of tweets contained hashtags. We may have been drawing from a much smaller and different pool of tweets when we restricted ourselves to just hash-tagged tweets.

What about retweets? Let’s compare these results if we instead allow retweets to be included in our searches:

Image made with Tableau by author

What’s interesting is that in both searches a majority of the tweets were retweets, but the results are strikingly similar to the results we got excluding retweets. The largest difference is the 11 point drop in positivity in the hash-tagged search, but it still left positive with the largest percentage of tweets.

If we recall, one of our issues with Twint was that it doesn’t scrape retweets. With these results, we might feel a lot safer using Twint to scrape tweets that contain “vaccine” since the sentiment ratio would be similar enough to a search containing retweets.

Section 3: Analytic Hazards of Using Tweets

Before you fire up your notebooks and begin your next project it’s important to know how representative Twitter is of the general population, after all, we still have one last question to answer: how useful is Twitter for capturing “national conversations”?

Photo by visuals on Unsplash

Selection bias should always be on our minds when we collect information from and about people. Understanding what biases exist when using Twitter might not help us mitigate them, but it can flag concerns for us to think about when drawing conclusions from our results.

According to a 2019 Pew Research study, only 22% of Americans actually use Twitter [3]. So we’re working with almost a quarter of the population, but how uniform is that usage? It turns out not uniform at all, another Pew study focused exclusively on Twitter usage[4] found that 10% of American Twitter users were responsible for 80% of tweets in the United States.

The same study found that American Twitter users were generally more likely to be younger, more educated, and more likely to be Democrats than the general public. When zooming in on the top 10% of American Twitter users, researchers found a much higher willingness to post about politics, with 69% of prolific tweeters saying they post about politics, compared to 39% of overall American Twitter users. While the gender divide was mostly evenly split among the bottom 90% of American Twitter users, women made up the majority of prolific tweeters.

Putting all of this together we can see that the American tweets collected are coming from a population that is much smaller and less representative than we might have thought.

These studies apply only to American Twitter users. Remember, our queries were for English language tweets, a language spoken by people all around the world, and who knows what biases exist in other countries regarding Twitter usage!

Conclusion

I hope this article leaves you more knowledgeable about using Twitter data for your next project. In review, we learned the following:

  • How to collect tweets using Tweepy and Twitter’s API as well as using Twint
  • The importance of testing multiple queries and comparing results
  • The biases you may encounter when using tweets based on usage statistics

Twitter, like any source of data, has its pros and cons, but don’t let the negatives keep you from experimenting. My advice is if you have an idea for a project or an interesting query, use the code I provided today and see what results you get. Just remember the importance of experimenting, what to watch out for when drawing conclusions, and one final tip: not everyone takes Twitter seriously.

References:

[1] E. Chi, G. Convertino, and L. Hong, Language Matters in Twitter: A Large Scale Study (2010), Fifth International AAAI Conference on Weblogs and Social Media

[2] Mention, 2018 Twitter Engagement Report (2018)

[3] A. Perron, M. Anderson, Social Media Usage Survey (2019), Pew Research Center

[4] A. Hughes, S. Wojcik, Sizing Up Twitter Users (2019), Pew Research Center

Feedback ↓

Sign Up for the Course
`; } else { console.error('Element with id="subscribe" not found within the page with class "home".'); } } }); // Remove duplicate text from articles /* Backup: 09/11/24 function removeDuplicateText() { const elements = document.querySelectorAll('h1, h2, h3, h4, h5, strong'); // Select the desired elements const seenTexts = new Set(); // A set to keep track of seen texts const tagCounters = {}; // Object to track instances of each tag elements.forEach(el => { const tagName = el.tagName.toLowerCase(); // Get the tag name (e.g., 'h1', 'h2', etc.) // Initialize a counter for each tag if not already done if (!tagCounters[tagName]) { tagCounters[tagName] = 0; } // Only process the first 10 elements of each tag type if (tagCounters[tagName] >= 2) { return; // Skip if the number of elements exceeds 10 } const text = el.textContent.trim(); // Get the text content const words = text.split(/\s+/); // Split the text into words if (words.length >= 4) { // Ensure at least 4 words const significantPart = words.slice(0, 5).join(' '); // Get first 5 words for matching // Check if the text (not the tag) has been seen before if (seenTexts.has(significantPart)) { // console.log('Duplicate found, removing:', el); // Log duplicate el.remove(); // Remove duplicate element } else { seenTexts.add(significantPart); // Add the text to the set } } tagCounters[tagName]++; // Increment the counter for this tag }); } removeDuplicateText(); */ // Remove duplicate text from articles function removeDuplicateText() { const elements = document.querySelectorAll('h1, h2, h3, h4, h5, strong'); // Select the desired elements const seenTexts = new Set(); // A set to keep track of seen texts const tagCounters = {}; // Object to track instances of each tag // List of classes to be excluded const excludedClasses = ['medium-author', 'post-widget-title']; elements.forEach(el => { // Skip elements with any of the excluded classes if (excludedClasses.some(cls => el.classList.contains(cls))) { return; // Skip this element if it has any of the excluded classes } const tagName = el.tagName.toLowerCase(); // Get the tag name (e.g., 'h1', 'h2', etc.) // Initialize a counter for each tag if not already done if (!tagCounters[tagName]) { tagCounters[tagName] = 0; } // Only process the first 10 elements of each tag type if (tagCounters[tagName] >= 10) { return; // Skip if the number of elements exceeds 10 } const text = el.textContent.trim(); // Get the text content const words = text.split(/\s+/); // Split the text into words if (words.length >= 4) { // Ensure at least 4 words const significantPart = words.slice(0, 5).join(' '); // Get first 5 words for matching // Check if the text (not the tag) has been seen before if (seenTexts.has(significantPart)) { // console.log('Duplicate found, removing:', el); // Log duplicate el.remove(); // Remove duplicate element } else { seenTexts.add(significantPart); // Add the text to the set } } tagCounters[tagName]++; // Increment the counter for this tag }); } removeDuplicateText(); //Remove unnecessary text in blog excerpts document.querySelectorAll('.blog p').forEach(function(paragraph) { // Replace the unwanted text pattern for each paragraph paragraph.innerHTML = paragraph.innerHTML .replace(/Author\(s\): [\w\s]+ Originally published on Towards AI\.?/g, '') // Removes 'Author(s): XYZ Originally published on Towards AI' .replace(/This member-only story is on us\. Upgrade to access all of Medium\./g, ''); // Removes 'This member-only story...' }); //Load ionic icons and cache them if ('localStorage' in window && window['localStorage'] !== null) { const cssLink = 'https://code.ionicframework.com/ionicons/2.0.1/css/ionicons.min.css'; const storedCss = localStorage.getItem('ionicons'); if (storedCss) { loadCSS(storedCss); } else { fetch(cssLink).then(response => response.text()).then(css => { localStorage.setItem('ionicons', css); loadCSS(css); }); } } function loadCSS(css) { const style = document.createElement('style'); style.innerHTML = css; document.head.appendChild(style); } //Remove elements from imported content automatically function removeStrongFromHeadings() { const elements = document.querySelectorAll('h1, h2, h3, h4, h5, h6, span'); elements.forEach(el => { const strongTags = el.querySelectorAll('strong'); strongTags.forEach(strongTag => { while (strongTag.firstChild) { strongTag.parentNode.insertBefore(strongTag.firstChild, strongTag); } strongTag.remove(); }); }); } removeStrongFromHeadings(); "use strict"; window.onload = () => { /* //This is an object for each category of subjects and in that there are kewords and link to the keywods let keywordsAndLinks = { //you can add more categories and define their keywords and add a link ds: { keywords: [ //you can add more keywords here they are detected and replaced with achor tag automatically 'data science', 'Data science', 'Data Science', 'data Science', 'DATA SCIENCE', ], //we will replace the linktext with the keyword later on in the code //you can easily change links for each category here //(include class="ml-link" and linktext) link: 'linktext', }, ml: { keywords: [ //Add more keywords 'machine learning', 'Machine learning', 'Machine Learning', 'machine Learning', 'MACHINE LEARNING', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, ai: { keywords: [ 'artificial intelligence', 'Artificial intelligence', 'Artificial Intelligence', 'artificial Intelligence', 'ARTIFICIAL INTELLIGENCE', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, nl: { keywords: [ 'NLP', 'nlp', 'natural language processing', 'Natural Language Processing', 'NATURAL LANGUAGE PROCESSING', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, des: { keywords: [ 'data engineering services', 'Data Engineering Services', 'DATA ENGINEERING SERVICES', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, td: { keywords: [ 'training data', 'Training Data', 'training Data', 'TRAINING DATA', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, ias: { keywords: [ 'image annotation services', 'Image annotation services', 'image Annotation services', 'image annotation Services', 'Image Annotation Services', 'IMAGE ANNOTATION SERVICES', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, l: { keywords: [ 'labeling', 'labelling', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, pbp: { keywords: [ 'previous blog posts', 'previous blog post', 'latest', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, mlc: { keywords: [ 'machine learning course', 'machine learning class', ], //Change your article link (include class="ml-link" and linktext) link: 'linktext', }, }; //Articles to skip let articleIdsToSkip = ['post-2651', 'post-3414', 'post-3540']; //keyword with its related achortag is recieved here along with article id function searchAndReplace(keyword, anchorTag, articleId) { //selects the h3 h4 and p tags that are inside of the article let content = document.querySelector(`#${articleId} .entry-content`); //replaces the "linktext" in achor tag with the keyword that will be searched and replaced let newLink = anchorTag.replace('linktext', keyword); //regular expression to search keyword var re = new RegExp('(' + keyword + ')', 'g'); //this replaces the keywords in h3 h4 and p tags content with achor tag content.innerHTML = content.innerHTML.replace(re, newLink); } function articleFilter(keyword, anchorTag) { //gets all the articles var articles = document.querySelectorAll('article'); //if its zero or less then there are no articles if (articles.length > 0) { for (let x = 0; x < articles.length; x++) { //articles to skip is an array in which there are ids of articles which should not get effected //if the current article's id is also in that array then do not call search and replace with its data if (!articleIdsToSkip.includes(articles[x].id)) { //search and replace is called on articles which should get effected searchAndReplace(keyword, anchorTag, articles[x].id, key); } else { console.log( `Cannot replace the keywords in article with id ${articles[x].id}` ); } } } else { console.log('No articles found.'); } } let key; //not part of script, added for (key in keywordsAndLinks) { //key is the object in keywords and links object i.e ds, ml, ai for (let i = 0; i < keywordsAndLinks[key].keywords.length; i++) { //keywordsAndLinks[key].keywords is the array of keywords for key (ds, ml, ai) //keywordsAndLinks[key].keywords[i] is the keyword and keywordsAndLinks[key].link is the link //keyword and link is sent to searchreplace where it is then replaced using regular expression and replace function articleFilter( keywordsAndLinks[key].keywords[i], keywordsAndLinks[key].link ); } } function cleanLinks() { // (making smal functions is for DRY) this function gets the links and only keeps the first 2 and from the rest removes the anchor tag and replaces it with its text function removeLinks(links) { if (links.length > 1) { for (let i = 2; i < links.length; i++) { links[i].outerHTML = links[i].textContent; } } } //arrays which will contain all the achor tags found with the class (ds-link, ml-link, ailink) in each article inserted using search and replace let dslinks; let mllinks; let ailinks; let nllinks; let deslinks; let tdlinks; let iaslinks; let llinks; let pbplinks; let mlclinks; const content = document.querySelectorAll('article'); //all articles content.forEach((c) => { //to skip the articles with specific ids if (!articleIdsToSkip.includes(c.id)) { //getting all the anchor tags in each article one by one dslinks = document.querySelectorAll(`#${c.id} .entry-content a.ds-link`); mllinks = document.querySelectorAll(`#${c.id} .entry-content a.ml-link`); ailinks = document.querySelectorAll(`#${c.id} .entry-content a.ai-link`); nllinks = document.querySelectorAll(`#${c.id} .entry-content a.ntrl-link`); deslinks = document.querySelectorAll(`#${c.id} .entry-content a.des-link`); tdlinks = document.querySelectorAll(`#${c.id} .entry-content a.td-link`); iaslinks = document.querySelectorAll(`#${c.id} .entry-content a.ias-link`); mlclinks = document.querySelectorAll(`#${c.id} .entry-content a.mlc-link`); llinks = document.querySelectorAll(`#${c.id} .entry-content a.l-link`); pbplinks = document.querySelectorAll(`#${c.id} .entry-content a.pbp-link`); //sending the anchor tags list of each article one by one to remove extra anchor tags removeLinks(dslinks); removeLinks(mllinks); removeLinks(ailinks); removeLinks(nllinks); removeLinks(deslinks); removeLinks(tdlinks); removeLinks(iaslinks); removeLinks(mlclinks); removeLinks(llinks); removeLinks(pbplinks); } }); } //To remove extra achor tags of each category (ds, ml, ai) and only have 2 of each category per article cleanLinks(); */ //Recommended Articles var ctaLinks = [ /* ' ' + '

Subscribe to our AI newsletter!

' + */ '

Take our 85+ lesson From Beginner to Advanced LLM Developer Certification: From choosing a project to deploying a working product this is the most comprehensive and practical LLM course out there!

'+ '

Towards AI has published Building LLMs for Production—our 470+ page guide to mastering LLMs with practical projects and expert insights!

' + '
' + '' + '' + '

Note: Content contains the views of the contributing authors and not Towards AI.
Disclosure: This website may contain sponsored content and affiliate links.

' + 'Discover Your Dream AI Career at Towards AI Jobs' + '

Towards AI has built a jobs board tailored specifically to Machine Learning and Data Science Jobs and Skills. Our software searches for live AI jobs each hour, labels and categorises them and makes them easily searchable. Explore over 10,000 live jobs today with Towards AI Jobs!

' + '
' + '

🔥 Recommended Articles 🔥

' + 'Why Become an LLM Developer? Launching Towards AI’s New One-Stop Conversion Course'+ 'Testing Launchpad.sh: A Container-based GPU Cloud for Inference and Fine-tuning'+ 'The Top 13 AI-Powered CRM Platforms
' + 'Top 11 AI Call Center Software for 2024
' + 'Learn Prompting 101—Prompt Engineering Course
' + 'Explore Leading Cloud Providers for GPU-Powered LLM Training
' + 'Best AI Communities for Artificial Intelligence Enthusiasts
' + 'Best Workstations for Deep Learning
' + 'Best Laptops for Deep Learning
' + 'Best Machine Learning Books
' + 'Machine Learning Algorithms
' + 'Neural Networks Tutorial
' + 'Best Public Datasets for Machine Learning
' + 'Neural Network Types
' + 'NLP Tutorial
' + 'Best Data Science Books
' + 'Monte Carlo Simulation Tutorial
' + 'Recommender System Tutorial
' + 'Linear Algebra for Deep Learning Tutorial
' + 'Google Colab Introduction
' + 'Decision Trees in Machine Learning
' + 'Principal Component Analysis (PCA) Tutorial
' + 'Linear Regression from Zero to Hero
'+ '

', /* + '

Join thousands of data leaders on the AI newsletter. It’s free, we don’t spam, and we never share your email address. Keep up to date with the latest work in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

',*/ ]; var replaceText = { '': '', '': '', '
': '
' + ctaLinks + '
', }; Object.keys(replaceText).forEach((txtorig) => { //txtorig is the key in replacetext object const txtnew = replaceText[txtorig]; //txtnew is the value of the key in replacetext object let entryFooter = document.querySelector('article .entry-footer'); if (document.querySelectorAll('.single-post').length > 0) { //console.log('Article found.'); const text = entryFooter.innerHTML; entryFooter.innerHTML = text.replace(txtorig, txtnew); } else { // console.log('Article not found.'); //removing comment 09/04/24 } }); var css = document.createElement('style'); css.type = 'text/css'; css.innerHTML = '.post-tags { display:none !important } .article-cta a { font-size: 18px; }'; document.body.appendChild(css); //Extra //This function adds some accessibility needs to the site. function addAlly() { // In this function JQuery is replaced with vanilla javascript functions const imgCont = document.querySelector('.uw-imgcont'); imgCont.setAttribute('aria-label', 'AI news, latest developments'); imgCont.title = 'AI news, latest developments'; imgCont.rel = 'noopener'; document.querySelector('.page-mobile-menu-logo a').title = 'Towards AI Home'; document.querySelector('a.social-link').rel = 'noopener'; document.querySelector('a.uw-text').rel = 'noopener'; document.querySelector('a.uw-w-branding').rel = 'noopener'; document.querySelector('.blog h2.heading').innerHTML = 'Publication'; const popupSearch = document.querySelector$('a.btn-open-popup-search'); popupSearch.setAttribute('role', 'button'); popupSearch.title = 'Search'; const searchClose = document.querySelector('a.popup-search-close'); searchClose.setAttribute('role', 'button'); searchClose.title = 'Close search page'; // document // .querySelector('a.btn-open-popup-search') // .setAttribute( // 'href', // 'https://medium.com/towards-artificial-intelligence/search' // ); } // Add external attributes to 302 sticky and editorial links function extLink() { // Sticky 302 links, this fuction opens the link we send to Medium on a new tab and adds a "noopener" rel to them var stickyLinks = document.querySelectorAll('.grid-item.sticky a'); for (var i = 0; i < stickyLinks.length; i++) { /* stickyLinks[i].setAttribute('target', '_blank'); stickyLinks[i].setAttribute('rel', 'noopener'); */ } // Editorial 302 links, same here var editLinks = document.querySelectorAll( '.grid-item.category-editorial a' ); for (var i = 0; i < editLinks.length; i++) { editLinks[i].setAttribute('target', '_blank'); editLinks[i].setAttribute('rel', 'noopener'); } } // Add current year to copyright notices document.getElementById( 'js-current-year' ).textContent = new Date().getFullYear(); // Call functions after page load extLink(); //addAlly(); setTimeout(function() { //addAlly(); //ideally we should only need to run it once ↑ }, 5000); }; function closeCookieDialog (){ document.getElementById("cookie-consent").style.display = "none"; return false; } setTimeout ( function () { closeCookieDialog(); }, 15000); console.log(`%c 🚀🚀🚀 ███ █████ ███████ █████████ ███████████ █████████████ ███████████████ ███████ ███████ ███████ ┌───────────────────────────────────────────────────────────────────┐ │ │ │ Towards AI is looking for contributors! │ │ Join us in creating awesome AI content. │ │ Let's build the future of AI together → │ │ https://towardsai.net/contribute │ │ │ └───────────────────────────────────────────────────────────────────┘ `, `background: ; color: #00adff; font-size: large`); //Remove latest category across site document.querySelectorAll('a[rel="category tag"]').forEach(function(el) { if (el.textContent.trim() === 'Latest') { // Remove the two consecutive spaces (  ) if (el.nextSibling && el.nextSibling.nodeValue.includes('\u00A0\u00A0')) { el.nextSibling.nodeValue = ''; // Remove the spaces } el.style.display = 'none'; // Hide the element } }); // Add cross-domain measurement, anonymize IPs 'use strict'; //var ga = gtag; ga('config', 'G-9D3HKKFV1Q', 'auto', { /*'allowLinker': true,*/ 'anonymize_ip': true/*, 'linker': { 'domains': [ 'medium.com/towards-artificial-intelligence', 'datasets.towardsai.net', 'rss.towardsai.net', 'feed.towardsai.net', 'contribute.towardsai.net', 'members.towardsai.net', 'pub.towardsai.net', 'news.towardsai.net' ] } */ }); ga('send', 'pageview'); -->