How to Learn AI

Author(s): Jeff Holmes MS MSCS

Originally published on Towards AI.

Common mistakes and misconceptions about learning AI/ML

A common misconception of beginners is that they can learn AI/ML from a few tutorials that implement the latest algorithms, so I thought I would share some notes and advice on learning AI. I also have a GitHub repo with lots of notes and links to AI/ML articles on various topics LearnAI.

Background

As a professional AI engineer with 25+ years of software engineering experience, I have designed and implemented a wide variety of software applications and technologies. Therefore, I have used and implemented hundreds (perhaps thousands) of software libraries, frameworks, and APIs. I also have an M.S. in Mathematics and an MSCS in Artificial Intelligence, so I am more than qualified to mentor and teach undergraduate mathematics and computer science courses, as well as many graduate courses in Math/CS.

The student of AI should be sure to learn the history of AI which is wrought with hype and alchemy making unfounded claims on what AI can do such as AGI. Thus, a student of AI also needs a healthy dose of skepticism [8].

Since this is an AI website, I will assume that most readers will have the following goals:

• You are interested in becoming an AI/ML engineer.
• You are interested in learning software engineering best practices [1][2].

How to Become an Artificial Intelligence Engineer

Prerequisites

Here are some undergraduate prerequisites that you should know before learning AI/ML:

• Calculus
• Discrete Mathematics
• Advanced Linear Algebra
• Probability and Statistics
• Statistical Programming

The following courses are usually required for a computer science degree:

• Algorithms and Data Structures
• Computer Organization
• Computer Networks
• Operating Systems
• Distributed Computing

What Google Recommends You do Before Taking Their Machine Learning or Data Science Course

Here are some resources on mathematics that I have found useful:

• Math Refresher for Scientists and Engineers
• Trigonometry Handbook
• Lessons on How to Lie with Statistics
• An Introduction To Recurrent Neural Networks And The Math That Powers Them
• Learn Data Science For Free

Common Mistakes to Avoid

Here are some beginner mistakes to avoid when learning ML [5][6][10]:

• Asking “What is the best algorithm for X problem?”
• Trying to code ML algorithms from scratch.
• Using custom datasets and/or solving real-world problems while learning AI.
• Trying to learn AI from research papers.
• Using neural networks (only as a last resort)

Avoid coding algorithms from scratch

In applied mathematics, the emphasis is on the application of mathematics rather than the derivation of the formulas. Likewise, AI engineering is the application of algorithms to solve problems. Thus, the emphasis is on knowing what algorithms are available for a given problem, how they work, and how to get the most out of them rather — not the implementation details.

Any competent software engineer can implement any machine learning algorithm — Lewis University Professor

Why? There are simply too many ML algorithms for a software developer to code from scratch and be able to understand all the details needed to properly implement them. In fact, more algorithms are constantly being developed. Just concentrating on the “more interesting” or SOTA algorithms is a recipe for disaster and a common beginner mistake. In fact, there are many other concepts that are more important than knowing how to implement an ML algorithm (see Core AI Concepts).

Once you have mastered using AI algorithms, you can go back later and try to understand how they work internally by coding them by hand, if needed.

Avoid trying to learn AI from research papers

Research papers are not a good resource for learning a topic. The reader is assumed to already know the core theory and concepts covered in textbooks. Thus, the best approach to learning AI is a good textbook.

Keep in mind that the results of AI research articles are for the most part, irreproducible. In fact, there is a serious problem with integrity in AI research right now. The further away you get from reputable publications such as IEEE, the worse it gets.

This is discussed in more detail in Russell and Norivg [8] and other graduate textbooks.

Tips for Learning

Here are some tips for learning AI:

• Favor simpler algorithms over more complex algorithms (Occam’s Razor). Thus, deep learning models should be your last choice.
• There is no such thing as best, only good enough (No Free Lunch Theorem).
• Know when not to use AI.
• AI Checklists

Core AI Concepts

Here are some key AI concepts to learn:

• How to define the problem?
• How to select a dataset?
• How to perform data preparation?
• How to perform feature engineering?
• How to select an algorithm?
• How to choose a performance metric?
• How to train a model?
• How to present results?
• How to design and deploy the solution?
• How to debug models?

In graduate school, we are taught that the proper technique for learning AI is the black-box [7] view of ML algorithms (top-down approach) since it is best not to waste time trying to hand-code algorithms (bottom-up approach), especially when you are just getting started.

Unless you are developing a commercial AI software library, you will find that there is very little to learn by focusing on the internal implementation of an algorithm. It is more important to focus on learning how to use and deploy AI algorithms.

In software engineering, there are many well-known principles, such as DRY, KISS, and Occam’s Razor. Perhaps the most important principle is “don’t reinvent the wheel” (DRTW). Nevertheless, it seems that every beginner in machine learning proceeds to try to implement a neural network from scratch. An AI engineer needs to know the best use of ALL software libraries, APIs, and algorithms that are used in an application, not just AI libraries.

Occam’s Razor: The simplest solution that fits the data is usually the best.

A professional AI engineer knows that the current best practice is a data-centric rather than model-centric approach to AI/ML. Therefore, a beginner should focus on learning data preparation, feature engineering, dimensionality reduction, and other key concepts and techniques rather than the implementation details of a particular model/algorithm. In addition, a shift in mindset is required to be effective at AI, from trying to find the “best” model/approach to “good enough”.

Today, automation of the full software development life cycle is a big part of AI software development (MLOps), so it is doubtful that you will implement and deploy a hand-coded algorithm. In fact, there are a multitude of reasons to avoid writing custom code when robust libraries and frameworks are readily available.

In short, it is best to understand the big picture (top-down approach) of the AI engineering process before delving into the implementation details.

How to Choose a Model

There have been times when I have tried to help beginners only to find that they have spent hundreds of hours coding from a tutorial (or worse, an example from Tensorflow/PyTorch documentation) using the wrong model and/or solving the wrong problem. In fact, they are usually unwilling to listen when told that they have made a mistake, which I call code fixation.

When learning AI, you should first spend time finding several good tutorials/examples that solve a similar problem that has been written by someone with good credentials.

Next, you need to spend time understanding your dataset to make sure it is being modeled correctly. For example, just because your data is stock data or has a time feature does not mean that the data is best modeled as a time series and/or using neural networks.

Finally, evaluate many simple models on your dataset (or a subset of your data if your dataset is very large) using an AutoML tool such as Orange to find the best-performing model(s) to be used as a baseline for comparison. Keep in mind that your final chosen model should perform much better than all simpler baseline models.

In general, even the most complex SOTA models will only achieve a small performance improvement (say 3–5%) which is not enough to justify their use in solving real-world problems.

Tutorials and Books

Choosing an ML learning resource is primarily a matter of personal preference. In general, I recommend that you refer to more than one resource (other than documentation) when learning AI. I have a medium article, AI Learning Resources, that has a list of some AI resources that I have found to be helpful. Since AI engineering is an extension of software engineering, you should develop a thorough understanding of the key concepts given in SWEBOK .

Do I need a Master’s Degree?

If you are going to spend the time needed to study AI, then you should consider an online degree, which will greatly increase your career opportunities (and a requirement for almost all AI engineering positions). It seems that many people forget that AI is considered a graduate-level topic. In fact, AI engineering is an engineering discipline. Most engineering disciplines require a specialized degree, but for some reason, software engineering does not, which is probably why 85% or more of AI projects fail [6].

Here are some articles that can help you decide:

• Do you need a Master’s Degree in Data Science?
• Do You Have the Degree it Takes to Get Hired as a Data Scientist?
• Why Machine Learning Engineers are Replacing Data Scientists
• Machine Learning Engineers Might Replace Data Scientists: Why?
• Masters in Artificial Intelligence — Top 10 Programs
• Lewis University MSCS in Artificial Intelligence

Conclusion

This article was mainly motivated by helping many Slack Discord forum users who are interested in a self-study approach. However, if you are going to spend the time needed to study AI/ML, then it would be wise to invest in an online degree, which will greatly increase your career opportunities.

References

[1] P. Bourque and R. E. Fairley, Guide to the Software Engineering Body of Knowledge, v. 3, IEEE, 2014.

[2] IEEE Code of Ethics

[3] ACM Code of Ethics

[4] J. Brownlee, “ 5 Mistakes Programmers Make when Starting in Machine Learning,” Machine Learning Mastery, Jan. 39, 2014.

[5] J. Brownlee, “ Stop Coding Machine Learning Algorithms From Scratch,” Machine Learning Mastery, Oct. 5, 2016.

[6] T. Shin, “ 4 Reasons Why You Shouldn’t Use Machine Learning,” Towards Data Science, Oct. 5, 2021.

[7] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software, 1st ed., Addison-Wesley Professional, ISBN-10: 0201633612, 1994.

[8] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 4th ed. Upper Saddle River, NJ: Prentice Hall, ISBN: 978–0–13–604259–4, 2021.

[9] M. Hunter, “ 6 Powerful Study Techniques to Help You Master the Toughest Topics in Data Science,” Towards Data Science, June 10, 2022.

[10] A. Joury, “ Forget about algorithms and models — Learn how to solve problems first,” Towards Data Science, Sept. 9, 2022.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI