How to Maximize ML Project Success with Efficient Scoping? | MLOps 5

Author(s): Akhil Theerthala

Originally published on Towards AI.

How to Maximize ML Project Success with Efficient Scoping? U+007C MLOps 5

In our past articles of this series, we have seen many things. We started our journey by looking at the lifecycle of a project. Following this, we got an overview of each phase of the lifecycle. If you have followed this series till now, you should know about the standard practices in different lifecycle phases. Let’s take a look at the lifecycle once.

After a brief overview of the lifecycle, we looked into the individual phases. After moving back to back from one stage of the project to another, it is time for us to look into the project’s very first stage, Scoping.

Fret not. Unlike the previous articles, this article is a short one. You can finish reading this within a few minutes! U+1F60C

Why should we scope a project?

Let’s take a step back and think from a company’s perspective. Companies generally invest vast amounts into any of their projects. Besides these costs, they must employ engineers and scientists to work on these projects. This requires additional capital. All this money will be invested into a project, which sometimes might yield abysmal results. Hence, companies are cautious before they start a new project.

They cross-check why they need the project, check for alternative cheaper options, list out additional benefits a project can bring, do market research for the projects, list out the different risks the project has, plan out the entire financial requirements, and make a profit estimate, and they estimate the time required for the project. They do all this before a project begins. However, even after doing this, the project might still fail or need more time or investments.

In Aug 2022, Forbes stated that somewhere between 60–80% of AI projects are failing, according to different sources. This shows that scoping the projects is one of the most important steps before starting a machine learning project.

Let’s speak examples.

Now that we have seen why scoping a project is essential. Let’s get into the details. Essentially, the question we have been asking till now is, “ How to pick a project to work on?”

Let’s use the example of an e-commerce retailer looking to increase sales. What can you recommend to him as an ML Engineer? You can guide him,

• A better product recommendation system.
• A better search engine for his site.
• To improve the catalog data of his store.
• A better inventory management system, etc.

We can also keep giving hundreds of other suggestions to him, both belonging to ML and not belonging to ML. But, there are only a few efficient suggestions that he needs to focus on. So, our problem now is to

• Identify the most profitable project to work on.
• Explain to him how this is the most profitable project using different metrics.
• Estimate the resources and costs of this project and ask him the amount.

That’s it. If you answer these three questions, you have finished scoping the project.

General Outline for Scoping.

Like all other fields, scoping ML projects have many community recommended outlines. Despite there being many outlines, their essence is the same. Now, we can look at one such strategy.

Step-1: Brainstorm the business problems.

• Before beginning the project, it is first essential to have a goal or a problem to solve. There are countless problems found everywhere. However, only a few are profitable.
• In this step, we list all the problems that can profit the company when solved.
• For our store, some problems can be reducing the inventory, increasing profit margin, increasing conversion, etc.,

Step-2: Brainstorm AI solutions.

• In this step, we identify different AI solutions for the identified problems. We pick the most feasible one among these in the later stages.
• For our store, it can be things like,
  – Improving the recommendation model to increase the conversion rate.
  – Optimizing what to sell to reduce the profit margin.
  – Using applications of marketing and demand prediction to reduce inventory.

Step-3: Feasibility Assessment

• AI is not a cure-all tool. It should not be applied to everything just because we can.
• In our earlier AI solutions, many need unrealistic requirements. Hence we need to filter these out.
• Since we are checking the possibility of an AI solution, we use different machine learning metrics to filter these out.
• For filtering these problems,
  – We can use external benchmarks like research papers, other companies etc.
  – We can look into Human-Level Performance (HLP) for unstructured data (or) the availability of predictive features for structured data.
  – We also generally look at the history of existing projects in the market and their performance to assess the rate of future improvement.

Step-4: Value Assessment

• Just as all the solutions are not feasible, all feasible solutions are not valuable. For example, building a linear regression to predict no. of A/Cs in the office is useless for an e-commerce store.
• In the value Assessment, we look at different Business metrics that affect the AI feasible projects. The business metrics include the metrics like User engagement, quality of the model compared to the market, possible revenue that can be generated from the model, etc.,
• In practical situations, the ML and Business teams compromise to some degree to find feasible and valuable projects.

Step-5: Draw out a plan.

• Determining the milestones generally involves determining the ML and software metrics of the projects. Some of such metrics were previously discussed in articles 2 and 3.1.
• However, here, we also try to determine the Business metrics and resources needed for the project like estimating the possible revenue, data required, the personnel that is allocated, etc.,
• Finally, after all the things are done. You now have a problem, a possible AI solution, and the metrics to evaluate the situation. But your job doesn’t end here. You need to give the company a timeline within which you will complete the project. (Yup, they don’t trust you with the budget.)

Step-6: Determine the Budget for the resources.

• Of course, doing all the above things and not having the resources to start working on the project is a waste of time.
• You can estimate the resources needed from the feasibility study the ML team has done for the project. Based on these resources, you need to determine the budget required for the project.

Thats it! You are done!

Could you leave the rest of the discussions and tasks for the Business team and wait until you get the approval for the project?

The answer would be yes. (probably, as I don’t know the structure of your company).

With this, we come to the end of our journey in the MLOps Series. I started this series as a follow-up to my notes on the MLOps specialization, and through the trip, I found areas that I needed to fill and enjoyed reading reports and articles on different topics.

If you are also taking the course, you might have noticed some areas where I made some changes and expanded in some areas (especially this article!). While writing this series, I found my long-lost passion for ML, as after some point, I found it a drag to read something and write semester exams on it. (I take ML courses at my uni.)

I went ahead and created this site to sustain my passion and also try and meet different people who also have a high passion for ML. I hope you share your comments and views on the areas covered in this article.

I will keep writing and publishing at least two articles a week from now on. So stay tuned! In the meantime, if you still need to read my other articles in this series, you can find them all here. You can also read some of my old articles on my medium page. Stay safe till next time!

Originally published at https://www.neuronuts.in on February 10, 2023.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI