Boost Your Data Science, ML, and CV Projects: Essential Tools for Effective Project Management

Author(s): Chinmay Bhalerao

Originally published on Towards AI.

Make your builds and projects faster with these tools

ML or data science projects are very huge to build as they contain many types of files and many different architectures. But surprisingly, I came across various tools for project management that will help you to fasten your project. These are independent tools that will help you to work for the library, and package management as well as to work asynchronously to make execution faster.

1. Cookiecutter

Cookiecutter is a tool that we use to cut very hard cookies made by people. [Sorry for my sarcasm]. Let's understand cookiecutter in detail.

As Cookiecutter’s official page describe,

Cookiecutter is a Python package, easily installable with pip or other package managers, that enables you to create and use templates for microservices and software projects. It is a command-line tool that requires no knowledge of Python to use.

when we start the ML-DL-CV project, we have a lot of things to do to add to our repository. Initially, while structuring projects, it is hard to know which things we will require in the future for the project. for example, APIs, models, front end, routes, main files, and many more stuff. Managing everything and routes for each type of file will get fuzzy while processing complex projects. Cookiecutter provides solutions for this kind of problem.

When done correctly, good project organization can save you time, but it can also take time and effort when you first begin a new project and are forced to create README files. There is always the option to simply clone an older project and delete or edit the existing files and folders, but this is neither more enjoyable nor quicker than beginning from scratch. Cookiecutter provides inbuild templates that we can use to set our project structure.

Installing Cookiecutter

pip install cookiecutter

Using conda

conda install -c conda-forge cookiecutter

We can also use the Inbuit template to create the structure of our project.

# Create project from the cookiecutter-pypackage.git repo template
# You'll be prompted to enter values.
# Then it'll create your Python package in the current working directory,
# based on those values.
$ cookiecutter https://github.com/audreyfeldroy/cookiecutter-pypackage
# For the sake of brevity, repos on GitHub can just use the 'gh' prefix
$ cookiecutter gh:audreyfeldroy/cookiecutter-pypackage

You can create a simple template with cookiecutter jinja template or can import inbuilt as mentioned above. It is very simple like installing any Python library and cloning any git repo.

cookiecutter documentation: Here

cookiecutter repository: Here

Few templates that I have found useful for ML — DL projects :

For data science project: Click here

For computer vision: Click here

You can explore cookiecutter more and choose which templates are suitable for your project.

2. Poetry

Poetry is an art but not here. here poetry is a package manager. Let's see it in detail.

Poetry is a cutting-edge Python package management tool that makes it easier to create, manage, and publish Python packages.

For managing dependencies, creating packages, and uploading them to PyPI (Python Package Index), the official repository for Python packages, it offers a simple-to-use command-line interface.

benefits of using Poetry for package management [Source]

• Dependency resolution
• Virtual environments
• Project scaffolding
• Built-in building and packaging
• Publishing to PyPI

Installation of Poetry

(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content U+007C py -

or Linux, macOS, and Windows Subsystem for Linux (WSL) users, open the terminal and run the following command

curl -sSL https://install.python-poetry.org U+007C python3 -

Create new project

poetry new My Project 

This will create a new folder called My Projectwith the following structure.

My Project
├── pyproject.toml
├── README.md
├── phone_number_validator
│ └── __init__.py
└── tests
 └── __init__.py

The poetry will create the structure of the files shown in the following image.

This is how poetry uses different files to manage dependencies for your project.

Poetry Documentation: Here

Poetry repo: Here

3. FastAPI

FastAPI is a cutting-edge web framework for creating Python RESTful APIs. Due to its simplicity, speed, and resilience, it has quickly been a favorite among developers since its initial release in 2018. For data validation, serialization, and deserialization, the FastAPI uses Pydantic and type hints.

Why FastAPI? because it has async support which will make applications faster. If you want async support for Django or Flask then you will require something like a salary in Python.

Installation

pip install fastapi

You will also need an ASGI server, for production such as Uvicorn or Hypercorn.

pip install "uvicorn[standard]"

Creating simple FastAPI.

from typing import Union

from fastapi import FastAPI

app = FastAPI()


@app.get("/")
def read_root():
 return {"Hello": "World"}


@app.get("/items/{item_id}")
def read_item(item_id: int, q: Union[str, None] = None):
 return {"item_id": item_id, "q": q}

At last to run FastAPI,

uvicorn main:app --reload

here I am assuming main is your .py file name. If your file name is different then you can use uvicorn your_file_name:app to run your app.

--reload: When you do changes and save the project then automatically fast API will reload and acquire changes. for that, we use this command.

You can also give a port and host if required.

How to test FastAPI? by Postman?

We don't require a postman here !!!!

FastAPI comes with a built-in Swagger interface to handle requests and responses.FastAPI provides its own frontend to test API and it is known as API SWAGGER.

Like-wise as the name suggest, FastAPI is very good to create API for data science, ML, and DL use cases.

FastAPI Documentation: Click here

FastAPI repository: Click here

These are various tools that I thought will be helpful for accelerating your projects. I gave a brief overview of these tools otherwise detailing would make this blog very long. But definitely, you can check the links that I have provided and can go deep down to understand these tools effectively.

If you have found this article insightful

It is a proven fact that “Generosity makes you a happier person”; therefore, Give claps to the article if you liked it. If you found this article insightful, follow me on Linkedin and Medium. You can also subscribe to get notified when I publish articles. Let’s create a community! Thanks for your support!

Also, medium doesn’t give me anything for writing, if you want to support me then you can click here to buy me coffee.

You can read my other blogs related to

How Filter Bubbles Are Biasing Your Opinions on Social Media

Filter Bubbles and Echo Chambers: How Social Media is Biasing Our Opinions and the Role of AI in it

medium.com

Understanding LangChain U+1F99C️U+1F517: PART 1

Theoretical understanding of chains, prompts, and other important modules in Langchain

pub.towardsai.net

Quantifying randomness in human behavior

Trust me, Human is the most random thing in the world

generativeai.pub

Comprehensive Guide: Top Computer Vision Resources All in One Blog

Save this blog for comprehensive resources for computer vision

medium.com

Signing off,

Chinmay!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI