Join thousands of AI enthusiasts and experts at the Learn AI Community.

Publication

Data Science   Machine Learning   Tutorial

Getting Started with Kaggle Competitions Using Monk AI

Last Updated on May 15, 2020 by Editorial Team

Author(s): sinchana S.R

Deep Learning

Monk is a low code deep learning tool and a suitable wrapper for computer vision.

Monk Features

  • Low-code.
  • Unified wrapper over major deep learning framework s — Keras, PyTorch, gluoncv.
  • Syntax invariant wrapper.

Monk Enables

  • To create, manage, and version control deep learning experiments.
  • To compare experiments across training metrics.
  • To quickly find the best hyperparameters.

Goals

  • To experiment with Models.
  • Understand how easy is it to use Monk.

Table of contents

  • About the dataset.
  • Installation.
  • Importing Pytorch Backend.
  • Creating and managing experiments.
  • List of all the available models.
  • Training a Pneumonia Vs. Normal image classifier.
  • Quick mode training.
  • Validating the trained classifier.
  • Running inference on test images.

About the dataset

Data: https://data.mendeley.com/datasets/rscbjbr9sj/2

License: CC BY 4.0

Paper: http://www.cell.com/cell/fulltext/S0092-8674(18)30154-5

Dataset structure

The dataset is organized into three folders (train, test, val) and contains subfolders for each image category (Pneumonia/Normal). There are 5,863 X-Ray images (JPEG) and two categories (Pneumonia/Normal).

Train 
-----Normal
-----Pneumonia
Test
-----Normal
-----Pneumonia
Validation
-----Normal
-----Pneumonia

Installation

Setting up MONK and its dependencies:

$ !git clone https://github.com/Tessellate-Imaging/monk_v1.git
$ !cd monk_v1/installation/Misc && pip install -r requirements_kaggle.txt

Since I am using Kaggle, I installed Kaggle dependencies.

If you are Linux user

$ cd monk_v1/installation/Linux && pip install -r requirements_cu9.txt

(Select the requirements file as per OS and CUDA version)

If using Colab install using the commands below

$ cd monk_v1/installation/Misc && pip install -r requirements_colab.txt

If using Kaggle uncomment the following command

$ cd monk_v1/installation/Misc && pip install -r requirements_kaggle.txt

You can find the installation procedure here.

MONK supports PyTorch, Keras, and mxnet-gluon backends.

Importing Pytorch Backend

#Using pytorch backend 
$ from pytorch_prototype import prototype
  • To use mxnet backend

$ from gluon_prototype import prototype

  • To use Keras backend

$ from keras_prototype import prototype

Creating and managing experiments

  • Provide a project name
  • Provide an experiment name
  • For specific data create a single project
  • Inside each project, multiple experiments can be created
  • Every experiment can have different hyper-parameters attached to it
$ gtf = prototype(verbose=1);
$ gtf.Prototype("PneumoniaClassificationMONK", "UsingPytorchBackend");

This creates files and directories as per the following structure workspace

|
|--------PneumoniaClassificationMONK (Project name can be different)
|
|
|-----UsingPytorchBackend (Experiment name can be different)
|
|-----experiment-state.json
|
|-----output
|
|------logs (All training logs and graphs saved here)
|
|------models (all trained models saved here)

List of all the available models

$ gtf.List_Models()

You can also find the list of available models, layers, blocks, optimizers here.

Training a Pneumonia vs. Normal image classifier

Quick mode training

  • Using the Default Function
  • dataset_path
  • model_name
  • num_epochs

Dataset folder structure parent_directory

|
|
|------Pneumonia
|
|------img1.jpg
|------img2.jpg
|------.... (and so on)
|------Normal
|
|------img1.jpg
|------img2.jpg
|------.... (and so on)

So for quick mode training,

$ gtf.Default(dataset_path="/kaggle/input/chest-xray-pneumonia/chest_xray/train", 
model_name="resnet50",
freeze_base_network=False,
num_epochs=25);

You can also find update mode and expert mode training.

Training the classifier

$ gtf.Train();

I trained my model for 25 epochs. After the end of the training, you can read the summary:

Validating the trained classifier

Load the experiment in validation mode

  • Set flag eval_infer as True
$ gtf = prototype(verbose=1);
$ gtf.Prototype("PneumoniaClassificationMONK", "UsingPytorchBackend",eval_infer=True);

Load the validation dataset

$ gtf.Dataset_Params(dataset_path="/kaggle/input/chest-xray-pneumonia/chest_xray/val");
$ gtf.Dataset();

Run validation

$ accuracy, class_based_accuracy = gtf.Evaluate();

Running inference on test images

Load the experiment in inference mode

  • Set flag eval_infer as True
$ gtf = prototype(verbose=1);
$ gtf.Prototype("PneumoniaClassificationMONK", "UsingPytorchBackend",eval_infer=True);

Select image and Run inference

$ img_name = "/kaggle/input/chest-xray-pneumonia/chest_xray/test/NORMAL/IM-0005-0001.jpeg";
$ predictions = gtf.Infer(img_name=img_name);

Training v/s Validation Accuracy Curves

$ from IPython.display import Image
$ Image(filename="/kaggle/working/workspace/PneumoniaClassificationMONK/UsingPytorchBackend/output/logs/train_val_accuracy.png")

Similarly, we can obtain Training v/s Validation loss Curves.

You can find the starter code on Kaggle’s website.

Resources

Monk_Object_Detection

A one-stop repository for low-code easily-installable object detection pipelines.

Monk_Gui

A Graphical User Interface for deep learning and computer vision over Monk Libraries.

Pytorch_Tutorial

A set of jupyter notebooks on PyTorch functions with examples.

I am an open-source contributor to the monk library. If you have any issues, please feel free to open an issue on GitHub.


Getting Started with Kaggle Competitions Using Monk AI was originally published in Towards AI — Multidisciplinary Science Journal on Medium, where people are continuing the conversation by highlighting and responding to this story.

Published via Towards AI

Feedback ↓