Challenge

As part of an Udacity course on AI Product Management, I was tasked with building an AI/ML model using Google's Vertex AI. The objective was to create a binary classifier and a 3-class model using clean/balanced data, clean/unbalanced data, and dirty/balanced data. There were several challenges posed in building the models. One of the main issues was dealing with unbalanced datasets which led to high accuracy scores but low precision and recall scores for the minority class. Another challenge was working with dirty data which resulted in overfitting or underfitting of the model.

Approach

To address the issue of unbalanced datasets, I used clean/balanced data for the binary classifier model. I also experimented with different score thresholds to increase precision and recall. For dealing with dirty data, I used a clean dataset for our 3-class model and increased the number of images for each class to improve accuracy.

Findings

Achieved precision and recall scores of 80% for the binary classifier with clean/balanced data
Increased precision by increasing score threshold while decreasing recall
Observed that unbalanced classes lead to high accuracy but low precision and recall scores for minority class
Determined that dirty data can lead to overfitting or underfitting of the model reducing precision and recall
Achieved F1 score of 0.91 for our 3-class model using clean dataset with increased number of images per class

AI/ML models using Google Vertex AI Console