During a customer visit, I was asked about a solution, which can automatically remove the background of an image without any human intervention.

The customer is a technology company, developing solutions for the automotive and financial services verticals. They have millions of images of cars that their clients upload to their websites with different backgrounds. They had to manually correct each of these images which is labour intensive and time consuming.

They asked me about the possibility of an application that can automatically detect a car from an image and mask the background with any colour of their choice and add their watermark on it.

Analyzing Images Using Open CV & Edge Detection

Initially, we decided to explore some already available solutions and created a tool using Open CV, Python, and Edge Detection. But these tools were not providing accurate results. The tool worked in basic car models, but most of the time the tool was not accurate in identifying the car from the image or the angle of the car by which the image was taken. Also, the jagged edges of the primary object (car) in the images were a major concern.

Exploring Machine Learning

At Feathersoft we had already worked on AI Solutions and have created different models especially in the agtech industry where we had used image recognition for disease detection and crop measurements.

This led me to think about whether the same technology can be used to solve the jagged edges issue. While exploring multiple computer vision AI projects in Yolo and Deeplab, I got to the conclusion that we could resolve the background removal problem using deep learning, semantic segmentation, and encoder-decoder layers.

Initially, we planned to use pre-existing models available in the market to tackle the issue. But unfortunately, we could not find a single model that has the potential to give an accurate output. This left us with two options. Either we train a model from scratch or improve an existing model using transfer learning. Transfer Learning is a method of reusing existing weights and retrain the model. This will ultimately help in reducing time and effort for training the model.

The Data Set

Data is the foundation for any machine learning project. Fortunately, we found a data set of car images and its mask in Kaggle. But we faced a big challenge while working on the data set. The images from Kaggle have a similar background which was not suitable for our purpose.
The workflow of our requirement was like, a user would take a car picture and upload it to a car listing website, which should then remove the background of the car with transparent background.
To train our model, none of the backgrounds should be the same. To deal with this data set issue, we generated images manually using custom-made python scripts with different backgrounds and using different car models. We created around 1000 images and their masks to train the model.

The Training

The training nearly took four days to get completed. The results came after 2000 epochs over our training data. We tested the model and were very satisfied with the prediction accuracy.

Deployment

We decided to host the solution (model) in Amazon SageMaker and for converting the millions of existing images, all images will be placed in an Amazon S3 bucket, from which the job could be scheduled, and run. This job triggers a lambda function and will convert all the images from the S3 bucket and deliver it to the output bucket.

Conclusion

The original scope was to find a background remover, to remove the background of the car images. However, the Model we created has the potential to be used in a variety of verticals including advertising, e-commerce, and real estate. The model can be customized to remove the background of any primary object. Automation of image background removal using machine learning, helped in displacing the labour-intensive process and improved efficiency.

Does your business have similar challenges? I am open to new challenges and experiences, and I would love to help you in finding the best solutions for your business problems.

The author, Sudhish Chandran is the Chief Technology Officer (CTO) of Feathersoft