Fashionably

A computer vision web app that demonstrates the significant environmental impact of the textile industry using machine learning.

Comment

Inspiration

We were inspired by the terrible practice of fast fashion. Fast fashion is the business strategy and trend of closely following the latest fashion trends, manufacturing dozens of new designs over a short period. After some extensive research, we found that this practice is incredibly bad for the environment due to the high water costs and monumental carbon dioxide emissions produced. We concluded that the careless purchasing of clothing without regard for the environment is a severe issue that needs to be addressed.

What it does

Fashionably aims to bring more awareness to the environmental impact of the textile industry to the average person by showing how much damage their outfit can inflict. The user takes a photo of their outfit, and then the program figures out what they are wearing specifically and calculates the water used, carbon dioxide released, toxic chemicals, and more about their clothes. Our initial goal was to compare purchasing clothing firsthand versus secondhand. The program is web-based and allows the user to either take a picture directly on the website or upload their image.

How we built it

We used YOLOv5 to develop a supervised vision-based model to determine the clothes the person is currently wearing. The model is trained based on our own custom data set. Using web-scraped outfits from large fashion retailers, and pictures taken of our fellow hackers (with consent of course) with various outfits to create a more accurate model. We developed a web scraper using Selenium and Python to scrape eCommerce websites for professional photos of clothing.

After detecting the clothing worn, we then use predetermined constants we found through our research to calculate the total environmental impact of the outfit through a Python backend. The program aims to find an approximation of the environmental impact based on the detections from the vision model. We planned on using Flask to implement the backend into an API but did not finish this portion.

Finally, the user interface was created in the React framework, using the Material UI library for buttons and icons, and React Camera to capture the photos. We made several TypeScript components and then merged them on our main page. However, we were unable to implement the model on the website due to several scheduling and tech issues.

Challenges we ran into

While developing our model, we experienced a few issues with one of our laptops due to the extensive processing power needed and being unable to use GPU power. After extensive debugging over both days of the hackathon and professional help from the mentors we decided to use Google Colab. Additionally, our initial dataset was gathered with our custom web-scraping tool from clothing websites such as Levi's, but we realized that the original model would have difficulty processing real images due to the change in lighting and styles. To solve this problem, we crowdsourced additional images to train the model for more accurate conditions, providing a more realistic quality. Scheduling conflicts within our individual team's schedule also set us behind.

Accomplishments that we're proud of

We are especially proud of our dataset due to its uniqueness. We found very few models online that had the same level of detail; most datasets had very inconsistent quality or used professional photos. Our dataset provides a more realistic and higher-quality set of images for training clothing detection models We are quite proud of our ability to push through our constant problems. As our team member, Arshan, stated, "[We] are proud of the perseverance of our team through the variety of challenges we faced, from building a dataset to training the model, we faced our fair share of troubles."

What we learned

We learned a lot more about the React framework and TypeScript, an unfamiliar tech stack. We also learned more about machine learning, especially YOLOv5, which we had zero experience with going into this solution. This project gave us a new perspective on machine learning and web design, which we will bring back to our daily lives.

What's Next for Fashionably

We would like to finish our front end and back end and completely rework the machine learning model being used. We want to implement further crowdsourcing measures through the website to improve the dataset and grow this solution into a viable tool to be used by many.

Share this project:

Updates