Intro to classification
with MobileNet

Are you ready to discover classification with MobileNet? In this module, you will create your image classification application! A few years ago, being able to automatically detect the content of a picture was science fiction, now you can do it easily on a smartphone!

Watch the video above first, then answer the quiz to make sure you understand the main notions. Some questions may need to be looked up elsewhere through a quick Internet search!

Create and prototype an application that might benefit from image classification using MobileNet.
Think of a coherent user journey: while MobileNet is able to say if a picture contains a dangerous shark or a cute dolphin… are you really going to use your phone while swimming in the sea? Especially if you see a fin getting closer to you? 🦈

Tools

p5.js Web Editor is a web editor for p5.js, a JavaScript library with the goal of making coding accessible to artists, designers, educators, and beginners.

TensorFlow.js is a machine learning framework developed by Google. MobileNet, the model we are using for this project is one of its available pre-trained models.

MobileNet classifies images with labels from the ImageNet database. MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints of a variety of use cases.

Project

If you don't have a GitHub account, now is the time to sign up!

1. Log in on the p5.js web editor with your GitHub account and create a copy of this project

2. Use MobileNet to create an application that detects automatically at least two objects of your choice from the classes available on MobileNet.

   You can find the full list of the classes that our pre-trained model can detect on its GitHub repository.

Describe a use case associated with your application in your webpage. We expect at least an explanation on who would be your users and how they would benefit from your application.

What do we expect when we ask for a "use case"?

A basic description of your users, the problem they are facing, and how your app will help them is enough.
A long description is not mandatory, but give us some context. On the other hand, if your use case is 50 characters long, something's wrong!

Example:

• This is an app for recognizing different species of mushrooms.
• This app, made for occasionnal hikers, is able to differenciate between gyromitra and hen-of-the-woods, two species of mushroom that look similar, one of them is venomous while the other isn't. By using our app on their phone they can identify which one of the two they are facing.
  Because a bug could have catastrophic consequences, we added clear explanations about how to differentiate them. With further development, other species of mushrooms and a map of the nearest profesionnals could be added to our app.

You can also follow the Five Ws method.

Describe a use case associated with your application in your webpage. For example, who would be your users and how they would benefit from your application.

Examples

Here are some examples of great projects that were submitted by students from previous sessions:

Going further

Definitions

• Machine Learning is a subfield of Artificial Intelligence, it's a research domain that develops algorithms, called models, which "learns" from data instead of classical algorithm which implements a set of predefined rules. The more (quality) data you give to the algorithm, to more accurate it gets (to a certain extend).
• Supervised Learning is a subfield of Machine Learning, it's a type of model where your training datasets includes both inputs and outputs. It means that you have the first set of data (the training dataset) where you know the expected output(s) and you want the model to be able to extrapolate and predict a new output from data it has never encountered. The opposite is Unsupervised Learning, when you're trying to make sense of a dataset without knowing the expected output (for example finding similarities in a client database).
• Classification is a subfield of Supervised Learning, where the expected output is discrete (think different categories, or yes/no). The opposite, when the output is continuous (a number), is called regression.
• MobileNets are a series of Convolutional Neural Networks, that are made to do image recognition tasks on low-latency and low-power application (embedded or mobile for example)
• ImageNet is an image database of nearly 15 million images ordered by categories.. that's why we can use them to train classification models! It's also the name of different machine learning competitions (see this one for example)

Tools

• TensorFlow is an open-source Python framework developed by Google, originally developed for doing complex calculations, it quickly became one of the most used tools for machine learning.
  Tensorflow.js is its Javascript implementation. You just learned how to use MobileNet, one of its pre-trained models.
• ml5.js is a Javascript framework developed by teachers from NYU. Built on top of Tensorflow.js it enables you to quickly use a pre-trained model in your browser.
• Runway ML is a desktop app that makes it easy to try a lot of machine learning models.

Resources

• TensorFlow Playground, let you with a neural network directly in a browser. You will learn how machine learning models really "learns" from data and what is a hyperparameter.
• 2D Visualisation of a CNN, trying to recognize handwritten digits.
• iNaturalist, an example of a crowdsourcing website for wildlife conservation