Friendly Flutter - Part 1/5

Published in

QuikNapp

5 min readMar 29, 2021

ML Kit in Flutter

In the first article of Friendly Flutter series, we are going to learn a little about ML Kit. At its I/O 2018 developer conference in May, Google introduced ML Kit, a cross-platform suite of machine learning tools for its Firebase mobile development platform. ML Kit is designed to compress and optimize machine learning models for mobile devices.

It also offers a couple of easy-to-use APIs for basic use cases. We will be learning some of these today.

Text Recognition
Barcode Scanning
Image Labelling

Setting up a new Firebase Project

Create a new Flutter project using the command below and then open it in an IDE. Here I’m naming the project mlKit_flutter

flutter create mlkit_flutter

In order to use Firebase ML Kit in your Android app, you have to first complete the Firebase setup.

Go to Firebase Console and click Add Project. Enter your project name and press Continue. For example, mlKit-flutter
Click Create project (or Add Firebase, if you’re using an existing Google Cloud project).
When your project is ready and looks something like this, click on the Android icon below “Get started by adding Firebase to your app”.

Now we are going to Add Firebase to our Android app

To know your package name, head over to android/app/src/main/AndroidManifest.xml
From the second line, you can add your package name, it must be in the format “com.example.projectName”
To get SHA-1 and SHA-256, run this command by changing the Username and then click on Register App

keytool -list -v -keystore "C:\Users\USERNAME\.android\debug.keystore" -alias androiddebugkey -storepass android -keypass android

Next, you need to download google-services.json file and place it in android/app folder. You need to modify your build.gradle file. Follow the instructions given there to do so.

Now we are ready to go!

Adding Plugins

We are going to add 3 Plugins to use ML Kit properly:

Head over to pubspec.yaml file and add these under dependencies

Create a stateful widget and few variables.

image of File type, a reference to a file on the file system
isImageLoaded of bool value to mark if image is loaded completely or not
third variable text, to store the recognized text
The new ImagePicker API does not rely in static methods anymore, so picker is a new instance of the plugin

Now we will create a async function getImage().

An async function is a function whose body is marked with an async modifier.
When you call an async function, it immediately returns a Future; the body of the function is scheduled for execution later. When the body has executed, the Future that was returned by the call is completed with the result.

setState is used to update a widget tree generally with some new data. Here, we are updating the value of the image and isImageLoaded.

Okay, the common part of all these features is complete. Let’s get down to each of these features and learn the implementation.

Text Recognition

With ML Kit’s text recognition APIs, you can recognize text in any Latin-based language.
Text recognition can automate tedious data entry for credit cards, receipts, and business cards. With the Cloud-based API, you can also extract text from pictures of documents, which you can use to increase accessibility or translate documents. Apps can even keep track of real-world objects, such as by reading the numbers on trains.

In the first line, we are creating a FirebaseVisionImage object from the image. In the next line, we are creating an instance of a detector. Here it is TextRecognizer. Next, we are calling processImage() with visionImage.

Now, we have to retrieve the texts from the visionText, then separate out the text from it and store it in the variable text. The texts are present in blocks -> lines -> elements.

Here is the UI that I used to display the image and the recognized text.

And it’s done!

Barcode Scanning

With ML Kit’s barcode scanning API, you can read data encoded using most standard barcode formats. Barcode scanning happens on the device, and doesn’t require a network connection.

Now, we have to retrieve the texts from the visionText, then separate out the text from it and store it in the variable text. The texts are present in blocks -> lines -> elements.

Here’s the UI sample that I used for barcode scan feature.

Image Labeling

With ML Kit’s image labeling APIs, you can recognize entities in an image without having to provide any additional contextual metadata, using either an on-device API or a cloud-based API.
Image labeling gives you insight into the content of images. When you use the API, you get a list of the entities that were recognized: people, things, places, activities, and so on. Each label found comes with a score that indicates the confidence the ML model has in its relevance.

Create a FirebaseVisionImage object from your image. The next step is to create an instance of the detector that we’d like to use.

We’re now ready to process the image using the detector. Since this will return a Future, we run the code in a function that returns a Future. With this object, we can call imageLabeler() to get a labeler and call processImage() to detect image labels. ImageLabel class contains information about labels. When the process of image labeling is successful, you’ll get a FirebaseVisionImageLabel object. Each object contains the a detected label from the image, its Knowledge Graph entity ID and the overall confidence of the result between 0.0 to 1.0.

Here’s the UI of the Image Labeling feature that I used.

That’s it for today. If you liked the blog, don’t forget to leave a clap. Try to explore other features of ML Kit. Till then, stay tuned for part 2 on next Monday.