DIY Projects

TensorFlow Lite Micro is available for ESP32 and ESP32-EYE (ESP32-CAM)

Espressif has just announced on its blog the availability of TensorFlow Lite Micro for ESP32 SoCs. For those who do not yet know, Tensor Flow is an open source platform dedicated to machine learning supported by Google. The source code is fully available on GitHub here. The Lite version eMicro is a light version intended for microcontrollers.


TensorFlow Lite Micro for ESP32 SoCs and other microcontrollers

TensorFlow Lite Micro (TFLM) is a lightweight yet experimental version suitable for microcontrollers.

TFLM is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory.

No compatibility with the operating system is necessary. It does not require any standard C or C ++ library, or dynamic memory allocation. The main runtime component uses only 16KB on an Arm Cortex M3 processor. For example for speech recognition, TFLM takes up only 22 KB of memory to run a keyword detection model.

Several examples are available in the GitHub repository maintained by Espressif here.

Hello World

This example is designed to demonstrate the basics of using TensorFlow Lite For Microcontrollers.
The model is trained to replicate a “sine wave” function and generates a data model to flash LEDs or control animation, depending on the capabilities of the device.

Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge STM32F746

Doorbell camera

This example shows how to create a smart doorbell connected with an ESP32-CAM module (ESP-EYE at Espressif). It is based on the standard TensorFlow person detection example. When a person is detected by the camera (note detected, unidentified), an email is sent with the captured image to a configured email address. This example uses a 250KB neural network to detect people in images captured by the camera.

Platforms ESP32-EYE

Pre-configured pinout

Before compiling, several parameters must be configured. Run this command to launch the configuration tool menuconfig

Choose the camera model, go to the menu Camera Pins -> Select Camera Pinout

SMTP Configuration to enter the connection parameters to your email account (gmail by default)

Example Connection Configuration to enter the connection parameters to the local WiFi network

A small demonstration of the operation in the form of a GIF animation.

More details for the compilation of the project on the Tensor Flow blog.

Image Recognition

This example shows how to use Tensorflow Lite Micro to perform image recognition on an STM32F746 development kit and STM32F4DIS-CAM camera module. It classifies the captured image into 1 of 10 different classes: Airplane, Car, Bird, Cat, Deer, Dog, Frog, Horse, Ship and Truck.

Magic wand

This example shows how to use TensorFlow Lite to run a 20 kilobyte neural network model to recognize gestures with an accelerometer.

The sample app reads the accelerometer data on an Arduino Nano 33 BLE Sense or SparkFun Edge board and indicates when it has detected a gesture, then sends the gesture to the serial port.

Platforms Arduino SparkFun Edge

Micro Speech

This example shows how to run a 20KB model that can recognize 2 keywords, “yes” and “no”, from voice data.

The application listens to its surroundings with a microphone and indicates when it has detected a word by lighting an LED or by displaying data on a screen.

The code has a small footprint (for example, about 22 kilobytes on a Cortex M3) and only uses about 10 kilobytes of RAM for working memory, so it is capable of running on systems like a STM32F103 with only 20 kilobytes of total SRAM and 64 kilobytes of Flash.

Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge STM32F746 macOS NXP FRDM K66F

Network Tester testeur réseau

The purpose of this app is to provide a quick way to test different networks.

It contains a test case and a default network model (network_model.h), default input data (input_data.h), and default expected output data (expected_output_data.h). The header files were created using the xxd command.

The default pattern is a single MaxPool2D operator, with an input form of {1, 4, 4, 1} and an output form of {1, 2, 2, 1}.

It is possible to define your own models.

Person detection 2 exemples

This example shows how to use Tensorflow Lite to run a 250 kilobyte neural network to recognize people in images captured by a camera.

Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge

Install the ESP-IDF environment

Before you can test the delivered samples, you must install the ESP-IDF framework on your computer.

Note. ESP32 SDK for Arduino IDE is not enough

Follow the instructions in this tutorial

The official page describing the installation procedure for Windows, macOS and Linux

Install gmake (make) on macOS (Mojave or higher)

On macOS, the Make command must have a version greater than 3.81, open a Terminal to check the installed version

make --version

However, it seems that the make command is causing compilation errors. In this case, it is better to install the gmake command using brew with the command

brew install make


brew reinstall make

To check the installed version, run

gmake --version
GNU Make 4.3
Built for x86_64-apple-darwin18.7.0

If the topic interests you, this book by Pete Warden and Daniel Situnayake is a good place to start.

Click to rate this post!
[Total: 0 Average: 0]
Exit mobile version