Espressif has just announced on its blog the availability of TensorFlow Lite Micro for ESP32 SoCs. For those who do not yet know, Tensor Flow is an open source platform dedicated to machine learning supported by Google. The source code is fully available on GitHub here. The Lite version eMicro is a light version intended for microcontrollers.
TensorFlow Lite Micro for ESP32 SoCs and other microcontrollers
TensorFlow Lite Micro (TFLM) is a lightweight yet experimental version suitable for microcontrollers.
TFLM is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory.
No compatibility with the operating system is necessary. It does not require any standard C or C ++ library, or dynamic memory allocation. The main runtime component uses only 16KB on an Arm Cortex M3 processor. For example for speech recognition, TFLM takes up only 22 KB of memory to run a keyword detection model.
Several examples are available in the GitHub repository maintained by Espressif here.
This example is designed to demonstrate the basics of using TensorFlow Lite For Microcontrollers.
The model is trained to replicate a “sine wave” function and generates a data model to flash LEDs or control animation, depending on the capabilities of the device.
Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge STM32F746
This example shows how to create a smart doorbell connected with an ESP32-CAM module (ESP-EYE at Espressif). It is based on the standard TensorFlow person detection example. When a person is detected by the camera (note detected, unidentified), an email is sent with the captured image to a configured email address. This example uses a 250KB neural network to detect people in images captured by the camera.
Platforms ESP32-EYE
Pre-configured pinout
- WROVER-KIT with OV2640 module (TTGO T-Camera for example)
- ESP-EYE DevKit by Espressif
- M5Stack Camera with PSRAM
- M5Stack Camera F (Wide)
- ESP32-CAM by AI-Thinker
- User pinout (Power Down, Reset, XCLK, SIOD, SIOC, VSYNC, HREF, PCLK, Y2, Y3, Y4, Y5, Y6, Y7, Y8, Y9)
Before compiling, several parameters must be configured. Run this command to launch the configuration tool
idf.py menuconfigChoose the camera model, go to the menu Camera Pins -> Select Camera Pinout
SMTP Configuration to enter the connection parameters to your email account (gmail by default)
Example Connection Configuration to enter the connection parameters to the local WiFi network
A small demonstration of the operation in the form of a GIF animation.
More details for the compilation of the project on the Tensor Flow blog.
This example shows how to use Tensorflow Lite Micro to perform image recognition on an STM32F746 development kit and STM32F4DIS-CAM camera module. It classifies the captured image into 1 of 10 different classes: Airplane, Car, Bird, Cat, Deer, Dog, Frog, Horse, Ship and Truck.
This example shows how to use TensorFlow Lite to run a 20 kilobyte neural network model to recognize gestures with an accelerometer.
The sample app reads the accelerometer data on an Arduino Nano 33 BLE Sense or SparkFun Edge board and indicates when it has detected a gesture, then sends the gesture to the serial port.
Platforms Arduino SparkFun Edge
This example shows how to run a 20KB model that can recognize 2 keywords, “yes” and “no”, from voice data.
The application listens to its surroundings with a microphone and indicates when it has detected a word by lighting an LED or by displaying data on a screen.
The code has a small footprint (for example, about 22 kilobytes on a Cortex M3) and only uses about 10 kilobytes of RAM for working memory, so it is capable of running on systems like a STM32F103 with only 20 kilobytes of total SRAM and 64 kilobytes of Flash.
Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge STM32F746 macOS NXP FRDM K66F
testeur réseau
The purpose of this app is to provide a quick way to test different networks.
It contains a test case and a default network model (network_model.h), default input data (input_data.h), and default expected output data (expected_output_data.h). The header files were created using the xxd command.
The default pattern is a single MaxPool2D operator, with an input form of {1, 4, 4, 1} and an output form of {1, 2, 2, 1}.
It is possible to define your own models.
2 exemples
This example shows how to use Tensorflow Lite to run a 250 kilobyte neural network to recognize people in images captured by a camera.
Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge
Install the ESP-IDF environment
Before you can test the delivered samples, you must install the ESP-IDF framework on your computer.
Note. ESP32 SDK for Arduino IDE is not enough
Follow the instructions in this tutorial
The official page describing the installation procedure for Windows, macOS and Linux
Install gmake (make) on macOS (Mojave or higher)
On macOS, the Make command must have a version greater than 3.81, open a Terminal to check the installed version
make --versionHowever, it seems that the make command is causing compilation errors. In this case, it is better to install the gmake command using brew with the command
brew install makeor
brew reinstall makeTo check the installed version, run
gmake --version
GNU Make 4.3
Built for x86_64-apple-darwin18.7.0If the topic interests you, this book by Pete Warden and Daniel Situnayake is a good place to start.
- New TTGO T5-4.7 ESP32 with ePaper and LiPo or 18650 holder
- TensorFlow Lite Micro is available for ESP32 and ESP32-EYE (ESP32-CAM)
- New STM32WL LoRa and STM32WB with Bluetooth Zigbee and OpenThread
- New LoLin W600-PICO and WeMos d1 mini compatible shields
