TensorFlow Lite Micro is available for ESP32 and ESP32-EYE (ESP32-CAM)

esp32-cam tensorflow lite example esp-eye
Share on facebook
Share on twitter
Share on linkedin
Share on pinterest
Share on email
Share on whatsapp

Espressif has just announced on its blog the availability of TensorFlow Lite Micro for ESP32 SoCs. For those who do not yet know, Tensor Flow is an open source platform dedicated to machine learning supported by Google. The source code is fully available on GitHub here. The Lite version eMicro is a light version intended for microcontrollers.

 

TensorFlow Lite Micro for ESP32 SoCs and other microcontrollers

TensorFlow Lite Micro (TFLM) is a lightweight yet experimental version suitable for microcontrollers.

TFLM is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory.

No compatibility with the operating system is necessary. It does not require any standard C or C ++ library, or dynamic memory allocation. The main runtime component uses only 16KB on an Arm Cortex M3 processor. For example for speech recognition, TFLM takes up only 22 KB of memory to run a keyword detection model.

Several examples are available in the GitHub repository maintained by Espressif here.

Hello World

This example is designed to demonstrate the basics of using TensorFlow Lite For Microcontrollers.
The model is trained to replicate a “sine wave” function and generates a data model to flash LEDs or control animation, depending on the capabilities of the device.

Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge STM32F746

Doorbell camera

This example shows how to create a smart doorbell connected with an ESP32-CAM module (ESP-EYE at Espressif). It is based on the standard TensorFlow person detection example. When a person is detected by the camera (note detected, unidentified), an email is sent with the captured image to a configured email address. This example uses a 250KB neural network to detect people in images captured by the camera.

Platforms ESP32-EYE

Pre-configured pinout

  • WROVER-KIT with OV2640 module (TTGO T-Camera for example)
  • ESP-EYE DevKit by Espressif
  • M5Stack Camera with PSRAM
  • M5Stack Camera F (Wide)
  • ESP32-CAM by AI-Thinker
  • User pinout (Power Down, Reset, XCLK, SIOD, SIOC, VSYNC, HREF, PCLK, Y2, Y3, Y4, Y5, Y6, Y7, Y8, Y9)

Before compiling, several parameters must be configured. Run this command to launch the configuration tool

idf.py menuconfig

Choose the camera model, go to the menu Camera Pins -> Select Camera Pinout

esp-eye esp32-cam tensorflow lite mcu camera pins setup

SMTP Configuration to enter the connection parameters to your email account (gmail by default)

Example Connection Configuration to enter the connection parameters to the local WiFi network

A small demonstration of the operation in the form of a GIF animation.

esp32-cam tensorflow lite example esp-eye

More details for the compilation of the project on the Tensor Flow blog.

Image Recognition

This example shows how to use Tensorflow Lite Micro to perform image recognition on an STM32F746 development kit and STM32F4DIS-CAM camera module. It classifies the captured image into 1 of 10 different classes: Airplane, Car, Bird, Cat, Deer, Dog, Frog, Horse, Ship and Truck.

Magic wand

This example shows how to use TensorFlow Lite to run a 20 kilobyte neural network model to recognize gestures with an accelerometer.

The sample app reads the accelerometer data on an Arduino Nano 33 BLE Sense or SparkFun Edge board and indicates when it has detected a gesture, then sends the gesture to the serial port.

Platforms Arduino SparkFun Edge

Micro Speech

This example shows how to run a 20KB model that can recognize 2 keywords, “yes” and “no”, from voice data.

The application listens to its surroundings with a microphone and indicates when it has detected a word by lighting an LED or by displaying data on a screen.

The code has a small footprint (for example, about 22 kilobytes on a Cortex M3) and only uses about 10 kilobytes of RAM for working memory, so it is capable of running on systems like a STM32F103 with only 20 kilobytes of total SRAM and 64 kilobytes of Flash.

Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge STM32F746 macOS NXP FRDM K66F

Network Tester testeur réseau

The purpose of this app is to provide a quick way to test different networks.

It contains a test case and a default network model (network_model.h), default input data (input_data.h), and default expected output data (expected_output_data.h). The header files were created using the xxd command.

The default pattern is a single MaxPool2D operator, with an input form of {1, 4, 4, 1} and an output form of {1, 2, 2, 1}.

It is possible to define your own models.

Person detection 2 exemples

This example shows how to use Tensorflow Lite to run a 250 kilobyte neural network to recognize people in images captured by a camera.

Platforms Synopsis DesignWare ARC EM Arduino ESP32 SparkFun Edge

Install the ESP-IDF environment

Before you can test the delivered samples, you must install the ESP-IDF framework on your computer.

Note. ESP32 SDK for Arduino IDE is not enough

Follow the instructions in this tutorial

The official page describing the installation procedure for Windows, macOS and Linux

Install gmake (make) on macOS (Mojave or higher)

On macOS, the Make command must have a version greater than 3.81, open a Terminal to check the installed version

make --version

However, it seems that the make command is causing compilation errors. In this case, it is better to install the gmake command using brew with the command

brew install make

or

brew reinstall make

To check the installed version, run

gmake --version
GNU Make 4.3
Built for x86_64-apple-darwin18.7.0

If the topic interests you, this book by Pete Warden and Daniel Situnayake is a good place to start.

Click to rate this post!
[Total: 0 Average: 0]

Are you having a problem with this topic?

Maybe someone has already found the solution, visit the forum before asking your question
Ask your question

Share on facebook
Share on twitter
Share on linkedin
Share on pinterest
Share on email
Share on whatsapp

Did you like this project ? Don't miss any more projects by subscribing to our weekly newsletter!

We will be happy to hear your thoughts

      Leave a Reply

      DIY Projects