Deploying SSCMA on Espressif Chips

This example is a tutorial for deploying the models from SSCMA to Espreessif chipsets, based on ESP-IDF and Tensorflow Lite Micro implementations.

Prerequisites

Hardware

• A Linux or macOS computer

• An ESP32-S3 development board with a camera (e.g. Seeed Studio XIAO)

• A USB cable

Install the ESP-IDF

SSCMA requires ESP-IDF 4.4.x for deployment in ESP32, please refer to the following tutorial ESP-IDF Get Started Guide to install and configure the toolchain of ESP-IDF.

After completed the ESP-IDF installation, please double-check IDF Environment Config Variables is finished:

• The IDF_PATH environment variable is set.

• Make sure that tools like idf.py and Xtensa-ESP32 (e.g. xtensa-esp32-elf-gcc) are included in $PATH.

TIP

We do not recommend that you configure ESP-IDF in a virtual environment, you can use the following command to exit a virtual environment (use the command multiple times to exit nested virtual environments).

conda deactivate

In addition, if your ESP-IDF is not configured in a virtual environment, any operations related to ESP-IDF, such as calls to idf.py, should be done in a non-virtual environment.

Get Examples and Submodules

Go to the root directory of the SSCMA project and run the following command to get the examples and its submodules.

git clone https://github.com/Seeed-Studio/sscma-example-esp32 -b 1.0.0 examples/esp32 && \
pushd examples/esp32 && \
git submodule init && \
git submodule update && \
popd

WARNING

You need to complete the installation and configuration of SSCMA first. If you have not installed SSCMA yet, you can refer to SSCMA Installation Guide.

Prepare the Model

Before you start compiling and deploying, you need to prepare the models that need to be deployed according to the actual application scenarios. Therefore, you may need to go through the steps of selecting a model or neural network, customizing the dataset, exporting or transforming the model, etc.

To help you understand the process in a more organized way, we have written complete documentation for different application scenarios.

• ESP32 Mask Detection

• ESP32 Meter Reader

WARNING

Before Compile and Deploy, you need to prepare the appropriate model.

Compile and Deploy

Compile

1. Go to the root directory of the SSCMA project and run the following command to access the examples directory examples/esp32.

cd examples/esp32 # SSCMA/examples/esp32

2. Set IDF_TARGET to esp32s3.

idf.py set-target esp32s3

3. Compile the example.

idf.py build

Deploy

1. Connect the ESP32 MCU to the computer and determine the serial port path. Under Linux, you can use the following command to check the currently available serial ports (the path to the serial port is typically /dev/ttyUSB0 for newly connected ESP32 devices on Linux):

lsusb -t && \
ls /dev | grep tty

2. Flash the firmware (replace <TARGET_SERIAL_PORT> with the ESP32 serial port path):

idf.py --port <TARGET_SERIAL_PORT> flash

3. Monitor serial output and wait for MCU reboot (replace <TARGET_SERIAL_PORT> with the ESP32 serial path):

idf.py --port <TARGET_SERIAL_PORT> monitor

TIP

The two commands to flash the firmware and monitor the serial port can be combined.

idf.py --port <TARGET_SERIAL_PORT> flash monitor

Use Ctrl+] to exit the monitor serial output screen.

Performance Profile

The performance of SSCMA related models, measured on different chips, is summarized in the following table.

Target	Model	Dataset	Input Resolution	Peak RAM	Inferencing Time	F1 Score	Link
ESP32-S3	Meter	Custom Meter	112x112 (RGB)	320KB	380ms	97%	pfld_meter_int8.tflite
ESP32-S3	Fomo	COCO MASK	96x96 (GRAY)	244KB	150ms	99.5%	fomo_mask_int8.tflite

TIP

For more models go to SSCMA Model Zoo

Contribute

• If you find any issues in these examples, or wish to submit an enhancement request, please use GitHub Issue.

• For ESP-IDF related issues please refer to ESP-IDF.

• For information about TensorFlow Lite Micro, please refer to TFLite-Micro.

• For SSCMA related information, please refer to SSCMA.

Licensing

These examples use ESP-IDF, which is released under the Apache 2.0 License.

The TensorFlow library code and third-party code contain their own licenses, which are described in TFLite-Micro.