ESP32 HMI 4.3-inch Arduino Tutorial¶
Overview¶
This example tutorial is a project for controlling the on and off of an LED. It demonstrates how to create a UI and how to control the on and off of an LED through buttons on the screen.
In this tutorial, we will show you how to design the UI with SquareLine Studio, and show you how to upload the code with Arduino IDE.
Hardware Preparation¶
CrowPanel ESP32 4.3'' HMI | Crowtail-DHT20 | Crowtail-LED |
---|---|---|
Design UI file with SquareLine Studio¶
Get Started with SquareLine Studio¶
Please click the card below to learn how to download the SquareLine Studio, and how to export the demo UI file.
Please click to download the demo for "Example Demonstration" section in "Get Started with SquareLine Studio".
Design UI file with SquareLine Studio¶
Let's start learning how to create our own UI after getting an initial understanding of SquareLine Studio.
-
Open the SquareLine Studio and create a project.
A. In the latest version of SLS, select "Elecrow"->"DIS06043H...Arduino-IDE"
B. In the earlier version of SLS, select "Arduino"->"Arduino with TFT_eSPI".
Note:
When select the Arduino framwork, there's only one option "Arduino with TFT_eSPI". By choosing this, the squareline studio will generate a template code suitable for the TFT_eSPI library. However, squareline studio not only supports the TFT_eSPI library, it supports a variety of libraries to suit different hardware and application needs. For example, Adafruit_GFX library, LovyanGFX etc.
After using SLS to generate UI code, we then use different graphics libraries according to different hardware and modify the corresponding code to display the content you design.
Set the name of the project, set the screen resolution to 480*272, set the color depth to 16bit, and keep other default settings. After setting, click CREATE to create the project.
- 16-bit color depth: can represent 65,536 colors through RGB 5:6:5 sub-pixel representation, that is, each RGB channel occupies 5 bits and 1 bit (a total of 16 bits) to represent colors.
-
After creation, enter the following interface with a blank background.
-
In the "Assets" area, click "ADD FILE TO ASSETS" to add custom images or icons.
Please click to download the custom images used in this tutorial.
Note:
Images only support PNG format. The pixels of the image need to be smaller than the pixel size of the screen used in your project. The size of each image should not exceed 100k, preferably within 30k, to provide a smooth display effect.
-
Add background.
Find "Inspector"->"STYLE SETTING", click to expand "STYLE(MAIN)", then click the 2nd "Background". Check the "Bg Image" and select the background image.
-
Add Button widget to control the LED.
Click "Button" in the "Widgets" area, and "Button1" will be added to the current Screen.
The position and size of the label can be adjusted by dragging the mouse. You can also directly enter numbers in the Inspector→BUTTON→Transform to adjust.
Add an identification symbol to the button. The button in this tutorial controls the LED switch, so you only need to mark the button "on" and "off". You can add LABEL widgets or add a background images to the button. This tutorial will demonstrate how to add a background image to a button.
Click the Button1, then find Inspector->STYLE SETTINGS ->STYLE(MAIN) ->Background, and select the image.
In the same way, duplicate a Button widget. And drag it to the corresponding position to modify different background image.
Set the status of the button to identify different states.
In "Inspector"->"STYLE SETTINGS"->"STATE", set display white background color by DEFAULT and red when on the PRESSED state.
Make the same settings for the "OFF" button.
-
Add events to buttons.
Note: Because the button controls the on and off of the LED, we can add any event here to generate the code framework for the button event when exporting the UI file. We will modify the code of the button event to control the LED latter.
Select the button and click "ADD EVENT".
Select "released" as the trigger condition, select a trigger event in "Action". It will be modified in the generated program to achieve the LED control function.
Complete the event. Here I choose to change the screen, and the screen to be switched is Screen1.
Add event to Button2 (OFF) in the same way.
-
Export UI files.
Click "File" -> "Project Settings" and make settings for the exported file.
Set the export path of the file (set the path according to your own file).
Fill in lvgl.h in LVGL Include Path. Check "Flat export(exports all files to one folder )".
Then click "APPLY CHANGES".
Tips: After selecting the flat export, the output files will be in the same folder, so that the output code does not need to modify the path in the program. If not, the output files will be classified and placed in different folders. The compiler may not be able to recognize different paths, which will cause some trouble. In this case, the user needs to modify it manually, so it is recommended to select all files to be output to the same folder.
Export UI files. The exported files will be in the path we set earlier.
In order to be used with the main program, all UI files generated need to be placed in the same folder as the main program.
Please click to download the main program(and the expert UI is included).
-
Modify the button event code.
Open the main program. And all the UI file in the same folder will open too.
First, define an extern variable in the ui.h file to store the status of the LED. In the main program, the LED light is controlled to turn on and off by judging the status of this variable.
Then find "FUNCTION" in the ui.c file. Here is the corresponding code generated when we add events in SquareLine.
Comment out or delete the code circled in red in the picture above, add new code to assign a value to the LED variable.
-
When button 1 (ON button) is pressed, set the LED value to 1.
-
When button 2 (OFF button) is pressed, set the LED value to 0.
-
The UI file export is completed, and the button event function is also modified. Next we're going to learn about the Arduino main program and learn how to upload the code to the board.
Build the Project with Arduino IDE¶
Get Started with Arduino IDE¶
Please click the card below to learn how to install Arduino IDE, and install ESP32 board in the Arduino IDE.
Install Libraries¶
In this project, we will use the following libraries:
#include <lvgl.h>
#include <SPI.h>
#include <Arduino_GFX_Library.h>
#include "ui.h"
#include "touch.h"
#include <lvgl.h>
: library for graphical interfaces.#include <Arduino_GFX_Library.h>
: library for screen drivetouch.h
: touch drive.#include "ui.h"
: is a custom header file that contain UI-related functions or definitions.
Please click to download the libraries we modified.
Then click "File" -> "Preferences" -> "Setting" to check the sketchbook location. Place the libraries downloaded to the sketchbook location.
Tips:
- If you install the lvgl libraries by Library Manager, you need to copy the demos folder and examples folder and paste them to src folder. And modify the lv_conf_template.h file and rename it to lv_conf.h, and place it in libraries folder.(Please refer to the library provided)
Code Explanation¶
Basic Definition
-
Backlight
-
Create a default Arduino_GFX object lcd
#if defined(DISPLAY_DEV_KIT) Arduino_GFX *lcd = create_default_Arduino_GFX(); #else /* !defined(DISPLAY_DEV_KIT) */ Arduino_ESP32RGBPanel *bus = new Arduino_ESP32RGBPanel( GFX_NOT_DEFINED /* CS */, GFX_NOT_DEFINED /* SCK */, GFX_NOT_DEFINED /* SDA */, 40 /* DE */, 41 /* VSYNC */, 39 /* HSYNC */, 42 /* PCLK */, 45 /* R0 */, 48 /* R1 */, 47 /* R2 */, 21 /* R3 */, 14 /* R4 */, 5 /* G0 */, 6 /* G1 */, 7 /* G2 */, 15 /* G3 */, 16 /* G4 */, 4 /* G5 */, 8 /* B0 */, 3 /* B1 */, 46 /* B2 */, 9 /* B3 */, 1 /* B4 */ ); // option 1: // ILI6485 LCD 480x272 Arduino_RPi_DPI_RGBPanel *lcd = new Arduino_RPi_DPI_RGBPanel( bus, 480 /* width */, 0 /* hsync_polarity */, 8 /* hsync_front_porch */, 4 /* hsync_pulse_width */, 43 /* hsync_back_porch */, 272 /* height */, 0 /* vsync_polarity */, 8 /* vsync_front_porch */, 4 /* vsync_pulse_width */, 12 /* vsync_back_porch */, 1 /* pclk_active_neg */, 7000000 /* prefer_speed */, true /* auto_flush */); #endif /* !defined(DISPLAY_DEV_KIT) */
/*It uses a class called Arduino_ESP32RGBPanel to initialize a display bus object bus. This object is used to control the ESP32-based RGB display panel. Then, according to option 1 in the comments, it uses the Arduino_RPi_DPI_RGBPanel class to initialize a display object called lcd. This object is used to control an ILI6485 LCD with a resolution of 480x272 pixels.*/
-
Resolution
-
Define a buffer variable to initialize the display driver of LVGL and prepare a buffer to store pixel data that will be rendered to the screen. The size of this buffer should be around 1/10 of the product of screen pixel values, which can balance the main control performance and the smoothness of screen display. Taking 10 as an example, the display will be smoother.
-
Define the led variable and the instance of the SPI library
Function
- The
my_disp_flush()
function is used to refresh the display. It receives a display driver pointerdisp
of typelv_disp_drv_t
, alv_area_t
structure pointer area representing the refresharea
, and alv_color_t
pointercolor_p
pointing to the color data. In the function, the widthw
and heighth
of the refresh area are first calculated. Then, depending on whether theLV_COLOR_16_SWAP
macro is defined, different methods are selected to draw the color data to the display device. If theLV_COLOR_16_SWAP
macro is defined, thedraw16bitBeRGBBitmap()
method is used to draw the color data; otherwise, thedraw16bitRGBBitmap()
method is used. Finally, thelv_disp_flush_ready()
function is called to notify LittlevGL that the display has been refreshed.
/* Display Refresh */
/* Display flushing */
void my_disp_flush(lv_disp_drv_t *disp, const lv_area_t *area, lv_color_t *color_p)
{
uint32_t w = (area->x2 - area->x1 + 1);
uint32_t h = (area->y2 - area->y1 + 1);
#if (LV_COLOR_16_SWAP != 0)
lcd->draw16bitBeRGBBitmap(area->x1, area->y1, (uint16_t *)&color_p->full, w, h);
#else
lcd->draw16bitRGBBitmap(area->x1, area->y1, (uint16_t *)&color_p->full, w, h);
#endif
lv_disp_flush_ready(disp);
}
- The
my_touchpad_read()
function is used to process touch input. It receives an input device driver pointerindev_driver
of typelv_indev_drv_t
, and a pointer to thelv_indev_data_t
structure data representing the input data. In the function, first check whether the touch screen has a signal. If there is a signal, further check whether the touch is pressed or released. If the touch is pressed, set the input state toLV_INDEV_STATE_PR
, and set the coordinates of the input point to the coordinates of the last touch of the touch screen. If the touch is released, set the input state toLV_INDEV_STATE_REL
. If there is no signal on the touch screen, set the input state toLV_INDEV_STATE_REL
, indicating that the input device is in the released state.
void my_touchpad_read(lv_indev_drv_t *indev_driver, lv_indev_data_t *data)
{
if (touch_has_signal())
{
if (touch_touched())
{
data->state = LV_INDEV_STATE_PR;
/*Set the coordinates*/
data->point.x = touch_last_x;
data->point.y = touch_last_y;
Serial.print( "Data x :" );
Serial.println( touch_last_x );
Serial.print( "Data y :" );
Serial.println( touch_last_y );
}
else if (touch_released())
{
data->state = LV_INDEV_STATE_REL;
}
}
else
{
data->state = LV_INDEV_STATE_REL;
}
}
Set Up Part
void setup()
{
Serial.begin( 9600 ); /*Initializing the Serial Port*/
//IO Port Pins
pinMode(38, OUTPUT);
digitalWrite(38, LOW);
pinMode(0, OUTPUT);//TOUCH-CS
lv_init(); // LVGL initialization
// Init Display
lcd->begin();
lcd->setTextSize(2);
lcd->fillScreen(BLACK);
delay(100);
touch_init(); // Touch initialization
//Backlight Pins
pinMode(27, OUTPUT);
digitalWrite(27, HIGH);
lcd.setRotation(1);
/*Screen display initialization, by calling the lv_disp_draw_buf_init function to initialize the drawing buffer. This function accepts three parameters: &draw_buf is a pointer to the lv_disp_draw_buf_t structure, buf1 is a buffer for storing pixel data, and the last parameter is NULL to indicate that no additional buffer is used. The buffer size calculated is screenWidth * screenHeight / 8, and it is associated with the drawing buffer structure.
The size of this buffer affects the smoothness. The larger the buffer, the smoother the display. The size is limited by its own memory.*/
screenWidth = lcd->width();
screenHeight = lcd->height();
lv_disp_draw_buf_init( &draw_buf, buf1, NULL, screenWidth * screenHeight / 8 );
/* Initialize the display */
lv_disp_drv_init(&disp_drv);
/* Change the following line to your display resolution */
disp_drv.hor_res = screenWidth;
disp_drv.ver_res = screenHeight;
disp_drv.flush_cb = my_disp_flush;
disp_drv.draw_buf = &draw_buf;
lv_disp_drv_register(&disp_drv);
/*Initialize the (virtual) input device driver. This code is used to initialize a virtual input device driver and associate it with the LVGL framework to handle the interaction of the input device. It provides the necessary configuration and functions for the input device driver by setting the input device type and defining the read callback function.*/
/* Initialize the (dummy) input device driver */
static lv_indev_drv_t indev_drv;
lv_indev_drv_init(&indev_drv);
indev_drv.type = LV_INDEV_TYPE_POINTER;
indev_drv.read_cb = my_touchpad_read;
lv_indev_drv_register(&indev_drv);
ui_init();
}
Main Program Part
void loop()
{
if(led == 1)
digitalWrite(38, HIGH);
if(led == 0)
digitalWrite(38, LOW);
lv_timer_handler(); /* let the GUI do its work */
delay( 10 );
}
Upload the Code¶
-
After completing the installation of the ESP32 board according to "Get Started with Arduino IDE" and the installation of the library according to "Install Libraries", open the program and connect the ESP32 HMI 4.3-inch to the computer via a USB-C cable.
-
Select Board: click "Tools" -> "Board" -> "esp32" and select "ESP32S3 Dev Module"
-
Board Setting: Under the "Tools" menu, see "Flash Size" select 4MB(32Mb); "Partition scheme" and select "Huge APP(3MB No OTA/1MB SPIFFS)"; "PSRAM" select "QSPI PSRAM"
-
Select Port
-
Upload the code: click the "Upload" icon to upload the code.
-
After the program is successfully uploaded, connect the LED to GPIO_D. Click the ON button on the screen and the LED will turn on. Click the OFF button and the LED will turn off.
Example Demo of ESP32 HMI Function¶
Note:
The library used in this section is in the compressed package of the library downloaded above.
Example1: LED blinking.¶
Connect the LED to GPIO_D(IO38) port, and upload the following code to the board. The LED will blinking.
#define D_PIN 38
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
pinMode(D_PIN, OUTPUT);
}
void loop() {
// put your main code here, to run repeatedly:
digitalWrite(D_PIN, HIGH);
delay(500);
digitalWrite(D_PIN, LOW);
delay(500);
}
Example2: Play music¶
Plug the speaker into SPK port. Please check if you have installed the library ESP32-audioI2S(Copy the ESP32-audioI2S-master folder in the libraries downloaded above to the .../arduino/libraries directory). Upload the following code to the board.
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <FS.h>
#include "Audio.h"
Audio audio;
#define I2S_DOUT 20
#define I2S_BCLK 35
#define I2S_LRC 19
//4.3
#define SD_MOSI 11
#define SD_MISO 13
#define SD_SCK 12
#define SD_CS 10
void setup() {
// put your setup code here, to run once:
Serial.begin( 9600 );
pinMode(SD_CS, OUTPUT); digitalWrite(SD_CS, HIGH);
SPI.begin(SD_SCK, SD_MISO, SD_MOSI);
SPI.setFrequency(1000000);
SD.begin(SD_CS);
audio.setPinout(I2S_BCLK, I2S_LRC, I2S_DOUT);
audio.setVolume(21); // 0...21
audio.connecttoFS(SD, "/123.mp3");
}
void loop() {
// put your main code here, to run repeatedly:
audio.loop();
//audio.stopSong();//CLOSE
}
Example3: Initialize SD Card slot¶
Please install the SD library and FS library. Upload the following code to the board.
#include <Wire.h>
#include <SPI.h>
#include <SD.h>
#include <FS.h>
//4.3
#define SD_MOSI 11
#define SD_MISO 13
#define SD_SCK 12
#define SD_CS 10
void setup() {
// put your setup code here, to run once:
Serial.begin( 9600 );
//SD卡
SPI.begin(SD_SCK, SD_MISO, SD_MOSI);
delay(100);
if (SD_init() == 1)
{
Serial.println("Card Mount Failed");
}
else
Serial.println("Initialize SD Card successfully");
}
void loop() {
// put your main code here, to run repeatedly:
}
//SD Card initialization
int SD_init()
{
if (!SD.begin(SD_CS))
{
Serial.println("Card Mount Failed");
return 1;
}
uint8_t cardType = SD.cardType();
if (cardType == CARD_NONE)
{
Serial.println("No TF card attached");
return 1;
}
uint64_t cardSize = SD.cardSize() / (1024 * 1024);
Serial.printf("TF Card Size: %lluMB\n", cardSize);
listDir(SD, "/", 2);
// listDir(SD, "/", 0);
// createDir(SD, "/mydir");
// listDir(SD, "/", 0);
// removeDir(SD, "/mydir");
// listDir(SD, "/", 2);
// writeFile(SD, "/hello.txt", "Hello ");
// appendFile(SD, "/hello.txt", "World!\n");
// readFile(SD, "/hello.txt");
// Serial.printf("Total space: %lluMB\n", SD.totalBytes() / (1024 * 1024));
// Serial.printf("Used space: %lluMB\n", SD.usedBytes() / (1024 * 1024));
// Serial.println("SD init over.");
return 0;
}
//Traverse the SD card directory
void listDir(fs::FS & fs, const char *dirname, uint8_t levels)
{
// Serial.printf("Listing directory: %s\n", dirname);
File root = fs.open(dirname);
if (!root)
{
//Serial.println("Failed to open directory");
return;
}
if (!root.isDirectory())
{
Serial.println("Not a directory");
return;
}
File file = root.openNextFile();
// i = 0;
while (file)
{
if (file.isDirectory())
{
// Serial.print(" DIR : ");
// Serial.println(file.name());
if (levels)
{
listDir(fs, file.name(), levels - 1);
}
}
else
{
Serial.print("FILE: ");
Serial.print(file.name());
// lcd.setCursor(0, 2 * i);
// lcd.printf("FILE:%s", file.name());
Serial.print("SIZE: ");
Serial.println(file.size());
// lcd.setCursor(180, 2 * i);
// lcd.printf("SIZE:%d", file.size());
// i += 16;
}
file = root.openNextFile();
}
}
Example4: Initialize the touch¶
Please check if you have install the library XPT2046_Touchscreen(Copy the XPT2046_Touchscreen folder in the libraries downloaded above to the .../arduino/libraries directory).
And put the touch.h file in the same directory as the program, as shown below
Upload the following code to the board, and open the serial monitor to check the touch information.
#include "touch.h"
void setup() {
// put your setup code here, to run once:
Serial.begin( 9600 ); /*初始化串口*/
touch_init();
}
void loop() {
// put your main code here, to run repeatedly:
if (touch_has_signal())
{
if (touch_touched())
{
Serial.print( "Data x :" );
Serial.println( touch_last_x );
Serial.print( "Data y :" );
Serial.println( touch_last_y );
}
}
}
Example5: Initialize the interact communication of Bluetooth¶
Please check if you have installed the library BLE(Copy the BLE folder in the libraries downloaded above to the .../arduino/libraries directory). Upload the following code to the board, and use the phone to search the Bluetooth device.
#include "BLEDevice.h" //BLE drive library
#include "BLEServer.h" //BLE server library
#include "BLEUtils.h" //BLE Utility library
#include "BLE2902.h" //Feature Add Descriptor Library
#include <BLECharacteristic.h> //BLE Feature function library
BLEAdvertising* pAdvertising = NULL;
BLEServer* pServer = NULL;
BLEService *pService = NULL;
BLECharacteristic* pCharacteristic = NULL;
#define bleServerName "ESP32SPI-BLE" //BLE Server name
#define SERVICE_UUID "6479571c-2e6d-4b34-abe9-c35116712345" //Server UUID
#define CHARACTERISTIC_UUID "826f072d-f87c-4ae6-a416-6ffdcaa02d73"
bool connected_state = false; //Create device connection identifier
class MyServerCallbacks: public BLEServerCallbacks //Create connection and disconnection calling classes
{
void onConnect(BLEServer *pServer)//Start connecting function
{
connected_state = true; //Device is connected correctly
}
void onDisconnect(BLEServer *pServer)//disconnect function
{
connected_state = false; //Device connection error
}
};
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
//BLE
BLEDevice::init(bleServerName); //Create BLE and set name
pServer = BLEDevice::createServer(); //Create BLE server
pServer->setCallbacks(new MyServerCallbacks()); //Set up connection and disconnection calling classes
pService = pServer->createService(SERVICE_UUID); //Create BLE service
pCharacteristic = pService->createCharacteristic( //Create ble feature(Characterristic_UUID)
CHARACTERISTIC_UUID,
BLECharacteristic::PROPERTY_READ | BLECharacteristic::PROPERTY_WRITE | BLECharacteristic::PROPERTY_NOTIFY);
pCharacteristic->setValue("ELECROW");
//start broadcast
pAdvertising = BLEDevice::getAdvertising(); //A bleadvertising class pointer padvertising is defined, which points to bledevice:: getadvertising()
pAdvertising->addServiceUUID(SERVICE_UUID);
pAdvertising->start(); //Start broadcasting
pService->start();
// pAdvertising->stop(); //Stop broadcasting
// pService->stop(); //Stop service
}
void loop() {
}
Example6: Initialize the WIFI¶
Please check if you have installed the library WiFi(Copy the WiFi folder in the libraries downloaded above to the .../arduino/libraries directory). Upload the following code to the board.
#include <WiFi.h>
const char *ssid = "elecrow888"; //Change to your SSID
const char *password = "elecrow2014"; //Change to your password
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
WiFi.begin(ssid, password);
WiFi.setAutoReconnect(true);//Set up automatic reconnection
while (WiFi.status() != WL_CONNECTED) {
delay(100);
Serial.println("connecting");
}
Serial.println("WiFi is connected.");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
// WiFi.disconnect();
}
void loop() {
}
Example7: Connect Crowtail-GPS module via UART to Get Location¶
Connect the Crowtail-GPS to the UART port.
Upload the following code to the board.
#define SERIAL_BAUD 9600
HardwareSerial cardSerial(1);//Declare the UART1
unsigned char buffer[256]; // buffer array for data recieve over serial port
int count_1 = 0; // counter for buffer array
void clearBufferArray() // function to clear buffer array
{
for (int i = 0; i < count_1; i++)
{
buffer[i] = 0; // clear all index of array with command NULL
}
}
void setup() {
// put your setup code here, to run once:
Serial.begin( 9600 ); /*Initialize the serial port*/
cardSerial.begin(SERIAL_BAUD, SERIAL_8N1, 18, 17);//4.3 /*初Initialize the serial port1*/18-RX\17-TX
}
void loop() {
// put your main code here, to run repeatedly:
if (cardSerial.available()) // if date is comming from softwareserial port ==> data is comming from SoftSerial shield
{
while (cardSerial.available()) // reading data into char array
{
buffer[count_1++] = cardSerial.read(); // writing data into array
if (count_1 == 256)break;
}
Serial.write(buffer, count_1); // if no data transmission ends, write buffer to hardware serial port
clearBufferArray(); // call clearBufferArray function to clear the storaged data from the array
count_1 = 0; // set counter of while loop to zero
}
if (Serial.available()) // if data is available on hardwareserial port ==> data is comming from PC or notebook
cardSerial.write(Serial.read()); // write it to the SoftSerial shield
}