Arduino H7 Video with LVGL memory/buffer related issues

[JwX000]

Hello!
It seems that there are some issues with memory/buffer management in Arduino H7 Video when using LVGL:

• Arduino_H7_Video::begin() allocates buf1 of size (X * Y * 1B) of lv_color_t (3B/px), where LVLG examples suggest buffer of BYTES_PER_PIXEL elements, which would lead to memory usage reduction
  'static lv_color_t * buf1 = (lv_color_t*)malloc((width() * height() / 10));'
• but to lv_display_set_buffers() it is passed only in fragment, because size is passed as X*Y/10 not sizeof(buf1), which creates buffer of size smaller that 1/10 of display resolution (which is recommended minimal size)
  lv_display_set_buffers(display, buf1, NULL, width() * height() / 10, LV_DISPLAY_RENDER_MODE_PARTIAL);
• trying to pass correct bigger size causes a fail on Arduino Giga, probably during screen rotation (disabling rotation allows for passing bigger /correct buffer size without fail),
• rotation inside lvgl_displayFlushing allocates buffer of X * Y * 4 inside internal RAM, which cannot success for Giga (trying to allocate 1.5MB when internal RAM is 1MB), additionally x4 factor is probably not needed - BYTES_PER_PIXEL would suffice
  rotated_buf = (uint8_t*)realloc(rotated_buf, w * h * 4);

[JwX000]

a correction - two first points can be reduced to: buffer smaller than recommended 1/10 of screen size using current code. There is cast to lv_color_t* but allocated is only X * Y / 10, not X * Y * sizeof(lv_color_t)/10
but other points seem still valid - buffer smaller than recommended, cannot be increased without fail, rotate trying to allocate more RAM than available.
And a question - as rotation buffer even after correction consumes nearly all internal RAM, can it be reallocated to SDRAM? which links to #1105 SDRAM should be protected from reinitializiation, leading potentially to memory overwriting

[JwX000]

ad.4 maybe X and Y in lvgl_displayFlushing are not full screen resolution because it is hard to understand how it works now if buffer size needed cannot fit RAM.... Maybe it is correlated with buf1 size if it causes malfunction when buf1 size is increased?

[leonardocavagnis]

Hi @JwX000 , thanks a lot for the detailed analysis!

You’re right, there’s currently some confusion in how buf1 is declared. LVGL expects the draw buffer size to be expressed using the following pattern: HOR_RES * VER_RES / 10 * BYTES_PER_PIXEL as documented here.

In our case BYTES_PER_PIXEL is 2, since we use 16-bit color depth (as defined in lv_conf_9.h):

ArduinoCore-mbed/libraries/Arduino_H7_Video/src/lv_conf_9.h

Line 25 in f4e9813

#define LV_COLOR_DEPTH 16

So yes, the current allocations:

ArduinoCore-mbed/libraries/Arduino_H7_Video/src/Arduino_H7_Video.cpp

Line 110 in f4e9813

static lv_color_t * buf1 = (lv_color_t*)malloc((width() * height() / 10)); /* Declare a buffer for 1/10 screen size */

ArduinoCore-mbed/libraries/Arduino_H7_Video/src/Arduino_H7_Video.cpp

Line 123 in f4e9813

lv_display_set_buffers(display, buf1, NULL, width() * height() / 10, LV_DISPLAY_RENDER_MODE_PARTIAL); /*Initialize the display buffer.*/

ArduinoCore-mbed/libraries/Arduino_H7_Video/src/Arduino_H7_Video.cpp

Line 137 in f4e9813

lv_disp_draw_buf_init(&draw_buf, buf1, NULL, width() * height() / 10); /* Initialize the display buffer. */

should be fixed to better match LVGL’s guidelines.

Quick question:
Have you tried correcting only the buffer size (multiplying by BYTES_PER_PIXEL) with rotation disabled?
If so, does the corrected buffer fit and work properly on GIGA?

Regarding rotation: this part is a bit more tricky. LVGL’s documentation on handling display rotation with custom drivers isn’t very clear, so the current implementation contains some workarounds that definitely need cleanup.
There’s a note about this directly in the code here:

ArduinoCore-mbed/libraries/Arduino_H7_Video/src/Arduino_H7_Video.cpp

Line 250 in f4e9813

// TODO: find a smart way to tackle sw rotation

[JwX000]

yes - I have created derived class of Arduino_H7_Video in the sketch (when I didn't know how to use cloned/local version of ArduinoCore-mbed with Arduino IDE) and did some experiments (that have turned in the end as mostly not needed, as my original problem turned out to be inability to successfully overwrite config file embedded with Arduino_H7_Video and thus increase LV_MEM_SIZE)
and even something like below works (alignment correction is another step of debugging - code works without it):

/* disable rotation */ 
if (this->isRotated()) 
{

    display = lv_display_create(height(), width());
    //lv_display_set_rotation(display, LV_DISPLAY_ROTATION_270);

    //[...]
  }

//[...]

//declare big buffer for LVGL (25% of screen size)
uint32_t buf_size = 1 * this->height() * this->width() /4 * lv_color_format_get_size(lv_display_get_color_format(display));

//[...]
static  LV_ATTRIBUTE_MEM_ALIGN  uint8_t* buf1 = (uint8_t *)ea_malloc(buf_size + LV_DRAW_BUF_ALIGN);

//[...]
lv_display_set_buffers(display, lv_draw_buf_align(buf1, LV_COLOR_FORMAT_RGB565), NULL, buf_size , LV_DISPLAY_RENDER_MODE_PARTIAL);

[leonardocavagnis]

To make the Arduino IDE use a locally modified ArduinoCore-mbed follow these instructions:

1. Open the Arduino folder in your Documents (the one that already contains the libraries directory).
2. Create this path inside it:

Arduino/
 └── hardware/
     └── arduino-git/
         └── ArduinoCore-mbed/

3. Clone or copy the ArduinoCore-mbed repository into that ArduinoCore-mbed folder.
4. Restart the Arduino IDE.

The IDE will now automatically detect this local core, so you can freely modify core files and test changes. You can then select it from: Tools → Board. The core will be listed with (arduino-git) in parentheses.

[JwX000]

yes - I have finally made it work, so I was able to use modified config (trying to include my local config from sketch was of limited/no effect), enabled logging, identified my issue as lack of memory, added some using LV_MEM_SIZE and resolved a case of LVGL demo clock hanging after short time when in standard dimensions of 150x150 (smaller clock [or clock with shorter arms] was working ok - probably it needed smaller work buffer), but it took me long time (and lots of digging in the codebase) to realize that my config is ignored by the library code despite defines being effective in the sketch...
So this ticket is a byproduct of searching why this demo is not working while LVGL team insisted it works ok, which means it seems to be of lesser importance than documenting the use of lv_config.h and means of modifying it when using Arduino IDE - core problem was in another area...

[leonardocavagnis]

Could you share a minimal sketch that reproduces the issue? That would help us validate the fix.
And if you’d like, feel free to open a PR on ArduinoCore-mbed to fix the buffer allocation, we’re happy to review it.
Thanks again!

[JwX000]

I will try to prepare something