Can MCU run realtime JPEG encode & UVC?

Realtime JPEG encoding on MCU – Test Overview

The STM32H7 series microcontrollers, specifically the STM32H753 and STM32H757, show promising capabilities for JPEG encoding according to their datasheets. However, achieving real-time JPEG encoding, especially for streaming applications such as USB Video Class (UVC), presents significant challenges. This article shares insights and test results from TrigoPi’s labs, where we explored the JPEG encoding pipeline on the STM32H757 and experimented with different feeding and encoding techniques to enable real-time performance on a low-cost MCU platform.

JPEG encoding pipeline diagram on STM32H757 MCU
Figure 1: JPEG encoding pipeline on STM32H757 MCU showing the data flow and processing stages.

Method & Procedure

Our investigation involved analyzing the JPEG pipeline architecture of the STM32H757, focusing on how frames are fed into the encoder. Unlike raster scanning, the encoder requires input in Minimum Coded Units (MCUs), which complicates frame feeding and timing (see Figure 2). We tested two main approaches for data feeding and encoding:

  • Polling-based frame feeding and encoding, where the CPU actively manages data transfer and encoding in chunks.
  • DMA-based transfer between SDRAM and AXI-RAM, offloading memory movement from the CPU to improve encoding throughput.
Diagram showing MCU and frame feeding in JPEG encoding pipeline
Figure 2: MCU and frame feeding in Minimum Coded Unit (MCU) format for JPEG encoding.

We ran encoding tests at 1080p and 720p resolutions with different chroma subsampling formats (4:2:2 and 4:2:0) and quality settings (Q20 and Q80). The tests measured output file sizes, compression ratios, and encoding times for both polling and DMA transfer methods.

Results & Analysis

Polling-based encoding results showed that at 1080p resolution with 4:2:2 chroma subsampling and Q80 quality, encoding took approximately 378 ms per frame, producing a 546 KB output with a compression ratio of 7.59 (see Figure 3). Lowering quality to Q20 improved compression ratio to 24.42 but only slightly reduced encoding time to 360 ms. Using 4:2:0 chroma subsampling reduced encoding time to 288 ms at Q80 and 274 ms at Q20 with output sizes around 510 KB and 159 KB respectively.

Encoding performance results for 1080p resolution using polling
Figure 3: Encoding results for 1080p resolution using polling method.

At 720p resolution, polling encoding times dropped to around 166 ms (Q80) and 156 ms (Q20) with 4:2:2 subsampling, and output sizes were correspondingly smaller (see Figure 4). However, these times are still not suitable for real-time video streaming at typical frame rates.

Encoding performance results for 720p resolution using polling
Figure 4: Encoding results for 720p resolution using polling method.

Switching to DMA transfers between SDRAM and AXI-RAM dramatically improved encoding speeds. At 720p with 4:2:2 subsampling and Q80 quality, encoding time dropped to 22 ms per frame, enabling 45 frames per second (fps) streaming. At 1080p with 4:2:0 subsampling, encoding took 50 ms per frame, roughly 20 fps (see Figure 5). This proves that with optimized DMA feeding, real-time JPEG encoding on STM32H7 MCUs is achievable.

Encoding performance results for 1080p and 720p using DMA
Figure 5: Encoding results using DMA transfers for 1080p and 720p resolutions.

Implications

Real-time JPEG encoding on STM32H7 MCUs is not straightforward with the default ST Micro implementation due to the MCU’s JPEG encoder input format and CPU load. However, by leveraging DMA and carefully managing frame feeding in MCUs, real-time encoding sufficient for UVC streaming is possible. This opens the door for low-cost embedded video streaming solutions where dedicated hardware codecs may be too expensive or power-hungry.

UVC streaming itself poses additional software challenges. The standard ST libraries require extensive modifications and tuning to handle compressed video streaming reliably and with low latency. Our team developed custom library adaptations and code generation tricks to achieve stable UVC streaming performance.

Screenshot showing UVC streaming monitoring
Figure 6: UVC streaming monitoring interface demonstrating real-time video output.

Key Findings

  • Polling-based encoding at 1080p takes around 360–378 ms per frame, too slow for real-time streaming.
  • Polling encoding at 720p reduces time to approximately 156–166 ms per frame, still insufficient for smooth video.
  • DMA-based encoding drastically improves speed: 22 ms/frame (45 fps) at 720p and 50 ms/frame (20 fps) at 1080p.
  • Chroma subsampling 4:2:0 is more efficient than 4:2:2 for encoding speed and compression ratio.
  • Real-time encoding “as-is” from ST Micro’s standard implementation is not feasible; custom feeding and DMA tricks are required.
  • UVC streaming with compressed JPEG video requires extensive library tuning and code customization.

Do’s & Don’ts

  • Do implement DMA transfers to offload CPU and speed up memory movement for JPEG encoding.
  • Do feed the JPEG encoder in MCU-sized chunks rather than full raster frames to match hardware expectations.
  • Do tune chroma subsampling and quality settings to balance compression ratio, image quality, and encoding speed.
  • Don’t rely on standard ST Micro JPEG encoding libraries without modification for real-time streaming.
  • Don’t underestimate the complexity of UVC streaming software; expect to invest time in library and driver customization.

Safety & Reliability Notes

When implementing real-time JPEG encoding and UVC streaming on STM32H7 MCUs, ensure careful memory management and synchronization between DMA operations and CPU tasks to avoid data corruption. Proper error handling and buffer management are critical to maintain stable streaming without frame drops or artifacts. Additionally, thermal considerations should be accounted for when running the MCU at high processing loads continuously.

At TrigoPi, we love testing the edges of what low-cost MCUs can do in demanding multimedia applications. It’s the kind of challenge that keeps our engineers awake at night—in a good way.

If you’re exploring embedded video streaming solutions and want to learn more about how to unlock real-time JPEG encoding on STM32H7 MCUs, we’d be happy to share our insights and approaches.

Download PDF file