Handling RAW Color Formats

Non-Image Processing

• V4L2 API standard supports both packed( RAW 6,7,9,10,14,20) and unpacked ( RAW8 and16) RAW Bayer formats, but most modern drivers use packed formats for efficiency (especially for >8-bit data). The exact behavior depends on the sensor and driver implementation. The Renesas MIPI and CRU assumes the camera sensor RAW data is packed ( see table below). 

  Format	MIPI CSI -2 bus Packing Standardized?	Standard MIPI v1.2 Format

  Format	MIPI CSI -2 bus Packing Standardized?	Standard MIPI v1.2 Format
  RAW6	✅ Yes (4px → 3B)	4 pixels in 3 bytes
  RAW7	✅ Yes (8px → 7B	8 pixels in 7 bytes
  RAW8	❌ N/A (unpacked)	unpacked 1 pixel in 1 byte
  RAW10	✅ Yes (4px → 5B)	4 pixels in 5 bytes
  RAW12	✅ Yes (2px → 3B)	packed 2px → 3B
  RAW14	✅ Yes (4px → 8B)	4 pixels in 8 bytes
  RAW16	❌ N/A (unpacked)	unpacked 1 pixel in 2 bytes
  RAW20	✅ Yes (2px → 5B)	2pixel in 5 bytes

  If the RZ/V2H CRU is setup to output RAW ( not RGB ) the image from the MIPI interface is output as 256bit block ( 4 x 64bit blocks). The table below shows the memory format.
  Padding differs depending on which RAW is used. 

• Open

  

   

  This layout is not the same as the standard V4L2 layout. It is designed to be compatible with the Renesas ISU ( Image Scaling Unit ) and ISP. The ISU features include scaling and format conversion. 
  The ISU can be used with the gstreamer by using the Renesas Gstreamer plugin vspmfilter.  

Image Processing

• RAW is passed through De-mosaic and output RGB888.

Bypass

• MIPI Camera dtata bypasses the CRU 

• MIPI-CSI Format is maintained

• Not supported on Standard RZ/V2H and RZ/V2N AI SDK. ( Contact FAE for support )

Terminology

• Bayer Format

  •  Most digital cameras use a Bayer filter—a color filter array (CFA) over the sensor that captures only one color per pixel (Red, Green, or Blue). The missing colors are interpolated later (demosaicing) to produce a full-color image.

  • V4L2 Bayer Formats

    [8]: 'RGGB' (8-bit Bayer RGRG/GBGB) [9]: 'BA81' (8-bit Bayer BGBG/GRGR) [10]: 'GRBG' (8-bit Bayer GRGR/BGBG) [11]: 'GBRG' (8-bit Bayer GBGB/RGRG) [12]: 'RG10' (10-bit Bayer RGRG/GBGB) [13]: 'BG10' (10-bit Bayer BGBG/GRGR) [14]: 'BA10' (10-bit Bayer GRGR/BGBG) [15]: 'GB10' (10-bit Bayer GBGB/RGRG) [16]: 'RG12' (12-bit Bayer RGRG/GBGB) [17]: 'BG12' (12-bit Bayer BGBG/GRGR) [18]: 'BA12' (12-bit Bayer GRGR/BGBG) [19]: 'GB12' (12-bit Bayer GBGB/RGRG) [20]: 'pREE' (14-bit Bayer RGRG/GBGB Packed) [21]: 'pBEE' (14-bit Bayer BGBG/GRGR Packed) [22]: 'pgEE' (14-bit Bayer GRGR/BGBG Packed) [23]: 'pGEE' (14-bit Bayer GBGB/RGRG Packed) [24]: 'RG16' (16-bit Bayer RGRG/GBGB) [25]: 'BYR2' (16-bit Bayer BGBG/GRGR) [26]: 'GR16' (16-bit Bayer GRGR/BGBG) [27]: 'GB16' (16-bit Bayer GBGB/RGRG)

  • 8-bit Bayer Formats (Uncompressed)

    • RGGB (8-bit): Red-Green top, Green-Blue bottom.

    • BA81 (8-bit): Alias for BGGR (Blue-Green top, Green-Red bottom).

    • GRBG (8-bit): Green-Red top, Blue-Green bottom.

    • GBRG (8-bit): Green-Blue top, Red-Green bottom.

    • 8-bit MIPI-CSI2 to V4L2 Codes:

      MIPI-CS2 Value V4L2 Equivalent (v4l2_fourcc) Description 'RGGB' (8) V4L2_PIX_FMT_SRGGB8 ('RGGB') 8-bit Bayer RGRG/GBGB 'BA81' (9) V4L2_PIX_FMT_SBGGR8 ('BA81') 8-bit Bayer BGBG/GRGR 'GRBG' (10) V4L2_PIX_FMT_SGRBG8 ('GRBG') 8-bit Bayer GRGR/BGBG 'GBRG' (11) V4L2_PIX_FMT_SGBRG8 ('GBRG') 8-bit Bayer GBGB/RGRG

  • 10-bit Bayer Formats (Uncompressed)

    • RG10, BG10, BA10, GB10: Same patterns as above but with 10 bits per pixel.

    • 10-bit MIPI-CSI2 to V4L2 Codes:

      MIPI-CS2 Value V4L2 Equivalent (v4l2_fourcc) Description 'RG10' (12) V4L2_PIX_FMT_SRGGB10 ('RG10') 10-bit Bayer RGRG/GBGB 'BG10' (13) V4L2_PIX_FMT_SBGGR10 ('BG10') 10-bit Bayer BGBG/GRGR 'BA10' (14) V4L2_PIX_FMT_SGRBG10 ('GR10') 10-bit Bayer GRGR/BGBG 'GB10' (15) V4L2_PIX_FMT_SGBRG10 ('GB10') 10-bit Bayer GBGB/RGRG

  • 12-bit Bayer Formats (Uncompressed)

    • RG12, BG12, BA12, GB12: Higher precision (12 bits per pixel).

    • 12 bit MIPI-CSI2 to V4L2 Codes:

      MIPI-CS2 Value V4L2 Equivalent (v4l2_fourcc) Description 'RG12' (16) V4L2_PIX_FMT_SRGGB12 ('RG12') 12-bit Bayer RGRG/GBGB 'BG12' (17) V4L2_PIX_FMT_SBGGR12 ('BG12') 12-bit Bayer BGBG/GRGR 'BA12' (18) V4L2_PIX_FMT_SGRBG12 ('GR12') 12-bit Bayer GRGR/BGBG 'GB12' (19) V4L2_PIX_FMT_SGBRG12 ('GB12') 12-bit Bayer GBGB/RGRG

  • 14-bit Packed Bayer Formats

    • pREE, pBEE, pgEE, pGEE: 14-bit Bayer data packed into bytes (space-efficient storage).

    • pREE = Packed Red-Green (RGGB) 14-bit.

    • pBEE = Packed Blue-Green (BGGR) 14-bit.

    • pgEE = Packed Green-Red (GRBG) 14-bit.

    • pGEE = Packed Green-Blue (GBRG) 14-bit.

    • 14-bit MIPI-CSI2 to V4L2 Codes:

      MIPI-CS2 Value V4L2 Equivalent (v4l2_fourcc) Description 'pREE' (20) V4L2_PIX_FMT_SRGGB14P ('pRAA' or custom) 14-bit Packed Bayer RGRG/GBGB 'pBEE' (21) V4L2_PIX_FMT_SBGGR14P ('pBAA' or custom) 14-bit Packed Bayer BGBG/GRGR 'pgEE' (22) V4L2_PIX_FMT_SGRBG14P ('pgAA' or custom) 14-bit Packed Bayer GRGR/BGBG 'pGEE' (23) V4L2_PIX_FMT_SGBRG14P ('pGAA' or custom) 14-bit Packed Bayer GBGB/RGRG

  • 16-bit Bayer Formats (Uncompressed)

    • RG16, BYR2, GR16, GB16: Full 16-bit depth per pixel.

    • BYR2 is an alias for BGGR (16-bit).

    • 16-bit MIPI-CSI2 to V4L2 Codes:

      MIPI-CS2 Value V4L2 Equivalent (v4l2_fourcc) Description 'RG16' (24) V4L2_PIX_FMT_SRGGB16 ('RG16') 16-bit Bayer RGRG/GBGB 'BYR2' (25) V4L2_PIX_FMT_SBGGR16 ('BG16') 16-bit Bayer BGBG/GRGR 'GR16' (26) V4L2_PIX_FMT_SGRBG16 ('GR16') 16-bit Bayer GRGR/BGBG 'GB16' (27) V4L2_PIX_FMT_SGBRG16 ('GB16') 16-bit Bayer GBGB/RGRG