Low Power Linux
This page discusses low power options for Linux based RZ devices.
Table of Contents
Supported Devices
Currently this topic only applies to the following devices because they are the only ones with the hardware support.
RZ/G3S
RZ/G3E
Hardware Architecture
Separated Power Domains
In order to achieve lower power numbers, the power rails inside the device need to be completely isolated so that they can be controlled independently.
The control of these internal power rails are not done by software, but rather by hardware. As in, to remove power from an internal power rail, you must completely remove power from the VDD supply pin to the device. This means that the external power supply circuitry, such as a PMIC, must be considered in the board design.
Removing power to a domain by means of the VDD pin helps to prevent "leakage currents" if only gate circuits were used.
The individual power domains are labeled as follows:
AWO domain: This stands for Always On because the intention is that this domain stays powered even in low power mode.
ISO domain: This stands for Isolated. The term refers to the internal circuits were carefully isolated from the rest of the device so they could be powered separately, and thus disconnect if needed. This design change is what allows for low power operations in the newer generation of RZ devices.
DDRIO domain : This refers to the DDR Controller I/O circuitry. It is on a separate domain so that it can be controlled independently.
VBAT domain: This stands for Voltage from Battery. This refers to a voltage rail that is intended to be connected to an external battery supply. This domain only contains the Real Time Clock (RTC) peripheral and is intended that in the lowest power mode, this would be the only circuity powered.
DDR Self Refresh
For normal DDR operations, a 'refresh cycle' is required by the DDR controller in order to ensure data integrity by periodically restoring the charge in memory cells. This process prevents data loss, as the charge on DRAM cells gradually weakens over time. However, in order to achieve a lower power number, the DDR controller circuit in the SoC must be compete disabled, and thus not be able to perform this refresh cycle.
When selecting DDR memory device for your design, you can choose a DDR that has the ability to perform SELF REFRESH cycles on its own. This is done by sending a SELF REFRESH command over the DDR bus to the device. However, while operating in self refresh mode, an external clock input is still needed and all the input pins to the device need to be activated. And of course, you need to continue to power the DDR memory device as well.
This is why there is a separate power domain ("DDRIO") for the DDR I/O circuity. The DDR controller itself can be powered down saving a lot of power, and only minimal DDR circuitry required for self refresh mode can used.
To shift DRAM to the self-refresh mode, the DRAM controller should issue a self-refresh command to the target
DRAM. Driving the CKE signal high or driving the RESET# signal low makes DRAM exit the self-refresh mode and
cancels the DRAM backup mode.
Power Modes
By powering some domains and not others, you can create "power modes" of operations.
ALL_ON : The device is fully powered and operational
AWO : “Always On” The main CA55 cores and other major peripherals are power down. However, the CM33 sub core remains powered (Always On) and operational.
VBAT : “Voltage from Battery” All cores and peripherals are powered down and only minimal circuitry is power by an external battery. This would be the case for a RTC that maintains time that can be used for wake the system up at periodic internals.
ALL_OFF : The entire system is powered down. Power = 0W.
Power Mode Transitions
The power mode transitions are accomplished by combining the following controls:
Internal control by register control inside this LSI. This such as enable/disable of CPU cores and clocks are controlled by software.
Power supply control for each power domain. For example, going into suspend means manually controlling a PMIC to change power rails from 1.8V to 0v. Additionally, moving back into ALL_ON state is manually controlling a PMIC to change power rails from 0V to 1.8V
Control flow branches considering the state before state transition and the state after transition. This means source code execution will vary depending on the previous state of the system.
As for 2 and 3, it depends on the system configuration including this LSI and power control system, so it is necessary to implement it according to the user’s system.
Renesas provides sample software to transition from ‘ALL_ON to VBATT’ and from ‘ALL_ON to AWO’ using the RZ/G3S SMARC Evaluation Kit.
Hardware Design Considerations
Below are some considerations when designing your system.
Renesas RZ/G3S Evaluation Board as Reference
Renesas provides the schematic and design details for the RZ/G3S evaluation board. Go to https://www.renesas.com/en/products/rz-g3s . Under the section “Software Downloads”, look for “RZ/G3S SMARC Module Board Design Data V3.00”
Included in the package is the schematic (pdf and Orcad), BOM, gerber data, GreenPAK device Data Sheets and GreenPAK Designer project files (.gp3, .gp5).
Please use this package as reference for your own design. Note that using GreenPAK is not mandatory, it was just a design choice by Renesas.
VBATT_VDD and VBATTRESETN
When transition from ALL_ON to VBATT+DDR Retention or AWO+DDR Retention, VBATT_VDD must be supplied even if you do not plan to connect a Real-Time Clock to the system. Additionally, the VBATTERESETN signal must remain high while the system is in low power state.
Software Architecture
Power State Coordination Interface (PSCI)
In summary, the kernel does not want to deal with the low level power operations (like it did with 32-bit ARM Linux) because they are different for each SoC and board, and that is too much to manage in the kernel code. So now (with 64-bit ARM Linux) an API was created for the kernel to call.
The Power State Coordination Interface (PSCI) is an ARM standard that describes a software interface for power management between an operating system (not just Linux) and it’s supervisory firmware. The interface allows an operating system to perform power management tasks such as putting CPUs into an idle (low-power) state, bringing CPUs online/offline and turning the system off.
The kernel prepares a message to send to the lower level (TrustedFirmware-A) by issuing a SMC (Secure Monitor Call) ARM instruction which will make the CPU enter EL3 (Exception Level 3) and then the CPU will execute the Secure Monitor code (TrustedFirmware-A BL31) that was loaded early in boot before the kernel started. To enter suspend, the PSCI API name is PSCI_SYSTEM_SUSPEND and is implemented in function psci_system_suspend() in the kernel file drivers/firmware/psci/psci.c. If there are no errors, the kernel will never return from that function (as in, CPU operation will be halted due to system suspend). For the system resume process, that is basically a warm boot, so the kernel starts executing from the beginning and detects that it needs to resume from a previous suspended state.
More details can be found here: https://www.thegoodpenguin.co.uk/blog/an-overview-of-psci/
The code that handles the PSCI Service calls is located in directory rzg_trusted-firmware-a/lib/psci
The code that handles the system suspend for RZ/G3S is in rzg_trusted-firmware-a/plat/renesas/rz/soc/g3s/plat_pm.c
Overview of Suspend-to-RAM (S2R)
Linux Suspend-to-RAM, also known as “S2R” or "sleep", is a low-power state in Linux where the system's memory (RAM) is powered on to retain its data, while other components are powered down to save energy. This allows for a quick wake-up time, making it ideal for short periods of inactivity.
As part of the system design, the application layer will initiate this transition into sleep mode which will then propagate through different layers of the system.
Here is an explanation of the flow of the process.
Refer to the diagram on the right for the numbers 1-8
1️⃣ Application Starts Suspend: The application layer requests to enter System Suspend-to-RAM via the system filesystem"("sysfs") by choosing one of the following runtime commands.
$ echo mem > /sys/power/state
or
$ echo deep > /sys/power/mem_sleep
$ echo mem > /sys/power/state
Because the application is the one initialing the S2R, the application should prepare the system first, especially for external components attached via I2C, USB, SPI, or remote client (TCP/IP). While the Linux device drivers resume operation, depending on the external component, they might also need to know operations will be stopped. This is part of the system design, not the Linux BSP operation.
💡 “Sysfs” is a virtual filesystem in Linux that provides a hierarchical view of devices, drivers, and other kernel objects to user space. It's a way for the kernel to expose information and control mechanisms to user applications. Mounted at /sys, it allows users to inspect and configure system components.
2️⃣ Kernel Support for Suspend: Sysfs sends commands to System Suspend Framework. This is the code within the Linux kernel that supports this feature. This is the same functionary for both an embedded system, or a full x86 PC.
The process consists of thing such as saving the states of individual drivers or sub-systems and informing individual drivers that this will be occurring. A driver or peripheral might need to know ahead of time in order to prepare the hardware safely.
Some operations that are done are: Invoking Suspend Notifiers, Freezing User-space Tasks, Suspend Devices, Suspending Timekeeping, CPU and Core Logic Actions
All user-space tasks and relevant kernel threads are frozen to prevent hardware or memory access during the transition.
All interrupts are masked except for those specifically configured as wake sources (e.g., RTC, power button, USB events)
While the peripherals are stopped by means of stopping the individual peripheral clocks, the peripheral IP still remains powered in some power down modes. This is because the CM33 might still need access and operation of some peripherals. For example, Linux might no longer need to use the serial console (SCIF-0), but while in S2R, the CM33 might continue to run and be using SCIF-1 to communicate with some sensor. This means it is not practical to power down the enter IP block, but rather just stop the clock source to individual channels. This is why all peripherals are on the AWO domain.
💡Phrases to enter into online AI tool such as Perplexity to get more detailed information: "In the Linux kernel, what happens to transition into sleep mode" or “How does the Linux kernel transition into suspend-to-idle state at the hardware level”
3️⃣ Kernel Prepares for Suspend: System Suspend Framework notifies device driver (On Chip Drivers) of moving to System Suspend via suspend/resume API Linux Power Management Framework.
4️⃣ Peripherals Prepared for Suspend. Drivers backup/restore corresponding SoC IPs registers
5️⃣ Kernel Suspends and tells Lower Level to Power Off: System Suspend Framework sends secure calls to TF-A
6️⃣ DDR is prepared for Suspend: TF-A set DDR to self-refresh mode so that contents can be preserved after the RZ DDR controller stops.
7️⃣ Real-time Clock is prepared. TF-A enables VBAT clock and power using during sleep
8️⃣ Power is Disconnected: TF-A sends commands to PMIC to set sleep flags and then turn off power
Source Code Modifications
This section list portions of TF-A, u-boot and kernel that might need to be customized for your design.
Trusted Firmware-A
🔷 TF-A Software Review
Please review the information on the ARM Trusted Firmware page. Especially the section on Secure Monitor Services.
🔷 Compiler #defines
The following compiler defines are used in the source code
PLAT_SYSTEM_SUSPEND : (required) This need to be defined in order to use the suspend features.
PLAT_SYSTEM_SUSPEND_awo : (optional) This defines the type of suspend you want to use is AWO mode. This is used in the sample code for the evaluation boards that support both AWO and VBATT modes. Normally, you will only use one mode or the other (not both). So, you probably will not need to defined this in your product code. But, you will need to define this if you want to use the sample code on the Renesas evolution boards.
PLAT_SYSTEM_SUSPEND_vbat : (optional) This defines the type of suspend you want to use is VBATT mode. This is used in the sample code for the evaluation boards that support both AWO and VBATT modes. Normally, you will only use one mode or the other (not both). So, you probably will not need to defined this in your product code. But, you will need to define this if you want to use the sample code on the Renesas evolution boards
🔷 Custom Flag for Cold/Warm Boot
Cold Boot means the device is booting for the first time from Power Off. Warm Boot means the device has already booted once and went into suspend, and not it is booting again (resume)
During a Warm Boot (resume from suspend) the CPU will being to execute the same code that was during the Cold Boot. That is BL2. However, the code will need some way to know if it is a Cold Boot or a Warm Boot (Resume).
During warm boot (resume), the process of BL2 loading BL3 images is skipped, and the CPU jumps directly to running BL31 that was already loaded during the original cold boot. Please refer to file bl2_plat_setup.c to see this.
As the system designer, you are required to implement how you want to flag the BL2 software of a resume from suspend.
👉 You need to modify or create your own pwrc_board_is_resume() function.
This function is used 2 times in order to change the flow of the code.
1. This function is used early in boot in BL2 to understand if it needs to copy other images (BL3) into DDR, or if they are already there.
bl2_plat_get_boot_mode
↳ sys_is_resume_reboot()
↳ pwrc_board_is_resume()
2. This function is also used in the DDR setup. During a warm boot, DDR initialization is skipped.
plat_ddr_setup() ↳ sys_is_resume_reboot() ↳ pwrc_board_is_resume()
For the Renesas Evaluation boards, the example code uses the GreenPAK devices to save the warm boot/suspend flag. In the function pwrc_board_is_resume(), you can see that I2C commands are used in order to read a register in the GreenPAK to determine if this is a Cold or Warm Boot.
You may choose your own method of saving a flag to signal Warm Boot/Resume. For example, you could define a location in internal RAM that you set to a specific value before you suspend the system.
👉 The function pwrc_board_suspend_on() is called during suspend, so you will add your impregnation in that function.
Example implementations of pwrc_board_is_resume() and pwrc_board_suspend_on() can be found in:
(TF-A v2.10.5 or later)
plat/renesas/rz/board/g3s_smarc/pwrc_board.c
plat/renesas/rz/board/g3e_smarc/pwrc_board.c
(TF-A v2.7.0)
lat/renesas/rz/soc/g3s/drivers/pwrc/pwrc.c
RZ/G3S Reference Example using GreenPAK
The Renesas evaluation board contains a Renesas GreenPAK device. This is a programmable device that can automate operations.
For the Renesas example, since the GreenPAK chip on the board is always powered, an internal general purpose register inside the GreenPAK was used as the flag to signal a resume from suspend.
Since the PMIC power rail outputs (pins) must be manually controlled to achieve the different system power mode, I2C commands need to be sent to the PMIC. Given that in modes such as VBATT where all CPUs are powered down, ‘someone’ needs to send those I2C commands. Therefore, the GreenPAK was programmed such then when a physical button the boards was pressed, the necessary I2C command were send from the GreenPAK to the PMIC in order to bring all the power rails back up.
u-boot
No specific modifications are needed to support low power.
Linux Kernel
The CPG driver files might be needed if plan on using AWO mode in order to keep peripherals running that only the CM33 uses.
CM33 Sub-core
No specific modifications are needed to support low power.
Notes about Suspend-Resume Operations
Below is a collection of interesting facts about using Linux Suspend to RAM.
Must re-open /dev/ttySCx (SCIF) driver handles after resume
If you open a serial port (SCIF) and keep it open during suspend/resume, communications will not work until you close and re-open the handle to the driver after the system resumes. Therefore, when preparing the system for suspend, it is recommended to first close the current handles to any /dev/ttySC, then re-open after system resume.
USB Host issues a USB RESET during Suspend
When entering system suspend, by the fact that the USB IP block is being powered down, the USB D+ and D- lines will both go to low level creating a "single-ended zero" (SE0) state. Any time a SE0 state lasts longer than 10ms, it is considered a "USB Reset". This is the hardware behavior of the USB hardware and cannot be changed. If a USB Device is connected and remains powered during RZ system suspend, please consider a this point. By contrast, when a PC is put into sleep/suspend, it is common that the motherboard chipset will not behave this same way and will not USB reset (SE0 state). Therefore, a USB device may act differently on a RZ suspend/resume system as opposed to a PC suspend/resume system.
Failure to Retain Data while DRAM is in SELFREFRESH
Although the DRAM device is responsible for preserving correct data in the self-refresh mode, the user should also take
care about the correctness of data because a serious problem may arise in the system if DRAM fails to retain some data.
Even when a correct sequence is executed to enter the backup mode, some data may be lost or corrupted due to
unexpected fluctuations in power supplies, signals, or temperature.
As a way of detecting a failure to retain data, the user can write dummy data to an empty memory area that can be used
without causing any problem in the system before entering the backup mode, check whether the data is correct after
release from the backup mode, and if the data is not correct, discard the retained data and restart execution with a cold
boot.
As the DRAM device stores electronic charges in capacitors, a failure to retain data is usually found as a value of "0"
being read instead of the correct value of "1".
Some Devices take longer than others to resume
Devices with more CPU cores take longer to resume
Device with more active USB channels will take longer to resume