CPUIDLE

CPUIDLE is a hardware block present on R40/H6 and newer SoCs that implements the CPU/cluster idle process in hardware. It can also be used to implement CPU hot removal in a clean and race-free manner.

CPU On/Off Sequence

CPU Power Off

At least the following actions are taken:

1. DBGPWRDUP is cleared for the core.
2. The C0_PWROFF_GATING_REG bit for the core is set.
3. The CPU power switch is turned off, using the sequence and delays programmed below.
4. The CPUn_OFF or CPUn_VOTE bit in CPUIDLE_STAT_REG, as appropriate, is set.

CPU Power On

At least the following actions are taken:

1. The CPUn_OFF and CPUn_VOTE bits in CPUIDLE_STAT_REG are cleared.
2. The CPU power switch is turned on, using the sequence and delays programmed below.
3. The C0_PWROFF_GATING_REG bit for the core is cleared.
4. The C0_PWRON_RESET_REG bit for the core is set (deasserted).
5. The C0_RST_CTRL_REG bit for the core is set (deasserted).
6. DBGPWRDUP is set for the core.

Cluster On/Off Sequence

Cluster Power Off

At least the following actions are taken. There may be more actions (such as flushing L2 cache), but those happen too quickly to be observable.

1. C0_PWROFF_GATING_REG is set to 0x1f (cluster and all CPUs gated).
2. C0_PWRON_RESET_REG is set to 0x0 (all resets asserted).
3. The CLUSTER_OFF bit in CPUIDLE_STAT_REG is set.

Notably, CPU_SYS_RESET is not asserted, so at least part of the GIC is still functional.

Cluster Power On

At least the following actions are taken:

1. The CLUSTER_OFF bit in CPUIDLE_STAT_REG is cleared.
2. The C0_PWROFF_GATING_REG bit for the cluster (bit 4) is cleared.
3. The C0_PWRON_RESET_REG bit for the cluster (bit 16) is set (deasserted).
4. The C0_RST_CTRL_REG bits for various cluster resets are set (deasserted).
5. RVBARADDR for CPU 0 is set to the hardcoded value 0x20060 (in SRAM A1, after an assumed eGON header).
6. The CPU power on sequence is performed for the lead core.

Note: AA64nAA32 is not restored, making this functionality unsuitable for a 64-bit SoC.

Registers

The CPUIDLE hardware is controlled by several registers in the R_CPUCFG MMIO space. They are only partially documented. Many of the register and field names are guesses based on the observed functionality.

Register Layout (sun8i)

The register layout is partially documented in the A40i/T3/other manuals.

Register Layout (sun50i)

The register layout is not documented, but is very similar to the sun8i layout.

Register	Offset
CPUIDLE_EN_REG	0x100
CLOSE_FLAG_REG	0x104
CPUIDLE_PEND_REG	0x108
CPUIDLE_WAKE_REG	0x10c
CPUIDLE_STAT_REG	0x110
PWR_SW_DELAY_REG	0x140
CONFIG_DELAY_REG	0x144
PWR_DOWN_CFG_REG	0x148
PWR_UP_CFG_REGn	0x150 + 0x4 * n

Register Descriptions

CPUIDLE_EN_REG

Offset	0x100
Name	Bits	R/W	Default	Values	Description
KEY_FIELD	31:16	RW	0	0x16aa 0xaa16	This register must be programmed using two write cycles, once with the first key, then with the second key and the desired value for the low 16 bits.
/	15:1	/	/
CPUIDLE_EN	0	RW	0	0: Disable 1: Enable	Enable bit for CPUIDLE hardware. While this bit is zero, registers may be modified but no action is taken.

CLOSE_FLAG_REG

Offset	0x104
Name	Bits	R/W	Default	Values	Description
/	31:20	/	/
CLOSE_CPUn_AND_CLUSTER	19:16	R/WAC	0	0: No action 1: Request core and cluster power off	One bit exists for each core.
/	15:4	/	/
CLOSE_CPUn	3:0	R/WAC	0	0: No action 1: Request core power off	One bit exists for each core.

From the ARM CPUs, only the bits corresponding to the local CPU can be set. From the ARISC, any bit can be set.

No action will be taken until both:

1. The corresponding CPUn_IDLE bit in CPUIDLE_STAT_REG is set (i.e. the core is in WFI), and
2. The corresponding CPUn_VOTE or CPUn_OFF bit (as applicable) in CPUIDLE_STAT_REG is cleared

The bit will automatically clear when the the power off sequence finishes.

The bits cannot be cleared by writing to this register or by disabling the CPUIDLE hardware. If bits in this register are set while the CPU is already off, they can be cleared by clearing the CPUn_VOTE or CPUn_OFF bit in CPUIDLE_STAT_REG, thus re-triggering the power off sequence. However, if the CPU is on, there is no way to abort the sequence.

The power off sequence will proceed regardless of the value of CPUIDLE_PEND_REG.

CPUIDLE_PEND_REG

Offset	0x108
Name	Bits	R/W	Default	Values	Description
/	31:4	/	/
CPUn_IRQ_PENDING	3:0	RO	/	0: Not pending 1: Pending	One bit exists for each core.

There are probably bits for FIQs, but currently Group 0 interrupts are unused.

CPUIDLE_WAKE_REG

Offset	0x10c
Name	Bits	R/W	Default	Values	Description
/	31:4	/	/
CPUn_IRQ_WAKE	3:0	RW	0	0: CPU will not wake up 1: CPU will wake up if IRQ is pending	One bit exists for each core.

There are probably bits for FIQs, but currently Group 0 interrupts are unused.

The Allwinner blob initializes this register to 0xffff at boot.

CPUIDLE_STAT_REG

Offset	0x110
Name	Bits	R/W	Default	Values	Description
/	31:14	/	/
CLUSTER_IDLE	13	RO	/	0: Cluster is active 1: Cluster is idle	Appears to be driven by STANDBYWFIL2.
CLUSTER_OFF	12	RW	0	0: Cluster is on, power off is allowed 1: Cluster is off, power off is inhibited	Only effect is to inhibit the bits in CLOSE_FLAG_REG.
CPUn_VOTE	3:0	RW	0	0: CPU votes for cluster power on 1: CPU votes for cluster power off	One bit exists for each core.
CPUn_OFF	3:0	RW	0	0: CPU is on, power off is allowed 1: CPU is off, power off is inhibited	One bit exists for each core.
CPUn_IDLE	3:0	RO	/	0: CPU is active 1: CPU is idle	One bit exists for each core. Appears to be driven by STANDBYWFI.

The cluster is automatically powered off any time the following conditions are true:

1. CLUSTER_IDLE is set,
2. CLUSTER_OFF is clear, and
3. all CPUn_VOTE bits are set.

At cluster idle time, as coordinated by PSCI or SCPI, software can set the CPUn_VOTE bits to trigger a cluster power off. (Since all CPUs are already off at that point, CLUSTER_IDLE should be set automatically.) Therefore, it is never necessary to use CLOSE_CPUn_AND_CLUSTER at CPU hotplug/idle time.

Entry and Exit Time Measurements

Here is some information explaining the idle state timings for A64, but the same general concepts apply to other SoCs:

Powering off idle CPUs saves about 30 mW compared to using WFI only. Additional power savings are possible by idling the L2 and downclocking the cluster when all CPUs are idle, but those were not explored.

Entry and exit latency were measured using a logic analyzer, with GPIO pins toggled in Linux after the calls to trace_cpu_idle() in cpuidle_enter_state(), and in the power management firmware after CPU power-off completes and immediately after detecting an interrupt.

800 us and 1500 us are worst-case values, largely driven by the fact that the power management firmware is single threaded. It can only handle commands to power off CPUs one at a time, and it cannot process any commands while powering on a CPU in response to an interrupt.

The cluster suspend process reliably takes 36 us; I rounded this up to 50 us. If all CPUs enter the cluster idle state at the same time, exit latency is actually reduced, because there is no contention in that case. However, if only some CPUs enter the cluster idle state, behavior is the same as for CPU idle.

Polling delay for the power management firmware to detect a pending interrupt is insignificant; it is less than 20 us.

min-residency was chosen as the point where enabling the idle state consumed no more average power than disabling the idle state at a variety of interrupt rates.