A33 Suspend

Overview

I have managed to make suspend to ram work for Allwinner A33 SOCs and a recent mainline kernel.  I've tested it on 4 different tablets and the sin-a33 development board and it works on all of them.  The code is mostly cut and paste from the Allwinner A33 SDK with a bunch of ugly hacks to make it work with the mainline 4.11 kernel from sunxi-next.  It's unlikely I will be submitting the changes to mainline since I feel that the hacks are far from acceptable to get accepted into mainline, but the patch appears to work well enough for people that need to get suspend working in a hurry on specific devices.  If anyone wants to try to clean this up enough to get it into mainline, please go for it!

Major hacks that were needed to get the 3.4 kernel SDK code to work in 4.11 are:

• The big difference between 3.4 and 4.11 kernels is that the 4.11 kernel starts up in secure arm mode then switches to non-secure arm mode, while the 3.4 kernel just stays in secure arm mode.  Secure and non-secure have their own MMUs so the resume code has to setup both MMUs instead of just the secure MMU.  A huge hack was done in u-boot of using the PSCI interface to get the secure MMU register settings from non-secure mode.
• The security domain of memory pages may have very loose permissions when resuming from suspend.
• You must have a r_uart node enabled in your dts or else resume will hang!
• The clocks are not setup correctly in the dts for arisc so you must boot the kernel with the clk_ignore_unused parameter or else the arisc cpu won't start
• On a tablet with a bettery, if the battery is at 100% suspend will resume immediately. The wakeup flags are not set correctly so that it wakes up when the battery is fully charged./li>
• The ahb0 clock uses different clock-div and clock-mult settings (see sun8i-a23-a33.dtsi)
• The RSB bus frequency was reduced from 3Mhz to 2 Mhz since the 3Mhz was unreliable with the changed ahb0 clock-div and clock mult settings (see sun8i-a23-a33.dtsi)
• The DRAM settings from the Allwinner FEX file are parameters of the arisc node for each device.  The dram settings are specific to each device and have to be extracted from the FEX file for the device.  Code could probably be written to dynamically generate the dram settings from the dram registers instead of hardcoding them in the dts file, but I couldn't figure it out easily.
• The factory ar100 binary firmware is loaded into the ar100 chip by the arisc driver.  There is no available source code for the binary, but I have disassembled enough to see that it would not be impossible to rewrite just enough functionality to make our own open source firmware.
• There are random printk statements and delays sprinkled through the code to make the MMC wifi resume from suspend correctly.
• SDIO wifi seems to work correctly after resume if the rtl8703as driver module is unloaded before suspend and then reloaded after resume
• USB seems to work correctly after resume if USB is build as a module and the modules are unloaded before suspend and reloaded after resume.
• LCD screen, touchscreen, and gsensor all seem to work correctly after resume .

Its a big patch, which involves both u-boot, buildroot and linux-sunxi-next git repositories, so I've provided step by step instructions to build a working image in the hopes that someone else can recreate and continue this work to eventually get this functionality into the mainline kernel.

JTAG debugging is invaluable if something goes wrong.  A micro sdcard breakout board from Sparkfun and a FTDI C232HM Serial Cable for JTAG works great with the allwinner devices for debugging.  You are able to step through the u-boot and kernel code to find out where things go sideways.

Instructions to build image

These instructions will be to build an image for the sun8i-a33-ga10h-v1.1 tablet.  I have a serial cable attached to the tablet's r_uart (not the mmc serial port) to issue the suspend command.  You may be able to do it across the network interface if you modify the instructions to include network functionality, but it's not guaranteed.  I have been able to suspend from wifi and ethernet but the build instructions are much more complicated, so for these instructions it is assumed you have a serial console available.  The instructions should also work for a sinlinx sin-a33 development board (the patch includes a modified sun8i-a33-sinlinx-sina33.dts with an arisc node with dram parameters).  If you have some other tablet or board just add the arisc node to the dts file for your device and include the correct DRAM settings from your FEX file.  If the DRAM settings in the arisc node are incorrect, resume will hang with no feedback!

Setup a working directory for all the projects

• Create a new directory to hold all the different projects

mkdir ~/a33_suspend_test_build

Download the patches

cd ~/a33_suspend_test_build

wget http://micile.com/patches/u-boot_suspend.patch

wget http://micile.com/patches/linux-sunxi_suspend.patch

Build sunxi-tools

export ARCH=arm

export CROSS_COMPILE=arm-linux-gnueabi-

cd ~/a33_suspend_test_build

git clone https://github.com/linux-sunxi/sunxi-tools

cd sunxi-tools

git checkout d9b1d7e7dff1e70fea91a3f259e9e8ac9508de35

make

Build U-boot

• Clone the u-boot repository and then checkout a specific commit so everyone is building off the same code base, apply the patch, and then build.  To build for sin-a33, change the ga10h_v1_1_defconfig to Sinlinx_SinA33_defconfig

export ARCH=arm

export CROSS_COMPILE=arm-linux-gnueabi-

cd ~/a33_suspend_test_build

git clone git://git.denx.de/u-boot.git

cd u-boot

git checkout d53ecad92f06d2e38a5cbc13af7473867c7fa277

patch -p1 < ../u-boot_suspend.patch

make distclean

make ga10h_v1_1_defconfig

make -j9

Download some Allwinner Stuff

• In order to build the suspend code, we need gen_check_code from the allwinner sunxi-tools repository (which has the same name but is completely different from the sunxi-tools git repository)

  cd ~/a33_suspend_test_build

  git clone https://github.com/allwinner-zh/sunxi-tools allwinner-sunxi-tools

  cd allwinner-sunxi-tools

  git checkout d0b77a4b0a8cc381a8a18201ef12a558cd778d8c

Build Kernel

• Clone the linux-sunxi-next repository and then checkout a specific commit so everyone is building off the same code base, apply the patch, and then build.
  
• The build depends on the gen_check_code executable from the allwinner-sunxi-tools repository
  
• The build depends on the arisc binary firmware from the A80SDK (but the firmware is for A33)
  
• If the git commit for the kernel disappears, the patch should apply to anything close to 4.12-rc1 in sunxi-next branch
  

  export ARCH=arm

  export CROSS_COMPILE=arm-linux-gnueabi-

  cd ~/a33_suspend_test_build

  git clone https://github.com/linux-sunxi/linux-sunxi

  cd linux-sunxi

  git checkout cdb41f6bb3262b3c52e788fec9bb6f92a1cabb15

  git checkout 2ea659a

  patch -p1 < ../linux-sunxi_suspend.patch

  cp ../allwinner-sunxi-tools/tools-for-kernel-standby/gen_check_code arch/arm/mach-sunxi/pm/standby/

  wget http://dl.linux-sunxi.org/SDK/A80/A80_SDK_20140728_stripped/lichee/linux-3.4/drivers/arisc/binary/arisc_sun8iw5p1.code -P drivers/arisc/binary/

  make distclean

  make sunxi_defconfig

  make -j9 zImage dtbs

Build buildroot

• Clone the buildroot repository and then checkout a specific commit so everyone is building off the same code base, apply the patch, and then build.
  
• Use the buildroot defconfig, but manually change the target architecture to match A33 of little endian ARM and cortex-A8
  
• Change the output format to be compatible with a filesystem that u-boot can load into RAM
  

  cd ~/a33_suspend_test_build

  git clone git://git.buildroot.net/buildroot

  cd buildroot

  git checkout efd6d5fd24706d825abae5f2bb39771fce586f09

  make defconfig

  make menuconfig

  Inside the menuconfig, change the following settings
  Target options -> Target Architecture = ARM (little endian)
  Target options -> Target Architecture Variant = coretx-A8
  Filesystem images -> cpio the root filesystem = Y
  Filesystem images -> cpio the root filesystem -> Create U-boot image of the root filesystem = Y
  

  make

Boot the device into FEL mode

• Use the instructions at http://linux-sunxi.org/FEL to get your device into FEL mode. My preferred method is using the FEL sdcard image, but choose whatever works for you
• Verify device is in FEL mode using the instructions at  http://linux-sunxi.org/FEL

Boot the kernel using FEL

To boot the Sin-A33 board, change the dts file passed into u-boot to the the Sin-A33 dtb file instead of the GA10H-v1.1 file.

cd ~/a33_suspend_test_build

./sunxi-tools/sunxi-fel -v uboot ./u-boot/u-boot-sunxi-with-spl.bin write-with-progress 0x42000000 ./linux-sunxi/arch/arm/boot/zImage write-with-progress 0x43000000 ./linux-sunxi/arch/arm/boot/dts/sun8i-a33-ga10h-v1.1.dtb     write 0x43300000 ./buildroot/output/images/rootfs.cpio.uboot

• u-boot will boot and then dump you to a u-boot prompt since there are no valid boot commands
• manually boot the kernel by issuing the following commands from the u-boot prompt (be careful, the lines seem to be too long for the u-boot serial buffer to cut and paste using putty so the long line may need to be pasted in 2 parts)

setenv bootargs console=ttyS0,115200 loglevel=10 panic=10 earlyprintk debug no_console_suspend clk_ignore_unused

bootz 0x42000000 0x43300000 0x43000000

• The kernel will boot along with the buildroot root filesystem and you will end up at a login prompt on your serial console
• login as root

Execute the suspend command

• Run the following command from the shell prompt on the serial console

echo standby > /sys/power/state

• The table will go into suspend.  To wake up, click the power button and the serial console will respond to commands again.
• While in suspend, power consumption drops to about 1/2 watt. I think it can drop much lower with some better tuning of which clocks are enabled and disabled.
• On a tablet with a bettery, if the battery is at 100% suspend will resume immediately. The wakeup flags are not set correctly so that it wakes up when the battery is fully charged./li>

Tada! And suspend and resume should work if everything went right...