	Xvisor on Raspberry Pi4 board (or BCM2838) using SD Booting

This section of readme explains how to build & run Xvisor on actual BCM2838
SOC boards (Raspberry Pi4 for example) using SD booting.

By default, Xvisor for BCM2838 supports Virt-v8 guest so we will show how
to boot Linux using Basic Firmware on Virt-v8 guest.

The Raspberry Pi4 usually comes with a SD card containig the GPU firmware
necessary to boot the ARM processor.

To create your own SD Card for booting the Raspberry Pi4, download and write
prebuild raspbian image to SD card as described in:
https://www.raspberrypi.org/downloads/raspbian/

As next step, we install u-boot-2020.01 on the Raspberry Pi4 SD card
as follow:
 1) Download and build u-boot-2020.01
 # export ARCH=arm64
 # export CROSS_COMPILE=aarch64-none-linux-gnu-
 # wget ftp://ftp.denx.de/pub/u-boot/u-boot-2020.01.tar.bz2
 # cd u-boot-2020.01
 # make rpi_4_defconfig
 # make all
 # cd ..
 2) Mount the boot partition of pre-installed SD card
 3) Copy u-boot.bin to <raspi4_sd_card_boot_partition>
 4) Create <raspi4_sd_card_boot_partition>/config.txt if it does not exist
 5) Add following lines to <raspi4_sd_card_boot_partition>/config.txt:
 enable_uart=1
 kernel=u-boot.bin
 gpu_mem=64
 arm_64bit=1
 6) Unmount the boot partition of pre-installed SD card

Once we have u-boot installed on SD card, we add 3 more images to the boot
partition of SD card:
 1) U-Boot compatible Xvisor binary (uvmm.bin)
 2) DTB for Xvisor (bcm2838-rpi-4-b.dtb)
 3) U-Boot compatible Xvisro disk containing guest binaries (udisk.img)

Following are steps to create these images and boot Xvisor on Raspberry Pi4:

  [1. Build environment]
  # export CROSS_COMPILE=aarch64-none-linux-gnu-

  [2. GoTo Xvisor source directory]
  # cd <xvisor_source_directory>

  [3. Configure Xvisor with Generic-v8 default settings]
  # make ARCH=arm generic-v8-defconfig

  [4. Build Xvisor and DTBs]
  # make

  [5. Make U-Boot compatible uvmm.bin for use with bootm command]
  # mkimage -A arm64 -O linux -T kernel -C none -a 0x00080000 -e 0x00080000 -n Xvisor -d build/vmm.bin build/uvmm.bin

  [6. Build Basic Firmware]
  # make -C tests/arm64/virt-v8/basic

  [7. GoTo Linux source directory]
  # cd <linux_source_directory>

  [8. Configure Linux in build directory]
  # cp arch/arm64/configs/defconfig arch/arm64/configs/tmp-virt-v8_defconfig
  # <xvisor_source_directory>/tests/common/scripts/update-linux-defconfig.sh -p arch/arm64/configs/tmp-virt-v8_defconfig -f <xvisor_source_directory>/tests/arm64/virt-v8/linux/linux_extra.config
  # make O=<linux_build_directory> ARCH=arm64 tmp-virt-v8_defconfig

  [9. Build Linux in build directory]
  # make O=<linux_build_directory> ARCH=arm64 Image dtbs

  [10. Create BusyBox RAMDISK to be used as RootFS for Linux kernel]
  (Note: For subsequent steps, we will assume that your RAMDISK is located at <busybox_rootfs_directory>/rootfs.img)
  (Note: Please refer tests/common/busybox/README.md for creating rootfs.img using BusyBox)

  [11. GoTo Xvisor source directory]
  # cd <xvisor_source_directory>

  [12. Create disk image for Xvisor with Guest Linux and Guest Basic Firmware]
  # mkdir -p ./build/disk/tmp
  # mkdir -p ./build/disk/system
  # cp -f ./docs/banner/roman.txt ./build/disk/system/banner.txt
  # cp -f ./docs/logo/xvisor_logo_name.ppm ./build/disk/system/logo.ppm
  # mkdir -p ./build/disk/images/arm64/virt-v8
  # dtc -q -I dts -O dtb -o ./build/disk/images/arm64/virt-v8-guest.dtb ./tests/arm64/virt-v8/virt-v8-guest.dts
  # cp -f ./build/tests/arm64/virt-v8/basic/firmware.bin ./build/disk/images/arm64/virt-v8/firmware.bin
  # cp -f ./tests/arm64/virt-v8/linux/nor_flash.list ./build/disk/images/arm64/virt-v8/nor_flash.list
  # cp -f ./tests/arm64/virt-v8/linux/cmdlist ./build/disk/images/arm64/virt-v8/cmdlist
  # cp -f ./tests/arm64/virt-v8/xscript/one_novgic_guest_virt-v8.xscript ./build/disk/boot.xscript
  # cp -f <linux_build_directory>/arch/arm64/boot/Image ./build/disk/images/arm64/virt-v8/Image
  # dtc -q -I dts -O dtb -o ./build/disk/images/arm64/virt-v8/virt-v8.dtb ./tests/arm64/virt-v8/linux/virt-v8.dts
  # cp -f <busybox_rootfs_directory>/rootfs.img ./build/disk/images/arm64/virt-v8/rootfs.img
  # genext2fs -B 1024 -b 32768 -d ./build/disk ./build/disk.img
  OR
  [12. Create disk image for Xvisor with only Guest Basic Firmware]
  # mkdir -p ./build/disk/tmp
  # mkdir -p ./build/disk/system
  # cp -f ./docs/banner/roman.txt ./build/disk/system/banner.txt
  # cp -f ./docs/logo/xvisor_logo_name.ppm ./build/disk/system/logo.ppm
  # mkdir -p ./build/disk/images/arm64/virt-v8
  # dtc -q -I dts -O dtb -o ./build/disk/images/arm64/virt-v8-guest.dtb ./tests/arm64/virt-v8/virt-v8-guest.dts
  # cp -f ./build/tests/arm64/virt-v8/basic/firmware.bin ./build/disk/images/arm64/virt-v8/firmware.bin
  # cp -f ./tests/arm64/virt-v8/linux/nor_flash.list ./build/disk/images/arm64/virt-v8/nor_flash.list
  # cp -f ./tests/arm64/virt-v8/xscript/one_novgic_guest_virt-v8.xscript ./build/disk/boot.xscript
  # genext2fs -B 1024 -b 32768 -d ./build/disk ./build/disk.img

  [13. Make U-boot compatible udisk.img for use with bootm command]
  # mkimage -A arm64 -O linux -T ramdisk -a 0x00000000 -n "Xvisor Ramdisk" -d build/disk.img build/udisk.img

  [14. Unplug bootable SD card from the Raspberry Pi4 board and plug it on your development machine]

  [15. Mount data partition (or partition 2) of SD card on your development machine]

  [16. Copy uvmm.bin, DTB, and udisk.img to data partition of SD card]
  # cp -f build/uvmm.bin <path_to_mounted_data_partition_of_sd_card>
  # cp -f build/arch/arm/dts/broadcom/bcm2838-rpi-4-b.dtb <path_to_mounted_data_partition_of_sd_card>
  # cp -f build/udisk.img <path_to_mounted_data_partition_of_sd_card>

  [17. Unmount data partition (or partition 2) of SD card from your development machine]

  [18. Unplug SD card from your development machine and plug it back on Raspberry Pi4 board]

  [19. Connect to serial port of Raspberry Pi4 board using Putty or Minicom]

  [20. Power-up or Reset Raspberry Pi4 board and press any key when U-Boot shows auto-boot count down]

  [21. Enable MMC interface from U-Boot command line]
  U-Boot> mmc dev 0:0

  [22. Copy Xvisor from SD card to SDRAM]
  U-Boot> ext4load mmc 0:2 0x200000 uvmm.bin

  [23. Copy the Xvisor DTB from SD card to SDRAM]
  U-Boot> ext4load mmc 0:2 0x800000 bcm2838-rpi-4-b.dtb

  [24. Copy disk image from SD card to SDRAM]
  U-Boot> ext4load mmc 0:2 0x2000000 udisk.img

  [25. Jump into Xvisor after disabling MMU, interrupts, etc]
  U-Boot> bootm 0x200000 0x2000000 0x800000

  (Note: If you want to "autoboot" then you need to create a boot.scr file
   on the root of the SD card. To do so edit a boot.cmd text file and add all
   the commmands you want to execute automatically. For example you can put
   the following commands in the "boot.cmd" file:

    mmc dev 0:0
    ext4load mmc 0:1 0x200000 uvmm.bin
    ext4load mmc 0:1 0x800000 bcm2838-rpi-4-b.dtb
    ext4load mmc 0:1 0x2000000 udisk.img
    bootm 0x200000 0x2000000 0x800000

   You can also create U-Boot boot.scr file with the following command:
   mkimage -C none -A arm64 -T script -d boot.cmd boot.scr
   Put the boot.scr file a the boot partition of your SD card.)

  [26. Kick Guest0 for starting Basic Firmware]
  XVisor# guest kick guest0

  [27. Bind to virtual UART]
  XVisor# vserial bind guest0/uart0

  [28. Try few commands of Basic firmware or goto next step]
  [guest0/uart0] basic# hi
  [guest0/uart0] basic# hello
  [guest0/uart0] basic# help

  [29. Copy linux from NOR flash to RAM and start linux booting from RAM]
  [guest0/uart0] basic# autoexec
  (Note: "autoexec" is a short-cut command)
  (Note: The <xvisor_source_directory>/tests/arm64/virt-v8/linux/cmdlist
   file which we have added to guest NOR flash contains set of commands for
   booting linux from NOR flash)

  [30. Wait for Linux prompt to come-up and then try out some commands]
  [guest0/uart0] / # ls

  [31. Enter character seqence 'ESCAPE+x+q" return to Xvisor prompt]
  [guest0/uart0] / #

  (Note: replace all <> brackets based on your workspace)
  (Note: for more info on your desired ARM host refer docs/arm/)
  (Note: you are free to change the ordering of above steps based
   on your workspace)
