	Xvisor on Raspberry-Pi (or BCM2835 SOC) using SD Booting 

This section of readme explains how to build & run Xvisor on actual 
Raspberry Pi board using SD booting feature of Raspberry Pi (or BCM2835 SOC).

By default, Xvisor for BCM2835 supports Realview-EB-MPCore guest so we will 
show how to run Basic Firmware (and/or Linux) on Realview-EB-MPCore guest.

The Raspberry Pi usually comes with and SD card containig the GPU firmware
necessary to boot the ARM processor.
To create your own SD Card to boot the Raspberry Pi, do the following:
 1) Download and write prebuild raspbian image (not older than 7th Sept 2017)
 to SD card as described in: https://www.raspberrypi.org/downloads/raspbian/
 2) Optionally download the following files from the Raspberry Official
 Repository (https://github.com/raspberrypi/firmware/tree/master/boot)
 to the SD card if you are not able detect complete 1GB RAM:
 - fixup.dat
 - fixup_cd.dat
 - start.elf
 - start_cd.elf
 (Note: the _cd files are used when gpu_mem=16)

As next step, we install u-boot-2017.09 on the Raspberry Pi SD card
as follow:
 1) Download and build u-boot-2017.09
 # export ARCH=arm
 # export CROSS_COMPILE=arm-linux-gnueabi-
 # wget ftp://ftp.denx.de/pub/u-boot/u-boot-2017.09.tar.bz2
 # tar -xvf u-boot-2017.09.tar.bz2
 # cd u-boot-2017.09
 # make rpi_defconfig
 # make all
 2) Mount the boot partition of pre-installed SD card
 3) Copy u-boot.bin to <raspi_sd_card_boot_partition>
 4) Create <raspi_sd_card_boot_partition>/config.txt if it does not exist
 5) Add following lines to <raspi_sd_card_boot_partition>/config.txt:
 kernel=u-boot.bin
 gpu_mem=64
 6) Unmount the boot partition of pre-installed SD card

Once we have u-boot installed on SD card, we add 3 more images to the data
partition of SD card:
 1) U-Boot compatible Xvisor binary (uvmm.bin)
 2) DTB for Xvisor (bcm2835-rpi-b.dtb)
 3) U-Boot compatible Xvisor disk containing guest binaries (udisk.img)

Following are steps to create these images and boot Xvisor on Raspberry Pi:

  [1. Build environment for Xvisor]
  # export CROSS_COMPILE=arm-linux-gnueabi-

  [2. GoTo Xvisor source directory]
  # cd <xvisor_source_directory>

  [3. Initialize Xvisor submodules]
  # git submodule init
  # git submodule update
  (Note: This is required to be done only once in freshly cloned xvisor source)

  [4. Configure Xvisor with Generic v6 default settings]
  # make ARCH=arm generic-v6-defconfig

  [5. Build Xvisor & DTBs]
  # make

  [6. Make U-Boot compatible uvmm.bin for use with bootm command]
  # mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Xvisor -d build/vmm.bin build/uvmm.bin

  [7. Build Basic Firmware]
  # make -C tests/arm32/realview-eb-mpcore/basic

  [8. Copy defconfig to Linux build directory]
  # cp tests/arm32/realview-eb-mpcore/linux/linux-<linux_version>_defconfig <linux_build_directory>/.config

  [9. GoTo Linux source directory]
  # cd <linux_source_directory>

  [10. Configure Linux in build directory]
  # sed -i 's/0xff800000UL/0xff000000UL/' arch/arm/include/asm/pgtable.h
  # make O=<linux_build_directory> ARCH=arm oldconfig

  [11. Build Linux in build directory]
  # make O=<linux_build_directory> ARCH=arm Image dtbs

  [12. Patch Linux kernel to replace sensitive non-priviledged instructions]
  # <xvisor_source_directory>/arch/arm/cpu/arm32/elf2cpatch.py -f <linux_build_directory>/vmlinux | <xvisor_source_directory>/build/tools/cpatch/cpatch32 <linux_build_directory>/vmlinux 0

  [13. Extract patched Linux kernel image]
  # ${CROSS_COMPILE}objcopy -O binary <linux_build_directory>/vmlinux <linux_build_directory>/arch/arm/boot/Image

  [14. Create BusyBox RAMDISK to be used as RootFS for Linux kernel]
  (Note: For subsequent steps, we will assume that your RAMDISK is located at <busybox_rootfs_directory>/rootfs.img)
  (Note: Please refer tests/common/busybox/README.md for creating rootfs.img using BusyBox)

  [15. GoTo Xvisor source directory]
  # cd <xvisor_source_directory>

  [16. Create disk image for Xvisor with Guest Linux and Guest Basic Firmware]
  # mkdir -p ./build/disk/tmp
  # mkdir -p ./build/disk/system
  # cp -f ./docs/banner/roman.txt ./build/disk/system/banner.txt
  # cp -f ./docs/logo/xvisor_logo_name.ppm ./build/disk/system/logo.ppm
  # mkdir -p ./build/disk/images/arm32/realview-eb-mpcore
  # ./build/tools/dtc/bin/dtc -I dts -O dtb -o ./build/disk/images/arm32/realview-eb-mpcore.dtb ./tests/arm32/realview-eb-mpcore/realview-eb-mpcore.dts
  # cp -f ./build/tests/arm32/realview-eb-mpcore/basic/firmware.bin.patched ./build/disk/images/arm32/realview-eb-mpcore/firmware.bin
  # cp -f ./tests/arm32/realview-eb-mpcore/linux/nor_flash.list ./build/disk/images/arm32/realview-eb-mpcore/nor_flash.list
  # cp -f ./tests/arm32/realview-eb-mpcore/linux/cmdlist ./build/disk/images/arm32/realview-eb-mpcore/cmdlist
  # cp -f ./tests/arm32/realview-eb-mpcore/xscript/one_guest_ebmp.xscript ./build/disk/boot.xscript
  # cp -f <linux_build_directory>/arch/arm/boot/Image ./build/disk/images/arm32/realview-eb-mpcore/Image
  # cp -f <linux_build_directory>/arch/arm/boot/dts/arm-realview-eb-11mp-ctrevb.dtb ./build/disk/images/arm32/realview-eb-mpcore/arm-realview-eb-11mp-ctrevb.dtb
  # cp -f <busybox_rootfs_directory>/rootfs.img ./build/disk/images/arm32/rootfs.img
  # genext2fs -B 1024 -b 32768 -d ./build/disk ./build/disk.img
  OR
  [16. Create disk image for Xvisor with only Guest Basic Firmware]
  # mkdir -p ./build/disk/tmp
  # mkdir -p ./build/disk/system
  # cp -f ./docs/banner/roman.txt ./build/disk/system/banner.txt
  # cp -f ./docs/logo/xvisor_logo_name.ppm ./build/disk/system/logo.ppm
  # mkdir -p ./build/disk/images/arm32/realview-eb-mpcore
  # ./build/tools/dtc/bin/dtc -I dts -O dtb -o ./build/disk/images/arm32/realview-eb-mpcore.dtb ./tests/arm32/realview-eb-mpcore/realview-eb-mpcore.dts
  # cp -f ./build/tests/arm32/realview-eb-mpcore/basic/firmware.bin.patched ./build/disk/images/arm32/realview-eb-mpcore/firmware.bin
  # cp -f ./tests/arm32/realview-eb-mpcore/basic/nor_flash.list ./build/disk/images/arm32/realview-eb-mpcore/nor_flash.list
  # cp -f ./tests/arm32/realview-eb-mpcore/xscript/one_guest_ebmp.xscript ./build/disk/boot.xscript
  # genext2fs -B 1024 -b 32768 -d ./build/disk ./build/disk.img

  [17. Make U-boot compatible initrd for use with bootm command]
  # mkimage -A arm -O linux -T ramdisk -a 0x01000000 -n "Xvisor ramdisk" -d build/disk.img build/udisk.img

  [18. Unplug bootable SD card from Raspberry Pi and plug it on your development machine]

  [19. Mount data partition (or partition 2) of SD card on your development machine]

  [20. Copy uvmm.bin, DTB, and disk.im to data partition of SD card]
  # cp -f build/uvmm.bin <path_to_mounted_data_partition_of_sd_card>
  # cp -f build/arch/arm/board/generic/dts/broadcom/bcm2835-rpi-b.dtb <path_to_mounted_data_partition_of_sd_card>
  # cp -f build/udisk.img <path_to_mounted_dat_partition_of_sd_card>

  [21. Unmount data partition (or partition 2) of SD card from your development machine]

  [22. Unplug SD card from your development machine and plug it back on Raspberry Pi]

  [23. Connect to serial port of Raspberry Pi using Putty or Minicom]

  [24. Power-up or Reset Raspberry Pi and press any key when U-Boot shows auto-boot count down]

  [25. Enable MMC interface from U-Boot command line]
  U-Boot> mmc dev 0:0

  [26. Copy Xvisor from SD card to SDRAM]
  U-Boot> ext4load mmc 0:2 0x00200000 uvmm.bin

  [27. Copy Xvisor DTB from SD card to SDRAM]
  U-Boot> ext4load mmc 0:2 0x00800000 bcm2835-rpi-b.dtb

  [28. Copy disk image from SD card to SDRAM]
  U-Boot> ext4load mmc 0:2 0x02000000 udisk.img

  [29. Jump into Xvisor after disabling MMU, interrupts, etc]
  U-Boot> bootm 0x00200000 0x02000000 0x00800000

  [30. Kick Guest0 for starting Basic Firmware]
  XVisor# guest kick guest0

  [31. Bind to virtual UART]
  XVisor# vserial bind guest0/uart0

  [32. Try few commands of Basic firmware or goto next step]
  [guest0/uart0] basic# hi
  [guest0/uart0] basic# hello
  [guest0/uart0] basic# help

  [33. Copy linux from NOR flash to RAM and start linux booting from RAM]
  [guest0/uart0] basic# autoexec
  (Note: "autoexec" is a short-cut command)
  (Note: The <xvisor_source_directory>/tests/arm32/realview-eb-mpcore/linux/cmdlist
   file which we have added to guest NOR flash contains set of commands for booting
   linux from NOR flash)

  [34. Wait for Linux prompt to come-up and then try out some commands]
  [guest0/uart0] / # ls

  [35. Enter character seqence 'ESCAPE+x+q" return to Xvisor prompt]
  [guest0/uart0] / #

  (Note: replace all <> brackets based on your workspace)
  (Note: some of the above steps will need to be adapted for other
   types of ARM host)
  (Note: for more info on your desired ARM host refer docs/arm/)
  (Note: you are free to change the ordering of above steps based
   on your workspace)


	Xvisor on QEMU Raspberry-Pi (or BCM2835 SOC) Host

This section of readme explains how to build & run Xvisor on Raspberry-Pi QEMU
(Raspberry-Pi QEMU source, https://github.com/Torlus/qemu.git, and use the
rpi branch).

By default, Xvisor for BCM2835 supports Realview-EB-MPCore guest so we will 
show how to run Basic Firmware (and/or Linux) on Realview-EB-MPCore guest.

Please follow the steps below to build & run Basic Firmware on 
Realview-EB-MPCore guest with Xvisor running on QEMU Raspberry-Pi Host:

  [1. Build environment for Xvisor]
  # export CROSS_COMPILE=arm-linux-gnueabi-

  [2. GoTo Xvisor source directory]
  # cd <xvisor_source_directory>

  [3. Initialize Xvisor submodules]
  # git submodule init
  # git submodule update
  (Note: This is required to be done only once in freshly cloned xvisor source)

  [4. Configure Xvisor with Generic v6 default settings]
  # make ARCH=arm generic-v6-defconfig

  [5. Build Xvisor & DTBs]
  # make

  [6. Build Basic Firmware]
  # make -C tests/arm32/realview-eb-mpcore/basic

  [7. Copy defconfig to Linux build directory]
  # cp tests/arm32/realview-eb-mpcore/linux/linux-<linux_version>_defconfig <linux_build_directory>/.config

  [8. GoTo Linux source directory]
  # cd <linux_source_directory>

  [9. Configure Linux in build directory]
  # sed -i 's/0xff800000UL/0xff000000UL/' arch/arm/include/asm/pgtable.h
  # make O=<linux_build_directory> ARCH=arm oldconfig

  [10. Build Linux in build directory]
  # make O=<linux_build_directory> ARCH=arm Image dtbs

  [11. Patch Linux kernel to replace sensitive non-priviledged instructions]
  # <xvisor_source_directory>/arch/arm/cpu/arm32/elf2cpatch.py -f <linux_build_directory>/vmlinux | <xvisor_source_directory>/build/tools/cpatch/cpatch32 <linux_build_directory>/vmlinux 0

  [12. Extract patched Linux kernel image]
  # ${CROSS_COMPILE}objcopy -O binary <linux_build_directory>/vmlinux <linux_build_directory>/arch/arm/boot/Image

  [13. Create BusyBox RAMDISK to be used as RootFS for Linux kernel]
  (Note: For subsequent steps, we will assume that your RAMDISK is located at <busybox_rootfs_directory>/rootfs.img)
  (Note: Please refer tests/common/busybox/README.md for creating rootfs.img using BusyBox)

  [14. GoTo Xvisor source directory]
  # cd <xvisor_source_directory>

  [15. Create disk image for Xvisor with Guest Linux and Guest Basic Firmware]
  # mkdir -p ./build/disk/tmp
  # mkdir -p ./build/disk/system
  # cp -f ./docs/banner/roman.txt ./build/disk/system/banner.txt
  # cp -f ./docs/logo/xvisor_logo_name.ppm ./build/disk/system/logo.ppm
  # mkdir -p ./build/disk/images/arm32/realview-eb-mpcore
  # ./build/tools/dtc/bin/dtc -I dts -O dtb -o ./build/disk/images/arm32/realview-eb-mpcore.dtb ./tests/arm32/realview-eb-mpcore/realview-eb-mpcore.dts
  # cp -f ./build/tests/arm32/realview-eb-mpcore/basic/firmware.bin.patched ./build/disk/images/arm32/realview-eb-mpcore/firmware.bin
  # cp -f ./tests/arm32/realview-eb-mpcore/linux/nor_flash.list ./build/disk/images/arm32/realview-eb-mpcore/nor_flash.list
  # cp -f ./tests/arm32/realview-eb-mpcore/linux/cmdlist ./build/disk/images/arm32/realview-eb-mpcore/cmdlist
  # cp -f ./tests/arm32/realview-eb-mpcore/xscript/one_guest_ebmp.xscript ./build/disk/boot.xscript
  # cp -f <linux_build_directory>/arch/arm/boot/Image ./build/disk/images/arm32/realview-eb-mpcore/Image
  # cp -f <linux_build_directory>/arch/arm/boot/dts/arm-realview-eb-11mp-ctrevb.dtb ./build/disk/images/arm32/realview-eb-mpcore/arm-realview-eb-11mp-ctrevb.dtb
  # cp -f <busybox_rootfs_directory>/rootfs.img ./build/disk/images/arm32/rootfs.img
  # genext2fs -B 1024 -b 32768 -d ./build/disk ./build/disk.img
  OR
  [15. Create disk image for Xvisor with only Guest Basic Firmware]
  # mkdir -p ./build/disk/tmp
  # mkdir -p ./build/disk/system
  # cp -f ./docs/banner/roman.txt ./build/disk/system/banner.txt
  # cp -f ./docs/logo/xvisor_logo_name.ppm ./build/disk/system/logo.ppm
  # mkdir -p ./build/disk/images/arm32/realview-eb-mpcore
  # ./build/tools/dtc/bin/dtc -I dts -O dtb -o ./build/disk/images/arm32/realview-eb-mpcore.dtb ./tests/arm32/realview-eb-mpcore/realview-eb-mpcore.dts
  # cp -f ./build/tests/arm32/realview-eb-mpcore/basic/firmware.bin.patched ./build/disk/images/arm32/realview-eb-mpcore/firmware.bin
  # cp -f ./tests/arm32/realview-eb-mpcore/basic/nor_flash.list ./build/disk/images/arm32/realview-eb-mpcore/nor_flash.list
  # cp -f ./tests/arm32/realview-eb-mpcore/xscript/one_guest_ebmp.xscript ./build/disk/boot.xscript
  # genext2fs -B 1024 -b 32768 -d ./build/disk ./build/disk.img

  [16. Launch QEMU]
  # qemu-system-arm -M raspi -m 512M -display none -serial stdio -kernel build/vmm.bin -dtb build/arch/arm/board/generic/dts/broadcom/bcm2835-rpi-b.dtb -initrd build/disk.img

  [17. Kick Guest0 for starting Basic Firmware]
  XVisor# guest kick guest0

  [18. Bind to virtual UART]
  XVisor# vserial bind guest0/uart0

  [19. Try few commands of Basic firmware or goto next step]
  [guest0/uart0] basic# hi
  [guest0/uart0] basic# hello
  [guest0/uart0] basic# help

  [20. Copy linux from NOR flash to RAM and start linux booting from RAM]
  [guest0/uart0] basic# autoexec
  (Note: "autoexec" is a short-cut command)
  (Note: The <xvisor_source_directory>/tests/arm32/realview-eb-mpcore/linux/cmdlist
   file which we have added to guest NOR flash contains set of commands for booting
   linux from NOR flash)

  [21. Wait for Linux prompt to come-up and then try out some commands]
  [guest0/uart0] / # ls

  [22. Enter character seqence 'ESCAPE+x+q" return to Xvisor prompt]
  [guest0/uart0] / #

  (Note: replace all <> brackets based on your workspace)
  (Note: some of the above steps will need to be adapted for other
   types of ARM host)
  (Note: for more info on your desired ARM host refer docs/arm/)
  (Note: you are free to change the ordering of above steps based
   on your workspace)
