Qemu and other emulators
This note covers setting up Qemu on an x86-based development system running Linux. This allows native (rather than cross) development tools to be run, which can be useful where the target system has performance/resource issues (e.g. some ARM systems), is not run natively due to company policy (older versions of Microsoft Windows) or is quite simply unavailable at a reasonable price (e.g. SGI MIPS systems or the fabled Chinese MIPS-based systems). It also briefly mentions User Mode Linux and the Hercules emulator for IBM zSeries mainframes despite the fact that these are not particularly relevant to Free Pascal, it does not consider x86-on-x86 virtualisation systems such as VMware.
The Host System
In the current case, the host is a Compaq rack-mount server running at around 3GHz. It has two internal drive cages, the first is connected to a RAID controller and is used for the host operating system and tools, the second is connected to a SCSI controller and contains 6x discs each of which is used for a different guest system.
The host IP address is 192.168.1.22 and the system is named pye-dev-07, the default gateway and name server are on 192.168.1.1. Guest systems are on the 192.168.22.x subnet and are named pye-dev-07a (192.168.22.16), pye-dev-07b (192.168.22.17) and so on, they have their own gateway 192.168.22.1 which is known to the site router and firewalls.
The host operating system is Debian "Squeeze", the host normally runs headless and may be accessed by SSH, X using XDMCP, or VNC. The display manager is gdm since this has a better XDMCP implementation than the alternatives, however in practice graphical login is most often handled by VNC.
The following guests are implemented:
- Debian on ARM (little-endian, armel) using Qemu
- Debian on MIPS (little-endian, mipsel) using Qemu
- Slackware 13.37 using User Mode Linux
- Windows 2K using Qemu
- Debian on zSeries using the Hercules emulator
Originally, pye-dev-07a was earmarked for big-endian ARM, but this appears to be being phased out by Debian so is probably no longer a viable target. Anybody planning to port FPC to the AVR-based Arduino?
In general, multiple guests can run simultaneously although this has not been exhaustively tested recently.
In the case of Linux the guest systems are each installed on an 18Gb disc, in the case of Windows a 36Gb disc is used. Each disc is assigned a label using e2label (arm, armel and so on), so that the startup script can mount it by name irrespective of which drive cage slot it's in.
Debian Guest using Qemu
Select a suitable Debian mirror and version, for example
Fetch a kernel and initrd image for Debian Squeeze, as below.
For ARM (little-endian):
In addition for this architecture you also need:
For MIPS (little-endian):
Copy these to the disc reserved for the guest, e.g. /export/mipsel.
Create a filesystem for Qemu, e.g.:
# qemu-img create -f qcow mipsel_hda.img 16G
Expect that to round up to around 17.1 Gb, if it doesn't then experiment. Start Qemu, telling it what kernel, initrd and filesystem to use:
For ARM (little-endian):
# qemu-system-arm -M versatilepb -kernel vmlinuz-2.6.32-5-versatile -initrd initrd.gz \ -hda armel_hda.img -append root=/dev/ram
Note that the above command requires X access, e.g. ssh with the -X option.
For MIPS (little-endian):
# qemu-system-mipsel -M malta -kernel vmlinux-2.6.32-5-4kc-malta -initrd initrd.gz \ -hda mipsel_hda.img -append "root=/dev/ram console=ttyS0" -nographic
Install the guest operating system as usual, splitting the disc into 16.5Gb for / with the remainder (around 700Mb) as swap. This will be slow, 8 or 9 hours is not implausible, so make sure that nobody's about to turn off your mains or disconnect you from the Internet.
Don't worry if it tells you it's not installing a loader- it's not needed since the kernel and initrd are loaded into memory by the host.
Boot the operating system and set network addresses etc. Use 192.168.22.16 or similar, with a gateway of 192.168.22.1.
For ARM (little-endian):
# qemu-system-arm -M versatilepb -kernel vmlinuz-2.6.32-5-versatile \ -initrd initrd.img-2.6.32-5-versatile -hda armel_hda.img -append "root=/dev/sda1"
For MIPS (little-endian):
# qemu-system-mipsel -M malta -kernel vmlinux-2.6.32-5-4kc-malta \ -hda mipsel_hda.img -append "root=/dev/sda1 console=ttyS0" -nographic
Finally, you should be able to boot the operating system with an operational network. This relies on having /etc/qemu-ifup and /etc/qemu-ifdown files (see below), and passes additional parameters to them in shell variables. In outline:
For ARM (little-endian):
# qemu-system-arm -M versatilepb -m 256 -hda armel_hda.img \ -kernel vmlinuz-2.6.32-5-versatile -initrd initrd.img-2.6.32-5-versatile \ -append 'root=/dev/sda1 text' \ -net nic,macaddr=00:16:3e:00:00:01 -net tap,ifname=tun1
For MIPS (little-endian):
# qemu-system-mipsel -M malta -m 256 -hda mipsel_hda.img \ -kernel vmlinux-2.6.32-5-4kc-malta -no-reboot \ -append 'root=/dev/sda1 console=ttyS0' -nographic \ -net nic,macaddr=00:16:3e:00:00:02 -net tap,ifname=tun2
Remember that if you change the network interface type or MAC address you will probably need to delete entries from the guest's /etc/udev/rules.d/z??_persistent-net.rules file.
Windows 2K Guest using Qemu
Use dd to save a .iso image of the installation CD. Create a filesystem image:
# qemu-img create -f qcow2 win2k.img 32G
Boot using startup script as below. Note that this must specify a non-default network card, since Qemu's current (as of 2011) default is not supported by Windows 2K.
TODO: run with kernel support module.
Common Qemu startup, ifup and ifdown scripts
There is much commonality irrespective of whether the guest is running Linux or Windows.
First startup script (e.g. /export/C):
#!/bin/sh mount -L mipsel cd /export/mipsel . ./C-2
Second startup script for ARM (little-endian):
#!/bin/sh # Routine startup of a Qemu guest relies on (the host) running /etc/qemu-ifup # to condition ARP, forwarding etc. QEMU_ID=1 QEMU='qemu-system-arm -M versatilepb' QEMU_RAM='-m 256' QEMU_HD='-hda armel_hda.img' QEMU_CD= QEMU_BOOT="-kernel vmlinuz-2.6.32-5-versatile -append 'root=/dev/sda1 text' -initrd initrd.img-2.6.32-5-versatile" # QEMU_MONITOR='-monitor stdio -nographic' # QEMU_MONITOR='-nographic' QEMU_VGA= VNC_ID=$(($QEMU_ID+1)) # QEMU_VNC="-vnc :$VNC_ID -k en-gb" QEMU_VNC= QEMU_NET="-net nic,macaddr=00:16:3e:00:00:0$QEMU_ID -net tap,ifname=tun$QEMU_ID" QEMU_GUEST_IP_ADDRESS=192.168.22.17 QEMU_GUEST_IP_GATEWAY=192.168.22.1 QEMU_HOST_GATEWAY_IF=eth1 export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY_IF echo \* $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \ $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC screen -S QEMU_$QEMU_ID \ sh -c "$QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \ $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC" cd ..
Second startup script for MIPS (little-endian):
#!/bin/sh # Routine startup of a Qemu guest relies on (the host) running /etc/qemu-ifup # to condition ARP, forwarding etc. QEMU_ID=2 QEMU='qemu-system-mipsel -M malta' QEMU_RAM='-m 256' QEMU_HD='-hda mipsel_hda.img' QEMU_CD= QEMU_BOOT="-kernel vmlinux-2.6.32-5-4kc-malta -append 'root=/dev/sda1 console=ttyS0' -no-reboot" # QEMU_MONITOR='-monitor stdio -nographic' QEMU_MONITOR='-nographic' QEMU_VGA= VNC_ID=$(($QEMU_ID+1)) # QEMU_VNC="-vnc :$VNC_ID -k en-gb" QEMU_VNC= QEMU_NET="-net nic,macaddr=00:16:3e:00:00:0$QEMU_ID -net tap,ifname=tun$QEMU_ID" QEMU_GUEST_IP_ADDRESS=192.168.22.18 QEMU_GUEST_IP_GATEWAY=192.168.22.1 QEMU_HOST_GATEWAY_IF=eth1 export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY_IF echo \* $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \ $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC screen -S QEMU_$QEMU_ID \ sh -c "$QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \ $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC" cd ..
Second startup script for Windows:
#!/bin/sh # Routine startup of a Qemu guest relies on (the host) running /etc/qemu-ifup # to condition ARP, forwarding etc. QEMU_ID=4 QEMU=qemu QEMU_RAM='-m 256' QEMU_HD='-hda win2k.img' QEMU_CD='-cdrom Windows2k-SP4.iso' QEMU_BOOT='-boot c' QEMU_MONITOR='-monitor stdio' QEMU_VGA='-vga cirrus' VNC_ID=$(($QEMU_ID+1)) QEMU_VNC="-vnc :$VNC_ID -k en-gb" QEMU_NET="-net nic,macaddr=00:16:3e:00:00:0$QEMU_ID,model=rtl8139 -net tap,ifname=tun$QEMU_ID" QEMU_GUEST_IP_ADDRESS=192.168.22.20 QEMU_GUEST_IP_GATEWAY=192.168.22.1 QEMU_HOST_GATEWAY_IF=eth1 export QEMU_GUEST_IP_ADDRESS QEMU_GUEST_IP_GATEWAY QEMU_HOST_GATEWAY_IF echo \* $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \ $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC screen -S QEMU_$QEMU_ID \ $QEMU $QEMU_RAM $QEMU_HD $QEMU_CD $QEMU_BOOT \ $QEMU_MONITOR $QEMU_VGA $QEMU_NET $QEMU_VNC cd ..
/etc/ifup (for both Linux and Windows):
#!/bin/bash # if-up file for qemu, heavily cribbed from the command sequence embedded in # User Mode Linux. MarkMLl. echo Running /etc/qemu-ifup $1 $2... # For compatibility with UML the only parameter here is $1 which is the # interface name. I've put in a reference to $2 so we can see it if anything # changes. # I'm going to assume that qemu is always run by root. This is fairly # reasonable since it allows guest OSes to be fired up which themselves might # give access to confidential data etc. if compromised. # Here's my equivalent to the host-side UML setup for Qemu. We're hamstrung # here by the fact that the emulator is not telling us what IP address it's # trying to enable, there isn't a 1:1 correspondence between IP addresses and # interfaces since the latter depends on the order the sessions are started. # # As a hack, assume that the caller exports QEMU_GUEST_IP_ADDRESS (e.g. # 192.168.17.16), QEMU_GUEST_IP_GATEWAY (e.g. 192.168.17.1) and # QEMU_HOST_GATEWAY_IF (e.g. eth0). echo \* modprobe tun modprobe tun echo \* ifconfig $1 $QEMU_GUEST_IP_GATEWAY netmask 255.255.255.255 up ifconfig $1 $QEMU_GUEST_IP_GATEWAY netmask 255.255.255.255 up X=`cat /proc/sys/net/ipv4/ip_forward` if [ "$X" == "0" ]; then # Use either this... # echo Global forwarding is not enabled. Please refer to the administrator # echo responsible for this machine, enabling it might be a security hazard. # ...or this. echo Forcibly enabling global forwarding, note that this might be a security hazard. echo \* echo 1 \> /proc/sys/net/ipv4/ip_forward echo 1 > /proc/sys/net/ipv4/ip_forward X=`cat /proc/sys/net/ipv4/ip_forward` if [ "$X" == "0" ]; then echo Unable to enable global forwarding. Please refer to the administrator echo responsible for this machine. fi fi echo \* route add -host $QEMU_GUEST_IP_ADDRESS dev $1 route add -host $QEMU_GUEST_IP_ADDRESS dev $1 echo \* echo 1 \> /proc/sys/net/ipv4/conf/$1/proxy_arp echo 1 > /proc/sys/net/ipv4/conf/$1/proxy_arp X=`cat /proc/sys/net/ipv4/conf/$1/proxy_arp` if [ "$X" == "0" ]; then echo -n Retrying while [ "$X" == "0" ]; do sleep 1 echo -n . echo 1 > /proc/sys/net/ipv4/conf/$1/proxy_arp X=`cat /proc/sys/net/ipv4/conf/$1/proxy_arp` done echo OK fi echo \* arp -Ds $QEMU_GUEST_IP_ADDRESS $1 pub arp -Ds $QEMU_GUEST_IP_ADDRESS $1 pub echo \* arp -Ds $QEMU_GUEST_IP_ADDRESS $QEMU_HOST_GATEWAY_IF pub arp -Ds $QEMU_GUEST_IP_ADDRESS $QEMU_HOST_GATEWAY_IF pub # Set up experimental UDP proxies. Depending on the protocol of interest # messages in one or both directions might need to be relayed. # # UDP port 79 is used for Dialarm signals, a unidirectional proxy is # adequate for this but detection of hosts changing state (i.e. being # added to or removed from the population of cooperating systems) is far # more responsive if a bidirectional proxy is available. PROXY_ID=1 case "$1" in tun1) PROXY_ID=2 ;; tun2) PROXY_ID=3 ;; tun3) PROXY_ID=4 ;; tun4) PROXY_ID=5 ;; tun5) PROXY_ID=6 ;; esac # echo \* udp-broadcast-relay -f $PROXY_ID 79 $QEMU_HOST_GATEWAY_IF $1 # /usr/local/src/udp-broadcast-relay/udp-broadcast-relay-0.3/udp-broadcast-relay \ -f $PROXY_ID 79 $QEMU_HOST_GATEWAY_IF $1 # Alternatively use this one which is oriented towards IP addresses # rather than interfaces. # Note attempt to counteract any niceness applied to Qemu itself. ps ax | grep 'udp-proxy[ ]-z 79 ' >/dev/null 2>&1 if [ $? != 0 ]; then echo \* udp-proxy -z 79 $QEMU_GUEST_IP_ADDRESS /usr/bin/nice --adjustment=20 /usr/local/src/udp-proxy/udp-proxy -z 79 $QEMU_GUEST_IP_ADDRESS else echo \* Already running udp-proxy -z 79 $QEMU_GUEST_IP_ADDRESS fi # echo \* udp-proxy -z 13264 $QEMU_GUEST_IP_ADDRESS # /usr/local/src/udp-proxy/udp-proxy -z 13264 $QEMU_GUEST_IP_ADDRESS echo .../qemu/qemu-ifup completed.
/etc/ifdown (for both Linux and Windows):
#!/bin/sh echo \* route del -host $QEMU_GUEST_IP_ADDRESS dev $1 route del -host $QEMU_GUEST_IP_ADDRESS dev $1 echo \* ifconfig $1 down ifconfig $1 down
In actual fact, these operations were cribbed from User Mode Linux (below) where they are embedded inside a host library.
Slackware Guest using User Mode Linux
User Mode Linux runs a guest kernel as a standard program, i.e. there is no emulation or virtualisation involved. The guest kernel can be allocated either physical discs or filesystems contained in files.
Put a .iso corresponding to a recent Slackware DVD in /export/uml. Unpack the initrd using zcat and cpio, save it as an ext3 image initrd_unpacked. Using dd create an empty file root_fs_slackware which will be partitioned and formatted during installation.
Use the sources from e.g. a recent Slackware to compile kernel plus modules with ARCH=um using a suffix -uml. Save the kernel to /export/uml/linux, install the modules and then copy them into the initrd filesystem.
Boot the UML kernel, telling it to use the initrd image and DVD iso:
# ./linux ubd0=initrd_unpacked ubd1=root_fs_slackware fake_ide ubd2r=slackware-13.37-install-dvd.iso rw
Run fdisk and setup as normal, you might need to tell it to install to /dev/ubd1 and use /dev/ubd2 for source. Finally, copy the modules onto the target filesystem.
When complete start up like this:
# Routine startup of a UML guest relies on (the host) running /usr/lib/uml/uml_net # to condition ARP, forwarding etc. echo \* ./linux ubd0=initrd_unpacked ubd1=root_fs_slackware fake_ide ubd2r=slackware-13.37-install-dvd.iso \ root=/dev/ubdb1 eth0=tuntap,,,192.168.1.22 screen -S UML_3 \ ./linux ubd0=initrd_unpacked ubd1=root_fs_slackware fake_ide ubd2r=slackware-13.37-install-dvd.iso \ root=/dev/ubdb1 eth0=tuntap,,,192.168.1.22 cd ..
Note that this is usually run from an X session, since the multiple virtual consoles appear as separate xterms.
Debian Guest using Hercules
Hercules is a commercial-grade emulator for IBM mainframes. Once the emulator is running, enter
to boot Linux from device 120. Hopefully SSH will be operational so you won't need to interact with the console, but if you do then prefix each line that is to go to the guest operating system (i.e. rather than to the console itself) with a dot.
Refer to the URL in the script below for more details.
#!/bin/sh # PREREQUISITE: Boot with ipl 120 # Note that this makes no attempt to support IPv6. iptables -t nat -A POSTROUTING -o eth1 -s 192.168.22.0/24 -j MASQUERADE iptables -A FORWARD -s 192.168.22.0/24 -j ACCEPT iptables -A FORWARD -d 192.168.22.0/24 -j ACCEPT echo 1 > /proc/sys/net/ipv4/ip_forward echo 1 > /proc/sys/net/ipv4/conf/all/proxy_arp # http://www.josefsipek.net/docs/s390-linux/hercules-s390.html screen -S HERC_5 \ hercules cd ..
CPUSERIAL 000069 # CPU serial number CPUMODEL 9672 # CPU model number MAINSIZE 256 # Main storage size in megabytes XPNDSIZE 0 # Expanded storage size in megabytes CNSLPORT 3270 # TCP port number to which consoles connect NUMCPU 2 # Number of CPUs LOADPARM 0120.... # IPL parameter OSTAILOR LINUX # OS tailoring PANRATE SLOW # Panel refresh rate (SLOW, FAST) ARCHMODE ESAME # Architecture mode ESA/390 or ESAME # .-----------------------Device number # | .-----------------Device type # | | .---------File name and parameters # | | | # V V V #--- ---- -------------------- # console 001F 3270 # terminal 0009 3215 # reader 000C 3505 /export/zlinux/rdr/kernel.debian /export/zlinux/rdr/parmfile.debian /export/zlinux/rdr/initrd.debian autopad eof # printer 000E 1403 /export/zlinux/prt/print00e.txt crlf # dasd 0120 3390 /export/zlinux/dasd/3390.LINUX.0120 0121 3390 /export/zlinux/dasd/3390.LINUX.0121 # tape 0581 3420 # network s390 realbox # 0A00,0A01 CTCI -n /dev/net/tun -t 1500 10.1.1.2 10.1.1.1 0A00,0A01 CTCI -n /dev/net/tun -t 1500 192.168.22.21 192.168.1.22
Note that the guest network is configured as SLIP. Best not fooled with.
Performance is, in general, disappointing. Using an informal "torture test" which exercises the CPU and disc access:
Compaq ProLiant ML530 G2, 2.8GHz, 3Gb, 8 jobs, 390W 0m12.170 79 Linksys NSLU2, 266MHz 32Mb, 1 job, 7W 6m35.014s 46 Qemu ARM, 1 job, 390W 42m58.925s 16,757 Qemu MIPS, 1 job, 390W 17m49.103s 6,949 Qemu x86, 1 job, 390W 47m0.441s 18,330 UML, 1 job, 390W 8m26.529s 3,289 Hercules zSeries, 4 jobs, 390W 9m43.330s 3,790
The final column in the above list is the W-Mins to complete the job. These timings are without the benefit of kernel support from kqemu (obsolete) or KVM, but the fact that Qemu's support for MIPS is significantly better than that for other architectures, and the fact that the Hercules emulator is in all cases better than Qemu, does make one wonder how efficient the code is.
Compiling FPC 2.4.2 using time make NOGDB=1 OPT='-O- -gl' all :
Host (x86, Linux, Debian): real 4m47.842s user 3m42.126s sys 0m33.506s Slug (32Mb, ARM, Linux, Debian): real 284m58.543s user 86m45.570s sys 20m46.500s Qemu (ARM, Linux, Debian): real 406m31.931s user 236m49.030s sys 148m58.110s Qemu (x86, Linux, Slackware): real 141m45.700s user 122m40.724s sys 17m15.670s Qemu (x86, Windows 2000): Elapsed 108m UML (x86, Linux, Slackware): real 238m41.257s user 45m54.460s sys 3m44.140s
Compiling Lazarus 0.9.30 using time make LCL_PLATFORM=gtk2 bigide :
Host (x86, Linux, Debian): real 2m21.072s user 2m6.452s sys 0m12.285s Slug (32Mb, ARM, Linux, Debian): real 9385m49.319s user 67m23.460s sys 430m9.870s Qemu (ARM, Linux, Debian): real 281m55.536s user 153m3.150s sys 53m27.470s Qemu (x86, Linux, Slackware): real 70m53.957s user 60m3.474s sys 8m8.801s Qemu (x86, Windows 2000, default platform): Elapsed 53m UML (x86, Linux, Slackware): real 489m40.233s user 81m43.740s sys 7m51.280s
Graphical Access using VNC etc.
All of the above assumes that SSH is available so that shell (command line) sessions can be run over the network. Where necessary install e.g. the Debian ssh (or openssh, openssh-server etc.) package. It is also possible to run a program using a GUI over SSH by X forwarding, see the SSH client's -X option and the corresponding server configuration-file settings.
There are several ways of running a program using a GUI without forwarding the X protocol over SSH:
- Running Qemu or UML in an xterm.
- Using Qemu's host VNC support.
- Connecting to a Qemu (or UML etc.) guest using X.
- Connecting to a Qemu (or UML etc.) guest using VNC.
In all cases it is worth using a low-resource window manager such as FluxBox or xfwm rather than KDE or Gnome.
Running Qemu or UML in an xterm or with host VNC support
This depends on the build options of the guest system kernel etc. In general, display resolution and depth will be limited by the emulated hardware, the network ports are associated with the host (rather than guest) system. See the Qemu documentation for details.
Connecting to a Qemu (or UML etc.) guest using X
Install the Gnome Display Manager (gdm) on the guest system. This allows a desktop system to log into it using XDMCP, the desktop system determines the display resolution and depth.
Connecting to a Qemu (or UML etc.) guest using VNC
This method also works well for arbitrary systems, even if Qemu etc. is not being used.
Install gdm and RealVNC (or, with limitations, TightVNC) on the guest system; the latter might be needed for ARM guests but RealVNC is to be preferred when available. This allows a desktop system to log into it using VNC, the guest system determines the display resolution and depth and network ports are associated with the guest (rather than host) system.
In /etc/inittab, insert something like this:
6:23:respawn:/sbin/getty 38400 tty6 # Added in lieu of trying to get it working in inetd.conf. MarkMLl. 8:23:respawn:/usr/local/vnc/vncshim-0-10 # 9:23:respawn:/usr/local/vnc/vncshim-4-14 # Example how to put a getty on a serial line (for a terminal) # #T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100
Note here that I routinely run two VNC servers per system, more than this don't work for reasons that I've not researched in detail.
Create /usr/local/vnc/vncshim-0-10 to read:
#!/bin/bash # This forms a shim between /etc/inittab and the VNC server, necessitated by # the facts that (a) I can't get this command working reliably in inetd.conf # and (b) lines in initab have restricted length. # # Note that the name of the file reflects the fact that it might request non- # standard combinations of VNC port and X display number. MarkMLl. # My preference is to select default linear dimensions that appear somewhere # in the list of conventional graphics modes. 1200x960 fits inside a 1280x1024 # screen, even allowing that the title bar etc. will take additional space. SIZE=1200x960 # The DPI setting can usefully be set to match the user's normal screen, leave # undefined (commented out) if in doubt. DPI='-dpi 96' # Each VNC server running on a host must have a distinct suffix to prevent port # and display number clashes. SUFFIX=0 # Some option formats vary between RealVNC (top) and TightVNC (bottom). OPTIONS='DisconnectClients=0 -NeverShared' # OPTIONS='-dontdisconnect -nevershared' ####################### CHANGE NOTHING BELOW HERE ########################### VNCPORT=590$SUFFIX XDISPLAY=:1$SUFFIX # Try to work out what this system is. Note particular care to allow spaces # in the desktop name. WHATAMI=`dirname $0`/whatami if [ -x $WHATAMI ]; then WHATAMI2=`$WHATAMI` else if [ -r $WHATAMI ]; then WHATAMI2=`cat $WHATAMI` else WHATAMI2=$$ fi fi DESKTOP='-desktop '\'$WHATAMI2\' # Optional Java viewer. if [ -r `dirname $0`/classes/index.vnc ]; then JAVA="-httpd `dirname $0`/classes -httpPort 580$SUFFIX" fi # This is the only way I can get it working the way I want, i.e. with reliable # handling of spaces in the desktop name. I'm hardly proud of it. rm -f `dirname $0`/.vncshim-$SUFFIX-1$SUFFIX cat >`dirname $0`/.vncshim-$SUFFIX-1$SUFFIX <<EOT #!/bin/sh exec /usr/bin/Xvnc -geometry $SIZE $DPI -depth 16 $DESKTOP \ -rfbport $VNCPORT -rfbauth /root/.vnc/passwd-$SUFFIX $JAVA \ $OPTIONS -query localhost -once $XDISPLAY EOT chmod +x `dirname $0`/.vncshim-$SUFFIX-1$SUFFIX exec `dirname $0`/.vncshim-$SUFFIX-1$SUFFIX
Create /usr/local/vnc/whatami to read:
#!/bin/bash echo -n "`hostname`, " _WHATAMI=`uname -m` if [[ $_WHATAMI == i?86 ]]; then _WHATAMI='i586' fi if [[ $_WHATAMI == arm* ]]; then _WHATAMI='arm' fi if [ -x `dirname $0`/whatami-$_WHATAMI ]; then `dirname $0`/whatami-$_WHATAMI else uname -m fi
Additional scripts e.g. whatami-arm can query /proc/cpuinfo, which is different for each architecture. Make sure that all scripts are set +x for root.
Use vncpasswd to set up password files /root/.vnc/passwd-0 etc.
So far, I find that RealVNC works fine when invoked as above, but TightVNC doesn't correctly invoke the display manager.
QEMU User Emulation Mode in Chrooted Environment
In previous chapters QEMU's full system emulation mode was described: a complete guest system is emulated including bios and hardware. Although this approaches a real environment as close as possible, the overhead created by emulating a complete system is considerable. QEMU's user emulation mode allows to run "guest" code directly on the "host" system. The cpu is still emulated but system calls are translated into system calls on the host system. Libraries used by the "guest" code are the "guest" libraries, not the host libraries. To run a small program created for fe. ARM with no dependencies you can simply do a
$ qemu-arm program
When the program has more and more dependencies, it becomes increasingly difficult to put all dependencies in locations that can be found by qemu-arm without messing up your host system. The solution that will be developed here is to create a chroot environment on the host in which all code is "guest" code. Advantage of this solution:
- the chroot isolates the guest from the host
- no need for vnc, ssh or any other solution to export screens.
- no booting of a guest OS system.
- guest "immersion" for user programs is almost complete. Only very low level and hardware related programming will notice the difference.
- a spectacular speed improvement for graphics (X) related tasks (running lazarus f.e.)
- hard disk resources are common to host and guest. Host can read and modify "guest" files but not the other way around.
Setting up the system.
For a better comprehension of what follows, here is the directory structure used in this how-to:
home qemu arm-linux-user sparc-linux-user scripts disks // here are qemu images for the different systems emulated chroot mount arm // here we'll chroot the arm environment sparc
Get and build the latest qemu
$ cd $ git clone git://git.savannah.nongnu.org/qemu.git qemu $ cd qemu
Build all emulations (full system and user emulation mode, all processors)
$ ./configure --static to build
or only the user emulations needed
$ ./configure --static --target-list=sparc-linux-user,arm-linux-user
$ make clean all
The --static option makes that qemu-arm and qemu-sparc are linked statically. Since they will be running in the chroot environment later on, they must be build without dependencies. Some distributions provide already statically linked binaries. fe qemu-arm-static on debian. Although they are not the latest and greatest, using these packages makes it somewhat easier. Building QEMU from source is really just a matter of minutes.
Now we need to copy the complete guest system into chroot/arm or chroot/sparc. There are several ways of doing this but when you don't have a real arm or sparc system at hand, using a QEMU image of a full system emulation as described in previous chapters is the easiest.
Mount the QEMU image.
WARNING: do not mount a qemu image when the associated guest is running!
Only raw QEMU images can be mounted as a loop-back device. If you have a qcow formatted image, convert it to a raw image.
$ qemu-img convert disks/armel_hda.img -O raw disks/armel_hda.raw
$ cd chroot $ su # mount ../disks/armel_hda.raw mount -o loop,offset=32256
The offset is the start of the first partition on the ARM disk when the default disk layout was used during installation. If mount fails then the offset is probably wrong. To calculate the offset:
# fdisk -lu ../disks/armel_hda.raw
Find the first sector of the partition that will be mounted as root. Calculate offset as (start sector number * 512). For the default SPARC debian etch installation using a sun partition table with a /boot first partition the offset is 98703360.
Check if the correct partition is mounted:
# ls mount
Copy the complete system
# cp -r mount/* arm/ # umount mount
Copy the QEMU arm emulator to the arm directory
# cd arm # cp ../../arm-linux-user/qemu-arm usr/local/bin
Our "guest" system is complete. Since we are sharing our "host" devices and system, we need to mount a few special directories:
# mount --bind /proc proc # mount --bind /dev dev # mount --bind /sys sys # mount --bind /tmp tmp
Why /tmp? The /tmp directory is used by the X window system. When you don't mount tmp on the host /tmp, you will get "can't open display 0:0" errors when trying to open a window in the guest.
We are going to use the host network:
# sudo cp /etc/resolv.conf etc
Remains one last thing: the emulator is copied into the guest but how do we tell the system that the emulator needs to kick in when a "quest" elf is detected? This is done with binfmt_misc. A configuration script is included with QEMU that registers the QEMU user mode emulators with binfmt_misc: (on my system I had to do a chmod +x on the script)
When you look at this script at the end of the binary data you'll notice that the emulator is registered as /usr/local/bin/qemu-arm. And that is where we put it a few lines ago.
Launch the system:
# chroot .
If all goes well:
# uname -m armv7l
Everything you do from here is in your "guest" system. To exit the "guest" system.
And unmount the special directories.
# umount proc # umount dev # umount sys # umount tmp
The ARM emulation works very well, except for gdb. On the contrary there are still quite some problems with the SPARC emulation. There is a bug in duplicating pipes which makes the use of bash impossible. The sh shell works but several unix commands don't function correctly. What works fine is fpc and lazarus! Programs can't be launched from inside lazarus. Because of the improved graphics speed when using the user emulation mode it is still worthwhile for using lazarus. Instead of starting a shell in chroot, you start directly lazarus:
$ sudo chroot . /your/lazarus/dir/lazarus
To run the program compiled in lazarus, you can launch it from another "host" terminal:
$ cd qemu/chroot/arm $ sudo chroot . /your/program/dir/project
Here also gdb is not working.
http://www.aurel32.net/info/debian_arm_qemu.php http://www.aurel32.net/info/debian_mips_qemu.php http://en.wikibooks.org/wiki/QEMU/Windows_XP http://user-mode-linux.sourceforge.net/ http://www.josefsipek.net/docs/s390-linux/hercules-s390.html
See also Native ARM Systems and Native MIPS Systems.