1.1 --- a/INSTALL Sun Sep 13 19:03:03 2009 -0700
1.2 +++ b/INSTALL Sun Sep 13 20:19:56 2009 -0700
1.3 @@ -1,29 +1,43 @@
1.4 -Dependencies:
1.5 - Event::Lib >= 1.0 (tested 1.03) - http://search.cpan.org/~vparseval/Event-Lib/
1.6 +1. Install the dependencies.
1.7 +
1.8 + QEMU >= 0.9.1, or KVM >= 75 (tested kvm-75)
1.9 + http://www.qemu.org
1.10 + http://www.linux-kvm.org
1.11 + These features are specifically used by Qemudo: qcow2, snapshots, and
1.12 + CPU selection support
1.13 +
1.14 + Event::Lib >= 1.0 (tested 1.03)
1.15 + http://search.cpan.org/~vparseval/Event-Lib/
1.16 Any distro: $ env FTP_PASSIVE=1 cpan Event::Lib
1.17 - OSSP::uuid (tested 1.3.2 and 1.5.1) - http://www.ossp.org/pkg/lib/uuid/
1.18 +
1.19 + OSSP::uuid (tested 1.3.2 and 1.5.1)
1.20 + http://www.ossp.org/pkg/lib/uuid/
1.21 Debian/Ubuntu: $ apt-get install libossp-uuid-perl
1.22 - JSON >= 1.0 including 2.x (tested 1.0) - http://search.cpan.org/~makamaka/JSON/
1.23 +
1.24 + JSON >= 1.0 (tested 1.0, 2.x should work too)
1.25 + http://search.cpan.org/~makamaka/JSON/
1.26 Debian/Ubuntu: $ apt-get install libjson-perl
1.27 - Qemu 0.9.0 (qcow2 + snapshots) 0.9.1 (cpu selection support)
1.28
1.29 -Shipped with Qemudo:
1.30 - SCGI >= 0.4 (tested 0.6) - http://search.cpan.org/~vipercode/SCGI/
1.31 - http://downloads.sourceforge.net/vnc-tight/tightvnc-1.3.9_javabin.zip
1.32 +FYI these dependencies are shipped with Qemudo:
1.33
1.34 -You will need a webserver supporting SCGI, for example lighttpd, apache, etc.
1.35 + SCGI >= 0.4 (Qemudo includes 0.6)
1.36 + http://search.cpan.org/~vipercode/SCGI/
1.37
1.38 -Lighttpd: add the following to your config file:
1.39 ----begin---
1.40 -server.modules += ( "mod_scgi" )
1.41 -scgi.server = ( "/q/" =>
1.42 - ((
1.43 - "host" => "127.0.0.1",
1.44 - "port" => 29029,
1.45 - "check-local" => "disable",
1.46 - ))
1.47 -)
1.48 ----end---
1.49 + TightVNC Java Applet >= 1.3 (Qemudo includes 1.3.9):
1.50 + http://downloads.sourceforge.net/vnc-tight/tightvnc-1.3.9_javabin.zip
1.51 +
1.52 +
1.53 +2. Configure a web server supporting SCGI.
1.54 +
1.55 +You may use any server supporting SCGI (Apache, Lighttpd, etc) and configure
1.56 +it to redirect HTTP requests for http://host/qemudo/... to Qemudo which runs
1.57 +the SCGI listener on port 29029/tcp by default.
1.58 +
1.59 +Example config for lighttpd:
1.60 + server.modules += ( "mod_scgi" )
1.61 + scgi.server = ( "/qemudo/" =>
1.62 + (( "host" => "127.0.0.1", "port" => 29029, "check-local" => "disable" ))
1.63 + )
1.64
1.65 Other than the standard SCGI variables (SCGI, CONTENT_LENGTH), Qemudo relies
1.66 on the following ones:
1.67 @@ -33,256 +47,34 @@
1.68 REMOTE_USER
1.69
1.70
1.71 +3. Copy the static files to the web server.
1.72
1.73 -*** WARNING ***
1.74 -Despite being easy to use, you will need advanced Unix skills to
1.75 -properly set up Qemudo and make full use of its features. Before
1.76 -attempting to install it, make sure you know how to: edit config files
1.77 -or simple Perl data structures by hand, install and enable CGI support
1.78 -in a web server, install sshd, use ssh-keygen(1), write an
1.79 -.ssh/authorized_keys file, create disk images with qemu-img, etc. In
1.80 -addition you may need to: export and mount read-only NFS shares,
1.81 -create user accounts dedicated to Qemudo, use screen(1), edit init
1.82 -scripts to run things at boot time, etc. Qemudo is a powerful but
1.83 -complex piece of software. You have been warned !
1.84 -*** WARNING ***
1.85 +The content of the "static-qemudo" repository must be surver by the
1.86 +HTTP server as http://host/static-qemudo/...
1.87
1.88 +Example if your server root is /var/www:
1.89 + $ cp -R static-qemudo /var/www
1.90
1.91 -1. Understanding Qemudo's Architecture
1.92
1.93 -Qemudo consits of two distinct parts:
1.94 -- A web interface (qemudo.cgi) that can run under any CGI-enabled
1.95 - web server.
1.96 -- A daemon (qd - Qemudo daemon) and its CLI tool (qsh - Qemudo shell)
1.97 - that have to be installed on each host that is going to run VMs.
1.98 -In order to understand how to install Qemudo and how to best use the
1.99 -flexibility it offers, here is an example of two different possible
1.100 -use cases:
1.101 +4. Launch the Qemudo daemon
1.102
1.103 - o As a workstation user, you are accustomed to running QEMU in its
1.104 - traditional graphical mode. You may decide to switch to Qemudo. You
1.105 - run both the Qemudo web interface and qd+qsh on your workstation, and
1.106 - use your VMs through a web browser connected to 127.0.0.1. This allows
1.107 - you to have your VMs running in different tabs in a single browser
1.108 - window. You can also log off your X11 session, re-log in, possibly as
1.109 - a different user, reopen a browser, and find your VMs back in the
1.110 - exact same state they were when you left (qd keeps them up and
1.111 - running). You can also decide to make your web server accessible to
1.112 - other people on the network, in order to give them access to your VMs.
1.113 +The Qemudo daemon acts as an SCGI listener and as a daemon controlling
1.114 +VMs running on the host system. The simplest way to configure and launch it
1.115 +is to do it from the Qemudo source tree itself.
1.116
1.117 - o As a datacenter administrator, you have 2, 10 or 20 servers at your
1.118 - disposition and need to run dozens of VMs per servers. Your users need
1.119 - a simple interface (say a browser) that lists all of the available VMs
1.120 - and lets them access any VM in a transparent way by completely
1.121 - abstracting and hiding the fact that they are actually running on a
1.122 - cluster of servers. You install the Qemudo web interface on a web
1.123 - server (possibly one of the servers that will also run QEMU), and
1.124 - qd+qsh on every server in the cluster.
1.125 +Create a config file for qd (Qemudo daemon), for example node/qd.conf
1.126 +(you might want to look at node/qd.conf.example). The 2 most important
1.127 +settings that probably need to be modified are the path to the directory
1.128 +containg VMs (repo), and the name of the command to launch QEMU/UVM
1.129 +(path_qemu). So this 2-line config file may be sufficient:
1.130 + repo = /path/to/repo (default is /srv/qemudo)
1.131 + path_qemu = qemu-system-x86_64 (default is qemu)
1.132
1.133 -Whatever is your need, it will determine how and where you will
1.134 -install qemudo.cgi and qd+qsh.
1.135 +Create the repo:
1.136 + $ mkdir /path/to/repo
1.137
1.138 -2. Installing qd (and qsh) on the Nodes
1.139 +Then launch qd from the root of Qemudo's source tree. By default
1.140 +it runs in the foreground and logs info to the standard output:
1.141 + $ env PERLLIB=`pwd`/lib ./node/qd --conf node/qd.conf
1.142
1.143 -Do the following on each Qemudo node that will run qd+qsh.
1.144 -
1.145 -Step 1: Install QEMU 0.8.1 or later
1.146 -
1.147 - Make sure QEMU is properly installed. Qemudo, or more precisely qd,
1.148 - needs to find "qemu" and "qemu-img" in the system PATH. QEMU version
1.149 - 0.8.1 or later is required because there is no VNC support in earlier
1.150 - versions.
1.151 -
1.152 - As of February 2007, Qemudo has been tested and is known to work well
1.153 - with QEMU versions 0.8.1 up to 0.9.0.
1.154 -
1.155 -Step 2: Install a web server listenning on port 80 and serving the TightVNC
1.156 -JAR file from the web root as "/VncViewer.jar"
1.157 -
1.158 - This web server only needs to serve this one static file. thttpd
1.159 - will do just fine. The TightVNC JAR file (VNC applet) can be obtained
1.160 - from <http://www.tightvnc.com/download.html> by downloading the "Java
1.161 - (viewer only)" version of TightVNC (file named
1.162 - tightvnc-*_javabin.zip). Unzip the file, you should find the
1.163 - VncViewer.jar file in the "classes" directory. Both the development
1.164 - and stable versions of TightVNC should work.
1.165 -
1.166 - The VNC applet must be served by each Qemudo node because it is
1.167 - downloaded by the users' browsers and the default Java security policy
1.168 - only allows applets to connect back to their originating host
1.169 - (needed to establish the VNC connection).
1.170 -
1.171 -Step 3: Get the node/ directory from the Qemudo tarball
1.172 -
1.173 - Extract the Qemudo tarball somewhere. Only the content of the node/
1.174 - directory is required to be present on a node (since it contains qd
1.175 - and qsh). You may delete all other extracted files, and copy the node/
1.176 - directory somewhere (the location doesn't matter).
1.177 -
1.178 - Note: If you are using QEMU 0.8.x you need to edit node/qd and change
1.179 - this line:
1.180 - exec qemu -monitor stdio -vnc :\$((\$QEMUDO_VNC_PORT - 5900)) \\
1.181 - to:
1.182 - exec qemu -monitor stdio -vnc \$((\$QEMUDO_VNC_PORT - 5900)) \\
1.183 - The colon character (:) after -vnc needs to be removed because
1.184 - QEMU versions before 0.9.0 used to require this slightly different
1.185 - syntax.
1.186 -
1.187 -Step 4: In node/, rename qd.conf.example to qd.conf and edit the options
1.188 -
1.189 - "dir" must be a path to an initially empty directory. The VMs created
1.190 - via Qemudo will result in actual files being created in this
1.191 - directory. Example: /space/qemudo/vm
1.192 -
1.193 - "dir_templates" must be a path to a directory containing QEMU disk
1.194 - image files that Qemudo will consider as being "system templates"
1.195 - (read-only "base" images). Any dot-file or subdirectory in
1.196 - "dir_templates" will be ignored. The idea behind these system templates
1.197 - is that you, as an end-user, create a set of fully functionning VMs
1.198 - and move their disk images to the "dir_templates" directory. Then,
1.199 - from the Qemudo web interface, you won't directly boot up a template
1.200 - but instead you will have to create a new VM by specifying the
1.201 - template it will be based off. At this point Qemudo will internally
1.202 - create a new copy-on-write qcow image based on the template you
1.203 - specified (using qemu-img -b). This powerful feature lets you create
1.204 - very quickly, say, 10 independent VMs based on the same template while
1.205 - using a fraction of the disk space it would normally require. It is
1.206 - important to understand that Qemudo *never* writes to or modifies
1.207 - files in "dir_templates". Qemudo only needs read-only access to it.
1.208 - And this directory must present the same content to all Qemudo nodes
1.209 - and to the web server, so you might want to make it a read-only NFS
1.210 - share. Example: /space/qemudo/template
1.211 -
1.212 - "log" must be a path to a log file. When running in the foreground,
1.213 - qd will log to stderr and ignore the "log" option. Example:
1.214 - /space/qemudo/qd.log
1.215 -
1.216 - "socket" is a path to a unix socket file created by qd on startup. The
1.217 - CLI tool qsh communicates with qd via this socket. Example: /var/run/qd
1.218 -
1.219 - "vncports" must define a range of VNC ports that will be assigned by
1.220 - qd to the QEMU instances. Example: 5900-5999
1.221 -
1.222 -Step 5: Run qd ("qd --conf qd.conf")
1.223 -
1.224 - You may run qd directly within the node/ directory, or copy it to your
1.225 - PATH. Run qd as a user that is allowed to create the Unix socket file
1.226 - (specified by the "socket" config option). This will make it run in
1.227 - the foreground:
1.228 -
1.229 - $ ./qd --conf qd.conf
1.230 -
1.231 - Alternatively you can run it in the background as a daemon:
1.232 -
1.233 - $ ./qd --conf qd.conf --detach
1.234 -
1.235 - You might want to run it within a screen(1) session, or start it up as
1.236 - a daemon from an initscript.
1.237 -
1.238 -Step 6: Place qsh in the system PATH and test it ("qsh --socket /var/run/qd")
1.239 -
1.240 - The qsh script (found in the node/ directory) needs to be placed in
1.241 - the system PATH (or anywhere else where the user created in step 7 can
1.242 - find it in his own PATH).
1.243 -
1.244 - Make sure that qsh can communicate with the running qd instance via
1.245 - the Unix socket (by default qsh tries to connect to "/var/run/qd"),
1.246 - and that everything appears to work ("version" should report the
1.247 - Qemudo version). Control-D, or "exit" exits qsh.
1.248 -
1.249 - $ qsh --socket /var/run/qd
1.250 - qsh$ version
1.251 - 100 qd version x.y.z
1.252 - qsh$ exit
1.253 -
1.254 -Step 7: Install sshd
1.255 -
1.256 - qsh is the CLI tool internally used by qemudo.cgi (web interface) to
1.257 - communicate with Qemudo nodes. qsh has also been designed to be
1.258 - usable by humans (see the "help" command in qsh for documentation).
1.259 - qemudo.cgi remotely invokes qsh over SSH like this:
1.260 -
1.261 - $ ssh -p PORT USER@NODE "echo COMMAND | qsh"
1.262 -
1.263 - This is similar to the way CVS is used over SSH connections. A current
1.264 - inconvenient in Qemudo is that the usage of SSH is always required,
1.265 - even when it shouldn't be necessary (e.g. when qemudo.cgi is running
1.266 - on the same host running qd). So even in this case you need to have
1.267 - sshd listenning on 127.0.0.1.
1.268 -
1.269 - Install sshd on the node. Also, you might want to create a dedicated
1.270 - user account that qemudo.cgi will use when connecting via SSH. Make
1.271 - sure this user can find "qsh" in his PATH (as explained in step 6).
1.272 -
1.273 -3. Installing the Qemudo Web Interface
1.274 -
1.275 -On the machine that will act as the main web server for the Qemudo
1.276 -interface, follow these steps.
1.277 -
1.278 -Step 1: Install a web server able to execute Perl CGI scripts.
1.279 -
1.280 - As for the nodes, thttpd will also do just fine.
1.281 -
1.282 -Step 2: Extract the Qemudo tarball, place web/ under the web root
1.283 -
1.284 - Extract the Qemudo tarball somewhere. At this point, only the content
1.285 - of the web/ subdirectory is actually required for the web server.
1.286 - Copy the whole web/ directory somewhere under the web root, example:
1.287 -
1.288 - $ cp -R web/ /var/www/qemudo
1.289 -
1.290 -Step 3: Install the Perl module CGI::Request
1.291 -
1.292 - Qemudo depends on the Perl module CGI::Request. Install it. The
1.293 - Debian or Ubuntu package name for this module is libcgi-perl:
1.294 -
1.295 - http://packages.debian.org/unstable/perl/libcgi-perl
1.296 -
1.297 - The above webpage contains a link to the original tarball:
1.298 -
1.299 - http://ftp.debian.org/debian/pool/main/libc/libcgi-perl/
1.300 - libcgi-perl_2.76.orig.tar.gz
1.301 -
1.302 - It seems this Perl module is not actively maintained anymore. Future
1.303 - Qemudo versions will probably abandon CGI::Request.
1.304 -
1.305 -Step 4: In the web root, rename qemudo.conf.example to qemudo.conf and edit
1.306 -the options
1.307 -
1.308 - "dir_templates" must be a path to a directory containing the same
1.309 - QEMU disk images than the "dir_templates" directories defined in the
1.310 - nodes' qd.conf. As explained before, a good idea is to make it a
1.311 - read-only NFS share mounted on every node and on the web server.
1.312 - Example: /space/qemudo/template
1.313 -
1.314 -Step 5: Edit qemudo.cgi, define the nodes in @nodes
1.315 -
1.316 - Edit qemudo.cgi and change the definition of the variable @nodes near
1.317 - the beginning of the Perl file to reflect the Qemudo nodes in your
1.318 - cluster. As explained before, qemudo.cgi will run ssh(1) to to
1.319 - connect to the nodes and remotely invoke qsh. So in addition to the
1.320 - hostname you also need to specify the TCP port sshd(8) is listenning
1.321 - on, usually 22, and the username to connect with via ssh.
1.322 -
1.323 -Step 6: Set up ssh keys for qemudo.cgi
1.324 -
1.325 - Create ssh keys so that the userid that CGI scripts are running as
1.326 - (www-data, www...) can ssh into every node with no password.
1.327 - In other words, this must not ask you for a passwd:
1.328 -
1.329 - $ su www-data "ssh -p PORT USER@NODE uname -a"
1.330 -
1.331 - Also, verify that qsh can be invoked remotely on every node defined in
1.332 - @nodes:
1.333 -
1.334 - $ su www-data \
1.335 - "ssh -p PORT USER@NODE 'echo version | qsh --socket /var/run/qd'"
1.336 -
1.337 - The decision to have the CGI directly run ssh to communicate with the
1.338 - nodes has been taken purely to quickly get a working prototype of
1.339 - Qemudo. It brings many disadvantages: high latency to process web
1.340 - request, unconventional usage of ssh keys, configuration overhead in
1.341 - case the web server is running in a chroot, etc. Future versions of
1.342 - Qemudo will be improved so the CGI won't need to run ssh.
1.343 -
1.344 -Qemudo is now properly installed. See `USAGE' for how to use it.
1.345 +Qemudo is now properly installed and running. See `USAGE' for how to use it.
2.1 --- a/NEWS Sun Sep 13 19:03:03 2009 -0700
2.2 +++ b/NEWS Sun Sep 13 20:19:56 2009 -0700
2.3 @@ -1,4 +1,4 @@
2.4 -Qemudo 0.2.0 (?)
2.5 +Qemudo 0.2.x (?)
2.6
2.7 o Complete rewrite, based on a fully distributed architecture, using a
2.8 unicast/multicast UDP synchronization protocol, SCGI as a web
3.1 --- a/README Sun Sep 13 19:03:03 2009 -0700
3.2 +++ b/README Sun Sep 13 20:19:56 2009 -0700
3.3 @@ -1,8 +1,8 @@
3.4 qemudo - QEMU Web Interface
3.5
3.6 -Qemudo is a Web interface to QEMU [1] offering a way for users to access
3.7 -and control multiple virtual machines (guest systems) running on one
3.8 -or more remote physical machines (host systems).
3.9 +Qemudo is a Web interface to QEMU [1] or KVM [2] offering a way for
3.10 +users to access and control multiple virtual machines (guest systems)
3.11 +running on one or more remote physical machines (host systems).
3.12
3.13 From the Web interface, VMs can be created, started up, shut down,
3.14 reset, and accessed (monitor, keyboard and mouse). Access to the VMs
3.15 @@ -14,7 +14,9 @@
3.16 VMs running on a cluster of physical machines. Of course it can also
3.17 be used just as easily to manage only a couple of VMs running locally.
3.18
3.19 -[1] http://fabrice.bellard.free.fr/qemu
3.20 +[1] http://www.qemu.org
3.21 +[2] http://www.linux-kvm.org
3.22 +
3.23
3.24 See `INSTALL' for installation instructions.
3.25 See `USAGE' for documentation.
4.1 --- a/USAGE Sun Sep 13 19:03:03 2009 -0700
4.2 +++ b/USAGE Sun Sep 13 20:19:56 2009 -0700
4.3 @@ -1,20 +1,6 @@
4.4 -1. Using Qemudo
4.5 +After installing Qemudo as described in INSTALL, browse this URL:
4.6
4.7 -$ cd /path/to/qemudo-x.x.x
4.8 -$ export PERLLIB=`pwd`/lib
4.9 -$ mkdir /srv/qemudo # default location for the disk image repository
4.10 -$ ./node/qd # runs Qemudo in the foreground
4.11 -Fri May 23 00:19:06 2008 -- starting ./node/qd (pid 7122)
4.12 -
4.13 -qd will listen for incoming SCGI requests on port 29029/tcp by default.
4.14 -Configure a webserver to redirect requests for /qemudo to this SCGI listener.
4.15 -
4.16 -Also configure the webserver to serve the "static-files" directory as
4.17 -http://<host>/static-files
4.18 -
4.19 -Then browse http://<host>/qemudo
4.20 -
4.21 -2. Browser compatibility
4.22 + http://host/qemudo
4.23
4.24 The Qemudo Web UI makes use of W3C-standardized technologies such as
4.25 XMLHttpRequest (technically still a draft), CSS2, DOM Level 1, etc.
4.26 @@ -29,28 +15,4 @@
4.27 - Opera 9.x
4.28 - Konqueror 3.5.x
4.29
4.30 -3. Bridging the VMs to the local network
4.31 -
4.32 -Users often want to bridge the VMs to the local ethernet interface
4.33 -to make them appear as being independent machines on the local network.
4.34 -Even though this topic is not directly related to Qemudo, here is the
4.35 -author's suggestion on how to accomplish this on a Debian-based Linux
4.36 -host system. First, a bridge containing the host's physical interface
4.37 -(say eth0) needs to be defined in /etc/network/interfaces:
4.38 -
4.39 - auto br0
4.40 - iface br0 inet dhcp
4.41 - bridge_ports eth0
4.42 -
4.43 -Qemudo always passes the "-net tap" option to QEMU to create a TAP interface
4.44 -so all that remains to be done is to write a /etc/qemu-ifup script that
4.45 -will be executed by QEMU when a VM is started (the TAP interface name is
4.46 -passed as an argument to the script, as documented in qemu(1)):
4.47 -
4.48 - #!/bin/sh
4.49 - ifconfig "$1" up && brctl addif br0 "$1"
4.50 -
4.51 -Now, each time a VM is started up in Qemudo, its TAP interface will
4.52 -be automatically placed in the bridge br0.
4.53 -
4.54 Enjoy !