Tucker's Tech

Virtualization links on del.icio.us

Nice set of links here (courtesy of Ken Robson), covering VMware, Xen, Microsoft, hardware issues, Solaris Zones, Linux VServer, grid computing, and some of the academic work in the area.  But he spelled "virtualization" wrong :-).

A new source browser is born

Back when I was at Sun working on OpenSolaris, I decided that we needed a good web-based source browser to help show off the "product" (which was, essentially, the source code itself).  At the time, we had a pilot site using CVSweb, which was easy to set up but (IMO) incredibly painful to use.  In particular, it didn't have any support for cross-reference links (the ability to jump to the definition of a function or variable by clicking on the name in the source code), and the search capabilities were pretty minimal.  I wanted something more usable for studying the source and general development activities; i.e., something that I (a diehard cscope user) would actually find useful in my own development work.

At the time, the state of the art was LXR, the Linux Cross-Reference system.  LXR provides cross-reference support, as well as freetext searches (via Glimpse), but the setup was somewhat cumbersome, support for revision history seemed awkward, and there were licensing issues with Glimpse.  Fortunately, it turned out that a Sun engineer was already playing around with web-based source browsers.  We talked with Chandan about our requirements, and he agreed to work on a new source browser.  The result was OpenGrok.

OpenGrok has actually been usable for a while as the source browser for OpenSolaris, but the source code was just released a couple of days ago.  This means people can now download it and use it for whatever source tree they're interested in (including proprietary code).  I did this recently with some internal VMware code, and (aside from a few minor issues that Chandan's working on fixing) it was remarkably easy.  OpenGrok is all Java based and runs on top of Tomcat, so once you have a recent version of Java (1.5) and Tomcat set up, it's pretty simple (even for a Tomcat novice like me).

The result is a source browser that not only has the cross-reference links and freetext search capabilities of LXR, but supports syntax highlighting and has a nice clean look and feel.  It's also completely free (in both senses) - released under the CDDL license.  (I suppose some people may have problems with the Java dependency, but at least that's free-as-in-beer for everyone.)  And, somewhat surprisingly given the fact that it's all Java, it's fast.  I admit I was skeptical when I first heard that Chandan was planning to use a Java search engine (Apache Lucene), but the results are impressive.  I'd suggest that anyone who spends much time looking at complex source trees check this out.

VMware and OpenSolaris

As a former Solaris engineer, I'm interested in helping people use Solaris and OpenSolaris with VMware.  VMware and Sun have announced that full support for Solaris will be added in future versions of VMware products.  In the meantime, the existing products can be made to work, but there are a few issues.  Here's some of the ones I know of (with workarounds).  Note: these aren't official recommendations from VMware (except where I've linked to vmware.com docs); they've worked for me, but your mileage may vary.

There are a couple of issues with virtual LSI Logic SCSI disks. Solaris 8 and 9 (GA) didn't come with a driver for the LSI Logic disk; if you try to install you'll get an error message saying something about "divide by zero error".  The fix is to either download the driver from the LSI website and install it as an ITU (details available with the driver), or upgrade to a recent update of Solaris 9 (which includes the driver) or to Solaris 10 or later.  More details are available here.

In addition, the LSI (actually "mpt") driver in versions of Solaris later than Solaris 10 (i.e., Solaris Express or OpenSolaris) contains some changes that don't work with Workstation 5.0.  You can either install on a virtual IDE drive (select the "custom" option when creating the VM), or use the beta of Workstation 5.5 (or the new VMware Player) instead. Setting up the X server can be a bit of a challenge.

The Xsun server available in Solaris 9 is actually pretty straightforward: kdmconfig will autoselect the VMware driver and a 1024x768 screen.  To change the screen size, run kdmconfig (as root) and pick a new size.  The Xorg server used by default in Solaris 10 and later is a bit more of a problem - it comes up OK, but the default screen size is pretty small (800x600). To change that, you can either switch back to the Xsun server by running kdmconfig, or run /usr/X11/bin/xorgconfig to reconfigure the Xorg server.  Some tips from other folks who have gone the xorgconfig route are available here and here.

Hello again

I'm a Principal Engineer at VMware , and am part of the team working on the ESX Server  product.  We're pretty excited about the recent announcement  of ESX 3, which has been in development for quite a while.  I actually can't take much credit since I only came in on the tail end of the ESX 3 development - up until last May I was at Sun, working on Solaris and OpenSolaris (see my Sun blog for more details).

Recently, I've been working on Distributed Resource Scheduling (DRS), which will be available with ESX 3.  The basic idea behind DRS is to provide the ability to automatically schedule VMs across a cluster of machines, in much the same way that an operating system schedules processes on different CPUs.  In addition to determining where VMs should run when initially powered on, DRS uses hot migration (aka VMotion) of VMs between hosts to adapt to dynamic changes in load or available resources.

The thing I find interesting about DRS is the way it decouples the application infrastructure (including the guest operating system itself) from the physical hardware.  If you want to take a machine down for maintenance, you can put it in maintenance mode and the VMs it was running will automatically migrate to other systems.  Once you're done, power it back on and VMs will migrate back.  If you have a spike in the load on a particular application, the scheduler can compensate by moving that VM to a host with more available resources. If you decide that a particular VM is more important than you initially thought and want to give it more resources, you can change the resource settings on the fly and the scheduler will adjust.

Of course, operating systems have been doing this sort of thing for years.  When I run a multithreaded application on an SMP I don't need to worry about how the threads are scheduled onto processors - the OS takes care of that.  And various kinds of batch and grid schedulers have been able to do initial placement scheduling - deciding what machine a new job should run on.  But being able adapt on the fly to changes in load and available resources - migrating workloads between independent machines - without application or OS changes - using commodity (i.e. cheap) hardware - in an enterprise-class product - that's something different.