beer planet

The DailyMail reports on some crazy ocean foaming in Sydney. What exactly are they talking about? Perhaps the pictures will explain better (or not).

At the recent Seattle Conference on Scalability organized by Google, Cuong Do, an engineering manager at YouTube, talks about YouTube's growth over the past 2 years and the scalability problems they have overcome. All in all, it's a very interesting presentation that I can recommend to anyone remotely interested in large-scale projects, such as YouTube.

One interesting fact that Cuong mentions is that the pre-Google YouTube tech team consisted of only 2 sysadmins, 2 scalability software architects, 2 developers, 2 networks engineers, and 1 DBA. It's quite impressive that such a small team managed to maintain and scale such a widely popular service.

Here's the presentation (52min long):

Updated: November 9th, 2007

In this article, I will show how to resize a VMware disk if you didn't make it large enough when it was created. Furthermore, I will show how to resize partitions following the disk augmentation using OpenSUSE 10.2 as an example. You will even be able to resize the Linux root partition (/) that is mounted and is normally unmountable.

The story: this is simple - you made a VMware disk without thinking ahead and now it ran out of space. Here you have a few options, the most notable ones being create another disk and mount it or grow the existing disk and resize the existing partitions. The first option is trivial, so let's explore the 2nd one.

1. Backup the existing VMware disk file by copying it somewhere safe.

2. Grow the VMware disk file (note that I'm using Windows in this step because my host system is Windows XP). In order to do this, use the 'vmware-vdiskmanager.exe' utility that comes bundled with VMware and most likely sits in C:\Program Files\VMware\VMware Workstation. If the target size of the new disk is 10GB, issue the following command:

vmware-vdiskmanager.exe -x 10Gb "YOUR_DISK.vmdk"

The disk will crunch for a bit and hopefully happiy finish with Grow: 100% done. Now the maximum disk space has been increased to 10GB. At this point, all partitions remain the same size, with a bunch of unpartitioned free space added at the end of the disk.

3. Now you can boot the guest OS (OpenSUSE 10.2 in my example) and use some magical utility to resize your partitions. In YaST, for instance, you would go to System->Partitioner. You can resize pretty much any partition that can be unmounted (unmount it beforehand and never try to repartition a mounted one). That represents a problem if you, like me, want to grow the root partition /. Still with me? Then let's go to the next step.

4.

a) In order to repartition or resize a root partition, I will use a LiveCD of another OS. Or even better, a LiveCD that lives and breathes resizing things left and right. I'm talking about the GParted Live CD from http://gparted.sourceforge.net/livecd.php. Since a LiveCD is an operating system on its own loaded into RAM, it won't need to mount any of your existing partitions. There's a bug with the latest version 0.3.4-8, which prevents GParted from seeing any partitions in VMware systems at all, but 0.3.4-5 works like a charm. Download the ISO image, right click on the CD/DVD icon in the lower right part of the VMware window and click Edit. Then change the Connection to "Use ISO image" and browse to the freshly downloaded GParted ISO. This will effectively mount the image without having to burn it onto a CD.

b) Reboot the guest OS. Quickly press F2 on the VMware preboot load screen to enter the virtual VMware BIOS. Go to the Boot tab and bring the CD-ROM drive to the top by scrolling to it and clicking + repeatedly. Now save and exit. The next time the OS tries to boot, it will first try to load the LiveCD, which is exactly what we want.

c) GParted should load after a few minutes where everything should be very straightforward. The screenshots here may help understand what I'm talking about. Fiddle with the settings, increase the root partition size, click Apply, and go have a few beers because this may take a while to finish. When it does, so are we, as we now have successfully grown the root partition.

d) Optionally revert the steps in part 4a and 4b.

5. Reboot into the guest OS again. Enjoy all the new extra space.

This concludes the article. Feel free to leave any questions or comments.

Updated: November 9th, 2007

How many times did I want to watch a directory waiting for a file to appear in it? Constant ls, for example, quickly got boring. A quick Perl script that would reload ls every 5 seconds… yeah it works but it takes a while to type up, and often enough I'm too lazy for that. And then I found 'watch' - a utility that comes with most *nix distros. Look at this beauty [man watch]:

NAME
watch - execute a program periodically, showing output fullscreen

SYNOPSIS
watch [-dhvt] [-n ] [--differences[=cumulative]] [--help]
[--interval=] [--no-title] [--version]

DESCRIPTION
watch runs command repeatedly, displaying its output (the first screen
full). This allows you to watch the program output change over time.
By default, the program is run every 2 seconds; use -n or –interval to
specify a different interval.

The -d or –differences flag will highlight the differences between
successive updates. The –cumulative option makes highlighting
"sticky", presenting a running display of all positions that have ever
changed. The -t or –no-title option turns off the header showing the
interval, command, and current time at the top of the display, as well
as the following blank line.

watch will run until interrupted.

NOTE
Note that command is given to "sh -c" which means that you may need to
use extra quoting to get the desired effect.

Note that POSIX option processing is used (i.e., option processing
stops at the first non-option argument). This means that flags after
command don't get interpreted by watch itself.

EXAMPLES
To watch for mail, you might do

watch -n 60 from

To watch the contents of a directory change, you could use

watch -d ls -l

If you're only interested in files owned by user joe, you might use

watch -d 'ls -l | fgrep joe'

To see the effects of quoting, try these out

watch echo $$
watch echo '$$'
watch echo "'"'$$'"'"

You can watch for your administrator to install the latest kernel with

watch uname -r

(Just kidding.)

Now if anyone knows how to make watch print the bottom part of the command output rather than the top, please post a comment.

Edit: BSD users, "watch" out. watch does something completely different on BSD (it lets you snoop on other users' ttys). The corresponding BSD command is actually cmdwatch.

If you haven't seen The Bourne Ultimatum yet, go see it. It's a great finale of the Bourne series. Matt Damon really delivers. In fact, I'm starting to feel obsessed with this movie.

Here's the clip of a shared scene between the 2 movies (this one is from The Bourne Supremacy). The song at the end is still breathtaking: