beer planet

At the recent Seattle Conference on Scalability organized by Google, Cuong Do, an engineering manager at YouTube, talks about YouTube's growth over the past 2 years and the scalability problems they have overcome. All in all, it's a very interesting presentation that I can recommend to anyone remotely interested in large-scale projects, such as YouTube.

One interesting fact that Cuong mentions is that the pre-Google YouTube tech team consisted of only 2 sysadmins, 2 scalability software architects, 2 developers, 2 networks engineers, and 1 DBA. It's quite impressive that such a small team managed to maintain and scale such a widely popular service.

Here's the presentation (52min long):

Updated: November 9th, 2007

In this article, I will show how to resize a VMware disk if you didn't make it large enough when it was created. Furthermore, I will show how to resize partitions following the disk augmentation using OpenSUSE 10.2 as an example. You will even be able to resize the Linux root partition (/) that is mounted and is normally unmountable.

The story: this is simple - you made a VMware disk without thinking ahead and now it ran out of space. Here you have a few options, the most notable ones being create another disk and mount it or grow the existing disk and resize the existing partitions. The first option is trivial, so let's explore the 2nd one.

1. Backup the existing VMware disk file by copying it somewhere safe.

2. Grow the VMware disk file (note that I'm using Windows in this step because my host system is Windows XP). In order to do this, use the 'vmware-vdiskmanager.exe' utility that comes bundled with VMware and most likely sits in C:\Program Files\VMware\VMware Workstation. If the target size of the new disk is 10GB, issue the following command:

vmware-vdiskmanager.exe -x 10Gb "YOUR_DISK.vmdk"

The disk will crunch for a bit and hopefully happiy finish with Grow: 100% done. Now the maximum disk space has been increased to 10GB. At this point, all partitions remain the same size, with a bunch of unpartitioned free space added at the end of the disk.

3. Now you can boot the guest OS (OpenSUSE 10.2 in my example) and use some magical utility to resize your partitions. In YaST, for instance, you would go to System->Partitioner. You can resize pretty much any partition that can be unmounted (unmount it beforehand and never try to repartition a mounted one). That represents a problem if you, like me, want to grow the root partition /. Still with me? Then let's go to the next step.

4.

a) In order to repartition or resize a root partition, I will use a LiveCD of another OS. Or even better, a LiveCD that lives and breathes resizing things left and right. I'm talking about the GParted Live CD from http://gparted.sourceforge.net/livecd.php. Since a LiveCD is an operating system on its own loaded into RAM, it won't need to mount any of your existing partitions. There's a bug with the latest version 0.3.4-8, which prevents GParted from seeing any partitions in VMware systems at all, but 0.3.4-5 works like a charm. Download the ISO image, right click on the CD/DVD icon in the lower right part of the VMware window and click Edit. Then change the Connection to "Use ISO image" and browse to the freshly downloaded GParted ISO. This will effectively mount the image without having to burn it onto a CD.

b) Reboot the guest OS. Quickly press F2 on the VMware preboot load screen to enter the virtual VMware BIOS. Go to the Boot tab and bring the CD-ROM drive to the top by scrolling to it and clicking + repeatedly. Now save and exit. The next time the OS tries to boot, it will first try to load the LiveCD, which is exactly what we want.

c) GParted should load after a few minutes where everything should be very straightforward. The screenshots here may help understand what I'm talking about. Fiddle with the settings, increase the root partition size, click Apply, and go have a few beers because this may take a while to finish. When it does, so are we, as we now have successfully grown the root partition.

d) Optionally revert the steps in part 4a and 4b.

5. Reboot into the guest OS again. Enjoy all the new extra space.

This concludes the article. Feel free to leave any questions or comments.

Updated: November 9th, 2007

How many times did I want to watch a directory waiting for a file to appear in it? Constant ls, for example, quickly got boring. A quick Perl script that would reload ls every 5 seconds… yeah it works but it takes a while to type up, and often enough I'm too lazy for that. And then I found 'watch' - a utility that comes with most *nix distros. Look at this beauty [man watch]:

NAME
watch - execute a program periodically, showing output fullscreen

SYNOPSIS
watch [-dhvt] [-n ] [--differences[=cumulative]] [--help]
[--interval=] [--no-title] [--version]

DESCRIPTION
watch runs command repeatedly, displaying its output (the first screen
full). This allows you to watch the program output change over time.
By default, the program is run every 2 seconds; use -n or –interval to
specify a different interval.

The -d or –differences flag will highlight the differences between
successive updates. The –cumulative option makes highlighting
"sticky", presenting a running display of all positions that have ever
changed. The -t or –no-title option turns off the header showing the
interval, command, and current time at the top of the display, as well
as the following blank line.

watch will run until interrupted.

NOTE
Note that command is given to "sh -c" which means that you may need to
use extra quoting to get the desired effect.

Note that POSIX option processing is used (i.e., option processing
stops at the first non-option argument). This means that flags after
command don't get interpreted by watch itself.

EXAMPLES
To watch for mail, you might do

watch -n 60 from

To watch the contents of a directory change, you could use

watch -d ls -l

If you're only interested in files owned by user joe, you might use

watch -d 'ls -l | fgrep joe'

To see the effects of quoting, try these out

watch echo $$
watch echo '$$'
watch echo "'"'$$'"'"

You can watch for your administrator to install the latest kernel with

watch uname -r

(Just kidding.)

Now if anyone knows how to make watch print the bottom part of the command output rather than the top, please post a comment.

Edit: BSD users, "watch" out. watch does something completely different on BSD (it lets you snoop on other users' ttys). The corresponding BSD command is actually cmdwatch.

Updated: November 9th, 2007

So today I was looking for a proper sources.list file that would work for apt-get in SUSE 10.2 since the one in yast and smart repositories comes with a broken list. And by broken I mean completely f***ed.

apt-get update
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386 release
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Get:1 ftp://ftp4.gwdg.de SuSE/10.2-i386 release
Ign ftp://ftp4.gwdg.de SuSE/10.2-i386 release
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/base pkglist
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/base release
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/update pkglist
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Get:1 ftp://ftp4.gwdg.de SuSE/10.2-i386/base pkglist
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/update release
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Err ftp://ftp4.gwdg.de SuSE/10.2-i386/base pkglist
Unable to fetch file, server said 'Failed to open file.  '
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/security pkglist
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Get:2 ftp://ftp4.gwdg.de SuSE/10.2-i386/base release
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/security release
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Ign ftp://ftp4.gwdg.de SuSE/10.2-i386/base release
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/rpmkeys pkglist
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Get:3 ftp://ftp4.gwdg.de SuSE/10.2-i386/update pkglist
Err ftp://mirrors.mathematik.uni-bielefeld.de SuSE/10.2-i386/rpmkeys release
Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Err ftp://ftp4.gwdg.de SuSE/10.2-i386/update pkglist
Unable to fetch file, server said 'Failed to open file.  '
Get:4 ftp://ftp4.gwdg.de SuSE/10.2-i386/update release
Ign ftp://ftp4.gwdg.de SuSE/10.2-i386/update release
Get:5 ftp://ftp4.gwdg.de SuSE/10.2-i386/security pkglist
Err ftp://ftp4.gwdg.de SuSE/10.2-i386/security pkglist
Unable to fetch file, server said 'Failed to open file.  '
Get:6 ftp://ftp4.gwdg.de SuSE/10.2-i386/security release
Ign ftp://ftp4.gwdg.de SuSE/10.2-i386/security release
Get:7 ftp://ftp4.gwdg.de SuSE/10.2-i386/rpmkeys pkglist
Err ftp://ftp4.gwdg.de SuSE/10.2-i386/rpmkeys pkglist
Unable to fetch file, server said 'Failed to open file.  '
Get:8 ftp://ftp4.gwdg.de SuSE/10.2-i386/rpmkeys release
Ign ftp://ftp4.gwdg.de SuSE/10.2-i386/rpmkeys release
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/release  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.base  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/release.base  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.update  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/release.update  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.security  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/release.security  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.rpmkeys  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/SuSE/10.2-i386/base/release.rpmkeys  Could not resolve 'mirrors.mathematik.uni-bielefeld.de'
Failed to fetch ftp://ftp4.gwdg.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.base  Unable to fetch file, server said 'Failed to open file.  '
Failed to fetch ftp://ftp4.gwdg.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.update  Unable to fetch file, server said 'Failed to open file.  '
Failed to fetch ftp://ftp4.gwdg.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.security  Unable to fetch file, server said 'Failed to open file.  '
Failed to fetch ftp://ftp4.gwdg.de/pub/linux/suse/apt/SuSE/10.2-i386/base/pkglist.rpmkeys  Unable to fetch file, server said 'Failed to open file.  '

The easiest solution is to download and install the rpms from ftp://ftp5.gwdg.de/pub/opensuse/repositories/home%3A/rbos/openSUSE_10.2/i586/ instead, which come with the correct sources.list as pointed to by this post. Here it is (with the additional sources uncommented):

# A very basic sources.list file.
#
# Fastest repository comes first, with netselect it is possible to determine
# the fastest repository easily.
 
repomd http://ftp-1.gwdg.de/pub/suse/i386 update/10.2
repomd http://ftp-1.gwdg.de/pub/opensuse/distribution 10.2/repo/oss/suse
 
# Visit http://software.opensuse.org/download/ to determine which additional
# components are available.  Some examples are given below, just uncomment
# the line to enable it.
 
repomd ftp://ftp-1.gwdg.de/pub/opensuse/repositories home:/rbos/openSUSE_10.2
repomd ftp://ftp-1.gwdg.de/pub/opensuse/repositories KDE:/KDE3/openSUSE_10.2
repomd ftp://ftp-1.gwdg.de/pub/opensuse/repositories KDE:/Backports/openSUSE_10.2
repomd ftp://ftp-1.gwdg.de/pub/opensuse/repositories KDE:/Playground/openSUSE_10.2
repomd ftp://ftp-1.gwdg.de/pub/opensuse/repositories mozilla/openSUSE_10.2
repomd ftp://ftp-1.gwdg.de/pub/opensuse/repositories openSUSE:/Tools/openSUSE_10.2
repomd ftp://ftp.gwdg.de/pub/linux/misc              packman/suse/10.2
 
# The apt format
# rpm ftp://mirrors.mathematik.uni-bielefeld.de/pub/linux/suse/apt/ @ftp_dir@ base update security rpmkeys
# rpm ftp://ftp4.gwdg.de/pub/linux/suse/apt/ @ftp_dir@ base update security rpmkeys

Updating this list produces a much nicer result:

apt update
Get:1 http://ftp-1.gwdg.de update/10.2 repomd.xml [1231B]
Get:2 http://ftp-1.gwdg.de 10.2/repo/oss/suse repomd.xml [951B]
Get:3 ftp://ftp-1.gwdg.de home:/rbos/openSUSE_10.2 repomd.xml [951B]
Get:4 ftp://ftp.gwdg.de packman/suse/10.2 repomd.xml [951B]
Get:5 ftp://ftp-1.gwdg.de KDE:/KDE3/openSUSE_10.2 repomd.xml [951B]
Get:6 ftp://ftp-1.gwdg.de KDE:/Backports/openSUSE_10.2 repomd.xml [951B]
Get:7 ftp://ftp-1.gwdg.de KDE:/Playground/openSUSE_10.2 repomd.xml [951B]
Get:8 ftp://ftp-1.gwdg.de mozilla/openSUSE_10.2 repomd.xml [951B]
Get:9 ftp://ftp-1.gwdg.de openSUSE:/Tools/openSUSE_10.2 repomd.xml [951B]
Fetched 8839B in 8s (1059B/s)
Get:1 http://ftp-1.gwdg.de update/10.2/ primary.xml [724kB]
Get:2 ftp://ftp-1.gwdg.de home:/rbos/openSUSE_10.2/ primary.xml [28.8kB]
Get:3 ftp://ftp.gwdg.de packman/suse/10.2/ primary.xml [2286kB]
Get:4 ftp://ftp-1.gwdg.de home:/rbos/openSUSE_10.2/ filelists.xml [46.5kB]
Get:5 http://ftp-1.gwdg.de update/10.2/ filelists.xml [4061kB]
Get:6 ftp://ftp-1.gwdg.de KDE:/KDE3/openSUSE_10.2/ primary.xml [112kB]
Get:7 ftp://ftp-1.gwdg.de KDE:/KDE3/openSUSE_10.2/ filelists.xml [1212kB]
Get:8 ftp://ftp.gwdg.de packman/suse/10.2/ filelists.xml [3021kB]
Get:9 ftp://ftp-1.gwdg.de KDE:/Backports/openSUSE_10.2/ primary.xml [174kB]
Get:10 ftp://ftp-1.gwdg.de KDE:/Backports/openSUSE_10.2/ filelists.xml [445kB]
Get:11 http://ftp-1.gwdg.de 10.2/repo/oss/suse/ primary.xml [6272kB]
Get:12 ftp://ftp-1.gwdg.de KDE:/Playground/openSUSE_10.2/ primary.xml [16.7kB]
Get:13 ftp://ftp-1.gwdg.de KDE:/Playground/openSUSE_10.2/ filelists.xml [39.0kB]
Get:14 ftp://ftp-1.gwdg.de mozilla/openSUSE_10.2/ primary.xml [22.8kB]
Get:15 ftp://ftp-1.gwdg.de mozilla/openSUSE_10.2/ filelists.xml [339kB]
Get:16 ftp://ftp-1.gwdg.de openSUSE:/Tools/openSUSE_10.2/ primary.xml [6095B]
Get:17 ftp://ftp-1.gwdg.de openSUSE:/Tools/openSUSE_10.2/ filelists.xml [14.5kB]
Get:18 http://ftp-1.gwdg.de 10.2/repo/oss/suse/ filelists.xml [12.4MB]
Fetched 31.2MB in 1m18s (400kB/s)
Reading Package Lists... Done
Building Dependency Tree... Done

Now we got a working apt for SUSE 10.2.

Updated: March 20th, 2008

The purpose of this article is to:

• introduce the new Linux section of the site (specifically dedicated to openSUSE).
• provide a very short description of Linux and compare it to other operating systems.
• answer some installation questions.

I'm a Windows user, I admit it. Not because I enjoy frequent reboots, freezes, and other unexplainable quirks. It's mostly because I have so many programs I'm used to, it would be impossible to switch to anything else, and I know ins and outs that allow me to be very comfortable with the Windows. I'm talking 20-30 programs I'm not willing to give up any time soon.

However, I also have deep respect for *nix based systems. I started using them back in college and continued later on when I worked at Intel (Solaris) and other jobs. Stability and abundance of open-source software that just works attracted me to Linux. Finally, being a user just wasn't going to cut it: I had way too many questions about how things work, how the OS boots, what the conventions are, etc, etc, etc. So, finally, a couple of days ago I picked openSUSE 10.2 as my distro (distribution) of choice to play around with. The Linux section here on the site has been empty for a while, so I'm going to fill it up by documenting all my experiences and questions throughout the whole learning process as seen with the eyes of an amateur, which, I think, is what most tutorials are missing. So, let's begin.

OK, so… what are all these distributions? Ubuntu, who hasn't heard that by now? SUSE? Fedora? Why can't Linux be just Linux, just like Windows is just Windows? In short, in my opinion, because nobody but Microsoft has access to the Windows code (technically, after that leak a couple of years ago, everyone had a chance to see kernel source, but nobody can actually modify the OS or release another one based on it). Linux is open source, anybody can contribute to its development. That's the beauty of Linux: collaboration of hundreds thousands of people often working on their own time with passion no employee on payroll can match. Some motivated individuals go even further: they compile a set of Linux components that they think would work best together and release it as a distribution. openSUSE is a fine example of that. A bunch of smart people over at Novell got together and produced openSUSE a while ago, with its main goals being:

• Make openSUSE the easiest Linux distribution for anyone to obtain and the most widely used open source platform.
• Provide an environment for open source collaboration that makes openSUSE the world's best Linux distribution for new and experienced Linux users.
• Dramatically simplify and open the development and packaging processes to make openSUSE the platform of choice for Linux hackers and application developers.

See this openSUSE.org about page for more info.

Installing Linux is easier than ever. Back in the day, Linux puzzled average users. Nowadays, for everyday activities Linux is as easy to use as Windows (or Mac if you're into big shiny objects). A short installation guide by Novel is available here and a very nice long guide by HowToForge is available here. If you breeze through the installation steps, you should have a shiny desktop ready to rock and roll in about 30 minutes. While the installation itself is very easy (just burn a CD/DVD from http://download.opensuse.org and reboot the computer), I had a couple of questions about things I wasn't quite sure about. Here they are:

• You're prompted to select your Desktop. The choices are KDE, GNOME, and Other (the Other menu has serveral options, one being no graphical desktop, which you would use if you didn't want a GUI at all, for example for server installs or if you imagined myself as an uber leet hax0r). A graphical desktop is a bit like a skin for the OS. Each also includes different sets of tools, for example KDE's default editor is KEdit while GNOME's is gedit. There are other differences, but for now we have to pick one. The good news is that you can install both later via the software update tool called YaST (actually it's yast2 in SUSE 10.2, yast is just a symlink to /sbin/yast2) and switch between them with ease. For my particular installation, I picked KDE as the desktop manager, which I ended up enjoying quite a lot.

• Correct partitioning of my 110GB hard drive raised the most questions out of all the installation prompts. After consulting with a friendly sysadmin from work, I ended up with
  • 1.0GB swap partition with swap filesystem. I have 2GB RAM, so 1GB swap in case I run out of RAM should suffice.
  • 100MB /boot partition with ext3 filesystem. /boot is where a boot loader like GRUB will reside. We don't want to piss off GRUB, so we'll give it a separate partition.
  • ###MB /tmp partition with ext3 filesystem. /tmp stores temp files for all users. It is better to assign a separate partition to it so that malicious users cannot potentially flood it with useless data and cause the server to run out of space. Don't make this space too small though, so that legitimate users don't run out of temp space.
  • ###MB /var partition with ext3 filesystem. /var can also run the server out of space quickly if a runaway log file fills it up (normally, in /var/log). Assign a few gigs to this dir depending on total space.
  • The rest of the drive I assigned to / with ext3 filesystem. Some people also recommend assigning a separate partition to /home, which is probably a good idea if you're planning on having multiple users on the box. I picked ext3 after reading up online about various filesystem types. ReiserFS was not really an option anymore, since it wasn't too stable yet, and the creator of Reiser was recently arrested for murdering his wife. 2 days after the news broke out, SUSE creators officially announced ext3 as the recommended SUSE filesystem. When ext4 comes out, ext3 partitions will be easily upgradeable to ext4 without the need for repartitioning. The only drawback that I could find for ext3 is the ~32k directory and file limit in a single directory. Oh well, as long as I know about it, I'll be able to craft my applications with that in mind.
  • Update 3/20/2008: I found a great Filesystem Hierarchy Standard document that explains all the dirs in detail here.

• Since I was planning on letting my router (Linksys WRT54G with HyperWRT firmware) handle the firewall duties, I disabled the firewall in the Network part of the installation. It's probably a good idea to leave it on and open ports like SSH (22), FTP (20-21), or HTTP (80) manually. I'm probably going to turn the firewall back on after a couple of weeks once I get the hang of everything.

Well, that should do it for this introduction. In the following articles, I will share my experiences dealing with something something specific, writing as I learn myself. They may be dedicated to software installation and usage, for example, proftpd ftp server. In other articles I may visit Linux concepts, like how to make programs start up at boot time.