blogs

Getting good performance out of USB hard drives in Linux

I recently bought two Western Digital MyBook 500 GB external USB 2.0 hard drives to house our CD collection ripped to FLAC. As they will be storing relatively small numbers of very large files, I opted to use a filesystem with a large blocksize and few inodes:

# mkfs.ext3 -T largefile4 /dev/sde1
# mkfs.ext3 -T largefile4 /dev/sdf1

Keep in mind, however, that the resulting filesystem has a fairly small number of inodes so if you start copying lots of small files to the filesystem, it will quickly become full even though a simple df might report tons of free blocks (df -i will give you information about inodes). Of course, you could get around this by tarring and bzipping directories with lots of files.

I noticed a problem, however. While rsyncing data from one drive to another, I noticed abysmal transfer speeds, on the order of 400 KB/sec. Not good. My first instinct was to check that I plugged the drives into USB 2.0 ports and not USB 1.1 ports. Nope - I had the right ports. So I looked at hdparm but that is ATA-specific. There is a sdparm utility for SCSI devices, but it doesn't seem to have much support for USB and Firewire. Eventually, I found this Linux USB FAQ, which talked about the max_sectors setting. I did:

# echo 1024 > /sys/block/sde/device/max_sectors
# echo 1024 > /sys/block/sdf/device/max_sectors

This increased max_sectors from 240 to 1024 (it wouldn't go any higher than 1024) and now rsyncs were transferring on the order of 15 MB/sec. Quite an improvement from 400 KB/sec to 15 MB/sec.

This is on an Ubuntu system with the 2.6.17-11-386 kernel. YMMV, but this worked like a charm for me.

TreeSize for Linux?

I just happened upon a post about a graphical disk usage program called gdmap.

Interesting program, but I'd prefer to see my disk usage in a tree view. On Windows, I really like TreeSize Free. Do you happen to know of anything like that for Linux?

XpressLinux

XpressLinux is based on Kubuntu but has some nice additions for better Windows integration. It might be the perfect distro for a die-hard Windows user who wants to try Linux or have a dual-boot system:

From http://www.xpresslinux.com/about/:

XPressLinux is designed to be easy to use and familiar for Windows users. It will install easily over an existing Windows system, allowing the user to access their existing programs & documents.

Major features of XPressLinux:

  • XPressLinux is based on Kubuntu and uses the KDE desktop.

The UNIX-Haters Handbook

I remember seeing this in a bookstore years ago, but I didn't buy it. Well, now it's free for download.

http://research.microsoft.com/~daniel/unix-haters.html

Mildly amusing. Some of the critiques are still true; others have long been addressed or are no longer relevant (do people think X is a "disaster" because it's a resource hog?) and then a bunch of the book is simply humor and trivia.

For all it's warts, I love UNIX/GNU/Linux and it's the best choice out there for me. Plan 9 is certainly not going to replace it, at least for me.

Playing with Plan 9

I thought it might be interesting to check out the Plan 9 operating system, since it's sort of the sequel to UNIX and is supposed to have cool new features (like better support for the "everything is a file" concept of UNIX). Luckily, you can download an ISO file for a live/install CD, so I downloaded it and fired it up in QEMU....

It's pretty weird. It looks kind of like UNIX and Squeak had a child. The GUI is pretty strange and not very pretty, IMHO. A 3 button mouse is pretty much a necessity as they've mapped the buttons to different things. I tried to move a window by dragging the title bar of the window, but that caused it to resize rather than move. It seems that you have to use Button 3 to move windows. This was pretty excruciating, because I was on my PowerBook with a 1 button trackpad. For some reason, the scrollbars in apps were totally confusing and unusable to me. It looked inspired by Squeak, with touches like using one of the mouse buttons to execute the word under the cursor and even a cutesy mascot, Glenda the bunny, reminiscent of the Squeak mouse. I thought the shell, rc, might be interesting, but I lost interest when I saw that there seemed to be no filename completion, as least with the tab key.

Patch to add -p option to touch, a la mkdir -p

The other day I realized that often when I'm using touch, I'd like it to have the ability to create ancestor directories that don't exist, a la mkdir -p. I quickly hacked together a shell script to do what I want.

Then I thought that this might be a generally useful extension to touch, so I thought I'd take a stab at adding it directly to the touch source code in GNU coreutils. Here's my patch, which I sent to the mailing list:

OS/2 Warp 4 in QEMU on Linux

Well OS/2 does work in QEMU, though I guess you might say that I cheated a bit - I did this by doing the install in VMware Player and then using qemu-img to convert the file from a vmdk to a qcow.

os2warp4_qemu

Loopback mountin' a specific partition inside a disk image

Loopback mounts are cool as heck. You can take a file containing an image of a partition and mount on top of your regular filesystem. But what if you have a file containing an entire disk image rather than just an image of a single partition? It's common to deal with such files when dealing with User Mode Linux (UML) root filesystems.

Well, you can still do it. It turns out that losetup and hence mount which uses losetup to do loopback mounting, allow you to specify an offset option to specify the starting offset into the file from which to read. Now the only trick is to use fdisk to figure out where the partitions are:

ls -h

I've been using Linux for years, but I just realized yesterday that ls has a -h option to display human-readable sizes. I've always used -h with du, but I never knew that ls had it too.

You learn something new everyday!

Fedora Core 6 live CD

It just got a whole lot easier to take Fedora for a quick spin:

lwn.net: Announcing the Fedora 6 Zod live CD and live CD tools