Debugging linux initramfs

Nov 17, 2011 — I’ve written previously about debugging the linux boot process. That was awhile back, and I think it was on RHEL5, but if not, it was another linux which was using an initial RAM disk (initrd). Nowadays, the initrd has been replaced by the initramfs. What’s the difference? Well considering just the files that are in /boot, that is, /boot/initrd-whatever vs. /boot/initramfs-whatever, not much, at least in principle. Both are compressed cpio archives. There is one very important difference.

If you’re packing up an initrd file, you do:

find . | cpio -o -H newc | gzip -9 > ../initrd-whatever

The same thing does not work with an initramfs. For the initramfs to work, you have to do:

find . | cpio -R 0:0 -o -H newc | gzip -9 > ../initramfs-whatever

You have to specify “-R 0:0” as an argument to cpio to make it create all the files in the archive with user and group of 0.

Why exactly this (mostly undocumented AFACT) difference exists, I’m not sure. I suspect it has to do with the contents that are in the initramfs vs. the initrd moreso than the fundamental differences in how the kernel sees these two constructions.

So what is the difference between the initrd vs. the initramfs for the kernel?

Mainly, it’s how the memory is used. In the initrd case, memory is allocated, and that memory is treated like the blocks of a disk, and a filesystem is made in that memory, and the cpio archive is unpacked into that memory that’s being treated as a filesystem on a disk that’s made of RAM.

Some very clever person noticed that this memory-based disk was being cached by the buffer cache. Consequently, the data was in memory twice. And it was realized that there’s no point in flushing out this cache from one chunk of memory to another “backing store” of yet more memory. In fact, the “backing store” can be dispensed with altogether — and the filesystem(!) — and just use the buffer cache without flushing to any backing store! That is what a ramfs is. As such, since you don’t have to allocate memory specifically for it, it’s just stuff in the cache, it grows as needed. That means you can (if you don’t mind assuming the host has quite a lot of memory available for the buffer cache) stuff a lot more in there and not have to worry too much about it. (It’s quite likely that I’ve botched this description in some way, so dig around here.

Consequently, the distros tend to stuff a lot more “complete” environments into the initramfs compressed cpio archive than they do into the initrd compressed cpio archive.

Anyway… this is all a bunch of stuff I figured out today. The main thing I figured out is that you have to use the “-R 0:0” option to cpio when packing up an initramfs image, which is different from what I was used to for packing up an initrd image, and so far as I can tell this is a difference documented nowhere, or at least nowhere I was able to find.

So, if it’s not documented, how did I figure it out? Well, after numerous failures packing it up in the usual way, I did the following:

cd /bin
mv cpio realcpio
cat > cpio
#!/bin/sh
echo cpio $@ >> /tmp/logfile
realcpio $@
^D
chmod +x cpio

Then, I ran “make install” in a kernel source tree (which builds a successful initramfs image as one of its effects) and then examined the logfile to see how it was using cpio.

~ by scaryreasoner on November 18, 2011.

2 Responses to “Debugging linux initramfs”

  1. I am a linux newbie sys admin. I got this as one of my internet question. Difference between initrd and intrdramfs. I could not understand all of it. But yes , i got n idea.

    Thanks

    [email address redacted]

    • I think the main difference is that the initrd used a ram disk (which uses memory as backing store for filesystem buffers, and is therefore a fixed size) and the initramfs uses a ram filesystem — which is a filesystem on filesystem buffers without any backing store — and as such it is not a fixed size and can grow (or shrink) as needed. Because of that, you can (depending on your application) get away with cramming more stuff into an initramfs vs. initrd. And so if you crack open an initrd and an initramfs in typical use cases (e.g. distro boot images) you’ll find the initramfs to be way more complicated and doing a lot more stuff than the initrd typically does. I’m not really sure that’s a good thing, but it seems to be the direction the distros are moving. <— the preceding paragraph might contain gross errors of fact.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: