Swimming with the Penguin - Adventures in Gentoo Linux

For the last 13 years, I've been rolling my own Linux distro with Linux from Scratch. The first time was a great learning experience, but each successive build was a little less learning and a little more tiresome. If you're going to do it right, it can take a couple of weeks up front just to figure out the dependencies and the build order, and another month and some change to actually do the build. After three or four rounds, I just couldn't face another two months to build and debug the latest version. So I went hunting for penguins, and found Gentoo.

Gentoo Linux is essentially an automated version of Linux from Scratch. You still build all the software from source, and you can still tailor it to your hardware and needs, but the folks at Gentoo have taken care of (for the most part) the worries about dependencies and build order. I've recently migrated to Gentoo, and thought I'd share some of my observations and experiences.

I heartily suggest following the Gentoo Handbook. While it might tell you a lot of things you already know, they kind of expect you to follow it, and doing so (at least approximately) will make your life easier.

Partitions and Filesystems

To /boot or not to /boot

The Gentoo Handbook suggests rather strongly that you dedicate a small partition at the beginning of your hard drive to /boot. This may be a good idea if you're installing to old hardware and can afford to re-partition your entire hard drive. The hardware I'm running on is over eight years old and I have Grub installed in the middle with no ill effects. If you're installing to a running system with partitions that you don't want to mess with, I see no reason to make /boot a separate partition.
Caveat: Whenever you build a new kernel, be sure to copy it to whatever partition you have Grub installed on.

Read-only /

It might be just me, but it seems that if you're going to install a hardened system (as I did), running with a read-only root filesystem is a no-brainer. And yet nobody does anything to make that easy. Here is how I did it:

  1. I created separate partitions for / and /var (where all the day-to-day writing will take place).
    It's important that the both partitions be at least 10 GB. / needs to be that big because of good old-fashioned code bloat; if you want Firefox and Thunderbird and Gimp and LibreOffice and the like, you're going to need room for them. Not to mention the size of a kernel tree (in /usr/src). And /var needs to be large as well; all that software is going to build there and they are going to need a LOT of space.
  2. After untarring the stage 3 tarball, I moved the following directories:
    cd /
    mv etc var/ && ln -s var/etc etc
    mv home var/ && ln -s var/home home
    mv root var/ && ln -s var/root root
    chmod 0750 var/root
    rm -rf var/tmp && mv tmp/ var/ && ln -s var/tmp tmp
  3. You need to fix a link for Portage:
    cd etc/portage
    rm make.profile
    ln -s ../../../usr/portage/profiles/hardened/linux/amd64 make.profile
  4. Boot from the install CD (or another system on the same PC) and with your root filesystem mounted on /mnt/,
    cd /mnt/
    rm -rf run
    rm var/run
    mkdir var/rootrun
    chmod 1777 var/rootrun
    ln -s var/rootrun run
    cd var
    ln -s rootrun run
    ln -s rootrun/lock lock
    This looks a little klunky, but it is important to preserve the relationships between the directory in /var (rootrun, which is taking the place of /run) and /var/{run,lock}.
  5. A small change is required in /etc/init.d/bootmisc to prevent it from undoing the above; comment out the lines:
           if [ "$RC_UNAME" = Linux -a -d /run ]; then
                   migrate_to_run  /var/lock /run/lock
                   migrate_to_run  /var/run /run
    in the start function.
  6. At this point, we have everything on a single partition. The next step (which should be done from the install CD or another system on the same PC) is to modify /etc/fstab to contain something like
    /dev/sda1      /            ext4     ro                      1     1
    /dev/sda2      /var         ext4     noatime                 0     0
    and then (assuming /dev/sda1 is mounted on /mnt and /dev/sda2 is mounted on /media)
    cp -a /mnt/var/* /media/
    Since /var will be mounted while openrc is bringing up the system, IT IS VITALLY IMPORTANT THAT /var BE IDENTICAL ON THE TWO PARTITIONS. Later you can feel free to modify the contents of the second partition to your heart's content, but anytime a change is made to anything which runs during startup, be sure to make the change to both partitions.
  7. During startup, you will get a red error flag when an attempt is made to write to /etc/mtab but you can safely ignore that since during operation, mtab is just a link to /proc/self/mounts.
  8. Anytime you want to install new software, you can simply
    mount -wo remount /
    and when you're finished, just
    mount -ro remount /
This might seem a little kludgy, but it is by far the simplest way to accomplish the goal of a read-only root filesystem than anything else I've run across, and it's the way I've been running for well over a decade on my previous Linux from Scratch systems.
You'll probably want to insert
UNINSTALL_IGNORE="/etc /lib/modules/*"
into /etc/portage/make.conf to avoid a lot of emerge warnings about the symbolic link /etc.

USE Flags

When I got to the handbook section titled "Configuring the USE variable", I decided to make a valiant attempt to get my ducks in a row. Of course, I missed things, and so ended up using "emerge --newuse" on more than one occasion. However, there are a couple of things that helped:

Kernel Build

I mentioned earlier that a kernel source tree takes a lot of space. For an example, 4.9.16 after a build takes up just under 1 GB.

Building your own kernel is one of those things that just keeps on giving you learning opportunities. Expect to do it several times before you get it right. Then expect to go through that again whenever you upgrade the kernel.

A couple of things worth mentioning (besides what is in the handbook):

Installing Software

Read the Portage introduction section of the handbook early and often.

To find out what was available to me, I did a
find /usr/portage -maxdepth 2 -type d | grep -ve libs | sort | less
and made a list of what I wanted. If you do that, you only need to think about packages which you explicitly use; Portage will handle the dependencies, but you may have to alter USE variables. Rather than let emerge do that for me, I elected to make the changes myself, or, in a few cases, place a USE declaration directly in front of emerge (i.e., USE="aspell icu" emerge --ask gimp).

Keep a tally of what gets installed (one good reason for always 'emerge --ask'ing, since without --ask things will happen very quickly and you might not get a chance to puruse the list of emerged packages easily). Often one of your desired packages will be pulled in by another package.

In general, I started with simple standalone packages first, then did progressively more complex packages, and waited to do the obviously X-only packages after xorg-x11 was installed.

A few specific comments:

Updating Software

I have found so far that updating about once a month is doable (it will take you a weekend to do the updates and solve any problems they cause). But I make sure I "emerge --check-news" once a day to see if there is anything I need to do more immediately. After a few iterations, my usual update workflow is:
eclean -d distfiles
emerge --sync
emerge --ask --update --newuse --deep --backtrack=30 @world

An emerge --update can take a while; to see a list of what is scheduled for updating, take a look at /var/cache/edb/mtimedb.

If any of these are suggested by emerge, I:
perl-cleaner --all
emerge @preserved-rebuild
revdep-rebuild --ignore --pretend
If the output of that looks OK, I
Finally, I
emerge --pretend --depclean
Be very careful with depclean - it tends to want to remove things you need (in my case nano), and if something in the build failed, it will try to remove things that the old version still needs. I run it as above, check each package that I'm not sure about with
equery depends <package>
and then preserve any necessary packages with
emerge --noreplace <package>
Finally, I re-run the depclean without the pretend.

If you ever need to add environment variables or configure parameters for a specific package, do the following:

  1. Create the directory /etc/portage/env.
  2. Create a file in that directory containing the shell commands to set the required variables. For instance, to set the configure option "--with-system-malloc", create a file system.malloc with the contents
  3. Add a line to the file /etc/portage/package.env specifying the package and filename for that option. For instance,
    app-editors/xemacs system.malloc

Because I have a healthy dose of paranoia, I have created a directory called save where I put a copy of everything I have configured in /etc/, reproducing the subdirectories as needed (i.e., /etc/samba/smb.conf is saved in save/samba/smb.conf). Then I wrote a short script to check to see if any updates changed my configuration files:

# checketc

for f in `find . -type f`; do
    fb=$(basename $f)
    n=$(find /etc/ -type f -name "$fb")
    nw=$(echo $n | tr ' ' '\n' | grep -e $f)
    if [ -n "$nw" ]; then
        cmp $f $nw
Then whenever I update software, I
cd save
If a gcc update changes a major release, be sure to read any news items suggested by emerge. Updates to gcc can have wide-ranging ripple effects. It wouldn't hurt to check the Upgrading_GCC wiki page. If you continue to have problems you can try "emerge -e <package>" but be prepared for a long emerge.

Some Comments

I include these not to incite any flames, but to merely put a few ideas out there:

©2017, Kenneth R. Koehler. All Rights Reserved. This document may be freely reproduced provided that this copyright notice is included.

Please send comments or suggestions to the author.