Booting LiveCD’s with Cobbler

As the size of my network grows it becomes increasingly more convenient to be able to boot a live CD rescue environment easily.  Even though most of the hosts are virtual and can be booted directly with an ISO I think it’s even better to PXE boot them directly from Cobbler since it’s already set up.  The documentation for booting live CD’s on Cobbler’s wiki did not seem to work for the Ubuntu Rescue Remix, it’s possible that some modifications need to be made for Ubuntu-based images (as the functionality provided in the livecd-iso-to-pxeboot script seems to be based on RedHat-style distros) but instead I used nfsroot which worked without issue.

First, you’ll want to mount your Live CD via loopback and run a Cobbler import:

# mount ubuntu-rescue-remix-12.04.iso /mnt -o loop
# cobbler import --name=ubuntu-rescue-remix-12.04 --path=/mnt --breed=ubuntu

Next I configured NFS, my configuration is a bit unique as I serve NFS from my file/VM server and Cobbler is a VM which has it’s own application files mounted via NFS as well:

# grep cobbler /etc/exports
/srv/nfs/cobbler <cobblervmip>(rw,async,subtree_check,no_root_squash)
/srv/nfs/cobbler/webroot/cobbler <dhcp-class-c>/24(ro,sync,subtree_check,no_wdelay,insecure_locks,no_root_squash,insecure)

Create the distro as usual and set the kernel options to boot from NFS:

# cobbler distro add --name=ubuntu-rescue-remix-12.04 --kernel=/var/www/cobbler/ks_mirror/ubuntu-rescue-remix-12.04/casper/vmlinuz --initrd=/var/www/cobbler/ks_mirror/ubuntu-rescue-remix-12.04/casper/initrd.gz
# cobbler distro edit --name=ubuntu-rescue-remix-12.04 --kopts='nfsroot=<cobbler IP>:/var/www/cobbler/ks_mirror/ubuntu-rescue-remix-12.04 ip=dhcp netboot=nfs boot=casper

There was one minor issue which I encountered while attempting to mount the nfsroot, “short read: 24 < 28“.  The only thing a Google search turned up was a posting which goes into a bit more detail on the source of the problem.  Apparently if you are have  configured a hosts.allow/deny file it will be an issue because the kernel will use NFSv3 for an nfsroot.  I mistakenly assumed NFS was working fine when I was booted up since my system was configured to use version 4.  Also, the tcp_wrapper hosts.allow/deny files do not recognize CIDR notation (unlike /etc/exports), you’ll have to use “192.168.0.” to specify a /24.

You will also need to add a profile to actually boot the live CD but that is straightforward. Other then that it should work without issue!

Provisioning Ubuntu VM’s with Cobbler

I’ve been playing with Cobbler at home lately now that my server was upgraded to an quad core with the magic vmx flag and ran into an issue deploying Ubuntu VM’s with it.  The install itself and import of the distro is pretty straight-forward, Canonical has some documentation on the process.  Koan can be used for provisoning VM’s (as is mentioned in the docs), however I have some prior Cobbler experience with CentOS and would like to develop this further with Ubuntu.

The problem I encountered was the following: during the install process Ub would not detect the virtual disks and an error is thrown, “no root filesystem is defined”.  I’m using the default KVM virtio disk bus type here and apparently the debian-installer will not detect these with the default configuration.  If you launch a shell and check, /dev/vda exists and running fdisk on it seems to suggest all is good.  Also, running the install via a CD/ISO works just fine as well.  The problem lies with Cobbler.

Eventually I narrowed it down to the preseed file.  When you import the distro it’s mentioned in the Ubuntu Cobbler preseed docs that a default preseed file is generated.  What it doesn’t mention is that something in this configuration is not compatible with virtio disk types.  I haven’t narrowed it down, instead I just copied the Ubuntu profile to a new one and changed the kickstart/preseed to /var/www/cobbler/ks_mirror/[ubuntu-distro-name]/preseed/ubuntu-server-minimalvm.seed.  I’ve got some more work to do on the preseed, I’m not all too familiar with them yet but planning to change that.

mdadm: Device or resource busy error

I recently needed to rebuild a RAID1 array after a reboot for some odd reason and afterwards I was unable to assemble the array.  mdadm came back and reported “Device or resource busy” on one of the drives.  I couldn’t figure out what the issue was originally as it wasn’t mounted and no other processes were using the drive via lsof.  Eventually I tracked it down to a changed UUID – my fstab was trying to mount the old mdadm array and it locked the resource.  I checked it by doing ‘ls’ on /dev/disk/by-uuid/.  Updating fstab with a new UUID, rebooting to clean up things and reassembling the array solved it.  Just a useful item to keep in mind.

Debugging logcheck rules and security events

I’ve been a big fan of logcheck for monitoring my servers, when properly configured it works very well and is pretty flexible.   Unless you are using a centralized logging system such as Splunk most of us are guilty of not thoroughly checking our logs.  I like to use logcheck to perform a simple audit of what my systems are up to, it’s not perfect but certainly better then nothing.

My configuration has been tweaked a bit, adding some custom regex’s for ignoring a few common items.  I found a nice debugging tip on a old posting from the logcheck-devel mailing list which mentions using egrep to test new rules:

cat <logfile> | egrep -v -f /etc/logcheck/ignore.d.workstation/regex

This has saved me a lot of time and frustration when making the final tweaks to a regex.  However, recently I had some difficulties ignoring what seemed to be particularly stubborn security events.  From best I could tell, grep suggested that my expression was filtering properly, yet logcheck was still reporting on these events.  Finally reading through the README I discovered patterns cancelling security alarms must be places in violations.ignore.d, not ignore.d.workstation/server.  Something to be mindful of.

NFS root on SheevaPlug with Debian

I broke out my SheevaPlug the other day and decided to tinker with it a bit… recently I had some new projects ideas where it would be of use.  I also wanted to configure it with an NFS root to avoid flash wear issues and a way to easily back up the device.  This is an attempt at documenting the process in the hope it may help someone else with all the issues I ran into.  YMMV, damnum absque injuria, etc…

The plug had previously been sitting in my closet for some time, in fact it’s been so long that the Ubuntu Jaunty 9.04 install it came with was very outdated by this point.  Unfortunately the idea of upgrading to any more recent Ubuntu release will not work as they are compiled for newer ARM architectures; the CPU in the SheevaPlug is only an ARMv5 and 9.04 was the last version to support this core.  Ubuntu 9.10 was built for v6 and 10.04 and later target the Cortex ARMv7 core.

However we’re still in luck as Debian has an ARM port which supports Marvell’s Kirkwood platform.  And thanks to some wonderful work by Martin we have an easy to follow process for installing Squeeze on the SheevaPlug.  There’s a few caveats and issues to contend with and you will probably have to upgrade U-Boot (Marvell shipped their own variant which added support for plug computers but newer versions support it directly now).  This is all explained very well on Martin’s Debian installation page.  I recommend choosing the option for loading the installer via TFTP as you will need it later if you want to use NFS root.  I also initially installed Debian on an SD card, for some reason trying a USB drive did not work.

Once the installation is done, boot into your new Squeeze install and prepare an NFS mount on another box somewhere which will serve your rootfs.  I perused the DisklessUbuntuHowTo doc to get a basic idea of how it works, you’ll be doing something similar except the initrd update process will be different – this will be explained below.  Once your NFS share is created on the server side, mount it on the SheevaPlug and copy the files over (obviously replacing the IP’s as necessary):

mount -t nfs 192.168.1.2:/nfsroot /mnt
cp -ax /. /mnt/.
cp -ax /boot/. /mnt/boot/.

The Ubuntu HOW-TO also lists copying /dev but this will not be needed as it is mounted via udev.  On my install /boot was a separate partition so make sure you include that as well.  Edit a few files in the new rootfs on the server-side to reflect a few things – /nfsroot/etc/network/interfaces to assign a static IP and /nfsroot/etc/fstab to replace your rootfs device with /dev/nfs.

You’ll also want to copy over the uImage and uInitrd files from /nfsroot/boot over to your TFTP folder as they’ll be needed to boot the plug.

Now comes the fun part, creating a new initrd.  Unfortunately the one installed by Debian did not support a NFS root so we need to build a new one.  Do this on the server-side as the new initrd will be copied to the TFTP folder as well.  The steps are relatively straight-forward but I recommend you read through the details first to get an better understanding.  The steps I outline are taken from a post on editing initrd’s and another on Debian SheevaPlug installation.

The process consists of the following: copy the kirkwood initrd image, extract it, update the initramfs.conf, re-compress, then rebuild it adding a special header for u-Boot.

~$ mkdir tmp

~$ cd tmp

~/tmp$ cp /home/exports/sheeva_squeeze_nfs_root/boot/initrd.img-2.6.32-5-kirkwood ./initrd.img-2.6.32-5-kirkwood.gz

~/tmp$ gunzip initrd.img-2.6.32-5-kirkwood.gz 

~/tmp$ mkdir initrd

~/tmp$ cd initrd/

~/tmp/initrd$ cpio -id < ../initrd.img-2.6.32-5-kirkwood
25329 blocks

~/tmp/initrd$ ls
bin  conf  etc  init  lib  sbin  scripts

At this point you’ll have the initrd extracted.  Edit the conf/initramfs.conf file and search for BOOT=local, change it to BOOT=nfs.

~/tmp/initrd$ vim conf/initramfs.conf

~/tmp/initrd$ find . | cpio --create --format='newc' > ../initrd.img-2.6.32-5-kirkwood-nfs
25329 blocks

~/tmp/initrd$ cd ..

~/tmp$ ls
initrd  initrd.img-2.6.32-5-kirkwood  initrd.img-2.6.32-5-kirkwood-nfs

~/tmp$ gzip -c initrd.img-2.6.32-5-kirkwood-nfs > initrd.img-2.6.32-5-kirkwood-nfs.gz

~/tmp$ ls
initrd  initrd.img-2.6.32-5-kirkwood  initrd.img-2.6.32-5-kirkwood-nfs  initrd.img-2.6.32-5-kirkwood-nfs.gz

~/tmp$ mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n 'Debian ramdisk' -d initrd.img-2.6.32-5-kirkwood-nfs.gz initrd.img-2.6.32-5-kirkwood-nfs
Image Name:   Debian ramdisk
Created:      Thu Oct 20 20:43:31 2011
Image Type:   ARM Linux RAMDisk Image (gzip compressed)
Data Size:    5473115 Bytes = 5344.84 kB = 5.22 MB
Load Address: 0x00000000
Entry Point:  0x00000000

~/tmp$ ls
initrd  initrd.img-2.6.32-5-kirkwood  initrd.img-2.6.32-5-kirkwood-nfs  initrd.img-2.6.32-5-kirkwood-nfs.gz

~/tmp$ cp initrd.img-2.6.32-5-kirkwood-nfs /var/lib/tftpboot/uImage-nfsroot

Now that you have the new initrd created you’ll need to update your uBoot environment variables.  This can be quite complex in-and-of itself.  The previous u-Boot environment variables I had did not work with this new setup, my attempts at abstracting away most of the complexity into many variables did not seem to work, there were many instances where variables referenced in commands and other variables were not getting set properly.  I basically re-worked it from the ground up using sample material at DENX’s (developers of u-Boot) website.  Their manual has a lot of information, you’ll probably want to reference the CommandLineParsing, UBootCmdGroupEnvironment, and LinuxNfsRoot pages primarily.  The working copy of my u-Boot environment is listed below:

addip=setenv bootargs ${bootargs} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}:${netdev}:off panic=1
addmisc=setenv bootargs ${bootargs}
addtty=setenv bootargs ${bootargs} console=ttyS0,${baudrate}
baudrate=115200
boot_mmc=setenv bootargs $(bootargs_console); run bootcmd_mmc; bootm ${kernel_addr_r} ${fdt_addr_r}
boot_nfs=tftpboot ${kernel_addr_r} ${bootfile}; tftpboot ${fdt_addr_r} ${fdt_file}; run nfsargs addip addtty; bootm ${kernel_addr_r} ${fdt_addr_r}
bootargs=root=/dev/nfs rw nfsroot=192.168.1.2:/home/exports/sheeva_squeeze_nfs_root ip=192.168.1.11:192.168.1.2:192.168.1.1:255.255.255.0:sheeva:eth0:off panic=1 console=ttyS0,115200
bootargs_console=console=ttyS0,115200 mtdparts=orion_nand:0x00100000@0x00000000(uBoot)ro,0x00400000@0x00100000(uImage),0x1fb00000@0x00500000(rootfs)
bootargs_nfs=$(bootargs_console) root=/dev/nfs rw nfsroot=$(serverip):$(rootpath) ip=$(ipaddr):$(serverip):$(gateway):$(netmask):sheeva:eth0:none
bootcmd=run boot_nfs
bootcmd_mmc=mmc init; ext2load mmc 0:1 0x00800000 /uImage; ext2load mmc 0:1 0x01100000 /uInitrd
bootcmd_nfs=tftpboot ${kernel_addr_r} ${bootfile}; tftpboot ${fdt_addr_r} ${fdt_file}
bootdelay=3
bootfile=uImage.sheeva.squeeze
console=console=ttyS0,115200 mtdparts=nand_mtd:0x00100000@0x00000000(uBoot)ro,0x00400000@0x00100000(uImage),0x1fb00000@0x00500000(rootfs)
ethact=egiga0
ethaddr=00:50:43:xx:xx:xx
fdt_addr=0x01100000
fdt_addr_r=0x01100000
fdt_file=uInitrd-nfsroot
fileaddr=800000
filesize=539CDD
gatewayip=192.168.1.1
hostname=sheeva
ipaddr=192.168.1.11
kernel_addr_r=0x00800000
netdev=eth0
netmask=255.255.255.0
nfsargs=setenv bootargs root=/dev/nfs rw nfsroot=${serverip}:${rootpath}
rootpath=/home/exports/sheeva_squeeze_nfs_root
serverip=192.168.1.2
stderr=serial
stdin=serial
stdout=serial
x_bootargs=console=ttyS0,115200 mtdparts=orion_nand:512k(uboot),4m@1m(kernel),507m@5m(rootfs) rw
x_bootargs_root=ubi.mtd=2 root=ubi0:rootfs rootfstype=ubifs
x_bootcmd_kernel=nand read 0x6400000 0x100000 0x400000
x_bootcmd_sata=ide reset;
x_bootcmd_usb=usb start;

By default it’s set to boot from NFS via the bootcmd=run boot_nfs line.  If you’re having issues getting NFS root to work, you may want to remove the panic=1 from the addip variable, this way it can drop you to an initrd prompt.  The boot_mmc variable is adapted from Martin’s example.  You may need to modify these slightly if your SheevaPlug is different, specifically the bootargs_console variable – mtdparts reflects the flash partition setup on my plug, update if yours differs (I think these are the factory default settings), /proc/mtd will have this information in it if you boot again from the Debian install.  Also I changed it to orion_nand, I believe it used to be orion_mtd initially.

That’s basically it.   Hope this is of use to someone!

Migrating mail server VM to a new host

I’ve been working on migrating a virtual host over to Rackspace which mainly runs a mail server among a few other small items.  I wasn’t 100% sure how smooth the process would be, expecting to hit at least a few road bumps along the way.  The first one I encountered was issues surrounding MX entries and the simplistic nature of the DNS record editor at Rackspace – most of my emails sent from my home PC were bouncing back 550 failed recipient verification.  This was just a dry run however as when the domain was with my previous hoster I just used my registrar’s DNS, when I switched back the problem seemed to be resolved.

However the second issue I hit had me stumped for a few days.  One of the reasons I migrated (besides price) was greater flexibility; Rackspace gave me more options for distros to choose from and I thought their overall interface was cleaner and designed better.  So when I provisioned the new VM I gave Ubuntu a shot since I run it on my home network I’m a bit more familiar with how I want to configure the box for the software I run at least.  After the DNS/mail issue was resolved everything seemed solid except for a random, albeit fairly minor problem.  For some odd reason hostname resolution replied with “hostname: temporary failure in name resolution” randomly.  I was getting emails from cronjobs running with this error which I found a bit strange.  While I was tinkering with the mail problem I also built a CentOS VM real quick and didn’t notice the error occurring with that host.  I double-checked and made sure the resolv.conf was identical, then /etc/hosts, then nsswitch.conf and so on, all the files seemed the same or at least close enough that I didn’t think it would be a problem.  I made sure DNS resolution worked on the machine and ensured any iptables rules were not in place.  What caught me as the strangest part was the fact it randomly worked and randomly didn’t, there did not seem to be any sort of reproducibility in the issue.  I even ran an strace and compared logs from instances it worked and when it didn’t.  ‘hostname -f’ also took a second or two to reply rather then an immediate response.

Eventually I figured I’d just add an alias to /etc/hosts with the local non-FQDN hostname.  I also noticed then that the /etc/hosts didn’t seem to have an extra carriage return at the end, I put one in and bingo!  Problem fixed.  Looking back through the strace logs I saw upon closer inspection that it didn’t actually read in the second line which had the FQDN hostname, the first for localhost was OK but then it stopped further parsing.  For some reason CentOS behaves differently as I saw – the hosts file was identical (except for the IP’s of course) – it too was missing a carriage return but strace revealed that it parsed the file just fine.  Just in case any one is wondering I was testing this on Ubuntu Lucid 10.04.2 LTS and CentOS 5.5.

::sigh::  Ah well at least I can cancel the plan with my original hoster now. 🙂

Permission issues on slave BIND nameservers

I’m working on several projects at work to enhance our infrastructure and bring automation to our environment through tools such as Cobbler/Puppet/Kerberos and most of these rely on a working DNS system to operate correctly.  After a quick refresh with BIND a primary nameserver was up and running pretty quickly.  The zonefiles were populated easily enough after hacking together some Python to auto-generate configuration files from our server MySQL database.  However I encountered issues when getting the slave nameserver up and running; I was getting errors such as “permission denied” when the slave was attempting to transfer the master zonefile.  I was pretty sure it was configured correctly, it only seemed to fail when creating the temporary file.  I checked the user permissions in the chroot and it all looked good, then I remembered this box was running SELinux.  Checking the BIND FAQ I quickly found the answer: by default named is only allowed to write to the following directories:

  • $ROOTDIR/var/named/slaves
  • $ROOTDIR/var/named/data
  • $ROOTDIR/var/tmp

with $ROOTDIR being the chroot specified in /etc/sysconfig/named.  The configuration files are in $ROOTDIR/var/named and of course I was naming my file “sec.ourdomain.net” as opposed to “slaves/sec.ourdomain.net”.  Oops.  Have to keep this one in mind!

Linux containers and ufw

I’ve been playing with Linux containers (LXC) recently thanks to the SysAdvent calendar and ran into a small issue where network traffic was blocked to the VM  (using a bridged interface) when ufw was running on the host. Granting an ufw allow rule to the guest IP did not seem to help either, it seems there is a bug filed for this.

Two solutions are presented on the Launchpad page:

  • Disable netfilter on the bridge on sysctl by adding the following lines to /etc/sysctl.conf, /etc/ufw/sysctl.conf or /etc/sysctl.d/ufw:
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
  • Configure iptables to permit forwarded traffic across the bridge by adding the following line (before the last COMMIT command) to /etc/ufw/before.rules:
-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT

Whichever you choose the relevant changes must be made active (reloading sysctl or restarting ufw).  I used the ufw/iptables solution and seemed to work fine.

I’m hoping to have another post up here soon about LXC itself, I’ve made a custom VM creation script for Debian and trying to get one working for Ubuntu as well.  From what I’ve seen so far it’s a very nice package although still much under development; I’m not so sure if I’d recommend it to be used in production environments but I see it maturing significantly in the near future.

LVM Loopback HOW-TO

This is a simple tutorial on setting up LVM on loopback devices, I’ve used it a few times for creating dynamic virtual disks; it came in particularly handy when archiving NEXRAD radar data for my radarwatchd project – using up all your inodes on several hundreds of thousands of 15Kb files doesn’t sound like my idea of fun.  Creating a virtual volume with reiserfs was a particularly handy solution in this case.

First Steps

  • Create several empty files with dd.  These will hold your physical volumes:
# dd if=/dev/zero of=lvmtest0.img bs=100 count=1M
1048576+0 records in
1048576+0 records out
104857600 bytes (105 MB) copied, 8.69891 s, 12.1 MB/s
# dd if=/dev/zero of=lvmtest1.img bs=100 count=1M
1048576+0 records in
1048576+0 records out
104857600 bytes (105 MB) copied, 8.69891 s, 12.1 MB/s
  • Link them to loopback devices with losetup: (see below if you run out of loopback devices)
# losetup /dev/loop0 lvmtest0.img
# losetup /dev/loop1 lvmtest1.img
  • Partition them with fdisk.  Create a single primary partition, full size of the device and set the type to Linux LVM (0x8e).  Shorthand commands with fdisk: n,p,1,Enter,Enter,t,8e,w.  I’ve also automated this somewhat with sfdisk:
  • # sfdisk /dev/loop0 << EOF
    ,,8e,,
    EOF
  • Install and configure LVM if needed.  In this test I used the filter settings in lvm.conf to ensure only loopback devices will be used with LVM:
filter = [ "a|/dev/loop.*|", "r/.*/" ]
  • Initialize LVM:
# vgscan
Reading all physical volumes.  This may take a while...
# vgchange -a
  • Create physical volumes with pvcreate:
# pvcreate /dev/loop0 /dev/loop1
 Physical volume "/dev/loop0" successfully created
 Physical volume "/dev/loop1" successfully created
  • Create a volume group then extend to include the second and any subsequent physical volumes:
# vgcreate testvg /dev/loop0
 Volume group "testvg" successfully created
# vgextend testvg /dev/loop1
 Volume group "testvg" successfully extended
  • Verify the operation was successful:
# vgdisplay -v
 Finding all volume groups
 Finding volume group "testvg"
 --- Volume group ---
 VG Name               testvg
 System ID
 Format                lvm2
 Metadata Areas        2
 Metadata Sequence No  2
 VG Access             read/write
 VG Status             resizable
 MAX LV                0
 Cur LV                0
 Open LV               0
 Max PV                0
 Cur PV                2
 Act PV                2
 VG Size               192.00 MB
 PE Size               4.00 MB
 Total PE              48
 Alloc PE / Size       0 / 0
 Free  PE / Size       48 / 192.00 MB
 VG UUID               1Gmt3W-ivMH-mXQH-HswP-tjHR-9mAZ-917z0g

 --- Physical volumes ---
 PV Name               /dev/loop0
 PV UUID               X11MYK-u8hk-4R26-CHuy-QUSw-2hLq-Notlnc
 PV Status             allocatable
 Total PE / Free PE    24 / 24

 PV Name               /dev/loop1
 PV UUID               dLKXlz-c536-9Elj-C2zZ-B4aw-69kj-zZ7PuN
 PV Status             allocatable
 Total PE / Free PE    24 / 24
  • Create a logical volume.  Use the largest available size reported to use by vgdisplay (Free PE/Size value):
# lvcreate -L192M -ntest testvg
 Logical volume "test" created
  • Finally create a filesystem on the new logical volume.  We’re using reiserfs for this example here:
# mkfs.reiserfs /dev/mapper/testvg-test
mkfs.reiserfs 3.6.21 (2009 www.namesys.com)

A pair of credits:
Alexander  Lyamin  keeps our hardware  running,  and was very  generous  to our
project in many little ways.

The  Defense  Advanced  Research  Projects Agency (DARPA, www.darpa.mil) is the
primary sponsor of Reiser4.  DARPA  does  not  endorse  this project; it merely
sponsors it.

Guessing about desired format.. Kernel 2.6.31-17-generic-pae is running.
Format 3.6 with standard journal
Count of blocks on the device: 49152
Number of blocks consumed by mkreiserfs formatting process: 8213
Blocksize: 4096
Hash function used to sort names: "r5"
Journal Size 8193 blocks (first block 18)
Journal Max transaction length 1024
inode generation number: 0
UUID: d50559af-5078-4de5-812a-264590e60177
ATTENTION: YOU SHOULD REBOOT AFTER FDISK!
 ALL DATA WILL BE LOST ON '/dev/mapper/testvg-test'!
Continue (y/n):y
Initializing journal - 0%....20%....40%....60%....80%....100%
Syncing..ok
ReiserFS is successfully created on /dev/mapper/testvg-test.
  • Mount the volume and you should be done:
# mount /dev/mapper/testvg-test /mnt/virtlvm
# df
Filesystem           1K-blocks      Used Available Use% Mounted on

/dev/mapper/testvg-test
 196596     32840    163756  17% /mnt/virtlvm

Growing the LVM volume

To expand the LVM volume you will follow similar steps to the ones stated above:

  • Create another virtual disk and partition with dd and fdisk:
# dd if=/dev/zero of=lvmtest2.img bs=100 count=1M
# fdisk /dev/loop2
  • Tie new disk image to a free loopback device with losetup:
# losetup /dev/loop2 lvmtest2.img
  • Create physical volumes on the new device with pvcreate:
# pvcreate /dev/loop2
Physical volume "/dev/loop2" successfully created
  • Extend the volume group:
# vgextend testvg /dev/loop2
 Volume group "testvg" successfully extended
  • Extend the logical volume:
# lvextend /dev/mapper/testvg-test /dev/loop2
 Extending logical volume test to 288.00 MB
 Logical volume test successfully resized
  • Resize the filesystem with resize_reiserfs:
# resize_reiserfs /dev/mapper/testvg-test
resize_reiserfs 3.6.21 (2009 www.namesys.com)

resize_reiserfs: On-line resizing finished successfully.

# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/testvg-test
                        294896     32840    262056  12% /mnt/virtlvm

Modifying the amount of loopback devices

  • To see all the loopback devices:
# ls -l /dev/loop*
  • Adding new loopback devices
  • If loopback support is compiled as a module:
# modprobe loop max_loop=64
  • To make permanent edit /etc/modules or /etc/modprobe.conf and use add one of the following:
loop max_loop=64
options loop max_loop=64
  • If loopback module is compiled into the kernel you will need to reboot, edit the kernel command-line parameters appending ‘max_loop=64’ to the end.
  • Loopback devices can also be dynamically created with mknod.  Loopback devices have a major number of 7 and minor number of the loopback device.  Keep in mind devices created this way will not be persistent and will disappear after a reboot:
# mknod -m660 /dev/loop8 b 7 8
# mknod -m660 /dev/loop9 b 7 9
# mknod -m660 /dev/loop10 b 7 10

That should cover mostly everything.  I’ve been informed that some of the steps may not be strictly necessary, such as the vgscan/vgchange.  I also believe the partitioning may not be needed as we’re using the full size of the virtual devices but it’s good practice nonetheless and definitely makes things clearer being able to see the LVM partitions.   Hope this helps!

Upgrading EOL Ubuntu installations

I have a number of Ubuntu boxes laying around and gotten a bit lazy keeping some of the lesser-used ones up to date.  I realized this after trying an apt-get update resulted in 404 errors, oops.  Since I couldn’t directly do a dist-upgrade I checked the Ubuntu wiki for upgrading EOL installations, the process is pretty simple.

All you basically need to do is update your /etc/apt/sources.list and replace us.archives.ubuntu.com (or whatever servers you are using) with old-releases.ubuntu.com, setting the release for your current distro correctly of course.  If it’s a desktop system you may need to install or ugprade update-manager package and/or ubuntu-desktop as well.  Then a simple aptitude update && aptitude safe-upgrade and do-release-upgrade should take care of your needs.  If you are multiple releases behind you will need to upgrade from one release to the next individually one at a time, you can’t skip directly to the latest so it may take some time.  Otherwise it’s pretty straightforward and from my experience thus far very pain-free which is always a plus.