HOWTO – Kubuntu 9.04, RAID-10, LVM2, and XFS…

Time to rebuild the Beast…

The poor thing had been in sparse use since it started shutting down (really, just POOF! –  it may as well be unplugged) randomly.  I didn’t pursue it for months, because I am fundamentally lazy at home.

(I think I mentioned this before.)

But with a trip coming up, and me needing a laptop for it, and my wife saying I could have hers if I got her Windows-based internet movie playing experience working on another computer, and the release of Ubuntu 9.04, well, let’s just say the planets finally aligned.

So I burned several CD’s – Xubuntu Live 9.04 (32-bit), MythBuntu 9.04 (64-bit), Kubuntu 9.04 Live (32-bit), and Kubuntu 9.04 Alternate Install (64-bit).  I rebuilt my targeted media PC first as Kubuntu.  I got a 1 TB SATA drive, put it in my beast computer, and used scp to back up everything in the house to it (execpt the Vista laptop).  It power dropped several times before I finally got it all (I hope).  Then I popped it into my media PC (not Microsoft – I call it that because I have it hooked up to the VGA port on my flat-screen TV).  I then loaded the Live CD of Kubuntu 32-bit and will use it as a backup in case my beastie dies while I am away.  This way, my wife is not stuck unable to watch her Japanese TV program downloads.

You have no idea how important that is to maintaining a happy family.  Seriously.

Anyway, enough boring crap.

I first installed the seven SATA drives I had (four pulled from the media PC, one was already installed, and two were sitting in a drawer), each identical 80 GB Hitachis, and left them powered up overnight to find any serious drive errors.  I got seek errors on the one I suspected of being bad, and tossed it.  Trust me – it was bad.

(That may explain why it was sitting in a drawer…)

The other six have stayed quite civilized.  Maybe they got the hint.

After trial and error, I used a combination of the Live CD to google and hand-build the file system, and the alternate 64-bit CD to install.

Why hand-build?  I guess that’s just how I roll…  And it gave me total control over how I built it.

Playing around with the drives, I found I could reliably pop the power just by running “hdparm -Tt /dev/sda”, so off I went to get a new power supply.  I found a 650W PS that more than makes up for my failing 450W PS, and let me clean up my cable mess as well.  Out with the old, in with the new, and everything is smooth as silk.

Back to googling, I found a collection of sites that allowed me to piece together what I think, and hope, is a very solid compromise between performance and reliability.  Space is not too much of an issue, since only one VM will be running on this system, and we are not huge downloaders.  As long as it has more space than the laptop (160 GB), it is fine.

System specs:

  • Athlon FX-53 (the old obsolete server-board-based one with 959 pins or something).
  • 2 GB of registered memory, I forget how fast.
  • An old NVidia AGP 7600 GT card (I think).
  • Four SATA ports onboard (two controllers, no hardware RAID enabled).
  • One four port add-in PCI SATA controller (RAID disabled).
  • No special BIOS tweaks.
  • Six SATA drives, 80 GB each, /dev/sda, /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde, /dev/sdf.
  • Fans.  Lots of fans.
  • 64-bit Kubuntu 9.04, alternate install CD.  Supports VMware-Server 2, and can run 64-bit and 32-bit virtual machine guests.

From the Live CD:

Opening Konsole and Konqueror:

sudo -i

Partitioning the drives:

cfdisk /dev/sda

  • sda1 primary 82 MB type FD (Linux RAID), bootable – this will be the RAID-1 /boot partition of six drives (ext3)
  • sda2 primary 404 MB type FD – this will be the swap partion on RAID-10 and LVM
  • sda3 primary 82 GB type FD – this will be the OS partition on RAID-10 and XFS

sfdisk -d /dev/sda | sfdisk /dev/sdb

sfdisk -d /dev/sda | sfdisk /dev/sdc

sfdisk -d /dev/sda | sfdisk /dev/sdd

sfdisk -d /dev/sda | sfdisk /dev/sde

sfdisk -d /dev/sda | sfdisk /dev/sdf

REBOOT (power cycle), run Live CD again, same apps opened:

sudo -i
apt-get install mdadm lvm2 (the live CD does not get RAID and LVM on its own – so install them)

RAID-1 and RAID-10 (all active, no spares):
Link = http://www.howtoforge.org/install-ubuntu-with-software-raid-10

  • boot partition: mdadm -v -C /dev/md0 -c 256 -n 6 -l 1 /dev/sd[abcdef]1 – RAID1 so LILO can boot it, all drives for max redundancy.
  • swap partition: mdadm -v -C /dev/md1 -c 256 -n 6 -l 10 -p f6 /dev/sd[abcdef]2
  • os partition: mdadm -v -C /dev/md2 -c 256 -n 6 -l 10 -p f2 /dev/sd[abcdef]3

cat /proc/mdstats to see RAID sets:

md2 : active raid10 sda3[0] sdf3[5] sde3[4] sdd3[3] sdc3[2] sdb3[1]
239817216 blocks 256K chunks 2 far-copies [6/6] [UUUUUU]
[=================>…] resync = 86.1% (206697408/239817216) finish=12.6min speed=43546K/sec

md1 : active raid10 sda2[0] sdf2[5] sde2[4] sdd2[3] sdc2[2] sdb2[1]
393472 blocks 6 near-copies [6/6] [UUUUUU]

md0 : active raid10 sda1[0] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1]
79872 blocks 256K chunks 6 far-copies [6/6] [UUUUUU]

unused devices:

Next, set up LVM:
Link = http://www.linuxdynasty.org/lvm2-how-to.html

Physical Volumes:

  • pvcreate /dev/md0
  • pvcreate /dev/md1
  • pvcreate /dev/md2

Volume Groups:
create, with useful names:

  • vgcreate boot-vg /dev/md0
  • vgcreate swap-vg /dev/md1
  • vgcreate os-vg /dev/md2

activate:

  • vgchange -a y boot-vg
  • vgchange -a y swap-vg
  • vgchange -a y os-vg

pvdisplay and pvscan to see physical volumes:

PV /dev/md2 VG os-vg lvm2 [228.71 GB / 4.00 MB free]
PV /dev/md1 VG swap-vg lvm2 [384.00 MB / 0 free]
PV /dev/md0 VG boot-vg lvm2 [76.00 MB / 0 free]
Total: 3 [229.16 GB] / in use: 3 [229.16 GB] / in no VG: 0 [0 ]

vgdisplay and vgscan to see volume groups:

Reading all physical volumes. This may take a while…
Found volume group “os-vg” using metadata type lvm2
Found volume group “swap-vg” using metadata type lvm2
Found volume group “boot-vg” using metadata type lvm2

Logical Volumes, create with useful names:

  • lvcreate -L 76M -n boot-lv boot-vg
  • lvcreate -L 384M -n swap-lv swap-vg
  • lvcreate -L 10G -n root-lv os-vg
  • lvcreate -L 2G -n var-lv os-vg
  • lvcreate -L 3G -n temp-lv os-vg
  • lvcreate -L 213.7G -n home-lv os-vg

lvdisplay and lvscan to see logical volumes:

ACTIVE ‘/dev/os-vg/root-lv’ [10.00 GB] inherit
ACTIVE ‘/dev/os-vg/var-lv’ [2.00 GB] inherit
ACTIVE ‘/dev/os-vg/temp-lv’ [3.00 GB] inherit
ACTIVE ‘/dev/os-vg/home-lv’ [213.70 GB] inherit
ACTIVE ‘/dev/swap-vg/swap-lv’ [384.00 MB] inherit
ACTIVE ‘/dev/boot-vg/boot-lv’ [76.00 MB] inherit

So far, partitioning, RAID-10, and LVM are done. Format using swap, ext3 (boot) and XFS:

  • mkfs.ext3 /dev/boot-vg/boot-lv
  • mkswap /dev/swap-vg/swap-lv

XFS Links:
http://www.csamuel.org/2008/03/23/btrfs-013-and-xfs-benchmarks
http://oss.oracle.com/projects/btrfs/dist/documentation/benchmark.html
http://everything2.com/index.pl?node_id=1479435

http://www.issociate.de/board/post/472270/New_XFS_benchmarks_using_David_Chinner%27s_recommendations_for_XFS-basedoptimizations..html

  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/root-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/var-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/temp-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/root-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/home-lv

df -h to see:

/dev/mapper/os–vg-root–lv
9.9G 4.1M 9.9G 1% /target
/dev/mapper/os–vg-var–lv
1.9G 4.1M 1.9G 1% /target/var
/dev/mapper/os–vg-temp–lv
2.9G 4.1M 2.9G 1% /target/temp
/dev/mapper/os–vg-home–lv
214G 4.1M 214G 1% /target/home
/dev/mapper/boot–vg-boot–lv
74M 5.6M 65M 8% /target/boot

Install using the 64-bit Alternate Install CD for Kubuntu 9.04.  Use the ext3 partition for /boot, the XFS partitions for /, /var, /tmp, and /home, and use the swap partition.  Do not format anything  – it will then only demand to format the swap partition.  I hand-formated to get additional control over how XFS was formatted.

I always separate /home to survive any OS rebuilds I might have to do, or a distro change.  I also separate out /var (so if it fills up, it does not fill up the root space), and /tmp (VMware stores lots of stuff there when snapshotting virtual machines, so make the room).

It installed LILO with the large-memory option on /dev/md0, and reran LILO successfully.  There was some misinformation out there that LILO would boot from a RAID-10 volume.  Yeah, only if it looks exactly like a RAID-1 mirror.  Whoops.

Mount options for XFS I put into /etc/fstab after successfully booting:

-o noatime,nodiratime,logbsize=256k,logbufs=8

Conclusion:  Well, it hasn’t thrown up yet.  I guess that is good.  It seems plenty fast, but I have not really exercised it.  I do not think I will mess with the kernel for a while – I need it to be very stable while I am away, which is exactly when it is most likely to break.

Next, I will put VMware-Server 2 on it and install a 32-bit XP VM for my wife to use with her JNet TV streaming addiction.  I am assuming it won’t work with Firefox and Linux, but I will try that also, to be sure.  She switched to that after Pandora TV changed for the worse.

Should it all work out, I will putty her Windows settings over (bookmarks really), finish up her XP VM, and finally get around to fixing her Vista laptop with a prescription-case extra-strength dose of Xubuntu.

I can hardly wait.

Advertisements