Stateless VMware ESXi 3.5 on an HP c7000 Blade Server…

NOTE:  This is only an overview.  Due to the detailed nature of this project, I will break it up over several more-focused articles over time for easier reference.

Well, despite my more negative impression of this year’s VMworld conference, it still really paid off.  There I learned about stateless ESX deployment.  Using this information, I was able to build in my lab, after a couple months of trial and error, a highly robust VMware environment, fully managed and licensed, using the midwife scripts I modified for this effort.  And configuration is hands-free.

Here are the system components:

  • SERVER – HP c7000 Blade Enclosure with sixteen Bl465c blades, two 4 GB FC modules, and four VC Enet modules
  • Each blade has two dual-core AMD CPUs, 16 GB RAM, two 72 GB SAS drives (hardware RAID-1), two embedded gig NICs, and a mezzanine card with two more gig NICs/iSCSI initiators and two FC HBAs
  • NETWORK – Cisco 6509 with two SUP 720 cards, two 48 port LC Gig-E fiber cards, and four 48 port gig copper cards
  • MANAGMENT – Dell 1850 with two 146 GB SAS drives (hardware RAID-1) for management and boot services
  • STORAGE – Scavenged proof-of-concept totally ghetto Dell Optiplex desktop with four internal 1.5 TB SATA drives (software RAID-10 formatted with tuned XFS) providing 3 TB of NFS shared storage
  • Scavenged HP IP-KVM box for OOB-management of the two Dells

Here are the steps I took:

  1. First I had to update all the firmware on the blade server.  This includes the two OA cards for the Onboard Administrator, the Virtual Connect software, the iLO2 software for each blade, the BIOS on each blade, and the Power Management Controller firmware.  There is a particular order this is done in, and it is not easy, but it really needs to be done.  The fixes that come with these updates are often vital to success.  Overall, I spent a week researching and updating.  I set all the blades to boot via PXE.
  2. Next, I built the storage server.  I really had no choice – nothing was available but a Dell Optiplex desktop.  It had four internal SATA ports available, and room for four 1GB RAM modules.  It also had a single dual-core Intel CPU and PCI slots for more NICs, and a PCI-Express mini-slot as well.  I had to order parts, and it took a little while, but once done, it had a total of four gig NICs (one embedded, two PCI, one PCI-Express), four 1.5 TB SATA drives, and 4 GB RAM.  I loaded it with 64-bit Ubuntu-9.04, hand-carved the partitions and RAID-10 setup, formatted the 3 TB volume with XFS, tuned as best I knew how, and then put it on the 2.6.31 kernel (I later updated it to 2.6.31.5).  There were no BIOS or other firmware updates needed.
  3. I then built the management server on the Dell 1850.  It only has one power supply (I cannot find a second one), but it does have 8 GB RAM and two dual-core CPUs.  I loaded 64-bit Ubuntu-9.04 on it afte installing two 146 GB SAS drives in a RAID-1 mirror (hardware-based).  I also updated the BIOS and other firmware on it.
  4. Having these components in place, I studied the blade server to see what I could get away with, and ultimately decided to use each NIC on a blade server to support a set of traffic types, and balanced the likelyhood of traffic demands across them.  For example, Vmotion traffic, while it may be intense, should be relatively infrequent, so it shares a V-Net with another type of traffic that is low-bandwidth (the alternate management  network).  Altogether, I ended up with a primary management network on up V-Net, Vmotion and the alternate on another V-Net, storage traffic (NFS and iSCSI) on a third V-Net, and VM traffic on its own V-Net.  Each V-Net maps to the its own NIC on a blade, the same NIC on each blade.

The physical network design:

For the V-Nets, the management network went on NIC 1 as an untagged VLAN.  It has to be untagged, because when it boots up, it needs to get a DHCP address and talk to the boot server for its image.  Since it comes up untagged, it will not be able to talk out to the DHCP/PXE server if the V-Net is set to pass through tags.  The other V-Nets support tagged VLANs to further separate traffic.  Each V-Net has four links to the Cisco 6509, except for the storage V-Net, which has eight.  Two links form an LACP bundle from the active side (VC-Enet module in Bay 1), and two make up an LACP bundle (or etherchannel) from the module in Bay 2, which is the offline side.  This is repeated for the other networks across the other modules in Bays 5 and 6.  Bays 3 and 4 house the Fiber Channel modules, which I am not using.  Everything is on its own individual private 10.x.x.x network as well, except for the VM traffic net, which will contain the virtual machine traffic.

The storage design:

Like I said, a really ghetto NFS server.  It does not have enough drives, so even though it would be overkill for a home PC, it will not cut it in this situation.  I expect it to run out of steam after only a few VMs are added, but it does tie everything together and provides the shared storage component needed for HA, Vmotion, and DRS.  I am working on an afforable and acceptable solution, rack-mounted, with more gig NICs and up to 24 hot-swap drives – more spindles should offer more thoughput.  I bonded the NICs together into a single LACP link, untagged back the the Cisco, on the NFS storage VLAN.  Once working, I stripped out all unneeded packages for a very minimal 64-bit Ubuntu server.  It boots in seconds, and has no GUI.  Unfortuately, I did not get into the weeds enough to align the partitions/volumes/etc.  I just forgot to do that.  I will have to figure that out next time I get a storage box in.

The management server:

It is also on a very minimal 64-bit Ubuntu-9.04 install.  Ithas four NICs, but I only use two (the other two are only 100 MB).  The two gig NICs are also bonded into one LACP link back to the Cisco, untagged.  The server is running a stripped down 2.6.31 kernel, and has VMware Server 2.0.x installed for the vCenter Server (running on a Windows 2003 server virtual machine).  On the Ubuntu host server, I have installed and configured DHCP, TFTP, and gPXE.  I also extracted the boot guts from the ESXi 3.5.0 Update 4 ISO and set up the tftpboot directory so that each blade will get the image installed.  On the vCenter Server virtual machine, I installed the Microsoft PowerShell tool (which installed ActiveState PERL), and the VMware PowerCLI tool.  I also downloaded the midwife scripts and installed Notepad++ for easy editing.  The vCenter Server VM is on a private 10.x.x.x net for isolated management, but this gets in the way of the Update Manager plugin, so I still have some work to do later to get around this.

Really key things I learned from this:

  1. The blade server VC-Enet modules are NOT layer-2 switches.  They may look and feel that way in some aspects, but they, by design, actually present themselves to network devices as server ports (NICs), not as more network devices.  Learn about them – RTFM.  It makes a difference.  For instance, it may be useful to know that the right side bay modules are placed in standby by default, and the left-side are active – they are linked via an internal 10Gig connection.  I know of another lab with the same hardware that could not figure out why they could not connect the blade modules to the network if all the modules were enabled, so they solved it by disabling all but Bay-1, instead of learning about the features and really getting the most out of it.
  2. Beware old 64-bit CPUs.  Just because it lets you load a cool 64-bit OS on it does NOT mean it will let you load a cool 64-bit virtual machine on it.  If it does not have virtualization instruction sets in its CPU(s), you will run into failure.  I found this out the hard way, after trying to get the RCLI appliance (64-bit) from VMware in order to manage the ESXi hosts.  I am glad I failed, because it forced me to try the PowerCLI/PowerShell tools.  Without those tools, I seriously doubt I could have gotten this project working.
  3. Learn PowerShell.  The PowerCLI scripts extend it for VMware management, but there are plenty of cool tricks you can do using the base PowerShell scripts as well.  I am no fan of Microsoft, so it is not often I express satisfaction with one of their products.  Remember where you were on this day, ‘cuz it could be a while before it happens again.
  4. Name resolution is pretty important.  HA wants it in a real bad way.  Point your hosts to a DNS server, or give them identical hosts files (a little ghetto, but a good failsafe for a static environment).  I did both.
  5. Remember those Enet modules?  Remember all that cool LACP stuff I mentioned?  Rememeber RTFM?  Do it, or you will miss the clue that while the E-net modules like to play with LACP, only one link per V-Net is set active to avoid loops.  So if, on your active V-Net, you have two LACP links, each for a different tagged VLAN, and your NFS devices won’t talk to anyone, you will know that it is because it saw your iSCSI V-Net first, so it set your NFS link offline.  Meaning, the iSCSI link on Bay-1 and it’s offline twin on Bay-2 both have to fail before your NFS link on Bay-1 will come up.  Play it safe – one LACP link per V-Net per bay.  Tag over multiple VLANs on the link instead. The E-net modules only see the LACP links, and do not care if they support different VLANs – only one is set active at a time.
  6. Be careful with spanning tree (this can be said for everything related to networking).    Use portfast on your interfaces to the E-net modules, and be careful with spanning tree guards on the Cisco side.  In testing, I would find that by pulling one of the pairs in a link, it would isolate the VLAN instead of carrying on as if nothing had happened.  Turns out a guard on the interface was disabling the link to avoid potential loops.  Once I disabled that, the port-channel link functioned as desired.
  7. Doesn’t it suck to get everything working, and then not have a clean way to import in VMs?  I mean, now that you built it, how do you get stuff into it?  I ended up restructuring my NFS server and installing Samba as well.  This is because when importing a VM from the GUI (say, by right-clicking on a resource pool), the “Other Virtual Machine” option is the only one that fits.  However, it then looks for a UNC path (Windows share-style) to the .vmx file.  I could browse the datastore and do it that way, but for VMs not on the NFS datastore already, I needed to provide a means for other labs to drop in their VMs.  Samba worked.  Now they can drop in their VMs on the NFS server via Samba, and the vCenter Server can import the VMs from the same place.

Currently, we are restructuring phycial paths between labs for better management.  It is part of an overall overhaul of the labs in my building.  Once done, my next step is to start building framework services, such as repository proxy servers, WSUS servers, DHCP/DNS/file/print, RADIUS/S-LDAP/AD, etc., etc.  I also need to wrap in a management service framework as well that extends to all the labs so everyone has an at-a-glance picture of what is happening to the network and the virtual environment.  One last issue I am fighting is that I am unable to complete importing VMs I made on ESX 3.5 U2 earlier this year.  It keeps failing to open the .vmdk files.  I will have to pin that down first.

The end result?

  1. If I run the midwife service on the vCenter server and reboot a blade, it is reloaded and reconfigured within minutes.
  2. If I upgrade to beefier blades, I pop them in and let them build.
  3. If I update to a newer release of ESXi (say, update 5 or 6), I extract from the ISO to the tftpboot directory and reboot the blades.  The old configs get applied on the new updated OS.
  4. All configs are identical – extremely important for cluster harmony.  No typos.
  5. If someone alters a config and “breaks” something, I reboot it and it gets the original config applied back.
  6. If I make a change to the config, I change it in the script once, not on each blade individually.  This also allows for immediate opportunity to DOCUMENT YOUR CHANGES AS YOU GO.  Which is just a little bit important.

As stated before, this is an overview.  I will add more detailed articles later, which will include scripts and pictures as appropriate.  I am at home now and do not have access to my documentation, but once I get them, I will post some goodies that hopefully help someone else out.  To include myself.

Kubuntu 9.04 64-Bit, Kernel 2.6.30.1, and NVIDIA…

I went ahead and decided to upgrade my kernel, and to go to the latest NVIDIA driver (180.51).  I downloaded the kernel and the nvidia driver file, built the kernel, and removed the nvidia restricted driver.  This is on a 64-bit build of Kubuntu 9.04.

However, I was not done.

When I tried to install the kernel image file, I kept getting dkms errors relating to nvidia-common.  I eventually removed the nvidia packages using “apt-get remove --purge nvidia*” (as root), but this still would not allow me to install the kernel. Also my xorg.conf file was empty.

I fixed xorg by typing “dpkg-reconfigure xserver-xorg” and adding in the line “Driver "vesa" ” to the “Device” section, so I would have something when I rebooted.

Only when I removed dkms (“apt-get remove dkms“), was I able to install kernel 2.6.30.1.  I use lilo since I run RAID-10, but did not have to update the /etc/lilo.conf file.  Upon reboot, I stopped X with “/etc/init.d/kdm stop“.

I next installed the NVidia driver first (and chose to install the 32-bit compatibility files as well). After that, I ran “nvidia-xconfig” and my xorg file was ready.  When testing with the “X” command, it just pulled up a blank screen, but I took a chance and started KDM (“/etc/init.d/kdm start“).

Everything came up fine.  Typing “glxgears” in a terminal showed decent enough acceleration (about 3000 fps).

So far, no other ill effects. And no firmware issues.

Tuning XFS – Useful Links…

These are both related to MythTV, but can apply to any XFS filesystem you may be using.

XFS Filesystem:  http://www.mythtv.org/wiki/XFS_Filesystem

Optimizing Performance:  http://www.mythtv.org/wiki/Optimizing_Performance

Impressions of Kubuntu 9.04 and VMware-Server 2.0.1…

So far, RAID-10/LVM/XFS is working quite well with Kubuntu 9.04.  Jaunty picks up hardware effortlessly.  I plugged in a USB thumb drive, and a little notification pops up.

Ok.

I plug in my camera, and it sees it fine,no muss, no fuss.

Better.

I plug in my webcam – no notification, it just works.

Sweeeet.

I plug in my HP printer, and I have to dig around to see that it was added as quietly and politely as you please, ready to print.

Awesome.

I ran out of things to plug in.  Kubuntu 8.04 (the previous version I was using) didn’t boot nearly as quickly, took longer to load the desktop after login, and was good about detecting devices, mostly, but needed polish and charm.

9.04 has it in spades.  I am really quite impressed with the hardware cababilities of it.  There are some programs, like adept, I am missing, but the learning curve for the newer stuff is really more like a learning bump.

Update:  It even loaded the sensors package to track temperatures.  Wow.

I am running 64-bit now, and flash and java work fine.  It took me a while to find the right libjavaplugin and link it into the Firefox plugins folder, but flash 10 worked fine and installed easily.

VMware-Server is a different story.  The 64-bit is slow, flaky, and cranky.  It times out all the time, it resets often, and it just stalls doing stuff.  I now have a VM ready for loading, but it took all day to fight it into doing so.  And I found no reliable cure, to include swapping out the java jre version used for a later version.  I am really dissapointed with the 2.0.1 release in terms of ease of install, performance, and reliability.  Oh well, at least it installed without needing a special patch or script.

Update:  After a huge fight, I got a new Windows XP VM made.  Using the command ‘watch “du -s –si /home/vmguests/WinXP” ‘, I was able to get a sense of the speed of the file system when I was creating the virtual disk files.  I chose to make one large file at once for each of the two disks; C drive (15 GB), and E drive (48 GB).  With the watch command updating every two seconds, I was able to see that the RAID-10 XFS filesystem was handling about 100 Mbps as the disk files were created.

Once I had made theVM, loading it was uneventful.  Just a regular Windows XP professional install, like any other.  The vmware-server played nice mostly after that and has continued to do so.  I have only had to log out once due to unresponsiveness, and have not had to restart the server services.  The VM is quite fast, and allows my wife to see her video streams in Media Player 11 with only minor stuttering of the video.   Audio is fine.

I really like the USB visibilty of vmware-server.  The VM picked up the printer as if it were directly connected, and once I loaded the drivers for it, I was printing from the VM like normal.  All of my USB devices can be presented to the VM, which is an area I had problems with in the past with the 1.x versions of vmware-server.

Anyway, my wife is set up with her login and has a shortcut to RDP to the Windows XP VM, where she can login and watch her JNet streams.

HOWTO – Kubuntu 9.04, RAID-10, LVM2, and XFS…

Time to rebuild the Beast…

The poor thing had been in sparse use since it started shutting down (really, just POOF! -  it may as well be unplugged) randomly.  I didn’t pursue it for months, because I am fundamentally lazy at home.

(I think I mentioned this before.)

But with a trip coming up, and me needing a laptop for it, and my wife saying I could have hers if I got her Windows-based internet movie playing experience working on another computer, and the release of Ubuntu 9.04, well, let’s just say the planets finally aligned.

So I burned several CD’s – Xubuntu Live 9.04 (32-bit), MythBuntu 9.04 (64-bit), Kubuntu 9.04 Live (32-bit), and Kubuntu 9.04 Alternate Install (64-bit).  I rebuilt my targeted media PC first as Kubuntu.  I got a 1 TB SATA drive, put it in my beast computer, and used scp to back up everything in the house to it (execpt the Vista laptop).  It power dropped several times before I finally got it all (I hope).  Then I popped it into my media PC (not Microsoft – I call it that because I have it hooked up to the VGA port on my flat-screen TV).  I then loaded the Live CD of Kubuntu 32-bit and will use it as a backup in case my beastie dies while I am away.  This way, my wife is not stuck unable to watch her Japanese TV program downloads.

You have no idea how important that is to maintaining a happy family.  Seriously.

Anyway, enough boring crap.

I first installed the seven SATA drives I had (four pulled from the media PC, one was already installed, and two were sitting in a drawer), each identical 80 GB Hitachis, and left them powered up overnight to find any serious drive errors.  I got seek errors on the one I suspected of being bad, and tossed it.  Trust me – it was bad.

(That may explain why it was sitting in a drawer…)

The other six have stayed quite civilized.  Maybe they got the hint.

After trial and error, I used a combination of the Live CD to google and hand-build the file system, and the alternate 64-bit CD to install.

Why hand-build?  I guess that’s just how I roll…  And it gave me total control over how I built it.

Playing around with the drives, I found I could reliably pop the power just by running “hdparm -Tt /dev/sda”, so off I went to get a new power supply.  I found a 650W PS that more than makes up for my failing 450W PS, and let me clean up my cable mess as well.  Out with the old, in with the new, and everything is smooth as silk.

Back to googling, I found a collection of sites that allowed me to piece together what I think, and hope, is a very solid compromise between performance and reliability.  Space is not too much of an issue, since only one VM will be running on this system, and we are not huge downloaders.  As long as it has more space than the laptop (160 GB), it is fine.

System specs:

  • Athlon FX-53 (the old obsolete server-board-based one with 959 pins or something).
  • 2 GB of registered memory, I forget how fast.
  • An old NVidia AGP 7600 GT card (I think).
  • Four SATA ports onboard (two controllers, no hardware RAID enabled).
  • One four port add-in PCI SATA controller (RAID disabled).
  • No special BIOS tweaks.
  • Six SATA drives, 80 GB each, /dev/sda, /dev/sdb, /dev/sdc, /dev/sdd, /dev/sde, /dev/sdf.
  • Fans.  Lots of fans.
  • 64-bit Kubuntu 9.04, alternate install CD.  Supports VMware-Server 2, and can run 64-bit and 32-bit virtual machine guests.

From the Live CD:

Opening Konsole and Konqueror:

sudo -i

Partitioning the drives:

cfdisk /dev/sda

  • sda1 primary 82 MB type FD (Linux RAID), bootable – this will be the RAID-1 /boot partition of six drives (ext3)
  • sda2 primary 404 MB type FD – this will be the swap partion on RAID-10 and LVM
  • sda3 primary 82 GB type FD – this will be the OS partition on RAID-10 and XFS

sfdisk -d /dev/sda | sfdisk /dev/sdb

sfdisk -d /dev/sda | sfdisk /dev/sdc

sfdisk -d /dev/sda | sfdisk /dev/sdd

sfdisk -d /dev/sda | sfdisk /dev/sde

sfdisk -d /dev/sda | sfdisk /dev/sdf

REBOOT (power cycle), run Live CD again, same apps opened:

sudo -i
apt-get install mdadm lvm2 (the live CD does not get RAID and LVM on its own – so install them)

RAID-1 and RAID-10 (all active, no spares):
Link = http://www.howtoforge.org/install-ubuntu-with-software-raid-10

  • boot partition: mdadm -v -C /dev/md0 -c 256 -n 6 -l 1 /dev/sd[abcdef]1 – RAID1 so LILO can boot it, all drives for max redundancy.
  • swap partition: mdadm -v -C /dev/md1 -c 256 -n 6 -l 10 -p f6 /dev/sd[abcdef]2
  • os partition: mdadm -v -C /dev/md2 -c 256 -n 6 -l 10 -p f2 /dev/sd[abcdef]3

cat /proc/mdstats to see RAID sets:

md2 : active raid10 sda3[0] sdf3[5] sde3[4] sdd3[3] sdc3[2] sdb3[1]
239817216 blocks 256K chunks 2 far-copies [6/6] [UUUUUU]
[=================>...] resync = 86.1% (206697408/239817216) finish=12.6min speed=43546K/sec

md1 : active raid10 sda2[0] sdf2[5] sde2[4] sdd2[3] sdc2[2] sdb2[1]
393472 blocks 6 near-copies [6/6] [UUUUUU]

md0 : active raid10 sda1[0] sdf1[5] sde1[4] sdd1[3] sdc1[2] sdb1[1]
79872 blocks 256K chunks 6 far-copies [6/6] [UUUUUU]

unused devices:

Next, set up LVM:
Link = http://www.linuxdynasty.org/lvm2-how-to.html

Physical Volumes:

  • pvcreate /dev/md0
  • pvcreate /dev/md1
  • pvcreate /dev/md2

Volume Groups:
create, with useful names:

  • vgcreate boot-vg /dev/md0
  • vgcreate swap-vg /dev/md1
  • vgcreate os-vg /dev/md2

activate:

  • vgchange -a y boot-vg
  • vgchange -a y swap-vg
  • vgchange -a y os-vg

pvdisplay and pvscan to see physical volumes:

PV /dev/md2 VG os-vg lvm2 [228.71 GB / 4.00 MB free]
PV /dev/md1 VG swap-vg lvm2 [384.00 MB / 0 free]
PV /dev/md0 VG boot-vg lvm2 [76.00 MB / 0 free]
Total: 3 [229.16 GB] / in use: 3 [229.16 GB] / in no VG: 0 [0 ]

vgdisplay and vgscan to see volume groups:

Reading all physical volumes. This may take a while…
Found volume group “os-vg” using metadata type lvm2
Found volume group “swap-vg” using metadata type lvm2
Found volume group “boot-vg” using metadata type lvm2

Logical Volumes, create with useful names:

  • lvcreate -L 76M -n boot-lv boot-vg
  • lvcreate -L 384M -n swap-lv swap-vg
  • lvcreate -L 10G -n root-lv os-vg
  • lvcreate -L 2G -n var-lv os-vg
  • lvcreate -L 3G -n temp-lv os-vg
  • lvcreate -L 213.7G -n home-lv os-vg

lvdisplay and lvscan to see logical volumes:

ACTIVE ‘/dev/os-vg/root-lv’ [10.00 GB] inherit
ACTIVE ‘/dev/os-vg/var-lv’ [2.00 GB] inherit
ACTIVE ‘/dev/os-vg/temp-lv’ [3.00 GB] inherit
ACTIVE ‘/dev/os-vg/home-lv’ [213.70 GB] inherit
ACTIVE ‘/dev/swap-vg/swap-lv’ [384.00 MB] inherit
ACTIVE ‘/dev/boot-vg/boot-lv’ [76.00 MB] inherit

So far, partitioning, RAID-10, and LVM are done. Format using swap, ext3 (boot) and XFS:

  • mkfs.ext3 /dev/boot-vg/boot-lv
  • mkswap /dev/swap-vg/swap-lv

XFS Links:
http://www.csamuel.org/2008/03/23/btrfs-013-and-xfs-benchmarks
http://oss.oracle.com/projects/btrfs/dist/documentation/benchmark.html
http://everything2.com/index.pl?node_id=1479435

http://www.issociate.de/board/post/472270/New_XFS_benchmarks_using_David_Chinner%27s_recommendations_for_XFS-basedoptimizations..html

  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/root-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/var-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/temp-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/root-lv
  • mkfs.xfs -f -d agcount=1 -i attr=2 -l lazy-count=1,size=128m,version=2 /dev/os-vg/home-lv

df -h to see:

/dev/mapper/os–vg-root–lv
9.9G 4.1M 9.9G 1% /target
/dev/mapper/os–vg-var–lv
1.9G 4.1M 1.9G 1% /target/var
/dev/mapper/os–vg-temp–lv
2.9G 4.1M 2.9G 1% /target/temp
/dev/mapper/os–vg-home–lv
214G 4.1M 214G 1% /target/home
/dev/mapper/boot–vg-boot–lv
74M 5.6M 65M 8% /target/boot

Install using the 64-bit Alternate Install CD for Kubuntu 9.04.  Use the ext3 partition for /boot, the XFS partitions for /, /var, /tmp, and /home, and use the swap partition.  Do not format anything  – it will then only demand to format the swap partition.  I hand-formated to get additional control over how XFS was formatted.

I always separate /home to survive any OS rebuilds I might have to do, or a distro change.  I also separate out /var (so if it fills up, it does not fill up the root space), and /tmp (VMware stores lots of stuff there when snapshotting virtual machines, so make the room).

It installed LILO with the large-memory option on /dev/md0, and reran LILO successfully.  There was some misinformation out there that LILO would boot from a RAID-10 volume.  Yeah, only if it looks exactly like a RAID-1 mirror.  Whoops.

Mount options for XFS I put into /etc/fstab after successfully booting:

-o noatime,nodiratime,logbsize=256k,logbufs=8

Conclusion:  Well, it hasn’t thrown up yet.  I guess that is good.  It seems plenty fast, but I have not really exercised it.  I do not think I will mess with the kernel for a while – I need it to be very stable while I am away, which is exactly when it is most likely to break.

Next, I will put VMware-Server 2 on it and install a 32-bit XP VM for my wife to use with her JNet TV streaming addiction.  I am assuming it won’t work with Firefox and Linux, but I will try that also, to be sure.  She switched to that after Pandora TV changed for the worse.

Should it all work out, I will putty her Windows settings over (bookmarks really), finish up her XP VM, and finally get around to fixing her Vista laptop with a prescription-case extra-strength dose of Xubuntu.

I can hardly wait.

Free Computer!

Amazing what some folks will throw away.

I was driving to work, and I happened to see a computer sitting on the curb in my neighborhood. This means it is for the garbage collector to pick up. I went to the house and asked what the story was, and the lady who answered the door told me that her husband had also found it on the curb somewhere else, but had been unable to make it work and didn’t have the time to continue, so he was throwing it out. She said I could have it, so I went to take a look.

No hard drive, no video card, dual SLI PCIe slots, a nice case and power supply, DVD writer…. and a gigabyte of Corsair RAM (PC3200) (!). Bonus!

So I took it home and a weekend later, had a monitor, PCI video card (no AGP slot), speakers, keyboard and mouse. I also had a couple of Linux CDs ready.

After checking all the connections and parts, this is what it had inside:

  • A 500W modular-plug power supply
  • A PCI Broadcom wireless G card (no antenna, but I had a spare)
  • A SoundBlaster Audigy PCI card
  • A generic PCI LAN card
  • An ASA A8N-Premium motherboard
  • A slew of USB ports
  • Onboard sound and dual gigabit LAN ports
  • 3 PCI slots, two 16x PCIe slots, and four memory slots
  • Eight SATA ports onboard, evenly split across two onboard RAID controllers (one supporting RAID-5)
  • Lots of case fans
  • And a partridge in a pear tree.

After checking it out, pulling the PCI LAN card, rearranging the WLAN and sound cards so the video card would fit ok (a puny NVIDIA 5500 with 128 MB RAM), and installing an 80 gig SATA-1 hard drive, I fired it up.

Everything worked. It was very very quiet.

I entered the BIOS first and checked it out. The hard drive wasn’t there, until I enabled both SATA controllers. Then it showed up fine after a reboot.

Holy crap! The thing had an AMD64 4400+ X2 processor – I had snagged a 64-bit dual core AMD for nothing!

It gets better – I went to install Linux, and there was already a CD in the drive – the OEM recovery CD, for Windows XP Pro! The license key was on the side of the case, so just for S&G I let it boot.  Hahaha.

After waiting for about 10 minutes into the blue “Initializing Windows Setup” screen with no sign of this changing, I rebooted, threw in a Linux Mint CD, and had it installed about 20 minutes later.  As I figured – maybe I’ll try to load XP in a virtual machine later…

Linux Mint came up fine, with everything found but the WLAN card. It was running a 2.6.17 kernel, so I reloaded with Kubuntu 7.04 to see if a more recent kernel would make a difference. It seemed a little better in seeing the card, but still wouldn’t configure it. Oh well.

I’ll Google it and get it running manually (that’ll be a new post). The SoundBlaster card seems to work well, and every thing else works fine. This thing is pretty quick.

In the mean time, I need to get a better video card for it, for starters. And yeah, “cat /proc/cpuinfo” shows two CPUs.

It just floors me what some people will toss out. I did some checking around, and I figure it had to have cost around $1000. I wonder if they decided to get a new computer with Vista pre-loaded?

Mandriva Cooker Update

Well, I ran urpmi yesterday – partly to see what would come down the pipe, and partly out of hope to fix both a nagging problem with being unable to boot new kernels and a new problem induced since the last update that prevented USB devices from working after about a minute.

Of course, I only discovered the new one after I desperately needed to copy some stuff over to a USB thumbdrive… (Figures.)

So I updated in run-level 3, and was nonplussed when I was unable to go to any other run-level after the update completed. Additionally, at some point during the update, the network interface was nuked, and I had to run “service network restart” in order to get eth0 visible again – running “ifconfig eth0 up” would fail, squawking that it no such device could be found. Once networking was working again, I reran the urpmi command and finished updating with only minor drama. Until I finished, reinstalled my nvidia kernel driver (ver. 9764), and typed “init 5″ in order to bring up the desktop. At this point all I got was a cryptic message that there were no init processes left, or words to that effect.

So, I rebooted, cuz it wasn’t yet enough drama and I was bored.

So, it failed to boot past a new message to enter the init level desired, instead always responding that there were no init processes left (or words to that effect).

So, I booted off my trusty RIP (Rescue Is Possible) liveCD, mounted my boot partition, copied over a working mdadm.conf to /etc/mdadm/mdadm.conf, followed instructions I left myself in the file, and soon had my RAID and LVM volumes up and mounted. I checked the mounted /etc/inittab, and it had next to nothing it it. Luckily, a backup existed, called inittab.rpmsave, which I copied back over the impaired inittab file. I also confirmed that the update had wiped all of the files from /etc/rc0.d through rc6.d directories. I unmounted the volumes and rebooted to see if the system would come up anyway.

It did, in the default run-level 3, but init # would not work. So I ran gdm to start the desktop and it works fine. I am guessing that Mandriva Cooker has been changed to move away from the current initscript process to a different one (sysV to BSD-style, maybe?). I understand that Slackware uses BSD-style initscripts, so maybe this is what is going on. Unfortuately, I am not even remotely familiar with this, so I will start studying.

Good news is, I got KDE 3.5.6, the USB problem is fixed, and the kernel boot problem I have had for a very long time (never found a cure, but am sure it was a Cooker-related problem) is gone. As soon as kernel 2.6.20.1 comes out (the git-6 changelog seems to indicate a lot of SATA and IDE changes), I will throw that on and see how it works. Beryl also got updated, and some quirks with the old version are gone. Nice.

Follow

Get every new post delivered to your Inbox.