Skip to content
{ 2010 02 22 }

Expanding Mac Mini Disk through iSCSI and big cheap drives

We use a Mac Mini with OS X 10.5 to serve our whole house with music, movies, television shows, and photographs. The Mini is perpetually out of disk space because it only has room for one 2.5″ internal hard drive (two if you remove the optical drive) and those notebook hard drives have smaller capacities, are slower, and cost a lot more than big capacity 3.5″ drives.

I also wanted to practice by Linux server skills, Xen virtualization skills, and use a server placed in our basement to house several big drives and serve it’s storage up to the Mini over the network. My initial plan was to use NFS and simply serve the big disks up over the network to the Mini (and other systems) using the nearly ubiquitous NFS protocol (NFS is available on all OS X, Linux, BSD, and Solaris systems). A friend recently challenged me to learn to do this with iSCSI instead and enjoy better performance as well as learning some small SAN technology.

Details for Geeks below the fold…

I have a low power consuming TranquilPC BBS that I bought a year ago for this specific purpose, but any old PC tower will serve effectively in this role. It does not have to be fast, nor new, nor particularly full of graphics features; re-using old hardware is actually best for this task.

I set up a Xen aware openSuSE Linux instance on the bare server (Centos or Fedora might be easier in hindsight). This is known in Xen virtualization terminology as a dom0 and it should do nothing except run the Xen virtual machines you configure later. You can add virtual machines of several operating system flavors so long as you have processor cycles, memory, and drive space to spare. My plan is to run Windows server, Centos servers, openSuSE servers, and my OpenFiler server all as virtual machines, known as domU in Xen terminology.

Installing openSuSE 11.1 with Xen virtualization on my TranquilPC BBS (mine is an older Atom330 based one) server:

    I had to upgrade the BIOS in my Atom330 based TranquilPC before it would work with my x86-64 (64 bit) openSuSE install DVD; so I had to burn a CD with the updated Intel BIOS, then install from a DVD iso of the 11.1 openSuSE operating system. Whatever system you use, make sure your install image matches the abilities of your hardware.

    In preparing the first drive, I created a 101MB /boot partition with ext3 filesystem and the entire remainder of that first drive as an Logical Volume Management (LVM) partition. That partition became the first disk in my first volume group (vg00 because of old AIX and HP-UX experiences) and I created a dom0_root (ext3 8GB) and a dom0_swap (swap 2GB; match your memory) logical volumes within it.

    During the installation, selected only Base System, Console Tools, Yast System Admin, Yast Installation Packages, and Software Management (manually de-selecting several other things) as I want this server to mostly be a headless server in the basement and need no graphics support.

    I configured the root password, IP address, and hostname manually, and made note of it because most configuration will be done through an ssh terminal session where I grab all the useful server software using the zypper command:

    zypper rr repo-non-oss
zypper rr repo-debug
zypper rr repo-oss
zypper rr repo-source
zypper rr repo-update
zypper rr "openSUSE 11.1-0"
zypper ar http://download.opensuse.org/distribution/11.1/repo/oss/ http://download.opensuse.org/distribution/11.1/repo/oss/
zypper ar http://download.opensuse.org/distribution/11.1/repo/non-oss/ http://download.opensuse.org/distribution/11.1/repo/non-oss/
zypper ar http://download.opensuse.org/update/11.1/ http://download.opensuse.org/update/11.1/
echo "YES" | zypper up
echo "YES" | zypper up
echo "install ipv6 /bin/true" >> /etc/modprobe.d/ipv6
sync ; sync ; sync ; reboot
echo "YES" | zypper install iputils iftop man net-snmp bind-utils procinfo vim-enhanced vim-data tcpdump rsync wget ksh findutils findutils-locate man man-pages postfix procmail sysstat sudo telnet termcap terminfo xntp cron tcsh lynx emacs patch ethtool vlan lsof bc mailx dos2unix dosfstools zsh ipmitool OpenIPMI vim-data acl tcptraceroute mcrypt xfsprogs acpid xterm tree

echo "YES" | zypper install xen xen-libs xen-tools xen-tools-ioemu kernel-xen vm-install libvirt libvirt-python

chkconfig avahi-dnsconfd off
chkconfig splash off
chkconfig splash_early off
chkconfig ntp off
chkconfig ipmi on
chkconfig acpid on
chkconfig powerd on

vi /etc/sysconfig/network/ifcfg-eth0 # (set your IP, netmask, etc)
vi /etc/resolv.conf # (set up your nameservers)
vi /etc/sysconfig/network/routes # (set your gateway)
vi /boot/grub/menu.lst # (change default to 0)
vi /etc/xen/xend-config.sxp # (uncomment the network-script network-bridge line so domUs can enjoy bridged networking)
sync ; sync ; sync ; reboot
echo "YES" | zypper up

The server with all the big disks in it needs to become an iSCSI target in SAN terms. FreeNAS, OpenFiler, and IET (iSCSI Enterprise Target) are all free and work with my hardware. I chose OpenFiler on the recommendation of friends, and because it serves up NFS, CIFS/Samba, and WebDav as well as iSCSI and it can handle software RAID and some volume management on the back end. In hindsight, I should probably have done IET instead; as I am more comfortable in the command line than I am with a web GUI. Configuring OpenFiler through the web GUI was very frustrating.

You can run OpenFiler as a bare metal install (where OpenFiler is itself the operating system on your big disk server), and this may be an attractive option if you are working with a server with very limited memory and CPU resources. I wanted greater flexibility and to have my little TranquilPC do more than just serve up storage, so I virtualized with Linux, Xen, and chose to run my OpenFiler instance as a domU (virtual machine).

Installing OpenFiler virtual server on your new Linux server:

    wget http://www.rpath.org/rbuilder/downloadImage?fileId=26671
wget http://www.rpath.org/rbuilder/downloadImage?fileId=26667
lvcreate -L 5G -n openfiler vg00 # (in my system that defaults to /dev/sdc, the first drive)
pvcreate /dev/sdb # (add another drive)
pvcreate /dev/sda # (add another drive)
lvcreate -l 1T -n sda vg00 /dev/sda # (size your volumes less than or equal to the space on each drive)
lvcreate -l 1T -n sdb vg00 /dev/sda # (only add second volume on second drive for mirroring)
mkfs.ext3 /dev/vg00/openfiler
e2label /dev/vg00/openfiler root
mount /dev/vg00/openfiler /mnt/
cd /mnt/
tar xzpvf ~/openfiler%2F16322%2Fopenfiler-2.3-x86_64.tar.gz
mkdir -p /opt/xenboot/openfiler
cp -p /mnt/boot/config-2.6.21.7-3.20.smp.gcc3.4.x86_64.xen.domU /opt/xenboot/openfiler/
cp -p /mnt/boot/initrd-2.6.21.7-3.20.smp.gcc3.4.x86_64.xen.domU.img /opt/xenboot/openfiler/
cp -p /mnt/boot/System.map-2.6.21.7-3.20.smp.gcc3.4.x86_64.xen.domU /opt/xenboot/openfiler/
cp -p /mnt/boot/vmlinuz-2.6.21.7-3.20.smp.gcc3.4.x86_64.xen.domU /opt/xenboot/openfiler/
cd ; umount /mnt
vi /etc/xen/vm/openfiler # (configure the xen virtual machine instance)
iname = 'openfiler'
memory = '512'
kernel = '/opt/xenboot/openfiler/vmlinuz-2.6.21.7-3.20.smp.gcc3.4.x86_64.xen.domU'
ramdisk = '/opt/xenboot/openfiler/initrd-2.6.21.7-3.20.smp.gcc3.4.x86_64.xen.domU.img'
disk = [ 'phy:/dev/vg00/openfiler,xvda,w', 'phy:/dev/vg00/sda,xvdb,w', 'phy:/dev/vg00/sdb,xvdc,w' ]
root = "/dev/xvda ro"
vfb = [ "type=vnc,vncunused=1,keymap=en-us" ] 
vif = [ "mac=00:16:3e:38:75:88,bridge=xenbr0" ]
on_reboot = 'restart'
on_crash = 'restart'

xm create /etc/xen/vm/openfiler
xm list

    Now you can point a web browser at the new OpenFiler server (port 446 on a DHCP provided address), log into it as openfiler with the password “password” and then change the password to something more sensible (Accounts/Admin Password).

    Change the IP address to a static one you want to use going forward (System/Network Setup). Change host name to “openfiler” or whatever you want.

    Create your first Volume Group, comprised of any Block Devices you passed to this domU from the dom0. Add Volumes that will be your iSCSI target LUNs, enable iSCSI target service (Services), and associate your Volume with a LUN and set up Network Access for it (Volumes/iSCSI Targets) using the LUN mapping and Network ACL sub menus.

Teach the Mac Mini to become an iSCSI Initiator (clients initiate the request for disks) by adding the GlobalSAN iSCSI initiator software, rebooting, and configuring it.

    Use System Preferences/globalSAN iSCSI to configure your portal, (the IP address of your openfiler server), then click on Targets and your iSCSI target should show up. Connect it; DiskUtility should pop up because the new drive is not formatted.

Use DiskUtility to partition and create a filesystem. Copy your media to the new iSCSI disk.

Implement a backup plan because RAID does not protect from human error.

Posted by snolan on Monday, February 22, 2010, at 11:35 am. Filed under Computers, Personal. Follow any responses to this post with its comments RSS feed. You can post a comment or trackback from your blog.

{ 1 } Comments