I used a Corel Linux CDROM ( http://www.corel.com ), and then upgraded to a more recent version of debian using dselect , but any Debian installation should work fine ( http://www.debian.org )
Source and debian packages are available from http://queenbee.fhcrc.org/Cluster-NFS/ .
If you are running a debian potato or more recent debian distribution,
cd /tmp wget http://queenbee.fhcrc.org/Cluster-NFS/Cluster-NFS.current.deb dpkg install Cluster-NFS.current.deb
Kernel Configuration, kernel 2.2.15 used as example
cd /usr/src wget http://www.us.kernel.org/pub/linux/kernel/v2.2/linux-2.2.15.tar.gz tar -xvzf linux-2.2.15.tar.gz
The MOSIX bundle is available from http://www.mosix.cs.huji.ac.il
The MOSIX installation program also builds the kernel sources, and allows you to configure the kernel. The first time through configure the kernel for the server machines. IE, DO NOT enable BOOTP or rootNFS.
cd /usr/src wget http://www.mosix.cs.huji.ac.il/ftps/MOSIX-0.97.4.tar.gz mkdir MOSIX cd MOSIX tar -xvzf ../MOSIX-0.97.4.tar.gz ./mosix.install
cd /usr/src/linux make mrproper make menuconfig ## or make config or make xconfig
To enable BOOTP and rootNFS, set the kernel configuration options:
[Code Maturity Options] ® [Prompt for development and/or incomplete code/drivers]
[Networking Options] ® [IP: kernel level autoconfiguration]
[Networking Options] ® [IP: kernel level autoconfiguration] ® [BOOTP support]
[Filesystems] ® [Network File Systems] ® [NFS filesystem support]
[Filesystems] ® [Network File Systems] ® [NFS filesystem support] ® [Root file system on NFS]
make dep make bzImage
Insert a formatted floppy in the drive and use ``dd'' to copy the kernel image to the floppy:
cd /usr/src/linux dd if=arch/i386/boot/bzImage of=/dev/fd0
On our system, the cluster nodes are assigned ip numbers 10.0.0.1 - 10.0.0.8 so this file only needs one range:
1 10.0.0.1 8
On our system, the cluster nodes are named bee1 through bee8, so /etc/hosts includes the lines
10.0.0.1 bee1 biostat1 b1 10.0.0.2 bee2 biostat2 b2 10.0.0.3 bee3 biostat3 b3 10.0.0.4 bee4 biostat4 b4 10.0.0.5 bee5 biostat5 b5 10.0.0.6 bee6 biostat6 b6 10.0.0.7 bee7 biostat7 b7 10.0.0.8 bee8 biostat8 b8
The BOOTP daemon uses /etc/bootptab to respond to the queries made by the compute nodes using the ethernet MAC address of the node.
Contents our /etc/bootptab :
.beehive:\
:bf=null:\
:dn=beehive.fhcrc.org:\
:ds=10.0.0.1:\
:gw=10.0.0.1:\
:hd=/:\
:hn:\
:nt=10.0.0.1:\
:rp=/:\
:sm=255.255.255.0:\
:to=auto:
bee2:ht=1:ha=00A0C98A1040:tc=.beehive:
bee3:ht=1:ha=00A0C98CF35A:tc=.beehive:
bee4:ht=1:ha=00A0C945BE60:tc=.beehive:
bee5:ht=1:ha=00a0c945c086:tc=.beehive:
bee6:ht=1:ha=000102241106:tc=.beehive:
bee7:ht=1:ha=000102241108:tc=.beehive:
bee8:ht=1:ha=00010224110a:tc=.beehive:
/ 10.0.0.1(rw,no_root_squash) / 10.0.0.2(rw,no_root_squash) / 10.0.0.3(rw,no_root_squash) / 10.0.0.4(rw,no_root_squash) / 10.0.0.5(rw,no_root_squash) / 10.0.0.6(rw,no_root_squash) / 10.0.0.7(rw,no_root_squash) / 10.0.0.8(rw,no_root_squash)
ALL: 10.0.0.1 ALL: 10.0.0.2 ALL: 10.0.0.3 ALL: 10.0.0.4 ALL: 10.0.0.5 ALL: 10.0.0.6 ALL: 10.0.0.7 ALL: 10.0.0.8
Files named filename$$CLIENT$$ will be served when any client requests the file filename .
On our system, we have
| /etc/exports$$CLIENT$$ | empty file |
| /etc/fstab$$CLIENT$$ | see below |
| /etc/inetd.conf$$CLIENT$$ | comment out bootp entry |
| /etc/mospe$$CLIENT$$ | soft link to a non-existant file |
| /etc/nsswitch.conf$$CLIENT$$ | remove ``dns'' from hosts entry |
| /etc/resolv.conf$$CLIENT$$ | remove name server entries |
| /etc/syslog.conf$$CLIENT$$ | see below |
| /etc/init.d/apache$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/apache-ssl$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/apcd$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/atd$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/cron$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/elmd$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/exim$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/gdm$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/genpower$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/gpm$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/lpd$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/mountall.sh$$CLIENT$$ | see below |
| /etc/init.d/network$$CLIENT$$ | see below |
| /etc/init.d/proftpd$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /etc/init.d/xfs$$CLIENT$$ | soft link to DoNotExecuteOnClients |
| /usr/sbin/ssmtp$$CLIENT$$ | see below |
Specific files
Forward all syslog information to the master node queenbee .
# forward everything to queenbee *.* @queenbee
This file needs to be modified so that it mounts the root nfs filesystem.
--- /etc/init.d/mountall.sh Mon Mar 13 10:07:59 2000 +++ /etc/init.d/mountall.sh$$CLIENT$$ Sat Mar 25 01:40:17 2000 @@ -8,8 +8,8 @@ # # Mount local file systems in /etc/fstab. # -[ "$VERBOSE" != no ] && echo "Mounting local file systems..." -mount -avt nonfs,noproc,nosmbfs +[ "$VERBOSE" != no ] && echo "Mounting ALL file systems..." +mount -avt noproc,nosmbfs # # We might have mounted something over /dev, see if /dev/initctl is there.
Usually, this file is used to start up the network connections. On the clients the network is already started by the time we get this far, so all that needs to be done is to set up the loopbback device:
echo "Setting up network " echo " current status " ifconfig /sbin/ifconfig lo 127.0.0.1 /sbin/route add -net 127.0.0.0 netmask 255.0.0.0 lo
I have not set up the nodes to connect to the outside, so mail creation requests need to be forwarded to the master node ( qb is an alias for queenbee ):
#!/bin/bash # funnel all email to queenbee, which forwards it all on to lynx ssh qb /usr/sbin/ssmtp $*
On the BioHive cluster, the following per-node files exist:
| /dev$$IP=10.0.0.2$$ | may not be necessary |
| /etc/X11/XF86Config$$10.0.0.2$$ | depends on VGA card used |
| /etc/adjtime$$IP=10.0.0.2$$ | may not be necessary |
| /etc/devices$$IP=10.0.0.2$$ | modified on bootup by each machine |
| /etc/hostname$$IP=10.0.0.2$$ | machine specific |
| /etc/mtab$$IP=10.0.0.2$$ | created by ``mount'' on each machine |
| /etc/rmtab$$IP=10.0.0.2$$ | created by ``mount'' on each machine |
| /tmp$$IP=10.0.0.2$$ | needed for temp files, lock files |
| /var$$IP=10.0.0.2$$ | needed for lock files, spool files |
Most of the files can be created by echo > filename\$\$10.0.0.2\$\$ .
The directories should be created by
cp -r \dirname \dirname\$\$IP=10.0.0.2\$\$