ClusterNFS: Simplifying Linux Clusters

Gregory R. Warnes, Ph.C.
Fred Hutchinson Cancer Research Center

May 11, 1999

Hydra.png

The cNFS Mascot, Henry the Hydra


Outline

  1. Why Linux Clusters?
  2. What Makes Clusters Hard?
  3. The Tools
  4. MOSIX + ClusterNFS in Action: The BioHive Cluster
  5. Cluster Recipe
  6. Ideas and Future Plans

  7.  

Why Use Linux Clusters?

  1. High performance
  2. Low Cost
  3. Relatively Easy to Build and Maintain

  4.  

Why Linux Clusters? High performance
Elapsed Time
Elapsed.png



Why Linux Clusters? High performance

Speedup
Speedup.png


Why Linux Clusters? Low Cost

hdnw.png


What Makes Clusters Hard?

  1. Setup - Administrator
  2. Maintenance - Administrator
  3. Running Jobs - Users

The Tools: MOSIX

Description:
MOSIX is an enhancement to the Linux kernel that provides adaptive (on-line) load-balancing and memory ushering between x86 Linux machines. It uses preemptive process migration to assign and reassign the processes among the nodes to take best advantage of the available resources.

Translation:
MOSIX moves processes around the cluster to balance the load, using faster machines first.

Source:
Amnon Barak, CS Department of the Hebrew University of Jerusalem

URL:
http://www.mosix.cs.huji.ac.il

The Tools: ClusterNFS (cNFS)

Description:

 

 
 
 
 
 

cNFS is a patch to the standard Universal-NFS server (uNFS) code that ``parses'' file requests to determine an appropriate match on the server.

Whenever a client requests the file filename, the server check for the existence of one of the following files, returning the first match:

filename$$UID=xxxx$$ user's id 
filename$$GID=xxxx$$ user's group id 
filename$$HOSTNAME=ssss$$ client hostname 
filename$$IP=xxx.xxx.xxx.xxx$$ client ip number 
filename$$CLIENT$$ always matches 
filename default 

Example:

When client machine bee3 asks for file /etc/hostname it gets the contents of /etc/hostname$$HOST=bee3$$.

Source:
Gregory Warnes, Fred Hutchinson Cancer Research Center
URL:
http://queenbee.fhcrc.org/ClusterNFS

MOSIX + ClusterNFS in Action: the BioHive Cluster

cluster.png
http://queenbee.fhcrc.org


Making Clusters Easy : MOSIX + ClusterNFS

  1. Setup - Administrator
  2. Maintenance - Administrator
  3. Running Jobs - Users

Making Clusters Easy for Users: MOSIX

MOSIX (http://www.mosix.cs.huji.ac.il) is a dynamic load-balancing system that transparently migrates tasks between machines.
  1. Users log into ``master'' node
  2. Jobs started on the master node automagically migrate to fastest / least loaded machine.
  3. Job Control (ps, top, kill) occurs as if whole cluster is one system
Users never need to know details of cluster configuration.


Diskless Servers: Traditional Method

Server: Client:

Diskless Servers: Traditional Method

This method requires a separate root directory structure for each node.
Hard to Set Up
Difficult to Maintain

Diskless Servers: ClusterNFS Method

Server: Client:

Diskless Servers: ClusterNFS Method

ClusterNFS allows all machines (including server) to share the root filesystem

Diskless Servers: ClusterNFS Method


ClusterNFS Recipe (Sketch)
On the Server

  1. Install and configure Debian Linux
  2. Install ClusterNFS
  3. Download and Compile MOSIX and Kernel, enabling BOOTP and RootNFS.
  4. Copy the Kernel to Floppies
  5. Add entries for each client to
  6. Create files that are the same for all clients, filename$$CLIENT$$.
  7. Create files that are specific to individual clients filename$$IP=xxx.xxx.xxx.xxx$$
  8. reboot server to restart all services

  9.  
The complete recipe is available at Recipe.html.

ClusterNFS Recipe
On the Client

  1. Insert boot floppy
  2. Boot
  3. Record Ethernet MAC address displayed by kernel
  4. Add to \etc\bootptab on server
  5. Reboot
The complete recipe is available at Recipe.html.

Plans and Ideas


(c) 2000 by Gregory R. Warnes