Fornax

Maintenance

Power off Fornax

This step should be performed by Computer Officer as it requires root permission.

1. Login to the Head Node to List Jobs and Close Job Queue Login to head node as root account, check for any running jobs, close job queues and terminate all running jobs:

ssh fornax.nus.edu.sg –l <rootaccount>

Kill all job and queue 2. Shutdown Compute Nodes On the head node, shutdown all compute nodes and check whether all compute nodes has been off. The output must be “noping”.

psh shutdown -h now

3. Stop PBS Pro services This command may take several minutes to complete.

/etc/init.d/pbs stop

4. Shutdown Head Node Login to head node and run shutdown command:

ssh fornax.nus.edu.sg –l <rootaccount>
shutdown –h now

Power on Fornax

1. Power On Head Node Login and check whether all services are running well.

/etc/init.d/pbs status
ps ax | grep zabbix
service nfs status

You will see following

rpc.svcgssd is stopped
rpc.mountd (pid 2107) is running...
nfsd (pid 2122 2121 2120 2119 2118 2117 2116 2115) is running...
rpc.rquotad (pid 2103) is running...

2. Power On Compute Nodes Turn on compute nodes manually.

Hardware

Node	Count	CPU	Memory	Storage	Comment
fornax.nus.edu.sg	1	2 x 16 (AMD 7513)	128 GB	14 TB	head node
smallmem	4	2 x 32 (AMD 7551)	128 GB	250 GB	fornax-c01 to c04
largemem	8	2 x 64 (AMD 7742)	512 GB	250 GB	fornax-c05 to c12
smallmem	1	2 x 32 (AMD 7543)	512 GB	250 GB	fornax-c13
genoa	7	2 x 32 (AMD 9354)	384 GB	200 GB	fornax-c14 to c20
fornax-scratch	1	N/A	128 GB	21 TB	scratch node

Software configuration

• Application source directory: /home/app/source
• Installation directory: /home/app
• Installation prefix: /home/app/<appname>
• Since /home/app directory is shared to all nodes, we do not need to compile or build the application in all nodes.
• There are two ways to load the settings:
  • Automatically load – write application profile in /etc/profile.d/
  • Manually load using environment modules – recommended, easy to switch between    different version.
• Environment modules directory: /home/app/modulefiles/
• An example of a simple module file:

   #%Module1.0
    prepend-path PATH /apps/<apps>/bin
    prepend-path LD_LIBRARY_PATH /app/<application>/lib
    setenv <APP>_HOME /app/<application>