This page discusses G-RSM on COMPAS Linux clusters at SIO.
COMPAS
- Comipler paths are different for the rack your model runs on. See /etc/motd on COMPAS. The installer script recognizes that MACHINE=linux (set in libs/configure-libs). To use correct compilers for your rack, you have to edit libs/opt_libs/options-linux-mpi and options-linux-single before running configure-libs and rsm/opt/options-linux-mpi and options-linux-single before running configure-model.
- The path to mpirun is different for rack4. It is /opt/mpich/pgi/bin/mpirun and you have to edit runscr/r_fcst.in .
- You need to a file called "node_list" in your $HOME directory. This is a list of rack and node numbers for your run and used by runscr/r_fcst as an argument to mpirun.
- If you're using rack 0 and 1, you have to edit opt/options-linux-mpi. Make sure F77=pgf90 not mpif90. This is particularly important if you're using the newest version of RSM/GSM.
COMPAS2
compas2 is a linux cluster with ethernet instead of myrinet. Aki changed two things to make G-RSM run on the machine.
1) delete "-lgm" option
2) setting a environmental parameter in ~/.bashrc
P4_GLOBMEMSIZE=xxx ; export P4_GLOBMEMSIZE
xxx is a number enough to run models.
3) modify gsm/configure-model
MPICH_DIR=/opt/mpich/pgi ( <- MPICH_DIR=/opt/mpich/myrinet/pgi )
4) modify runs/runscr/fcst(.in)
/opt/mpich/pgi/bin/mpirun ( <- /opt/mpich/myrinet/pgi/bin/mpirun )
5) runs/runscr/post ( if "batch mode" is selected )
if [ -z "ssh -f compas-5-0-eth2" ]
ssh -f compas-5-0-eth2
:
6) node_list
compas-5-x-eth2 ( high-speed ; recommended, to login to compas-5-x is needed )
or
compas-5-x (normal speed)
Atlas (Contribution by Kei) ===== This step is not necessary, except the step 3.
1. Check mpif90's location by $ where mpif90 or $ which mpif90.
The top low of the response should be "/share/apps/mpich1/pgi/bin/mpif90"
Otherwise, setenv your PATH in order to do so.
2. Make a directory, go there, and do $cvs co install, by your own preference. Then the install will be stuck somewhere.
3. Edit libs/lib/w3lib_linux/nainit.f
Find two "status(12)" and changed to "status(8)".
Go to libs/, make clean, make.
(if it fails in con, remove the line contains "era40_19900309.grib" in con/Makefile. Fixed in latest version)
4. Edit rsm(gsm)/configure-model
Find "MPICH_DIR=/usr/local/mpich" and change to
"MPICH_DIR=/share/apps/mpich1/pgi"
5. Edit rsm(gsm)/opt/opt-linux-mpi
Remove "-L/opt/gm/lib" in LOAD_FLAGS
Remove "-lgm" in EXTRA_LIBS
Add "-Mextend" in FORT_FLAGS and LOAD_FLAGS (just in case)
6. Go to rsm(gsm), do $configure-model, make clean, make
7. [Only for rsm-mpi] Because of make version problem,
automatic process won't work properly in case of RSM-MPI.
So, do the make process by hand following the rule in "src/Makefile".
$ cd rsm ; make clean
$ cd src ; make rsm (when stuck, remove "p2sig mtn" from RSM_PROGS in
Makefile)
$ cd rmpi ; make
$ cd ../sfcl_par ; make ; rm -f *.o ; make (because the first make would link unnecessary files)
$ cd ../rsml_par ; make ; rm -f *.o ; make (if stuck, check "stdamp2.o" in Makefile and ../rsml/Makefile. Fixed in the latest version)
$ cd ../rgsm_par ; make ; rm -f *.o ; make
$ cd ../rfcst_par ; make ; rm -f *.o ; make
$ cd ../rpgb_par ; make ; rm -f *.o ; make
$ cd ../ ; make r_rescon
8. Go to rruns(gruns)/runscr, edit r_fcst(fcst)
if [ ecpc_linux = compas_linux -a .......=> if [ ecpc_linx ecpc_linux -a .......
/opt/mpich/myrinet/pgi/bin/mpirun ==> /share/apps/mpich1/pgi/bin/mpirun
9. Run whatever run-script you like! (rsm, gsm, rsim, etc.)
(You need to have ~/node_list as similar as the case of using compas)