8th International Regional Spectral Model Workshop

as a part of 4th ICTP Workshop on the Theory and Use of Regional Climate Models.

Trieste, March 3-14, 2008

NOTES FOR THE EXPERIMENTS - DOWNSCALING OF SEASONAL FORECASTS

Hideki Kanamaru, Ana Nunes, Liqiang Sun, Masao Kanamitsu, and RSM workshop committe members

FIRST WEEK

Design

1. Experimental Climate Prediction Center (ECPC) of Scripps Institution of Oceanography (SIO) produces 12-member ensemble seasonal forecasts every month by ECPC Global Spectral Model (GSM). In this training we use forecasts completed on February 7th, 2008. The forecasts extend to the end of August.
2. 12-member ensembles were created with 4 initial conditions and 3 SST forecasts.

	initial condition 1	IC 2	IC 3	IC 4
SST plus	n01	n02	n03	n04
SST mean	l01	l02	l03	l04
SST minus	v01	v02	v03	v04

Roughly speaking plus and minus SST is one standard deviation plus and minus from the mean SST out of a number of SST forecasts.

1. For this exercise, we will downscale the forecasts on three regional domains.
   1. Latin America 60km resolution 108x89 grid cells, 85W-30W, 35S-10N
   2. Africa 60km 128x79 20W-50E, 20S-20N
   3. Asia 60km128x79 60E-130E, 0-40N
2. Downscaling simulation will cover the months of March to May. This should take less than 40 hours on a PC.
3. Each group/domain will run up to 10 member ensembles, depending on computer availability, and number of participants.
4. We will learn
   1. how to install the model
   2. how to run the model
   3. how to post-process the output
   4. how to visualize the output with GrADS
   5. how to interpret the results.

Set-up

1. set your path to include current directory and set stack size to unlimited by adding following lines to your ~/.cshrc
   1. set path = ( . $path )
   2. limit stacksize unlimited
2. Type $ source ~/.cshrc to make the change effective.
3. Copy ecpc_exp.tar from xxx to your home directory or scratch space.
4. Untar ecpc_exp.tar $ tar xvf ecpc_exp.tar
5. go to libs directory $ cd ecpc_exp/libs
6. Open configure-libs… Actually you don’t have to edit anything in this file!
7. Type $ configure-libs, $ make clean – this will clean up the directory
8. Type $ configure-libs – this will create Makefiles
9. Type $ make
10. go to ecpc_exp/rsm $ cd ../rsm
11. open configure-model with an editor $ vi configure-model
12. edit LIBS_DIR (provide full path)
13. edit MODEL_DEFINE (provide full path; note grid numbers will be different for different domains)
14. close configure-model and $ configure-model, $ make clean – this will clean up the directory
15. $ configure-model
16. $ make
17. go to rruns directory $ cd ../rruns
18. open configure-scr
19. eidt SRCS_DIR (provide full path)
20. edit RGN, RES, PROJ
21. open expscr/rsim.in template
    1. make sure YYYYS, MMS, DDS, HHS (2008,03,01,00) and YYYYE, MME, DDE, HHE (2008,06,01,00) are correct
    2. edit path to INITIAL_SIG_GLB, note each run should point to different sigma file under different directory (each GSM ensemble member)!
    3. edit path to INITIAL_SFC_GLB Note each run should point to different initial sfc file under different directory (each GSM ensemble member)!
    4. Edit SST_ANL and SST_ANL_T0
    5. Edit ICE_ANL and ICE_ANL_T0 . Set it to a climatology file.
    6. Edit INCHOUR if necessary
22. $configure-scr rsim
23. for Asia domain group, time step has to be smaller… open rsim script (which was just created) and edit DELTAT_REG_SPRING to 300, instead of 400sec
24. run the model by typing $ rsim
25. Sit back and relax for 40 hours – no! - you have to check the simulation periodically to make sure the model did not crash. If the model stopped working for some reason ask one of us to seek help.

SECOND WEEK DAY 1

1. Review of model configuration. See above.
2. Installing the model from scratch using CVS. See PPT. GRSM_CVS_2008.ppt
3. Post-processing (grmean, grvar, etc)
4. All notes for the day can be found in /home/nfs1/smr1934/post/var.list.txt and a number of scripts under the same directory. Here's the summary.

GROUP LEADER/COORDINATOR/PRESENTER

ASIA
SYED and BASANTA !!!

AFRICA
MATAYO and ABDOULAYE !!!

S AMERICA
DAVID and SIRAJ !!!

MAKING MONTHLY/DAILY MEANS

1. wgrib to extract variables
2. grmean to make means
3. force_grib_date_mon to set correct time for the mean files
4. grmap to create control files
5. run gribmap after fixing control file
6. use grads to visualize

File name convention

monthly:
$region_$ensemble_$var_$yyyy$mm.grib
e.g.
af_l01_tmphag_200803.grib
sa_n02_tmphag_200804.grib
as_v03_tmphag_200805.grib

daily:
$region_$ensemble_$var_$yyyy$mm$dd.grib
e.g.
af_n03_prate_20080302.grib

List of key variables

186 prate
193 tmphag
(grib record number)

wgrib r_pgb.ftxxx

to get the list of variables in a pgb file.

wgrib -V r_pgb.ftxxx

to get a long list of variables in a grib file.

example: extracting record number 186 (prate) from a grib file into another grib file:

wgrib -d 186 /scratch/RSM/rruns/SouthAmerical01/2008/r_pgb.ftxx -o output_grib -grib

try extract_var.sh and extract_var_tmphag.sh in /home/nfs1/smr1934/post

Files to be used for making means

for monthly means --- tmphag

March 2008
use ft06 to ft744

April 2008
use ft750 to ft1464

May 2008
use ft1470 to 2208

for daily means --- prate

ft06, 12, 18, 24 for 20080301
ft30, 36, 42, 48 for 20080302
etc

to use libs/etc/grmean

prepare a text file with the list of input pgb files.

ls prate/prate.ft?? > prate.list.01
ls prate/prate.ft1?? >> prate.list.01

and then type
grmean -s prate.list.01 -o output_grib

Scripts for grmean

copy files from
/home/nfs1/smr1934/post

./test.sh for tmphag monthly means
./test_prate.sh for prate daily means.

change the path to your extracted prate.ftxx files and tmphag.ftxx files
and libs/etc/grmean and libs/etc/force_grib...

also change region name two letter acronym, sa, af, as
change ensemble member name, l01, n02, v03, etc

GRMAP

Next we create ctl files to display the mean files!

TO DISPLAY TMP 2m CORRECTLY

open .ctl file
go to this line:

TMP 0 11,105,2 Temperature (K)

by default it appears as

0 11,105,0

this is not correct! replace the last 0 with 2 (means 2m).
and run $ gribmap -0 -i xxx.ctl
and open xxx.ctl in GrADS

when using grmap for the original r_pgb.ftxx files...

use -cnoah option so you can see 2m temperature, surface winds, soil moisture, etc correctly.

but -cnoah option does not work if you have extracted variables into a new grib file.

how to go from monthly/daily means to ctl files

/afs/ictp/home/a/anunes/RSM/ecpc_exp/libs/etc/grmap -g0 sa_l01_tmphag_20080301.grib

open sa_l01_tmphag_20080301.grib.ctl

change

TMP 0 11,105,0 Temperature (K)
to


TMP 0 11,105,2 Temperature (K)


because we have edited a ctl file we have to recreate idx file associated with the ctl file:

gribmap -0 -i sa_l01_tmphag_20080301.grib.ctl
this is gribmap not grmap. gribmap comes with wgrib.

The same process for precip daily means but no need to fix the control file.

DAY2

Tuesday, we will discuss how to display multiple grib files in one control file and create an animation.

how to display topography (hgtsfc)

how to take area average of precipitation and plot a time series

how to take ensemble means and spread (stdev). first transfer all mean files into one machine and use the same grmean and grvar_ver2 (standard deviation)

Other requests I received:

• explain what other output files mean from the model.
• how to restart the model
• we'll provide climatology file so you can look at anomalies.
• GSM forecast map that drove our regional climate model

PLAN for the next two days
1. cut-off time for additional ensemble member runs 4:30pm. exception: people running the first member.
2. we start copying mean files of all ensemble members to one common computer each group.
3. The file transfer should be done by 5:30pm and
4. we can start looking at ensemble means and spread tomorrow morning!

TEMPLATES for CTL FILES

1. you are able to open a number of grib files uisng a template function in GrADS
2. open sa_l01_prate_20080301.grib.ctl
3. edit the first line:
dset /home/nfs1/smr1934/post/sa_l01_prate_2008%m2%d2.grib
using wild card for two-digit month %m2 and two-digit day %d2
4. insert this line:
options template
5. change this line (92 and 1dy)
tdef 92 linear 00Z1MAR2008 1dy
6. close the control file
7. run gribmap -0 -i on the control file
8. open in grads
9. set t 1 92
10. d prate
11. you would see evolution of precipitation

12. you can do the same for monthly temperature file.
tdef line should read
tdef 3 ..... 1mo
which means interval 1 month and 3 time steps.

DISPLAYING TOPOGRAPHY

1. run grmap on one of the r_pgb.files
2. open it in grads (1st ctl file)
3. d hgtsfc (for surface height)
4. open sa_l01_prate_20080301.grib.ctl (2nd ctl file)
5. set gxout shaded for color shades
6. d prate.2 for prate plot
7. set gxout contour for contour lines
8. d hgtsfc
9. if you want to mask ocean,
10. d maskout(hgtsfc, land-1)
11. if you open multiple ctl files in one grads session, you have to specify which ctl file the variable is coming from.

AREA AVERAGE

1. open prate ctl file
2. set t 1 92
3. d aave(prate,lon=-70,lon=-50,lat=-15,lat=0)
longitude has to come first.

TO USE GRADS SCRIPTS

1. add this line to your ~/.cshrc
setenv GASCRP /home/nfs1/smr1934/post/grads_scripts
2. source ~/.cshrc
3. start grads
4. you will be able to use functions such as cbar to display color bars for color shades.

DISPLAY TIME SERIES OF AREA AVERAGE PRECIPITATION

use tloop!
1. open prate ctl file
2. set t 1 92
3. set x 1
4. set y 1
5. d tloop( aave(prate,lon=-70,lon=-50,lat=-15,lat=0)*60*60*24 )
6. now you should be seeing a line plot for the time series of area average precipitation

NUMBER OF RAINY DAYS

test_gridpoint.gs - this script looks at an individual grid point, and goes through all 92 days and evaluate each day and check if there is precipitation above xxx mm/day. And write out the number of rainy days for that grid. 109x98 numbers - rainy days for all grids

test.gs - another version - if you are interested in area average instead of one grid point.

test_gridpoint_ver2.gs -- updated version! also writes number of rainy days into a text file file.txt

COPY ALL DAILY MEAN PRECIP and MONTHLY TMPHAG FILES UNDER respective directories

/home/nfs1/smr1934/SA for south america
L01, V02, N03, etc
/home/nfs1/smr1934/AS for asia
/home/nfs1/smr1934/AF for africa

South America

	1	2	3
n	OK
l	OK	OK	OK
v	OK	OK

Asia

	1	2	3
n	OK	OK	OK
l	OK	OK	OK
v	OK	OK	OK

Africa

	1	2	3
n	OK	OK
l			OK
v		OK	OK

PLANS for DAY3

PRIORITIES

• start looking at ensemble means - you can use grmean for this analysis
• start looking at ensemble spread - you can use grvar_ver2 for this - this program works in the same way as grmean but calculates standard deviation.
• compare with observed climatology
• compare with GSM forecasts that drove the regional model

Other requests I will get to:

• fix grmap on some machines
• how to plot accumulated daily precip
• possibly get GSM forecast in GRIB format?
• ECMWF model output as inputs to RSM?

OBSERVED CLIMATOLOGY FILE

go to
/home/nfs1/smr1934/climatology

MAPS from IRI

Go to http://iri.columbia.edu
Top right corner, middle icon: forecasts

• Seasonal forecasts from the driving GSM (used for our experiments): Click on Individual AGCM Predictions. For precipitation, there are six panels, the right column is for total precipitation. For temperature, the right column is temperature itself not anomalies. Also available are probabilistic forecasts.
• Skill of the driving GSM (used for our experiments): Click on Multi-Model Ensemble Predictions -> individual model skill maps on right column. Choose ECPC-NOAH. Verification against CRU 1950-2000 observation.

FYI - what's available on ECPC web site.

ECPC seasonal forecasts (GLOBAL)
http://ecpc.ucsd.edu/projects/GSM_seasons.html
CLICK on SEASONAL FORECASTS
Feb 2008 is the GSM forecasts that we use in this workshop.

ECPC downscaled forecasts over the US
http://ecpc.ucsd.edu/projects/GSM_seasons.html
CLICK on DOWNSCALED FORECASTS
Downscaled forecasts are still old. Jan 2008 version is the latest.

ENSEMBLE MEAN AND SPREAD

you can do this within GrADS but you can also create one mean file and one stdev file from all members.
see /home/nfs1/smr1934/post
ensemble.sa.list
is the list of currently available south america members, temperature monthly means for 200803
Look at ensemble.sh
I take mean from the four members and write it to sa.tmphag.mean.200803.monthly.grib
fix the date with force_grib_date_mon
and
run grmap, gribmap
now you have an ensemble mean!

Similarly for standard deviation, you can use grvar_ver2 instead of grmean.

PLOTTING NUMBER OF RAINY DAYS

contributed by Yurdanur
1. go to my post directory
2. copy fconv.f and ndaysP.ctl
3. you have to run test_gridpoints_ver2.gs to create file.txt with number of rainy days for all gridpoints within the domain
4. edit fconv.f - you have to change the number of grid points.
5. make fconv
6. this creates a binary fconv
7. run ./fconv
8. edit ndaysP.ctl - most importantly you have to copy the correct xdef and ydef from your domain's ctl file
9. open ndaysP.ctl in GrADS

SST FILES

/home/nfs1/smr1934/SST
grib files that were used for the experiments
and
PPT files that someone created (coming soon)

Good news of the day - free intel compiler works with G-RSM on susie linux!

http://g-rsm.wikispaces.com/intel

What are other files in R_000/2008?

• r_pgb.ft$hour: Output in GRIB format - pressure, surface and other levels.
• r_sigit.ft$hour, r_sigitdt.ft$hour: Sigma level files - necessary for restarts.
• r_sfc.ft$hour: surface files - necessary for restarts.
• r_sfci, r_sigitdt, r_sigit: rsim script automatically copies the latest r_sfc.ft$hour, r_sigitdt.ft$hour, r_sigit.ft$hour to r_sfci, r_sigitdt, r_sigit every INCHOUR (every time rfcst.x exits)
• The frequency of creating r_sfc.ft$hour, r_sigitdt.ft$hour, r_sigit.ft$hour is controlled by INCRESTART.
• nfdr.* : Fire danger forecast files
• r_fcstout.ft$hour : log file for rfcst.x created every INCHOUR

How to restart the model

• Look for the last r_sigit.ft, r_sigitdt.ft and r_sfc.ft before the model crashed, compare with r_sigit, r_sigitdt, and r_sfci. If they are not the same, copy the latest r_sigit.ft, r_sigitdt.ft and r_sfc.ft to r_sigit, r_sigitdt, and r_sfci.
• You can check if the cause of crash is a blow-up by looking at the last r_fcstout.ft and check for spdmx. spdmx is the maximum wind speed found in each 28 level.
• If the model blew up, decrease the time step in rsim script.
• Simply rerun the model by typing rsim

If you want to create ensemble seasonal forecast...

• first, for each ensemble member, create a seasonal mean (grmean). You already have monthly mean of tmphag. so you just have to average 3 monthly files into one seasonal mean. For precipitation make a list of 92 daily mean files and feed it into grmean to get a seasonal mean.
• next, gather seasonal means from all members
• take ensemble mean of all seasonal means (grmean) and ensemble spread of all seasonal means (grvar_ver2)

Another way to produce ensemble seasonal mean/spread

• If you are working with one merged (with cat command) daily precipitation grib file, you can create control files for all ensemble members, and open all control files in GrADS.
• d ave(prate.1,t=1,t=92) is a seasonal mean of prate of the first member.
• d (ave(prate.1,t=1,t=92) + ave(prate.2,t=1,t=92) + ave(prate.3,t=1,t=92) ...)/N is ensemble seasonal mean.
• ensemble spread for the season is a bit more complicated, but you get the idea...

You can remotely log in a ICTP machine through ssh with your user name and password.

ssh username@ssh.ictp.it
or
ssh username@eklund-xx.ictp.it
xx is number of the terminal (not accessible from a laptop but okay from other terminals)

DUAL-PROCESSORS

G-RSM runs on dual processors with improved performance compared to a single processor. You have to choose option thread instead of single. However the current version may not be bug free due to recent changes to the code. We would be happy to work with someone who is willing to debug the code for the thread option.

Intel Mac

We had a successful report of running G-RSM on Intel Mac with Intel compilers. We welcome your further inputs. If necessary, we'll commit intel-mac version of option files to our g-rsm code server.

GSM global seasonal forecast in GRIB format

Monthly mean GSM global forecast files (GRIB version of what we are using as forcings for the experiments by RSM), so you can compare them with downscaled forecasts in GrADS, if you would like to try. Look at directory GSM_FCST.
/home/nfs1/smr1934/GSM_FCST/l01/flx.avrg.grib.ctl (both prate and tmphag are here) and pgb.avrg.grib.ctl (for other variables)
replace l01 with ensemble name.
And you can directly call the ctl file from your GrADS without copying them to your directory.

INITIAL CONDITIONS USED IN THE GLOBAL FORECASTS (used in our experiments)

We (ECPC) have been running 50 year 12 member ensemble AMIP type integrations. The initial conditions are the extension of those runs.

USE OF GLOBAL FORECASTS FROM OTHER MODELS

It has not been uploaded to our code server yet, but p2sig program is able to handle any GRIB format input files.

DAY 4

TO DO LIST for YOU

• finish creating ensemble mean and spread maps
• choose gsm forecast map and forecast skill map that you will use in group presentations (from IRI and ECPC web sites)
• also look at climatology and compare with downscaled forecasts. (available in climatology directory)
• also look at SST forecasts ( available in SST directory )
• look at accumulated daily precipitation to determine onset of rainy days
• look at number of rainy days if you haven't done so
• plot hgtsfc (topography) if you haven't done so

SAMPLE SCRIPT FOR ACCUMULATED PRECIPITATION

contributed by Dian and Ana and Amjad.

look at directory
accppt/

first gs script writes out a binary file of accumulated precipitation. There is a loop in which GrADS writes out precipitation accumulated from day 1 to 92.
second gs script read the binary .dat and plot it
you also need a ctl file for the .dat to read it in GrADS.

CHANGE OUTPUT FREQUENCY in RSM

• open rruns/rsim
• INTPGB and INTPOST control the output frequency

CHANGE CONVECTION SCHEMES in RSM

• go to rsm/def/physics.h
• undef the current conv scheme and define a new scheme
• go to rsm
• configure-model
• make

CHANGE VEGETATION in RSM

• before running rfcst.x, sfc0 processes vegfrac (vegtation fraction) and vegtype (vegetation type). The files are found under libs/con. rruns/runscr/sfc0.in controls which vegfrac and vegtype files should be used in sfc0.x
• in the forecast program rfcst.x rruns/runscr/r_fcstparm.in has a name list for vegfrac because fraction varies from month to month.

TAR FILES FOR YOU TO TAKE HOME

In /home/nfs1/smr1934

ecpc_exp.tar -- the model package but we recommend downloading directly from our server via CVS.
grads_scripts.tar -- cbar.gs etc
GSM_FCST.tar -- GRIB files for the driving GSM forecasts
SST.tar --- 3 SST files for the experiments
takehome.tar --- a number of scripts/programs used in theworkshop.
takehome.tar