========================================================================================
Advice from Brian Eaton on run script and namelist settings to optimize waccm3.5.7 runs
========================================================================================

1. My run script includes a bunch of environment variables that are
designed to optimize performance on the IBM power5 machines.  I got these
from Jim Edwards and use them in all my run scripts.  They are the
following:

export OMP_DYNAMIC=FALSE
export XLSMPOPTS="stack=128000000:spins=0:yields=0"
export MP_WAIT_MODE=poll
export MP_POLLING_INTERVAL=20000000
export MP_USE_BULK_XFER=yes
export MP_BULK_MIN_MSG_SIZE=12k
export MP_EAGER_LIMIT=10240
export SAVE_ALL_TASKS=no
export MP_SHARED_MEMORY=yes
export MP_SINGLE_THREAD=yes	[NB: DO NOT USE THIS SETTING UNLESS RUNNING PURE MPI]
export MEMORY_AFFINITY=MCM
export AIXTHREAD_SCOPE=S
export MALLOCMULTIHEAP=true
export LDR_CNTRL="DATAPSIZE=64K@TEXTPSIZE=64K@STACKPSIZE=64K"

The other thing I do in my run script is to use a script provided by CISL
(/contrib/bin/bindproc.x) that does "process binding".  So the command I
use to execute cam is: 

% mpirun.lsf /contrib/bin/bindproc.x cam    [where cam is the name of the executable]

2. You should add the setting phys_loadbalance=2 to the namelist.  This
optimizes load balancing

3. Do not set mss_irt=XXX [where XXX ne 0]. This forces cam to issue commands
to archive to the mss, and you're suffering from waiting for these commands to
finish transferring data to the mss.  Fabrizio has volunteered to talk to
Matt and find out how to use the archiving script.  It just involves adding
a command or two to your current run script to execute the archiving
scripts when cam is done executing.  Then you can set mss_irt=0.  We don't
expect this to be hard since Matt introduced these scripts onto cam's trunk
at cam3_5_11 which is only a few tags after the code base you're currently
using.

Brian