LSP can be run on either single- or multiple-processor machines.
The multiple-processor version is initiated by using the MULTI_PROCESS
compiler directive (see section MULTI_PROCESS). An input
file must be present in order for LSP to run (see section Input Variables) and a
command file may be present (see section Command File), but is optional.
The single-processor version of LSP will run on Unix, Windows, and Mac OS X workstations. At the command line or shell prompt, enter
lsp [-r] [-ra] [-n N] [-s] input.lsp
where `input.lsp' is the user-supplied input file. To run in background with redirection of terminal output, one can use `lsp [-opt] input.lsp >&log&' (assuming a C shell), where `log' is the name of the file containing the redirected output.
The optional `-r' flag is used to restart a previous run from a restart
dump, which will have the name `restart.dat' or `restart.alt'.
This file is read after the input file is processed. The restart file
contains probe history, particle and field data from the previous run. If the
rename_restart_flag is ON (see section rename_restart_flag (flag)), then the most recent restart file may have the
`.alt' extension (this can be determined by looking at the file dates).
In this case, the file with the `.dat' extension can be removed from the
simulation directory before restarting. The code will then attempt a restart
from the `.alt' file. If this restart fails, the `.dat' file can be
used as a backup. Alternatively, the `-ra' option has the same effect as
the `-r' flag, except that the restart is performed from the
`restart.alt' file instead of the `restart.dat' file. This is a more
elegant way to restart from the `.alt' file than moving or deleting the
`.dat' file.
Probe history data reside in the restart file and are written to the history
file `history.p4' when the restart run begins, so this file need not be
preserved between restarts. The user ordinarily increases one of the
time_limit parameters on the input file prior to restarting a
simulation, unless the previous run was stopped before reaching this limit.
However, use of the number_of_steps parameter on a restart run will
cause the simulation to execute exactly that number of additional timesteps,
unless the time_limit is reached first.
The `-n N' sequence, where N is an unsigned integer, will cause
the code to run that number of timesteps, regardless of what is specified on
the input file. This may be useful for checking the simulation setup without
directly changing the input file.
The `-s' option is used to perform initialization only and stops the run immediately regardless of any time limits in the input file. This may also be useful for checking the simulation setup before submitting a batch run, for example.
The multiple-processor version of LSP uses the MPI message-passing library (`http://www-unix.mcs.anl.gov/mpi/'). This version can make use of decomposition of the simulation space into separate domains in order to distribute the work-load among multiple processes and thereby achieve faster running times.
The multiple-processor version of LSP produces a separate restart file
for each region, named `restart1.dat', `restart2.dat', ....
All of these files must be present for the restart to work correctly.
If the rename_restart_flag is ON (see section rename_restart_flag (flag)), then the most recent restart files may have the
`.alt' extension (this can be determined by looking at the file dates).
In this case, the files with the `.dat' extension can be moved out of the
current run directory before restarting. The code will then attempt a restart
from the `.alt' files. If this restart fails, the `.dat' files can be
used as a backup by restoring them to the run directory. The most common reason
for restart failure is a corrupted or incomplete restart file due to time-limit
termination while the file was being written. A better way to restart a
simulation from the `.alt' files than moving or deleting the `.dat'
files is to use the `-ra' option on the command line instead of the
`-r' flag.
On a workstation network with the MPICH version of MPI installed (`http://www-unix.mcs.anl.gov/mpi/mpich/'), start a multiple-processor version of LSP using the p4 (no relation to the P4 postprocessor, section P4 Postprocessor) communication device as follows:
lsp -p4pg pgroup [-opt] input.lsp
where the file `pgroup' specifies a p4 processor group, such as
local 0 cerebro 1 /usr1/run7/lsp achilles 1 /usr1/run7/lsp
which specifies that three processes are to be used (see section number_of_processes (integer)), one on the local computer, one on computer "cerebro" (which may also be the local computer), and one on computer "achilles". The path to the executable on each computer must be supplied. The path also specifies the run directory, so the executable, or a link to it, must be in the run directory.
On the DEC Unix cluster, start a multiple-processor version of LSP with:
dmpirun -np NP lsp [-opt] input.lsp
where `NP' is the number of processes to use. This
number must be the same as the number_of_processes parameter in
the input file (see section Control Input).
On the Intel Teraflop (ASCI Red), start LSP interactively on `NP' processors using
yod -sz NP lsp [-opt] input.lsp >&log&
An NQS (Network Queueing System) job for `NP' processors can be submitted to the `QUEUE' queue on the Teraflop for a time `HH:MM:SS' (hours:mins:secs) using
qsub -q QUEUE -lP NP -lT HH:MM:SS -o log.lsp script.lsp
where `script.lsp' is a shell-script file. An example of this file follows.
#!/bin/sh date cd $QSUB_WORKDIR /cougar/bin/yod lsp [-opt] input.lsp
where QSUB_WORKDIR is a shell variable which resolves to the
working directory from which the qsub command is issued.
On the ASCIQ system, start LSP interactively using
prun -n NP lsp [-opt] input.lsp >&log&
where `NP' is the number of processors.
An LSF (Load Sharing Facility) job for `NP' processors on `ND' nodes can be submitted to the `QUEUE' queue on the ASCIQ for a time `MMM' (min) using
bsub -q QUEUE -n NP -o log.lsp -W MMM prun -N ND lsp [-opt] input.lsp
Each node consists of 4 processors; therefore, a multiple of 4 must be used for `NP' when submitting a job beyond 4.
On the IBM-SP2, start LSP interactively using
lsp -procs NP [-opt] input.lsp >&log&
where `NP' is the number of processors.
Generally, the IP switch will be used for interactive jobs. At the command prompt, type setenv MP_EUILIB ip.
A LoadLeveler batch job can be submitted using
llsubmit script.lsp
where `script.lsp' is a control file such as
#@ job_name = run1
#@ initialdir = /scratch2/mydir/
#@ notification = always
#@ notify_user = myname@company.com
#@ error = lsp.$(Cluster).err
#@ output = lsp.$(Cluster).out
#@ job_type = parallel
#@ requirements = (Adapter == "hps_user")
#@ min_processors = NP
#@ max_processors = NP
#@ environment = MP_EUILIB=us;MP_INFOLEVEL=3;MP_LABELIO=yes
#@ checkpoint = no
#@ wall_clock_limit= HH:MM:SS
#@ account_no = ACCOUNT_NUMBER
#@ queue
set nodes = `echo $LOADL_PROCESSOR_LIST`
cat ${nodes}
set runid = "run1"
cp -f $HOME/mydir/lsp .
cp -f $HOME/mydir/runs/${runid} .
lsp ${runid}
tar cvf ${runid}.tar ${runid} *.dat *.out *.err
compress -f ${runid}.tar
hpsscp ${runid}.tar.Z hpss:/s/hpss/myname/${runid}.tar.Z
echo "Job completed"
where `NP' is the number of processors, `HH:MM:SS' (hours:mins:secs) is the wall-time and `ACCOUNT_NUMBER' is the user's account.
When LSP begins running, several lines of data relating to when and where it was compiled, the compiler directives used, the input file name, and the start time are generated. An example is:
Compiled Sat Nov 14 12:24:57 MST 1998 on achilles
Compiler flags: -O4 -std1 -warnprotos
Code options defined by user: -DSTATIC_FIELDS -DMULTI_PROCESS
Code options defined at compile-time:
STATIC_FIELDS
MULTI_PROCESS
Coordinate system used:
CARTESIAN
Input data file: input.lsp
Start time = Sat Nov 14 12:27:56 1998
The "code options" listed are those specified by the user at compilation time (see section Compiling LSP). These options are also referred to as "compiler directives".
When starting an LSP run, a variety of error checking takes place to diagnose possible mistakes in the choice of compiler directives and the specification of input data. These errors will usually cause a printed message in standard output and prevent the simulation from continuing. However, in some cases, a warning message may occur without stopping the code from running. The user should be alert for this type of occurrence.
Most of the data appearing in the input file for a simulation is syntax dependent. Interpretation by the LSP code is sensitive to the spelling and order of input parameters. At present, the code issues an error message and exits immediately upon finding items in the input text that cannot be identified.
The user is responsible for simulation set-up. However, LSP is able to check that at least the boundaries of each domain (and therefore the entire simulation space) are well defined. This means that some physical boundary condition is defined everywhere at the boundaries. If any cells along an outer boundary are found not to be covered by one of the conditions necessary for simulation fidelity, appropriate error messages are printed which locate the portion of the grid containing the fault, and the simulation is aborted. An example is:
P0: boundary check error at ZMIN,
x-range 0.00e+00:1.00e+00, y-range 0.00e+00:0.00e+00
*** Error(s) on domain boundary detected ***
Domain boundary = ZMIN
Check input data for completeness
which indicates that the domain under control of process 0 has been found to have an undefined boundary at the lower side in the z-coordinate. The most common cause for this kind of error is a "hole" in the simulation boundary which can be patched with a conducting surface.
While running a simulation, LSP checks periodically for a file named `command' in the directory from which it is started. This file may contain one of the following strings, which cause the code to perform certain actions at the current timestep:
dump: Write all particle, scalar, field, and diagnostic dumps.
stop: Write a restart dump and stop the run.
abort: Stop the run with no restart dump.
rdump: Write a restart dump and continue.
report: Print domain statistics to gauge load balance.
particles: Print particle statistics by species.
balance: Rebalance the computational load among processes.
Under either Windows, Unix or Mac OS X, one can create the file `command' containing the word `stop' with the command echo stop > command.
Go to the first, previous, next, last section, table of contents.