Plasma State File Reducer — HowTo
As the title already leaves out to think, this plasma state file (PSF) reducer is supposed
to read in a (larger) netcdf-formatted PSF (l-PSF), and produce a new (reduced) version of
that file (r-PSF) that contains definitions of only those variables that are needed for a
particular simulation. However, a set of l-PSFs are created live for each given time frame
and added onto a single folder that we will call the
“monitored folder”. Furthermore, our
goal is to perform a live-monitoring of the resulting r-PSF, for each incoming time-dependent
l-PSF. Therefore, we are reducing from many l-PSFs to a single r-PSF, which will be in turn
live-monitored by
Elvis.
During this reduction, the user has a choice of either replacing the previous data in the
r-PSF with the one from the currently “monitored” l-PSF, or simply just appending new data onto
the previous ones.
In the case of appending, an infinite dimension for storing the time slots is created and added onto
the dimensions of each existing variable. When replacing instead, the time dimension is fixed to describe a scalar.
In both cases, a time variable is also accordingly defined. Also, for every pre-existing dimension, we define a
vector variable that holds the numbers 1 through the dimension's length. The later is intended to help with the
visualization for some cases...
For testing purposes in a first time, instead of l-PSFs, we considered a set of matlab
generated ASCII files and produced a simple netcdf file; hence the allusions to ASCII files later on.
The implementation is comprised of four (4) main phases:
- The listener: plasma_listener.py
This consists simply of an infinite loop that listens to new l-PSFs entries into the monitored
folder, and dispatches the work to the “reducer”, for each entry, based on the user's selections.
It also initializes the file to record timing results in if the user opted for saving them.
***Notice that, in special cases such as simple testing where the set of l-PSFs is predefined,
one can always remove the infinite loop simply by commenting out the line containing “while True:”
in the definition of the plasma_listener().
- The reducer: plasma_red.py (imported by the listener)
For each given currently monitored entry, it creates or updates (appending or replacing) the r-PSF,
based on the format of the incoming entry(l-PSF or ASCII file).
***Notice that this assumes that every file in the monitored folder has the same structure,
so that there will be no need to redefine the already defined variables (and dimensions and attributes)
- The initializer: monitor_init.py (imported by the user)
This can be viewed as an agreement, between the user and the “listener”, on constant and default values.
It defines a set of flags, with their default values, and default functions needed, whose
implementation really depends on the the structure in which the time-dependent files are stored.
- The user: plasma_user.py (imported by the reducer)
The user clearly sets the flags values that he/she wants, and can provide new definitions of initial
functions if the structure of his data does not agree with the default ones.
***Make sure to add the following line at the beginning of the file: “from monitor_init import *”
*** Notice that the variables dims_red (names of needed dimensions) and vars_red (names of needed
variables) have to be defined for l-PSF entries.
Of all these, only the file
“plasma_user.py” is what really needs to be re-defined from one set of users to another, and provided the listener, reducer and initializer are set properly for the particular task. Please refer to the provided folder for a listing of initializing flags and functions, an example of a user file, and a look at the listener and reducer.
PSF_reducer
The current implementation is available (and can be downloaded) from the following link:
PSF_reducer.tar.gz
Please notice that
netcdf,
python, and
pycdf
need to be available on any system that this application is run from. (provide links to each).
After the initializing flags and function definitions are defined correctly for a given user (user-dependent
definition of plasma_user.py), simply run the following command from the folder containing all the files
(listener,reducer, etc...):
“python plasma_listener.py”.
A message of success creating (or updating) the r-PSF should then be printed on the screen for each
processed l-PSF entry; either that or an error message....
Please submit all comments or questions, with subject line “PSF_reducer”, to: lvoufo@cs.indiana.edu or bramley@cs.indiana.edu.