This software repository contains several python scripts to perform various tasks associated with the
Co-optimized Machine Learned Manifolds (CMLM) model, which is described in Perry, Henry de Frahan, and Yellapantula, CNF, 2022 (https://doi.org/10.1016/j.combustflame.2022.112286). This includes not only the code that defines the CMLM model, but also scripts to generate suitable training data, scripts to pre-process the data, scripts to train the CMLM model, and scripts to plot the output, as well as various other helper files. The scripts depend on several commonly used python libraries for data analysis and chemical reaction computations. The trained models that result from this tool are designed to work with the an interface being implemented in the Pele suite of reacting flow solvers (https://github.com/AMReX-Combustion).

### REQUIREMENTS ###
Python 3
Cantera
Pytorch
Pandas
Numpy
Scikit-Learn
Matplotlib
SHERPA: https://parameter-sherpa.readthedocs.io/en/latest/gettingstarted/installation.html

Additionally, to use Cantera to generate training data, it is necessary to download the drm19 (http://combustion.berkeley.edu/drm/) and HyChem Jet-A (https://web.stanford.edu/group/haiwanglab/HyChem/pages/download_cantera.html) chemical mechanisms and use Cantera's tools to convert to the appropriate format. Some data has been saved so the first example can be run without doing this.


--------------------Strained Premixed Flames Case ---------------------

For this example, the output of steps 1 and 2 (data generation and pre-processing) is saved and included already, so these steps may be skipped to focus only on training and analyzing the CMLM model.

### STEP 1 ###

run cantera to generate training data. This script solves a bunch of 1D flames at varied
strain rate:

    python get_strained_data.py

Or to run in parallel, e.g. on an interactivee node on Eagle:

    srun -n 36 python get_strained_data.py

### Step 2 ###
Process the data to the form required by the ML scripts

    python gen_trn_tst_data.py -i "strained_flames/strained*.csv" -o data/test_strained_data -f -g POSF10325 -m random

### Step 3 ###
Train the network (run on GPU node on Eagle or change use_GPU in script to False)

    python train_models_strained_flames_raw.py

### Step 4 ###
make some plots

    python plot_strained.py



-------------------- Autoignition Case ---------------------

### Step 1 ###
Run a bunch of 0D reactor cases with different equivalence ratios (fuel/air ratios) to generate data:

    python get_autoignition_data.py

### Step 2 ###
Process the data to the form required by the ML scripts

    python gen_trn_tst_data.py -i "autoignition/a*csv" -f -o data/test_autoignition_data -m patternfile -p "50_P"  -g CH4

### Step 3 ###
Train the network (run on GPU node on Eagle or change use_GPU in script to False)

    python train_models_autoignition.py

### Step 4 ###
Plot the network outputs

    python plot_autoignition.py

### Create a table of the data for comparison to the network

    python compile_2D_table.py


-------------------- Premixed Flame Case (2D C/Cvar Table) ---------------------

A single script solves a single cantera PMF, then convolutes with a beta PDF
and creates a chemtable.

    python get_pmfdat_and_ctable_withvar.py

This will generate the following files:

    * prem_drm19_phi1_p1_t298.ctb
      (2D chemtable file in Pele format, very similar to NGA format)

    * prem_drm19_phi1_p1_t298.dat
      Pele PMF data file (for detailed chemistry)

    * prem_drm19_phi1_p1_t298_allrxn.ctb
      Pele PMF data file (for manifold chemistry, but Cvar = 0 everywhere)

    * prem_drm19_phi1_p1_t298_mani_filter1.875e-02.dat
      Pele PMF data file (for manifold chemistry, filtered to get nonzero Cvar)

You can print some data about the chemtable to make sure this worked:

    python ctable_tools.py data/prem_drm19_phi1_p1_t298.ctb -p

You can also create some plots (in the `plots/` directory):

    python ctable_tools.py data/prem_drm19_phi1_p1_t298.ctb -sp justplot -o plots

-------------------- Premixed Flame Case ---------------------

The script to generate testing and training data has been modified to it
can read in chemtable files. So first, we generate a chemtable
*set ```net_info_file = None``` in this file

    python get_pmfdat_and_ctable.py

    python gen_trn_tst_data.py -i "data/prem_drm19_phi1_p1_t298_allrxn.ctb" -t -o data/test_premixed_data -m everyother -g CH4

    python train_models_premixed.py

Now rerun the flame solver to generate the initial condition file for the network model
*set ```net_info_file = cmlm_net_info.txt``` in this file

    python get_pmfdat_and_ctable.py

-------------------- Additional Notes ---------------------
### Net Array
The `train_models_autoignition_netarray.py` file has an example on how to
set up and train a NetArray model (defined in the `net_array` module). The
main differences in terms of usage from the single net case are that subnets
must be added when or just after initializing the net to define
the net array, and that the `save_torchscript` method should be
used to save a net array. The keyword arguments might also differ from
those for the other nets -- see the `__init__` method in the
`NetArray` class. Note that `save_torchscript` saves a directory rather
than a single file.

The conversion script (`convert_net_format.py`) will work on saved net array
directories. It requires the subnet file names to be sortable (with position
in sorted filename list corresponding to the net's postion in the net array).
The `NetArray` (and `MRNetArray`) classes do this by default. Note that one
needs to supply the `--array` argument to the conversion script to convert a
`NetArray` at present.

### Source Term Specific Loss Function
The module `manifold_reduction_nets` now defines a pair of loss functions
operating on species source terms and manifold source terms. The species
source term loss function does not account for biases introduced by the
scaling, so that will need to be added in. Whether the functions are
correct or not is up for debate.

The `train_models_autoignition_srcloss.py` script gives an example of
including this loss function when training the model. It only does so
for the FGM net (everything else is commented out).

### Flamesheet Case
Right now this just produces a simple net trained on a very small dataset.
It was used for testing whether the FlameSheet test would run and comparing
PyTorch vs. home-rolled net performance.




The script to generate testing and training data has been modified to it
can read in chemtable files. So first, we generate a chemtable
*set ```net_info_file = None``` in this file

    python get_pmfdat_and_ctable.py

    python gen_trn_tst_data.py -i "data/prem_drm19_phi1_p1_t298_allrxn.ctb" -t -o data/test_premixed_data -m everyother -g CH4

    python train_models_premixed.py

Now rerun the flame solver to generate the initial condition file for the network model
*set ```net_info_file = cmlm_net_info.txt``` in this file

    python get_pmfdat_and_ctable.py

-------------------- Additional Notes ---------------------
### Net Array
The `train_models_autoignition_netarray.py` file has an example on how to
set up and train a NetArray model (defined in the `net_array` module). The
main differences in terms of usage from the single net case are that subnets
must be added when or just after initializing the net to define
the net array, and that the `save_torchscript` method should be
used to save a net array. The keyword arguments might also differ from
those for the other nets -- see the `__init__` method in the
`NetArray` class. Note that `save_torchscript` saves a directory rather
than a single file.

The conversion script (`convert_net_format.py`) will work on saved net array
directories. It requires the subnet file names to be sortable (with position
in sorted filename list corresponding to the net's postion in the net array).
The `NetArray` (and `MRNetArray`) classes do this by default. Note that one
needs to supply the `--array` argument to the conversion script to convert a
`NetArray` at present.

### Source Term Specific Loss Function
The module `manifold_reduction_nets` now defines a pair of loss functions
operating on species source terms and manifold source terms. The species
source term loss function does not account for biases introduced by the
scaling, so that will need to be added in. Whether the functions are
correct or not is up for debate.

The `train_models_autoignition_srcloss.py` script gives an example of
including this loss function when training the model. It only does so
for the FGM net (everything else is commented out).

### Flamesheet Case
Right now this just produces a simple net trained on a very small dataset.
It was used for testing whether the FlameSheet test would run and comparing
PyTorch vs. home-rolled net performance.

-------------------- Nonpremixed with variance  ---------------------

Generate nonpremixed flamelets (make sure you don't already have
a nonpremixed_flames/ directory - this doesn't like to overwrite).
Also this will run for too long on it's own. Just let it go until it's
done several flames traversing toward equilibrium (should be files
with indices ~80 -> 117 in the nonpremixed_flames directory):

    python get_nonpremixed_flamelets.py

Compile the table:

    mpiexec -n 20 python compile_3D_table_nonpremixed.py

Make some plots of the table:

    python ctable_tools.py nonpremixed.ctb -sp "ZMIXVAR:0" -o plots
    python ctable_tools.py nonpremixed.ctb -sp "PROG:0.27" -o plots

-------------------- Dependencies with Poetry  ---------------------

1. Install poetry
2. run `poetry update` from this directory
3. prefix running python scripts with `poetry run`, e.g.:
          poetry run python get_nonpremixed_flamelets.py
4. Only manages dependencies for tables right now, not networks

-------------------- Additional Details ---------------------

Released under a BSD 3 clause license, see "LICENSE"

NREL Software Record Number: SWR-23-41