[1]:
import emat
from emat.util.loggers import timing_log
emat.versions()
emat 0.5.2, plotly 4.14.3

Meta-Model Creation

To demostrate the creation of a meta-model, we will use the Road Test example model included with TMIP-EMAT. We will first create and run a design of experiments, to have some experimental data to define the meta-model.

[2]:
import emat.examples
scope, db, model = emat.examples.road_test()
design = model.design_experiments(design_name='lhs')
results = model.run_experiments(design)

We can then create a meta-model automatically from these experiments.

[3]:
from emat.model import create_metamodel

with timing_log("create metamodel"):
    mm = create_metamodel(scope, results, suppress_converge_warnings=True)
<TIME BEGINS> create metamodel
< TIME ENDS > create metamodel <18.05s>

If you are using the default meta-model regressor, as we are doing here, you can directly access a cross-validation method that uses the experimental data to evaluate the quality of the regression model. The cross_val_scores provides a measure of how well the meta-model predicts the experimental outcomes, similar to an R^2 measure on a linear regression model.

[4]:
with timing_log("crossvalidate metamodel"):
    display(mm.cross_val_scores())
<TIME BEGINS> crossvalidate metamodel
Cross Validation Score
no_build_travel_time 0.9946
build_travel_time 0.9846
time_savings 0.9242
value_of_time_savings 0.8345
net_benefits 0.6746
cost_of_capacity_expansion 0.8952
present_cost_expansion 0.9445
< TIME ENDS > crossvalidate metamodel <14.22s>

We can apply the meta-model directly on a new design of experiments, and use the contrast_experiments visualization tool to review how well the meta-model is replicating the underlying model’s results.

[5]:
design2 = mm.design_experiments(design_name='lhs_meta', n_samples=10_000)
[6]:
with timing_log("apply metamodel"):
    results2 = mm.run_experiments(design2)
<TIME BEGINS> apply metamodel
< TIME ENDS > apply metamodel <0.18s>
[7]:
results2.info()
<class 'emat.experiment.experimental_design.ExperimentalDesign'>
Int64Index: 10000 entries, 0 to 9999
Data columns (total 20 columns):
 #   Column                      Non-Null Count  Dtype
---  ------                      --------------  -----
 0   alpha                       10000 non-null  float64
 1   amortization_period         10000 non-null  int64
 2   beta                        10000 non-null  float64
 3   debt_type                   10000 non-null  category
 4   expand_capacity             10000 non-null  float64
 5   input_flow                  10000 non-null  int64
 6   interest_rate               10000 non-null  float64
 7   interest_rate_lock          10000 non-null  bool
 8   unit_cost_expansion         10000 non-null  float64
 9   value_of_time               10000 non-null  float64
 10  yield_curve                 10000 non-null  float64
 11  free_flow_time              10000 non-null  int64
 12  initial_capacity            10000 non-null  int64
 13  no_build_travel_time        10000 non-null  float64
 14  build_travel_time           10000 non-null  float64
 15  time_savings                10000 non-null  float64
 16  value_of_time_savings       10000 non-null  float64
 17  net_benefits                10000 non-null  float64
 18  cost_of_capacity_expansion  10000 non-null  float64
 19  present_cost_expansion      10000 non-null  float64
dtypes: bool(1), category(1), float64(14), int64(4)
memory usage: 1.5 MB
[8]:
from emat.analysis import contrast_experiments
contrast_experiments(mm.scope, results2, results)

No Build Time