{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "086cde1a", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.284997Z", "start_time": "2021-06-28T16:15:47.751986Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "emat 0.5.2, plotly 4.14.3\n" ] } ], "source": [ "import emat\n", "emat.versions()" ] }, { "cell_type": "raw", "id": "918cf9c9", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. _methodology-experimental-design:" ] }, { "cell_type": "markdown", "id": "d4ee5e58", "metadata": {}, "source": [ "# Experimental Designs" ] }, { "cell_type": "markdown", "id": "450b74a1", "metadata": {}, "source": [ "A fundamental part of TMIP-EMAT is the development of a \"design of experiments\"\n", "to use in conjunction with a model. The experimental design lays out a list of \n", "model experiments to be conducted, and can be constructed in a number of different \n", "ways: it can be a systematic design, a random design, or something in between.\n", "\n", "Developing an experimental design using TMIP-EMAT requires that an exploratory\n", "scope is defined, to identify the model inputs and their potential values. To \n", "illustrate designs, we'll work with the scope from the road test model." ] }, { "cell_type": "code", "execution_count": 2, "id": "2be77ae2", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.339231Z", "start_time": "2021-06-28T16:15:49.287767Z" } }, "outputs": [], "source": [ "import emat.examples\n", "scope, db, model = emat.examples.road_test()" ] }, { "cell_type": "raw", "id": "fd396e99", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. _methodology-univariate-sensitivity:" ] }, { "cell_type": "markdown", "id": "e2e18cb9", "metadata": {}, "source": [ "## Univariate Sensitivity Tests" ] }, { "cell_type": "markdown", "id": "60fe24f6", "metadata": {}, "source": [ "One of the simplest experimental designs, and one used often in practice for \n", "transportation models, is a series of univariate sensitivity tests. In this \n", "design, a set of baseline or default model inputs is used as a starting point,\n", "and then input parameters are changed one at a time to non-default values.\n", "Univariate sensitivity tests are excellent tools for debugging and quality checking\n", "the model code, as they allow modelers to confirm that each modeled input is \n", "(or is intentionally not) triggering some change in the model outputs.\n", "\n", "TMIP-EMAT provides the `design_sensitivity_tests` function to automatically \n", "generate a design of univariate sensitivity tests, given an exploratory scope.\n", "This tool will systematically generate experiments where all of the inputs but one are\n", "set to their default values, and the holdout input is then set to alternative values.\n", "For numerical inputs, the alternative values are the minimum or maximum\n", "value (one experiment for each, if they are different from the default). For categorical\n", "and boolean inputs, one experiment is created for each possible input value." ] }, { "cell_type": "code", "execution_count": 3, "id": "aab6df3b", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.379999Z", "start_time": "2021-06-28T16:15:49.342917Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
free_flow_timeinitial_capacityalphabetainput_flowvalue_of_timeunit_cost_expansioninterest_rateyield_curveexpand_capacityamortization_perioddebt_typeinterest_rate_lock
0601000.154.01000.0751000.0300.010010.030GO BondFalse
1601000.104.01000.0751000.0300.010010.030GO BondFalse
2601000.204.01000.0751000.0300.010010.030GO BondFalse
3601000.153.51000.0751000.0300.010010.030GO BondFalse
4601000.155.51000.0751000.0300.010010.030GO BondFalse
5601000.154.0800.0751000.0300.010010.030GO BondFalse
6601000.154.01500.0751000.0300.010010.030GO BondFalse
7601000.154.01000.0011000.0300.010010.030GO BondFalse
8601000.154.01000.2501000.0300.010010.030GO BondFalse
9601000.154.01000.075950.0300.010010.030GO BondFalse
10601000.154.01000.0751450.0300.010010.030GO BondFalse
11601000.154.01000.0751000.0250.010010.030GO BondFalse
12601000.154.01000.0751000.0400.010010.030GO BondFalse
13601000.154.01000.0751000.030-0.002510.030GO BondFalse
14601000.154.01000.0751000.0300.020010.030GO BondFalse
15601000.154.01000.0751000.0300.01000.030GO BondFalse
16601000.154.01000.0751000.0300.0100100.030GO BondFalse
17601000.154.01000.0751000.0300.010010.015GO BondFalse
18601000.154.01000.0751000.0300.010010.050GO BondFalse
19601000.154.01000.0751000.0300.010010.030Rev BondFalse
20601000.154.01000.0751000.0300.010010.030PaygoFalse
21601000.154.01000.0751000.0300.010010.030GO BondTrue
\n", "
" ], "text/plain": [ " free_flow_time initial_capacity alpha beta input_flow value_of_time \\\n", "0 60 100 0.15 4.0 100 0.075 \n", "1 60 100 0.10 4.0 100 0.075 \n", "2 60 100 0.20 4.0 100 0.075 \n", "3 60 100 0.15 3.5 100 0.075 \n", "4 60 100 0.15 5.5 100 0.075 \n", "5 60 100 0.15 4.0 80 0.075 \n", "6 60 100 0.15 4.0 150 0.075 \n", "7 60 100 0.15 4.0 100 0.001 \n", "8 60 100 0.15 4.0 100 0.250 \n", "9 60 100 0.15 4.0 100 0.075 \n", "10 60 100 0.15 4.0 100 0.075 \n", "11 60 100 0.15 4.0 100 0.075 \n", "12 60 100 0.15 4.0 100 0.075 \n", "13 60 100 0.15 4.0 100 0.075 \n", "14 60 100 0.15 4.0 100 0.075 \n", "15 60 100 0.15 4.0 100 0.075 \n", "16 60 100 0.15 4.0 100 0.075 \n", "17 60 100 0.15 4.0 100 0.075 \n", "18 60 100 0.15 4.0 100 0.075 \n", "19 60 100 0.15 4.0 100 0.075 \n", "20 60 100 0.15 4.0 100 0.075 \n", "21 60 100 0.15 4.0 100 0.075 \n", "\n", " unit_cost_expansion interest_rate yield_curve expand_capacity \\\n", "0 100 0.030 0.0100 10.0 \n", "1 100 0.030 0.0100 10.0 \n", "2 100 0.030 0.0100 10.0 \n", "3 100 0.030 0.0100 10.0 \n", "4 100 0.030 0.0100 10.0 \n", "5 100 0.030 0.0100 10.0 \n", "6 100 0.030 0.0100 10.0 \n", "7 100 0.030 0.0100 10.0 \n", "8 100 0.030 0.0100 10.0 \n", "9 95 0.030 0.0100 10.0 \n", "10 145 0.030 0.0100 10.0 \n", "11 100 0.025 0.0100 10.0 \n", "12 100 0.040 0.0100 10.0 \n", "13 100 0.030 -0.0025 10.0 \n", "14 100 0.030 0.0200 10.0 \n", "15 100 0.030 0.0100 0.0 \n", "16 100 0.030 0.0100 100.0 \n", "17 100 0.030 0.0100 10.0 \n", "18 100 0.030 0.0100 10.0 \n", "19 100 0.030 0.0100 10.0 \n", "20 100 0.030 0.0100 10.0 \n", "21 100 0.030 0.0100 10.0 \n", "\n", " amortization_period debt_type interest_rate_lock \n", "0 30 GO Bond False \n", "1 30 GO Bond False \n", "2 30 GO Bond False \n", "3 30 GO Bond False \n", "4 30 GO Bond False \n", "5 30 GO Bond False \n", "6 30 GO Bond False \n", "7 30 GO Bond False \n", "8 30 GO Bond False \n", "9 30 GO Bond False \n", "10 30 GO Bond False \n", "11 30 GO Bond False \n", "12 30 GO Bond False \n", "13 30 GO Bond False \n", "14 30 GO Bond False \n", "15 30 GO Bond False \n", "16 30 GO Bond False \n", "17 15 GO Bond False \n", "18 50 GO Bond False \n", "19 30 Rev Bond False \n", "20 30 Paygo False \n", "21 30 GO Bond True " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from emat.experiment.experimental_design import design_sensitivity_tests\n", "ust = design_sensitivity_tests(scope, design_name='univariate_tests')\n", "ust" ] }, { "cell_type": "markdown", "id": "52b6cda9", "metadata": {}, "source": [ "The output of this function is an `emat.ExperimentalDesign`, which is a subclass of \n", "a typical pandas DataFrame. The `emat.ExperimentalDesign` subclass attaches some useful metadata to the \n", "actual contents of the design, namely: \n", "\n", "- `sampler_name`, which for univariate sensitivity test is `'uni'`, \n", "- `design_name`, a descriptive name for storing and retrieving experiments and results in a database, and\n", "- `scope`, which contains a reference to the scope used to define this design." ] }, { "cell_type": "code", "execution_count": 4, "id": "e71f5949", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.384812Z", "start_time": "2021-06-28T16:15:49.381873Z" } }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ust.scope is scope" ] }, { "cell_type": "code", "execution_count": 5, "id": "7155ed32", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.389097Z", "start_time": "2021-06-28T16:15:49.386484Z" } }, "outputs": [ { "data": { "text/plain": [ "'uni'" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ust.sampler_name" ] }, { "cell_type": "code", "execution_count": 6, "id": "2ed7b8aa", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.393028Z", "start_time": "2021-06-28T16:15:49.390455Z" } }, "outputs": [ { "data": { "text/plain": [ "'univariate_tests'" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ust.design_name" ] }, { "cell_type": "raw", "id": "29a6dd55", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. _methodology-latin-hypercube:" ] }, { "cell_type": "markdown", "id": "ed7f4bf5", "metadata": { "ExecuteTime": { "end_time": "2021-05-26T13:06:52.426484Z", "start_time": "2021-05-26T13:06:52.424166Z" } }, "source": [ "## Efficient Designs" ] }, { "cell_type": "markdown", "id": "76b62818", "metadata": {}, "source": [ "The univariate tests, while useful for model debugging and testing, are not the best choice for exploratory modeling, where we want to consider changing many input factors at once. To support multivariate exploratation, TMIP-EMAT includes a design_experiments function that offers several different multivariate samplers." ] }, { "cell_type": "code", "execution_count": 7, "id": "9211865e", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.396582Z", "start_time": "2021-06-28T16:15:49.394511Z" } }, "outputs": [], "source": [ "from emat.experiment.experimental_design import design_experiments" ] }, { "cell_type": "markdown", "id": "1c7485d5", "metadata": {}, "source": [ "The default multivariate sampler is a basic Latin Hypercube sampler. Meta-models for deterministic simulation experiments, such as a TDM, are best supported by a “space filling” design of experiments, such as Latin Hypercube draws (Sacks, Welch, Mitchell, & Wynn, 1989). \n", "\n", "A Latin Hypercube sample for one dimension is constructed by subdividing the distribution of each input factor into N equally probable ranges, and drawing one random sample within each range. For example, if an input factor is assumed to be uniformly distributed between 0.1 and 0.2, that distribution can be divided into four regions (0.1-0.125, 0.125-0.15, 0.15-0.175, and 0.175-0.2), and one random draw can be made in each region. In the example below, we illustrate exactly this, for the \"alpha\" parameter in the demo scope." ] }, { "cell_type": "code", "execution_count": 8, "id": "5c08f1cd", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.729575Z", "start_time": "2021-06-28T16:15:49.399373Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAABjCAYAAAB+OXB0AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAH6UlEQVR4nO3df6hfdR3H8ef7trnptra51RqbtmwyqZjTTDJDFkXkIFfsHyViURDSD/MPSSEIISoaVH8KZoFUFNgsR1AY4f5IU7yb23TMqQvFqSmOiXPpWt53f3zPza+X2935/rrf+/2c5wO+fM/3nPP58Lnv87mve77ne++5kZlIkso0NuwBSJIGx5CXpIIZ8pJUMENekgpmyEtSweYNquNly5bl+vXrB9V9o5w8eZJFixYNexjFsJ79ZT37Z8+ePS9n5rv62efAQn7VqlWMj48PqvtG2b17N5s3bx72MIphPfvLevZPRDzT7z69XCNJBTPkJalghrwkFcyQl6SCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQz5Dhx77RT7n32FY6+dGvZQhsYazA0eh+41rXbzhj2AUXHPvue4eecB5o+NcXpigh3bNnLNpjXDHtassgZzg8ehe02snWfyNRx77RQ37zzAG6cnOHHqP7xxeoJv7zzQmDMBsAZzhcehe02tnSFfw9HjrzN/7O2lmj82xtHjrw9pRLPPGswNHofuNbV2hnwNa5efzemJibetOz0xwdrlZw9pRLPPGswNHofuNbV2hnwNKxYvYMe2jSycP8aSBfNYOH+MHds2smLxgmEPbdZYg7nB49C9ptbOD15rumbTGq5cv5Kjx19n7fKzi58Y07EGc4PHoXtNrJ0h34EVixc0YlLMxBrMDR6H7jWtdl6ukaSCGfKSVDBDXpIKZshLUsEMeUkqmCEvSQUz5CWpYIa8JBXMkJekghnyklQwQ16SCmbIS1LBDHlJKpghL0kFM+QlqWCGvCQVzJCXpIIZ8pJUMENekgpmyEtSwQx5SSqYIS9JBYvMHEzHESeAwwPpvHlWAi8PexAFsZ79ZT37Z0NmLulnh/P62dkUhzPzsgH23xgRMW4t+8d69pf17J+IGO93n16ukaSCGfKSVLBBhvztA+y7aaxlf1nP/rKe/dP3Wg7sg1dJ0vB5uUaSCmbIS1LBaoV8RHwmIg5HxFMRccs02y+KiL9HxKmIuKlO24g4NyL+EhFPVs/Le/9yRkO39YyI8yLivog4FBEHI+JbbdtujYjnImJf9dgyW1/PMPU4N5+OiEereo23rXdudj43N7TNvX0R8WpE3Fhta+TchFr1/EJEHKgeD0TExWdq2/H8zMwZH8A7gCPABcBZwH7gA1P2eTfwEeD7wE112gI7gFuq5VuAH51pLCU8eqznauDSankJ8ERbPW9t37cJj15qWW17Glg5Tb/OzS7qOaWffwLvrV43bm52UM+PAcur5auBh87UttP5WedM/nLgqcz8R2b+G/gtsLV9h8x8KTMfBk530HYrcGe1fCfwuRpjKUHX9czMFzJzb7V8AjgErJmdYc9JvczNmTg3e6vnJ4EjmfnM4IY6EurU84HMPF69fBBYW6NtR/OzTsivAZ5te32U+sEyU9tVmfkCtMKL1hlCE/RSz/+JiHXAJcBDbau/Ub3t+0VDLjH0WssE7o2IPRHx1bb1zs2WruYmcC3wmynrmjY3ofN6fgX4U422Hc3POiEf06yr+3uXvbQtVc81iYjFwE7gxsx8tVp9G/B+YBPwAvDjHsY4Knqt5ZWZeSmtt8lfj4ir+jOskdWPuXkWcA1wV9vqJs5N6KCeEfEJWiF/c6dtz6ROyB8Fzmt7vRZ4vmb/M7V9MSJWA1TPL9Xsc9T1Uk8iYj6tgP91Zt49uT4zX8zMNzNzAvgZrbd7peuplpn5fPX8EvB73qqZc7Olo3pWrgb2ZuaLkysaOjehZj0jYiNwB7A1M4/VaNvR/KwT8g8DF0bE+6qf0tcCu2q0O1PbXcD2ank7cE/NPkdd1/WMiAB+DhzKzJ9M2ba67eXngcf6NN65rJdaLoqIJZPLwKd5q2bOzc6/1yddx5RLNQ2dm1CjnhFxPnA38MXMfKJm287mZ81PibfQ+k2OI8B3qnXXA9dXy++h9ZPnVeCVavmd/69ttX4F8Ffgyer53GF/Gj5bj27rCXyc1lu2A8C+6rGlavNL4NFq2y5g9bC/zjleywto/cbCfuCgc7O3elbbzgGOAUun9NnIuVmznncAx9u+n8dnaptdzE9vayBJBfMvXiWpYIa8JBXMkJekghnyklQwQ16SCmbIq0jVHSZX9rqPNOoMeUkqmCGvkRcRf6huMnZwyo3GiIh1EfF4RNxZ3SDrdxFxTtsu34yIvdV95S+q2lxe3dv7kep5w6x+QVIfGfIqwZcz88PAZcANEbFiyvYNwO2ZuZHWX2p+rW3by9m6SdltwOQ/wXgcuCozLwG+C/xgoKOXBsiQVwluiIj9tO7HfR5w4ZTtz2bm/dXyr2jdHmLS5E3e9gDrquWlwF0R8RjwU+CDgxi0NBsMeY20iNgMfAq4IjMvBh4BFk7Zbeq9O9pfn6qe3wTmVcvfA+7LzA8Bn52mP2lkGPIadUuB45n5r+qa+ken2ef8iLiiWr4O+FuNPp+rlr/Ul1FKQ2LIa9T9GZgXEQdonYE/OM0+h4Dt1T7n0rr+PpMdwA8j4n5a/2tTGlnehVJFq/5N4h+rSy9S43gmL0kF80xekgrmmbwkFcyQl6SCGfKSVDBDXpIKZshLUsH+Cwd5zD1APWunAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "n=4\n", "design = design_experiments(scope, n_samples=n, random_seed=0)\n", "ax = design.plot.scatter(x='alpha', y='initial_capacity', grid=True, figsize=(6,1))\n", "xlim = design.scope['alpha'].range\n", "ax.set_xlim(xlim)\n", "ax.set_xticks(np.linspace(*xlim, n+1))\n", "ax.set_ylabel('')\n", "ax.set_yticks([]);" ] }, { "cell_type": "markdown", "id": "5d308a8a", "metadata": {}, "source": [ "This ensures better coverage of the entire input range than making four random draws from the full space, which could easily result in a cluster of observations in one part of the space and large void elsewhere. \n", "\n", "Generating a multi-dimensional Latin Hypercube sample for use with multiple input variables follows this same basic technique, although the various draws in each dimension are randomly reordered before being joined with draws in other dimensions, to avoid unintended correlation. Various techniques exist to conduct this random reordering, although there is no agreement in the literature about the “best” orthogonal Latin hypercube design. In most instances, a simple random reordering is sufficient to get pretty good coverage of the space; this is the default in TMIP-EMAT. If a greater degree of orthogonality is desired, one simple method to achieve it is to generate numerous possible sets of random draws, and greedily select the set that minimizes the maximum correlation.\n", "\n", "In the figure below, we can see the result across two dimensions for the design of four draws we made previously. Each dimension is divided in 4, so that each grid row and each grid column has exactly one experiment. " ] }, { "cell_type": "code", "execution_count": 9, "id": "ae306a05", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.822285Z", "start_time": "2021-06-28T16:15:49.731539Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAELCAYAAAALC/uGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAYF0lEQVR4nO3de7SddX3n8ffnkGMSDR1DEhGJKAhNRRszI6Vjsa4o1SqtQY03Zix4K7qqU52lIzqjFUftshm1l5l6wUuljuAtIlg7FUuNaFtFLiFyMV4QNMBAjVE5nZCecL7zx35STuI5J+ck+5edbN6vtfbaz/Pbz+W7H3bOh+f2e1JVSJLUwsigC5AkDS9DRpLUjCEjSWrGkJEkNWPISJKaMWQkSc00DZkkH05yZ5LrJrWdm+TWJBu712mTPntDku8m2ZzkN6dZ5hFJvpjkO9374pbfQZK071rvyXwEeOoU7X9cVau6118DJDkReD7wqG6e9yQ5bIp5Xw9cVlUnAJd145Kkg1DTkKmqy4Efz3Ly04GPV9WOqvo+8F3g5GmmO78bPh94xv7WKUlqY96A1vvKJGcCVwKvqaptwNHA1yZNs6Vr29ORVXU7QFXdnuRB060kydnA2QALFix47DHHHNOv+u/zJiYmGBnxlF4/uC37y+3ZX9/+9rd/VFXL9nX+QYTMe4G3AtW9vwt4MZAppt2vPm+q6jzgPIAVK1bU5s2b92dxmmTDhg2sXr160GUMBbdlf7k9+yvJLfsz/wGP+6q6o6ruqaoJ4APce0hsC/DQSZMuB26bYhF3JDkKoHu/s2W9kqR9d8BDZldAdJ4J7Lry7BLg+UnmJzkWOAG4YopFXAKc1Q2fBVzcqlZJ0v5pergsyYXAamBpki3Am4HVSVbROxR2M/AygKq6PskngRuAncArquqebjkfBN5XVVcC7wA+meQlwA+A57T8DpKkfdc0ZKrqjCmaPzTD9G8H3j5F+0snDW8FTu1LgZKkprwEQ5LUjCEjSWrGkJEkNWPISJKaMWQOUVvHdnDtD3/C1rEdgy5FkqY1qG5ltB8u3ngr56zfxOjICOMTE6xbu5I1q6bqgUeSBss9mUPM1rEdnLN+E3ePT3DXjp3cPT7B69Zvco9G0kHJkDnEbNm2ndE9Ov8bHRlhy7btA6pIkqZnyBxili9eyPjExG5t4xMTLF+8cEAVSdL0DJlDzJJF81m3diULRkc4fP48FoyOsG7tSpYsmj/o0iTp53ji/xC0ZtXRnHL8UrZs287yxQsNGEkHLUPmELVk0XzDRdJBz8NlkqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjNNQybJh5PcmeS6SW1vTbIpycYklyZ5SNf+5CRXJflm9/6kaZZ5bpJbu/k3Jjmt5XeQJO271nsyHwGeukfb/6iqlVW1Cvgr4A+69h8BT6+qXwbOAj46w3L/uKpWda+/7nPNkqQ+mddy4VV1eZKH79H2s0mjDwCqa79mUvv1wIIk86tqR8saJUntNA2Z6SR5O3Am8FPgiVNMsha4ZoaAeWWSM4ErgddU1bZp1nM2cDbAsmXL2LBhw/6Wrs7Y2Jjbs0/clv3l9jy4pKrarqC3J/NXVfXoKT57A7Cgqt48qe1RwCXAU6rqe1PMcyS9Q2sFvBU4qqpevLc6VqxYUZs3b97n76HdbdiwgdWrVw+6jKHgtuwvt2d/Jbmqqk7a1/kHfXXZBfT2WgBIshy4CDhzqoABqKo7quqeqpoAPgCcfEAqlSTN2QEPmSQnTBpdA3yra38g8HngDVX19zPMf9Sk0WcC1003rSRpsJqek0lyIbAaWJpkC/Bm4LQkK4AJ4Bbg5d3krwSOB96U5E1d21Oq6s4kHwTeV1VXAuuSrKJ3uOxm4GUtv4Mkad+1vrrsjCmaPzTNtG8D3jbNZy+dNPw7/alOktTaoM/JSJKGmCEjSWrGkJEkNWPISJKaMWQkSc0YMpKkZgwZSVIzhowkqRlDRpLUjCEjSWrGkJEkNWPISJKaMWQkSc0YMpKkZgwZSVIzhowkqRlDRpLUjCEjSWrGkJEkNWPISJKaMWQkSc0YMpKkZgwZSVIzhowkqRlDRpLUjCEjSWpm3mwnTPI94GvAV4DLq+qGZlVJkobCXPZkTgTeDywB3pnkpiQXtSlLkjQM5hIy9wDj3fsEcAdwZ4uiJEnDYdaHy4CfAd8E3g18oKq2tilJkjQs5rIncwZwOfB7wMeTvCXJqW3KkiQNg1nvyVTVxcDFSX4JeBrwauB1wMI2pUmSDnWz3pNJsr67wuxPgUXAmcDiVoVJkg59czkn8w7g6qq6p1UxkqThMpeQ2Qi8IskTuvEvA++rqvG+VyVJGgpzCZn3AqPAe7rx3+naXtrvoiRJw2EuIfMrVfWYSeN/l+TafhckSRoec7oZM8kjdo0kOY7ejZmSJE1pLnsy/wX4UpKbgAAPA17UpCpJ0lCYy30ylyU5AVhBL2S+VVU7mlUmSTPYOraDLdu2s3zxQpYsmj/ocjSNvYZMkmdN89EjklBVn+lzTZI0o4s33so56zcxOjLC+MQE69auZM2qowddlqYwmz2Zp8/wWQGGjKQDZuvYDs5Zv4m7xye4mwkAXrd+E6ccv9Q9moPQbEJmY1X9aZLHV9VXm1ckSTPYsm07oyMj/xowAKMjI2zZtt2QOQjN5uqyXSf3/6xlIZI0G8sXL2R8YmK3tvGJCZYvthvFg9FsQubGJDcDK5JsmvT6ZpJNjeuTpN0sWTSfdWtXsmB0hMPnz2PB6Ajr1q50L+YgtdfDZVV1RpIHA18A1sxl4Uk+DPw2cGdVPbprew5wLvBI4OSqurJrfzhwI7C5m/1rVfXyKZZ5BPAJ4OHAzcBzq2rbXOqSdGhbs+poTjl+qVeXHQJmdTNmVf3fqnpMVd2y52vXNEnWTzHrR4Cn7tF2HfAses+m2dP3qmpV9/q5gOm8Hrisqk4ALuvGJd3HLFk0n8c89IEGzEFuLnf8781xezZU1eXAj/dou7GqNu857RycDpzfDZ8PPGM/liVJamgud/zvTfVhGccmuYbeo57fWFVfmWKaI6vqdoCquj3Jg6ZbWJKzgbMBli1bxoYNG/pQogDGxsbcnn3ituwvt+fBpZ8hs79uB46pqq1JHgt8Nsmjqupn+7rAqjoPOA9gxYoVtXr16v5UKjZs2IDbsz/clv3l9jy49PNwWfZn5qraUVVbu+GrgO8BvzjFpHckOQqge79zf9YrSWpnLo9fftVe2s7Zn0KSLEtyWDd8HHACcNMUk14CnNUNnwVcvD/rlSS1M5c9mbOmaHvhroGqunTPD5NcCPwjvXtstiR5SZJnJtkCPA74fJIvdJM/AdjUPaPm08DLq+rH3XI+mOSkbrp3AE9O8h3gyd24JOkgNJsOMs8A/gO9k/KXTProcGDrTPNW1RnTfHTRFNOuB6a6DJqqeumk4a3AqXspW5J0EJjNif9/oHdSfinwrkntdwHe8S9JmtZs7vi/BbiF3uEtSZJmbdaXMCe5i3vvhbkfMAr8c1X9QovCJEmHvrk8GfPwyeNJngGc3O+CJEnDY5/vk6mqzwJP6l8pkqRhM5fDZZMfwzwCnER/upKRJA2puXQrM/kxzDvpdbN/el+rkSQNlbmck3nR3qeSJOlec+lW5rgkn0vyT0nuTHJx1/2LJElTmsuJ/wuATwJHAQ8BPgVc2KIoSdJwmEvIpKo+WlU7u9f/xhP/kqQZzOXE/5eSvB74OL1weR69Di6PANjVmaUkSbvMJWSe172/bI/2F9MLHc/PSJJ2M5ery45tWYgkafjM6fHLSX4NePjk+arqL/tckyRpSMzljv+PAo8ANgL3dM0FGDKSpCnNZU/mJODEqvKKMknSrMzlEubrgAe3KkSSNHzmsiezFLghyRXAjl2NVbWm71VJkobCXELm3FZFSJKG01wuYf5yy0IkScNnryGT5KtV9fg9Hr8MEKB8/LIkaTp7DZmqenz3fvjeppUkabJ9fvyyJEl7Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBk1t3VsB9f+8CdsHdux94klDZW5PLRMmrOLN97KOes3MToywvjEBOvWrmTNqqMHXZakA8Q9GTWzdWwH56zfxN3jE9y1Yyd3j0/wuvWb3KOR7kMMGTWzZdt2Rkd2/4mNjoywZdv2AVUk6UAzZNTM8sULGZ+Y2K1tfGKC5YsXDqgiSQeaIaNmliyaz7q1K1kwOsLh8+exYHSEdWtXsmTR/EGXJukA8cS/mlqz6mhOOX4pW7ZtZ/nihQaMdB9jyKi5JYvmGy7SfZSHyyRJzRgykqRmBna4LMmrgN8FAnygqv4kySeAFd0kDwR+UlWrppj3ZuAu4B5gZ1WddCBqliTNzUBCJsmj6QXMycC/AH+T5PNV9bxJ07wL+OkMi3liVf2obaWSpP0xqMNljwS+VlX/r6p2Al8GnrnrwyQBngtcOKD6JEl9MKiQuQ54QpIlSe4PnAY8dNLnvw7cUVXfmWb+Ai5NclWSsxvXKknaRwM5XFZVNyb5I+CLwBhwLbBz0iRnMPNezClVdVuSBwFfTPKtqrp8z4m6ADobYNmyZWzYsKFfX+E+b2xszO3ZJ27L/nJ7HlxSVYOugSR/CGypqvckmQfcCjy2qrbMYt5zgbGqeudM061YsaI2b97cl3oFGzZsYPXq1YMuYyi4LfvL7dlfSa7an4urBnYJc7cXQpJjgGdx757LbwDfmi5gkjwgyeG7hoGn0Dv8Jkk6yAzyjv/1SZYA48Arqmpb1/589jhUluQhwAer6jTgSOCi3rUBzAMuqKq/OXBlS5Jma2AhU1W/Pk37C6dou43exQFU1U3AY5oWJ0nqC+/4lyQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjOGjCSpGUNGkgZk69gOrv3hT9g6tmPQpTQzb9AFSNJ90cUbb+Wc9ZsYHRlhfGKCdWtXsmbV0YMuq+/ck5GkA2zr2A7OWb+Ju8cnuGvHTu4en+B16zcN5R6NISNJB9iWbdsZHdn9z+/oyAhbtm0fUEXtGDKSdIAtX7yQ8YmJ3drGJyZYvnjhgCpqx5CRpANsyaL5rFu7kgWjIxw+fx4LRkdYt3YlSxbNH3RpfeeJf0kagDWrjuaU45eyZdt2li9eOJQBA4aMJA3MkkXzhzZcdvFwmSSpGUNGktSMISNJasaQkSQ1Y8hIkpoxZCRJzRgykqRmDBlJUjMDC5kk/znJ9UmuS3JhkgVJjkjyxSTf6d4XTzPvU5NsTvLdJK8/0LVLkmZnICGT5Gjg94GTqurRwGHA84HXA5dV1QnAZd34nvMeBvw58DTgROCMJCceqNolSbM3yMNl84CFSeYB9wduA04Hzu8+Px94xhTznQx8t6puqqp/AT7ezSdJOsgMpO+yqro1yTuBHwDbgUur6tIkR1bV7d00tyd50BSzHw38cNL4FuBXp1pPkrOBs7vRHUmu69uX0FLgR4MuYki4LfvL7dlfK/Zn5oGETHeu5XTgWOAnwKeSvGC2s0/RVlNNWFXnAed167yyqk6ae7Waituzf9yW/eX27K8kV+7P/IM6XPYbwPer6p+qahz4DPBrwB1JjgLo3u+cYt4twEMnjS+nd6hNknSQGVTI/AD490nunyTAqcCNwCXAWd00ZwEXTzHvN4ATkhyb5H70Lhi45ADULEmao0Gdk/l6kk8DVwM7gWvoHdZaBHwyyUvoBdFzAJI8BPhgVZ1WVTuTvBL4Ar2r0j5cVdfPYrXnNfgq92Vuz/5xW/aX27O/9mt7pmrK0xmSJO037/iXJDVjyEiSmhn6kLELmv2T5KFJvpTkxq4boFd17ecmuTXJxu512qBrPVQkuTnJN7vtdmXXNqsulXSvJCsm/f42JvlZklf725y9JB9Ocufkewhn+i0meUP3t3Rzkt+c1TqG+ZxM1wXNt4En07v0+RvAGVV1w0ALO4R0l5IfVVVXJzkcuIpeTwzPBcaq6p2DrO9QlORmel0q/WhS2zrgx1X1ju5/hhZX1TmDqvFQ0/1bv5Xejdkvwt/mrCR5AjAG/GXXxde0v8Wu+64L6fW68hDgb4FfrKp7ZlrHsO/J2AXNfqqq26vq6m74LnqXmh892KqG0my6VNL0TgW+V1W3DLqQQ0lVXQ78eI/m6X6LpwMfr6odVfV94Lv0/sbOaNhDZqouaPwDuY+SPBz4t8DXu6ZXJtnU7XJ7eGf2Crg0yVVd10cAu3WpBEzVpZKm93x6/5e9i7/NfTfdb3Gf/p4Oe8jMugsazSzJImA98Oqq+hnwXuARwCrgduBdg6vukHNKVf07ej2Jv6I7ZKF91N2UvQb4VNfkb7ONffp7OuwhYxc0fZBklF7AfKyqPgNQVXdU1T1VNQF8gFnsNqunqm7r3u8ELqK37WbTpZKm9jTg6qq6A/xt9sF0v8V9+ns67CFjFzT7qev250PAjVX17kntR02a7JmAPVzPQpIHdBdQkOQBwFPobbvZdKmkqZ3BpENl/jb323S/xUuA5yeZn+RY4ATgir0tbKivLgPoLl/8E+7tgubtg63o0JLk8cBXgG8CE13zf6X3D3sVvd3lm4GX7TqOq+klOY7e3gv0unW6oKrenmQJ8EngGLoulapqzxOy2kOS+9M7T3BcVf20a/so/jZnJcmFwGp6j0e4A3gz8Fmm+S0m+W/Ai+l1B/bqqvo/e13HsIeMJGlwhv1wmSRpgAwZSVIzhowkqRlDRpLUjCEjSWrGkJEkNWPISAdA173/0kHXMVmSlyc5sxt+YfeYc6mv5g26AEmDUVXvmzT6Qnp3xtvtkvrKPRkNvSQvSHJF9wCr9yf51a6H3gVdNy/XJ3l0ktVJLk9yUZIbkrwvyUi3jPcmubKb9i2Tln1zkrckubp7ENkvde1Lklya5Jok72fqzgUn13hmV9O13R3rJHl6kq93y/jbJEd27ecm+WiSv+seLPW7XfuiJJdNquX0vSz/3CSvTfJs4CTgY902+q0kF02a98lJPtOn/xy6r6kqX76G9gU8EvgcMNqNvwc4E3gb8E7gz4E3dJ+tBu4GjqPXDdEXgWd3nx3RvR8GbABWduM3A/+pG/494IPd8J8Bf9AN/xa9Lk6WTlPjo4DNuz6ftK7F3Nsrx0uBd3XD5wLXAgvpdQfyQ3oPkZoH/EI3zVJ6z/vIDMs/F3htN7yB3oPU6Ob5FrCsG78AePqg/1v6OjRfHi7TsDsVeCzwjV5fnyyk16vsf6fXgerdwO9Pmv6KqroJ/rVfp8cDnwae2z37ZR5wFHAisKmbZ9f/5V8FPKsbfsKu4ar6fJJtM9T4JODT1T0ps+7ts2w58Imuw8f7Ad+fNM/FVbUd2J7kS/R6Gv488IfdowMm6D3r48gZlj+lqqpub+cFSf4CeBy9YJbmzJDRsAtwflW9YbfG5MHAImAUWAD8c/fRnp35Vdfj7GuBX6mqbUk+0s2zy47u/R52/zc1244BM820/xN4d1VdkmQ1vT2P6ZZdwH8ElgGPrarx7jHPC2ZY/kz+gt4e4N3Ap6pq5xznlwDPyWj4XQY8O8mDAJIckeRhwHnAm4CPAX80afqTu0dDjADPA74K/AK9EPppd17kabNY7+X0/uiT5Gn0Dn3NVONzu56YSXJE1/5v6D23Hu7ten2X07tzSkvoHeb7Rjf9nV3APBF42F6WP9ldwOG7Rqr3zJvbgDcCH9nbl5Wm456MhlpV3ZDkjfQedzwCjNN7PsbOqrogyWHAPyR5Er1DTP8IvAP4ZXpBcVFVTSS5BrgeuAn4+1ms+i3AhUmuBr5Mr8v06Wq8PsnbgS8nuQe4ht7VXucCn0pyK/A14NhJs11B7/DYMcBbq+q2JB8DPpfkSmAjvfMqMy1/so8A70uyHXhcdyjuY/TOy9wwi+8rTcmu/qVOd0jqtVX12wMuZUZJzgXGquqdjdfzv4BrqupDLdej4eaejKSfk+QqeocIXzPoWnRoc09GOkC6cyKXTfHRqVW19UDXIx0IhowkqRmvLpMkNWPISJKaMWQkSc0YMpKkZv4/Ah9Nf0DkE98AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "ax = design.plot.scatter(x='expand_capacity', y='input_flow', grid=True)\n", "xlim = design.scope['expand_capacity'].range\n", "ylim = design.scope['input_flow'].range\n", "ax.set_xlim(xlim)\n", "ax.set_ylim(ylim)\n", "ax.set_xticks(np.linspace(*xlim, n+1))\n", "ax.set_yticks(np.linspace(*ylim, n+1));" ] }, { "cell_type": "markdown", "id": "5d1bdbca", "metadata": {}, "source": [ "The Latin Hypercube design of experiments is advantageous over a factorial or grid-based design, as every experimental observation can provide useful information, even when some input factors are potentially unimportant or spurious. \n", "\n", "As an example, consider the illustration below: 16 experiments are designed, either by making an evenly spaced grid over a two dimensional design space (i.e. a \"full factorial\" design) or by making the same number of latin hypercube draws." ] }, { "cell_type": "code", "execution_count": 10, "id": "9415057e", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:49.940134Z", "start_time": "2021-06-28T16:15:49.823669Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbOUlEQVR4nO3df5Ac5X3n8ffHCwitFozPWoGXBQRVsh3kAlvaExClsCAOFkJApe6uDgLOOci1JZ+lECiC4a4KYt8R+8Jd4EIcq1RCKRwOU5yNCCWDLFfZlM8QCLv8MhhQyVg2Yn23C+bXggqj5Xt/dG80jHp2Z3eme3pnPq+qqel5nqdnnuaL9jtPP890KyIwMzOr9oFWd8DMzMrJCcLMzDI5QZiZWSYnCDMzy+QEYWZmmQ5pdQeaaeHChbF48eJWd8PMbM4YHh5+OSJ6s+raKkEsXryYoaGhVnfDzGzOkPTLWnU+xWRmZpmcIMzMLJMThJmZZXKCMDOzTE4QAOPjsGtX8mzl4JiUk+NSPjnGJNcEIWmPpJ9KekLSQcuLJK2S9Hpa/4Sk6yrqVkt6XtJuSdfk0sH9+2HjRli0CJYvT543bkzKrTUck3JyXMqniJhERG4PYA+wcIr6VcD2jPIu4OfAScBhwJPAydN93vLly2NGNmyI6O6OgAOP7u6k3FrDMSknx6V8mhQTYChq/E1V5Hi5b0l7gIGIeLlG/SrgqohYW1V+BvAXEfHZ9PW1ABHxtak+b2BgIOr+HcT4eJJx9+07uG7+fBgdhZ6e+t7LmsMxKSfHpXyaGBNJwxExkFWX9xxEADslDUsarNHmDElPSrpf0tK07FjgxYo2e9Oyg0galDQkaWhsbKz+no2MQFdXdl1XV1JvxXJMyslxKZ+CYpL3L6lXRsSIpEXADyQ9FxE/rqh/DDghIsYlrQHuAZYAynivzKFORGwGNkMygqi7Z319MDGRXTcxkdRbsRyTcnJcyqegmOQ6goiIkfR5FNgGrKiqfyMixtPt+4BDJS0kGTEcV9G0H2ju15SeHli3Drq731/e3Z2Ue8hcPMeknByX8ikqJrUmJxp9AAuAIyq2HwJWV7U5Bv5lHmQF8CuS0cMhwAvAiRyYpF463WfOeJL63XeTCZ358yN6epLnDRuScmsNx6ScHJfyaVJMaMUktaSTSEYNpH/w74iIGyStTxPTJkkbgC8C+4F9wJUR8VC6/xrgZpIVTVsj4obpPnNGk9SVxseTc3Z9ff42VBaOSTk5LuXTYEymmqTOdRVT0WadIMzMOlQrVzGZmdkc5QRhZmaZnCDMzCxTW91RzswsD0uv38Fb7xz8u4MF87p45iurW9CjYngEYWY2jazkMFV5u3CCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzs2ksmJd9ae1a5e3Cy1zNzKbRzktZp+IE0eY6df22mTXOp5jaXKeu3zazxjlBmJlZJicIMzPL5ARhZmaZcp2klrQHeBOYAPZX35RC0iXAl9OX48AXI+LJevY1M7N8FbGK6ayIeLlG3S+AT0fEq5LOBTYDp9W5r9VhwbyumquYzMym0tJlrpP3n049DPS3qi/tqlVLWb281mzuy3sOIoCdkoYlDU7Tdh1w/0z3lTQoaUjS0NjYWBO6bM3g5bVmc1/eI4iVETEiaRHwA0nPRcSPqxtJOoskQfzeTPeNiM0kp6YYGBiIfA7DzKzz5DqCiIiR9HkU2AasqG4j6RRgC3BhRLwyk33NzCw/uSUISQskHTG5DZwDPF3V5njgbuBzEbFrJvuamVm+8jzFdDSwTdLk59wRETskrQeIiE3AdcCHgb9L200uZ83cN8e+mplZldwSRES8AJyaUb6pYvsLwBfq3dfmDi+vNZv7fDVXy4WXsprNfb7UhpmZZXKCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzM8vkBGFmZpmcIMzMLJMThJmZZXKCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzM8vky32bWcdbev2Omvcv6eRL1+eaICTtAd4EJjhwt7jKegH/E1gDvA18PiIeS+tWp3VdwJaI+HqefTWzmWmnP6pZxzFVeacoYgRxVkS8XKPuXGBJ+jgN+CZwmqQu4BvAHwB7gUcl3RsRPyugv2ZWB/9RbX+tnoO4EPhWJB4GjpL0EWAFsDsiXoiI3wJ3pm3NzKwgeSeIAHZKGpY0mFF/LPBixeu9aVmt8oNIGpQ0JGlobGysSd02M7O8E8TKiFhGcirpS5LOrKpXxj4xRfnBhRGbI2IgIgZ6e3sb662Zmf2LXBNERIykz6PANpJTR5X2AsdVvO4HRqYoNzNrugXzumZU3ilym6SWtAD4QES8mW6fA3y1qtm9wAZJd5JMUr8eEb+WNAYskXQi8BJwEfBHefXVzGZuwbyumquY5pq5tuqqKHmuYjoa2JasZOUQ4I6I2CFpPUBEbALuI1niuptkmeufpHX7JW0Avk+yzHVrRDyTY1/NbIb8R7X9KSLz1P6cNDAwEENDQ63uhpnZnCFpuPo3apNavczVzMxKygnCzMwyOUGYmVkmJwgzM8vkq7lax2mni8yZ5ckjCOs4vsicWX2cIADGx2HXruTZysExKSfHpXxyjElnJ4j9+2HjRli0CJYvT543bkzKrTUck3JyXMqngJh09hzEFVfA1q2wb9+Bsq1bk+dbbmlNnzqdY1JOjkv5FBCTzv0l9fh4knEr/+NOmj8fRkehp6e5HbSpFRSTxdd8r2bdnq+f1/D7tx3/WymfJsbEv6TOMjICXTUuKtbVldRbsQqKia/cOUP+t1I+BcWkc08x9fXBRI1VKxMTSb0Vq6CYeCnrDPnfSvkUFJPOHUH09MC6ddDd/f7y7u6k3EPm4jkm5eS4lE9BMencEQTATTclz7femgzLJibgsssOlFvxHJNyclzKp4CYdO4kdaXx8eScXV+fvw2VhWNSTo5L+TQYk6kmqXMfQUjqAoaAlyJibVXdnwOXVPTld4DeiPiNpD3Am8AEsL/WATRFTw989KO5vb3NgmNSTo5L+eQYkyJOMV0OPAscWV0RETcCNwJIOh+4IiJ+U9HkrIh4uYA+mplZlVwnqSX1A+cBW+pofjHw7Tz7Y2Zm9ct7FdPNwNXAe1M1ktQNrAa+W1EcwE5Jw5IGp9h3UNKQpKGxsbEmdNnMzCDHBCFpLTAaEcN1ND8feLDq9NLKiFgGnAt8SdKZWTtGxOaIGIiIgd7e3sY7bmZmQL4jiJXABelk853A2ZJur9H2IqpOL0XESPo8CmwDVuTXVTMzq5ZbgoiIayOiPyIWkySAH0bEpdXtJH0Q+DTwjxVlCyQdMbkNnAM8nVdfzczsYIX/UE7SeoCI2JQW/SGwMyLeqmh2NLBNEiR9vCMidhTaUTOzDucfypmZdTBfzdXMzGass6/F1EJLr9+ReQ/kBfO6fLVRMysFjyBaJCs5TFVuZlY0JwgzM8vkBGFmZpmcIMzMLJMnqW3O8QS/WTE8gmiRBfOybzheq9wO8AS/WTE8gmgRf9M1s7LzCMLMzDI5QZiZWaa6TzFJOg9YChw+WRYRX82jU2Zm1np1jSAkbQL+PbAREPDvgBNy7JdZTZ7gNytGvSOI342IUyQ9FRFfkfQ/gLvz7JhZLZ7gNytGvXMQ+9LntyX1Ae8CJ+bTJTMzK4N6RxDbJR0F3Ag8BgSwJa9OmZlZ69U7gviriHgtIr5LMvfwceC/1rOjpC5Jj0vanlG3StLrkp5IH9dV1K2W9Lyk3ZKuqbOfZmbWJPWOIP4JWAYQEe8A70h6bLJsGpcDzwJH1qj/PxGxtrJAUhfwDeAPgL3Ao5LujYif1dlfMzNr0JQJQtIxwLHAfEmfIlnBBMkf++7p3lxSP3AecANw5Qz6tQLYHREvpO9zJ3Ah4ARhZlaQ6UYQnwU+D/QDf11R/gbwn+p4/5uBq4EjpmhzhqQngRHgqoh4hiQpvVjRZi9wWtbOkgaBQYDjjz++ji6ZmVk9pkwQEXEbcJukf5POP9RN0lpgNCKGJa2q0ewx4ISIGJe0BrgHWMKBkcr7ulOjj5uBzQADAwOZbdqJr2RqZkWpd5L6QUm3SrofQNLJktZNs89K4AJJe4A7gbMl3V7ZICLeiIjxdPs+4FBJC0lGDMdVNO0nGWF0PF/J1MyKUm+C+Hvg+0Bf+noX8GdT7RAR10ZEf0QsBi4CfhgRl1a2kXSMJKXbK9L+vAI8CiyRdKKkw9L9762zr2Zm1gT1JoiFEXEX8B5AROwHZvWVVdJ6SevTl/8WeDqdg/gb4KJI7Ac2kCSlZ4G70rkJMzMrSL3LXN+S9GHSeQBJpwOv1/shEfEA8EC6vami/G+Bv62xz33AffV+hpmZNVe9CeJKklM8J0l6EOgl+fZvZmZtqt4E8TNgG/A28CbJaqNdOfXJprBgXlfNVUxmZs1Ub4L4FslvH/4yfX0x8A8kl/22Ankpq5kVpd4E8bGIOLXi9Y/SiWUzM2tT9a5iejydmAZA0mnAg/l0yczMymC6azH9lGTl0qHAH0v6Vfr6BHxdJDOztjbdKaa109SbmVmbmu5aTL8sqiNmZlYu9c5BmJlZh3GCMDOzTE4QZmaWyQnCzMwyOUGYmVkmJwgzM8vkBGFmZpnqvRaTmZm1WNH3pM99BCGpS9LjkrZn1F0i6an08ZCkUyvq9kj6qaQnJA3l3U8zs7Ir+p70RYwgLie5beiRGXW/AD4dEa9KOhfYDJxWUX9WRLxcQB/NzKxKriMISf3AecCWrPqIeCgiXk1fPgz059kfMzOrX96nmG4Grgbeq6PtOuD+itcB7JQ0LGmw1k6SBiUNSRoaGxtrqLNmZnZAbglC0lpgNCKG62h7FkmC+HJF8cqIWAacC3xJ0plZ+0bE5ogYiIiB3t7eZnTdzMzIdwSxErhA0h7gTuBsSbdXN5J0CskpqAsj4pXJ8ogYSZ9HSe6HvSLHvpqZlV6te8/ndU/63CapI+Ja4FoASauAqyLi0so2ko4H7gY+FxG7KsoXAB+IiDfT7XOAr+bVVzOzuaDoe9IX/kM5SeslrU9fXgd8GPi7quWsRwM/Se97/c/A9yJiR26dGh+HXbuSZysHx6ScHJfyyTEmhSSIiHggItam25siYlO6/YWI+FBEfDJ9DKTlL0TEqeljaUTckEvH9u+HjRth0SJYvjx53rgxKbfWcEzKyXEpnwJi0tm/pL7iCti6FfbtO1C2dWvyfMstrelTp3NMyslxKZ8CYqKIaMoblcHAwEAMDdX5o+vx8STjVv7HnTR/PoyOQk9PcztoU3NMyslxKZ8mxkTS8OTZm2qdO4IYGYGuGjP/XV1J/Uc/WmyfOl0HxKToa+k0RQfEZc4pKCadezXXvj6YqHH9komJpN6K1QExKfpaOk3RAXGZcwqKSecmiJ4eWLcOurvfX97dnZR7yFw8x6ScHJfyKSgmnXuKCeCmm5LnW29NhmUTE3DZZQfKrXiOSTk5LuVTQEw6d5K60vh4cs6ur8/fhsqiTWOy+Jrv1azb8/XzZvWehc5rtGlc5rQGY+JJ6un09HiSrWwck7oVOq/huJRPjjHp3DkIsxYo+lo6Zo3wCMKsQKVdymqWwSMIMzPL5ARhZmaZnCDM5jjPa1hePAdhNsd5XsPy4hGEmZllyj1BSOqS9Lik7Rl1kvQ3knZLekrSsoq61ZKeT+uuybufZmb2fkWMIC4Hnq1Rdy6wJH0MAt+EJKkA30jrTwYulnRy/l01M7NJuSYISf3AecCWGk0uBL4ViYeBoyR9BFgB7E7vLPdb4M60rZmZFSTvEcTNwNXAezXqjwVerHi9Ny2rVX4QSYOShiQNjY2NNdxhMzNL5JYgJK0FRiNieKpmGWUxRfnBhRGbI2IgIgZ6e3tn0VMzM8uS5zLXlcAFktYAhwNHSro9Ii6taLMXOK7idT8wAhxWo9zMzAqS2wgiIq6NiP6IWAxcBPywKjkA3Av8cbqa6XTg9Yj4NfAosETSiZIOS/e/N6++mpnZwQr/oZyk9QARsQm4D1gD7AbeBv4krdsvaQPwfaAL2BoRzxTdVzOzTuYbBpmZdbCpbhjkX1KbmVkmJwgzM8vkBGFmZpl8NVezCkuv35F5L+cF87p81VTrOB5BmFXISg5TlZu1MycIMzPL5ARhZmaZnCDMzCyTJ6nNbM7yooJ8eQRhVmHBvK4ZlVtreVFBvjyCMKvgb51mB3gEYWZmmZwgzMwskxOEmZllcoIwsznLiwry5UlqM5uzvKggX7klCEmHAz8G5qWf852IuL6qzZ8Dl1T05XeA3oj4jaQ9wJvABLC/1g0tzMwsH3mOIN4Bzo6IcUmHAj+RdH9EPDzZICJuBG4EkHQ+cEVE/KbiPc6KiJdz7KOZmdWQW4KI5F6m4+nLQ9PHVPc3vRj4dl79MTOzmcl1klpSl6QngFHgBxHxSI123cBq4LsVxQHslDQsaXCKzxiUNCRpaGxsrIm9NzPrbLkmiIiYiIhPAv3ACkmfqNH0fODBqtNLKyNiGXAu8CVJZ9b4jM0RMRARA729vc3svplZRytkmWtEvAY8QDJKyHIRVaeXImIkfR4FtgEr8uuhmZlVyy1BSOqVdFS6PR/4DPBcRrsPAp8G/rGibIGkIya3gXOAp/Pqq5mZHSzPVUwfAW6T1EWSiO6KiO2S1gNExKa03R8COyPirYp9jwa2SZrs4x0RsSPHvpqZWRUli43aw8DAQAwNDRX6mb4evZnNZZKGa/3OzJfaaJCvR29m7coJwszMMjlBmJlZJicIMzPL5ARhZmaZnCAa5OvRm1m78v0gGuSlrGbWrpwgzKww/t3Q3OJTTGZWGP9uaG5xgjAzs0xOEGZmlskJwszMMjlBmJlZJicIMyuMfzc0t3iZq5kVxktZ5xaPIADGx2HXruTZysExKSfHpXxyjEmetxw9XNI/S3pS0jOSvpLRZpWk1yU9kT6uq6hbLel5SbslXZNLJ/fvh40bYdEiWL48ed64MSm31nBMyslxKZ8iYhIRuTwAAT3p9qHAI8DpVW1WAdsz9u0Cfg6cBBwGPAmcPN1nLl++PGZkw4aI7u4IOPDo7k7KrTUck3JyXMqnSTEBhqLG39RCbjkqqRv4CfDFiHikonwVcFVErK1qfwbwFxHx2fT1tQAR8bWpPmdGtxwdH08y7r59B9fNnw+jo9DTU997WXM4JuXkuJRPE2PSsluOSuqS9AQwCvygMjlUOCM9DXW/pKVp2bHAixVt9qZlWZ8xKGlI0tDY2Fj9nRsZga4aKye6upJ6K5ZjUk6OS/kUFJNcE0RETETEJ4F+YIWkT1Q1eQw4ISJOBW4B7knLlfV2NT5jc0QMRMRAb29v/Z3r64OJGtd/mZhI6q1Yjkk5OS7lU1BMClnFFBGvAQ8Aq6vK34iI8XT7PuBQSQtJRgzHVTTtB5r7NaWnB9atg+7u95d3dyflHjIXzzEpJ8elfAqKSW6/g5DUC7wbEa9Jmg98BvhvVW2OAf5fRISkFSQJ6xXgNWCJpBOBl4CLgD9qeidvuil5vvXWZFg2MQGXXXag3IrnmJST41I+BcQkt0lqSacAt5GsSPoAcFdEfFXSeoCI2CRpA/BFYD+wD7gyIh5K918D3JzuvzUibpjuM2c0SV1pfDw5Z9fX529DZeGYlJPjUj4NxmSqSepCVjEVZdYJwsysQ7VsFZOZmc1dThBmZpbJCcLMzDI5QZiZWaa2mqSWNAb8cpa7LwRebmJ35gIfc/vrtOMFH/NMnRARmb8ybqsE0QhJQ7Vm8tuVj7n9ddrxgo+5mXyKyczMMjlBmJlZJieIAza3ugMt4GNuf512vOBjbhrPQZiZWSaPIMzMLJMThJmZZWr7BCFptaTnJe2WdE1G/ccl/ZOkdyRdNZN9y2q2xyzpOEk/kvSspGckXV5sz2evkTin9V2SHpe0vZgeN67B/7ePkvQdSc+l8T6juJ7PXoPHfEX6//XTkr4t6fDiej57dRzzJZKeSh8PSTq13n2nVetm1e3wILlU+M+Bk4DDgCeBk6vaLAL+NXADyf2x6963jI8Gj/kjwLJ0+whgV7sfc0X9lcAdwPZWH08Rx0xyKf4vpNuHAUe1+pjyPGaSWxb/Apifvr4L+Hyrj6lJx/y7wIfS7XOBR+rdd7pHu48gVgC7I+KFiPgtcCdwYWWDiBiNiEeBd2e6b0nN+pgj4tcR8Vi6/SbwLDXuBV4yjcQZSf3AecCWIjrbJLM+ZklHAmcCt6btfhvJXR/LrqE4k9wgbb6kQ4Bumn2XynzUc8wPRcSr6cuHSe7AWde+02n3BHEs8GLF673U/wevkX1bqSn9lrQY+BTwSHO6latGj/lm4GrgvSb2KW+NHPNJwBjw9+lptS2SFjS7gzmY9TFHxEvAfwd+BfwaeD0idja9h80302NeB9w/y30P0u4JQhll9a7rbWTfVmq435J6gO8CfxYRbzSlV/ma9TFLWguMRsRwc7uUu0bifAiwDPhmRHwKeAuYC3NsjcT5QyTfnk8E+oAFki5tYt/yUvcxSzqLJEF8eab71tLuCWIvcFzF637qH1Y2sm8rNdRvSYeSJIf/FRF3N7lveWnkmFcCF0jaQzIEP1vS7c3tXi4a/X97b0RMjg6/Q5Iwyq6RY/4M8IuIGIuId4G7Sc7dl11dx5ze4nkLcGFEvDKTfafS7gniUWCJpBMlHQZcBNxbwL6tNOt+SxLJeelnI+Kvc+xjs836mCPi2ojoj4jF6X4/jIi58M2ykWP+v8CLkj6WFv0+8LN8utlUjfyb/BVwuqTu9P/z3yeZYyu7aY9Z0vEkCe9zEbFrJvtOq9Wz9AWsAlhDshrn58B/TsvWA+vT7WNIMu0bwGvp9pG19p0Lj9keM/B7JEPQp4An0seaVh9P3nGueI9VzJFVTI0eM/BJYCiN9T2kq2DK/mjwmL8CPAc8DfwDMK/Vx9OkY94CvFrxb3Zoqn1n8vClNszMLFO7n2IyM7NZcoIwM7NMThBmZpbJCcLMzDI5QZiZWSYnCLMmkbRH0sJG25iVhROEmZllcoIwmwVJ90gaTu8vMFhVtzi9z8Jt6TX6vyOpu6LJRkmPSfqppI+n+6xIr+X/ePr8McxazAnCbHYui4jlwADwp5I+XFX/MWBzRJxC8qve/1hR93JELAO+CUze1OY54MxILp53HfCXufberA5OEGaz86eSniS5/v5xwJKq+hcj4sF0+3aSy5hMmrwI4jCwON3+IPC/JT0N3AQszaPTZjPhBGE2Q5JWkVwd9IyIOBV4HKi+fWX1NWwqX7+TPk+QXHob4L8AP4qITwDnZ7yfWeGcIMxm7oPAqxHxdjqHcHpGm+Mr7vN8MfCTOt7zpXT7803ppVmDnCDMZm4HcIikp0i++T+c0eZZ4D+kbf4VyXzDVP4K+JqkB0nuJWzWcr6aq1mTpbdr3Z6eLjKbszyCMDOzTB5BmJlZJo8gzMwskxOEmZllcoIwM7NMThBmZpbJCcLMzDL9fwQQtg6RR0v3AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import itertools\n", "factorial = pd.DataFrame(\n", " itertools.product(np.linspace(*design.scope['alpha'].range, 4),\n", " np.linspace(*design.scope['beta'].range, 4)), \n", " columns=['alpha', 'beta']\n", ")\n", "lhs = design_experiments(scope, n_samples=16, random_seed=4)\n", "\n", "ax = factorial.plot.scatter('alpha', 'beta', color='r', s=40)\n", "lhs.plot.scatter('alpha', 'beta', marker='s', s=40, ax=ax);" ] }, { "cell_type": "markdown", "id": "fbcb27f2", "metadata": {}, "source": [ "If it turns out that the \"beta\" input is not particularly important for some model outputs,\n", "the information we are actually getting on those outputs will have been derived from inputs that\n", "look like this:" ] }, { "cell_type": "code", "execution_count": 11, "id": "8ed8a37b", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:50.015131Z", "start_time": "2021-06-28T16:15:49.942333Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAABjCAYAAAClgtpWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAIJElEQVR4nO3dXYxcdRnH8e9ja1tahaLFVGlxISElYkBqJaCGoHiBaOVWo0aixhAT8SXEl5hoiFQTY9Q7ElJjiPgSrYQIicQLuRGE0KV2LbYSkJcWaWhNKwbTheDjxTkb1+2yOzPnnPmf7X4/yWRnzsz/nP8zz+xvZ87snInMRJI0fq8qPQFJWq4MYEkqxACWpEIMYEkqxACWpEJWDnPjDRs25MTEREdTkaRT0+Tk5NHMPGvu8qECeGJigt27d7c3K0laBiLiqfmWuwtCkgoxgCWpEANYkgoxgCWpEANYkgoxgCWpEANYkgoxgCWpEANYkgoxgCWpEANYkgoxgCWpEANYkgoxgCWpkKEOR1nChd+8hxemXz5p+brVK3jkpquLjxt1O011ud2m615oPNDp/dXF/bKUetz2XNta32LrKXUfzzXuefQ+gOe7MxZaPu5xo26nqS6323Tdo4xv6/7q4n5ZSj1ue65trW+x9ZS6jwfdXlfzcBeEJBViAEtSIQawJBViAEtSIb0P4Jl3zgddPu5xo26nqS6323TdC43v+v7qYv1Lqcdtz7Wt9S22nlL38aDb62oekZkD33jbtm3ptyJL0nAiYjIzt81d3vtnwJJ0qjKAJakQA1iSCjGAJakQA1iSCjGAJakQA1iSCjGAJakQA1iSChlPAE9NwY4d1U/1gz3pJ/vSPx32pNuPIh8/Dhs3wvT0/5atXg2HD8P69YOvR+2xJ/1kX/qnxZ6U+Sjy3MlDdXnjxk43qwXYk36yL/0zhp50F8BTUydPfsb0tC+xSrAn/WRf+mdMPekugO+6q9n1ap896Sf70j9j6kl3Abx9e7Pr1T570k/2pX/G1JNu34Rbs2b+p/GrV8OJE4OvR+2xJ/1kX/qnxZ6UeRPu8OFqsrPNvIuoMuxJP9mX/hlDT7oN4PXrq78Ue/fCzTdXP0+c8N9qSrIn/WRf+mcMPfEriSSpY34lkST1jAEsSYUYwJJUiAEsSYUYwJJUiAEsSYUYwJJUiAEsSYUYwJJUiAEsSYUYwJJUiAEsSYUMdTCeiDgCPDXitjYAR0ccu1RZ8/Kw3GpebvVC85rfnJlnzV04VAA3ERG75zsa0KnMmpeH5VbzcqsXuqvZXRCSVIgBLEmFjDOAbx3jtvrCmpeH5VbzcqsXOqp5bPuAJUn/z10QklSIASxJhTQO4Ii4OiL+GhGPRcRX57n+goj4Y0RMR8SNw4ztq1FrjojNEXFvROyPiEci4vPjnfnomvS5vn5FROyJiLvHM+PmGj6210fErog4UPf78vHNfHQNa/5i/bjeFxE/j4g145v56Aao+aMRMVWf7o+Iiwcdu6jMHPkErAAeB84DVgF7gbfMuc0bgHcAO4Abhxnbx1PDmt8IbK3PvxZ49FSvedb1XwJ+Btxdup5x1AzcBny6Pr8KWF+6pi5rBs4GngBOqy//EriudE0t1fxO4Mz6/PuBBwcdu9ip6TPgS4HHMvNvmfki8Avg2tk3yMznMvMh4KVhx/bUyDVn5rOZ+XB9/l/AfqoHbt816TMRsQn4ALBzHJNtycg1R8TpwBXAj+rbvZiZx8cy62Ya9RlYCZwWESuBtcDfu55wCwap+f7MPFZffADYNOjYxTQN4LOBg7MuH2LwQGkytqRW5h0RE8AlwIPtTKtTTWv+IfBl4D8tzqlrTWo+DzgC/Lje7bIzIta1PcEOjFxzZj4DfA94GngW+Gdm/q71GbZv2Jo/Bfx2xLEnaRrAMc+yQf+vrcnYkhrPOyJeA/wa+EJmPt/KrLo1cs0R8UHgucycbHdKnWvS55XAVuCWzLwEeAFYCu9xNOnzmVTP/s4F3gSsi4iPtTi3rgxcc0S8hyqAvzLs2FfSNIAPAZtnXd7E4C87mowtqdG8I+LVVOH708y8o+W5daVJze8CPhQRT1K9RHtvRNze7vQ60fSxfSgzZ17d7KIK5L5rUvP7gCcy80hmvgTcQbXvtO8GqjkiLqLahXZtZv5jmLELaRrADwHnR8S5EbEK+DDwmzGMLWnkeUdEUO0X3J+Z3+9wjm0buebM/FpmbsrMiXrc7zNzKTwzalLzYeBgRGypF10F/KWbabaqye/k08BlEbG2fpxfRfUeR98tWnNEnEP1B+XjmfnoMGMX1cK7iNdQvZv/OPD1etn1wPX1+Y1UfymeB47X509/pbFL4TRqzcC7qV6iTAF/qk/XlK6n6z7PWseVLJH/gmhaM/A2YHfd6zup30Xv+6lhzTcBB4B9wE+A1aXraanmncCxWb+zuxcaO8zJjyJLUiF+Ek6SCjGAJakQA1iSCjGAJakQA1iSCjGAtWRExJMRsaHpbaS+MIAlqRADWL0UEXdGxGR9fNnPzLluoj7O7m31MVp3RcTaWTf5XEQ8HBF/jogL6jGX1sdy3VP/3IJUmAGsvvpkZr4d2AbcEBGvn3P9FuDWzLyI6lNZn5113dHM3ArcAswcNPwAcEVWB8f5BvDtTmcvDcAAVl/dEBF7qY6/uhk4f871BzPzvvr87VQf854xc5CjSWCiPn8G8KuI2Af8ALiwi0lLwzCA1TsRcSXV0bUuz8yLgT3A3K+3mfsZ+tmXp+ufL1MdGhLgW8C9mflWYPs865PGzgBWH50BHMvMf9f7cC+b5zbnzPqetY8Afxhgnc/U569rZZZSQwaw+ugeYGVETFE9c31gntvsBz5R3+Z1VPt7F/Jd4DsRcR/Vd3lJxXk0NC059dc53V3vTpCWLJ8BS1IhPgOWpEJ8BixJhRjAklSIASxJhRjAklSIASxJhfwXcGLqSNk/jvYAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "factorial['beta'] = 0\n", "lhs['beta'] = 1\n", "ax = factorial.plot.scatter('alpha', 'beta', color='r', s=40, figsize=(6,1))\n", "lhs.plot.scatter('alpha', 'beta', marker='s', s=40, ax=ax)\n", "ax.set_ylim(-1,2)\n", "ax.set_ylabel('')\n", "ax.set_yticks([]);" ] }, { "cell_type": "markdown", "id": "5fd330ea", "metadata": {}, "source": [ "Essentially only four unique experiments remain under the full factorial design,\n", "while the Latin hypercube design still offers useful variance over all 16 experiments." ] }, { "cell_type": "markdown", "id": "0462fea0", "metadata": {}, "source": [ "As for the sensitivity tests, the return type for the `design_experiments` function is `emat.ExperimentalDesign`.\n", "The result can be manipulated, saved, or reloaded as any other pandas DataFrame, which allows the the modeler to\n", "review an experimental design before actually running experiments." ] }, { "cell_type": "code", "execution_count": 12, "id": "b3da49ea", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:50.031415Z", "start_time": "2021-06-28T16:15:50.017468Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
alphaamortization_periodbetadebt_typeexpand_capacityinput_flowinterest_rateinterest_rate_lockunit_cost_expansionvalue_of_timeyield_curvefree_flow_timeinitial_capacity
00.179365241GO Bond80.1626431200.034792False140.9629530.0492280.00739760100
10.105645331GO Bond64.6715711260.030968False107.5793800.1375050.01387360100
20.141456401GO Bond87.8680011310.033288True104.8011100.0655760.01242060100
30.118782261Rev Bond16.3238861040.037495False134.2075180.0978380.00485360100
40.128577291GO Bond94.5250911460.038427True117.6932570.1077050.01638060100
50.158509421Paygo54.7470601150.034314True142.4514420.0279200.01042860100
60.172526361Rev Bond11.2151461390.037112True110.9624720.0633300.00174960100
70.114949321Paygo83.8039171480.036221False126.5331010.0832070.01519660100
80.198825471Paygo37.148734880.030445True115.3984830.092398-0.00187460100
90.185078191Rev Bond1.0861241080.025081True95.6384730.0448810.00009060100
100.151665501Rev Bond24.641162990.028835True136.5401900.1246680.00621760100
110.110274151Rev Bond71.659310920.032085False104.1412100.1695720.01921360100
120.136319181Rev Bond29.768710830.028615True98.6889800.1508770.00406460100
130.143869451GO Bond60.9551261330.026766False125.1267530.0799680.00884660100
140.187749371Paygo49.882562960.039660False129.6892190.1143330.01768760100
150.168406221Paygo43.5829871180.027771False121.3630340.0181840.00127760100
\n", "
" ], "text/plain": [ " alpha amortization_period beta debt_type expand_capacity \\\n", "0 0.179365 24 1 GO Bond 80.162643 \n", "1 0.105645 33 1 GO Bond 64.671571 \n", "2 0.141456 40 1 GO Bond 87.868001 \n", "3 0.118782 26 1 Rev Bond 16.323886 \n", "4 0.128577 29 1 GO Bond 94.525091 \n", "5 0.158509 42 1 Paygo 54.747060 \n", "6 0.172526 36 1 Rev Bond 11.215146 \n", "7 0.114949 32 1 Paygo 83.803917 \n", "8 0.198825 47 1 Paygo 37.148734 \n", "9 0.185078 19 1 Rev Bond 1.086124 \n", "10 0.151665 50 1 Rev Bond 24.641162 \n", "11 0.110274 15 1 Rev Bond 71.659310 \n", "12 0.136319 18 1 Rev Bond 29.768710 \n", "13 0.143869 45 1 GO Bond 60.955126 \n", "14 0.187749 37 1 Paygo 49.882562 \n", "15 0.168406 22 1 Paygo 43.582987 \n", "\n", " input_flow interest_rate interest_rate_lock unit_cost_expansion \\\n", "0 120 0.034792 False 140.962953 \n", "1 126 0.030968 False 107.579380 \n", "2 131 0.033288 True 104.801110 \n", "3 104 0.037495 False 134.207518 \n", "4 146 0.038427 True 117.693257 \n", "5 115 0.034314 True 142.451442 \n", "6 139 0.037112 True 110.962472 \n", "7 148 0.036221 False 126.533101 \n", "8 88 0.030445 True 115.398483 \n", "9 108 0.025081 True 95.638473 \n", "10 99 0.028835 True 136.540190 \n", "11 92 0.032085 False 104.141210 \n", "12 83 0.028615 True 98.688980 \n", "13 133 0.026766 False 125.126753 \n", "14 96 0.039660 False 129.689219 \n", "15 118 0.027771 False 121.363034 \n", "\n", " value_of_time yield_curve free_flow_time initial_capacity \n", "0 0.049228 0.007397 60 100 \n", "1 0.137505 0.013873 60 100 \n", "2 0.065576 0.012420 60 100 \n", "3 0.097838 0.004853 60 100 \n", "4 0.107705 0.016380 60 100 \n", "5 0.027920 0.010428 60 100 \n", "6 0.063330 0.001749 60 100 \n", "7 0.083207 0.015196 60 100 \n", "8 0.092398 -0.001874 60 100 \n", "9 0.044881 0.000090 60 100 \n", "10 0.124668 0.006217 60 100 \n", "11 0.169572 0.019213 60 100 \n", "12 0.150877 0.004064 60 100 \n", "13 0.079968 0.008846 60 100 \n", "14 0.114333 0.017687 60 100 \n", "15 0.018184 0.001277 60 100 " ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lhs" ] }, { "cell_type": "raw", "id": "fe9171b4", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. _methodology-monte-carlo-design:" ] }, { "cell_type": "markdown", "id": "36f13ebf", "metadata": {}, "source": [ "In addition to the default Latin Hypercube sampler, TMIP-EMAT also includes several other samplers,\n", "which can be invoked through the `sampler` argument of the `design_experiments` function:\n", "\n", "- `'ulhs'` is a \"uniform\" latin hypercube sampler, which ignores all defined distribution shapes and\n", " correlations from the scope, instead using independent uniform distributions for all inputs. This \n", " sampler is useful when designing experiments to support a meta-model for a risk analysis that \n", " includes well characterized non-uniform uncertainties and correlations.\n", "- `'mc'` is a simple Monte Carlo sampler, which does not make attempts to ensure good coverage of the \n", " sampling space. This sampler minimizes overhead and is useful when running a very large number of\n", " simulations\n", " \n" ] }, { "cell_type": "raw", "id": "74fdc483", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. _methodology-reference-design:" ] }, { "cell_type": "markdown", "id": "1761ff39", "metadata": { "ExecuteTime": { "end_time": "2021-05-26T16:30:57.203824Z", "start_time": "2021-05-26T16:30:57.201261Z" } }, "source": [ "## Reference Designs" ] }, { "cell_type": "markdown", "id": "26c69feb", "metadata": {}, "source": [ "An additional convenience tool is included in TMIP-EMAT to create a \"reference\" design, which contains only a single experiment that has every input set at the default value. Although this one experiment is also included in the univariate sensitivity test design, an analyst may not need or want to run that full design. Having the reference case available can be helpful to add context to some exploratory visualizations. " ] }, { "cell_type": "code", "execution_count": 13, "id": "f7078e78", "metadata": { "ExecuteTime": { "end_time": "2021-06-28T16:15:50.046268Z", "start_time": "2021-06-28T16:15:50.034100Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
free_flow_timeinitial_capacityalphabetainput_flowvalue_of_timeunit_cost_expansioninterest_rateyield_curveexpand_capacityamortization_perioddebt_typeinterest_rate_lock
0601000.154.01000.0751000.030.0110.030GO BondFalse
\n", "
" ], "text/plain": [ " free_flow_time initial_capacity alpha beta input_flow value_of_time \\\n", "0 60 100 0.15 4.0 100 0.075 \n", "\n", " unit_cost_expansion interest_rate yield_curve expand_capacity \\\n", "0 100 0.03 0.01 10.0 \n", "\n", " amortization_period debt_type interest_rate_lock \n", "0 30 GO Bond False " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from emat.experiment.experimental_design import design_refpoint_test\n", "design_refpoint_test(scope)" ] }, { "cell_type": "raw", "id": "c347a21d", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. include:: design-api.irst" ] } ], "metadata": { "celltoolbar": "Raw Cell Format", "jupytext": { "encoding": "# -*- coding: utf-8 -*-", "formats": "ipynb,py:percent" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.1" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": false, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 5 }