{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "ExecuteTime": { "end_time": "2021-05-25T17:22:15.108524Z", "start_time": "2021-05-25T17:22:13.618693Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "emat 0.5.2, plotly 4.14.3\n" ] } ], "source": [ "import emat\n", "emat.versions()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Identifying Model Problems" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Once a series of experiments has been conducted for a core model, it\n", "is suggested that an analyst review the results to confirm that the \n", "model is behaving as expected. TMIP-EMAT provides some visualization\n", "tools to review results in a graphical manner, which is a generally \n", "convenient method to review this experimental data.\n", "\n", "To demonstrate these tools, we will use the Road Test example model included\n", "in TMIP-EMAT. We can quickly construct and run a design of experiments\n", "to exercise this model and populate some results to visualize, and in \n", "this example we will use an intentionally broken model input, to demonstrate\n", "one way that bad results might manifest." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "ExecuteTime": { "end_time": "2021-05-25T17:22:15.929633Z", "start_time": "2021-05-25T17:22:15.111197Z" } }, "outputs": [], "source": [ "import emat.examples\n", "scope, db, model = emat.examples.road_test(yamlfile='road_test_corrupt.yaml')\n", "design = model.design_experiments()\n", "results = model.run_experiments(design)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The \"corrupt\" scope for the Road Test model includes a policy lever to set\n", "the lane width for the link. You might expect that increasing the lane \n", "width might increase the effective capacity on the link, but as coded any\n", "deviation from exactly 10.0 feet will result in substantial extra delay in\n", "the build travel time, regardless of any other factors. Running the experiments\n", "with this broken input will invalidate the entire set of results, but here\n", "we'll assume that we don't know *a priori* that the lane width parameter is \n", "broken.\n", "\n", "Given this set of experimental results, we can display a scatter plot matrix\n", "to see the results. This is a collection of two-dimensional plots, each\n", "showing a contrast between two factors, typically an input parameter (i.e. \n", "an uncertainty or a policy lever) and an output performance measure, although\n", "it is also possible to plot inputs against inputs or outputs against outputs.\n", "\n", "The `display_experiments` function in the `emat.analysis` sub-package can \n", "automatically create a scatter plot matrix that crosses every parameter with\n", "every measure, simply by providing the scope and the results. By default, \n", "plots that display levers are shown in blue, plots that show uncertainties\n", "are in red. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "ExecuteTime": { "end_time": "2021-05-25T17:22:20.636355Z", "start_time": "2021-05-25T17:22:15.931793Z" } }, "outputs": [ { "data": { "text/html": [ "