{ "cells": [ { "cell_type": "markdown", "id": "521c6d01", "metadata": {}, "source": [ "# EasyVVUQ - Jinja encoder tutorial\n", "\n", "\n", "This is a small modification of the basic tutorial on the simple beam model. In particular, will show how to use mathmatical expressions inside a Jinja template. For more information in encoding and decoding see the `tutorials/encoder_decoder_tutorial.ipynb` notebook." ] }, { "cell_type": "markdown", "id": "6428be0a", "metadata": {}, "source": [ "## Campaign" ] }, { "cell_type": "markdown", "id": "24a4af40", "metadata": {}, "source": [ "We need to import EasyVVUQ as well as ChaosPy (we use it's distributions) and matplotlib for plotting later on." ] }, { "cell_type": "code", "execution_count": 1, "id": "dad22041", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:11.855724Z", "start_time": "2021-06-09T07:14:08.704997Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:12.597493Z", "iopub.status.busy": "2025-07-20T07:23:12.597301Z", "iopub.status.idle": "2025-07-20T07:23:13.774595Z", "shell.execute_reply": "2025-07-20T07:23:13.774326Z", "shell.execute_reply.started": "2025-07-20T07:23:12.597477Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Volumes/UserData/dpc/GIT/EasyVVUQ/env_3.12/lib/python3.12/site-packages/chaospy/__init__.py:9: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.\n", " import pkg_resources\n" ] } ], "source": [ "import os\n", "import easyvvuq as uq\n", "import chaospy as cp\n", "import matplotlib.pyplot as plt\n", "from easyvvuq.actions import CreateRunDirectory, Encode, Decode, CleanUp, ExecuteLocal, Actions" ] }, { "cell_type": "markdown", "id": "ee53f14d", "metadata": {}, "source": [ "We will describe the parameters. This is done for validation purposes (so that input parameters outside valid ranges given an error. Also this is where you can specify default values for input parameters that you don't want to vary in the analysis. Only the type and the default value fields are mandatory." ] }, { "cell_type": "code", "execution_count": 2, "id": "5bb692f0", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:11.858747Z", "start_time": "2021-06-09T07:14:11.856736Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:13.775062Z", "iopub.status.busy": "2025-07-20T07:23:13.774889Z", "iopub.status.idle": "2025-07-20T07:23:13.776820Z", "shell.execute_reply": "2025-07-20T07:23:13.776603Z", "shell.execute_reply.started": "2025-07-20T07:23:13.775054Z" } }, "outputs": [], "source": [ "params = {\n", " \"F\": {\"type\": \"float\", \"default\": 1.0}, \n", " \"L\": {\"type\": \"float\", \"default\": 1.5}, \n", " \"a\": {\"type\": \"float\", \"min\": 0.7, \"max\": 1.2, \"default\": 1.0}, \n", " \"D\": {\"type\": \"float\", \"min\": 0.75, \"max\": 0.85, \"default\": 0.8},\n", " \"d\": {\"type\": \"float\", \"default\": 0.1},\n", " \"E\": {\"type\": \"float\", \"default\": 200000},\n", " \"power\" : {\"type\": \"float\", \"default\": 0.5},\n", " \"outfile\": {\"type\": \"string\", \"default\": \"output.json\"}\n", "}" ] }, { "cell_type": "markdown", "id": "e15d9c0f", "metadata": {}, "source": [ "## Jinja encoder\n", "\n", "Below we find the only deviation from the basic tutorial, namely the use of the Jinja decoder with a mathmatical expression for the `a` variable. The standard Jinja template would look like:\n", "\n", "```\n", "{\"outfile\": \"{{outfile}}\", \n", "\"F\": {{F}},\n", "\"L\": {{L}}, \n", "\"a\": {{a}}, \n", "\"D\": {{D}}, \n", "\"d\": {{d}}, \n", "\"E\": {{E}}\n", "}\n", "```\n", "\n", "This is replaces every `{{variable}}` with a numeric value. If that the variable appears in the `vary`, the value will be drawn from the specified probability distribution. If it does not appear in `vary`, the default value as specified in the `params` dict will be used. The result will be a JSON file that is read by the beam model. It does not have to be a JSON file, this same principle will hold for any type of template file.\n", "\n", "The Jinja encoder is flexible in the sense that **mathematical expressions** can also be used. As an example, consider the followin template:\n", "\n", "```\n", "{\"outfile\": \"{{outfile}}\", \n", "\"F\": {{F}},\n", "\"L\": {{L}}, \n", "\"a\": {{a ** power}}, \n", "\"D\": {{D}}, \n", "\"d\": {{d}}, \n", "\"E\": {{E}}\n", "}\n", "```\n", "\n", "This is the same as before, except now the square root of `a` is taken. Here, `power` is also defined in the `params` dict. Since `power` is not in vary, the value of `0.5` is always used." ] }, { "cell_type": "code", "execution_count": 3, "id": "d51ada1b", "metadata": { "execution": { "iopub.execute_input": "2025-07-20T07:23:13.777088Z", "iopub.status.busy": "2025-07-20T07:23:13.777028Z", "iopub.status.idle": "2025-07-20T07:23:13.778493Z", "shell.execute_reply": "2025-07-20T07:23:13.778311Z", "shell.execute_reply.started": "2025-07-20T07:23:13.777082Z" } }, "outputs": [], "source": [ "encoder = uq.encoders.JinjaEncoder(template_fname='beam.jinja', target_filename='input.json')" ] }, { "cell_type": "markdown", "id": "2afdb7b8", "metadata": {}, "source": [ "The rest of this turorial proceeds unmodified from the basic tutorial." ] }, { "cell_type": "code", "execution_count": 4, "id": "77f1eff4", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:19.459501Z", "start_time": "2021-06-09T07:14:19.455107Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:13.778757Z", "iopub.status.busy": "2025-07-20T07:23:13.778699Z", "iopub.status.idle": "2025-07-20T07:23:13.780194Z", "shell.execute_reply": "2025-07-20T07:23:13.780016Z", "shell.execute_reply.started": "2025-07-20T07:23:13.778751Z" } }, "outputs": [], "source": [ "decoder = uq.decoders.JSONDecoder(target_filename='output.json', output_columns=['g1'])\n", "execute = ExecuteLocal('{}/beam input.json'.format(os.getcwd()))\n", "\n", "actions = Actions(CreateRunDirectory('/tmp', flatten=True), \n", " Encode(encoder), execute, Decode(decoder))" ] }, { "cell_type": "markdown", "id": "d5dc3398", "metadata": {}, "source": [ "Campaign is the central hub in which operations take place. It is responsible for running your simulations, gathering the results, storing them in the Database, retrieving them for analysis, etc. The Campaign in EasyVVUQ is very powerful and supports multiple applications, sampling, analysis and execution methods. It also lets you save progress and retrieve results later for analysis. Here we only look at a simple case." ] }, { "cell_type": "code", "execution_count": 5, "id": "916c375d", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:22.214216Z", "start_time": "2021-06-09T07:14:22.176515Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:13.780470Z", "iopub.status.busy": "2025-07-20T07:23:13.780420Z", "iopub.status.idle": "2025-07-20T07:23:13.796307Z", "shell.execute_reply": "2025-07-20T07:23:13.796021Z", "shell.execute_reply.started": "2025-07-20T07:23:13.780464Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/Volumes/UserData/dpc/GIT/EasyVVUQ/env_3.12/lib/python3.12/site-packages/cerberus/validator.py:618: UserWarning: These types are defined both with a method and in the'types_mapping' property of this validator: {'integer'}\n", " warn(\n", "/Volumes/UserData/dpc/GIT/EasyVVUQ/env_3.12/lib/python3.12/site-packages/cerberus/validator.py:618: UserWarning: These types are defined both with a method and in the'types_mapping' property of this validator: {'integer'}\n", " warn(\n" ] } ], "source": [ "campaign = uq.Campaign(name='beam', params=params, actions=actions, work_dir='/tmp')" ] }, { "cell_type": "markdown", "id": "531e7aec", "metadata": {}, "source": [ "First we need to define the input parameter distributions. We have chosen 4 of the 6 available inputs. This is partly because this means that we won't have to sample at too many points and partly because I've found that these influence the output variable the most." ] }, { "cell_type": "code", "execution_count": 6, "id": "e3bc3491", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:24.062778Z", "start_time": "2021-06-09T07:14:24.056744Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:13.798220Z", "iopub.status.busy": "2025-07-20T07:23:13.798076Z", "iopub.status.idle": "2025-07-20T07:23:13.800479Z", "shell.execute_reply": "2025-07-20T07:23:13.800158Z", "shell.execute_reply.started": "2025-07-20T07:23:13.798208Z" } }, "outputs": [], "source": [ "vary = {\n", " \"F\": cp.Normal(1, 0.1),\n", " \"L\": cp.Normal(1.5, 0.01),\n", " \"a\": cp.Uniform(0.7, 1.2),\n", " \"D\": cp.Triangle(0.75, 0.8, 0.85)\n", "}" ] }, { "cell_type": "markdown", "id": "ac53db70", "metadata": {}, "source": [ "We have to choose the sampler next. For this task we can use either [Stochastic Collocation](https://easyvvuq.readthedocs.io/en/dev/easyvvuq.sampling.html#module-easyvvuq.sampling.stochastic_collocation), [Polynomial Chaos Expansion](https://easyvvuq.readthedocs.io/en/dev/easyvvuq.sampling.html#module-easyvvuq.sampling.pce) or [QMC](https://easyvvuq.readthedocs.io/en/dev/easyvvuq.sampling.html#module-easyvvuq.sampling.qmc) samplers. Stochastic Collocation is fast for this problem size so that is what we chose." ] }, { "cell_type": "code", "execution_count": 7, "id": "927c07af", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:25.478795Z", "start_time": "2021-06-09T07:14:25.224527Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:13.800937Z", "iopub.status.busy": "2025-07-20T07:23:13.800846Z", "iopub.status.idle": "2025-07-20T07:23:13.937195Z", "shell.execute_reply": "2025-07-20T07:23:13.936916Z", "shell.execute_reply.started": "2025-07-20T07:23:13.800927Z" } }, "outputs": [], "source": [ "campaign.set_sampler(uq.sampling.PCESampler(vary=vary, polynomial_order=1))" ] }, { "cell_type": "markdown", "id": "d12571d5", "metadata": {}, "source": [ "For this tutorial we have chosen to run the simulation on the local machine. This will done in parallel with up to 8 tasks running concurrently. Alternatives are execution in the Cloud (via the [ExecuteKubernetes](https://easyvvuq.readthedocs.io/en/dev/easyvvuq.actions.html#module-easyvvuq.actions.execute_kubernetes) action) or on HPC machines." ] }, { "cell_type": "code", "execution_count": 8, "id": "0dbb0dab", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:28.266228Z", "start_time": "2021-06-09T07:14:26.904138Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:13.937522Z", "iopub.status.busy": "2025-07-20T07:23:13.937457Z", "iopub.status.idle": "2025-07-20T07:23:14.136057Z", "shell.execute_reply": "2025-07-20T07:23:14.135757Z", "shell.execute_reply.started": "2025-07-20T07:23:13.937515Z" } }, "outputs": [], "source": [ "campaign.execute().collate()" ] }, { "cell_type": "markdown", "id": "3d955b56", "metadata": {}, "source": [ "The execution can take a bit since we need to generate several hundred samples. We asked it to evaluate 8 samples in parallel. You can track progress by using the ```progress``` method. You can also check progress automatically and resume execution after it is done if you want to run this inside a script rather than interactively." ] }, { "cell_type": "code", "execution_count": 9, "id": "e197810e", "metadata": { "ExecuteTime": { "end_time": "2021-06-09T07:14:30.293648Z", "start_time": "2021-06-09T07:14:30.260808Z" }, "execution": { "iopub.execute_input": "2025-07-20T07:23:14.136573Z", "iopub.status.busy": "2025-07-20T07:23:14.136495Z", "iopub.status.idle": "2025-07-20T07:23:14.148094Z", "shell.execute_reply": "2025-07-20T07:23:14.147858Z", "shell.execute_reply.started": "2025-07-20T07:23:14.136564Z" } }, "outputs": [ { "data": { "text/html": [ "
| \n", " | run_id | \n", "iteration | \n", "F | \n", "L | \n", "a | \n", "D | \n", "d | \n", "E | \n", "power | \n", "outfile | \n", "g1 | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|
| \n", " | 0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "
| 0 | \n", "1 | \n", "0 | \n", "0.9 | \n", "1.49 | \n", "0.805662 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000008 | \n", "
| 1 | \n", "2 | \n", "0 | \n", "0.9 | \n", "1.49 | \n", "0.805662 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000006 | \n", "
| 2 | \n", "3 | \n", "0 | \n", "0.9 | \n", "1.49 | \n", "1.094338 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000006 | \n", "
| 3 | \n", "4 | \n", "0 | \n", "0.9 | \n", "1.49 | \n", "1.094338 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000005 | \n", "
| 4 | \n", "5 | \n", "0 | \n", "0.9 | \n", "1.51 | \n", "0.805662 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000008 | \n", "
| 5 | \n", "6 | \n", "0 | \n", "0.9 | \n", "1.51 | \n", "0.805662 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000007 | \n", "
| 6 | \n", "7 | \n", "0 | \n", "0.9 | \n", "1.51 | \n", "1.094338 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000006 | \n", "
| 7 | \n", "8 | \n", "0 | \n", "0.9 | \n", "1.51 | \n", "1.094338 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000005 | \n", "
| 8 | \n", "9 | \n", "0 | \n", "1.1 | \n", "1.49 | \n", "0.805662 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000010 | \n", "
| 9 | \n", "10 | \n", "0 | \n", "1.1 | \n", "1.49 | \n", "0.805662 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000008 | \n", "
| 10 | \n", "11 | \n", "0 | \n", "1.1 | \n", "1.49 | \n", "1.094338 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000007 | \n", "
| 11 | \n", "12 | \n", "0 | \n", "1.1 | \n", "1.49 | \n", "1.094338 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000006 | \n", "
| 12 | \n", "13 | \n", "0 | \n", "1.1 | \n", "1.51 | \n", "0.805662 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000010 | \n", "
| 13 | \n", "14 | \n", "0 | \n", "1.1 | \n", "1.51 | \n", "0.805662 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000008 | \n", "
| 14 | \n", "15 | \n", "0 | \n", "1.1 | \n", "1.51 | \n", "1.094338 | \n", "0.779588 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000008 | \n", "
| 15 | \n", "16 | \n", "0 | \n", "1.1 | \n", "1.51 | \n", "1.094338 | \n", "0.820412 | \n", "0.1 | \n", "200000 | \n", "0.5 | \n", "output.json | \n", "-0.000006 | \n", "