EasyVVUQ: Uncertainty intervals for everyone!¶
EasyVVUQ is a Python library designed to facilitate verification, validation and uncertainty quantification (VVUQ) for a wide variety of simulations. It was conceived and developed within the EU funded VECMA (Verified Exascale Computing for Multiscale Applications) project.
Goals¶
The purpose of EasyVVUQ is to make it as easy as possible to implement advanced techniques for uncertainty quantification for existing application codes (or workflows). We do not intend to re-invent the wheel, and plan on always building upon existing libraries such as Chaospy which focus on providing statistical functionality. Our aim is to expose these features in as accessible a way for users of scientific codes, in particular simulation software targeting HPC machines.
For technical details please see EasyVVUQ API Reference.
We also provide a range of interactive tutorials within the repository. This collection changes over time, but can be found here: https://github.com/UCL-CCS/EasyVVUQ/tree/dev/tutorials .
Another point of reference point are the non-regression tests which can be found at https://github.com/UCL-CCS/EasyVVUQ/tree/dev/tests . These are often useful in showing how pieces of the software work.
Table of contents¶
EasyVVUQ installation¶
Installation¶
Note
To use the library you will need Python 3.7+.
Installation should be straight forward:
pip install easyvvuq
To update an existing installation:
pip install easyvvuq --upgrade
Alternatively, to get the most current version, the code can be installed from Github as follows:
git clone https://github.com/UCL-CCS/EasyVVUQ.git
cd EasyVVUQ
pip install -r requirements.txt
python setup.py install
Note
The above assumes that your default python is Python 3. If that is not the case replace python with python3 and pip with pip3.
Depending on your setup you may not have permission to install packages. In that case, we recommend creating a virtual environment using conda or pipenv.
Questions & Troubleshooting¶
For any problems and questions you might have related to EasyVVUQ, please
feel free to file an https://github.com/UCL-CCS/EasyVVUQ/issues.
Conceptual basis¶
EasyVVUQ was created as part of the VECMA project, and is currently maintained and enhanced as part of the SEAVEA project. The aim of this project was to make state of the art VVUQ algorithms available for use in HPC applications (and specifically multiscale models). The basis of making generic tools within VECMA is the idea of Patterns, which are:
“abstractions that describe, in a non-application and non-domain specific manner, a workflow or algorithm for conducting validation, verification, uncertainty quantification or sensitivity analysis”.
Making use of Patterns in practice requires that they are decomposed into components which can be flexibly combined to implement a range of algorithms.
Figure 1: Decomposition of generalized VVUQ workflow into different functions. These are implemented as ‘Elements’ in EasyVVUQ. Rounded boxes are specified by users to tailor general workflows to their particular use case
EasyVVUQ is designed around a breakdown of such workflows into four distinct
stages (see Figure 1); Sampling, Model Evaluation, result Aggregation,
and Analysis.
In an HPC context the model evaluation step is generally equivalent to the
execution of a (computationally expensive) simulation.
The actual simulation execution is beyond the remit of the package but
EasyVVUQ is designed to wrap around simulation execution, providing functions
to generate input (an Encoder) and to transform simulation output into common
formats for analysis (a Decoder).
Below we describe the components of EasyVVUQ designed to perform each step in
more detail.
Parameter description¶
The first step in our generalised workflow is a description of the model parameters and how they might vary in the sampling phase of the VVUQ pattern. Typically the user will specifying all numerical parameters, the distribution from which they should be drawn and physically acceptable limits on their value.
Campaign¶
EasyVVUQ workflows are coordinated by an object called a Campaign.
This contains a common database, the CampaignDB, which contains
information on the application(s) being analysed alongside the runs
mandated by the sampling algorithm(s) employed. It also stores the
decoded results of the simulations themseleves. The Database is the
central location where all information about your campaign is
kept. The Campaign handles all validation and is transfers
information between each stage of the workflow.
The Basic Tutorial gives a good hands-on introduction to defining parameters and creating a campaign.
Elements¶
Within VECMA software components that can be reused in a wide range of
application scenarios are known as Elements.
Within EasyVVUQ we provide five classes of Elements (Samplers,
Decoders, Encoders, and those providing collation,
for the aggregation step, and Analysis functionality) which we
describe below.
Samplers¶
A Sampler populates the CampaignDB with a set of run specifications based on
the parameter description provided by the user.
Each Sampler is designed to employs one of a range of algorithms, such as
the Monte Carlo or Quasi Monte Carlo approaches (Sobol, 1998).
They deal with generic information in the sense that all parameters use the
nomenclature and units provided by the user rather than anything specific to
any application or workflow.
Detailed information on the Sampler modules is available here.
Encoders¶
The role of an Encoder is to convert generic parameter descriptions into
inputs (for example configuration files) which can be used in a specific
application.
Included in the base application is a simple templating system in which
values are substituted into a text input file.
For many applications it is envisioned that specific encoders will be
needed and the framework of EasyVVUQ means that any class derived from a
generic Encoder base class is picked up and may be used.
This enables EasyVVUQ to be easily extended for new applications by
experienced users.
Detailed information on the Encoder modules is available here.
Decoders¶
The role of a Decoder is twofold, to record simulation completion in the
CampaignDB and to extract the output information from the simulation runs.
Similarly to an Encoder, a Decoder is designed to be user extendable to
facilitate analysis of a wide range of applications.
The Encoder-Decoder tutorial provides a good introduction to using Encoders and Decoders within EasyVVUQ. Detailed information on the Decoder modules themselves is available here.
Analysis¶
The final goal of any VVUQ workflow is an analysis which provided information on the simulation output across a range of runs. Different types of analysis (for example bootstrapping of multiple runs from varied initial conditions) are, or will be, provided by EasyVVUQ.
Detailed information on the Analysis modules is available here
