easyvvuq.db package

Submodules

easyvvuq.db.base module

Provides a base class for CampaignDBs

class easyvvuq.db.base.BaseCampaignDB(location=None, new_campaign=False, name=None, info=None)

Bases: object

Baseclass for all EasyVVUQ CampaignDBs

Skeleton for class that provides database access for the campaign.

Parameters:

• location (str or None) – Location to look for database.

• new_campaign (bool) – Does the database need to be initialised as a new campaign.

• name (str or None) – Name of the campaign.

• info (easyvvuq.data_structs.CampaignInfo) – Information defining the campaign.

add_app(app_info)

Add application to the ‘app’ table.

Parameters:

app_info (AppInfo) – Application definition.

add_run(run_info=None, prefix='Run_')

Add run to the runs table in the database.

Parameters:

• run_info (easyvvuq.data_structs.RunInfo) – Contains relevant run fields: params, status (where in the EasyVVUQ workflow is this RunTable), campaign (id number), sample, app

• prefix (str) – Prefix for run id

add_sampler(sampler)

Add new Sampler to the ‘sampler’ table.

Parameters:

sampler (BaseSamplingElement)

app(name)

Get app information. Specific applications selected by name, otherwise first entry in database ‘app’ selected.

Parameters:

name (str or None) – Name of selected app, if None given then first app will be selected.

Returns:

Application information.

Return type:

dict

append_collation_dataframe(df, app_id)

Append the data in dataframe ‘df’ to that already collated in the database

Parameters:

• df (pandas dataframe) – The dataframe whose contents need to be appended to the collation store

• app_id (int) – The id of this app in the sql database. Used to determine which collation table is appended to.

campaign_dir(campaign_name=None)

Get campaign directory for campaign_name.

Returns:

Path to campaign directory.

Return type:

str

campaigns()

Get list of campaigns for which information is stored in the database.

Returns:

Campaign names.

Return type:

list

get_campaign_id(name)

Return the (database) id corresponding to the campaign with name ‘name’.

Parameters:

name (str) – Name of the campaign.

Returns:

The id of the campaign with the specified name

Return type:

int

get_collation_dataframe(app_id)

Returns a dataframe containing the full collated results stored in this database i.e. the total of what was added with the append_collation_dataframe() method.

Parameters:

app_id (int) – The id of this app in the sql database. Used to determine which collation table is returned.

Returns:

df – The dataframe with all contents that were appended to the table corresponding to this app_id.

Return type:

pandas dataframe

get_num_runs(campaign=None, sampler=None, status=None, not_status=None)

Returns the number of runs matching the filtering criteria.

Parameters:

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sampler id to filter for.

• status (enum(Status) or None) – Status string to filter for.

• not_status (enum(Status) or None) – Exclude runs with this status string

Returns:

The number of runs in the database matching the filtering criteria

Return type:

int

get_run_status(run_name, campaign=None, sampler=None)

Return the status (enum) for the run with name ‘run_name’ (and, optionally, filtering for campaign and sampler by id)

Parameters:

• run_name (str) – Name of the run

• campaign (int) – ID of the desired Campaign

• sampler (int) – ID of the desired Sampler

Returns:

status – Status of the run.

Return type:

enum(Status)

resurrect_app(app_name)

Return the ‘live’ encoder and decoder objects corresponding to the app with name ‘app_name’ in the database. They are deserialized from the states previously stored in the database.

Parameters:

app_name (string) – Name of the app to resurrect

Returns:

The ‘live’ encoder and decoder objects associated with this app

Return type:

BaseEncoder, BaseDecoder, BaseCollationElement

resurrect_sampler(sampler_id)

Return the sampler object corresponding to id sampler_id in the database. It is deserialized from the state stored in the database.

Parameters:

sampler_id (int) – The id of the sampler to resurrect

Returns:

The ‘live’ sampler object, deserialized from the state in the db

Return type:

BaseSamplingElement

run(run_name, campaign=None, sampler=None)

Get the information for a specified run.

Parameters:

• run_name (str) – Name of run to filter for.

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sample id to filter for.

Returns:

Containing run information (run_name, params, status, sample, campaign, app)

Return type:

dict

runs(campaign=None, sampler=None, status=None, not_status=None)

A generator to return all run information for selected campaign and sampler.

Parameters:

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sampler id to filter for.

• status (enum(Status) or None) – Status string to filter for.

• not_status (enum(Status) or None) – Exclude runs with this status string

Returns:

Information on each selected run (key = run_name, value = dict of run information fields.), one at a time.

Return type:

dict

runs_dir(campaign_name=None)

Get the directory used to store run information for campaign_name.

Parameters:

campaign_name (str) – Name of the selected campaign.

Returns:

Path containing run outputs.

Return type:

str

set_dir_for_run(run_name, run_dir, campaign=None, sampler=None)

Set the ‘run_dir’ path for the specified run in the database.

Parameters:

• run_name (str) – Name of run to filter for.

• run_dir (str) – Directory path associated to set for this run.

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sample id to filter for.

set_run_statuses(run_name_list, status)

Set the specified ‘status’ (enum) for all runs in the list run_ID_list

Parameters:

• run_name_list (list of str) – A list of run names run names (format is usually: prefix + int)

• status (enum(Status)) – The new status all listed runs should now have

update_sampler(sampler_id, sampler_element)

Update the state of the Sampler with id ‘sampler_id’ to that in the passed ‘sampler_element’

Parameters:

• sampler_id (int) – The id of the sampler in the db to update

• sampler_element (BaseSamplingElement) – The sampler whose state should be used as the new state

easyvvuq.db.sql module

Provides class that allows access to an SQL Database that serves as the back-end to EasyVVUQ.

class easyvvuq.db.sql.AppTable(*args: Any, **kwargs: Any)

Bases: declarative_base

An SQLAlchemy schema for the app table.

class easyvvuq.db.sql.CampaignDB(location=None)

Bases: BaseCampaignDB

An interface between the campaign database and the campaign.

Parameters:

location (str) – database URI as needed by SQLAlchemy

add_app(app_info)

Add application to the ‘app’ table.

Parameters:

app_info (AppInfo) – Application definition.

add_runs(run_info_list=None, run_prefix='run_', iteration=0)

Add list of runs to the runs table in the database.

Parameters:

• run_info_list (List of RunInfo objects) – Each RunInfo object contains relevant run fields: params, status (where in the EasyVVUQ workflow is this RunTable), campaign (id number), sample, app

• run_prefix (str) – Prefix for run name

• iteration (int) – Iteration number used by iterative workflows. For example, MCMC. Can be left as default zero in other cases.

add_sampler(sampler_element)

Add new Sampler to the ‘sampler’ table.

Parameters:

sampler_element (Sampler) – An EasyVVUQ sampler.

Returns:

The sampler id in the database.

Return type:

int

app(name=None)

Get app information. Specific applications selected by name, otherwise first entry in database ‘app’ selected.

Parameters:

name (str or None) – Name of selected app, if None given then first app will be selected.

Returns:

Information about the application.

Return type:

dict

campaign_dir(campaign_name=None)

Get campaign directory for campaign_name.

Parameters:

campaign_name (str) – Name of campaign to select

Returns:

Path to campaign directory.

Return type:

str

campaign_exists(name)

Check if campaign specified by that name already exists.

Parameters:

name (str)

Returns:

True if such a campaign already exists, False otherwise

Return type:

bool

campaigns()

Get list of campaigns for which information is stored in the database.

Returns:

Campaign names.

Return type:

list

create_campaign(info)

Creates a new campaign in the database.

Parameters:

info (CampaignInfo) – This easyvvuq.data_structs.CampaignInfo will contain information needed to construct the Campaign table.

dump()

Dump the database as JSON for debugging purposes.

Returns:

A database dump in JSON format.

Return type:

dict

get_active_app()

Returns active app table.

Return type:

AppTable

get_campaign_id(name)

Return the (database) id corresponding to the campaign with name ‘name’.

Parameters:

name (str) – Name of the campaign.

Returns:

The id of the campaign with the specified name

Return type:

int

get_num_runs(campaign=None, sampler=None, status=None, not_status=None)

Returns the number of runs matching the filtering criteria.

Parameters:

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sampler id to filter for.

• status (enum(Status) or None) – Status string to filter for.

• not_status (enum(Status) or None) – Exclude runs with this status string

Returns:

The number of runs in the database matching the filtering criteria

Return type:

int

get_results(app_name, sampler_id, status=Status.COLLATED, iteration=-1)

Returns the results as a pandas DataFrame.

Parameters:

• app_name (str) – Name of the app to return data for.

• sampler_id (int) – ID of the sampler.

• status (STATUS) – Run status to filter for.

• iteration (int) – If a positive integer will return the results for a given iteration only.

Returns:

Will construct a DataFrame from the decoder output dictionaries.

Return type:

DataFrame

get_run_status(run_id, campaign=None, sampler=None)

Return the status (enum) for the run with name ‘run_name’ (and, optionally, filtering for campaign and sampler by id)

Parameters:

• run_id (int) – id of the run

• campaign (int) – ID of the desired Campaign

• sampler (int) – ID of the desired Sampler

Returns:

Status of the run.

Return type:

enum(Status)

get_sampler_id(campaign_id)

Return the (database) id corresponding to the sampler currently set for the campaign with id ‘campaign_id’

Parameters:

campaign_id (int) – ID of the campaign.

Returns:

The id of the sampler set for the specified campaign

Return type:

int

relocate(new_path, campaign_name)

Update all runs in the db with the new campaign path.

Parameters:

• new_path (str) – new runs directory

• campaign_name (str) – name of the campaign

replace_actions(app_name, actions)

Replace actions for an app with a given name.

Parameters:

• app_name (str) – Name of the app.

• actions (Actions) – Actions instance, will replace the current Actions of an app.

resume_campaign(name)

Resumes campaign.

Parameters:

name (str) – Name of the Campaign to resume. Must already exist in the database.

resurrect_app(app_name)

Return the ‘live’ encoder, decoder and collation objects corresponding to the app with name ‘app_name’ in the database. They are deserialized from the states previously stored in the database.

Parameters:

app_name (string) – Name of the app to resurrect

Returns:

The ‘live’ Actions object associated with this app. Used to execute the simulation associated with the app as well as do any pre- and post-processing.

Return type:

Actions

resurrect_sampler(sampler_id)

Return the sampler object corresponding to id sampler_id in the database. It is deserialized from the state stored in the database.

Parameters:

sampler_id (int) – The id of the sampler to resurrect

Returns:

The ‘live’ sampler object, deserialized from the state in the db

Return type:

Sampler

run(name, campaign=None, sampler=None, status=None, not_status=None, app_id=None)

Get the information for a specified run.

Parameters:

• name (str) – Name of run to filter for.

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sampler id to filter for.

• status (enum(Status) or None) – Status string to filter for.

• not_status (enum(Status) or None) – Exclude runs with this status string

• app_id (int or None) – App id to filter for.

Returns:

Containing run information (run_name, params, status, sample, campaign, app)

Return type:

dict

run_ids(campaign=None, sampler=None, status=None, not_status=None, app_id=None)

A generator to return all run IDs for selected campaign and sampler.

Parameters:

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sampler id to filter for.

• status (enum(Status) or None) – Status string to filter for.

• not_status (enum(Status) or None) – Exclude runs with this status string

• app_id (int or None) – App id to filter for.

Yields:

str – run ID for each selected run, one at a time.

runs(campaign=None, sampler=None, status=None, not_status=None, app_id=None)

A generator to return all run information for selected campaign and sampler.

Parameters:

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sampler id to filter for.

• status (enum(Status) or None) – Status string to filter for.

• not_status (enum(Status) or None) – Exclude runs with this status string

• app_id (int or None) – App id to filter for.

Yields:

dict – Information on each selected run (key = run_name, value = dict of run information fields.), one at a time.

runs_dir(campaign_name=None)

Get the directory used to store run information for campaign_name.

Parameters:

campaign_name (str) – Name of the selected campaign.

Returns:

Path containing run outputs.

Return type:

str

set_active_app(name)

Set an app specified by name as active.

Parameters:

name (str) – name of the app to set as active

set_dir_for_run(run_name, run_dir, campaign=None, sampler=None)

Set the ‘run_dir’ path for the specified run in the database.

Parameters:

• run_name (str) – Name of run to filter for.

• run_dir (str) – Directory path associated to set for this run.

• campaign (int or None) – Campaign id to filter for.

• sampler (int or None) – Sample id to filter for.

set_run_statuses(run_id_list, status)

Set the specified ‘status’ (enum) for all runs in the list run_id_list

Parameters:

• run_id_list (list of int) – a list of run ids

• status (enum(Status)) – The new status all listed runs should now have

set_sampler(campaign_id, sampler_id)

Set specified campaign to be using specified sampler

Parameters:

• campaign_id (int) – ID of the campaign.

• sampler_id (int) – ID of the sampler.

store_result(run_id, result, change_status=True)

Stores results of a simulation inside the RunTable given a run id.

Parameters:

• run_id (int) – The id of a run to store the results in. This will be the run with which these results are associated with. Namely the run that has the inputs used to generate these results.

• result (dict) – Results in dictionary form. This is the same format as used by the Decoder.

• change_status (bool) – If set to False will not update the runs’ status to COLLATED. This is sometimes useful in scenarios where you want several apps to work on the same runs.

store_results(app_name, results)

Stores the results from a given run in the database.

Parameters:

• run_name (str) – name of the run

• results (dict) – dictionary with the results (from the decoder)

update_sampler(sampler_id, sampler_element)

Update the state of the Sampler with id ‘sampler_id’ to that in the passed ‘sampler_element’

Parameters:

• sampler_id (int) – The id of the sampler in the db to update

• sampler_element (Sampler) – The sampler that should be used as the new state

class easyvvuq.db.sql.CampaignTable(*args: Any, **kwargs: Any)

Bases: declarative_base

An SQLAlchemy schema for the campaign information table.

class easyvvuq.db.sql.DBInfoTable(*args: Any, **kwargs: Any)

Bases: declarative_base

An SQLAlchemy schema for the database information table.

class easyvvuq.db.sql.RunTable(*args: Any, **kwargs: Any)

Bases: declarative_base

An SQLAlchemy schema for the run table.

class easyvvuq.db.sql.SamplerTable(*args: Any, **kwargs: Any)

Bases: declarative_base

An SQLAlchemy schema for the run table.

easyvvuq.db.sql.set_sqlite_pragma(dbapi_connection, connection_record)

Module contents