easyvvuq.db package

Submodules

easyvvuq.db.base module

Provides a base class for CampaignDBs

class easyvvuq.db.base.BaseCampaignDB(location=None, new_campaign=False, name=None, info=None)

Bases: object

Baseclass for all EasyVVUQ CampaignDBs

Skeleton for class that provides database access for the campaign.

Parameters:

• location (str or None) – Location to look for database.
• new_campaign (bool) – Does the database need to be initialised as a new campaign.
• name (str or None) – Name of the campaign.
• info (easyvvuq.data_structs.CampaignInfo) – Information defining the campaign.

add_app(app_info)

Add application to the ‘app’ table.

Parameters:app_info (AppInfo) – Application definition.

add_run(run_info=None, prefix='Run_')

Add run to the runs table in the database.

Parameters:

• run_info (easyvvuq.data_structs.RunInfo) – Contains relevant run fields: params, status (where in the EasyVVUQ workflow is this RunTable), campaign (id number), sample, app
• prefix (str) – Prefix for run id

add_sampler(sampler)

Add new Sampler to the ‘sampler’ table.

Parameters:sampler (BaseSamplingElement)

app(name)

Get app information. Specific applications selected by name, otherwise first entry in database ‘app’ selected.

Parameters:name (str or None) – Name of selected app, if None given then first app will be selected. Returns:Application information. Return type:dict

append_collation_dataframe(df, app_id)

Append the data in dataframe ‘df’ to that already collated in the database

Parameters:

• df (pandas dataframe) – The dataframe whose contents need to be appended to the collation store
• app_id (int) – The id of this app in the sql database. Used to determine which collation table is appended to.

campaign_dir(campaign_name=None)

Get campaign directory for campaign_name.

Returns:Path to campaign directory. Return type:str

campaigns()

Get list of campaigns for which information is stored in the database.

Returns:Campaign names. Return type:list

get_campaign_id(name)

Return the (database) id corresponding to the campaign with name ‘name’.

Parameters:name (str) – Name of the campaign. Returns:The id of the campaign with the specified name Return type:int

get_collation_dataframe(app_id)

Returns a dataframe containing the full collated results stored in this database i.e. the total of what was added with the append_collation_dataframe() method.

Parameters:app_id (int) – The id of this app in the sql database. Used to determine which collation table is returned. Returns:df – The dataframe with all contents that were appended to the table corresponding to this app_id. Return type:pandas dataframe

get_num_runs(campaign=None, sampler=None, status=None, not_status=None)

Returns the number of runs matching the filtering criteria.

Parameters:

• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sampler id to filter for.
• status (enum(Status) or None) – Status string to filter for.
• not_status (enum(Status) or None) – Exclude runs with this status string

Returns:

The number of runs in the database matching the filtering criteria

Return type:

int

get_run_status(run_name, campaign=None, sampler=None)

Return the status (enum) for the run with name ‘run_name’ (and, optionally, filtering for campaign and sampler by id)

Parameters:

• run_name (str) – Name of the run
• campaign (int) – ID of the desired Campaign
• sampler (int) – ID of the desired Sampler

Returns:

status – Status of the run.

Return type:

enum(Status)

resurrect_app(app_name)

Return the ‘live’ encoder and decoder objects corresponding to the app with name ‘app_name’ in the database. They are deserialized from the states previously stored in the database.

Parameters:app_name (string) – Name of the app to resurrect Returns:The ‘live’ encoder and decoder objects associated with this app Return type:BaseEncoder, BaseDecoder, BaseCollationElement

resurrect_sampler(sampler_id)

Return the sampler object corresponding to id sampler_id in the database. It is deserialized from the state stored in the database.

Parameters:sampler_id (int) – The id of the sampler to resurrect Returns:The ‘live’ sampler object, deserialized from the state in the db Return type:BaseSamplingElement

run(run_name, campaign=None, sampler=None)

Get the information for a specified run.

Parameters:

• run_name (str) – Name of run to filter for.
• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sample id to filter for.

Returns:

Containing run information (run_name, params, status, sample, campaign, app)

Return type:

dict

runs(campaign=None, sampler=None, status=None, not_status=None)

A generator to return all run information for selected campaign and sampler.

Parameters:

• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sampler id to filter for.
• status (enum(Status) or None) – Status string to filter for.
• not_status (enum(Status) or None) – Exclude runs with this status string

Returns:

Information on each selected run (key = run_name, value = dict of run information fields.), one at a time.

Return type:

dict

runs_dir(campaign_name=None)

Get the directory used to store run information for campaign_name.

Parameters:campaign_name (str) – Name of the selected campaign. Returns:Path containing run outputs. Return type:str

set_dir_for_run(run_name, run_dir, campaign=None, sampler=None)

Set the ‘run_dir’ path for the specified run in the database.

Parameters:

• run_name (str) – Name of run to filter for.
• run_dir (str) – Directory path associated to set for this run.
• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sample id to filter for.

set_run_statuses(run_name_list, status)

Set the specified ‘status’ (enum) for all runs in the list run_ID_list

Parameters:

• run_name_list (list of str) – A list of run names run names (format is usually: prefix + int)
• status (enum(Status)) – The new status all listed runs should now have

update_sampler(sampler_id, sampler_element)

Update the state of the Sampler with id ‘sampler_id’ to that in the passed ‘sampler_element’

Parameters:

• sampler_id (int) – The id of the sampler in the db to update
• sampler_element (BaseSamplingElement) – The sampler whose state should be used as the new state

easyvvuq.db.sql module

Provides class that allows access to an SQL Database that serves as the back-end to EasyVVUQ.

class easyvvuq.db.sql.AppTable(**kwargs)

Bases: sqlalchemy.orm.decl_api.Base

An SQLAlchemy schema for the app table.

actions

id

name

params

class easyvvuq.db.sql.CampaignDB(location=None)

Bases: easyvvuq.db.base.BaseCampaignDB

An interface between the campaign database and the campaign.

Parameters:location (str) – database URI as needed by SQLAlchemy

add_app(app_info)

Add application to the ‘app’ table.

Parameters:app_info (AppInfo) – Application definition.

add_runs(run_info_list=None, run_prefix='run_', iteration=0)

Add list of runs to the runs table in the database.

Parameters:

• run_info_list (List of RunInfo objects) – Each RunInfo object contains relevant run fields: params, status (where in the EasyVVUQ workflow is this RunTable), campaign (id number), sample, app
• run_prefix (str) – Prefix for run name
• iteration (int) – Iteration number used by iterative workflows. For example, MCMC. Can be left as default zero in other cases.

add_sampler(sampler_element)

Add new Sampler to the ‘sampler’ table.

Parameters:sampler_element (Sampler) – An EasyVVUQ sampler. Returns:The sampler id in the database. Return type:int

app(name=None)

Get app information. Specific applications selected by name, otherwise first entry in database ‘app’ selected.

Parameters:name (str or None) – Name of selected app, if None given then first app will be selected. Returns:Information about the application. Return type:dict

campaign_dir(campaign_name=None)

Get campaign directory for campaign_name.

Parameters:campaign_name (str) – Name of campaign to select Returns:Path to campaign directory. Return type:str

campaign_exists(name)

Check if campaign specified by that name already exists.

Parameters:name (str) Returns:True if such a campaign already exists, False otherwise Return type:bool

campaigns()

Get list of campaigns for which information is stored in the database.

Returns:Campaign names. Return type:list

create_campaign(info)

Creates a new campaign in the database.

Parameters:info (CampaignInfo) – This easyvvuq.data_structs.CampaignInfo will contain information needed to construct the Campaign table.

dump()

Dump the database as JSON for debugging purposes.

Returns:A database dump in JSON format. Return type:dict

get_active_app()

Returns active app table.

Returns: Return type:AppTable

get_campaign_id(name)

Return the (database) id corresponding to the campaign with name ‘name’.

Parameters:name (str) – Name of the campaign. Returns:The id of the campaign with the specified name Return type:int

get_num_runs(campaign=None, sampler=None, status=None, not_status=None)

Returns the number of runs matching the filtering criteria.

Parameters:

• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sampler id to filter for.
• status (enum(Status) or None) – Status string to filter for.
• not_status (enum(Status) or None) – Exclude runs with this status string

Returns:

The number of runs in the database matching the filtering criteria

Return type:

int

get_results(app_name, sampler_id, status=<Status.COLLATED: 3>, iteration=-1)

Returns the results as a pandas DataFrame.

Parameters:

• app_name (str) – Name of the app to return data for.
• sampler_id (int) – ID of the sampler.
• status (STATUS) – Run status to filter for.
• iteration (int) – If a positive integer will return the results for a given iteration only.

Returns:

Will construct a DataFrame from the decoder output dictionaries.

Return type:

DataFrame

get_run_status(run_id, campaign=None, sampler=None)

Return the status (enum) for the run with name ‘run_name’ (and, optionally, filtering for campaign and sampler by id)

Parameters:

• run_id (int) – id of the run
• campaign (int) – ID of the desired Campaign
• sampler (int) – ID of the desired Sampler

Returns:

Status of the run.

Return type:

enum(Status)

get_sampler_id(campaign_id)

Return the (database) id corresponding to the sampler currently set for the campaign with id ‘campaign_id’

Parameters:campaign_id (int) – ID of the campaign. Returns:The id of the sampler set for the specified campaign Return type:int

relocate(new_path, campaign_name)

Update all runs in the db with the new campaign path.

Parameters:

• new_path (str) – new runs directory
• campaign_name (str) – name of the campaign

replace_actions(app_name, actions)

Replace actions for an app with a given name.

Parameters:

• app_name (str) – Name of the app.
• actions (Actions) – Actions instance, will replace the current Actions of an app.

resume_campaign(name)

Resumes campaign.

Parameters:name (str) – Name of the Campaign to resume. Must already exist in the database.

resurrect_app(app_name)

Return the ‘live’ encoder, decoder and collation objects corresponding to the app with name ‘app_name’ in the database. They are deserialized from the states previously stored in the database.

Parameters:app_name (string) – Name of the app to resurrect Returns:The ‘live’ Actions object associated with this app. Used to execute the simulation associated with the app as well as do any pre- and post-processing. Return type:Actions

resurrect_sampler(sampler_id)

Return the sampler object corresponding to id sampler_id in the database. It is deserialized from the state stored in the database.

Parameters:sampler_id (int) – The id of the sampler to resurrect Returns:The ‘live’ sampler object, deserialized from the state in the db Return type:Sampler

run(name, campaign=None, sampler=None, status=None, not_status=None, app_id=None)

Get the information for a specified run.

Parameters:

• name (str) – Name of run to filter for.
• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sampler id to filter for.
• status (enum(Status) or None) – Status string to filter for.
• not_status (enum(Status) or None) – Exclude runs with this status string
• app_id (int or None) – App id to filter for.

Returns:

Containing run information (run_name, params, status, sample, campaign, app)

Return type:

dict

run_ids(campaign=None, sampler=None, status=None, not_status=None, app_id=None)

A generator to return all run IDs for selected campaign and sampler.

Parameters:

• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sampler id to filter for.
• status (enum(Status) or None) – Status string to filter for.
• not_status (enum(Status) or None) – Exclude runs with this status string
• app_id (int or None) – App id to filter for.

Yields:

str – run ID for each selected run, one at a time.

runs(campaign=None, sampler=None, status=None, not_status=None, app_id=None)

A generator to return all run information for selected campaign and sampler.

Parameters:

• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sampler id to filter for.
• status (enum(Status) or None) – Status string to filter for.
• not_status (enum(Status) or None) – Exclude runs with this status string
• app_id (int or None) – App id to filter for.

Yields:

dict – Information on each selected run (key = run_name, value = dict of run information fields.), one at a time.

runs_dir(campaign_name=None)

Get the directory used to store run information for campaign_name.

Parameters:campaign_name (str) – Name of the selected campaign. Returns:Path containing run outputs. Return type:str

set_active_app(name)

Set an app specified by name as active.

Parameters:name (str) – name of the app to set as active

set_dir_for_run(run_name, run_dir, campaign=None, sampler=None)

Set the ‘run_dir’ path for the specified run in the database.

Parameters:

• run_name (str) – Name of run to filter for.
• run_dir (str) – Directory path associated to set for this run.
• campaign (int or None) – Campaign id to filter for.
• sampler (int or None) – Sample id to filter for.

set_run_statuses(run_id_list, status)

Set the specified ‘status’ (enum) for all runs in the list run_id_list

Parameters:

• run_id_list (list of int) – a list of run ids
• status (enum(Status)) – The new status all listed runs should now have

set_sampler(campaign_id, sampler_id)

Set specified campaign to be using specified sampler

Parameters:

• campaign_id (int) – ID of the campaign.
• sampler_id (int) – ID of the sampler.

store_result(run_id, result, change_status=True)

Stores results of a simulation inside the RunTable given a run id.

Parameters:

• run_id (int) – The id of a run to store the results in. This will be the run with which these results are associated with. Namely the run that has the inputs used to generate these results.
• result (dict) – Results in dictionary form. This is the same format as used by the Decoder.
• change_status (bool) – If set to False will not update the runs’ status to COLLATED. This is sometimes useful in scenarios where you want several apps to work on the same runs.

store_results(app_name, results)

Stores the results from a given run in the database.

Parameters:

• run_name (str) – name of the run
• results (dict) – dictionary with the results (from the decoder)

update_sampler(sampler_id, sampler_element)

Update the state of the Sampler with id ‘sampler_id’ to that in the passed ‘sampler_element’

Parameters:

• sampler_id (int) – The id of the sampler in the db to update
• sampler_element (Sampler) – The sampler that should be used as the new state

class easyvvuq.db.sql.CampaignTable(**kwargs)

Bases: sqlalchemy.orm.decl_api.Base

An SQLAlchemy schema for the campaign information table.

active_app

campaign_dir

campaign_dir_prefix

easyvvuq_version

id

name

runs_dir

sampler

class easyvvuq.db.sql.DBInfoTable(**kwargs)

Bases: sqlalchemy.orm.decl_api.Base

An SQLAlchemy schema for the database information table.

id

next_run

class easyvvuq.db.sql.RunTable(**kwargs)

Bases: sqlalchemy.orm.decl_api.Base

An SQLAlchemy schema for the run table.

app

campaign

execution_info

id

iteration

params

result

run_dir

run_name

sampler

status

class easyvvuq.db.sql.SamplerTable(**kwargs)

Bases: sqlalchemy.orm.decl_api.Base

An SQLAlchemy schema for the run table.

id

sampler

easyvvuq.db.sql.set_sqlite_pragma(dbapi_connection, connection_record)

Module contents