easyvvuq.db package¶
Submodules¶
easyvvuq.db.base module¶
Provides a base class for CampaignDBs
- class easyvvuq.db.base.BaseCampaignDB(location=None, new_campaign=False, name=None, info=None)[source]¶
Bases:
object
Baseclass for all EasyVVUQ CampaignDBs
Skeleton for class that provides database access for the campaign.
- Parameters:
location (str or None) – Location to look for database.
new_campaign (bool) – Does the database need to be initialised as a new campaign.
name (str or None) – Name of the campaign.
info (easyvvuq.data_structs.CampaignInfo) – Information defining the campaign.
- add_app(app_info)[source]¶
Add application to the ‘app’ table.
- Parameters:
app_info (AppInfo) – Application definition.
- add_run(run_info=None, prefix='Run_')[source]¶
Add run to the runs table in the database.
- Parameters:
run_info (easyvvuq.data_structs.RunInfo) – Contains relevant run fields: params, status (where in the EasyVVUQ workflow is this RunTable), campaign (id number), sample, app
prefix (str) – Prefix for run id
- add_sampler(sampler)[source]¶
Add new Sampler to the ‘sampler’ table.
- Parameters:
sampler (BaseSamplingElement)
- app(name)[source]¶
Get app information. Specific applications selected by name, otherwise first entry in database ‘app’ selected.
- Parameters:
name (str or None) – Name of selected app, if None given then first app will be selected.
- Returns:
Application information.
- Return type:
dict
- append_collation_dataframe(df, app_id)[source]¶
Append the data in dataframe ‘df’ to that already collated in the database
- Parameters:
df (pandas dataframe) – The dataframe whose contents need to be appended to the collation store
app_id (int) – The id of this app in the sql database. Used to determine which collation table is appended to.
- campaign_dir(campaign_name=None)[source]¶
Get campaign directory for campaign_name.
- Returns:
Path to campaign directory.
- Return type:
str
- campaigns()[source]¶
Get list of campaigns for which information is stored in the database.
- Returns:
Campaign names.
- Return type:
list
- get_campaign_id(name)[source]¶
Return the (database) id corresponding to the campaign with name ‘name’.
- Parameters:
name (str) – Name of the campaign.
- Returns:
The id of the campaign with the specified name
- Return type:
int
- get_collation_dataframe(app_id)[source]¶
Returns a dataframe containing the full collated results stored in this database i.e. the total of what was added with the append_collation_dataframe() method.
- Parameters:
app_id (int) – The id of this app in the sql database. Used to determine which collation table is returned.
- Returns:
df – The dataframe with all contents that were appended to the table corresponding to this app_id.
- Return type:
pandas dataframe
- get_num_runs(campaign=None, sampler=None, status=None, not_status=None)[source]¶
Returns the number of runs matching the filtering criteria.
- Parameters:
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sampler id to filter for.
status (enum(Status) or None) – Status string to filter for.
not_status (enum(Status) or None) – Exclude runs with this status string
- Returns:
The number of runs in the database matching the filtering criteria
- Return type:
int
- get_run_status(run_name, campaign=None, sampler=None)[source]¶
Return the status (enum) for the run with name ‘run_name’ (and, optionally, filtering for campaign and sampler by id)
- Parameters:
run_name (str) – Name of the run
campaign (int) – ID of the desired Campaign
sampler (int) – ID of the desired Sampler
- Returns:
status – Status of the run.
- Return type:
enum(Status)
- resurrect_app(app_name)[source]¶
Return the ‘live’ encoder and decoder objects corresponding to the app with name ‘app_name’ in the database. They are deserialized from the states previously stored in the database.
- Parameters:
app_name (string) – Name of the app to resurrect
- Returns:
The ‘live’ encoder and decoder objects associated with this app
- Return type:
BaseEncoder, BaseDecoder, BaseCollationElement
- resurrect_sampler(sampler_id)[source]¶
Return the sampler object corresponding to id sampler_id in the database. It is deserialized from the state stored in the database.
- Parameters:
sampler_id (int) – The id of the sampler to resurrect
- Returns:
The ‘live’ sampler object, deserialized from the state in the db
- Return type:
- run(run_name, campaign=None, sampler=None)[source]¶
Get the information for a specified run.
- Parameters:
run_name (str) – Name of run to filter for.
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sample id to filter for.
- Returns:
Containing run information (run_name, params, status, sample, campaign, app)
- Return type:
dict
- runs(campaign=None, sampler=None, status=None, not_status=None)[source]¶
A generator to return all run information for selected campaign and sampler.
- Parameters:
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sampler id to filter for.
status (enum(Status) or None) – Status string to filter for.
not_status (enum(Status) or None) – Exclude runs with this status string
- Returns:
Information on each selected run (key = run_name, value = dict of run information fields.), one at a time.
- Return type:
dict
- runs_dir(campaign_name=None)[source]¶
Get the directory used to store run information for campaign_name.
- Parameters:
campaign_name (str) – Name of the selected campaign.
- Returns:
Path containing run outputs.
- Return type:
str
- set_dir_for_run(run_name, run_dir, campaign=None, sampler=None)[source]¶
Set the ‘run_dir’ path for the specified run in the database.
- Parameters:
run_name (str) – Name of run to filter for.
run_dir (str) – Directory path associated to set for this run.
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sample id to filter for.
- set_run_statuses(run_name_list, status)[source]¶
Set the specified ‘status’ (enum) for all runs in the list run_ID_list
- Parameters:
run_name_list (list of str) – A list of run names run names (format is usually: prefix + int)
status (enum(Status)) – The new status all listed runs should now have
- update_sampler(sampler_id, sampler_element)[source]¶
Update the state of the Sampler with id ‘sampler_id’ to that in the passed ‘sampler_element’
- Parameters:
sampler_id (int) – The id of the sampler in the db to update
sampler_element (BaseSamplingElement) – The sampler whose state should be used as the new state
easyvvuq.db.sql module¶
Provides class that allows access to an SQL Database that serves as the back-end to EasyVVUQ.
- class easyvvuq.db.sql.AppTable(*args: Any, **kwargs: Any)[source]¶
Bases:
declarative_base
An SQLAlchemy schema for the app table.
- class easyvvuq.db.sql.CampaignDB(location=None)[source]¶
Bases:
BaseCampaignDB
An interface between the campaign database and the campaign.
- Parameters:
location (str) – database URI as needed by SQLAlchemy
- add_app(app_info)[source]¶
Add application to the ‘app’ table.
- Parameters:
app_info (AppInfo) – Application definition.
- add_runs(run_info_list=None, run_prefix='run_', iteration=0)[source]¶
Add list of runs to the runs table in the database.
- Parameters:
run_info_list (List of RunInfo objects) – Each RunInfo object contains relevant run fields: params, status (where in the EasyVVUQ workflow is this RunTable), campaign (id number), sample, app
run_prefix (str) – Prefix for run name
iteration (int) – Iteration number used by iterative workflows. For example, MCMC. Can be left as default zero in other cases.
- add_sampler(sampler_element)[source]¶
Add new Sampler to the ‘sampler’ table.
- Parameters:
sampler_element (Sampler) – An EasyVVUQ sampler.
- Returns:
The sampler id in the database.
- Return type:
int
- app(name=None)[source]¶
Get app information. Specific applications selected by name, otherwise first entry in database ‘app’ selected.
- Parameters:
name (str or None) – Name of selected app, if None given then first app will be selected.
- Returns:
Information about the application.
- Return type:
dict
- campaign_dir(campaign_name=None)[source]¶
Get campaign directory for campaign_name.
- Parameters:
campaign_name (str) – Name of campaign to select
- Returns:
Path to campaign directory.
- Return type:
str
- campaign_exists(name)[source]¶
Check if campaign specified by that name already exists.
- Parameters:
name (str)
- Returns:
True if such a campaign already exists, False otherwise
- Return type:
bool
- campaigns()[source]¶
Get list of campaigns for which information is stored in the database.
- Returns:
Campaign names.
- Return type:
list
- create_campaign(info)[source]¶
Creates a new campaign in the database.
- Parameters:
info (CampaignInfo) – This easyvvuq.data_structs.CampaignInfo will contain information needed to construct the Campaign table.
- dump()[source]¶
Dump the database as JSON for debugging purposes.
- Returns:
A database dump in JSON format.
- Return type:
dict
- get_campaign_id(name)[source]¶
Return the (database) id corresponding to the campaign with name ‘name’.
- Parameters:
name (str) – Name of the campaign.
- Returns:
The id of the campaign with the specified name
- Return type:
int
- get_num_runs(campaign=None, sampler=None, status=None, not_status=None)[source]¶
Returns the number of runs matching the filtering criteria.
- Parameters:
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sampler id to filter for.
status (enum(Status) or None) – Status string to filter for.
not_status (enum(Status) or None) – Exclude runs with this status string
- Returns:
The number of runs in the database matching the filtering criteria
- Return type:
int
- get_results(app_name, sampler_id, status=Status.COLLATED, iteration=-1)[source]¶
Returns the results as a pandas DataFrame.
- Parameters:
app_name (str) – Name of the app to return data for.
sampler_id (int) – ID of the sampler.
status (STATUS) – Run status to filter for.
iteration (int) – If a positive integer will return the results for a given iteration only.
- Returns:
Will construct a DataFrame from the decoder output dictionaries.
- Return type:
DataFrame
- get_run_status(run_id, campaign=None, sampler=None)[source]¶
Return the status (enum) for the run with name ‘run_name’ (and, optionally, filtering for campaign and sampler by id)
- Parameters:
run_id (int) – id of the run
campaign (int) – ID of the desired Campaign
sampler (int) – ID of the desired Sampler
- Returns:
Status of the run.
- Return type:
enum(Status)
- get_sampler_id(campaign_id)[source]¶
Return the (database) id corresponding to the sampler currently set for the campaign with id ‘campaign_id’
- Parameters:
campaign_id (int) – ID of the campaign.
- Returns:
The id of the sampler set for the specified campaign
- Return type:
int
- relocate(new_path, campaign_name)[source]¶
Update all runs in the db with the new campaign path.
- Parameters:
new_path (str) – new runs directory
campaign_name (str) – name of the campaign
- replace_actions(app_name, actions)[source]¶
Replace actions for an app with a given name.
- Parameters:
app_name (str) – Name of the app.
actions (Actions) – Actions instance, will replace the current Actions of an app.
- resume_campaign(name)[source]¶
Resumes campaign.
- Parameters:
name (str) – Name of the Campaign to resume. Must already exist in the database.
- resurrect_app(app_name)[source]¶
Return the ‘live’ encoder, decoder and collation objects corresponding to the app with name ‘app_name’ in the database. They are deserialized from the states previously stored in the database.
- Parameters:
app_name (string) – Name of the app to resurrect
- Returns:
The ‘live’ Actions object associated with this app. Used to execute the simulation associated with the app as well as do any pre- and post-processing.
- Return type:
- resurrect_sampler(sampler_id)[source]¶
Return the sampler object corresponding to id sampler_id in the database. It is deserialized from the state stored in the database.
- Parameters:
sampler_id (int) – The id of the sampler to resurrect
- Returns:
The ‘live’ sampler object, deserialized from the state in the db
- Return type:
Sampler
- run(name, campaign=None, sampler=None, status=None, not_status=None, app_id=None)[source]¶
Get the information for a specified run.
- Parameters:
name (str) – Name of run to filter for.
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sampler id to filter for.
status (enum(Status) or None) – Status string to filter for.
not_status (enum(Status) or None) – Exclude runs with this status string
app_id (int or None) – App id to filter for.
- Returns:
Containing run information (run_name, params, status, sample, campaign, app)
- Return type:
dict
- run_ids(campaign=None, sampler=None, status=None, not_status=None, app_id=None)[source]¶
A generator to return all run IDs for selected campaign and sampler.
- Parameters:
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sampler id to filter for.
status (enum(Status) or None) – Status string to filter for.
not_status (enum(Status) or None) – Exclude runs with this status string
app_id (int or None) – App id to filter for.
- Yields:
str – run ID for each selected run, one at a time.
- runs(campaign=None, sampler=None, status=None, not_status=None, app_id=None)[source]¶
A generator to return all run information for selected campaign and sampler.
- Parameters:
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sampler id to filter for.
status (enum(Status) or None) – Status string to filter for.
not_status (enum(Status) or None) – Exclude runs with this status string
app_id (int or None) – App id to filter for.
- Yields:
dict – Information on each selected run (key = run_name, value = dict of run information fields.), one at a time.
- runs_dir(campaign_name=None)[source]¶
Get the directory used to store run information for campaign_name.
- Parameters:
campaign_name (str) – Name of the selected campaign.
- Returns:
Path containing run outputs.
- Return type:
str
- set_active_app(name)[source]¶
Set an app specified by name as active.
- Parameters:
name (str) – name of the app to set as active
- set_dir_for_run(run_name, run_dir, campaign=None, sampler=None)[source]¶
Set the ‘run_dir’ path for the specified run in the database.
- Parameters:
run_name (str) – Name of run to filter for.
run_dir (str) – Directory path associated to set for this run.
campaign (int or None) – Campaign id to filter for.
sampler (int or None) – Sample id to filter for.
- set_run_statuses(run_id_list, status)[source]¶
Set the specified ‘status’ (enum) for all runs in the list run_id_list
- Parameters:
run_id_list (list of int) – a list of run ids
status (enum(Status)) – The new status all listed runs should now have
- set_sampler(campaign_id, sampler_id)[source]¶
Set specified campaign to be using specified sampler
- Parameters:
campaign_id (int) – ID of the campaign.
sampler_id (int) – ID of the sampler.
- store_result(run_id, result, change_status=True)[source]¶
Stores results of a simulation inside the RunTable given a run id.
- Parameters:
run_id (int) – The id of a run to store the results in. This will be the run with which these results are associated with. Namely the run that has the inputs used to generate these results.
result (dict) – Results in dictionary form. This is the same format as used by the Decoder.
change_status (bool) – If set to False will not update the runs’ status to COLLATED. This is sometimes useful in scenarios where you want several apps to work on the same runs.
- class easyvvuq.db.sql.CampaignTable(*args: Any, **kwargs: Any)[source]¶
Bases:
declarative_base
An SQLAlchemy schema for the campaign information table.
- class easyvvuq.db.sql.DBInfoTable(*args: Any, **kwargs: Any)[source]¶
Bases:
declarative_base
An SQLAlchemy schema for the database information table.
- class easyvvuq.db.sql.RunTable(*args: Any, **kwargs: Any)[source]¶
Bases:
declarative_base
An SQLAlchemy schema for the run table.
- class easyvvuq.db.sql.SamplerTable(*args: Any, **kwargs: Any)[source]¶
Bases:
declarative_base
An SQLAlchemy schema for the run table.
- easyvvuq.db.sql.set_sqlite_pragma(dbapi_connection, connection_record)¶