easyvvuq.actions package

Submodules

easyvvuq.actions.action_statuses module

class easyvvuq.actions.action_statuses.ActionStatuses(statuses, batch_size=8, poll_sleep_time=1)

Bases: object

A class that tracks statuses of a list of actions.

Parameters:

• statuses (list of ActionStatus) – a list of action statuses to track
• poll_sleep_time (int) – a time to sleep for after iterating over all active statuses before starting again

job_handler(status)

Will handle the execution of this action status.

Parameters:status (ActionStatus) – ActionStatus of an action to be executed.

progress()

Some basic stats about the action statuses status.

Returns: Return type:A dictionary with four keys - ‘ready’, ‘active’ and ‘finished’, ‘failed’.

start()

Start the actions.

Returns: Return type:A list of Python futures represending action execution.

wait(poll_interval=1)

A command that will automatically poll job statuses. For use in scripts.

Parameters:poll_interval (int) – Polling interval in seconds.

easyvvuq.actions.base module

Templates for elements that perform actions on a specified directory.

class easyvvuq.actions.base.BaseAction

Bases: object

Baseclass for all EasyVVUQ Actions.

act_on_dir(target_dir)

Function that will perform some action on the specified directory. Must be implemented by subclass.

Parameters:target_dir (str) – Directory upon which the action will be performed

campaign

easyvvuq.actions.execute_kubernetes module

Provides an action element to execute a simulation on a Kubernetes cluster and retrieve the output. The successful use of this actions requires that the Kubernetes cluster is properly set-up on the users system. Namely the ~/.kube/config file should contain valid information. Exact details will depend on the cloud service provider. Otherwise this action works similarly to how ExecuteLocal works. The difference is that the simulations are executed on a Kubernetes cluster. The input files are passed to the Pods via the ConfigMap mechanism. This probably limits the size of the configuration files but this can be alleviated with some kind of a pre-processing script on the Pod side. Likewise, output from the simulation is retrieved using the Kubernetes log mechanism. Therefore the simulation output needs to be printed to stdout on the Pod side. Again, if the simulation produces complicated or large output you should extract the quantitities of interest on the Pod using some kind of script and print them to stdout.

Examples

class easyvvuq.actions.execute_kubernetes.ActionStatusKubernetes(api, body, config_names, namespace, outfile)

Bases: object

Provides a way to track the status of an on-going Kubernetes action.

Parameters:

• api (CoreV1Api) – will be used to communicate with the cluster
• pod_name (str) – pod identifier
• config_names (list of str) – list of ConfigMap identifiers
• namespace (str) – Kubernetes namespace
• outfile (str) – a filename to write the output of the simulation

create_config_maps(file_names)

Create Kubernetes ConfigMaps for the input files to the simulation.

create_volumes(file_names, dep)

Create descriptions of Volumes that will hold the input files.

finalise()

Will read the logs from the Kubernetes pod, output them to a file and delete the Kubernetes resources we have allocated.

finished()

Will return True if the pod has finished, otherwise will return False.

start()

Will create the Kubernetes pod and hence start the action.

started()

Will return true if start() was called.

succeeded()

Will return True if the pod has finished successfully, otherwise will return False. If the job hasn’t finished yet will return False.

class easyvvuq.actions.execute_kubernetes.ExecuteKubernetes(image, command, input_file_names=None, output_file_name=None)

Bases: easyvvuq.actions.base.BaseAction

Provides an action element to run a shell command in a specified directory.

Parameters:

• pod_config (str) – Filename of the YAML file with the Kubernetes Pod configuration.
• input_file_names (list of str) – A list of input file names for your simulation.
• output_file_name (str) – An output file name for the output of the simulation.

act_on_dir(target_dir)

Executes a dockerized simulation on input files found in target_dir.

target_dir : str

Directory in which to execute simulation.

easyvvuq.actions.execute_local module

Provides element to execute a shell command in a given directory.

class easyvvuq.actions.execute_local.ActionStatusLocal

Bases: object

finalise()

finished()

start()

succeeded()

class easyvvuq.actions.execute_local.ActionStatusLocalV2(full_cmd, target_dir)

Bases: object

finalise()

Performs clean-up if necessary. In this case it isn’t. I think.

finished()

Returns true if action is finished. In this case if calling poll on the popen object returns a non-None value.

start()

started()

succeeded()

Will return True if the process finished successfully. It judges based on the return code and will return False if that code is not zero.

class easyvvuq.actions.execute_local.ExecuteLocal(run_cmd, interpret=None)

Bases: easyvvuq.actions.base.BaseAction

act_on_dir(target_dir)

Executes self.run_cmd in the shell in target_dir.

target_dir : str

Directory in which to execute command.

class easyvvuq.actions.execute_local.ExecuteLocalV2(run_cmd, interpret=None)

Bases: easyvvuq.actions.execute_local.ExecuteLocal

An improvement over ExecuteLocal that uses Popen and provides the non-blocking execution that allows you to track progress. In line with other Action classes in EasyVVUQ.

act_on_dir(target_dir)

Executes self.run_cmd in the shell in target_dir.

target_dir : str

Directory in which to execute command.

class easyvvuq.actions.execute_local.ExecutePython(function)

Bases: easyvvuq.actions.base.BaseAction

act_on_dir(target_dir)

Function that will perform some action on the specified directory. Must be implemented by subclass.

Parameters:target_dir (str) – Directory upon which the action will be performed

easyvvuq.actions.execute_slurm module

Provides a simple action element for interacting with a SLURM job. This lets you execute your simulation on a SLURM cluster.

class easyvvuq.actions.execute_slurm.ActionStatusSLURM(script, script_name, target_dir)

Bases: object

An ActionStatus to track the execution of a SLURM job.

Parameters:

• script (str) – The body of the script. Will be written to a file relative to target_dir named script_name.
• script_name (str) – Name of the script file. This will be written to. So beware of overwriting issues.
• target_dir (str) – Name of the execution directory for this job.

finalise()

Performs clean-up if necessary. In this case it isn’t. I think.

finished()

Returns true if action is finished. In this case if calling poll on the popen object returns a non-None value.

start()

Start the SLURM job.

started()

Returns True if the job has started.

succeeded()

Will return True if the process finished successfully. It judges based on the return code and will return False if that code is not zero.

class easyvvuq.actions.execute_slurm.ExecuteSLURM(template_script, variable)

Bases: easyvvuq.actions.base.BaseAction

An improvement over ExecuteLocal that uses Popen and provides the non-blocking execution that allows you to track progress. In line with other Action classes in EasyVVUQ.

Parameters:

• template_script (str) – filename for the template slurm script
• variable (str a string in the template script that will be) – replaced with the target_dir

act_on_dir(target_dir)

Executes self.run_cmd in the shell in target_dir.

target_dir : str

Directory in which to execute command.

Module contents