Vytautas Jancauskas 2020
EasyVVUQ Mega Tutorial¶
Introduction¶
In this tutorial we will show you how you can use EasyVVUQ to investigate the properties of a simple epidemiological model. The model is very simplistic and is not intended to realisticly portray a real epidemic such as COVID-19. However it is also fun enough to experiment with and we can use it as an example to show how you can use EasyVVUQ to answer questions about your scientific models. EasyVVUQ is successfully used by researchers in various different fields, such as plasma physics, weather and air pollution modelling and materials science.
Installing EasyVVUQ¶
Before we do anything else we need to install EasyVVUQ for use in this notebook. Please skip this if you already have EasyVVUQ installed in your system. This is meant for situations where this notebook is hosted externally to your local computer. We also want to clone the git repository because it contains some files used in this tutorial. Please note that you might need to restart the runtime after the installation is complete. If the following code examples don’t work please try that first. I have added a command that kills the runtime (which causes it to be restarted). But I’m not sure if it will always work.
[ ]:
!pip install git+https://github.com/UCL-CCS/EasyVVUQ
[2]:
!git clone https://github.com/UCL-CCS/EasyVVUQ
Cloning into 'EasyVVUQ'...
remote: Enumerating objects: 125, done.
remote: Counting objects: 100% (125/125), done.
remote: Compressing objects: 100% (107/107), done.
remote: Total 14554 (delta 75), reused 40 (delta 18), pack-reused 14429
Receiving objects: 100% (14554/14554), 4.26 MiB | 4.24 MiB/s, done.
Resolving deltas: 100% (10606/10606), done.
[ ]:
import os
os.kill(os.getpid(), 9)
Epidemiological Model¶
In our model individuals are placed on a two dimensional square grid. During each turn, the individual moves at random one square in one of eight possible directions (N, NE, E, SE, S, SW, W or NW). If the individual encounters another individual and that person is ill, the individual will also become sick for a specified number of turns (that number is a parameter to the model). Once ill, the individual cannot become sick anymore and after getting over the disease they will have immunity. Immunity means that this person cannot get infected anymore. The simulation continues until no one is sick. At that point the disease counts as erradicated.
Let us load the model and see how it operates. We create a population on a 10x10 grid containing 20 individuals. Once infected the individual can transmit the disease for 28 turns. After that they have a 20% chance of dying.
[4]:
import matplotlib.pyplot as plt
import numpy as np
import EasyVVUQ.tutorials.epidemic.epidemic as epidemic
population = epidemic.Population(grid_size=10, n=20, duration=28, mortality=0.2)
The code cell below is supposed to be run multiple times. Each time you run it the image below will update to show the state of the population. The black squares are empty, the white squares represent individuals who are not immune and are not sick. The red squares represent ill individuals and the intensity of the red shows how many turns they will stay ill for. Green squares represent individuals who are immune. Once the disease is eradicated, the graphic will change to a plot showing the evolution of the disease over time.
[5]:
population.move()
if np.count_nonzero(population.ill):
plt.imshow(population.get_im())
plt.show()
else:
plt.plot(population.ill_history, label="Sick Individuals")
plt.plot(population.immune_history, label="Immune Individuals")
plt.plot(population.n_history, label="Population")
plt.xlabel("Time")
plt.ylabel("Count")
plt.legend()
plt.show()
Since EasyVVUQ is meant to be a general framework (non-Python specific) we don’t call Python functions directly to get results of the simulation. After all, many simulations are still written in Fortran and operate by taking an input file and producing a file with outputs of the simulation. To do statistical analysis we need to be able to provide an appropriate input file to the simulation and be able to parse the outputs of the simulation. You can run our simulation as in the following example.
[8]:
!cd EasyVVUQ/tutorials/epidemic/; time python3 epidemic.py example.json output.csv
real 0m1.769s
user 0m1.308s
sys 0m0.332s
This will have produced a file called output.csv that consists of four columns labeled “iteration”, “ill”, “immune” and “population”. These should be fairly self explanatory.
[10]:
!cat EasyVVUQ/tutorials/epidemic/output.csv
iteration,ill,immune,population
0,1,0,20
1,1,0,20
2,2,1,20
3,3,2,20
4,3,2,20
5,3,2,20
6,3,2,20
7,3,2,20
8,3,2,20
9,3,2,20
10,5,4,20
11,5,4,20
12,5,4,20
13,5,4,20
14,5,4,20
15,7,6,20
16,9,8,20
17,9,8,20
18,10,9,20
19,11,10,20
20,12,11,20
21,12,11,20
22,13,12,20
23,14,13,20
24,14,13,20
25,14,13,20
26,14,13,20
27,16,16,20
28,16,16,20
29,15,16,20
30,14,15,19
31,14,15,19
32,15,16,19
33,15,16,19
34,15,16,19
35,16,17,19
36,16,17,19
37,15,18,19
38,16,19,19
39,16,19,19
40,16,19,19
41,16,19,19
42,14,19,19
43,12,19,19
44,12,19,19
45,11,19,19
46,10,19,19
47,9,19,19
48,9,19,19
49,8,19,19
50,7,19,19
51,7,19,19
52,7,19,19
53,7,19,19
54,4,19,19
55,4,19,19
56,4,19,19
57,4,19,19
58,4,19,19
59,3,19,19
60,3,19,19
61,3,19,19
62,2,19,19
63,2,19,19
64,1,19,19
65,0,19,19
Let us plot this data and see what the evolution of the disease looks like in our population.
[12]:
import matplotlib.pyplot as plt
import pandas as pd
df = pd.read_csv("EasyVVUQ/tutorials/epidemic/output.csv")
plt.plot(df['ill'], label="Sick Individuals")
plt.plot(df['immune'], label="Immune Individuals")
plt.plot(df['population'], label="Population")
plt.xlabel("Time")
plt.ylabel("Count")
plt.legend()
plt.show()
A short summary is in order so that we can start exploring the sample space of our model:
- We have a script that takes a JSON file with parameters and produces a CSV file with the output.
- The model takes 4 input parameters - grid size, population size, disease duration and mortality rate.
- The model produces 4 columns of output - iteration number, number of sick people, number of immune people and the current population size.
We will use EasyVVUQ to help us answer some questions about the model. Here are some simple ones that arise from toying with the model:
- Given that every time the length of time before the disease is erradicated is different even with the same parameters (due to the fact that each individual chooses where to move to at random), we might want to know, within a given certainty range, what the expected value of that is. This tells us, with needed confidence, how long we can expect the disease to last given certain parameters.
- We might also do the same thing but for a set of parameter values. Namely we might want to performa a parameter sweep with corresponding error bars.
- We might also want to improve our results by adding more samples to our analysis - hence we will see how we can restart a simulation and draw more samples to improve the accuracy.
- At some point we will want to use external resources to execute our simulations. We will quickly discuss how this can be done.
- Finally, given that our model is quite computationally expensive, we might want to explore the possibility of creating surrogate models to stand-in in place of the original model. These are usually expensive to create but very cheap to evaluate. Hence there is a possibility that we will be able to extract knowledge about our model from them that would be too expensive (computationally or otherwise) with a full simulation.
But first we need to set EasyVVUQ up to produce the configuration files in the suitable format and read in the output of the simulation. We also need to give a description of the parameter space. We also need to specify how we will execute our simulation. The next sections is concerened with these tasks.
EasyVVUQ Set-up¶
For the examples in this tutorial we import some libraries that will be used throughout. EasyVVUQ will be referred to as ‘uq’ in the code. We also need Chaospy because we use it for the probability distribution classes. We use numpy for certain small tasks and we use pandas DataFrame as the standard data exchange format as is customary in the Python data science infrastucture.
[13]:
import easyvvuq as uq
import chaospy as cp
import easyvvuq.collate
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import os
from pathlib import Path
While we are at it we also want to describe the arguments to our model. This takes a form of a Python dictionary. The dictionary is based on the Cerberus validator dictionary format. For more information refer to Cerberus documentation. This dictionary is used in both validation of the results and for the default values when we don’t want to vary a certain parameter.
[14]:
params = {
"grid_size": {"type": "float", "default": 10},
"n": {"type": "float", "min": 1, "max": 100, "default": 20},
"duration": {"type": "float", "min": 0, "max": 365, "default": 28},
"mortality": {"type": "float", "min": 0.0, "max": 1.0, "default": 0.2},
"ensemble" : {"type": "integer", "default": 0}
}
We will also want to set-up some elements that will stay the same for all the examples. These components are the encoder - which is responsible for creating input files for our simulation and the decoder - which is responsible for parsing the output of the simulation.
For the Encoder we use the GenericEncoder class. It is a very simple template based encoder. It finds a specified delimiter, and replaces the variable name that follows that delimiter with the corresponding value. In our case the template file looks like follows:
{
"grid_size" : $grid_size,
"n" : $n,
"duration" : $duration,
"mortality" : $mortality
}
From this template, a JSON file will be created and then passed to the simulation script as an argument. EasyVVUQ has other encoders as well. For example the Jinja encoder.
[15]:
encoder = uq.encoders.GenericEncoder(
template_fname='EasyVVUQ/tutorials/epidemic/epidemic.template',
delimiter='$',
target_filename='epidemic_in.json')
Since the quantity of interest (number of turns until the disease is erradicated) is a function of the simulation output (it is the iteration number of the last row) we need to extend the Decoder class to take this in to account. To this end we inherit from SimpleCSV decoder and redefine the parse_sim_output method to take the last value of the iteration column in the file produced by the simulation. This gives us the length in turns for which the simulation ran or in other words before
the disease disappeared in our simulation.
[58]:
class EpidemicDecoder(uq.decoders.SimpleCSV, decoder_name='epidemic_decoder'):
def parse_sim_output(self, run_info={}):
result = super().parse_sim_output(run_info)
return {'iteration': result['iteration'][-1]}
decoder = EpidemicDecoder(
target_filename="output.csv", output_columns=["iteration"])
We will also define a helper function that will execute the simulation with the provided input files. This function takes a campaign object, creates the directories with input files and then runs our script in them with those files as inputs. The exact details of this process can be found here and here.
[59]:
def campaign_execute(campaign):
campaign.apply_for_each_run_dir(
uq.actions.ExecuteLocal(
"{} epidemic_in.json output.csv".format(
os.path.abspath('EasyVVUQ/tutorials/epidemic/epidemic.py')),
interpret="python3"))
Basic Example¶
We start by creating an EasyVVUQ Campaign. Here we call it ‘epidemic_basic’. :
[60]:
campaign = uq.Campaign(name='epidemic_basic')
print(campaign)
db_location = sqlite:////Users/di73kuj2/Programming/EasyVVUQ/tutorials/epidemic_basicqy4s3zwf/campaign.db
active_sampler_id = None
campaign_name = epidemic_basic
campaign_dir = /Users/di73kuj2/Programming/EasyVVUQ/tutorials/epidemic_basicqy4s3zwf
campaign_id = 1
log = []
We then want to describe our application. This means passing parameter dictionary, enoder, decoder and collater to the campaign object.
[61]:
# Add the app (automatically set as current app)
campaign.add_app(
name="epidemic",
params=params,
encoder=encoder,
decoder=decoder)
For this particular task we are not interested in the relationship between input parameters and the behavior of the simulation. All we want is to see how much the result varies between runs that are identical but for the random seed.
[62]:
from easyvvuq.sampling import EmptySampler
campaign.set_sampler(EmptySampler())
EmptySampler is a convenience class for such cases. However, another option is, if your simulation provides the option to specify different seeds, to draw the seeds from a probability distribution. In cases like these you could specify the sampler like this:
from easyvvuq.sampling import RandomSampler
vary = {'seed' : cp.DiscreteUniform(0, MAX_SEED)}
sampler = RandomSampler(vary)
In the above example MAX_SEED is the maximum value the seed can take plus one.
Calling the campaign’s draw_samples() method will cause the specified number of samples to be added as runs to the campaign database, awaiting encoding and execution. Please note that nothing is executed at this stage. Neither any changes are made to the file system (e.g. no input files are created). This happens at a later stage. If no arguments are passed to draw_samples() then all samples will be drawn, unless the sampler is not finite. In this case let us try 20 samples :
[63]:
campaign.draw_samples(20)
Let us now create the input files for our simulations.
[64]:
campaign.populate_runs_dir()
[64]:
['Run_1',
'Run_2',
'Run_3',
'Run_4',
'Run_5',
'Run_6',
'Run_7',
'Run_8',
'Run_9',
'Run_10',
'Run_11',
'Run_12',
'Run_13',
'Run_14',
'Run_15',
'Run_16',
'Run_17',
'Run_18',
'Run_19',
'Run_20']
We now want to execute the simulations. Please note, that after this stage the results are not yet processed. The output files are just sitting on the filesystem awaiting collation.
[65]:
campaign_execute(campaign)
We can now see what the result is. It will be a DataFrame containing the number of iterations before the disease is erradicated. Calling collate on the campaign object will find and parse the output files of our simulation and then produce a single DataFrame with the results.
[66]:
campaign.collate()
df = campaign.get_collation_result()
print(df)
run_id grid_size n duration mortality ensemble iteration
0 0 0 0 0 0 0
0 1 10 20 28 0.2 0 62.0
1 2 10 20 28 0.2 0 60.0
2 3 10 20 28 0.2 0 62.0
3 4 10 20 28 0.2 0 62.0
4 5 10 20 28 0.2 0 92.0
5 6 10 20 28 0.2 0 87.0
6 7 10 20 28 0.2 0 61.0
7 8 10 20 28 0.2 0 83.0
8 9 10 20 28 0.2 0 87.0
9 10 10 20 28 0.2 0 55.0
10 11 10 20 28 0.2 0 66.0
11 12 10 20 28 0.2 0 81.0
12 13 10 20 28 0.2 0 99.0
13 14 10 20 28 0.2 0 57.0
14 15 10 20 28 0.2 0 97.0
15 16 10 20 28 0.2 0 102.0
16 17 10 20 28 0.2 0 77.0
17 18 10 20 28 0.2 0 63.0
18 19 10 20 28 0.2 0 85.0
19 20 10 20 28 0.2 0 74.0
This collated data is stored in the campaign database. An analysis element, here EnsembleBoot (which performs bootstraping), can then be applied to the campaign’s collation result. :
[68]:
analysis = uq.analysis.EnsembleBoot(qoi_cols=[("iteration", 0)], stat_func=np.mean, stat_name='mean', alpha=0.05)
campaign.apply_analysis(analysis)
The output of this is dependent on the type of analysis element. :
[69]:
# Get Descriptive Statistics
results = campaign.get_last_analysis()
print(results)
iteration
0
mean low high
True 75.25 69.15 82.05
The above gives the mean value of the number of iterations before the disease eradication and the 95% confidence region for that estimator.
Parameter Sweep¶
Suppose we want to examine the behaviour of the model by doing a sweep across a range of parameter values. Let’s say that given a fixed population size and fixed mortality rate we want to see how does the model behaves if we vary the disease duration parameter from 4 days to, for example, 28 days. We will do this in 2 day increments. Also, in order to perform some kind of statistical analysis of the results we will want to do sample the same parameter set several times. We will do this using the replicas mechanism in the EasyVVUQ.
First we define a new campaign.
[70]:
campaign = uq.Campaign(name='epidemic_sweep')
campaign.add_app(
name="epidemic_sweep",
params=params,
encoder=encoder,
decoder=decoder)
We need to define a different kind of sampler. In our case we want BasicSweep which allows one to sample an n-dimensional grid of values. In this case we specify a single parameter and give it a list of values. If you add more parameters the sampler will then sample a Cartesian product of those lists. For examples {'a' : [1, 2], 'b' : [3, 4]} will result in {'a': 1, 'b': 3}, {'a': 1, 'b': 4}, {'a': 2, 'b': 3} and {'a': 2, 'b': 4} being sampled.
[71]:
sweep = {
"duration" : list(range(2, 30, 2))
}
sweep_sampler = uq.sampling.BasicSweep(sweep=sweep)
We will also want to sample each point multiple times in order to construct confidence intervals. To this end we use a ReplicaSampler element which wraps around our sweep_sampler. In essence it will simply run the simulation with the same inputs multiple times with different random number generator seeds.
[72]:
from easyvvuq.sampling import ReplicaSampler
sampler = ReplicaSampler(sweep_sampler)
campaign.set_sampler(sampler)
Finally we want to draw some samples from this new sampler.
[73]:
campaign.draw_samples(20 * 14)
campaign.populate_runs_dir()
[73]:
['Run_1',
'Run_2',
'Run_3',
'Run_4',
'Run_5',
'Run_6',
'Run_7',
'Run_8',
'Run_9',
'Run_10',
'Run_11',
'Run_12',
'Run_13',
'Run_14',
'Run_15',
'Run_16',
'Run_17',
'Run_18',
'Run_19',
'Run_20',
'Run_21',
'Run_22',
'Run_23',
'Run_24',
'Run_25',
'Run_26',
'Run_27',
'Run_28',
'Run_29',
'Run_30',
'Run_31',
'Run_32',
'Run_33',
'Run_34',
'Run_35',
'Run_36',
'Run_37',
'Run_38',
'Run_39',
'Run_40',
'Run_41',
'Run_42',
'Run_43',
'Run_44',
'Run_45',
'Run_46',
'Run_47',
'Run_48',
'Run_49',
'Run_50',
'Run_51',
'Run_52',
'Run_53',
'Run_54',
'Run_55',
'Run_56',
'Run_57',
'Run_58',
'Run_59',
'Run_60',
'Run_61',
'Run_62',
'Run_63',
'Run_64',
'Run_65',
'Run_66',
'Run_67',
'Run_68',
'Run_69',
'Run_70',
'Run_71',
'Run_72',
'Run_73',
'Run_74',
'Run_75',
'Run_76',
'Run_77',
'Run_78',
'Run_79',
'Run_80',
'Run_81',
'Run_82',
'Run_83',
'Run_84',
'Run_85',
'Run_86',
'Run_87',
'Run_88',
'Run_89',
'Run_90',
'Run_91',
'Run_92',
'Run_93',
'Run_94',
'Run_95',
'Run_96',
'Run_97',
'Run_98',
'Run_99',
'Run_100',
'Run_101',
'Run_102',
'Run_103',
'Run_104',
'Run_105',
'Run_106',
'Run_107',
'Run_108',
'Run_109',
'Run_110',
'Run_111',
'Run_112',
'Run_113',
'Run_114',
'Run_115',
'Run_116',
'Run_117',
'Run_118',
'Run_119',
'Run_120',
'Run_121',
'Run_122',
'Run_123',
'Run_124',
'Run_125',
'Run_126',
'Run_127',
'Run_128',
'Run_129',
'Run_130',
'Run_131',
'Run_132',
'Run_133',
'Run_134',
'Run_135',
'Run_136',
'Run_137',
'Run_138',
'Run_139',
'Run_140',
'Run_141',
'Run_142',
'Run_143',
'Run_144',
'Run_145',
'Run_146',
'Run_147',
'Run_148',
'Run_149',
'Run_150',
'Run_151',
'Run_152',
'Run_153',
'Run_154',
'Run_155',
'Run_156',
'Run_157',
'Run_158',
'Run_159',
'Run_160',
'Run_161',
'Run_162',
'Run_163',
'Run_164',
'Run_165',
'Run_166',
'Run_167',
'Run_168',
'Run_169',
'Run_170',
'Run_171',
'Run_172',
'Run_173',
'Run_174',
'Run_175',
'Run_176',
'Run_177',
'Run_178',
'Run_179',
'Run_180',
'Run_181',
'Run_182',
'Run_183',
'Run_184',
'Run_185',
'Run_186',
'Run_187',
'Run_188',
'Run_189',
'Run_190',
'Run_191',
'Run_192',
'Run_193',
'Run_194',
'Run_195',
'Run_196',
'Run_197',
'Run_198',
'Run_199',
'Run_200',
'Run_201',
'Run_202',
'Run_203',
'Run_204',
'Run_205',
'Run_206',
'Run_207',
'Run_208',
'Run_209',
'Run_210',
'Run_211',
'Run_212',
'Run_213',
'Run_214',
'Run_215',
'Run_216',
'Run_217',
'Run_218',
'Run_219',
'Run_220',
'Run_221',
'Run_222',
'Run_223',
'Run_224',
'Run_225',
'Run_226',
'Run_227',
'Run_228',
'Run_229',
'Run_230',
'Run_231',
'Run_232',
'Run_233',
'Run_234',
'Run_235',
'Run_236',
'Run_237',
'Run_238',
'Run_239',
'Run_240',
'Run_241',
'Run_242',
'Run_243',
'Run_244',
'Run_245',
'Run_246',
'Run_247',
'Run_248',
'Run_249',
'Run_250',
'Run_251',
'Run_252',
'Run_253',
'Run_254',
'Run_255',
'Run_256',
'Run_257',
'Run_258',
'Run_259',
'Run_260',
'Run_261',
'Run_262',
'Run_263',
'Run_264',
'Run_265',
'Run_266',
'Run_267',
'Run_268',
'Run_269',
'Run_270',
'Run_271',
'Run_272',
'Run_273',
'Run_274',
'Run_275',
'Run_276',
'Run_277',
'Run_278',
'Run_279',
'Run_280']
The following steps are the same as in our more basic example. Note that execution might take some time. That is because we are running 280 simulations on the machine that is hosting this notebook.
[86]:
statuses = campaign.apply_for_each_run_dir(
uq.actions.ExecuteLocal("{} epidemic_in.json output.csv".format(
os.path.abspath('EasyVVUQ/tutorials/epidemic/epidemic.py')), interpret="python3"))
[87]:
print(statuses.progress())
{'ready': 0, 'active': 0, 'finished': 0, 'failed': 0}
[88]:
campaign.collate()
df = campaign.get_collation_result()
print(df)
run_id duration ensemble grid_size n mortality iteration
0 0 0 0 0 0 0
0 1 2 0 10 20 0.2 2.0
1 2 4 1 10 20 0.2 8.0
2 3 6 2 10 20 0.2 8.0
3 4 8 3 10 20 0.2 7.0
4 5 10 4 10 20 0.2 9.0
.. ... ... ... ... .. ... ...
275 276 20 9 10 20 0.2 96.0
276 277 22 10 10 20 0.2 68.0
277 278 24 11 10 20 0.2 64.0
278 279 26 12 10 20 0.2 59.0
279 280 28 13 10 20 0.2 82.0
[280 rows x 7 columns]
One important difference in the analysis stage below is the addition of the groupby keyword argument. This means, essentially, that bootstrapping will be done on rows with the same ensemble_id value separately. So in the end we will get (30 - 2) / 2 = 14 triples of the mean value and lower/upper confidence bounds.
[91]:
analysis = uq.analysis.EnsembleBoot(qoi_cols=[("iteration", 0)], groupby=('ensemble', 0), stat_func=np.mean, stat_name='mean')
campaign.apply_analysis(analysis)
We can now view the resulting DataFrame.
[92]:
results = campaign.get_last_analysis()
print(results)
iteration
0
mean low high
(ensemble, 0)
0 1.200 1.00000 1.45000
1 5.750 4.64875 6.95000
2 9.350 7.89875 11.00125
3 16.950 13.54875 20.50000
4 25.550 19.45000 31.80000
5 38.675 30.09750 46.45500
6 36.950 30.14875 43.70125
7 55.100 46.84625 62.90500
8 49.050 41.19875 56.05000
9 63.850 60.29875 68.90125
10 66.350 58.19875 74.00000
11 72.400 62.94875 82.95250
12 75.850 69.09750 84.10250
13 73.175 68.30000 78.25000
[96]:
#plt.errorbar(list(range(2, 30, 2)),
# results[('iteration', 0)]['mean'].values,
# np.array([results['iteration']['mean'].values - results['iteration']['low'].values,
# results['iteration']['high'].values - results['iteration']['mean'].values]))
#plt.xlabel('duration (days)')
#plt.ylabel('days until eradication')
#plt.show()
Remote Execution¶
This part of the tutorial assumes that you have a Kubernetes cluster access configured. In other words the ~/.kube/config file needs to be populated with the data that Kubernetes API can use to connect to a cluster. The exact details for how to do this will depend on your cloud service provider. For example, in order to start a cluster on GKE I would need to run the following command (or similar):
[ ]:
!gcloud container clusters create easyvvuq
To use this functionality you would first need to create a Docker image for your simulation. There are many resource on how to do this. See for example here. For your convenience below is the Dockerfile we have used when creating an EasyVVUQ image that will be used in his example. You also need to publish your Docker image to where it can be downloaded by the Kubernetes cluster. An obvious place is Docker Hub.
FROM ubuntu:latest
RUN apt-get update && \
apt-get install -y python3-pip && \
apt-get install -y git && \
apt-get install -y tini && \
pip3 install easyvvuq && \
git clone https://github.com/UCL-CCS/EasyVVUQ.git
ENTRYPOINT ["tini", "--"]
After this, the only other bit of boring admin you need to do is to create a Kubernetes pod configuration. Again here is the one that was used in this tutorial:
apiVersion: v1
kind: Pod
metadata:
name: epidemic
spec:
restartPolicy: Never
containers:
- name: epidemic
image: orbitfold/easyvvuq:latest
command: ["/bin/sh", "-c"]
args: ["python3 /EasyVVUQ/docs/epidemic/epidemic.py /config/example.json out.csv && cat out.csv"]
It will likely be the same for your simulation. Please note the image name, and the command that will be used to execute the simulation. In the current implementation your simulation needs to put all output to the standard output. So after creating the output CSV file we use cat to print it to the screen. Your way of doing this is likely to be different.
We have, so far, drawn 20 samples in our parameter sweep which has resulted in fairly wide error bars. We can shrink them by adding more samples. To this end will draw 80 more samples and create the corresponding input files.
[105]:
campaign.draw_samples(60 * 14)
campaign.populate_runs_dir()
[105]:
['Run_1121',
'Run_1122',
'Run_1123',
'Run_1124',
'Run_1125',
'Run_1126',
'Run_1127',
'Run_1128',
'Run_1129',
'Run_1130',
'Run_1131',
'Run_1132',
'Run_1133',
'Run_1134',
'Run_1135',
'Run_1136',
'Run_1137',
'Run_1138',
'Run_1139',
'Run_1140',
'Run_1141',
'Run_1142',
'Run_1143',
'Run_1144',
'Run_1145',
'Run_1146',
'Run_1147',
'Run_1148',
'Run_1149',
'Run_1150',
'Run_1151',
'Run_1152',
'Run_1153',
'Run_1154',
'Run_1155',
'Run_1156',
'Run_1157',
'Run_1158',
'Run_1159',
'Run_1160',
'Run_1161',
'Run_1162',
'Run_1163',
'Run_1164',
'Run_1165',
'Run_1166',
'Run_1167',
'Run_1168',
'Run_1169',
'Run_1170',
'Run_1171',
'Run_1172',
'Run_1173',
'Run_1174',
'Run_1175',
'Run_1176',
'Run_1177',
'Run_1178',
'Run_1179',
'Run_1180',
'Run_1181',
'Run_1182',
'Run_1183',
'Run_1184',
'Run_1185',
'Run_1186',
'Run_1187',
'Run_1188',
'Run_1189',
'Run_1190',
'Run_1191',
'Run_1192',
'Run_1193',
'Run_1194',
'Run_1195',
'Run_1196',
'Run_1197',
'Run_1198',
'Run_1199',
'Run_1200',
'Run_1201',
'Run_1202',
'Run_1203',
'Run_1204',
'Run_1205',
'Run_1206',
'Run_1207',
'Run_1208',
'Run_1209',
'Run_1210',
'Run_1211',
'Run_1212',
'Run_1213',
'Run_1214',
'Run_1215',
'Run_1216',
'Run_1217',
'Run_1218',
'Run_1219',
'Run_1220',
'Run_1221',
'Run_1222',
'Run_1223',
'Run_1224',
'Run_1225',
'Run_1226',
'Run_1227',
'Run_1228',
'Run_1229',
'Run_1230',
'Run_1231',
'Run_1232',
'Run_1233',
'Run_1234',
'Run_1235',
'Run_1236',
'Run_1237',
'Run_1238',
'Run_1239',
'Run_1240',
'Run_1241',
'Run_1242',
'Run_1243',
'Run_1244',
'Run_1245',
'Run_1246',
'Run_1247',
'Run_1248',
'Run_1249',
'Run_1250',
'Run_1251',
'Run_1252',
'Run_1253',
'Run_1254',
'Run_1255',
'Run_1256',
'Run_1257',
'Run_1258',
'Run_1259',
'Run_1260',
'Run_1261',
'Run_1262',
'Run_1263',
'Run_1264',
'Run_1265',
'Run_1266',
'Run_1267',
'Run_1268',
'Run_1269',
'Run_1270',
'Run_1271',
'Run_1272',
'Run_1273',
'Run_1274',
'Run_1275',
'Run_1276',
'Run_1277',
'Run_1278',
'Run_1279',
'Run_1280',
'Run_1281',
'Run_1282',
'Run_1283',
'Run_1284',
'Run_1285',
'Run_1286',
'Run_1287',
'Run_1288',
'Run_1289',
'Run_1290',
'Run_1291',
'Run_1292',
'Run_1293',
'Run_1294',
'Run_1295',
'Run_1296',
'Run_1297',
'Run_1298',
'Run_1299',
'Run_1300',
'Run_1301',
'Run_1302',
'Run_1303',
'Run_1304',
'Run_1305',
'Run_1306',
'Run_1307',
'Run_1308',
'Run_1309',
'Run_1310',
'Run_1311',
'Run_1312',
'Run_1313',
'Run_1314',
'Run_1315',
'Run_1316',
'Run_1317',
'Run_1318',
'Run_1319',
'Run_1320',
'Run_1321',
'Run_1322',
'Run_1323',
'Run_1324',
'Run_1325',
'Run_1326',
'Run_1327',
'Run_1328',
'Run_1329',
'Run_1330',
'Run_1331',
'Run_1332',
'Run_1333',
'Run_1334',
'Run_1335',
'Run_1336',
'Run_1337',
'Run_1338',
'Run_1339',
'Run_1340',
'Run_1341',
'Run_1342',
'Run_1343',
'Run_1344',
'Run_1345',
'Run_1346',
'Run_1347',
'Run_1348',
'Run_1349',
'Run_1350',
'Run_1351',
'Run_1352',
'Run_1353',
'Run_1354',
'Run_1355',
'Run_1356',
'Run_1357',
'Run_1358',
'Run_1359',
'Run_1360',
'Run_1361',
'Run_1362',
'Run_1363',
'Run_1364',
'Run_1365',
'Run_1366',
'Run_1367',
'Run_1368',
'Run_1369',
'Run_1370',
'Run_1371',
'Run_1372',
'Run_1373',
'Run_1374',
'Run_1375',
'Run_1376',
'Run_1377',
'Run_1378',
'Run_1379',
'Run_1380',
'Run_1381',
'Run_1382',
'Run_1383',
'Run_1384',
'Run_1385',
'Run_1386',
'Run_1387',
'Run_1388',
'Run_1389',
'Run_1390',
'Run_1391',
'Run_1392',
'Run_1393',
'Run_1394',
'Run_1395',
'Run_1396',
'Run_1397',
'Run_1398',
'Run_1399',
'Run_1400',
'Run_1401',
'Run_1402',
'Run_1403',
'Run_1404',
'Run_1405',
'Run_1406',
'Run_1407',
'Run_1408',
'Run_1409',
'Run_1410',
'Run_1411',
'Run_1412',
'Run_1413',
'Run_1414',
'Run_1415',
'Run_1416',
'Run_1417',
'Run_1418',
'Run_1419',
'Run_1420',
'Run_1421',
'Run_1422',
'Run_1423',
'Run_1424',
'Run_1425',
'Run_1426',
'Run_1427',
'Run_1428',
'Run_1429',
'Run_1430',
'Run_1431',
'Run_1432',
'Run_1433',
'Run_1434',
'Run_1435',
'Run_1436',
'Run_1437',
'Run_1438',
'Run_1439',
'Run_1440',
'Run_1441',
'Run_1442',
'Run_1443',
'Run_1444',
'Run_1445',
'Run_1446',
'Run_1447',
'Run_1448',
'Run_1449',
'Run_1450',
'Run_1451',
'Run_1452',
'Run_1453',
'Run_1454',
'Run_1455',
'Run_1456',
'Run_1457',
'Run_1458',
'Run_1459',
'Run_1460',
'Run_1461',
'Run_1462',
'Run_1463',
'Run_1464',
'Run_1465',
'Run_1466',
'Run_1467',
'Run_1468',
'Run_1469',
'Run_1470',
'Run_1471',
'Run_1472',
'Run_1473',
'Run_1474',
'Run_1475',
'Run_1476',
'Run_1477',
'Run_1478',
'Run_1479',
'Run_1480',
'Run_1481',
'Run_1482',
'Run_1483',
'Run_1484',
'Run_1485',
'Run_1486',
'Run_1487',
'Run_1488',
'Run_1489',
'Run_1490',
'Run_1491',
'Run_1492',
'Run_1493',
'Run_1494',
'Run_1495',
'Run_1496',
'Run_1497',
'Run_1498',
'Run_1499',
'Run_1500',
'Run_1501',
'Run_1502',
'Run_1503',
'Run_1504',
'Run_1505',
'Run_1506',
'Run_1507',
'Run_1508',
'Run_1509',
'Run_1510',
'Run_1511',
'Run_1512',
'Run_1513',
'Run_1514',
'Run_1515',
'Run_1516',
'Run_1517',
'Run_1518',
'Run_1519',
'Run_1520',
'Run_1521',
'Run_1522',
'Run_1523',
'Run_1524',
'Run_1525',
'Run_1526',
'Run_1527',
'Run_1528',
'Run_1529',
'Run_1530',
'Run_1531',
'Run_1532',
'Run_1533',
'Run_1534',
'Run_1535',
'Run_1536',
'Run_1537',
'Run_1538',
'Run_1539',
'Run_1540',
'Run_1541',
'Run_1542',
'Run_1543',
'Run_1544',
'Run_1545',
'Run_1546',
'Run_1547',
'Run_1548',
'Run_1549',
'Run_1550',
'Run_1551',
'Run_1552',
'Run_1553',
'Run_1554',
'Run_1555',
'Run_1556',
'Run_1557',
'Run_1558',
'Run_1559',
'Run_1560',
'Run_1561',
'Run_1562',
'Run_1563',
'Run_1564',
'Run_1565',
'Run_1566',
'Run_1567',
'Run_1568',
'Run_1569',
'Run_1570',
'Run_1571',
'Run_1572',
'Run_1573',
'Run_1574',
'Run_1575',
'Run_1576',
'Run_1577',
'Run_1578',
'Run_1579',
'Run_1580',
'Run_1581',
'Run_1582',
'Run_1583',
'Run_1584',
'Run_1585',
'Run_1586',
'Run_1587',
'Run_1588',
'Run_1589',
'Run_1590',
'Run_1591',
'Run_1592',
'Run_1593',
'Run_1594',
'Run_1595',
'Run_1596',
'Run_1597',
'Run_1598',
'Run_1599',
'Run_1600',
'Run_1601',
'Run_1602',
'Run_1603',
'Run_1604',
'Run_1605',
'Run_1606',
'Run_1607',
'Run_1608',
'Run_1609',
'Run_1610',
'Run_1611',
'Run_1612',
'Run_1613',
'Run_1614',
'Run_1615',
'Run_1616',
'Run_1617',
'Run_1618',
'Run_1619',
'Run_1620',
'Run_1621',
'Run_1622',
'Run_1623',
'Run_1624',
'Run_1625',
'Run_1626',
'Run_1627',
'Run_1628',
'Run_1629',
'Run_1630',
'Run_1631',
'Run_1632',
'Run_1633',
'Run_1634',
'Run_1635',
'Run_1636',
'Run_1637',
'Run_1638',
'Run_1639',
'Run_1640',
'Run_1641',
'Run_1642',
'Run_1643',
'Run_1644',
'Run_1645',
'Run_1646',
'Run_1647',
'Run_1648',
'Run_1649',
'Run_1650',
'Run_1651',
'Run_1652',
'Run_1653',
'Run_1654',
'Run_1655',
'Run_1656',
'Run_1657',
'Run_1658',
'Run_1659',
'Run_1660',
'Run_1661',
'Run_1662',
'Run_1663',
'Run_1664',
'Run_1665',
'Run_1666',
'Run_1667',
'Run_1668',
'Run_1669',
'Run_1670',
'Run_1671',
'Run_1672',
'Run_1673',
'Run_1674',
'Run_1675',
'Run_1676',
'Run_1677',
'Run_1678',
'Run_1679',
'Run_1680',
'Run_1681',
'Run_1682',
'Run_1683',
'Run_1684',
'Run_1685',
'Run_1686',
'Run_1687',
'Run_1688',
'Run_1689',
'Run_1690',
'Run_1691',
'Run_1692',
'Run_1693',
'Run_1694',
'Run_1695',
'Run_1696',
'Run_1697',
'Run_1698',
'Run_1699',
'Run_1700',
'Run_1701',
'Run_1702',
'Run_1703',
'Run_1704',
'Run_1705',
'Run_1706',
'Run_1707',
'Run_1708',
'Run_1709',
'Run_1710',
'Run_1711',
'Run_1712',
'Run_1713',
'Run_1714',
'Run_1715',
'Run_1716',
'Run_1717',
'Run_1718',
'Run_1719',
'Run_1720',
'Run_1721',
'Run_1722',
'Run_1723',
'Run_1724',
'Run_1725',
'Run_1726',
'Run_1727',
'Run_1728',
'Run_1729',
'Run_1730',
'Run_1731',
'Run_1732',
'Run_1733',
'Run_1734',
'Run_1735',
'Run_1736',
'Run_1737',
'Run_1738',
'Run_1739',
'Run_1740',
'Run_1741',
'Run_1742',
'Run_1743',
'Run_1744',
'Run_1745',
'Run_1746',
'Run_1747',
'Run_1748',
'Run_1749',
'Run_1750',
'Run_1751',
'Run_1752',
'Run_1753',
'Run_1754',
'Run_1755',
'Run_1756',
'Run_1757',
'Run_1758',
'Run_1759',
'Run_1760',
'Run_1761',
'Run_1762',
'Run_1763',
'Run_1764',
'Run_1765',
'Run_1766',
'Run_1767',
'Run_1768',
'Run_1769',
'Run_1770',
'Run_1771',
'Run_1772',
'Run_1773',
'Run_1774',
'Run_1775',
'Run_1776',
'Run_1777',
'Run_1778',
'Run_1779',
'Run_1780',
'Run_1781',
'Run_1782',
'Run_1783',
'Run_1784',
'Run_1785',
'Run_1786',
'Run_1787',
'Run_1788',
'Run_1789',
'Run_1790',
'Run_1791',
'Run_1792',
'Run_1793',
'Run_1794',
'Run_1795',
'Run_1796',
'Run_1797',
'Run_1798',
'Run_1799',
'Run_1800',
'Run_1801',
'Run_1802',
'Run_1803',
'Run_1804',
'Run_1805',
'Run_1806',
'Run_1807',
'Run_1808',
'Run_1809',
'Run_1810',
'Run_1811',
'Run_1812',
'Run_1813',
'Run_1814',
'Run_1815',
'Run_1816',
'Run_1817',
'Run_1818',
'Run_1819',
'Run_1820',
'Run_1821',
'Run_1822',
'Run_1823',
'Run_1824',
'Run_1825',
'Run_1826',
'Run_1827',
'Run_1828',
'Run_1829',
'Run_1830',
'Run_1831',
'Run_1832',
'Run_1833',
'Run_1834',
'Run_1835',
'Run_1836',
'Run_1837',
'Run_1838',
'Run_1839',
'Run_1840',
'Run_1841',
'Run_1842',
'Run_1843',
'Run_1844',
'Run_1845',
'Run_1846',
'Run_1847',
'Run_1848',
'Run_1849',
'Run_1850',
'Run_1851',
'Run_1852',
'Run_1853',
'Run_1854',
'Run_1855',
'Run_1856',
'Run_1857',
'Run_1858',
'Run_1859',
'Run_1860',
'Run_1861',
'Run_1862',
'Run_1863',
'Run_1864',
'Run_1865',
'Run_1866',
'Run_1867',
'Run_1868',
'Run_1869',
'Run_1870',
'Run_1871',
'Run_1872',
'Run_1873',
'Run_1874',
'Run_1875',
'Run_1876',
'Run_1877',
'Run_1878',
'Run_1879',
'Run_1880',
'Run_1881',
'Run_1882',
'Run_1883',
'Run_1884',
'Run_1885',
'Run_1886',
'Run_1887',
'Run_1888',
'Run_1889',
'Run_1890',
'Run_1891',
'Run_1892',
'Run_1893',
'Run_1894',
'Run_1895',
'Run_1896',
'Run_1897',
'Run_1898',
'Run_1899',
'Run_1900',
'Run_1901',
'Run_1902',
'Run_1903',
'Run_1904',
'Run_1905',
'Run_1906',
'Run_1907',
'Run_1908',
'Run_1909',
'Run_1910',
'Run_1911',
'Run_1912',
'Run_1913',
'Run_1914',
'Run_1915',
'Run_1916',
'Run_1917',
'Run_1918',
'Run_1919',
'Run_1920',
'Run_1921',
'Run_1922',
'Run_1923',
'Run_1924',
'Run_1925',
'Run_1926',
'Run_1927',
'Run_1928',
'Run_1929',
'Run_1930',
'Run_1931',
'Run_1932',
'Run_1933',
'Run_1934',
'Run_1935',
'Run_1936',
'Run_1937',
'Run_1938',
'Run_1939',
'Run_1940',
'Run_1941',
'Run_1942',
'Run_1943',
'Run_1944',
'Run_1945',
'Run_1946',
'Run_1947',
'Run_1948',
'Run_1949',
'Run_1950',
'Run_1951',
'Run_1952',
'Run_1953',
'Run_1954',
'Run_1955',
'Run_1956',
'Run_1957',
'Run_1958',
'Run_1959',
'Run_1960']
Now we call apply_for_each_run_dir with the ExecuteKubernetes action. This will submit the jobs to the Kubernetes cluster and the execution will automatically start.
[106]:
statuses = campaign.apply_for_each_run_dir(
uq.actions.ExecuteKubernetes('../tutorials/kubernetes/epidemic.yaml',
['epidemic_in.json'], 'output.csv'), batch_size=8)
[50]:
statuses = campaign.sample_and_apply(
60 * 14,
uq.actions.ExecuteKubernetes(
'../tutorials/kubernetes/epidemic.yaml',
['epidemic_in.json'],
'output.csv'), 8)
[110]:
statuses.start()
[110]:
[<Future at 0x129f93760 state=running>,
<Future at 0x129f934f0 state=running>,
<Future at 0x129f9ae20 state=running>,
<Future at 0x129f9f550 state=running>,
<Future at 0x11c8f2220 state=running>,
<Future at 0x129f9fc10 state=running>,
<Future at 0x129fa4160 state=running>,
<Future at 0x129fa4670 state=running>,
<Future at 0x129fa4b80 state=pending>,
<Future at 0x129fa4d90 state=pending>,
<Future at 0x129fa4ee0 state=pending>,
<Future at 0x129fa90a0 state=pending>,
<Future at 0x129fa91f0 state=pending>,
<Future at 0x129fa9370 state=pending>,
<Future at 0x129fa94c0 state=pending>,
<Future at 0x129fa9640 state=pending>,
<Future at 0x129fa9790 state=pending>,
<Future at 0x129fa9910 state=pending>,
<Future at 0x129fa9a60 state=pending>,
<Future at 0x129fa9be0 state=pending>,
<Future at 0x129fa9d30 state=pending>,
<Future at 0x129d6cf70 state=pending>,
<Future at 0x129fa9f40 state=pending>,
<Future at 0x129fb00a0 state=pending>,
<Future at 0x129fb0220 state=pending>,
<Future at 0x129fb03d0 state=pending>,
<Future at 0x129fb0550 state=pending>,
<Future at 0x129fb0640 state=pending>,
<Future at 0x129fb07c0 state=pending>,
<Future at 0x129fb0970 state=pending>,
<Future at 0x129fb0af0 state=pending>,
<Future at 0x129fb0be0 state=pending>,
<Future at 0x129fb0d60 state=pending>,
<Future at 0x129fb0f10 state=pending>,
<Future at 0x129fb80a0 state=pending>,
<Future at 0x129fb81f0 state=pending>,
<Future at 0x129fb8370 state=pending>,
<Future at 0x129fb84c0 state=pending>,
<Future at 0x129fb8640 state=pending>,
<Future at 0x129fb8790 state=pending>,
<Future at 0x129fb8910 state=pending>,
<Future at 0x129fb8a60 state=pending>,
<Future at 0x129fb8be0 state=pending>,
<Future at 0x129fb8d30 state=pending>,
<Future at 0x129fb8eb0 state=pending>,
<Future at 0x129fc1040 state=pending>,
<Future at 0x129fc11c0 state=pending>,
<Future at 0x129fc1310 state=pending>,
<Future at 0x129fc1490 state=pending>,
<Future at 0x129fc15e0 state=pending>,
<Future at 0x129fc1760 state=pending>,
<Future at 0x129fc18b0 state=pending>,
<Future at 0x129fc1a30 state=pending>,
<Future at 0x129fc1b80 state=pending>,
<Future at 0x129fc1d00 state=pending>,
<Future at 0x129fc1e50 state=pending>,
<Future at 0x129fc1fd0 state=pending>,
<Future at 0x12a047160 state=pending>,
<Future at 0x12a0472e0 state=pending>,
<Future at 0x12a047430 state=pending>,
<Future at 0x12a0475b0 state=pending>,
<Future at 0x12a047700 state=pending>,
<Future at 0x12a047880 state=pending>,
<Future at 0x12a0479d0 state=pending>,
<Future at 0x12a047b50 state=pending>,
<Future at 0x12a047ca0 state=pending>,
<Future at 0x12a047e20 state=pending>,
<Future at 0x12a047f70 state=pending>,
<Future at 0x12a04d130 state=pending>,
<Future at 0x12a04d280 state=pending>,
<Future at 0x12a04d400 state=pending>,
<Future at 0x12a04d550 state=pending>,
<Future at 0x12a04d6d0 state=pending>,
<Future at 0x12a04d820 state=pending>,
<Future at 0x12a04d9a0 state=pending>,
<Future at 0x12a04daf0 state=pending>,
<Future at 0x12a04dc70 state=pending>,
<Future at 0x12a04ddc0 state=pending>,
<Future at 0x12a04df40 state=pending>,
<Future at 0x12a0530d0 state=pending>,
<Future at 0x12a053250 state=pending>,
<Future at 0x12a0533a0 state=pending>,
<Future at 0x12a053520 state=pending>,
<Future at 0x12a053670 state=pending>,
<Future at 0x12a0537f0 state=pending>,
<Future at 0x12a053940 state=pending>,
<Future at 0x12a053ac0 state=pending>,
<Future at 0x12a053c10 state=pending>,
<Future at 0x12a053d90 state=pending>,
<Future at 0x12a053ee0 state=pending>,
<Future at 0x12a0580a0 state=pending>,
<Future at 0x12a0581f0 state=pending>,
<Future at 0x12a058370 state=pending>,
<Future at 0x12a0584c0 state=pending>,
<Future at 0x12a058640 state=pending>,
<Future at 0x12a058790 state=pending>,
<Future at 0x12a058910 state=pending>,
<Future at 0x12a058a60 state=pending>,
<Future at 0x12a058be0 state=pending>,
<Future at 0x12a058d30 state=pending>,
<Future at 0x12a058eb0 state=pending>,
<Future at 0x12a05d040 state=pending>,
<Future at 0x12a05d1c0 state=pending>,
<Future at 0x12a05d310 state=pending>,
<Future at 0x12a05d490 state=pending>,
<Future at 0x12a05d5e0 state=pending>,
<Future at 0x12a05d760 state=pending>,
<Future at 0x12a05d8b0 state=pending>,
<Future at 0x12a05da30 state=pending>,
<Future at 0x12a05db80 state=pending>,
<Future at 0x12a05dd00 state=pending>,
<Future at 0x12a05de50 state=pending>,
<Future at 0x12a05dfd0 state=pending>,
<Future at 0x12a062160 state=pending>,
<Future at 0x12a0622e0 state=pending>,
<Future at 0x12a062430 state=pending>,
<Future at 0x12a0625b0 state=pending>,
<Future at 0x12a062700 state=pending>,
<Future at 0x12a062880 state=pending>,
<Future at 0x12a0629d0 state=pending>,
<Future at 0x12a062b50 state=pending>,
<Future at 0x12a062ca0 state=pending>,
<Future at 0x12a062e20 state=pending>,
<Future at 0x12a062f70 state=pending>,
<Future at 0x12a068130 state=pending>,
<Future at 0x12a068280 state=pending>,
<Future at 0x12a068400 state=pending>,
<Future at 0x12a068550 state=pending>,
<Future at 0x12a0686d0 state=pending>,
<Future at 0x12a068820 state=pending>,
<Future at 0x12a0689a0 state=pending>,
<Future at 0x12a068af0 state=pending>,
<Future at 0x12a068c70 state=pending>,
<Future at 0x12a068dc0 state=pending>,
<Future at 0x12a068f40 state=pending>,
<Future at 0x12a06d0d0 state=pending>,
<Future at 0x12a06d250 state=pending>,
<Future at 0x12a06d3a0 state=pending>,
<Future at 0x12a06d520 state=pending>,
<Future at 0x12a06d670 state=pending>,
<Future at 0x12a06d7f0 state=pending>,
<Future at 0x12a06d940 state=pending>,
<Future at 0x12a06dac0 state=pending>,
<Future at 0x12a06dc10 state=pending>,
<Future at 0x12a06dd90 state=pending>,
<Future at 0x12a06dee0 state=pending>,
<Future at 0x12a0720a0 state=pending>,
<Future at 0x12a0721f0 state=pending>,
<Future at 0x12a072370 state=pending>,
<Future at 0x12a0724c0 state=pending>,
<Future at 0x12a072640 state=pending>,
<Future at 0x12a072790 state=pending>,
<Future at 0x12a072910 state=pending>,
<Future at 0x12a072a60 state=pending>,
<Future at 0x12a072be0 state=pending>,
<Future at 0x12a072d30 state=pending>,
<Future at 0x12a072eb0 state=pending>,
<Future at 0x12a077040 state=pending>,
<Future at 0x12a0771c0 state=pending>,
<Future at 0x12a077310 state=pending>,
<Future at 0x12a077490 state=pending>,
<Future at 0x12a0775e0 state=pending>,
<Future at 0x12a077760 state=pending>,
<Future at 0x12a0778b0 state=pending>,
<Future at 0x12a077a30 state=pending>,
<Future at 0x12a077b80 state=pending>,
<Future at 0x12a077d00 state=pending>,
<Future at 0x12a077e50 state=pending>,
<Future at 0x12a077fd0 state=pending>,
<Future at 0x12a07d160 state=pending>,
<Future at 0x12a07d2e0 state=pending>,
<Future at 0x12a07d430 state=pending>,
<Future at 0x12a07d5b0 state=pending>,
<Future at 0x12a07d700 state=pending>,
<Future at 0x12a07d880 state=pending>,
<Future at 0x12a07d9d0 state=pending>,
<Future at 0x12a07db50 state=pending>,
<Future at 0x12a07dca0 state=pending>,
<Future at 0x12a07de20 state=pending>,
<Future at 0x12a07df70 state=pending>,
<Future at 0x12a083130 state=pending>,
<Future at 0x12a083280 state=pending>,
<Future at 0x12a083400 state=pending>,
<Future at 0x12a083550 state=pending>,
<Future at 0x12a0836d0 state=pending>,
<Future at 0x12a083820 state=pending>,
<Future at 0x12a0839a0 state=pending>,
<Future at 0x12a083af0 state=pending>,
<Future at 0x12a083c70 state=pending>,
<Future at 0x12a083dc0 state=pending>,
<Future at 0x12a083f40 state=pending>,
<Future at 0x12a0880d0 state=pending>,
<Future at 0x12a088250 state=pending>,
<Future at 0x12a0883a0 state=pending>,
<Future at 0x12a088520 state=pending>,
<Future at 0x12a088670 state=pending>,
<Future at 0x12a0887f0 state=pending>,
<Future at 0x12a088940 state=pending>,
<Future at 0x12a088ac0 state=pending>,
<Future at 0x12a088c10 state=pending>,
<Future at 0x12a088d90 state=pending>,
<Future at 0x12a088ee0 state=pending>,
<Future at 0x12a08d760 state=pending>,
<Future at 0x12a08d160 state=pending>,
<Future at 0x12a08d2e0 state=pending>,
<Future at 0x12a08d430 state=pending>,
<Future at 0x12a08d5b0 state=pending>,
<Future at 0x129835070 state=pending>,
<Future at 0x129835fa0 state=pending>,
<Future at 0x129b5cf40 state=pending>,
<Future at 0x129b5c700 state=pending>,
<Future at 0x129b5c8b0 state=pending>,
<Future at 0x129b5c0a0 state=pending>,
<Future at 0x129b5c9d0 state=pending>,
<Future at 0x129b5c2e0 state=pending>,
<Future at 0x129b5c910 state=pending>,
<Future at 0x129b5c400 state=pending>,
<Future at 0x129b5c160 state=pending>,
<Future at 0x129b5c760 state=pending>,
<Future at 0x129b5cdc0 state=pending>,
<Future at 0x129af8d60 state=pending>,
<Future at 0x129af8be0 state=pending>,
<Future at 0x129af8df0 state=pending>,
<Future at 0x129af8a00 state=pending>,
<Future at 0x129af8430 state=pending>,
<Future at 0x129af8dc0 state=pending>,
<Future at 0x129af8ca0 state=pending>,
<Future at 0x129af84c0 state=pending>,
<Future at 0x129af8ac0 state=pending>,
<Future at 0x129bc4a30 state=pending>,
<Future at 0x129bc49d0 state=pending>,
<Future at 0x129bc4bb0 state=pending>,
<Future at 0x129bc4c10 state=pending>,
<Future at 0x129bc4a90 state=pending>,
<Future at 0x129bc4eb0 state=pending>,
<Future at 0x129bc42e0 state=pending>,
<Future at 0x129bc4970 state=pending>,
<Future at 0x129bc4220 state=pending>,
<Future at 0x129bc43d0 state=pending>,
<Future at 0x129bc41c0 state=pending>,
<Future at 0x129bc4670 state=pending>,
<Future at 0x129bc47f0 state=pending>,
<Future at 0x129bc4490 state=pending>,
<Future at 0x129bc4700 state=pending>,
<Future at 0x129b36280 state=pending>,
<Future at 0x129b36250 state=pending>,
<Future at 0x129b36c10 state=pending>,
<Future at 0x129b364c0 state=pending>,
<Future at 0x129b36190 state=pending>,
<Future at 0x129b36be0 state=pending>,
<Future at 0x129b36f40 state=pending>,
<Future at 0x129b36ee0 state=pending>,
<Future at 0x129b36520 state=pending>,
<Future at 0x129b36910 state=pending>,
<Future at 0x129b369d0 state=pending>,
<Future at 0x129b36640 state=pending>,
<Future at 0x129b36130 state=pending>,
<Future at 0x129b362e0 state=pending>,
<Future at 0x124d02580 state=pending>,
<Future at 0x129a757f0 state=pending>,
<Future at 0x129a750d0 state=pending>,
<Future at 0x129a75c10 state=pending>,
<Future at 0x129a754f0 state=pending>,
<Future at 0x129a75be0 state=pending>,
<Future at 0x129a75e50 state=pending>,
<Future at 0x129a87f40 state=pending>,
<Future at 0x129a874f0 state=pending>,
<Future at 0x129a87340 state=pending>,
<Future at 0x129a874c0 state=pending>,
<Future at 0x129a87070 state=pending>,
<Future at 0x129a87700 state=pending>,
<Future at 0x129a879d0 state=pending>,
<Future at 0x129a87b20 state=pending>,
<Future at 0x129a87f10 state=pending>,
<Future at 0x129a87910 state=pending>,
<Future at 0x129a87970 state=pending>,
<Future at 0x129a87610 state=pending>,
<Future at 0x124d6c490 state=pending>,
<Future at 0x1122cb100 state=pending>,
<Future at 0x1297e8580 state=pending>,
<Future at 0x1298b43a0 state=pending>,
<Future at 0x1298b4d60 state=pending>,
<Future at 0x1298b4a00 state=pending>,
<Future at 0x12983ca30 state=pending>,
<Future at 0x12983cc10 state=pending>,
<Future at 0x12983cdf0 state=pending>,
<Future at 0x12996de50 state=pending>,
<Future at 0x12996da00 state=pending>,
<Future at 0x12996d730 state=pending>,
<Future at 0x12996d100 state=pending>,
<Future at 0x1298c3d60 state=pending>,
<Future at 0x1298c3d30 state=pending>,
<Future at 0x1298c3df0 state=pending>,
<Future at 0x1298c3280 state=pending>,
<Future at 0x129776a90 state=pending>,
<Future at 0x128ffae20 state=pending>,
<Future at 0x129832250 state=pending>,
<Future at 0x129832be0 state=pending>,
<Future at 0x1297ba3d0 state=pending>,
<Future at 0x1297ba1f0 state=pending>,
<Future at 0x11c811e50 state=pending>,
<Future at 0x11c811ee0 state=pending>,
<Future at 0x1298c4400 state=pending>,
<Future at 0x1298c4430 state=pending>,
<Future at 0x1298c47c0 state=pending>,
<Future at 0x1298c47f0 state=pending>,
<Future at 0x1298c46a0 state=pending>,
<Future at 0x1298c4190 state=pending>,
<Future at 0x1298c43d0 state=pending>,
<Future at 0x129a82400 state=pending>,
<Future at 0x129a82e50 state=pending>,
<Future at 0x129a82310 state=pending>,
<Future at 0x129a823a0 state=pending>,
<Future at 0x129a82550 state=pending>,
<Future at 0x129a82280 state=pending>,
<Future at 0x129a82100 state=pending>,
<Future at 0x1298811c0 state=pending>,
<Future at 0x129881f10 state=pending>,
<Future at 0x1298818e0 state=pending>,
<Future at 0x129881e80 state=pending>,
<Future at 0x12981d610 state=pending>,
<Future at 0x12981dbb0 state=pending>,
<Future at 0x12981d4c0 state=pending>,
<Future at 0x129963f10 state=pending>,
<Future at 0x129963fa0 state=pending>,
<Future at 0x129963760 state=pending>,
<Future at 0x129963d90 state=pending>,
<Future at 0x129963130 state=pending>,
<Future at 0x129963eb0 state=pending>,
<Future at 0x129963e80 state=pending>,
<Future at 0x12987c8b0 state=pending>,
<Future at 0x12987c0a0 state=pending>,
<Future at 0x12987cb80 state=pending>,
<Future at 0x12987c730 state=pending>,
<Future at 0x12987c790 state=pending>,
<Future at 0x12987c8e0 state=pending>,
<Future at 0x129f59520 state=pending>,
<Future at 0x129f595b0 state=pending>,
<Future at 0x129f59700 state=pending>,
<Future at 0x129f59790 state=pending>,
<Future at 0x129f598e0 state=pending>,
<Future at 0x129f59970 state=pending>,
<Future at 0x129f59ac0 state=pending>,
<Future at 0x129f59b50 state=pending>,
<Future at 0x129f59ca0 state=pending>,
<Future at 0x129f59d30 state=pending>,
<Future at 0x129f59e80 state=pending>,
<Future at 0x129f59f10 state=pending>,
<Future at 0x129b57040 state=pending>,
<Future at 0x129b57820 state=pending>,
<Future at 0x129b57400 state=pending>,
<Future at 0x129b57b50 state=pending>,
<Future at 0x129b57550 state=pending>,
<Future at 0x129b57130 state=pending>,
<Future at 0x129b575b0 state=pending>,
<Future at 0x129b57a30 state=pending>,
<Future at 0x129b57e50 state=pending>,
<Future at 0x129b572e0 state=pending>,
<Future at 0x129b51b50 state=pending>,
<Future at 0x129b51790 state=pending>,
<Future at 0x129b51af0 state=pending>,
<Future at 0x129b51ee0 state=pending>,
<Future at 0x129b519d0 state=pending>,
<Future at 0x129b51640 state=pending>,
<Future at 0x129b51d30 state=pending>,
<Future at 0x129b51be0 state=pending>,
<Future at 0x129b51ca0 state=pending>,
<Future at 0x129b51ac0 state=pending>,
<Future at 0x129b517f0 state=pending>,
<Future at 0x129d31040 state=pending>,
<Future at 0x129d317c0 state=pending>,
<Future at 0x129d318e0 state=pending>,
<Future at 0x129d314c0 state=pending>,
<Future at 0x129d319d0 state=pending>,
<Future at 0x129d31070 state=pending>,
<Future at 0x129d31460 state=pending>,
<Future at 0x129d31b20 state=pending>,
<Future at 0x129d31ee0 state=pending>,
<Future at 0x129d31b80 state=pending>,
<Future at 0x129d31d30 state=pending>,
<Future at 0x129d23a90 state=pending>,
<Future at 0x129d23040 state=pending>,
<Future at 0x129d23160 state=pending>,
<Future at 0x129d23ee0 state=pending>,
<Future at 0x129d23700 state=pending>,
<Future at 0x129d23790 state=pending>,
<Future at 0x129d23730 state=pending>,
<Future at 0x129d23bb0 state=pending>,
<Future at 0x129d23dc0 state=pending>,
<Future at 0x129d231f0 state=pending>,
<Future at 0x129d23af0 state=pending>,
<Future at 0x129c78580 state=pending>,
<Future at 0x129c78070 state=pending>,
<Future at 0x129c782b0 state=pending>,
<Future at 0x129c78220 state=pending>,
<Future at 0x129c784f0 state=pending>,
<Future at 0x129c788e0 state=pending>,
<Future at 0x129c78d30 state=pending>,
<Future at 0x129c78b20 state=pending>,
<Future at 0x129c78e50 state=pending>,
<Future at 0x129c78b80 state=pending>,
<Future at 0x129c78970 state=pending>,
<Future at 0x129b4d400 state=pending>,
<Future at 0x129b4d310 state=pending>,
<Future at 0x129b4d640 state=pending>,
<Future at 0x129b4d7f0 state=pending>,
<Future at 0x129b4deb0 state=pending>,
<Future at 0x129b4da60 state=pending>,
<Future at 0x129b4d520 state=pending>,
<Future at 0x129b4df40 state=pending>,
<Future at 0x129b4dfd0 state=pending>,
<Future at 0x129b4d940 state=pending>,
<Future at 0x129b4ddc0 state=pending>,
<Future at 0x129d37580 state=pending>,
<Future at 0x129d37070 state=pending>,
<Future at 0x129d37340 state=pending>,
<Future at 0x129d37220 state=pending>,
<Future at 0x129d373d0 state=pending>,
<Future at 0x129d376a0 state=pending>,
<Future at 0x129d378e0 state=pending>,
<Future at 0x129d37a60 state=pending>,
<Future at 0x129d37bb0 state=pending>,
<Future at 0x129d37d30 state=pending>,
<Future at 0x129d37e80 state=pending>,
<Future at 0x129d37820 state=pending>,
<Future at 0x129d675e0 state=pending>,
<Future at 0x129d67190 state=pending>,
<Future at 0x129d67280 state=pending>,
<Future at 0x129d67460 state=pending>,
<Future at 0x129d67550 state=pending>,
<Future at 0x129d67bb0 state=pending>,
<Future at 0x129d67e80 state=pending>,
<Future at 0x129d67a00 state=pending>,
<Future at 0x129d67c10 state=pending>,
<Future at 0x129d67a60 state=pending>,
<Future at 0x129d67790 state=pending>,
<Future at 0x129d610d0 state=pending>,
<Future at 0x129d61220 state=pending>,
<Future at 0x129d613a0 state=pending>,
<Future at 0x129d614f0 state=pending>,
<Future at 0x129d61670 state=pending>,
<Future at 0x129d617c0 state=pending>,
<Future at 0x129d61940 state=pending>,
<Future at 0x129d61a90 state=pending>,
<Future at 0x129d61c10 state=pending>,
<Future at 0x129d61d60 state=pending>,
<Future at 0x129d61ee0 state=pending>,
<Future at 0x129d610a0 state=pending>,
<Future at 0x129d8e280 state=pending>,
<Future at 0x129d8e310 state=pending>,
<Future at 0x129d8e460 state=pending>,
<Future at 0x129d8e4f0 state=pending>,
<Future at 0x129d8e640 state=pending>,
<Future at 0x129d8e6d0 state=pending>,
<Future at 0x129d8e820 state=pending>,
<Future at 0x129d8e8b0 state=pending>,
<Future at 0x129d8ea00 state=pending>,
<Future at 0x129d8ea90 state=pending>,
<Future at 0x129d8ebe0 state=pending>,
<Future at 0x129d8ec70 state=pending>,
<Future at 0x129d8edc0 state=pending>,
<Future at 0x129d8ee50 state=pending>,
<Future at 0x129d8efa0 state=pending>,
<Future at 0x129d8e130 state=pending>,
<Future at 0x129d3c100 state=pending>,
<Future at 0x129d3c280 state=pending>,
<Future at 0x129d3c3d0 state=pending>,
<Future at 0x129d3c550 state=pending>,
<Future at 0x129d3c6a0 state=pending>,
<Future at 0x129d3c820 state=pending>,
<Future at 0x129d3c970 state=pending>,
<Future at 0x129d3caf0 state=pending>,
<Future at 0x129d3cc40 state=pending>,
<Future at 0x129d3cdc0 state=pending>,
<Future at 0x129d3cf10 state=pending>,
<Future at 0x129d3ceb0 state=pending>,
<Future at 0x129d3cbe0 state=pending>,
<Future at 0x129d3c910 state=pending>,
<Future at 0x129d3c5b0 state=pending>,
<Future at 0x129d28040 state=pending>,
<Future at 0x129d28ca0 state=pending>,
<Future at 0x129d28070 state=pending>,
<Future at 0x129d289d0 state=pending>,
<Future at 0x129d281f0 state=pending>,
<Future at 0x129d28dc0 state=pending>,
<Future at 0x125014490 state=pending>,
<Future at 0x127530be0 state=pending>,
<Future at 0x129b6afd0 state=pending>,
<Future at 0x129b3a1f0 state=pending>,
<Future at 0x129b3a7f0 state=pending>,
<Future at 0x129b3a940 state=pending>,
<Future at 0x129b3a640 state=pending>,
<Future at 0x129b3a490 state=pending>,
<Future at 0x129b3ad00 state=pending>,
<Future at 0x129b3afd0 state=pending>,
<Future at 0x129b3a790 state=pending>,
<Future at 0x129b3adc0 state=pending>,
<Future at 0x129b3a910 state=pending>,
<Future at 0x129b3a730 state=pending>,
<Future at 0x129b3ed00 state=pending>,
<Future at 0x129b3eb80 state=pending>,
<Future at 0x129b3e910 state=pending>,
<Future at 0x129b3ee50 state=pending>,
<Future at 0x129b3ec10 state=pending>,
<Future at 0x129b3e400 state=pending>,
<Future at 0x129b3e3d0 state=pending>,
<Future at 0x129b3e250 state=pending>,
<Future at 0x129b3e370 state=pending>,
<Future at 0x129b3e760 state=pending>,
<Future at 0x129b3e130 state=pending>,
<Future at 0x129b3e0d0 state=pending>,
<Future at 0x129b48be0 state=pending>,
<Future at 0x129b48c70 state=pending>,
<Future at 0x129b48040 state=pending>,
<Future at 0x129b48d30 state=pending>,
<Future at 0x129b485e0 state=pending>,
<Future at 0x129b48eb0 state=pending>,
<Future at 0x129b48700 state=pending>,
<Future at 0x129b48910 state=pending>,
<Future at 0x129b487f0 state=pending>,
<Future at 0x129b481c0 state=pending>,
<Future at 0x129b48e20 state=pending>,
<Future at 0x129d1cf10 state=pending>,
<Future at 0x129d1cd00 state=pending>,
<Future at 0x129d1c7c0 state=pending>,
<Future at 0x129d1c610 state=pending>,
<Future at 0x129d1c3a0 state=pending>,
<Future at 0x129d1cac0 state=pending>,
<Future at 0x129d1c100 state=pending>,
<Future at 0x129d1cd60 state=pending>,
<Future at 0x129d1c640 state=pending>,
<Future at 0x129f71070 state=pending>,
<Future at 0x129f711c0 state=pending>,
<Future at 0x129f71fa0 state=pending>,
<Future at 0x129d2d280 state=pending>,
<Future at 0x129d2d3d0 state=pending>,
<Future at 0x129d2d460 state=pending>,
<Future at 0x129d2d5b0 state=pending>,
<Future at 0x129d2d640 state=pending>,
<Future at 0x129d2d790 state=pending>,
<Future at 0x129d2d820 state=pending>,
<Future at 0x129d2d970 state=pending>,
<Future at 0x129d2da00 state=pending>,
<Future at 0x129d2db50 state=pending>,
<Future at 0x129d2dbe0 state=pending>,
<Future at 0x129d2dd30 state=pending>,
<Future at 0x129d2ddc0 state=pending>,
<Future at 0x129d2df10 state=pending>,
<Future at 0x129d2dfa0 state=pending>,
<Future at 0x129e06880 state=pending>,
<Future at 0x129e068b0 state=pending>,
<Future at 0x129e06a60 state=pending>,
<Future at 0x129e06a90 state=pending>,
<Future at 0x129e06c40 state=pending>,
<Future at 0x129e06c70 state=pending>,
<Future at 0x129e06640 state=pending>,
<Future at 0x129e06220 state=pending>,
<Future at 0x129e062e0 state=pending>,
<Future at 0x129e06370 state=pending>,
<Future at 0x129e064c0 state=pending>,
<Future at 0x129e06550 state=pending>,
<Future at 0x129e06070 state=pending>,
<Future at 0x129e06040 state=pending>,
<Future at 0x129e06760 state=pending>,
<Future at 0x129b77be0 state=pending>,
<Future at 0x129b77e20 state=pending>,
<Future at 0x129b77af0 state=pending>,
<Future at 0x12a3bf850 state=pending>,
<Future at 0x12a3bf910 state=pending>,
<Future at 0x12a3bfa00 state=pending>,
<Future at 0x12a3bfaf0 state=pending>,
<Future at 0x12a3bfbe0 state=pending>,
<Future at 0x12a3bfcd0 state=pending>,
<Future at 0x12a3bfdc0 state=pending>,
<Future at 0x12a3bfeb0 state=pending>,
<Future at 0x12a3bffa0 state=pending>,
<Future at 0x12a3d90d0 state=pending>,
<Future at 0x12a3d91c0 state=pending>,
<Future at 0x12a3d92b0 state=pending>,
<Future at 0x12a3d93a0 state=pending>,
<Future at 0x12a3d9490 state=pending>,
<Future at 0x12a3d9580 state=pending>,
<Future at 0x12a3d9670 state=pending>,
<Future at 0x12a3d9760 state=pending>,
<Future at 0x12a3d9850 state=pending>,
<Future at 0x12a3d9940 state=pending>,
<Future at 0x12a3d9a30 state=pending>,
<Future at 0x12a3d9b20 state=pending>,
<Future at 0x12a3d9c10 state=pending>,
<Future at 0x12a3d9d00 state=pending>,
<Future at 0x12a3d9df0 state=pending>,
<Future at 0x12a3d9ee0 state=pending>,
<Future at 0x12a3d9fd0 state=pending>,
<Future at 0x12a3de100 state=pending>,
<Future at 0x12a3de1f0 state=pending>,
<Future at 0x12a3de2e0 state=pending>,
<Future at 0x12a3de3d0 state=pending>,
<Future at 0x12a3de4c0 state=pending>,
<Future at 0x12a3de5b0 state=pending>,
<Future at 0x12a3de6a0 state=pending>,
<Future at 0x12a3de790 state=pending>,
<Future at 0x12a3de880 state=pending>,
<Future at 0x12a3de970 state=pending>,
<Future at 0x12a3dea60 state=pending>,
<Future at 0x12a3deb50 state=pending>,
<Future at 0x12a3dec40 state=pending>,
<Future at 0x12a3ded30 state=pending>,
<Future at 0x12a3dee20 state=pending>,
<Future at 0x12a3def10 state=pending>,
<Future at 0x12a3e3040 state=pending>,
<Future at 0x12a3e3130 state=pending>,
<Future at 0x12a3e3220 state=pending>,
<Future at 0x12a3e3310 state=pending>,
<Future at 0x12a3e3400 state=pending>,
<Future at 0x12a3e34f0 state=pending>,
<Future at 0x12a3e35e0 state=pending>,
<Future at 0x12a3e36d0 state=pending>,
<Future at 0x12a3e37c0 state=pending>,
<Future at 0x12a3e38b0 state=pending>,
<Future at 0x12a3e39a0 state=pending>,
<Future at 0x12a3e3a90 state=pending>,
<Future at 0x12a3e3b80 state=pending>,
<Future at 0x12a3e3c70 state=pending>,
<Future at 0x12a3e3d60 state=pending>,
<Future at 0x12a3e3e50 state=pending>,
<Future at 0x12a3e3f40 state=pending>,
<Future at 0x12a3e7070 state=pending>,
<Future at 0x12a3e7160 state=pending>,
<Future at 0x12a3e7250 state=pending>,
<Future at 0x12a3e7340 state=pending>,
<Future at 0x12a3e7430 state=pending>,
<Future at 0x12a3e7520 state=pending>,
<Future at 0x12a3e7610 state=pending>,
<Future at 0x12a3e7700 state=pending>,
<Future at 0x12a3e77f0 state=pending>,
<Future at 0x12a3e78e0 state=pending>,
<Future at 0x12a3e79d0 state=pending>,
<Future at 0x12a3e7ac0 state=pending>,
<Future at 0x12a3e7bb0 state=pending>,
<Future at 0x12a3e7ca0 state=pending>,
<Future at 0x12a3e7d90 state=pending>,
<Future at 0x12a3e7e80 state=pending>,
<Future at 0x12a3e7f70 state=pending>,
<Future at 0x12a3ec0a0 state=pending>,
<Future at 0x12a3ec190 state=pending>,
<Future at 0x12a3ec280 state=pending>,
<Future at 0x12a3ec370 state=pending>,
<Future at 0x12a3ec460 state=pending>,
<Future at 0x12a3ec550 state=pending>,
<Future at 0x12a3ec640 state=pending>,
<Future at 0x12a3ec730 state=pending>,
<Future at 0x12a3ec820 state=pending>,
<Future at 0x12a3ec910 state=pending>,
<Future at 0x12a3eca00 state=pending>,
<Future at 0x12a3ecaf0 state=pending>,
<Future at 0x12a3ecbe0 state=pending>,
<Future at 0x12a3eccd0 state=pending>,
<Future at 0x12a3ecdc0 state=pending>,
<Future at 0x12a3eceb0 state=pending>,
<Future at 0x12a3ecfa0 state=pending>,
<Future at 0x12a3f10d0 state=pending>,
<Future at 0x12a3f11c0 state=pending>,
<Future at 0x12a3f12b0 state=pending>,
<Future at 0x12a3f13a0 state=pending>,
<Future at 0x12a3f1490 state=pending>,
<Future at 0x12a3f1580 state=pending>,
<Future at 0x12a3f1670 state=pending>,
<Future at 0x12a3f1760 state=pending>,
<Future at 0x12a3f1850 state=pending>,
<Future at 0x12a3f1940 state=pending>,
<Future at 0x12a3f1a30 state=pending>,
<Future at 0x12a3f1b20 state=pending>,
<Future at 0x12a3f1c10 state=pending>,
<Future at 0x12a3f1d00 state=pending>,
<Future at 0x12a3f1df0 state=pending>,
<Future at 0x12a3f1ee0 state=pending>,
<Future at 0x12a3f1fd0 state=pending>,
<Future at 0x12a3f5100 state=pending>,
<Future at 0x12a3f51f0 state=pending>,
<Future at 0x12a3f52e0 state=pending>,
<Future at 0x12a3f53d0 state=pending>,
<Future at 0x12a3f54c0 state=pending>,
<Future at 0x12a3f55b0 state=pending>,
<Future at 0x12a3f56a0 state=pending>,
<Future at 0x12a3f5790 state=pending>,
<Future at 0x12a3f5880 state=pending>,
<Future at 0x12a3f5970 state=pending>,
<Future at 0x12a3f5a60 state=pending>,
<Future at 0x12a3f5b50 state=pending>,
<Future at 0x12a3f5c40 state=pending>,
<Future at 0x12a3f5d30 state=pending>,
<Future at 0x12a3f5e20 state=pending>,
<Future at 0x12a3f5f10 state=pending>,
<Future at 0x12a3fb040 state=pending>,
<Future at 0x12a3fb130 state=pending>,
<Future at 0x12a3fb220 state=pending>,
<Future at 0x12a3fb310 state=pending>,
<Future at 0x12a3fb400 state=pending>,
<Future at 0x12a3fb4f0 state=pending>,
<Future at 0x12a3fb5e0 state=pending>,
<Future at 0x12a3fb6d0 state=pending>,
<Future at 0x12a3fb7c0 state=pending>,
<Future at 0x12a3fb8b0 state=pending>,
<Future at 0x12a3fb9a0 state=pending>,
<Future at 0x12a3fba90 state=pending>,
<Future at 0x12a3fbb80 state=pending>,
<Future at 0x12a3fbc70 state=pending>,
<Future at 0x12a3fbd60 state=pending>,
<Future at 0x12a3fbe50 state=pending>,
<Future at 0x12a3fbf40 state=pending>,
<Future at 0x12a3ff070 state=pending>,
<Future at 0x12a3ff160 state=pending>,
<Future at 0x12a3ff250 state=pending>,
<Future at 0x12a3ff340 state=pending>,
<Future at 0x12a3ff430 state=pending>,
<Future at 0x12a3ff520 state=pending>,
<Future at 0x12a3ff610 state=pending>,
<Future at 0x12a3ff700 state=pending>,
<Future at 0x12a3ff7f0 state=pending>,
<Future at 0x12a3ff8e0 state=pending>,
<Future at 0x12a3ff9d0 state=pending>,
<Future at 0x12a3ffac0 state=pending>,
<Future at 0x12a3ffbb0 state=pending>,
<Future at 0x12a3ffca0 state=pending>,
<Future at 0x12a3ffd90 state=pending>,
<Future at 0x12a3ffe80 state=pending>,
<Future at 0x12a3fff70 state=pending>,
<Future at 0x12a005070 state=pending>,
<Future at 0x12a0051c0 state=pending>,
<Future at 0x12a005250 state=pending>,
<Future at 0x12a0053a0 state=pending>,
<Future at 0x12a005430 state=pending>,
<Future at 0x12a005580 state=pending>,
<Future at 0x12a005610 state=pending>,
<Future at 0x12a005760 state=pending>,
<Future at 0x12a0057f0 state=pending>,
<Future at 0x12a005940 state=pending>,
<Future at 0x12a0059d0 state=pending>,
<Future at 0x12a005b20 state=pending>,
<Future at 0x12a005bb0 state=pending>,
<Future at 0x12a005d00 state=pending>,
<Future at 0x12a005d90 state=pending>,
<Future at 0x12a005ee0 state=pending>,
<Future at 0x12a005f70 state=pending>,
<Future at 0x12a00c100 state=pending>,
<Future at 0x12a00c190 state=pending>,
<Future at 0x12a00c2e0 state=pending>,
<Future at 0x12a00c370 state=pending>,
<Future at 0x12a00c4c0 state=pending>,
<Future at 0x12a00c550 state=pending>,
<Future at 0x12a00c6a0 state=pending>,
<Future at 0x12a00c730 state=pending>,
<Future at 0x12a00c880 state=pending>,
<Future at 0x12a00c910 state=pending>,
<Future at 0x12a00ca60 state=pending>,
<Future at 0x12a00caf0 state=pending>,
<Future at 0x12a00cc40 state=pending>,
<Future at 0x12a00ccd0 state=pending>,
<Future at 0x12a00ce20 state=pending>,
<Future at 0x12a00ceb0 state=pending>,
<Future at 0x12a00cfd0 state=pending>,
<Future at 0x12a014100 state=pending>,
<Future at 0x12a0141f0 state=pending>,
<Future at 0x12a0142e0 state=pending>,
<Future at 0x12a0143d0 state=pending>,
<Future at 0x12a0144c0 state=pending>,
<Future at 0x12a0145b0 state=pending>,
<Future at 0x12a0146a0 state=pending>,
<Future at 0x12a014790 state=pending>,
<Future at 0x12a014880 state=pending>,
<Future at 0x12a014970 state=pending>,
<Future at 0x12a014a60 state=pending>,
<Future at 0x12a014b50 state=pending>,
<Future at 0x12a014c40 state=pending>,
<Future at 0x12a014d30 state=pending>,
<Future at 0x12a014e20 state=pending>,
<Future at 0x12a014f10 state=pending>,
<Future at 0x12a02c040 state=pending>,
<Future at 0x12a02c130 state=pending>,
<Future at 0x12a02c220 state=pending>,
<Future at 0x12a02c310 state=pending>,
<Future at 0x12a02c400 state=pending>,
<Future at 0x12a02c4f0 state=pending>,
<Future at 0x12a02c5e0 state=pending>,
<Future at 0x12a02c6d0 state=pending>,
<Future at 0x12a02c7c0 state=pending>,
<Future at 0x12a02c8b0 state=pending>,
<Future at 0x12a02c9a0 state=pending>,
<Future at 0x12a02ca90 state=pending>,
<Future at 0x12a02cb80 state=pending>,
<Future at 0x12a02cc70 state=pending>,
<Future at 0x12a02cd60 state=pending>,
<Future at 0x12a02ce50 state=pending>,
<Future at 0x12a02cf40 state=pending>,
<Future at 0x12a032070 state=pending>,
<Future at 0x12a032160 state=pending>,
<Future at 0x12a032250 state=pending>,
<Future at 0x12a032340 state=pending>,
<Future at 0x12a032430 state=pending>,
<Future at 0x12a032520 state=pending>,
<Future at 0x12a032610 state=pending>,
<Future at 0x12a032700 state=pending>,
<Future at 0x12a0327f0 state=pending>,
<Future at 0x12a0328e0 state=pending>,
<Future at 0x12a0329d0 state=pending>,
<Future at 0x12a032ac0 state=pending>,
<Future at 0x12a032bb0 state=pending>,
<Future at 0x12a032ca0 state=pending>,
<Future at 0x12a032d90 state=pending>,
<Future at 0x12a032e80 state=pending>,
<Future at 0x12a032f70 state=pending>,
<Future at 0x12a0390a0 state=pending>,
<Future at 0x12a039190 state=pending>,
<Future at 0x12a039280 state=pending>,
<Future at 0x12a039370 state=pending>,
<Future at 0x12a039460 state=pending>,
<Future at 0x12a039550 state=pending>,
<Future at 0x12a039640 state=pending>,
<Future at 0x12a039730 state=pending>,
<Future at 0x12a039820 state=pending>,
<Future at 0x12a039910 state=pending>,
<Future at 0x12a039a00 state=pending>,
<Future at 0x12a039af0 state=pending>,
<Future at 0x12a039be0 state=pending>,
<Future at 0x12a039cd0 state=pending>,
<Future at 0x12a039dc0 state=pending>,
<Future at 0x12a039eb0 state=pending>,
<Future at 0x12a039fa0 state=pending>,
<Future at 0x12a0400d0 state=pending>,
<Future at 0x12a0401c0 state=pending>,
<Future at 0x12a0402b0 state=pending>,
<Future at 0x12a0403a0 state=pending>,
<Future at 0x12a040490 state=pending>,
<Future at 0x12a040580 state=pending>,
<Future at 0x12a040670 state=pending>,
<Future at 0x12a040760 state=pending>,
<Future at 0x12a040850 state=pending>,
<Future at 0x12a040940 state=pending>,
<Future at 0x12a040a30 state=pending>,
<Future at 0x12a040b20 state=pending>,
<Future at 0x12a040c10 state=pending>,
<Future at 0x12a040d00 state=pending>,
<Future at 0x12a040df0 state=pending>,
<Future at 0x12a040ee0 state=pending>,
<Future at 0x12a040fd0 state=pending>,
<Future at 0x129fe5100 state=pending>,
<Future at 0x129fe51f0 state=pending>,
<Future at 0x129fe52e0 state=pending>,
<Future at 0x129fe53d0 state=pending>,
<Future at 0x129fe54c0 state=pending>,
<Future at 0x129fe55b0 state=pending>,
<Future at 0x129fe56a0 state=pending>,
<Future at 0x129fe5790 state=pending>,
<Future at 0x129fe5880 state=pending>,
<Future at 0x129fe5970 state=pending>,
<Future at 0x129fe5a60 state=pending>,
<Future at 0x129fe5b50 state=pending>,
<Future at 0x129fe5c40 state=pending>,
<Future at 0x129fe5d30 state=pending>,
<Future at 0x129fe5e20 state=pending>,
<Future at 0x129fe5f10 state=pending>,
<Future at 0x129fcc040 state=pending>,
<Future at 0x129fcc130 state=pending>,
<Future at 0x129fcc220 state=pending>,
<Future at 0x129fcc310 state=pending>,
<Future at 0x129fcc400 state=pending>,
<Future at 0x129fcc4f0 state=pending>,
<Future at 0x129fcc5e0 state=pending>,
<Future at 0x129fcc6d0 state=pending>,
<Future at 0x129fcc7c0 state=pending>,
<Future at 0x129fcc8b0 state=pending>,
<Future at 0x129fcc9a0 state=pending>,
<Future at 0x129fcca90 state=pending>,
<Future at 0x129fccb80 state=pending>,
<Future at 0x129fccc70 state=pending>,
<Future at 0x129fccd60 state=pending>,
<Future at 0x129fcce50 state=pending>,
<Future at 0x129fccf40 state=pending>,
<Future at 0x129fd1070 state=pending>,
<Future at 0x129fd1160 state=pending>,
<Future at 0x129fd1250 state=pending>,
<Future at 0x129fd1340 state=pending>,
<Future at 0x129fd1430 state=pending>,
<Future at 0x129fd1520 state=pending>,
<Future at 0x129fd1610 state=pending>,
<Future at 0x129fd1700 state=pending>,
<Future at 0x129fd17f0 state=pending>,
<Future at 0x129fd18e0 state=pending>,
<Future at 0x129fd19d0 state=pending>,
<Future at 0x129fd1ac0 state=pending>,
<Future at 0x129fd1bb0 state=pending>,
<Future at 0x129fd1ca0 state=pending>,
<Future at 0x129fd1d90 state=pending>,
<Future at 0x129fd1e80 state=pending>,
<Future at 0x129fd1f70 state=pending>,
<Future at 0x129fef0a0 state=pending>,
<Future at 0x129fef190 state=pending>,
<Future at 0x129fef280 state=pending>,
<Future at 0x129fef370 state=pending>,
<Future at 0x129fef460 state=pending>,
<Future at 0x129fef550 state=pending>,
<Future at 0x129fef640 state=pending>,
<Future at 0x129fef730 state=pending>,
<Future at 0x129fef820 state=pending>,
<Future at 0x129fef910 state=pending>,
<Future at 0x129fefa00 state=pending>,
<Future at 0x129fefaf0 state=pending>,
<Future at 0x129fefbe0 state=pending>,
<Future at 0x129fefcd0 state=pending>,
<Future at 0x129fefdc0 state=pending>,
<Future at 0x129fefeb0 state=pending>,
<Future at 0x129feffa0 state=pending>,
<Future at 0x129ff40d0 state=pending>,
<Future at 0x129ff41c0 state=pending>,
<Future at 0x129ff42b0 state=pending>,
<Future at 0x129ff43a0 state=pending>,
<Future at 0x129ff4490 state=pending>,
<Future at 0x129ff4580 state=pending>,
<Future at 0x129ff4670 state=pending>,
<Future at 0x129ff4760 state=pending>,
<Future at 0x129ff4850 state=pending>,
<Future at 0x129ff4940 state=pending>,
<Future at 0x129ff4a30 state=pending>,
<Future at 0x129ff4b20 state=pending>,
<Future at 0x129ff4c10 state=pending>,
<Future at 0x129ff4d00 state=pending>,
<Future at 0x129ff4df0 state=pending>,
<Future at 0x129ff4ee0 state=pending>,
<Future at 0x129ff4fd0 state=pending>,
<Future at 0x129ffb100 state=pending>,
<Future at 0x129ffb1f0 state=pending>,
<Future at 0x129ffb2e0 state=pending>,
<Future at 0x129ffb3d0 state=pending>,
<Future at 0x129ffb4c0 state=pending>,
<Future at 0x129ffb5b0 state=pending>,
<Future at 0x129ffb6a0 state=pending>,
<Future at 0x129ffb790 state=pending>,
<Future at 0x129ffb880 state=pending>,
<Future at 0x129ffb970 state=pending>,
<Future at 0x129ffba60 state=pending>,
<Future at 0x129ffbb50 state=pending>,
<Future at 0x129ffbc40 state=pending>,
<Future at 0x129ffbd30 state=pending>,
<Future at 0x129ffbe20 state=pending>,
<Future at 0x129ffbf10 state=pending>,
<Future at 0x12a002040 state=pending>,
<Future at 0x12a002130 state=pending>,
<Future at 0x12a002220 state=pending>,
<Future at 0x12a002310 state=pending>,
<Future at 0x12a002400 state=pending>,
<Future at 0x12a0024f0 state=pending>,
<Future at 0x12a0025e0 state=pending>,
<Future at 0x12a0026d0 state=pending>,
<Future at 0x12a0027c0 state=pending>,
<Future at 0x12a0028b0 state=pending>,
<Future at 0x12a0029a0 state=pending>,
<Future at 0x12a002a90 state=pending>,
<Future at 0x12a002b80 state=pending>,
<Future at 0x12a002c70 state=pending>,
<Future at 0x12a002d60 state=pending>,
<Future at 0x12a002e50 state=pending>,
<Future at 0x12a002f40 state=pending>,
<Future at 0x129e4f070 state=pending>,
<Future at 0x129e4f160 state=pending>,
<Future at 0x129e4f250 state=pending>,
<Future at 0x129e4f340 state=pending>,
<Future at 0x129e4f430 state=pending>,
<Future at 0x129e4f520 state=pending>,
<Future at 0x129e4f610 state=pending>,
<Future at 0x129e4f700 state=pending>,
<Future at 0x129e4f7f0 state=pending>,
<Future at 0x129e4f8e0 state=pending>,
<Future at 0x129e4f9d0 state=pending>,
<Future at 0x129e4fac0 state=pending>,
<Future at 0x129e4fbb0 state=pending>,
<Future at 0x129e4fca0 state=pending>,
<Future at 0x129e4fd90 state=pending>,
<Future at 0x129e4fe80 state=pending>,
<Future at 0x129e4ff70 state=pending>,
<Future at 0x129e4a0a0 state=pending>,
<Future at 0x129e4a190 state=pending>,
<Future at 0x129e4a280 state=pending>,
<Future at 0x129e4a370 state=pending>,
<Future at 0x129e4a460 state=pending>,
<Future at 0x129e4a550 state=pending>,
<Future at 0x129e4a640 state=pending>,
<Future at 0x129e4a730 state=pending>,
<Future at 0x129e4a820 state=pending>,
<Future at 0x129e4a910 state=pending>,
<Future at 0x129e4aa00 state=pending>,
<Future at 0x129e4aaf0 state=pending>,
<Future at 0x129e4abe0 state=pending>,
<Future at 0x129e4acd0 state=pending>,
<Future at 0x129e4adc0 state=pending>,
<Future at 0x129e4aeb0 state=pending>,
<Future at 0x129e4afa0 state=pending>,
<Future at 0x129e5c0d0 state=pending>,
<Future at 0x129e5c1c0 state=pending>,
<Future at 0x129e5c2b0 state=pending>,
<Future at 0x129e5c3a0 state=pending>,
...]
[111]:
print(statuses.progress())
statuses.actions
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-111-45e45d9aa8bc> in <module>
----> 1 print(statuses.progress())
2 statuses.actions
~/Programming/EasyVVUQ/easyvvuq/actions/action_statuses.py in progress(self)
86 running += 1
87 elif action.done():
---> 88 if not action.result():
89 failed += 1
90 else:
~/miniconda3/envs/easyvvuq/lib/python3.8/concurrent/futures/_base.py in result(self, timeout)
430 raise CancelledError()
431 elif self._state == FINISHED:
--> 432 return self.__get_result()
433
434 self._condition.wait(timeout)
~/miniconda3/envs/easyvvuq/lib/python3.8/concurrent/futures/_base.py in __get_result(self)
386 def __get_result(self):
387 if self._exception:
--> 388 raise self._exception
389 else:
390 return self._result
~/miniconda3/envs/easyvvuq/lib/python3.8/concurrent/futures/thread.py in run(self)
55
56 try:
---> 57 result = self.fn(*self.args, **self.kwargs)
58 except BaseException as exc:
59 self.future.set_exception(exc)
~/Programming/EasyVVUQ/easyvvuq/actions/action_statuses.py in job_handler(self, status)
52 ActionStatus of an action to be executed.
53 """
---> 54 status.start()
55 while not status.finished():
56 time.sleep(self.poll_sleep_time)
~/Programming/EasyVVUQ/easyvvuq/actions/execute_kubernetes.py in start(self)
89 if self.started():
90 raise RuntimeError('The pod has already started!')
---> 91 self.create_config_maps(self.config_names)
92 self.create_volumes(self.config_names, self.body)
93 self.core_v1.create_namespaced_pod(body=self.body, namespace="default")
~/Programming/EasyVVUQ/easyvvuq/actions/execute_kubernetes.py in create_config_maps(self, file_names)
162 metadata=metadata
163 )
--> 164 self.core_v1.create_namespaced_config_map(namespace='default', body=configmap)
165
166
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py in create_namespaced_config_map(self, namespace, body, **kwargs)
6628 """
6629 kwargs['_return_http_data_only'] = True
-> 6630 return self.create_namespaced_config_map_with_http_info(namespace, body, **kwargs) # noqa: E501
6631
6632 def create_namespaced_config_map_with_http_info(self, namespace, body, **kwargs): # noqa: E501
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/api/core_v1_api.py in create_namespaced_config_map_with_http_info(self, namespace, body, **kwargs)
6723 auth_settings = ['BearerToken'] # noqa: E501
6724
-> 6725 return self.api_client.call_api(
6726 '/api/v1/namespaces/{namespace}/configmaps', 'POST',
6727 path_params,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/api_client.py in call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, async_req, _return_http_data_only, collection_formats, _preload_content, _request_timeout, _host)
346 """
347 if not async_req:
--> 348 return self.__call_api(resource_path, method,
349 path_params, query_params, header_params,
350 body, post_params, files,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/api_client.py in __call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout, _host)
178
179 # perform request and return response
--> 180 response_data = self.request(
181 method, url, query_params=query_params, headers=header_params,
182 post_params=post_params, body=body,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/api_client.py in request(self, method, url, query_params, headers, post_params, body, _preload_content, _request_timeout)
389 _request_timeout=_request_timeout)
390 elif method == "POST":
--> 391 return self.rest_client.POST(url,
392 query_params=query_params,
393 headers=headers,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/rest.py in POST(self, url, headers, query_params, post_params, body, _preload_content, _request_timeout)
272 def POST(self, url, headers=None, query_params=None, post_params=None,
273 body=None, _preload_content=True, _request_timeout=None):
--> 274 return self.request("POST", url,
275 headers=headers,
276 query_params=query_params,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/kubernetes/client/rest.py in request(self, method, url, query_params, headers, body, post_params, _preload_content, _request_timeout)
165 if body is not None:
166 request_body = json.dumps(body)
--> 167 r = self.pool_manager.request(
168 method, url,
169 body=request_body,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/request.py in request(self, method, url, fields, headers, **urlopen_kw)
76 )
77 else:
---> 78 return self.request_encode_body(
79 method, url, fields=fields, headers=headers, **urlopen_kw
80 )
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/request.py in request_encode_body(self, method, url, fields, headers, encode_multipart, multipart_boundary, **urlopen_kw)
168 extra_kw.update(urlopen_kw)
169
--> 170 return self.urlopen(method, url, **extra_kw)
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/poolmanager.py in urlopen(self, method, url, redirect, **kw)
373 response = conn.urlopen(method, url, **kw)
374 else:
--> 375 response = conn.urlopen(method, u.request_uri, **kw)
376
377 redirect_location = redirect and response.get_redirect_location()
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
686 # Request a connection from the queue.
687 timeout_obj = self._get_timeout(timeout)
--> 688 conn = self._get_conn(timeout=pool_timeout)
689
690 conn.timeout = timeout_obj.connect_timeout
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/connectionpool.py in _get_conn(self, timeout)
278 conn = None
279
--> 280 return conn or self._new_conn()
281
282 def _put_conn(self, conn):
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/connectionpool.py in _new_conn(self)
232 )
233
--> 234 conn = self.ConnectionCls(
235 host=self.host,
236 port=self.port,
~/miniconda3/envs/easyvvuq/lib/python3.8/site-packages/urllib3/connection.py in __init__(self, *args, **kw)
123 self.proxy_config = kw.pop("proxy_config", None)
124
--> 125 _HTTPConnection.__init__(self, *args, **kw)
126
127 @property
TypeError: __init__() got an unexpected keyword argument 'assert_hostname'
[120]:
campaign.collate()
df = campaign.get_collation_result()
[121]:
analysis = uq.analysis.EnsembleBoot(qoi_cols=["iteration"], groupby='ensemble', stat_func=np.mean, stat_name='mean')
campaign.apply_analysis(analysis)
[123]:
results = campaign.get_last_analysis()
print(results)
<easyvvuq.analysis.ensemble_boot.ResultsBoot object at 0x12b03a490>
[ ]:
plt.errorbar(list(range(2, 30, 2)),
results['iteration']['mean'].values,
np.array([results['iteration']['mean'].values - results['iteration']['low'].values,
results['iteration']['high'].values - results['iteration']['mean'].values]))
plt.xlabel('duration (days)')
plt.ylabel('days until eradication')
plt.show()
Sensitivy Analysis with QMC¶
[ ]:
campaign = uq.Campaign(name='sobol_method')
campaign.add_app(
name="sobol_method",
params=params,
encoder=encoder,
decoder=decoder,
collater=collater)
[ ]:
vary = {
"duration": cp.DiscreteUniform(7, 14),
"mortality": cp.Uniform(0.1, 0.3),
}
qmc_sampler = uq.sampling.QMCSampler(vary, 100)
[ ]:
campaign.set_sampler(qmc_sampler)
[ ]:
statuses = campaign.sample_and_apply(
0, uq.actions.ExecuteLocalV2("{} epidemic_in.json output.csv".format(
os.path.abspath('EasyVVUQ/tutorials/epidemic/epidemic.py')), interpret="python3"), 8)
[ ]:
statuses.progress()
[ ]:
campaign.collate()
df = campaign.get_collation_result()
print(df)
[ ]:
from easyvvuq.analysis import QMCAnalysis
analysis = QMCAnalysis(qmc_sampler, ['n', 'duration', 'mortality'])
campaign.apply_analysis(analysis)
results = campaign.get_last_analysis()
results['sobols_first']['n']['n'][0], results['sobols_first']['mortality']['mortality'], results['sobols_first']['duration']['duration'])
Using Stochastic Collocation¶
[ ]:
params = {
"grid_size": {"type": "float", "default": 10},
"n": {"type": "float", "min": 1, "max": 100, "default": 20},
"duration": {"type": "float", "min": 0, "max": 365, "default": 28},
"mortality": {"type": "float", "min": 0.0, "max": 1.0, "default": 0.2}
}
campaign = uq.Campaign(name='epidemic_sc')
campaign.add_app(
name="epidemic_sc",
params=params,
encoder=encoder,
decoder=decoder,
collater=collater)
[ ]:
vary = {
"n": cp.DiscreteUniform(10, 100),
"duration": cp.DiscreteUniform(7, 56),
"mortality": cp.Uniform(0.0, 1.0),
}
sampler = uq.sampling.SCSampler(vary=vary, polynomial_order=9)
campaign.set_sampler(sampler)
[ ]:
statuses = campaign.sample_and_apply(
0,
uq.actions.ExecuteKubernetes(
'tutorial_files/kubernetes/epidemic.yaml',
['epidemic_in.json'],
'output.csv'), 8)
[ ]:
print(statuses.progress())
[ ]:
campaign.collate()
df = campaign.get_collation_result()
print(df)
And the analysis can be done with: :
[ ]:
analysis = uq.analysis.SCAnalysis(sampler=sampler, qoi_cols=["iteration"])
campaign.apply_analysis(analysis)
[ ]:
n = np.linspace(5, 100, 20)
duration = np.linspace(7, 50, 20)
mortality = 0.7
grid = np.meshgrid(n, duration, mortality)
z = np.array([analysis.surrogate('iteration', [grid[0].flatten()[i], grid[1].flatten()[i], mortality]) for i in range(n.shape[0] * duration.shape[0])]).flatten()
fig = plt.figure(figsize=(10, 10), dpi= 80, facecolor='w', edgecolor='k')
ax = fig.gca(projection='3d')
ax.set_zlim(0, 200)
ax.plot_trisurf(grid[0].flatten(), grid[1].flatten(), z)
ax.set_xlabel('n')
ax.set_ylabel('duration')
ax.set_zlabel('time')
plt.show()
Let us quickly check the correspondence of the surrogate model to the data we have collected during our parameter sweep experiment. What we’re looking for is the curve should be within the confidence interval we have calculated.
[ ]:
n = [20] * 14
duration = list(range(2, 30, 2))
mortality = [0.2] * 14
plt.plot(([analysis.surrogate('iteration', [n_, duration_, mortality_])[0] for n_, duration_, mortality_ in zip(n, duration, mortality)]))
plt.ylim(0, 100)
plt.show()
[ ]:
