Aggregate Run¶
-
class
deepobs.analyzer.analyze_utils.
AggregateRun
(path, runs, name, metric, testproblem, conv_perf)[source]¶ DeepOBS class for a group of runs witht the same settings (but possibly different seeds).
Parameters: - path (str) -- Path to the parent folder of the aggregate run folder (i.e. the settings folder).
- runs (list) -- List of run names all with the same setting.
- name (str) -- Name of the aggregate run (folder).
- metric (str) -- Metric to use for this test problem. If available this
will be
test_accuracies
, otherwisetest_losses
. - testproblem (str) -- Name of the test problem this aggregate run (folder) belongs to.
- conv_perf (float) -- Convergence performance of the test problem this aggregate run (folder) belongs to.
-
name
¶ Name of the aggregate run (folder).
-
testproblem
¶ Name of the test problem this aggregate run (folder) belongs to.
-
conv_perf
¶ Convergence performance for this test problem.
-
runs
¶ List of run names all with the same setting.
-
num_runs
¶ Number of runs (with the same setting).
-
metric
¶ Metric to use for this test problem. If available this will be
test_accuracies
, otherwisetest_losses
.
-
output
¶ Dictionary including all aggregate information of the runs with this setting. All performance metrics have a mean and a standard deviation (can be zero if there is only one run with this setting).
-
final_value
¶ Final (mean) value of the test problem's metric
-
best_value
¶ Best (mean) value of the test problem's metric