Aggregate Run

class deepobs.analyzer.analyze_utils.AggregateRun(path, runs, name, metric, testproblem, conv_perf)[source]

DeepOBS class for a group of runs witht the same settings (but possibly different seeds).

Parameters:
  • path (str) -- Path to the parent folder of the aggregate run folder (i.e. the settings folder).
  • runs (list) -- List of run names all with the same setting.
  • name (str) -- Name of the aggregate run (folder).
  • metric (str) -- Metric to use for this test problem. If available this will be test_accuracies, otherwise test_losses.
  • testproblem (str) -- Name of the test problem this aggregate run (folder) belongs to.
  • conv_perf (float) -- Convergence performance of the test problem this aggregate run (folder) belongs to.
name

Name of the aggregate run (folder).

testproblem

Name of the test problem this aggregate run (folder) belongs to.

conv_perf

Convergence performance for this test problem.

runs

List of run names all with the same setting.

num_runs

Number of runs (with the same setting).

metric

Metric to use for this test problem. If available this will be test_accuracies, otherwise test_losses.

output

Dictionary including all aggregate information of the runs with this setting. All performance metrics have a mean and a standard deviation (can be zero if there is only one run with this setting).

final_value

Final (mean) value of the test problem's metric

best_value

Best (mean) value of the test problem's metric