Tuner

The base class for all tuning methods in DeepOBS.

class deepobs.tuner.tuner.Tuner(optimizer_class, hyperparam_names, ressources, runner)[source]

The base class for all tuning methods in DeepOBS.

__init__(optimizer_class, hyperparam_names, ressources, runner)[source]
Parameters:
  • optimizer_class (framework optimizer class) -- The optimizer class of the optimizer that is run on the testproblems. For PyTorch this must be a subclass of torch.optim.Optimizer. For TensorFlow a subclass of tf.train.Optimizer.
  • hyperparam_names (dict) -- A nested dictionary that lists all hyperparameters of the optimizer, their type and their default values (if they have any) in the form: {'<name>': {'type': <type>, 'default': <default value>}}, e.g. for torch.optim.SGD with momentum: {'lr': {'type': float}, 'momentum': {'type': float, 'default': 0.99}, 'uses_nesterov': {'type': bool, 'default': False}}
  • ressources (int) -- The number of evaluations the tuner is allowed to perform on each testproblem.
  • runner -- The DeepOBS runner that the tuner uses for evaluation.
tune(testproblem, *args, output_dir='./results', random_seed=42, rerun_best_setting=True, **kwargs)[source]

Tunes hyperparaneter of the optimizer_class on a testproblem. :param testproblem: Testproblem for which to generate commands. :type testproblem: str :param output_dir: The output path where the execution results are written to. :type output_dir: str :param random_seed: The random seed for the tuning. :type random_seed: int :param rerun_best_setting: Whether to rerun the best setting with 10 different seeds. :type rerun_best_setting: bool

tune_on_testset(testset, *args, **kwargs)[source]

Tunes the hyperparameter on a whole testset. :param testset: A list of testproblems. :type testset: list