# Estimate Runtime¶

A convenience script to estimate the run time overhead of a new optimization method compared to SGD.

By default this script runs SGD as well as the new optimizer 5 times for 3 epochs on the multi-layer perceptron on MNIST while measuring the time. It will output the mean run time overhead of the new optimizer for these runs.

Optionally the setup can be changed, by varying the test problem, the number of epochs, the number of runs, etc. if this allows for a fairer evaluation.

Usage:

Run a new run script and compare its runtime to SGD.

usage: deepobs_estimate_runtime.py [-h] [--test_problem TEST_PROBLEM]
[--data_dir DATA_DIR] [--bs BS] [--lr LR]
[-N NUM_EPOCHS] [--num_runs NUM_RUNS]
[--saveto SAVETO]
[--optimizer_args OPTIMIZER_ARGS]
run_script


## Positional Arguments¶

 run_script Path to the new run_script.

## Named Arguments¶

 --test_problem Name of the test problem to run both scripts. Default: "mnist_mlp" --data_dir Path to the base data dir. If not set, deepobs uses its default. Default: "data_deepobs" --bs, --batch_size The batch size (positive integer). Default: 128 --lr, --learning_rate The learning rate of both SGD and the new optimizer, defaults to 1e-5. Default: 1e-05 -N, --num_epochs Total number of training epochs per run. Default: 3 --num_runs Total number of runs for each optimizer. Default: 5 --saveto Folder for saving a txt files with a summary. --optimizer_args Additional arguments for the new optimizer