Test Problems¶
Currently DeepOBS includes twenty-six different test problems. A test problem is given by a combination of a data set and a model and is characterized by its loss function.
Each test problem inherits from the same base class with the following signature.
-
class
deepobs.tensorflow.testproblems.testproblem.
TestProblem
(batch_size, l2_reg=None)[source]¶ Base class for DeepOBS test problems.
Parameters: - batch_size (int) -- Batch size to use.
- l2_reg (float) -- L2-Regularization (weight decay) factor to use. If not specified, the test problems revert to their respective defaults. Note: Some test problems do not use regularization and this value will be ignored in such a case.
-
dataset
¶ The dataset used by the test problem (datasets.DataSet instance).
-
train_init_op
¶ A tensorflow operation initializing the test problem for the training phase.
-
train_eval_init_op
¶ A tensorflow operation initializing the test problem for evaluating on training data.
-
test_init_op
¶ A tensorflow operation initializing the test problem for evaluating on test data.
-
losses
¶ A tf.Tensor of shape (batch_size, ) containing the per-example loss values.
-
regularizer
¶ A scalar tf.Tensor containing a regularization term (might be a constant 0.0 for test problems that do not use regularization).
-
accuracy
¶ A scalar tf.Tensor containing the mini-batch mean accuracy.
Note
Some of the test problems described here are based on more general implementations. For example the Wide ResNet 40-4 network on Cifar-100 is based on the general Wide ResNet architecture which is also implemented. Therefore, it is very easy to include new Wide ResNets if necessary.