Quadratic Test Problems

DeepOBS includes a stochastic quadratic problem with an eigenspectrum similar to what has been reported for neural networks.

Other stochastic quadratic problems (of different dimensionality or with a different Hessian structure) can be created easily using the quadratic_base class.

class deepobs.tensorflow.testproblems._quadratic._quadratic_base(batch_size, weight_decay=None, hessian=array([[1., 0., 0., ..., 0., 0., 0.], [0., 1., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.], ..., [0., 0., 0., ..., 1., 0., 0.], [0., 0., 0., ..., 0., 1., 0.], [0., 0., 0., ..., 0., 0., 1.]]))[source]

DeepOBS base class for a stochastic quadratic test problems creating lossfunctions of the form

\(0.5* (\theta - x)^T * Q * (\theta - x)\)

with Hessian Q and "data" x coming from the quadratic data set, i.e., zero-mean normal.

Parameters:
  • batch_size (int) -- Batch size to use.
  • weight_decay (float) -- No weight decay (L2-regularization) is used in this test problem. Defaults to None and any input here is ignored.
  • hessian (np.array) -- Hessian of the quadratic problem. Defaults to the 100 dimensional identity.
dataset

The DeepOBS data set class for the quadratic test problem.

train_init_op

A tensorflow operation initializing the test problem for the training phase.

train_eval_init_op

A tensorflow operation initializing the test problem for evaluating on training data.

test_init_op

A tensorflow operation initializing the test problem for evaluating on test data.

losses

A tf.Tensor of shape (batch_size, ) containing the per-example loss values.

regularizer

A scalar tf.Tensor containing a regularization term. Will always be 0.0 since no regularizer is used.

set_up()[source]

Sets up the stochastic quadratic test problem. The parameter Theta will be initialized to (a vector of) 1.0.

Test Problems