Tolstoi Char RNN

class deepobs.tensorflow.testproblems.tolstoi_char_rnn.tolstoi_char_rnn(batch_size, weight_decay=None)[source]

DeepOBS test problem class for a two-layer LSTM for character-level language modelling (Char RNN) on Tolstoi's War and Peace.

Some network characteristics:

  • 128 hidden units per LSTM cell
  • sequence length 50
  • cell state is automatically stored in variables between subsequent steps
  • when the phase placeholder swithches its value from one step to the next, the cell state is set to its zero value (meaning that we set to zero state after each round of evaluation, it is therefore important to set the evaluation interval such that we evaluate after a full epoch.)

Working training parameters are:

  • batch size 50
  • 200 epochs
  • SGD with a learning rate of \(\approx 0.1\) works
Parameters:
  • batch_size (int) -- Batch size to use.
  • weight_decay (float) -- No weight decay (L2-regularization) is used in this test problem. Defaults to None and any input here is ignored.
dataset

The DeepOBS data set class for Tolstoi.

train_init_op

A tensorflow operation initializing the test problem for the training phase.

train_eval_init_op

A tensorflow operation initializing the test problem for evaluating on training data.

test_init_op

A tensorflow operation initializing the test problem for evaluating on test data.

losses

A tf.Tensor of shape (batch_size, ) containing the per-example loss values.

regularizer

A scalar tf.Tensor containing a regularization term.

accuracy

A scalar tf.Tensor containing the mini-batch mean accuracy.

set_up()[source]

Set up the Char RNN test problem instance on Tolstoi.