Tolstoi Char RNN¶
DeepOBS test problem class for a two-layer LSTM for character-level language modelling (Char RNN) on Tolstoi's War and Peace.
Some network characteristics:
128hidden units per LSTM cell
- sequence length
- cell state is automatically stored in variables between subsequent steps
- when the phase placeholder swithches its value from one step to the next, the cell state is set to its zero value (meaning that we set to zero state after each round of evaluation, it is therefore important to set the evaluation interval such that we evaluate after a full epoch.)
Working training parameters are:
- batch size
- SGD with a learning rate of \(\approx 0.1\) works
- batch_size (int) -- Batch size to use.
- weight_decay (float) -- No weight decay (L2-regularization) is used in this
test problem. Defaults to
Noneand any input here is ignored.
The DeepOBS data set class for Tolstoi.
A tensorflow operation initializing the test problem for the training phase.
A tensorflow operation initializing the test problem for evaluating on training data.
A tensorflow operation initializing the test problem for evaluating on test data.
A tf.Tensor of shape (batch_size, ) containing the per-example loss values.
A scalar tf.Tensor containing a regularization term.
A scalar tf.Tensor containing the mini-batch mean accuracy.
Set up the Char RNN test problem instance on Tolstoi.