This ipython notebook will teach you the basics of how multilayer perceptrons work, and show you how to use multilayer perceptrons in pylearn2.
To do this, we will go over several concepts:
Part 1: What pylearn2 is doing for you in this example
Review of softmax regression, and how MLPs are similar
The multilayer perceptron model
Some beneficial properties of MLPs
Some detrimental properties of MLPs
Part 2: How to use pylearn2 to train an MLP
Part 3: A deeper MLP, and pylearn2 polymorphism
Part 4: Regularization, and pylearn2 costs
Note that this won't explain in detail how the individual classes are implemented. The classes follow pretty good naming conventions and have pretty good docstrings, but if you have trouble understanding them, write to me and I might add a part 3 explaining how some of the parts work under the hood.
Please write to pylearn-dev@googlegroups.com if you encounter any problem with this tutorial.
Before running this notebook, you must have installed pylearn2. Follow the download and installation instructions if you have not yet done so.
This tutorial also assumes you already know about softmax regression, and know how to train and evaluate a softmax regression model in pylearn2. If not, work through softmax_regression.ipynb before starting this tutorial.
It's also strongly recommend that you run this notebook with THEANO_FLAGS="device=gpu". This is a processing intensive example and the GPU will make it run a lot faster, if you have one available. Execute the next cell to verify that you are using the GPU.
import theano
print theano.config.device
gpu
Using gpu device 0: GeForce GTX 285
In this part, we won't get into any specifics of pylearn2 yet. We'll just discuss how to train a multilayer perceptron (MLP). If you already know about MLPs, feel free to skip straight to part 2, where we show how to do all of this in pylearn2.
In softmax_regression.ipynb, we saw how softmax regression is a classification model that learns to map an input vector $x$ to a probability distribution $p(y\mid x)$ where $y$ is a categorical value with $k$ different values. We then described how a dataset $\mathcal{D}$ of $(x, y)$ tuples could be used to train a softmax regression model by maximizing the log likelihood,
$$ \sum_{x,y \in \mathcal{D} } \log P(y \mid x). $$A multilayer perceptron is a very general machine learning model. In many cases, we can think of it as mapping $x$ to $P(y\mid x)$, and train it by maximizing the log likelihood. We'll start with that basic perspective, because of its similarity to softmax regression. (It is, however, possible to interpret the output of a multiplayer perceptron non-probabilistically, to use it for regression rather than classification, and to train it by optimizing functions other than the log likelihood)
Everything we described above is still relevant to the MLP. However, there is one more fact about softmax regression that does not apply to the MLP. Specifically, softmax regression assumes that
$$ p(y \mid x) = \frac { \exp( x^T W + b ) } { \sum_i \exp(x^T W + b)_i } = \text{softmax}( x^T W + b). $$The MLP makes a different assumption about the functional form of $p(y \mid x)$.
The multilayer perceptron model assumption is very weak. Essentially, the assumption is that the relationship between inputs and outputs can be represented by the composition of several simpler functions. Each function being composed can be thought of as another "layer" or stage of processing. The number of compositions determines the "depth" of the model.
Suppose we have a sequence of functions implementing the layers, $g_1, g_2, \dots, g_L$. Then the output of our MLP is
$$f(x) = g_L(g_{L-1}( \dots g_2( g_1 ( x )) \dots )).$$In the first example for this tutorial, we will use just two layers. The final layer will be
$ g_2(g_1) = \text{softmax}( g_1^T W^{(2)} + b^{(2)}),$
so we can think of this model as using $g_1$ to transform $x$ into a different space, then doing softmax regression in that space.
For the first layer, we will use an affine transform followed by elementwise-application of the logistic sigmoid function, $\sigma(z) = \frac {1 } { 1 + \exp(-z) }.$ This is a very commonly used type of layer in multilayer perceptrons. Putting it all together, we get
$ g_1(x) = \sigma ( x^T W^{(1)} + b^{(1)} ).$
The full model is thus
$$ f(x) = \text{softmax}( \sigma ( x^T W^{(1)} + b^{(1)} )^T W^{(2)} + b^{(2)}). $$If we interpret $f(x)$ as defining $p(y \mid x)$, it makes sense to train the parameters $W^{(1)}$, $W^{(2)}$, $b^{(1)}$, and $b^{(2)}$ by maximizing the log likelihood of the training data.
An obvious problem with softmax regression and other linear classifiers is that linear functions are very simple. They prevent solutions to even very simple classification problems, such as the class of 2 bit patterns whose XOR is true. XOR is true when $x=[1,0]$ or $x=[0,1]$ but not when $x=[0,0]$ or $x=[1,1]$. Suppose we draw a line that separates $[0,0]$ from $[0,1]$. Then it must pass through some point $[0,p]$. We require that this line also pass through $[q,1]$ in order to separate $[0,1]$ from $[1,1]$. But this means it slope must be negative and its $x$-intercept must be negative. Since a line only has one $x$ intercept, it does not pass between $[0,0]$ and $[1,0]$. Those two points belong to different classes, so any linear classifier must fail.
An MLP solves this problem by introducing extra stages of processing. In our two layer example, suppose the dimensionality of the first layer is 2. We call the outputs of this layer "hidden units" because they are neither inputs nor outputs of the system; they are unobserved variables that the network must decide what to do with. The MLP can set one of these hidden units to be active when the sum of the two input variables is less than 1. It can set the other to be active when the sum of the two input variables is greater than 1. It can then set the output unit to be active by default, and to deactivate when either of the two hidden variables is active.
More generally, an MLP with one sufficient large hidden layer can represent any function. This result is known as the "universal approximator theorem."
Another advantage of MLPs is that they can be made deeper and deeper, rather than just wider and wider. Many functions can be represented more efficiently (using fewer parameters) with a deep architecture than with a wide one. Using fewer parameters is beneficial both because the MLP takes less memory to represent, but also because the parameters may be estimated more accurately from a smaller amount of data.
Unfortunately, just because an MLP can represent any function does not mean that it will learn to represent the right function. The problem of overfitting can still make the MLP perform badly on the test set even if it classifies the training set perfectly. While larger MLPs are capable of fitting more complicated training sets, they are also likely to overfit worse than smaller MLPs.
A related issue with MLPs is that they have many configuration options. The model itself imposes design decisions such as what type of function to use for each layer, the dimensionality of each layer. Also, the log likelihood is no longer generally concave, so the choice of optimization procedure matters more than it did with softmax regression. These configuration options are known as "hyperparameters." Choosing the right hyperparameters is an open and exciting research problem.
Most of the hyperparameters in this tutorial were not chosen particularly carefully. Feel free to play with all of the settings in this notebook. If you find better ones, write to me and I'll put your settings and your name in the tutorial!
Now that we've described the theory of what we're going to do, it's time to do it! This part describes how to use pylearn2 to run the algorithms described above.
As in the softmax regression tutorial, we will use the MLP to do optical character recognition on the MNIST dataset. The yaml string we construct is similar ot the one we use before. The main difference is that the MLP model class takes a "layers" argument describing the various layers of the model.
Note that for each layer, we need to specify what class to load. The identity of this class determines what type of layer appears at each position in the network. Here, we use a sigmoid hidden layer followed by a softmax output layer.
Every layer of the MLP needs a unique name. Here we name the first hidden layer 'h0' and the output label representing the prediction of the class $y$ 'y'. These layer names are used to generate monitor channel names later so that we can track properties of each layer separately.
The hidden layer needs some configuration that is pretty similar to the configuration for the output layer. Much as we need to tell the output layer its size (10 classes) we also need to tell the hidden layer its dimension, or the number of hidden units to go in that layer. In this case we use 500. We also need to tell it how to initialize its weights. The Sigmoid class supports the irange argument that we demonstrated for Softmax in the softmax regression tutorial, and we could use that here. Instead, we demonstrate a different argument, sparse_init. When sparse_init is specified, each unit gets exactly sparse_init non-zero weights initially. These weights are drawn from $N(0,1)$, so they are quite large compared to how weights are usually initialized.
import os
import pylearn2
path = os.path.join(pylearn2.__path__[0], 'scripts', 'tutorials', 'multilayer_perceptron', 'mlp_tutorial_part_2.yaml')
with open(path, 'r') as f:
train = f.read()
hyper_params = {'train_stop' : 50000,
'valid_stop' : 60000,
'dim_h0' : 500,
'max_epochs' : 10000,
'save_path' : '.'}
train = train % (hyper_params)
print train
!obj:pylearn2.train.Train { dataset: &train !obj:pylearn2.datasets.mnist.MNIST { which_set: 'train', start: 0, stop: 50000 }, model: !obj:pylearn2.models.mlp.MLP { layers: [ !obj:pylearn2.models.mlp.Sigmoid { layer_name: 'h0', dim: 500, sparse_init: 15, }, !obj:pylearn2.models.mlp.Softmax { layer_name: 'y', n_classes: 10, irange: 0. } ], nvis: 784, }, algorithm: !obj:pylearn2.training_algorithms.bgd.BGD { batch_size: 10000, line_search_mode: 'exhaustive', conjugate: 1, updates_per_batch: 10, monitoring_dataset: { 'train' : *train, 'valid' : !obj:pylearn2.datasets.mnist.MNIST { which_set: 'train', start: 50000, stop: 60000 }, 'test' : !obj:pylearn2.datasets.mnist.MNIST { which_set: 'test', } }, termination_criterion: !obj:pylearn2.termination_criteria.And { criteria: [ !obj:pylearn2.termination_criteria.MonitorBased { channel_name: "valid_y_misclass" }, !obj:pylearn2.termination_criteria.EpochCounter { max_epochs: 10000 } ] } }, extensions: [ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest { channel_name: 'valid_y_misclass', save_path: "mlp_best.pkl" }, ] }
Note that we still do not specify a cost to be minimized. In the case of LogisticRegression, the model requested the negative log likelihood by default. In the case of the MLP, it is up to the final layer of the MLP to specify the default cost if the user does not provide one. In this case, since the final layer is a Softmax layer, we still have the same objective function as in the SoftmaxRegression tutorial.
Now, we use pylearn2's yaml_parse.load to construct the Train object, and run its main loop. The same thing could be accomplished by running pylearn2's train.py script on a file containing the yaml string.
Execute the next cell to train the model. This will take several minutes and possible as much as a few hours depending on how fast your computer is.
from pylearn2.config import yaml_parse
train = yaml_parse.load(train)
train.main_loop()
compiling begin_record_entry...
/u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit. warnings.warn("MLP changing the recursion limit.")
compiling begin_record_entry done. Time elapsed: 0.479222 seconds Monitored channels: ave_grad_mult ave_grad_size ave_step_size test_h0_col_norms_max test_h0_col_norms_mean test_h0_col_norms_min test_h0_max_x_max_u test_h0_max_x_mean_u test_h0_max_x_min_u test_h0_mean_x_max_u test_h0_mean_x_mean_u test_h0_mean_x_min_u test_h0_min_x_max_u test_h0_min_x_mean_u test_h0_min_x_min_u test_h0_row_norms_max test_h0_row_norms_mean test_h0_row_norms_min test_objective test_y_col_norms_max test_y_col_norms_mean test_y_col_norms_min test_y_max_max_class test_y_mean_max_class test_y_min_max_class test_y_misclass test_y_nll test_y_row_norms_max test_y_row_norms_mean test_y_row_norms_min train_h0_col_norms_max train_h0_col_norms_mean train_h0_col_norms_min train_h0_max_x_max_u train_h0_max_x_mean_u train_h0_max_x_min_u train_h0_mean_x_max_u train_h0_mean_x_mean_u train_h0_mean_x_min_u train_h0_min_x_max_u train_h0_min_x_mean_u train_h0_min_x_min_u train_h0_row_norms_max train_h0_row_norms_mean train_h0_row_norms_min train_objective train_y_col_norms_max train_y_col_norms_mean train_y_col_norms_min train_y_max_max_class train_y_mean_max_class train_y_min_max_class train_y_misclass train_y_nll train_y_row_norms_max train_y_row_norms_mean train_y_row_norms_min valid_h0_col_norms_max valid_h0_col_norms_mean valid_h0_col_norms_min valid_h0_max_x_max_u valid_h0_max_x_mean_u valid_h0_max_x_min_u valid_h0_mean_x_max_u valid_h0_mean_x_mean_u valid_h0_mean_x_min_u valid_h0_min_x_max_u valid_h0_min_x_mean_u valid_h0_min_x_min_u valid_h0_row_norms_max valid_h0_row_norms_mean valid_h0_row_norms_min valid_objective valid_y_col_norms_max valid_y_col_norms_mean valid_y_col_norms_min valid_y_max_max_class valid_y_mean_max_class valid_y_min_max_class valid_y_misclass valid_y_nll valid_y_row_norms_max valid_y_row_norms_mean valid_y_row_norms_min Compiling accum... graph size: 160 graph size: 157 graph size: 157 Compiling accum done. Time elapsed: 11.082528 seconds Monitoring step: Epochs seen: 0 Batches seen: 0 Examples seen: 0 ave_grad_mult: 0.0 ave_grad_size: 0.0 ave_step_size: 0.0 test_h0_col_norms_max: 6.23503398895 test_h0_col_norms_mean: 3.82355618477 test_h0_col_norms_min: 2.06193995476 test_h0_max_x_max_u: 0.999900639057 test_h0_max_x_mean_u: 0.909942150116 test_h0_max_x_min_u: 0.508436858654 test_h0_mean_x_max_u: 0.901069939137 test_h0_mean_x_mean_u: 0.476713299751 test_h0_mean_x_min_u: 0.152832776308 test_h0_min_x_max_u: 0.480607658625 test_h0_min_x_mean_u: 0.0718067958951 test_h0_min_x_min_u: 0.000174344575498 test_h0_row_norms_max: 5.89326095581 test_h0_row_norms_mean: 2.98549151421 test_h0_row_norms_min: 0.0 test_objective: 2.30258440971 test_y_col_norms_max: 0.0 test_y_col_norms_mean: 0.0 test_y_col_norms_min: 0.0 test_y_max_max_class: 0.0999999940395 test_y_mean_max_class: 0.099990285933 test_y_min_max_class: 0.0999999940395 test_y_misclass: 0.901999950409 test_y_nll: 2.30258440971 test_y_row_norms_max: 0.0 test_y_row_norms_mean: 0.0 test_y_row_norms_min: 0.0 train_h0_col_norms_max: 6.23503303528 train_h0_col_norms_mean: 3.82355594635 train_h0_col_norms_min: 2.06193971634 train_h0_max_x_max_u: 0.999884188175 train_h0_max_x_mean_u: 0.910601377487 train_h0_max_x_min_u: 0.542480230331 train_h0_mean_x_max_u: 0.899177610874 train_h0_mean_x_mean_u: 0.477026820183 train_h0_mean_x_min_u: 0.158626437187 train_h0_min_x_max_u: 0.458495438099 train_h0_min_x_mean_u: 0.0697233080864 train_h0_min_x_min_u: 0.000107248379209 train_h0_row_norms_max: 5.89326000214 train_h0_row_norms_mean: 2.98549151421 train_h0_row_norms_min: 0.0 train_objective: 2.30258440971 train_y_col_norms_max: 0.0 train_y_col_norms_mean: 0.0 train_y_col_norms_min: 0.0 train_y_max_max_class: 0.0999999940395 train_y_mean_max_class: 0.0999902933836 train_y_min_max_class: 0.0999999940395 train_y_misclass: 0.901359915733 train_y_nll: 2.30258440971 train_y_row_norms_max: 0.0 train_y_row_norms_mean: 0.0 train_y_row_norms_min: 0.0 valid_h0_col_norms_max: 6.23503398895 valid_h0_col_norms_mean: 3.82355618477 valid_h0_col_norms_min: 2.06193995476 valid_h0_max_x_max_u: 0.999902307987 valid_h0_max_x_mean_u: 0.910734891891 valid_h0_max_x_min_u: 0.505713641644 valid_h0_mean_x_max_u: 0.897212743759 valid_h0_mean_x_mean_u: 0.477113306522 valid_h0_mean_x_min_u: 0.159442692995 valid_h0_min_x_max_u: 0.474104195833 valid_h0_min_x_mean_u: 0.0706818476319 valid_h0_min_x_min_u: 0.000110276472697 valid_h0_row_norms_max: 5.89326095581 valid_h0_row_norms_mean: 2.98549151421 valid_h0_row_norms_min: 0.0 valid_objective: 2.30258440971 valid_y_col_norms_max: 0.0 valid_y_col_norms_mean: 0.0 valid_y_col_norms_min: 0.0 valid_y_max_max_class: 0.0999999940395 valid_y_mean_max_class: 0.099990285933 valid_y_min_max_class: 0.0999999940395 valid_y_misclass: 0.900900006294 valid_y_nll: 2.30258440971 valid_y_row_norms_max: 0.0 valid_y_row_norms_mean: 0.0 valid_y_row_norms_min: 0.0 Time this epoch: 35.338505 seconds Monitoring step: Epochs seen: 1 Batches seen: 5 Examples seen: 50000 ave_grad_mult: 0.566698908806 ave_grad_size: 0.567735552788 ave_step_size: 0.291175425053 test_h0_col_norms_max: 6.24065446854 test_h0_col_norms_mean: 3.83268666267 test_h0_col_norms_min: 2.0723836422 test_h0_max_x_max_u: 0.999798893929 test_h0_max_x_mean_u: 0.930105090141 test_h0_max_x_min_u: 0.600322246552 test_h0_mean_x_max_u: 0.863031387329 test_h0_mean_x_mean_u: 0.476889610291 test_h0_mean_x_min_u: 0.171247333288 test_h0_min_x_max_u: 0.412737071514 test_h0_min_x_mean_u: 0.0536084063351 test_h0_min_x_min_u: 0.000199288566364 test_h0_row_norms_max: 5.89763784409 test_h0_row_norms_mean: 2.99287319183 test_h0_row_norms_min: 0.0068221190013 test_objective: 0.350786328316 test_y_col_norms_max: 2.74948716164 test_y_col_norms_mean: 2.56346487999 test_y_col_norms_min: 2.34412789345 test_y_max_max_class: 0.999794960022 test_y_mean_max_class: 0.840726792812 test_y_min_max_class: 0.207839608192 test_y_misclass: 0.0983999967575 test_y_nll: 0.350786328316 test_y_row_norms_max: 0.701220929623 test_y_row_norms_mean: 0.34330791235 test_y_row_norms_min: 0.0764839723706 train_h0_col_norms_max: 6.24065446854 train_h0_col_norms_mean: 3.83268642426 train_h0_col_norms_min: 2.07238340378 train_h0_max_x_max_u: 0.999829530716 train_h0_max_x_mean_u: 0.930867910385 train_h0_max_x_min_u: 0.617025732994 train_h0_mean_x_max_u: 0.860394179821 train_h0_mean_x_mean_u: 0.477169722319 train_h0_mean_x_min_u: 0.177841931581 train_h0_min_x_max_u: 0.386521846056 train_h0_min_x_mean_u: 0.0524694435298 train_h0_min_x_min_u: 0.000151637359522 train_h0_row_norms_max: 5.89763736725 train_h0_row_norms_mean: 2.99287295341 train_h0_row_norms_min: 0.0068221190013 train_objective: 0.372914284468 train_y_col_norms_max: 2.74948716164 train_y_col_norms_mean: 2.56346464157 train_y_col_norms_min: 2.34412789345 train_y_max_max_class: 0.999826908112 train_y_mean_max_class: 0.833846986294 train_y_min_max_class: 0.198893502355 train_y_misclass: 0.106319993734 train_y_nll: 0.372914284468 train_y_row_norms_max: 0.701220929623 train_y_row_norms_mean: 0.343307882547 train_y_row_norms_min: 0.0764839798212 valid_h0_col_norms_max: 6.24065446854 valid_h0_col_norms_mean: 3.83268666267 valid_h0_col_norms_min: 2.0723836422 valid_h0_max_x_max_u: 0.999864041805 valid_h0_max_x_mean_u: 0.930580854416 valid_h0_max_x_min_u: 0.638543665409 valid_h0_mean_x_max_u: 0.858349621296 valid_h0_mean_x_mean_u: 0.477255016565 valid_h0_mean_x_min_u: 0.177810654044 valid_h0_min_x_max_u: 0.361713379622 valid_h0_min_x_mean_u: 0.0531250722706 valid_h0_min_x_min_u: 0.000215846084757 valid_h0_row_norms_max: 5.89763784409 valid_h0_row_norms_mean: 2.99287319183 valid_h0_row_norms_min: 0.0068221190013 valid_objective: 0.339448153973 valid_y_col_norms_max: 2.74948716164 valid_y_col_norms_mean: 2.56346487999 valid_y_col_norms_min: 2.34412789345 valid_y_max_max_class: 0.999945104122 valid_y_mean_max_class: 0.845010101795 valid_y_min_max_class: 0.196165680885 valid_y_misclass: 0.0965999960899 valid_y_nll: 0.339448153973 valid_y_row_norms_max: 0.701220929623 valid_y_row_norms_mean: 0.34330791235 valid_y_row_norms_min: 0.0764839723706 Time this epoch: 35.029214 seconds Monitoring step: Epochs seen: 2 Batches seen: 10 Examples seen: 100000 ave_grad_mult: 0.648920476437 ave_grad_size: 0.385089039803 ave_step_size: 0.205155700445 test_h0_col_norms_max: 6.2453122139 test_h0_col_norms_mean: 3.8378276825 test_h0_col_norms_min: 2.07804393768 test_h0_max_x_max_u: 0.999864637852 test_h0_max_x_mean_u: 0.93498313427 test_h0_max_x_min_u: 0.613258361816 test_h0_mean_x_max_u: 0.847131431103 test_h0_mean_x_mean_u: 0.476234823465 test_h0_mean_x_min_u: 0.172577545047 test_h0_min_x_max_u: 0.381593316793 test_h0_min_x_mean_u: 0.0493729114532 test_h0_min_x_min_u: 0.000119279786304 test_h0_row_norms_max: 5.90795898438 test_h0_row_norms_mean: 2.99731445312 test_h0_row_norms_min: 0.0140750305727 test_objective: 0.296338170767 test_y_col_norms_max: 3.20915484428 test_y_col_norms_mean: 3.00029850006 test_y_col_norms_min: 2.73683047295 test_y_max_max_class: 0.9999589324 test_y_mean_max_class: 0.878535091877 test_y_min_max_class: 0.236884206533 test_y_misclass: 0.0850000008941 test_y_nll: 0.296338170767 test_y_row_norms_max: 0.839111089706 test_y_row_norms_mean: 0.403169810772 test_y_row_norms_min: 0.0928392037749 train_h0_col_norms_max: 6.24531269073 train_h0_col_norms_mean: 3.83782744408 train_h0_col_norms_min: 2.07804393768 train_h0_max_x_max_u: 0.999843478203 train_h0_max_x_mean_u: 0.935774207115 train_h0_max_x_min_u: 0.630811154842 train_h0_mean_x_max_u: 0.843988478184 train_h0_mean_x_mean_u: 0.476507484913 train_h0_mean_x_min_u: 0.179330348969 train_h0_min_x_max_u: 0.372446238995 train_h0_min_x_mean_u: 0.048459071666 train_h0_min_x_min_u: 0.000123051402625 train_h0_row_norms_max: 5.90795898438 train_h0_row_norms_mean: 2.99731445312 train_h0_row_norms_min: 0.014075031504 train_objective: 0.310930907726 train_y_col_norms_max: 3.20915460587 train_y_col_norms_mean: 3.00029873848 train_y_col_norms_min: 2.73683071136 train_y_max_max_class: 0.999969184399 train_y_mean_max_class: 0.872422754765 train_y_min_max_class: 0.206743046641 train_y_misclass: 0.0889399945736 train_y_nll: 0.310930907726 train_y_row_norms_max: 0.839111089706 train_y_row_norms_mean: 0.403169810772 train_y_row_norms_min: 0.0928391963243 valid_h0_col_norms_max: 6.2453122139 valid_h0_col_norms_mean: 3.8378276825 valid_h0_col_norms_min: 2.07804393768 valid_h0_max_x_max_u: 0.999864220619 valid_h0_max_x_mean_u: 0.935237765312 valid_h0_max_x_min_u: 0.672344446182 valid_h0_mean_x_max_u: 0.842247903347 valid_h0_mean_x_mean_u: 0.476582586765 valid_h0_mean_x_min_u: 0.178887397051 valid_h0_min_x_max_u: 0.358671993017 valid_h0_min_x_mean_u: 0.0488182529807 valid_h0_min_x_min_u: 0.000185967219295 valid_h0_row_norms_max: 5.90795898438 valid_h0_row_norms_mean: 2.99731445312 valid_h0_row_norms_min: 0.0140750305727 valid_objective: 0.286341637373 valid_y_col_norms_max: 3.20915484428 valid_y_col_norms_mean: 3.00029850006 valid_y_col_norms_min: 2.73683047295 valid_y_max_max_class: 0.999980926514 valid_y_mean_max_class: 0.880788624287 valid_y_min_max_class: 0.193636313081 valid_y_misclass: 0.0813999921083 valid_y_nll: 0.286341637373 valid_y_row_norms_max: 0.839111089706 valid_y_row_norms_mean: 0.403169810772 valid_y_row_norms_min: 0.0928392037749 Time this epoch: 35.009148 seconds Monitoring step: Epochs seen: 3 Batches seen: 15 Examples seen: 150000 ave_grad_mult: 0.747792065144 ave_grad_size: 0.265085607767 ave_step_size: 0.150685995817 test_h0_col_norms_max: 6.24948835373 test_h0_col_norms_mean: 3.84261131287 test_h0_col_norms_min: 2.08266615868 test_h0_max_x_max_u: 0.99994790554 test_h0_max_x_mean_u: 0.937485575676 test_h0_max_x_min_u: 0.633630394936 test_h0_mean_x_max_u: 0.859075248241 test_h0_mean_x_mean_u: 0.475113451481 test_h0_mean_x_min_u: 0.166715249419 test_h0_min_x_max_u: 0.368945479393 test_h0_min_x_mean_u: 0.0472293719649 test_h0_min_x_min_u: 5.30257530045e-05 test_h0_row_norms_max: 5.91970491409 test_h0_row_norms_mean: 3.00150084496 test_h0_row_norms_min: 0.0220027510077 test_objective: 0.269680500031 test_y_col_norms_max: 3.56634759903 test_y_col_norms_mean: 3.29666876793 test_y_col_norms_min: 3.00721621513 test_y_max_max_class: 0.999979376793 test_y_mean_max_class: 0.893490552902 test_y_min_max_class: 0.250094264746 test_y_misclass: 0.0763000026345 test_y_nll: 0.269680500031 test_y_row_norms_max: 0.959613263607 test_y_row_norms_mean: 0.443394243717 test_y_row_norms_min: 0.103941932321 train_h0_col_norms_max: 6.24948787689 train_h0_col_norms_mean: 3.84261083603 train_h0_col_norms_min: 2.08266615868 train_h0_max_x_max_u: 0.99988758564 train_h0_max_x_mean_u: 0.938323676586 train_h0_max_x_min_u: 0.649454653263 train_h0_mean_x_max_u: 0.846590101719 train_h0_mean_x_mean_u: 0.475384742022 train_h0_mean_x_min_u: 0.171920359135 train_h0_min_x_max_u: 0.365952074528 train_h0_min_x_mean_u: 0.0464779213071 train_h0_min_x_min_u: 6.07749607298e-05 train_h0_row_norms_max: 5.91970491409 train_h0_row_norms_mean: 3.00150060654 train_h0_row_norms_min: 0.022002749145 train_objective: 0.278353452682 train_y_col_norms_max: 3.56634736061 train_y_col_norms_mean: 3.29666852951 train_y_col_norms_min: 3.00721621513 train_y_max_max_class: 0.999987363815 train_y_mean_max_class: 0.889036417007 train_y_min_max_class: 0.227912455797 train_y_misclass: 0.0788599997759 train_y_nll: 0.278353452682 train_y_row_norms_max: 0.959613204002 train_y_row_norms_mean: 0.443394213915 train_y_row_norms_min: 0.103941932321 valid_h0_col_norms_max: 6.24948835373 valid_h0_col_norms_mean: 3.84261131287 valid_h0_col_norms_min: 2.08266615868 valid_h0_max_x_max_u: 0.999919652939 valid_h0_max_x_mean_u: 0.937573850155 valid_h0_max_x_min_u: 0.684871912003 valid_h0_mean_x_max_u: 0.850003778934 valid_h0_mean_x_mean_u: 0.475453108549 valid_h0_mean_x_min_u: 0.170857235789 valid_h0_min_x_max_u: 0.353432744741 valid_h0_min_x_mean_u: 0.0467779003084 valid_h0_min_x_min_u: 6.80360026308e-05 valid_h0_row_norms_max: 5.91970491409 valid_h0_row_norms_mean: 3.00150084496 valid_h0_row_norms_min: 0.0220027510077 valid_objective: 0.26020783186 valid_y_col_norms_max: 3.56634759903 valid_y_col_norms_mean: 3.29666876793 valid_y_col_norms_min: 3.00721621513 valid_y_max_max_class: 0.999977052212 valid_y_mean_max_class: 0.896274268627 valid_y_min_max_class: 0.17623616755 valid_y_misclass: 0.0750000029802 valid_y_nll: 0.26020783186 valid_y_row_norms_max: 0.959613263607 valid_y_row_norms_mean: 0.443394243717 valid_y_row_norms_min: 0.103941932321 Time this epoch: 35.058853 seconds Monitoring step: Epochs seen: 4 Batches seen: 20 Examples seen: 200000 ave_grad_mult: 0.788351774216 ave_grad_size: 0.187993511558 ave_step_size: 0.113317854702 test_h0_col_norms_max: 6.25235366821 test_h0_col_norms_mean: 3.84656834602 test_h0_col_norms_min: 2.08510184288 test_h0_max_x_max_u: 0.999974727631 test_h0_max_x_mean_u: 0.938515424728 test_h0_max_x_min_u: 0.650707960129 test_h0_mean_x_max_u: 0.87255191803 test_h0_mean_x_mean_u: 0.474163293839 test_h0_mean_x_min_u: 0.160470247269 test_h0_min_x_max_u: 0.364907234907 test_h0_min_x_mean_u: 0.0464833118021 test_h0_min_x_min_u: 2.23769111471e-05 test_h0_row_norms_max: 5.93058395386 test_h0_row_norms_mean: 3.0049738884 test_h0_row_norms_min: 0.0284670460969 test_objective: 0.252513170242 test_y_col_norms_max: 3.77643465996 test_y_col_norms_mean: 3.49576759338 test_y_col_norms_min: 3.21715569496 test_y_max_max_class: 0.999990880489 test_y_mean_max_class: 0.902969479561 test_y_min_max_class: 0.223742827773 test_y_misclass: 0.0724000036716 test_y_nll: 0.252513170242 test_y_row_norms_max: 1.04190921783 test_y_row_norms_mean: 0.47004455328 test_y_row_norms_min: 0.109351947904 train_h0_col_norms_max: 6.25235366821 train_h0_col_norms_mean: 3.84656858444 train_h0_col_norms_min: 2.08510160446 train_h0_max_x_max_u: 0.999940037727 train_h0_max_x_mean_u: 0.939188420773 train_h0_max_x_min_u: 0.661542713642 train_h0_mean_x_max_u: 0.85992783308 train_h0_mean_x_mean_u: 0.474434643984 train_h0_mean_x_min_u: 0.163209468126 train_h0_min_x_max_u: 0.358978569508 train_h0_min_x_mean_u: 0.0456797704101 train_h0_min_x_min_u: 3.15167126246e-05 train_h0_row_norms_max: 5.93058395386 train_h0_row_norms_mean: 3.00497412682 train_h0_row_norms_min: 0.0284670442343 train_objective: 0.257761448622 train_y_col_norms_max: 3.77643465996 train_y_col_norms_mean: 3.49576735497 train_y_col_norms_min: 3.21715545654 train_y_max_max_class: 0.999995172024 train_y_mean_max_class: 0.898737490177 train_y_min_max_class: 0.233332633972 train_y_misclass: 0.0732599943876 train_y_nll: 0.257761448622 train_y_row_norms_max: 1.04190921783 train_y_row_norms_mean: 0.470044583082 train_y_row_norms_min: 0.109351947904 valid_h0_col_norms_max: 6.25235366821 valid_h0_col_norms_mean: 3.84656834602 valid_h0_col_norms_min: 2.08510184288 valid_h0_max_x_max_u: 0.999963521957 valid_h0_max_x_mean_u: 0.938330054283 valid_h0_max_x_min_u: 0.685399234295 valid_h0_mean_x_max_u: 0.864110708237 valid_h0_mean_x_mean_u: 0.474497437477 valid_h0_mean_x_min_u: 0.161501988769 valid_h0_min_x_max_u: 0.347681999207 valid_h0_min_x_mean_u: 0.0459976904094 valid_h0_min_x_min_u: 2.87672100967e-05 valid_h0_row_norms_max: 5.93058395386 valid_h0_row_norms_mean: 3.0049738884 valid_h0_row_norms_min: 0.0284670460969 valid_objective: 0.242218419909 valid_y_col_norms_max: 3.77643465996 valid_y_col_norms_mean: 3.49576759338 valid_y_col_norms_min: 3.21715569496 valid_y_max_max_class: 0.999983727932 valid_y_mean_max_class: 0.90525239706 valid_y_min_max_class: 0.237812787294 valid_y_misclass: 0.070799998939 valid_y_nll: 0.242218419909 valid_y_row_norms_max: 1.04190921783 valid_y_row_norms_mean: 0.47004455328 valid_y_row_norms_min: 0.109351947904 Time this epoch: 34.824181 seconds Monitoring step: Epochs seen: 5 Batches seen: 25 Examples seen: 250000 ave_grad_mult: 0.822910606861 ave_grad_size: 0.140246614814 ave_step_size: 0.0910708159208 test_h0_col_norms_max: 6.2554602623 test_h0_col_norms_mean: 3.85085010529 test_h0_col_norms_min: 2.08709287643 test_h0_max_x_max_u: 0.999985814095 test_h0_max_x_mean_u: 0.939129829407 test_h0_max_x_min_u: 0.667058110237 test_h0_mean_x_max_u: 0.881521999836 test_h0_mean_x_mean_u: 0.473096251488 test_h0_mean_x_min_u: 0.148683413863 test_h0_min_x_max_u: 0.366505622864 test_h0_min_x_mean_u: 0.0459363907576 test_h0_min_x_min_u: 9.1133879323e-06 test_h0_row_norms_max: 5.94399118423 test_h0_row_norms_mean: 3.00873041153 test_h0_row_norms_min: 0.0347110852599 test_objective: 0.236052155495 test_y_col_norms_max: 3.98437142372 test_y_col_norms_mean: 3.68210268021 test_y_col_norms_min: 3.41360712051 test_y_max_max_class: 0.99999153614 test_y_mean_max_class: 0.909221351147 test_y_min_max_class: 0.227106332779 test_y_misclass: 0.0672999992967 test_y_nll: 0.236052155495 test_y_row_norms_max: 1.12676775455 test_y_row_norms_mean: 0.494562119246 test_y_row_norms_min: 0.114525236189 train_h0_col_norms_max: 6.2554602623 train_h0_col_norms_mean: 3.85085010529 train_h0_col_norms_min: 2.08709263802 train_h0_max_x_max_u: 0.999965369701 train_h0_max_x_mean_u: 0.939886808395 train_h0_max_x_min_u: 0.672379374504 train_h0_mean_x_max_u: 0.869372367859 train_h0_mean_x_mean_u: 0.473366141319 train_h0_mean_x_min_u: 0.151700764894 train_h0_min_x_max_u: 0.357233524323 train_h0_min_x_mean_u: 0.0450618416071 train_h0_min_x_min_u: 1.45595986396e-05 train_h0_row_norms_max: 5.9439907074 train_h0_row_norms_mean: 3.00873041153 train_h0_row_norms_min: 0.0347110852599 train_objective: 0.239308148623 train_y_col_norms_max: 3.9843711853 train_y_col_norms_mean: 3.68210220337 train_y_col_norms_min: 3.41360712051 train_y_max_max_class: 0.999996185303 train_y_mean_max_class: 0.905649185181 train_y_min_max_class: 0.236008346081 train_y_misclass: 0.0679599940777 train_y_nll: 0.239308148623 train_y_row_norms_max: 1.12676763535 train_y_row_norms_mean: 0.494562089443 train_y_row_norms_min: 0.114525228739 valid_h0_col_norms_max: 6.2554602623 valid_h0_col_norms_mean: 3.85085010529 valid_h0_col_norms_min: 2.08709287643 valid_h0_max_x_max_u: 0.999980926514 valid_h0_max_x_mean_u: 0.939110815525 valid_h0_max_x_min_u: 0.683836042881 valid_h0_mean_x_max_u: 0.873598277569 valid_h0_mean_x_mean_u: 0.473425507545 valid_h0_mean_x_min_u: 0.149841591716 valid_h0_min_x_max_u: 0.346154510975 valid_h0_min_x_mean_u: 0.0454438403249 valid_h0_min_x_min_u: 1.18227362691e-05 valid_h0_row_norms_max: 5.94399118423 valid_h0_row_norms_mean: 3.00873041153 valid_h0_row_norms_min: 0.0347110852599 valid_objective: 0.22658072412 valid_y_col_norms_max: 3.98437142372 valid_y_col_norms_mean: 3.68210268021 valid_y_col_norms_min: 3.41360712051 valid_y_max_max_class: 0.999987483025 valid_y_mean_max_class: 0.911411643028 valid_y_min_max_class: 0.217763110995 valid_y_misclass: 0.0644000023603 valid_y_nll: 0.22658072412 valid_y_row_norms_max: 1.12676775455 valid_y_row_norms_mean: 0.494562119246 valid_y_row_norms_min: 0.114525236189 Time this epoch: 35.012249 seconds Monitoring step: Epochs seen: 6 Batches seen: 30 Examples seen: 300000 ave_grad_mult: 0.849331319332 ave_grad_size: 0.110973127186 ave_step_size: 0.0771789103746 test_h0_col_norms_max: 6.25832700729 test_h0_col_norms_mean: 3.85529947281 test_h0_col_norms_min: 2.08869576454 test_h0_max_x_max_u: 0.999991595745 test_h0_max_x_mean_u: 0.93943220377 test_h0_max_x_min_u: 0.680398881435 test_h0_mean_x_max_u: 0.887371778488 test_h0_mean_x_mean_u: 0.472293674946 test_h0_mean_x_min_u: 0.139431104064 test_h0_min_x_max_u: 0.367107391357 test_h0_min_x_mean_u: 0.0457468703389 test_h0_min_x_min_u: 3.69549866264e-06 test_h0_row_norms_max: 5.9600777626 test_h0_row_norms_mean: 3.01261997223 test_h0_row_norms_min: 0.0412151031196 test_objective: 0.222071394324 test_y_col_norms_max: 4.16519927979 test_y_col_norms_mean: 3.85762476921 test_y_col_norms_min: 3.61017894745 test_y_max_max_class: 0.999991238117 test_y_mean_max_class: 0.913735508919 test_y_min_max_class: 0.246407344937 test_y_misclass: 0.0631999969482 test_y_nll: 0.222071394324 test_y_row_norms_max: 1.19918644428 test_y_row_norms_mean: 0.517221450806 test_y_row_norms_min: 0.117476500571 train_h0_col_norms_max: 6.25832748413 train_h0_col_norms_mean: 3.85529899597 train_h0_col_norms_min: 2.08869576454 train_h0_max_x_max_u: 0.999979615211 train_h0_max_x_mean_u: 0.94024169445 train_h0_max_x_min_u: 0.675026059151 train_h0_mean_x_max_u: 0.87550008297 train_h0_mean_x_mean_u: 0.472564071417 train_h0_mean_x_min_u: 0.142730906606 train_h0_min_x_max_u: 0.356041908264 train_h0_min_x_mean_u: 0.044754832983 train_h0_min_x_min_u: 6.11660334471e-06 train_h0_row_norms_max: 5.96007823944 train_h0_row_norms_mean: 3.01261997223 train_h0_row_norms_min: 0.0412151031196 train_objective: 0.222275063396 train_y_col_norms_max: 4.16519880295 train_y_col_norms_mean: 3.85762453079 train_y_col_norms_min: 3.61017894745 train_y_max_max_class: 0.999996602535 train_y_mean_max_class: 0.910623729229 train_y_min_max_class: 0.235357835889 train_y_misclass: 0.062839999795 train_y_nll: 0.222275063396 train_y_row_norms_max: 1.19918644428 train_y_row_norms_mean: 0.517221450806 train_y_row_norms_min: 0.11747649312 valid_h0_col_norms_max: 6.25832700729 valid_h0_col_norms_mean: 3.85529947281 valid_h0_col_norms_min: 2.08869576454 valid_h0_max_x_max_u: 0.999989330769 valid_h0_max_x_mean_u: 0.939590632915 valid_h0_max_x_min_u: 0.678366243839 valid_h0_mean_x_max_u: 0.879810392857 valid_h0_mean_x_mean_u: 0.472620040178 valid_h0_mean_x_min_u: 0.140709280968 valid_h0_min_x_max_u: 0.344533830881 valid_h0_min_x_mean_u: 0.0452971383929 valid_h0_min_x_min_u: 4.94029472975e-06 valid_h0_row_norms_max: 5.9600777626 valid_h0_row_norms_mean: 3.01261997223 valid_h0_row_norms_min: 0.0412151031196 valid_objective: 0.213480621576 valid_y_col_norms_max: 4.16519927979 valid_y_col_norms_mean: 3.85762476921 valid_y_col_norms_min: 3.61017894745 valid_y_max_max_class: 0.999992728233 valid_y_mean_max_class: 0.915528953075 valid_y_min_max_class: 0.230840429664 valid_y_misclass: 0.0590999983251 valid_y_nll: 0.213480621576 valid_y_row_norms_max: 1.19918644428 valid_y_row_norms_mean: 0.517221450806 valid_y_row_norms_min: 0.117476500571 Time this epoch: 34.796789 seconds Monitoring step: Epochs seen: 7 Batches seen: 35 Examples seen: 350000 ave_grad_mult: 0.921035170555 ave_grad_size: 0.0949304848909 ave_step_size: 0.0732585340738 test_h0_col_norms_max: 6.26188564301 test_h0_col_norms_mean: 3.86070275307 test_h0_col_norms_min: 2.09020781517 test_h0_max_x_max_u: 0.999995708466 test_h0_max_x_mean_u: 0.940146625042 test_h0_max_x_min_u: 0.672576725483 test_h0_mean_x_max_u: 0.892456889153 test_h0_mean_x_mean_u: 0.47117972374 test_h0_mean_x_min_u: 0.127655550838 test_h0_min_x_max_u: 0.367071986198 test_h0_min_x_mean_u: 0.0451025255024 test_h0_min_x_min_u: 1.38111693104e-06 test_h0_row_norms_max: 5.97794675827 test_h0_row_norms_mean: 3.01733326912 test_h0_row_norms_min: 0.0475185476243 test_objective: 0.2069362849 test_y_col_norms_max: 4.37119436264 test_y_col_norms_mean: 4.05648756027 test_y_col_norms_min: 3.72235488892 test_y_max_max_class: 0.999992549419 test_y_mean_max_class: 0.920760273933 test_y_min_max_class: 0.212535321712 test_y_misclass: 0.0597999989986 test_y_nll: 0.2069362849 test_y_row_norms_max: 1.28081488609 test_y_row_norms_mean: 0.54237049818 test_y_row_norms_min: 0.120768107474 train_h0_col_norms_max: 6.26188564301 train_h0_col_norms_mean: 3.86070251465 train_h0_col_norms_min: 2.09020781517 train_h0_max_x_max_u: 0.999989151955 train_h0_max_x_mean_u: 0.941006839275 train_h0_max_x_min_u: 0.670265555382 train_h0_mean_x_max_u: 0.880909919739 train_h0_mean_x_mean_u: 0.471454769373 train_h0_mean_x_min_u: 0.130571871996 train_h0_min_x_max_u: 0.354819297791 train_h0_min_x_mean_u: 0.0440064184368 train_h0_min_x_min_u: 2.32596198657e-06 train_h0_row_norms_max: 5.97794628143 train_h0_row_norms_mean: 3.0173330307 train_h0_row_norms_min: 0.0475185438991 train_objective: 0.205675914884 train_y_col_norms_max: 4.3711938858 train_y_col_norms_mean: 4.05648708344 train_y_col_norms_min: 3.72235488892 train_y_max_max_class: 0.999997496605 train_y_mean_max_class: 0.917994856834 train_y_min_max_class: 0.242114007473 train_y_misclass: 0.0586799941957 train_y_nll: 0.205675914884 train_y_row_norms_max: 1.2808150053 train_y_row_norms_mean: 0.542370438576 train_y_row_norms_min: 0.120768100023 valid_h0_col_norms_max: 6.26188564301 valid_h0_col_norms_mean: 3.86070275307 valid_h0_col_norms_min: 2.09020781517 valid_h0_max_x_max_u: 0.999994754791 valid_h0_max_x_mean_u: 0.940389454365 valid_h0_max_x_min_u: 0.653915822506 valid_h0_mean_x_max_u: 0.885270357132 valid_h0_mean_x_mean_u: 0.471503049135 valid_h0_mean_x_min_u: 0.129038855433 valid_h0_min_x_max_u: 0.343496620655 valid_h0_min_x_mean_u: 0.0445692464709 valid_h0_min_x_min_u: 1.89789943761e-06 valid_h0_row_norms_max: 5.97794675827 valid_h0_row_norms_mean: 3.01733326912 valid_h0_row_norms_min: 0.0475185476243 valid_objective: 0.199690312147 valid_y_col_norms_max: 4.37119436264 valid_y_col_norms_mean: 4.05648756027 valid_y_col_norms_min: 3.72235488892 valid_y_max_max_class: 0.999996244907 valid_y_mean_max_class: 0.922058641911 valid_y_min_max_class: 0.22336602211 valid_y_misclass: 0.055799998343 valid_y_nll: 0.199690312147 valid_y_row_norms_max: 1.28081488609 valid_y_row_norms_mean: 0.54237049818 valid_y_row_norms_min: 0.120768107474 Time this epoch: 34.805092 seconds Monitoring step: Epochs seen: 8 Batches seen: 40 Examples seen: 400000 ave_grad_mult: 0.991648554802 ave_grad_size: 0.0825677365065 ave_step_size: 0.0698289051652 test_h0_col_norms_max: 6.26615095139 test_h0_col_norms_mean: 3.86642217636 test_h0_col_norms_min: 2.0920112133 test_h0_max_x_max_u: 0.999997377396 test_h0_max_x_mean_u: 0.940795004368 test_h0_max_x_min_u: 0.66545778513 test_h0_mean_x_max_u: 0.901528179646 test_h0_mean_x_mean_u: 0.470299869776 test_h0_mean_x_min_u: 0.121718779206 test_h0_min_x_max_u: 0.370387345552 test_h0_min_x_mean_u: 0.0449309423566 test_h0_min_x_min_u: 5.55576320949e-07 test_h0_row_norms_max: 5.99863862991 test_h0_row_norms_mean: 3.02230143547 test_h0_row_norms_min: 0.0541109740734 test_objective: 0.1924007833 test_y_col_norms_max: 4.68016433716 test_y_col_norms_mean: 4.25164651871 test_y_col_norms_min: 3.82015967369 test_y_max_max_class: 0.999988377094 test_y_mean_max_class: 0.924113929272 test_y_min_max_class: 0.210057422519 test_y_misclass: 0.0555000007153 test_y_nll: 0.1924007833 test_y_row_norms_max: 1.36218941212 test_y_row_norms_mean: 0.566706836224 test_y_row_norms_min: 0.123096778989 train_h0_col_norms_max: 6.26615047455 train_h0_col_norms_mean: 3.86642193794 train_h0_col_norms_min: 2.0920112133 train_h0_max_x_max_u: 0.999993860722 train_h0_max_x_mean_u: 0.941651582718 train_h0_max_x_min_u: 0.657282650471 train_h0_mean_x_max_u: 0.89084905386 train_h0_mean_x_mean_u: 0.470575273037 train_h0_mean_x_min_u: 0.124335050583 train_h0_min_x_max_u: 0.357388138771 train_h0_min_x_mean_u: 0.0438850969076 train_h0_min_x_min_u: 9.09530456283e-07 train_h0_row_norms_max: 5.99863910675 train_h0_row_norms_mean: 3.02230119705 train_h0_row_norms_min: 0.0541109666228 train_objective: 0.187867701054 train_y_col_norms_max: 4.680164814 train_y_col_norms_mean: 4.25164651871 train_y_col_norms_min: 3.82015943527 train_y_max_max_class: 0.999996304512 train_y_mean_max_class: 0.922073721886 train_y_min_max_class: 0.237471118569 train_y_misclass: 0.0530999973416 train_y_nll: 0.187867701054 train_y_row_norms_max: 1.36218929291 train_y_row_norms_mean: 0.566706776619 train_y_row_norms_min: 0.123096778989 valid_h0_col_norms_max: 6.26615095139 valid_h0_col_norms_mean: 3.86642217636 valid_h0_col_norms_min: 2.0920112133 valid_h0_max_x_max_u: 0.999996244907 valid_h0_max_x_mean_u: 0.940959215164 valid_h0_max_x_min_u: 0.634269952774 valid_h0_mean_x_max_u: 0.894827961922 valid_h0_mean_x_mean_u: 0.470626890659 valid_h0_mean_x_min_u: 0.123129568994 valid_h0_min_x_max_u: 0.344170331955 valid_h0_min_x_mean_u: 0.0444831475616 valid_h0_min_x_min_u: 7.30816509531e-07 valid_h0_row_norms_max: 5.99863862991 valid_h0_row_norms_mean: 3.02230143547 valid_h0_row_norms_min: 0.0541109740734 valid_objective: 0.184409946203 valid_y_col_norms_max: 4.68016433716 valid_y_col_norms_mean: 4.25164651871 valid_y_col_norms_min: 3.82015967369 valid_y_max_max_class: 0.999994754791 valid_y_mean_max_class: 0.926723182201 valid_y_min_max_class: 0.219980046153 valid_y_misclass: 0.047499999404 valid_y_nll: 0.184409946203 valid_y_row_norms_max: 1.36218941212 valid_y_row_norms_mean: 0.566706836224 valid_y_row_norms_min: 0.123096778989 Time this epoch: 35.663056 seconds Monitoring step: Epochs seen: 9 Batches seen: 45 Examples seen: 450000 ave_grad_mult: 1.00632071495 ave_grad_size: 0.0730155408382 ave_step_size: 0.0651284307241 test_h0_col_norms_max: 6.27027750015 test_h0_col_norms_mean: 3.87175488472 test_h0_col_norms_min: 2.09271168709 test_h0_max_x_max_u: 0.99999833107 test_h0_max_x_mean_u: 0.941553533077 test_h0_max_x_min_u: 0.65441852808 test_h0_mean_x_max_u: 0.903928875923 test_h0_mean_x_mean_u: 0.469605773687 test_h0_mean_x_min_u: 0.114903002977 test_h0_min_x_max_u: 0.373793333769 test_h0_min_x_mean_u: 0.044343251735 test_h0_min_x_min_u: 2.48894650667e-07 test_h0_row_norms_max: 6.01675319672 test_h0_row_norms_mean: 3.0269382 test_h0_row_norms_min: 0.0595724433661 test_objective: 0.178400695324 test_y_col_norms_max: 4.93448925018 test_y_col_norms_mean: 4.4312376976 test_y_col_norms_min: 3.912296772 test_y_max_max_class: 0.99998986721 test_y_mean_max_class: 0.929982662201 test_y_min_max_class: 0.206445708871 test_y_misclass: 0.0520999990404 test_y_nll: 0.178400695324 test_y_row_norms_max: 1.42163467407 test_y_row_norms_mean: 0.588779568672 test_y_row_norms_min: 0.124702431262 train_h0_col_norms_max: 6.27027750015 train_h0_col_norms_mean: 3.87175512314 train_h0_col_norms_min: 2.09271168709 train_h0_max_x_max_u: 0.999995946884 train_h0_max_x_mean_u: 0.942493140697 train_h0_max_x_min_u: 0.638945221901 train_h0_mean_x_max_u: 0.893475353718 train_h0_mean_x_mean_u: 0.469883978367 train_h0_mean_x_min_u: 0.117275975645 train_h0_min_x_max_u: 0.360578835011 train_h0_min_x_mean_u: 0.0432931296527 train_h0_min_x_min_u: 3.94163265582e-07 train_h0_row_norms_max: 6.01675271988 train_h0_row_norms_mean: 3.0269382 train_h0_row_norms_min: 0.0595724396408 train_objective: 0.173733517528 train_y_col_norms_max: 4.93448877335 train_y_col_norms_mean: 4.4312376976 train_y_col_norms_min: 3.91229653358 train_y_max_max_class: 0.999996066093 train_y_mean_max_class: 0.92810434103 train_y_min_max_class: 0.229242756963 train_y_misclass: 0.0490399971604 train_y_nll: 0.173733517528 train_y_row_norms_max: 1.42163455486 train_y_row_norms_mean: 0.588779509068 train_y_row_norms_min: 0.124702423811 valid_h0_col_norms_max: 6.27027750015 valid_h0_col_norms_mean: 3.87175488472 valid_h0_col_norms_min: 2.09271168709 valid_h0_max_x_max_u: 0.999997377396 valid_h0_max_x_mean_u: 0.941749632359 valid_h0_max_x_min_u: 0.622578442097 valid_h0_mean_x_max_u: 0.897465348244 valid_h0_mean_x_mean_u: 0.469932496548 valid_h0_mean_x_min_u: 0.116939790547 valid_h0_min_x_max_u: 0.347404718399 valid_h0_min_x_mean_u: 0.0439214892685 valid_h0_min_x_min_u: 3.13890211601e-07 valid_h0_row_norms_max: 6.01675319672 valid_h0_row_norms_mean: 3.0269382 valid_h0_row_norms_min: 0.0595724433661 valid_objective: 0.172197133303 valid_y_col_norms_max: 4.93448925018 valid_y_col_norms_mean: 4.4312376976 valid_y_col_norms_min: 3.912296772 valid_y_max_max_class: 0.999996781349 valid_y_mean_max_class: 0.932501792908 valid_y_min_max_class: 0.216077208519 valid_y_misclass: 0.0454999953508 valid_y_nll: 0.172197133303 valid_y_row_norms_max: 1.42163467407 valid_y_row_norms_mean: 0.588779568672 valid_y_row_norms_min: 0.124702431262 Time this epoch: 35.404834 seconds Monitoring step: Epochs seen: 10 Batches seen: 50 Examples seen: 500000 ave_grad_mult: 1.06833612919 ave_grad_size: 0.0678643658757 ave_step_size: 0.0653440654278 test_h0_col_norms_max: 6.27522420883 test_h0_col_norms_mean: 3.87793588638 test_h0_col_norms_min: 2.09417295456 test_h0_max_x_max_u: 0.999998867512 test_h0_max_x_mean_u: 0.942130804062 test_h0_max_x_min_u: 0.645175695419 test_h0_mean_x_max_u: 0.909636974335 test_h0_mean_x_mean_u: 0.468845933676 test_h0_mean_x_min_u: 0.104815065861 test_h0_min_x_max_u: 0.378569096327 test_h0_min_x_mean_u: 0.0440588444471 test_h0_min_x_min_u: 1.15133666156e-07 test_h0_row_norms_max: 6.03866481781 test_h0_row_norms_mean: 3.03230404854 test_h0_row_norms_min: 0.065353885293 test_objective: 0.167283341289 test_y_col_norms_max: 5.2253780365 test_y_col_norms_mean: 4.62542486191 test_y_col_norms_min: 4.01688957214 test_y_max_max_class: 0.999992907047 test_y_mean_max_class: 0.933511257172 test_y_min_max_class: 0.242168530822 test_y_misclass: 0.0492999963462 test_y_nll: 0.167283341289 test_y_row_norms_max: 1.50107598305 test_y_row_norms_mean: 0.612406551838 test_y_row_norms_min: 0.125712171197 train_h0_col_norms_max: 6.27522373199 train_h0_col_norms_mean: 3.87793540955 train_h0_col_norms_min: 2.09417295456 train_h0_max_x_max_u: 0.999997496605 train_h0_max_x_mean_u: 0.943212330341 train_h0_max_x_min_u: 0.628583967686 train_h0_mean_x_max_u: 0.899803757668 train_h0_mean_x_mean_u: 0.469121694565 train_h0_mean_x_min_u: 0.107625767589 train_h0_min_x_max_u: 0.36565092206 train_h0_min_x_mean_u: 0.0430302321911 train_h0_min_x_min_u: 1.75549445203e-07 train_h0_row_norms_max: 6.03866481781 train_h0_row_norms_mean: 3.03230404854 train_h0_row_norms_min: 0.065353885293 train_objective: 0.159167990088 train_y_col_norms_max: 5.22537755966 train_y_col_norms_mean: 4.62542486191 train_y_col_norms_min: 4.01688957214 train_y_max_max_class: 0.999997138977 train_y_mean_max_class: 0.931973934174 train_y_min_max_class: 0.241810530424 train_y_misclass: 0.0449799969792 train_y_nll: 0.159167990088 train_y_row_norms_max: 1.50107610226 train_y_row_norms_mean: 0.612406492233 train_y_row_norms_min: 0.125712156296 valid_h0_col_norms_max: 6.27522420883 valid_h0_col_norms_mean: 3.87793588638 valid_h0_col_norms_min: 2.09417295456 valid_h0_max_x_max_u: 0.999998152256 valid_h0_max_x_mean_u: 0.942497193813 valid_h0_max_x_min_u: 0.619423508644 valid_h0_mean_x_max_u: 0.903488636017 valid_h0_mean_x_mean_u: 0.469177812338 valid_h0_mean_x_min_u: 0.108095638454 valid_h0_min_x_max_u: 0.349716216326 valid_h0_min_x_mean_u: 0.04355686903 valid_h0_min_x_min_u: 1.34484281489e-07 valid_h0_row_norms_max: 6.03866481781 valid_h0_row_norms_mean: 3.03230404854 valid_h0_row_norms_min: 0.065353885293 valid_objective: 0.160998404026 valid_y_col_norms_max: 5.2253780365 valid_y_col_norms_mean: 4.62542486191 valid_y_col_norms_min: 4.01688957214 valid_y_max_max_class: 0.999998152256 valid_y_mean_max_class: 0.936175227165 valid_y_min_max_class: 0.220791786909 valid_y_misclass: 0.0441000014544 valid_y_nll: 0.160998404026 valid_y_row_norms_max: 1.50107598305 valid_y_row_norms_mean: 0.612406551838 valid_y_row_norms_min: 0.125712171197 Time this epoch: 35.425083 seconds Monitoring step: Epochs seen: 11 Batches seen: 55 Examples seen: 550000 ave_grad_mult: 1.14648592472 ave_grad_size: 0.0634888410568 ave_step_size: 0.0666681230068 test_h0_col_norms_max: 6.28032588959 test_h0_col_norms_mean: 3.88434314728 test_h0_col_norms_min: 2.09576916695 test_h0_max_x_max_u: 0.999999403954 test_h0_max_x_mean_u: 0.942800343037 test_h0_max_x_min_u: 0.63667178154 test_h0_mean_x_max_u: 0.915984809399 test_h0_mean_x_mean_u: 0.468008965254 test_h0_mean_x_min_u: 0.101051539183 test_h0_min_x_max_u: 0.390204340219 test_h0_min_x_mean_u: 0.0434938073158 test_h0_min_x_min_u: 5.66224080956e-08 test_h0_row_norms_max: 6.06034469604 test_h0_row_norms_mean: 3.03785538673 test_h0_row_norms_min: 0.0704936757684 test_objective: 0.15456405282 test_y_col_norms_max: 5.50095510483 test_y_col_norms_mean: 4.82304191589 test_y_col_norms_min: 4.1173620224 test_y_max_max_class: 0.999992728233 test_y_mean_max_class: 0.936915397644 test_y_min_max_class: 0.252786010504 test_y_misclass: 0.0443000011146 test_y_nll: 0.15456405282 test_y_row_norms_max: 1.60092997551 test_y_row_norms_mean: 0.636273026466 test_y_row_norms_min: 0.124862372875 train_h0_col_norms_max: 6.28032636642 train_h0_col_norms_mean: 3.88434290886 train_h0_col_norms_min: 2.09576892853 train_h0_max_x_max_u: 0.999998629093 train_h0_max_x_mean_u: 0.944033026695 train_h0_max_x_min_u: 0.631079792976 train_h0_mean_x_max_u: 0.90686249733 train_h0_mean_x_mean_u: 0.468293100595 train_h0_mean_x_min_u: 0.103928506374 train_h0_min_x_max_u: 0.373679548502 train_h0_min_x_mean_u: 0.0424839258194 train_h0_min_x_min_u: 8.3652395233e-08 train_h0_row_norms_max: 6.06034517288 train_h0_row_norms_mean: 3.03785514832 train_h0_row_norms_min: 0.0704936683178 train_objective: 0.146077007055 train_y_col_norms_max: 5.50095510483 train_y_col_norms_mean: 4.82304239273 train_y_col_norms_min: 4.11736249924 train_y_max_max_class: 0.999997377396 train_y_mean_max_class: 0.935088992119 train_y_min_max_class: 0.235717624426 train_y_misclass: 0.0411599949002 train_y_nll: 0.146077007055 train_y_row_norms_max: 1.60092973709 train_y_row_norms_mean: 0.636273086071 train_y_row_norms_min: 0.124862357974 valid_h0_col_norms_max: 6.28032588959 valid_h0_col_norms_mean: 3.88434314728 valid_h0_col_norms_min: 2.09576916695 valid_h0_max_x_max_u: 0.999998867512 valid_h0_max_x_mean_u: 0.943427741528 valid_h0_max_x_min_u: 0.627752363682 valid_h0_mean_x_max_u: 0.910161554813 valid_h0_mean_x_mean_u: 0.468341171741 valid_h0_mean_x_min_u: 0.104510381818 valid_h0_min_x_max_u: 0.357529014349 valid_h0_min_x_mean_u: 0.0429090820253 valid_h0_min_x_min_u: 6.19904838572e-08 valid_h0_row_norms_max: 6.06034469604 valid_h0_row_norms_mean: 3.03785538673 valid_h0_row_norms_min: 0.0704936757684 valid_objective: 0.149976089597 valid_y_col_norms_max: 5.50095510483 valid_y_col_norms_mean: 4.82304191589 valid_y_col_norms_min: 4.1173620224 valid_y_max_max_class: 0.999998509884 valid_y_mean_max_class: 0.939062952995 valid_y_min_max_class: 0.239928662777 valid_y_misclass: 0.0416000001132 valid_y_nll: 0.149976089597 valid_y_row_norms_max: 1.60092997551 valid_y_row_norms_mean: 0.636273026466 valid_y_row_norms_min: 0.124862372875 Time this epoch: 35.174293 seconds Monitoring step: Epochs seen: 12 Batches seen: 60 Examples seen: 600000 ave_grad_mult: 1.16790962219 ave_grad_size: 0.0593062080443 ave_step_size: 0.0650760680437 test_h0_col_norms_max: 6.28521823883 test_h0_col_norms_mean: 3.89019036293 test_h0_col_norms_min: 2.09752202034 test_h0_max_x_max_u: 0.999999582767 test_h0_max_x_mean_u: 0.94396853447 test_h0_max_x_min_u: 0.629440486431 test_h0_mean_x_max_u: 0.920006334782 test_h0_mean_x_mean_u: 0.467411011457 test_h0_mean_x_min_u: 0.0957048162818 test_h0_min_x_max_u: 0.389512062073 test_h0_min_x_mean_u: 0.0425479598343 test_h0_min_x_min_u: 3.20807167498e-08 test_h0_row_norms_max: 6.08012914658 test_h0_row_norms_mean: 3.04289364815 test_h0_row_norms_min: 0.0743318274617 test_objective: 0.144802451134 test_y_col_norms_max: 5.74849033356 test_y_col_norms_mean: 5.00328540802 test_y_col_norms_min: 4.21305179596 test_y_max_max_class: 0.999994158745 test_y_mean_max_class: 0.941429018974 test_y_min_max_class: 0.231030538678 test_y_misclass: 0.0408000014722 test_y_nll: 0.144802451134 test_y_row_norms_max: 1.7184125185 test_y_row_norms_mean: 0.658156752586 test_y_row_norms_min: 0.125041946769 train_h0_col_norms_max: 6.28521871567 train_h0_col_norms_mean: 3.89019012451 train_h0_col_norms_min: 2.09752202034 train_h0_max_x_max_u: 0.999999046326 train_h0_max_x_mean_u: 0.945232570171 train_h0_max_x_min_u: 0.634238958359 train_h0_mean_x_max_u: 0.911378622055 train_h0_mean_x_mean_u: 0.467698544264 train_h0_mean_x_min_u: 0.0993719547987 train_h0_min_x_max_u: 0.373709738255 train_h0_min_x_mean_u: 0.0415380932391 train_h0_min_x_min_u: 4.67483047828e-08 train_h0_row_norms_max: 6.08012914658 train_h0_row_norms_mean: 3.04289340973 train_h0_row_norms_min: 0.0743318200111 train_objective: 0.135217413306 train_y_col_norms_max: 5.74849033356 train_y_col_norms_mean: 5.00328493118 train_y_col_norms_min: 4.21305131912 train_y_max_max_class: 0.999997973442 train_y_mean_max_class: 0.939604878426 train_y_min_max_class: 0.252161383629 train_y_misclass: 0.0378999970853 train_y_nll: 0.135217413306 train_y_row_norms_max: 1.71841263771 train_y_row_norms_mean: 0.658156752586 train_y_row_norms_min: 0.125041931868 valid_h0_col_norms_max: 6.28521823883 valid_h0_col_norms_mean: 3.89019036293 valid_h0_col_norms_min: 2.09752202034 valid_h0_max_x_max_u: 0.99999922514 valid_h0_max_x_mean_u: 0.944651842117 valid_h0_max_x_min_u: 0.645568966866 valid_h0_mean_x_max_u: 0.914412498474 valid_h0_mean_x_mean_u: 0.467748105526 valid_h0_mean_x_min_u: 0.0972835198045 valid_h0_min_x_max_u: 0.359061449766 valid_h0_min_x_mean_u: 0.0419331230223 valid_h0_min_x_min_u: 3.37856427279e-08 valid_h0_row_norms_max: 6.08012914658 valid_h0_row_norms_mean: 3.04289364815 valid_h0_row_norms_min: 0.0743318274617 valid_objective: 0.141469165683 valid_y_col_norms_max: 5.74849033356 valid_y_col_norms_mean: 5.00328540802 valid_y_col_norms_min: 4.21305179596 valid_y_max_max_class: 0.999998867512 valid_y_mean_max_class: 0.943582773209 valid_y_min_max_class: 0.241308540106 valid_y_misclass: 0.0379000008106 valid_y_nll: 0.141469165683 valid_y_row_norms_max: 1.7184125185 valid_y_row_norms_mean: 0.658156752586 valid_y_row_norms_min: 0.125041946769 Time this epoch: 35.417259 seconds Monitoring step: Epochs seen: 13 Batches seen: 65 Examples seen: 650000 ave_grad_mult: 1.26017534733 ave_grad_size: 0.0564817748964 ave_step_size: 0.066411331296 test_h0_col_norms_max: 6.29147386551 test_h0_col_norms_mean: 3.89687585831 test_h0_col_norms_min: 2.09867763519 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.944756031036 test_h0_max_x_min_u: 0.627687215805 test_h0_mean_x_max_u: 0.920987069607 test_h0_mean_x_mean_u: 0.46668151021 test_h0_mean_x_min_u: 0.0932114943862 test_h0_min_x_max_u: 0.397424399853 test_h0_min_x_mean_u: 0.041878964752 test_h0_min_x_min_u: 1.39939251298e-08 test_h0_row_norms_max: 6.10229206085 test_h0_row_norms_mean: 3.04865932465 test_h0_row_norms_min: 0.0787999555469 test_objective: 0.135829210281 test_y_col_norms_max: 6.01105213165 test_y_col_norms_mean: 5.20304250717 test_y_col_norms_min: 4.33085250854 test_y_max_max_class: 0.999994754791 test_y_mean_max_class: 0.945015072823 test_y_min_max_class: 0.230172082782 test_y_misclass: 0.0381999947131 test_y_nll: 0.135829210281 test_y_row_norms_max: 1.85168874264 test_y_row_norms_mean: 0.682141900063 test_y_row_norms_min: 0.125363498926 train_h0_col_norms_max: 6.29147338867 train_h0_col_norms_mean: 3.89687561989 train_h0_col_norms_min: 2.09867739677 train_h0_max_x_max_u: 0.999999403954 train_h0_max_x_mean_u: 0.946107804775 train_h0_max_x_min_u: 0.63179987669 train_h0_mean_x_max_u: 0.912519574165 train_h0_mean_x_mean_u: 0.466963618994 train_h0_mean_x_min_u: 0.0961530357599 train_h0_min_x_max_u: 0.379027783871 train_h0_min_x_mean_u: 0.0408683530986 train_h0_min_x_min_u: 1.94427727251e-08 train_h0_row_norms_max: 6.10229253769 train_h0_row_norms_mean: 3.04865932465 train_h0_row_norms_min: 0.0787999555469 train_objective: 0.12386597693 train_y_col_norms_max: 6.01105213165 train_y_col_norms_mean: 5.20304203033 train_y_col_norms_min: 4.33085203171 train_y_max_max_class: 0.999997973442 train_y_mean_max_class: 0.943518102169 train_y_min_max_class: 0.246507614851 train_y_misclass: 0.034559994936 train_y_nll: 0.12386597693 train_y_row_norms_max: 1.85168862343 train_y_row_norms_mean: 0.682141840458 train_y_row_norms_min: 0.125363498926 valid_h0_col_norms_max: 6.29147386551 valid_h0_col_norms_mean: 3.89687585831 valid_h0_col_norms_min: 2.09867763519 valid_h0_max_x_max_u: 0.999999403954 valid_h0_max_x_mean_u: 0.945632517338 valid_h0_max_x_min_u: 0.651219964027 valid_h0_mean_x_max_u: 0.915507853031 valid_h0_mean_x_mean_u: 0.467023015022 valid_h0_mean_x_min_u: 0.0969914197922 valid_h0_min_x_max_u: 0.364903271198 valid_h0_min_x_mean_u: 0.0411523580551 valid_h0_min_x_min_u: 1.40916096569e-08 valid_h0_row_norms_max: 6.10229206085 valid_h0_row_norms_mean: 3.04865932465 valid_h0_row_norms_min: 0.0787999555469 valid_objective: 0.133389517665 valid_y_col_norms_max: 6.01105213165 valid_y_col_norms_mean: 5.20304250717 valid_y_col_norms_min: 4.33085250854 valid_y_max_max_class: 0.999999165535 valid_y_mean_max_class: 0.946852385998 valid_y_min_max_class: 0.214304342866 valid_y_misclass: 0.03579999879 valid_y_nll: 0.133389517665 valid_y_row_norms_max: 1.85168874264 valid_y_row_norms_mean: 0.682141900063 valid_y_row_norms_min: 0.125363498926 Time this epoch: 35.366187 seconds Monitoring step: Epochs seen: 14 Batches seen: 70 Examples seen: 700000 ave_grad_mult: 1.40761697292 ave_grad_size: 0.0550340935588 ave_step_size: 0.0714166760445 test_h0_col_norms_max: 6.29854393005 test_h0_col_norms_mean: 3.90459442139 test_h0_col_norms_min: 2.1004254818 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.94591987133 test_h0_max_x_min_u: 0.614766418934 test_h0_mean_x_max_u: 0.921078026295 test_h0_mean_x_mean_u: 0.466091096401 test_h0_mean_x_min_u: 0.0916782915592 test_h0_min_x_max_u: 0.396448850632 test_h0_min_x_mean_u: 0.0410171151161 test_h0_min_x_min_u: 7.75897479599e-09 test_h0_row_norms_max: 6.12455701828 test_h0_row_norms_mean: 3.05531406403 test_h0_row_norms_min: 0.0834630578756 test_objective: 0.125504016876 test_y_col_norms_max: 6.29601860046 test_y_col_norms_mean: 5.42890501022 test_y_col_norms_min: 4.46609354019 test_y_max_max_class: 0.999994575977 test_y_mean_max_class: 0.949410498142 test_y_min_max_class: 0.201501131058 test_y_misclass: 0.0355000011623 test_y_nll: 0.125504016876 test_y_row_norms_max: 2.01360034943 test_y_row_norms_mean: 0.709331393242 test_y_row_norms_min: 0.125074863434 train_h0_col_norms_max: 6.29854393005 train_h0_col_norms_mean: 3.90459418297 train_h0_col_norms_min: 2.10042524338 train_h0_max_x_max_u: 0.999999523163 train_h0_max_x_mean_u: 0.947336554527 train_h0_max_x_min_u: 0.624508261681 train_h0_mean_x_max_u: 0.912684559822 train_h0_mean_x_mean_u: 0.466372013092 train_h0_mean_x_min_u: 0.0946839675307 train_h0_min_x_max_u: 0.383265286684 train_h0_min_x_mean_u: 0.0400523841381 train_h0_min_x_min_u: 1.04573256721e-08 train_h0_row_norms_max: 6.12455654144 train_h0_row_norms_mean: 3.05531382561 train_h0_row_norms_min: 0.0834630504251 train_objective: 0.112524747849 train_y_col_norms_max: 6.29601955414 train_y_col_norms_mean: 5.42890501022 train_y_col_norms_min: 4.46609306335 train_y_max_max_class: 0.999997973442 train_y_mean_max_class: 0.948245584965 train_y_min_max_class: 0.237888276577 train_y_misclass: 0.031159998849 train_y_nll: 0.112524747849 train_y_row_norms_max: 2.01360034943 train_y_row_norms_mean: 0.709331333637 train_y_row_norms_min: 0.125074848533 valid_h0_col_norms_max: 6.29854393005 valid_h0_col_norms_mean: 3.90459442139 valid_h0_col_norms_min: 2.1004254818 valid_h0_max_x_max_u: 0.999999582767 valid_h0_max_x_mean_u: 0.946813523769 valid_h0_max_x_min_u: 0.649647653103 valid_h0_mean_x_max_u: 0.915705919266 valid_h0_mean_x_mean_u: 0.466423898935 valid_h0_mean_x_min_u: 0.0953802764416 valid_h0_min_x_max_u: 0.369607925415 valid_h0_min_x_mean_u: 0.0402967631817 valid_h0_min_x_min_u: 7.3467920636e-09 valid_h0_row_norms_max: 6.12455701828 valid_h0_row_norms_mean: 3.05531406403 valid_h0_row_norms_min: 0.0834630578756 valid_objective: 0.124651312828 valid_y_col_norms_max: 6.29601860046 valid_y_col_norms_mean: 5.42890501022 valid_y_col_norms_min: 4.46609354019 valid_y_max_max_class: 0.999999046326 valid_y_mean_max_class: 0.950519561768 valid_y_min_max_class: 0.27420938015 valid_y_misclass: 0.0340000018477 valid_y_nll: 0.124651312828 valid_y_row_norms_max: 2.01360034943 valid_y_row_norms_mean: 0.709331393242 valid_y_row_norms_min: 0.125074863434 Time this epoch: 35.379965 seconds Monitoring step: Epochs seen: 15 Batches seen: 75 Examples seen: 750000 ave_grad_mult: 1.47251427174 ave_grad_size: 0.0522134304047 ave_step_size: 0.071938700974 test_h0_col_norms_max: 6.30543804169 test_h0_col_norms_mean: 3.91161727905 test_h0_col_norms_min: 2.10170149803 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.947175860405 test_h0_max_x_min_u: 0.611504435539 test_h0_mean_x_max_u: 0.926122069359 test_h0_mean_x_mean_u: 0.466326773167 test_h0_mean_x_min_u: 0.0923119410872 test_h0_min_x_max_u: 0.401742935181 test_h0_min_x_mean_u: 0.040495429188 test_h0_min_x_min_u: 3.92714794017e-09 test_h0_row_norms_max: 6.1450252533 test_h0_row_norms_mean: 3.0613322258 test_h0_row_norms_min: 0.0881084352732 test_objective: 0.118968196213 test_y_col_norms_max: 6.55995035172 test_y_col_norms_mean: 5.62976980209 test_y_col_norms_min: 4.59543800354 test_y_max_max_class: 0.999994218349 test_y_mean_max_class: 0.951547503471 test_y_min_max_class: 0.228803291917 test_y_misclass: 0.0349999964237 test_y_nll: 0.118968196213 test_y_row_norms_max: 2.14034724236 test_y_row_norms_mean: 0.733418226242 test_y_row_norms_min: 0.12729588151 train_h0_col_norms_max: 6.30543756485 train_h0_col_norms_mean: 3.91161704063 train_h0_col_norms_min: 2.10170149803 train_h0_max_x_max_u: 0.999999880791 train_h0_max_x_mean_u: 0.948555886745 train_h0_max_x_min_u: 0.618095517159 train_h0_mean_x_max_u: 0.918341517448 train_h0_mean_x_mean_u: 0.466599404812 train_h0_mean_x_min_u: 0.0956889539957 train_h0_min_x_max_u: 0.392006248236 train_h0_min_x_mean_u: 0.0395230464637 train_h0_min_x_min_u: 5.12299225264e-09 train_h0_row_norms_max: 6.14502477646 train_h0_row_norms_mean: 3.06133174896 train_h0_row_norms_min: 0.088108420372 train_objective: 0.103299617767 train_y_col_norms_max: 6.55995082855 train_y_col_norms_mean: 5.62976932526 train_y_col_norms_min: 4.5954375267 train_y_max_max_class: 0.999997377396 train_y_mean_max_class: 0.951093494892 train_y_min_max_class: 0.249234974384 train_y_misclass: 0.0284799989313 train_y_nll: 0.103299617767 train_y_row_norms_max: 2.14034700394 train_y_row_norms_mean: 0.733418226242 train_y_row_norms_min: 0.12729588151 valid_h0_col_norms_max: 6.30543804169 valid_h0_col_norms_mean: 3.91161727905 valid_h0_col_norms_min: 2.10170149803 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.947986066341 valid_h0_max_x_min_u: 0.642793953419 valid_h0_mean_x_max_u: 0.920964062214 valid_h0_mean_x_mean_u: 0.466656267643 valid_h0_mean_x_min_u: 0.0940045118332 valid_h0_min_x_max_u: 0.377251118422 valid_h0_min_x_mean_u: 0.03970798105 valid_h0_min_x_min_u: 3.59755203405e-09 valid_h0_row_norms_max: 6.1450252533 valid_h0_row_norms_mean: 3.0613322258 valid_h0_row_norms_min: 0.0881084352732 valid_objective: 0.119057364762 valid_y_col_norms_max: 6.55995035172 valid_y_col_norms_mean: 5.62976980209 valid_y_col_norms_min: 4.59543800354 valid_y_max_max_class: 0.999998807907 valid_y_mean_max_class: 0.953496754169 valid_y_min_max_class: 0.279151201248 valid_y_misclass: 0.0322999954224 valid_y_nll: 0.119057364762 valid_y_row_norms_max: 2.14034724236 valid_y_row_norms_mean: 0.733418226242 valid_y_row_norms_min: 0.12729588151 Time this epoch: 35.163641 seconds Monitoring step: Epochs seen: 16 Batches seen: 80 Examples seen: 800000 ave_grad_mult: 1.55044400692 ave_grad_size: 0.0495749413967 ave_step_size: 0.071437291801 test_h0_col_norms_max: 6.31254959106 test_h0_col_norms_mean: 3.91860723495 test_h0_col_norms_min: 2.10440206528 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.948059678078 test_h0_max_x_min_u: 0.594348907471 test_h0_mean_x_max_u: 0.927690863609 test_h0_mean_x_mean_u: 0.465816915035 test_h0_mean_x_min_u: 0.0875093266368 test_h0_min_x_max_u: 0.399484992027 test_h0_min_x_mean_u: 0.0397286936641 test_h0_min_x_min_u: 2.14466333581e-09 test_h0_row_norms_max: 6.16418838501 test_h0_row_norms_mean: 3.06732678413 test_h0_row_norms_min: 0.091041892767 test_objective: 0.111787736416 test_y_col_norms_max: 6.79865264893 test_y_col_norms_mean: 5.8274474144 test_y_col_norms_min: 4.71656274796 test_y_max_max_class: 0.999996483326 test_y_mean_max_class: 0.954726696014 test_y_min_max_class: 0.287018150091 test_y_misclass: 0.0328000001609 test_y_nll: 0.111787736416 test_y_row_norms_max: 2.27131104469 test_y_row_norms_mean: 0.757337749004 test_y_row_norms_min: 0.12875507772 train_h0_col_norms_max: 6.31254959106 train_h0_col_norms_mean: 3.91860699654 train_h0_col_norms_min: 2.10440182686 train_h0_max_x_max_u: 0.999999880791 train_h0_max_x_mean_u: 0.949532628059 train_h0_max_x_min_u: 0.603366672993 train_h0_mean_x_max_u: 0.920095324516 train_h0_mean_x_mean_u: 0.466088950634 train_h0_mean_x_min_u: 0.0905980989337 train_h0_min_x_max_u: 0.391806066036 train_h0_min_x_mean_u: 0.0387711115181 train_h0_min_x_min_u: 2.82344658764e-09 train_h0_row_norms_max: 6.16418838501 train_h0_row_norms_mean: 3.06732654572 train_h0_row_norms_min: 0.0910418853164 train_objective: 0.0944318547845 train_y_col_norms_max: 6.79865264893 train_y_col_norms_mean: 5.82744646072 train_y_col_norms_min: 4.71656322479 train_y_max_max_class: 0.999998569489 train_y_mean_max_class: 0.954577803612 train_y_min_max_class: 0.255649060011 train_y_misclass: 0.0261199977249 train_y_nll: 0.0944318547845 train_y_row_norms_max: 2.27131080627 train_y_row_norms_mean: 0.757337749004 train_y_row_norms_min: 0.128755062819 valid_h0_col_norms_max: 6.31254959106 valid_h0_col_norms_mean: 3.91860723495 valid_h0_col_norms_min: 2.10440206528 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.948857724667 valid_h0_max_x_min_u: 0.6251745224 valid_h0_mean_x_max_u: 0.922635018826 valid_h0_mean_x_mean_u: 0.46614703536 valid_h0_mean_x_min_u: 0.0918302312493 valid_h0_min_x_max_u: 0.379304587841 valid_h0_min_x_mean_u: 0.0389546044171 valid_h0_min_x_min_u: 1.96204941183e-09 valid_h0_row_norms_max: 6.16418838501 valid_h0_row_norms_mean: 3.06732678413 valid_h0_row_norms_min: 0.091041892767 valid_objective: 0.110771089792 valid_y_col_norms_max: 6.79865264893 valid_y_col_norms_mean: 5.8274474144 valid_y_col_norms_min: 4.71656274796 valid_y_max_max_class: 0.999999165535 valid_y_mean_max_class: 0.95663100481 valid_y_min_max_class: 0.264041811228 valid_y_misclass: 0.0305000003427 valid_y_nll: 0.110771089792 valid_y_row_norms_max: 2.27131104469 valid_y_row_norms_mean: 0.757337749004 valid_y_row_norms_min: 0.12875507772 Time this epoch: 35.246666 seconds Monitoring step: Epochs seen: 17 Batches seen: 85 Examples seen: 850000 ave_grad_mult: 1.59982562065 ave_grad_size: 0.0473937280476 ave_step_size: 0.0712730288506 test_h0_col_norms_max: 6.31961965561 test_h0_col_norms_mean: 3.92528343201 test_h0_col_norms_min: 2.10622811317 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.949356853962 test_h0_max_x_min_u: 0.59099650383 test_h0_mean_x_max_u: 0.927693426609 test_h0_mean_x_mean_u: 0.465647280216 test_h0_mean_x_min_u: 0.0867232903838 test_h0_min_x_max_u: 0.39404541254 test_h0_min_x_mean_u: 0.0387796163559 test_h0_min_x_min_u: 1.4791411429e-09 test_h0_row_norms_max: 6.18163251877 test_h0_row_norms_mean: 3.07301926613 test_h0_row_norms_min: 0.0938726961613 test_objective: 0.106328338385 test_y_col_norms_max: 7.01830482483 test_y_col_norms_mean: 6.0149974823 test_y_col_norms_min: 4.83683490753 test_y_max_max_class: 0.999997198582 test_y_mean_max_class: 0.95773011446 test_y_min_max_class: 0.291382759809 test_y_misclass: 0.0320000015199 test_y_nll: 0.106328338385 test_y_row_norms_max: 2.38739275932 test_y_row_norms_mean: 0.780075967312 test_y_row_norms_min: 0.130353063345 train_h0_col_norms_max: 6.31961917877 train_h0_col_norms_mean: 3.92528319359 train_h0_col_norms_min: 2.10622787476 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.950781822205 train_h0_max_x_min_u: 0.600085794926 train_h0_mean_x_max_u: 0.920143485069 train_h0_mean_x_mean_u: 0.465922415257 train_h0_mean_x_min_u: 0.0898449495435 train_h0_min_x_max_u: 0.391092181206 train_h0_min_x_mean_u: 0.0378985367715 train_h0_min_x_min_u: 1.99124361444e-09 train_h0_row_norms_max: 6.18163204193 train_h0_row_norms_mean: 3.07301878929 train_h0_row_norms_min: 0.0938726961613 train_objective: 0.088271394372 train_y_col_norms_max: 7.01830387115 train_y_col_norms_mean: 6.0149974823 train_y_col_norms_min: 4.83683490753 train_y_max_max_class: 0.999998629093 train_y_mean_max_class: 0.957574307919 train_y_min_max_class: 0.276376664639 train_y_misclass: 0.023999998346 train_y_nll: 0.088271394372 train_y_row_norms_max: 2.3873925209 train_y_row_norms_mean: 0.780075907707 train_y_row_norms_min: 0.130353048444 valid_h0_col_norms_max: 6.31961965561 valid_h0_col_norms_mean: 3.92528343201 valid_h0_col_norms_min: 2.10622811317 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.950090706348 valid_h0_max_x_min_u: 0.620329141617 valid_h0_mean_x_max_u: 0.922675073147 valid_h0_mean_x_mean_u: 0.465976387262 valid_h0_mean_x_min_u: 0.0912392660975 valid_h0_min_x_max_u: 0.378798425198 valid_h0_min_x_mean_u: 0.0380813851953 valid_h0_min_x_min_u: 1.35891120578e-09 valid_h0_row_norms_max: 6.18163251877 valid_h0_row_norms_mean: 3.07301926613 valid_h0_row_norms_min: 0.0938726961613 valid_objective: 0.107352338731 valid_y_col_norms_max: 7.01830482483 valid_y_col_norms_mean: 6.0149974823 valid_y_col_norms_min: 4.83683490753 valid_y_max_max_class: 0.999998867512 valid_y_mean_max_class: 0.959039092064 valid_y_min_max_class: 0.278402447701 valid_y_misclass: 0.0296999998391 valid_y_nll: 0.107352338731 valid_y_row_norms_max: 2.38739275932 valid_y_row_norms_mean: 0.780075967312 valid_y_row_norms_min: 0.130353063345 Time this epoch: 35.302343 seconds Monitoring step: Epochs seen: 18 Batches seen: 90 Examples seen: 900000 ave_grad_mult: 1.79280376434 ave_grad_size: 0.0464615598321 ave_step_size: 0.0771328359842 test_h0_col_norms_max: 6.32822799683 test_h0_col_norms_mean: 3.93359160423 test_h0_col_norms_min: 2.10832476616 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.95045799017 test_h0_max_x_min_u: 0.586617648602 test_h0_mean_x_max_u: 0.929918944836 test_h0_mean_x_mean_u: 0.465379714966 test_h0_mean_x_min_u: 0.083888605237 test_h0_min_x_max_u: 0.397964477539 test_h0_min_x_mean_u: 0.0380010083318 test_h0_min_x_min_u: 7.37118366345e-10 test_h0_row_norms_max: 6.20447731018 test_h0_row_norms_mean: 3.08011174202 test_h0_row_norms_min: 0.0980293303728 test_objective: 0.100425355136 test_y_col_norms_max: 7.28403282166 test_y_col_norms_mean: 6.2393155098 test_y_col_norms_min: 4.98830795288 test_y_max_max_class: 0.999997019768 test_y_mean_max_class: 0.959611177444 test_y_min_max_class: 0.283116281033 test_y_misclass: 0.03039999865 test_y_nll: 0.100425355136 test_y_row_norms_max: 2.53001952171 test_y_row_norms_mean: 0.806962490082 test_y_row_norms_min: 0.131183430552 train_h0_col_norms_max: 6.32822799683 train_h0_col_norms_mean: 3.93359088898 train_h0_col_norms_min: 2.10832476616 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.951834976673 train_h0_max_x_min_u: 0.594883143902 train_h0_mean_x_max_u: 0.922610759735 train_h0_mean_x_mean_u: 0.465650081635 train_h0_mean_x_min_u: 0.0870256572962 train_h0_min_x_max_u: 0.393012464046 train_h0_min_x_mean_u: 0.0370769426227 train_h0_min_x_min_u: 9.6733221433e-10 train_h0_row_norms_max: 6.20447683334 train_h0_row_norms_mean: 3.0801115036 train_h0_row_norms_min: 0.0980293378234 train_objective: 0.0801135376096 train_y_col_norms_max: 7.28403186798 train_y_col_norms_mean: 6.23931598663 train_y_col_norms_min: 4.98830747604 train_y_max_max_class: 0.999998509884 train_y_mean_max_class: 0.960199356079 train_y_min_max_class: 0.269580304623 train_y_misclass: 0.0213200002909 train_y_nll: 0.0801135376096 train_y_row_norms_max: 2.53001952171 train_y_row_norms_mean: 0.806962549686 train_y_row_norms_min: 0.131183415651 valid_h0_col_norms_max: 6.32822799683 valid_h0_col_norms_mean: 3.93359160423 valid_h0_col_norms_min: 2.10832476616 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.951114416122 valid_h0_max_x_min_u: 0.613163709641 valid_h0_mean_x_max_u: 0.925033152103 valid_h0_mean_x_mean_u: 0.465698361397 valid_h0_mean_x_min_u: 0.0884924307466 valid_h0_min_x_max_u: 0.386198699474 valid_h0_min_x_mean_u: 0.0373685508966 valid_h0_min_x_min_u: 6.73591848965e-10 valid_h0_row_norms_max: 6.20447731018 valid_h0_row_norms_mean: 3.08011174202 valid_h0_row_norms_min: 0.0980293303728 valid_objective: 0.101348236203 valid_y_col_norms_max: 7.28403282166 valid_y_col_norms_mean: 6.2393155098 valid_y_col_norms_min: 4.98830795288 valid_y_max_max_class: 0.99999922514 valid_y_mean_max_class: 0.961142122746 valid_y_min_max_class: 0.255374312401 valid_y_misclass: 0.028299998492 valid_y_nll: 0.101348236203 valid_y_row_norms_max: 2.53001952171 valid_y_row_norms_mean: 0.806962490082 valid_y_row_norms_min: 0.131183430552 Time this epoch: 35.215917 seconds Monitoring step: Epochs seen: 19 Batches seen: 95 Examples seen: 950000 ave_grad_mult: 1.94697141647 ave_grad_size: 0.0453744120896 ave_step_size: 0.0806727781892 test_h0_col_norms_max: 6.33764886856 test_h0_col_norms_mean: 3.94183731079 test_h0_col_norms_min: 2.11102938652 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.951863646507 test_h0_max_x_min_u: 0.580549895763 test_h0_mean_x_max_u: 0.932179749012 test_h0_mean_x_mean_u: 0.465434730053 test_h0_mean_x_min_u: 0.0796971917152 test_h0_min_x_max_u: 0.390337795019 test_h0_min_x_mean_u: 0.0372235476971 test_h0_min_x_min_u: 6.10773209786e-10 test_h0_row_norms_max: 6.22417736053 test_h0_row_norms_mean: 3.08713316917 test_h0_row_norms_min: 0.101160049438 test_objective: 0.0948458611965 test_y_col_norms_max: 7.54131317139 test_y_col_norms_mean: 6.45906209946 test_y_col_norms_min: 5.14208126068 test_y_max_max_class: 0.999998509884 test_y_mean_max_class: 0.962593019009 test_y_min_max_class: 0.309717655182 test_y_misclass: 0.0273999981582 test_y_nll: 0.0948458611965 test_y_row_norms_max: 2.65757870674 test_y_row_norms_mean: 0.83378046751 test_y_row_norms_min: 0.132128432393 train_h0_col_norms_max: 6.3376493454 train_h0_col_norms_mean: 3.94183754921 train_h0_col_norms_min: 2.1110291481 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.953146517277 train_h0_max_x_min_u: 0.591403305531 train_h0_mean_x_max_u: 0.925151884556 train_h0_mean_x_mean_u: 0.465704083443 train_h0_mean_x_min_u: 0.0828539133072 train_h0_min_x_max_u: 0.392235994339 train_h0_min_x_mean_u: 0.0363104119897 train_h0_min_x_min_u: 8.38429381478e-10 train_h0_row_norms_max: 6.2241768837 train_h0_row_norms_mean: 3.08713316917 train_h0_row_norms_min: 0.101160041988 train_objective: 0.073119558394 train_y_col_norms_max: 7.54131317139 train_y_col_norms_mean: 6.45906209946 train_y_col_norms_min: 5.14208078384 train_y_max_max_class: 0.999999344349 train_y_mean_max_class: 0.963022887707 train_y_min_max_class: 0.268300741911 train_y_misclass: 0.0194799974561 train_y_nll: 0.073119558394 train_y_row_norms_max: 2.65757846832 train_y_row_norms_mean: 0.833780527115 train_y_row_norms_min: 0.132128432393 valid_h0_col_norms_max: 6.33764886856 valid_h0_col_norms_mean: 3.94183731079 valid_h0_col_norms_min: 2.11102938652 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.952474594116 valid_h0_max_x_min_u: 0.606046676636 valid_h0_mean_x_max_u: 0.927380979061 valid_h0_mean_x_mean_u: 0.465753525496 valid_h0_mean_x_min_u: 0.0843893289566 valid_h0_min_x_max_u: 0.386562854052 valid_h0_min_x_mean_u: 0.0366778969765 valid_h0_min_x_min_u: 5.66411417768e-10 valid_h0_row_norms_max: 6.22417736053 valid_h0_row_norms_mean: 3.08713316917 valid_h0_row_norms_min: 0.101160049438 valid_objective: 0.09637324512 valid_y_col_norms_max: 7.54131317139 valid_y_col_norms_mean: 6.45906209946 valid_y_col_norms_min: 5.14208126068 valid_y_max_max_class: 0.999999463558 valid_y_mean_max_class: 0.96346116066 valid_y_min_max_class: 0.277560830116 valid_y_misclass: 0.0262000001967 valid_y_nll: 0.09637324512 valid_y_row_norms_max: 2.65757870674 valid_y_row_norms_mean: 0.83378046751 valid_y_row_norms_min: 0.132128432393 Time this epoch: 34.760706 seconds Monitoring step: Epochs seen: 20 Batches seen: 100 Examples seen: 1000000 ave_grad_mult: 2.02213191986 ave_grad_size: 0.0437575168908 ave_step_size: 0.081667304039 test_h0_col_norms_max: 6.34621286392 test_h0_col_norms_mean: 3.94933509827 test_h0_col_norms_min: 2.11350440979 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.953083157539 test_h0_max_x_min_u: 0.574586033821 test_h0_mean_x_max_u: 0.934979915619 test_h0_mean_x_mean_u: 0.465407788754 test_h0_mean_x_min_u: 0.0830942466855 test_h0_min_x_max_u: 0.386586099863 test_h0_min_x_mean_u: 0.0363725870848 test_h0_min_x_min_u: 3.24080540182e-10 test_h0_row_norms_max: 6.2420706749 test_h0_row_norms_mean: 3.09350514412 test_h0_row_norms_min: 0.104648023844 test_objective: 0.0911609381437 test_y_col_norms_max: 7.76595830917 test_y_col_norms_mean: 6.65801715851 test_y_col_norms_min: 5.27815532684 test_y_max_max_class: 0.999998688698 test_y_mean_max_class: 0.964522898197 test_y_min_max_class: 0.28780567646 test_y_misclass: 0.0263999979943 test_y_nll: 0.0911609381437 test_y_row_norms_max: 2.76887655258 test_y_row_norms_mean: 0.858034849167 test_y_row_norms_min: 0.135387971997 train_h0_col_norms_max: 6.34621238708 train_h0_col_norms_mean: 3.94933462143 train_h0_col_norms_min: 2.11350440979 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.95438170433 train_h0_max_x_min_u: 0.584669828415 train_h0_mean_x_max_u: 0.928267598152 train_h0_mean_x_mean_u: 0.465672910213 train_h0_mean_x_min_u: 0.0862845480442 train_h0_min_x_max_u: 0.389768064022 train_h0_min_x_mean_u: 0.0354867391288 train_h0_min_x_min_u: 4.38173886064e-10 train_h0_row_norms_max: 6.2420706749 train_h0_row_norms_mean: 3.0935049057 train_h0_row_norms_min: 0.104648023844 train_objective: 0.0672194138169 train_y_col_norms_max: 7.76595830917 train_y_col_norms_mean: 6.65801715851 train_y_col_norms_min: 5.27815580368 train_y_max_max_class: 0.999999523163 train_y_mean_max_class: 0.965664386749 train_y_min_max_class: 0.276637971401 train_y_misclass: 0.0176799986511 train_y_nll: 0.0672194138169 train_y_row_norms_max: 2.76887631416 train_y_row_norms_mean: 0.858034789562 train_y_row_norms_min: 0.135387957096 valid_h0_col_norms_max: 6.34621286392 valid_h0_col_norms_mean: 3.94933509827 valid_h0_col_norms_min: 2.11350440979 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.953744530678 valid_h0_max_x_min_u: 0.597306787968 valid_h0_mean_x_max_u: 0.930288851261 valid_h0_mean_x_mean_u: 0.465717792511 valid_h0_mean_x_min_u: 0.087476670742 valid_h0_min_x_max_u: 0.389485627413 valid_h0_min_x_mean_u: 0.0357559174299 valid_h0_min_x_min_u: 3.05218794683e-10 valid_h0_row_norms_max: 6.2420706749 valid_h0_row_norms_mean: 3.09350514412 valid_h0_row_norms_min: 0.104648023844 valid_objective: 0.0925975292921 valid_y_col_norms_max: 7.76595830917 valid_y_col_norms_mean: 6.65801715851 valid_y_col_norms_min: 5.27815532684 valid_y_max_max_class: 0.999999761581 valid_y_mean_max_class: 0.965861082077 valid_y_min_max_class: 0.303610026836 valid_y_misclass: 0.0258000008762 valid_y_nll: 0.0925975292921 valid_y_row_norms_max: 2.76887655258 valid_y_row_norms_mean: 0.858034849167 valid_y_row_norms_min: 0.135387971997 Time this epoch: 35.213061 seconds Monitoring step: Epochs seen: 21 Batches seen: 105 Examples seen: 1050000 ave_grad_mult: 2.08118438721 ave_grad_size: 0.0415316298604 ave_step_size: 0.080756470561 test_h0_col_norms_max: 6.35434007645 test_h0_col_norms_mean: 3.95622348785 test_h0_col_norms_min: 2.11573195457 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.953851401806 test_h0_max_x_min_u: 0.567606449127 test_h0_mean_x_max_u: 0.933193147182 test_h0_mean_x_mean_u: 0.465032488108 test_h0_mean_x_min_u: 0.0830998793244 test_h0_min_x_max_u: 0.383445978165 test_h0_min_x_mean_u: 0.0356372632086 test_h0_min_x_min_u: 2.19485554731e-10 test_h0_row_norms_max: 6.25859546661 test_h0_row_norms_mean: 3.09933209419 test_h0_row_norms_min: 0.107006825507 test_objective: 0.0886002033949 test_y_col_norms_max: 7.9637556076 test_y_col_norms_mean: 6.83463764191 test_y_col_norms_min: 5.3923330307 test_y_max_max_class: 0.99999833107 test_y_mean_max_class: 0.965270340443 test_y_min_max_class: 0.310471683741 test_y_misclass: 0.0262000001967 test_y_nll: 0.0886002033949 test_y_row_norms_max: 2.86672186852 test_y_row_norms_mean: 0.879510939121 test_y_row_norms_min: 0.136433556676 train_h0_col_norms_max: 6.35433912277 train_h0_col_norms_mean: 3.95622301102 train_h0_col_norms_min: 2.11573171616 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.955171644688 train_h0_max_x_min_u: 0.579390406609 train_h0_mean_x_max_u: 0.926286578178 train_h0_mean_x_mean_u: 0.465295374393 train_h0_mean_x_min_u: 0.0863517001271 train_h0_min_x_max_u: 0.384008407593 train_h0_min_x_mean_u: 0.0348346866667 train_h0_min_x_min_u: 2.7790120205e-10 train_h0_row_norms_max: 6.25859498978 train_h0_row_norms_mean: 3.09933185577 train_h0_row_norms_min: 0.107006818056 train_objective: 0.0625123158097 train_y_col_norms_max: 7.96375513077 train_y_col_norms_mean: 6.83463668823 train_y_col_norms_min: 5.3923330307 train_y_max_max_class: 0.99999922514 train_y_mean_max_class: 0.967036545277 train_y_min_max_class: 0.273270666599 train_y_misclass: 0.0158599987626 train_y_nll: 0.0625123158097 train_y_row_norms_max: 2.8667216301 train_y_row_norms_mean: 0.879510939121 train_y_row_norms_min: 0.136433571577 valid_h0_col_norms_max: 6.35434007645 valid_h0_col_norms_mean: 3.95622348785 valid_h0_col_norms_min: 2.11573195457 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.95438760519 valid_h0_max_x_min_u: 0.590360045433 valid_h0_mean_x_max_u: 0.928494334221 valid_h0_mean_x_mean_u: 0.465345591307 valid_h0_mean_x_min_u: 0.0878760442138 valid_h0_min_x_max_u: 0.384474813938 valid_h0_min_x_mean_u: 0.0351748354733 valid_h0_min_x_min_u: 1.96166291544e-10 valid_h0_row_norms_max: 6.25859546661 valid_h0_row_norms_mean: 3.09933209419 valid_h0_row_norms_min: 0.107006825507 valid_objective: 0.0909144356847 valid_y_col_norms_max: 7.9637556076 valid_y_col_norms_mean: 6.83463764191 valid_y_col_norms_min: 5.3923330307 valid_y_max_max_class: 0.999999403954 valid_y_mean_max_class: 0.966769099236 valid_y_min_max_class: 0.282997220755 valid_y_misclass: 0.025399999693 valid_y_nll: 0.0909144356847 valid_y_row_norms_max: 2.86672186852 valid_y_row_norms_mean: 0.879510939121 valid_y_row_norms_min: 0.136433556676 Time this epoch: 35.132773 seconds Monitoring step: Epochs seen: 22 Batches seen: 110 Examples seen: 1100000 ave_grad_mult: 2.14148879051 ave_grad_size: 0.0403550490737 ave_step_size: 0.0810787156224 test_h0_col_norms_max: 6.3625164032 test_h0_col_norms_mean: 3.96336507797 test_h0_col_norms_min: 2.11782503128 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.954942882061 test_h0_max_x_min_u: 0.564869940281 test_h0_mean_x_max_u: 0.935991108418 test_h0_mean_x_mean_u: 0.465213596821 test_h0_mean_x_min_u: 0.0809470117092 test_h0_min_x_max_u: 0.385282725096 test_h0_min_x_mean_u: 0.0350002162158 test_h0_min_x_min_u: 1.53522847213e-10 test_h0_row_norms_max: 6.27728748322 test_h0_row_norms_mean: 3.10535025597 test_h0_row_norms_min: 0.109762132168 test_objective: 0.0847353041172 test_y_col_norms_max: 8.15684700012 test_y_col_norms_mean: 7.01448202133 test_y_col_norms_min: 5.519551754 test_y_max_max_class: 0.999998867512 test_y_mean_max_class: 0.967154860497 test_y_min_max_class: 0.283250451088 test_y_misclass: 0.0249000005424 test_y_nll: 0.0847353041172 test_y_row_norms_max: 2.96138525009 test_y_row_norms_mean: 0.901844441891 test_y_row_norms_min: 0.138287782669 train_h0_col_norms_max: 6.3625164032 train_h0_col_norms_mean: 3.96336531639 train_h0_col_norms_min: 2.11782479286 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.956164836884 train_h0_max_x_min_u: 0.575154662132 train_h0_mean_x_max_u: 0.929373860359 train_h0_mean_x_mean_u: 0.465472817421 train_h0_mean_x_min_u: 0.0842097327113 train_h0_min_x_max_u: 0.385039448738 train_h0_min_x_mean_u: 0.0342263542116 train_h0_min_x_min_u: 1.99991759264e-10 train_h0_row_norms_max: 6.27728748322 train_h0_row_norms_mean: 3.10535001755 train_h0_row_norms_min: 0.109762117267 train_objective: 0.0575138144195 train_y_col_norms_max: 8.15684700012 train_y_col_norms_mean: 7.01448202133 train_y_col_norms_min: 5.51955223083 train_y_max_max_class: 0.999999582767 train_y_mean_max_class: 0.96871650219 train_y_min_max_class: 0.287014901638 train_y_misclass: 0.0142399985343 train_y_nll: 0.0575138144195 train_y_row_norms_max: 2.96138525009 train_y_row_norms_mean: 0.901844382286 train_y_row_norms_min: 0.138287782669 valid_h0_col_norms_max: 6.3625164032 valid_h0_col_norms_mean: 3.96336507797 valid_h0_col_norms_min: 2.11782503128 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.955441474915 valid_h0_max_x_min_u: 0.589357554913 valid_h0_mean_x_max_u: 0.93136715889 valid_h0_mean_x_mean_u: 0.46551990509 valid_h0_mean_x_min_u: 0.086060911417 valid_h0_min_x_max_u: 0.390778958797 valid_h0_min_x_mean_u: 0.0345365256071 valid_h0_min_x_min_u: 1.45148310038e-10 valid_h0_row_norms_max: 6.27728748322 valid_h0_row_norms_mean: 3.10535025597 valid_h0_row_norms_min: 0.109762132168 valid_objective: 0.0865774899721 valid_y_col_norms_max: 8.15684700012 valid_y_col_norms_mean: 7.01448202133 valid_y_col_norms_min: 5.519551754 valid_y_max_max_class: 0.999999761581 valid_y_mean_max_class: 0.96779280901 valid_y_min_max_class: 0.273192465305 valid_y_misclass: 0.0244999974966 valid_y_nll: 0.0865774899721 valid_y_row_norms_max: 2.96138525009 valid_y_row_norms_mean: 0.901844441891 valid_y_row_norms_min: 0.138287782669 Time this epoch: 35.193111 seconds Monitoring step: Epochs seen: 23 Batches seen: 115 Examples seen: 1150000 ave_grad_mult: 2.29178571701 ave_grad_size: 0.0395583026111 ave_step_size: 0.0849489048123 test_h0_col_norms_max: 6.37209796906 test_h0_col_norms_mean: 3.97117829323 test_h0_col_norms_min: 2.11957788467 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.956251323223 test_h0_max_x_min_u: 0.561847269535 test_h0_mean_x_max_u: 0.934819757938 test_h0_mean_x_mean_u: 0.465812414885 test_h0_mean_x_min_u: 0.0860762521625 test_h0_min_x_max_u: 0.382852345705 test_h0_min_x_mean_u: 0.0343066453934 test_h0_min_x_min_u: 1.00234549827e-10 test_h0_row_norms_max: 6.29464244843 test_h0_row_norms_mean: 3.11194372177 test_h0_row_norms_min: 0.112373262644 test_objective: 0.0813909471035 test_y_col_norms_max: 8.37556743622 test_y_col_norms_mean: 7.21202421188 test_y_col_norms_min: 5.66676425934 test_y_max_max_class: 0.99999922514 test_y_mean_max_class: 0.969460964203 test_y_min_max_class: 0.304885983467 test_y_misclass: 0.0245999991894 test_y_nll: 0.0813909471035 test_y_row_norms_max: 3.05758142471 test_y_row_norms_mean: 0.926100432873 test_y_row_norms_min: 0.141218408942 train_h0_col_norms_max: 6.37209796906 train_h0_col_norms_mean: 3.97117805481 train_h0_col_norms_min: 2.11957764626 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.957390427589 train_h0_max_x_min_u: 0.571023106575 train_h0_mean_x_max_u: 0.928050994873 train_h0_mean_x_mean_u: 0.466074705124 train_h0_mean_x_min_u: 0.089665055275 train_h0_min_x_max_u: 0.383693158627 train_h0_min_x_mean_u: 0.0335417687893 train_h0_min_x_min_u: 1.25948085294e-10 train_h0_row_norms_max: 6.29464149475 train_h0_row_norms_mean: 3.11194324493 train_h0_row_norms_min: 0.112373247743 train_objective: 0.0530071258545 train_y_col_norms_max: 8.37556743622 train_y_col_norms_mean: 7.21202325821 train_y_col_norms_min: 5.6667637825 train_y_max_max_class: 0.999999761581 train_y_mean_max_class: 0.971323847771 train_y_min_max_class: 0.274939656258 train_y_misclass: 0.0134199988097 train_y_nll: 0.0530071258545 train_y_row_norms_max: 3.05758142471 train_y_row_norms_mean: 0.926100373268 train_y_row_norms_min: 0.141218394041 valid_h0_col_norms_max: 6.37209796906 valid_h0_col_norms_mean: 3.97117829323 valid_h0_col_norms_min: 2.11957788467 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.956661045551 valid_h0_max_x_min_u: 0.585293292999 valid_h0_mean_x_max_u: 0.930199086666 valid_h0_mean_x_mean_u: 0.466111898422 valid_h0_mean_x_min_u: 0.0879896134138 valid_h0_min_x_max_u: 0.389526426792 valid_h0_min_x_mean_u: 0.0339160002768 valid_h0_min_x_min_u: 9.11426628614e-11 valid_h0_row_norms_max: 6.29464244843 valid_h0_row_norms_mean: 3.11194372177 valid_h0_row_norms_min: 0.112373262644 valid_objective: 0.0844431295991 valid_y_col_norms_max: 8.37556743622 valid_y_col_norms_mean: 7.21202421188 valid_y_col_norms_min: 5.66676425934 valid_y_max_max_class: 0.999999761581 valid_y_mean_max_class: 0.970053553581 valid_y_min_max_class: 0.252451866865 valid_y_misclass: 0.0244999974966 valid_y_nll: 0.0844431295991 valid_y_row_norms_max: 3.05758142471 valid_y_row_norms_mean: 0.926100432873 valid_y_row_norms_min: 0.141218408942 Time this epoch: 35.101327 seconds Monitoring step: Epochs seen: 24 Batches seen: 120 Examples seen: 1200000 ave_grad_mult: 2.4745285511 ave_grad_size: 0.037980530411 ave_step_size: 0.0874084308743 test_h0_col_norms_max: 6.38188457489 test_h0_col_norms_mean: 3.97928380966 test_h0_col_norms_min: 2.12256121635 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.957348406315 test_h0_max_x_min_u: 0.559364676476 test_h0_mean_x_max_u: 0.934902369976 test_h0_mean_x_mean_u: 0.465777903795 test_h0_mean_x_min_u: 0.0820802301168 test_h0_min_x_max_u: 0.37371224165 test_h0_min_x_mean_u: 0.0335819907486 test_h0_min_x_min_u: 6.84129905504e-11 test_h0_row_norms_max: 6.31191539764 test_h0_row_norms_mean: 3.11876320839 test_h0_row_norms_min: 0.114646181464 test_objective: 0.0791404470801 test_y_col_norms_max: 8.59417057037 test_y_col_norms_mean: 7.40912103653 test_y_col_norms_min: 5.81003856659 test_y_max_max_class: 0.99999922514 test_y_mean_max_class: 0.969864010811 test_y_min_max_class: 0.260499119759 test_y_misclass: 0.0230999998748 test_y_nll: 0.0791404470801 test_y_row_norms_max: 3.15858983994 test_y_row_norms_mean: 0.950569629669 test_y_row_norms_min: 0.144145652652 train_h0_col_norms_max: 6.38188409805 train_h0_col_norms_mean: 3.97928357124 train_h0_col_norms_min: 2.12256097794 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.958476305008 train_h0_max_x_min_u: 0.567814290524 train_h0_mean_x_max_u: 0.928138375282 train_h0_mean_x_mean_u: 0.46603512764 train_h0_mean_x_min_u: 0.0855919569731 train_h0_min_x_max_u: 0.379186630249 train_h0_min_x_mean_u: 0.0329259894788 train_h0_min_x_min_u: 8.38127969804e-11 train_h0_row_norms_max: 6.31191492081 train_h0_row_norms_mean: 3.11876296997 train_h0_row_norms_min: 0.114646181464 train_objective: 0.0484027862549 train_y_col_norms_max: 8.59417057037 train_y_col_norms_mean: 7.40912055969 train_y_col_norms_min: 5.81003761292 train_y_max_max_class: 0.999999701977 train_y_mean_max_class: 0.972274065018 train_y_min_max_class: 0.297603964806 train_y_misclass: 0.0116999996826 train_y_nll: 0.0484027862549 train_y_row_norms_max: 3.15858960152 train_y_row_norms_mean: 0.95056951046 train_y_row_norms_min: 0.144145637751 valid_h0_col_norms_max: 6.38188457489 valid_h0_col_norms_mean: 3.97928380966 valid_h0_col_norms_min: 2.12256121635 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.957733333111 valid_h0_max_x_min_u: 0.57974678278 valid_h0_mean_x_max_u: 0.930305242538 valid_h0_mean_x_mean_u: 0.466072052717 valid_h0_mean_x_min_u: 0.0875690802932 valid_h0_min_x_max_u: 0.382509231567 valid_h0_min_x_mean_u: 0.0332807153463 valid_h0_min_x_min_u: 6.19569395788e-11 valid_h0_row_norms_max: 6.31191539764 valid_h0_row_norms_mean: 3.11876320839 valid_h0_row_norms_min: 0.114646181464 valid_objective: 0.0832240283489 valid_y_col_norms_max: 8.59417057037 valid_y_col_norms_mean: 7.40912103653 valid_y_col_norms_min: 5.81003856659 valid_y_max_max_class: 0.999999761581 valid_y_mean_max_class: 0.970567047596 valid_y_min_max_class: 0.264748305082 valid_y_misclass: 0.023999998346 valid_y_nll: 0.0832240283489 valid_y_row_norms_max: 3.15858983994 valid_y_row_norms_mean: 0.950569629669 valid_y_row_norms_min: 0.144145652652 Time this epoch: 35.537865 seconds Monitoring step: Epochs seen: 25 Batches seen: 125 Examples seen: 1250000 ave_grad_mult: 2.61537218094 ave_grad_size: 0.0366696789861 ave_step_size: 0.0890378654003 test_h0_col_norms_max: 6.39218759537 test_h0_col_norms_mean: 3.98763632774 test_h0_col_norms_min: 2.1254658699 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.958310782909 test_h0_max_x_min_u: 0.550610423088 test_h0_mean_x_max_u: 0.936276137829 test_h0_mean_x_mean_u: 0.46598726511 test_h0_mean_x_min_u: 0.0805417820811 test_h0_min_x_max_u: 0.375468403101 test_h0_min_x_mean_u: 0.0328881442547 test_h0_min_x_min_u: 7.84403653142e-11 test_h0_row_norms_max: 6.33220767975 test_h0_row_norms_mean: 3.1257724762 test_h0_row_norms_min: 0.116853624582 test_objective: 0.0754533782601 test_y_col_norms_max: 8.81067371368 test_y_col_norms_mean: 7.60889148712 test_y_col_norms_min: 5.96597194672 test_y_max_max_class: 0.999999761581 test_y_mean_max_class: 0.971746265888 test_y_min_max_class: 0.288777351379 test_y_misclass: 0.0232999995351 test_y_nll: 0.0754533782601 test_y_row_norms_max: 3.24519085884 test_y_row_norms_mean: 0.975342810154 test_y_row_norms_min: 0.148321658373 train_h0_col_norms_max: 6.3921880722 train_h0_col_norms_mean: 3.98763632774 train_h0_col_norms_min: 2.1254658699 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.959453582764 train_h0_max_x_min_u: 0.561280608177 train_h0_mean_x_max_u: 0.929621398449 train_h0_mean_x_mean_u: 0.466242551804 train_h0_mean_x_min_u: 0.0841432288289 train_h0_min_x_max_u: 0.38017898798 train_h0_min_x_mean_u: 0.0322615392506 train_h0_min_x_min_u: 1.0068777756e-10 train_h0_row_norms_max: 6.33220720291 train_h0_row_norms_mean: 3.1257724762 train_h0_row_norms_min: 0.116853624582 train_objective: 0.043993473053 train_y_col_norms_max: 8.81067276001 train_y_col_norms_mean: 7.60889053345 train_y_col_norms_min: 5.96597194672 train_y_max_max_class: 0.999999821186 train_y_mean_max_class: 0.974251687527 train_y_min_max_class: 0.270618349314 train_y_misclass: 0.0104199992493 train_y_nll: 0.043993473053 train_y_row_norms_max: 3.24519062042 train_y_row_norms_mean: 0.975342690945 train_y_row_norms_min: 0.148321658373 valid_h0_col_norms_max: 6.39218759537 valid_h0_col_norms_mean: 3.98763632774 valid_h0_col_norms_min: 2.1254658699 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.958616793156 valid_h0_max_x_min_u: 0.574836432934 valid_h0_mean_x_max_u: 0.931737542152 valid_h0_mean_x_mean_u: 0.466278731823 valid_h0_mean_x_min_u: 0.0861588418484 valid_h0_min_x_max_u: 0.383438080549 valid_h0_min_x_mean_u: 0.032565869391 valid_h0_min_x_min_u: 7.15359646519e-11 valid_h0_row_norms_max: 6.33220767975 valid_h0_row_norms_mean: 3.1257724762 valid_h0_row_norms_min: 0.116853624582 valid_objective: 0.0792490914464 valid_y_col_norms_max: 8.81067371368 valid_y_col_norms_mean: 7.60889148712 valid_y_col_norms_min: 5.96597194672 valid_y_max_max_class: 0.999999821186 valid_y_mean_max_class: 0.972301781178 valid_y_min_max_class: 0.278648257256 valid_y_misclass: 0.0228000003844 valid_y_nll: 0.0792490914464 valid_y_row_norms_max: 3.24519085884 valid_y_row_norms_mean: 0.975342810154 valid_y_row_norms_min: 0.148321658373 Time this epoch: 35.095306 seconds Monitoring step: Epochs seen: 26 Batches seen: 130 Examples seen: 1300000 ave_grad_mult: 2.71106290817 ave_grad_size: 0.0348753891885 ave_step_size: 0.0883127823472 test_h0_col_norms_max: 6.4017291069 test_h0_col_norms_mean: 3.99520802498 test_h0_col_norms_min: 2.12854385376 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.959264814854 test_h0_max_x_min_u: 0.55078792572 test_h0_mean_x_max_u: 0.935866773129 test_h0_mean_x_mean_u: 0.466203004122 test_h0_mean_x_min_u: 0.078706741333 test_h0_min_x_max_u: 0.367448121309 test_h0_min_x_mean_u: 0.0321592055261 test_h0_min_x_min_u: 4.60236952715e-11 test_h0_row_norms_max: 6.34829235077 test_h0_row_norms_mean: 3.13210654259 test_h0_row_norms_min: 0.118406176567 test_objective: 0.0733289569616 test_y_col_norms_max: 9.00574874878 test_y_col_norms_mean: 7.78995084763 test_y_col_norms_min: 6.10382938385 test_y_max_max_class: 0.999999761581 test_y_mean_max_class: 0.972537279129 test_y_min_max_class: 0.267330288887 test_y_misclass: 0.0230999998748 test_y_nll: 0.0733289569616 test_y_row_norms_max: 3.33722496033 test_y_row_norms_mean: 0.997784733772 test_y_row_norms_min: 0.151363104582 train_h0_col_norms_max: 6.40172863007 train_h0_col_norms_mean: 3.9952082634 train_h0_col_norms_min: 2.12854361534 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.960306882858 train_h0_max_x_min_u: 0.562210321426 train_h0_mean_x_max_u: 0.929156780243 train_h0_mean_x_mean_u: 0.466451466084 train_h0_mean_x_min_u: 0.0823357179761 train_h0_min_x_max_u: 0.377736270428 train_h0_min_x_mean_u: 0.0315755605698 train_h0_min_x_min_u: 5.56277697517e-11 train_h0_row_norms_max: 6.34829139709 train_h0_row_norms_mean: 3.13210630417 train_h0_row_norms_min: 0.118406184018 train_objective: 0.0409014374018 train_y_col_norms_max: 9.00574874878 train_y_col_norms_mean: 7.78995037079 train_y_col_norms_min: 6.10382938385 train_y_max_max_class: 0.999999821186 train_y_mean_max_class: 0.975649058819 train_y_min_max_class: 0.290484070778 train_y_misclass: 0.00971999950707 train_y_nll: 0.0409014374018 train_y_row_norms_max: 3.33722496033 train_y_row_norms_mean: 0.997784733772 train_y_row_norms_min: 0.151363104582 valid_h0_col_norms_max: 6.4017291069 valid_h0_col_norms_mean: 3.99520802498 valid_h0_col_norms_min: 2.12854385376 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.959495425224 valid_h0_max_x_min_u: 0.576115489006 valid_h0_mean_x_max_u: 0.931356489658 valid_h0_mean_x_mean_u: 0.466476142406 valid_h0_mean_x_min_u: 0.084395840764 valid_h0_min_x_max_u: 0.37360149622 valid_h0_min_x_mean_u: 0.0320250503719 valid_h0_min_x_min_u: 4.11951479873e-11 valid_h0_row_norms_max: 6.34829235077 valid_h0_row_norms_mean: 3.13210654259 valid_h0_row_norms_min: 0.118406176567 valid_objective: 0.0791732370853 valid_y_col_norms_max: 9.00574874878 valid_y_col_norms_mean: 7.78995084763 valid_y_col_norms_min: 6.10382938385 valid_y_max_max_class: 0.999999821186 valid_y_mean_max_class: 0.973247587681 valid_y_min_max_class: 0.254454284906 valid_y_misclass: 0.0232999995351 valid_y_nll: 0.0791732370853 valid_y_row_norms_max: 3.33722496033 valid_y_row_norms_mean: 0.997784733772 valid_y_row_norms_min: 0.151363104582 Time this epoch: 35.406078 seconds Monitoring step: Epochs seen: 27 Batches seen: 135 Examples seen: 1350000 ave_grad_mult: 2.80285286903 ave_grad_size: 0.0334513224661 ave_step_size: 0.088192678988 test_h0_col_norms_max: 6.41177082062 test_h0_col_norms_mean: 4.00275707245 test_h0_col_norms_min: 2.13091373444 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.960214793682 test_h0_max_x_min_u: 0.547308385372 test_h0_mean_x_max_u: 0.936411142349 test_h0_mean_x_mean_u: 0.466299444437 test_h0_mean_x_min_u: 0.0786798894405 test_h0_min_x_max_u: 0.366961061954 test_h0_min_x_mean_u: 0.0315478779376 test_h0_min_x_min_u: 4.00019496694e-11 test_h0_row_norms_max: 6.36605072021 test_h0_row_norms_mean: 3.13841247559 test_h0_row_norms_min: 0.119428776205 test_objective: 0.0725825279951 test_y_col_norms_max: 9.19264411926 test_y_col_norms_mean: 7.96643924713 test_y_col_norms_min: 6.2465171814 test_y_max_max_class: 0.999999821186 test_y_mean_max_class: 0.974697828293 test_y_min_max_class: 0.284473180771 test_y_misclass: 0.0219000000507 test_y_nll: 0.0725825279951 test_y_row_norms_max: 3.41140413284 test_y_row_norms_mean: 1.01977562904 test_y_row_norms_min: 0.155052781105 train_h0_col_norms_max: 6.41176986694 train_h0_col_norms_mean: 4.00275659561 train_h0_col_norms_min: 2.13091373444 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.96123111248 train_h0_max_x_min_u: 0.556509852409 train_h0_mean_x_max_u: 0.929732501507 train_h0_mean_x_mean_u: 0.466540902853 train_h0_mean_x_min_u: 0.0823666229844 train_h0_min_x_max_u: 0.371994018555 train_h0_min_x_mean_u: 0.0308939814568 train_h0_min_x_min_u: 5.01665688157e-11 train_h0_row_norms_max: 6.36605024338 train_h0_row_norms_mean: 3.13841223717 train_h0_row_norms_min: 0.119428783655 train_objective: 0.0370035469532 train_y_col_norms_max: 9.19264411926 train_y_col_norms_mean: 7.96643972397 train_y_col_norms_min: 6.24651670456 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.977714180946 train_y_min_max_class: 0.2884734869 train_y_misclass: 0.00885999947786 train_y_nll: 0.0370035469532 train_y_row_norms_max: 3.41140389442 train_y_row_norms_mean: 1.01977562904 train_y_row_norms_min: 0.155052781105 valid_h0_col_norms_max: 6.41177082062 valid_h0_col_norms_mean: 4.00275707245 valid_h0_col_norms_min: 2.13091373444 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.960400640965 valid_h0_max_x_min_u: 0.574073672295 valid_h0_mean_x_max_u: 0.931940674782 valid_h0_mean_x_mean_u: 0.466566413641 valid_h0_mean_x_min_u: 0.0844538062811 valid_h0_min_x_max_u: 0.369769692421 valid_h0_min_x_mean_u: 0.0312857404351 valid_h0_min_x_min_u: 3.6578275131e-11 valid_h0_row_norms_max: 6.36605072021 valid_h0_row_norms_mean: 3.13841247559 valid_h0_row_norms_min: 0.119428776205 valid_objective: 0.0765716135502 valid_y_col_norms_max: 9.19264411926 valid_y_col_norms_mean: 7.96643924713 valid_y_col_norms_min: 6.2465171814 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.974825143814 valid_y_min_max_class: 0.268961429596 valid_y_misclass: 0.0228999983519 valid_y_nll: 0.0765716135502 valid_y_row_norms_max: 3.41140413284 valid_y_row_norms_mean: 1.01977562904 valid_y_row_norms_min: 0.155052781105 Time this epoch: 34.780491 seconds Monitoring step: Epochs seen: 28 Batches seen: 140 Examples seen: 1400000 ave_grad_mult: 3.07722043991 ave_grad_size: 0.0323846936226 ave_step_size: 0.0927985981107 test_h0_col_norms_max: 6.42322683334 test_h0_col_norms_mean: 4.01182746887 test_h0_col_norms_min: 2.13467645645 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.961281061172 test_h0_max_x_min_u: 0.549638330936 test_h0_mean_x_max_u: 0.939934492111 test_h0_mean_x_mean_u: 0.466522186995 test_h0_mean_x_min_u: 0.0823039337993 test_h0_min_x_max_u: 0.357347339392 test_h0_min_x_mean_u: 0.030734334141 test_h0_min_x_min_u: 5.08886266459e-11 test_h0_row_norms_max: 6.38524675369 test_h0_row_norms_mean: 3.1459903717 test_h0_row_norms_min: 0.121352598071 test_objective: 0.0716430544853 test_y_col_norms_max: 9.41203117371 test_y_col_norms_mean: 8.17550086975 test_y_col_norms_min: 6.39991140366 test_y_max_max_class: 0.999999821186 test_y_mean_max_class: 0.974777877331 test_y_min_max_class: 0.270264923573 test_y_misclass: 0.0223999992013 test_y_nll: 0.0716430544853 test_y_row_norms_max: 3.5011806488 test_y_row_norms_mean: 1.04629290104 test_y_row_norms_min: 0.159884780645 train_h0_col_norms_max: 6.42322635651 train_h0_col_norms_mean: 4.01182699203 train_h0_col_norms_min: 2.13467645645 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.962265074253 train_h0_max_x_min_u: 0.556583762169 train_h0_mean_x_max_u: 0.93360298872 train_h0_mean_x_mean_u: 0.4667532444 train_h0_mean_x_min_u: 0.0861736312509 train_h0_min_x_max_u: 0.366083562374 train_h0_min_x_mean_u: 0.0301264487207 train_h0_min_x_min_u: 6.59544779902e-11 train_h0_row_norms_max: 6.38524627686 train_h0_row_norms_mean: 3.1459903717 train_h0_row_norms_min: 0.121352590621 train_objective: 0.0347100757062 train_y_col_norms_max: 9.41203117371 train_y_col_norms_mean: 8.17550086975 train_y_col_norms_min: 6.39991092682 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.978323638439 train_y_min_max_class: 0.29167419672 train_y_misclass: 0.00763999950141 train_y_nll: 0.0347100757062 train_y_row_norms_max: 3.50118041039 train_y_row_norms_mean: 1.04629290104 train_y_row_norms_min: 0.159884765744 valid_h0_col_norms_max: 6.42322683334 valid_h0_col_norms_mean: 4.01182746887 valid_h0_col_norms_min: 2.13467645645 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.961273550987 valid_h0_max_x_min_u: 0.570200264454 valid_h0_mean_x_max_u: 0.935558438301 valid_h0_mean_x_mean_u: 0.466782003641 valid_h0_mean_x_min_u: 0.0883660390973 valid_h0_min_x_max_u: 0.355543404818 valid_h0_min_x_mean_u: 0.0304867494851 valid_h0_min_x_min_u: 4.62212004781e-11 valid_h0_row_norms_max: 6.38524675369 valid_h0_row_norms_mean: 3.1459903717 valid_h0_row_norms_min: 0.121352598071 valid_objective: 0.0746665000916 valid_y_col_norms_max: 9.41203117371 valid_y_col_norms_mean: 8.17550086975 valid_y_col_norms_min: 6.39991140366 valid_y_max_max_class: 0.999999821186 valid_y_mean_max_class: 0.975300252438 valid_y_min_max_class: 0.280865699053 valid_y_misclass: 0.0222999975085 valid_y_nll: 0.0746665000916 valid_y_row_norms_max: 3.5011806488 valid_y_row_norms_mean: 1.04629290104 valid_y_row_norms_min: 0.159884780645 Time this epoch: 35.278322 seconds Monitoring step: Epochs seen: 29 Batches seen: 145 Examples seen: 1450000 ave_grad_mult: 3.31815242767 ave_grad_size: 0.0319525785744 ave_step_size: 0.0989938527346 test_h0_col_norms_max: 6.43632364273 test_h0_col_norms_mean: 4.02131462097 test_h0_col_norms_min: 2.13684439659 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.96216905117 test_h0_max_x_min_u: 0.548579275608 test_h0_mean_x_max_u: 0.941554307938 test_h0_mean_x_mean_u: 0.46652469039 test_h0_mean_x_min_u: 0.0799239650369 test_h0_min_x_max_u: 0.357737779617 test_h0_min_x_mean_u: 0.0300854835659 test_h0_min_x_min_u: 2.19761518011e-11 test_h0_row_norms_max: 6.40747022629 test_h0_row_norms_mean: 3.15389037132 test_h0_row_norms_min: 0.123720750213 test_objective: 0.0690323263407 test_y_col_norms_max: 9.64226436615 test_y_col_norms_mean: 8.3891248703 test_y_col_norms_min: 6.57722139359 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.976467430592 test_y_min_max_class: 0.293188840151 test_y_misclass: 0.0212999973446 test_y_nll: 0.0690323263407 test_y_row_norms_max: 3.5816681385 test_y_row_norms_mean: 1.07294213772 test_y_row_norms_min: 0.162085324526 train_h0_col_norms_max: 6.43632364273 train_h0_col_norms_mean: 4.02131462097 train_h0_col_norms_min: 2.13684439659 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.963165700436 train_h0_max_x_min_u: 0.55665397644 train_h0_mean_x_max_u: 0.935373008251 train_h0_mean_x_mean_u: 0.466756403446 train_h0_mean_x_min_u: 0.0838716328144 train_h0_min_x_max_u: 0.36525374651 train_h0_min_x_mean_u: 0.0295039452612 train_h0_min_x_min_u: 2.68275124338e-11 train_h0_row_norms_max: 6.40747022629 train_h0_row_norms_mean: 3.15389037132 train_h0_row_norms_min: 0.123720750213 train_objective: 0.0306258164346 train_y_col_norms_max: 9.64226341248 train_y_col_norms_mean: 8.3891248703 train_y_col_norms_min: 6.57722091675 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.980324864388 train_y_min_max_class: 0.283443570137 train_y_misclass: 0.00647999951616 train_y_nll: 0.0306258164346 train_y_row_norms_max: 3.58166790009 train_y_row_norms_mean: 1.07294213772 train_y_row_norms_min: 0.162085309625 valid_h0_col_norms_max: 6.43632364273 valid_h0_col_norms_mean: 4.02131462097 valid_h0_col_norms_min: 2.13684439659 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.962217152119 valid_h0_max_x_min_u: 0.571269631386 valid_h0_mean_x_max_u: 0.937248468399 valid_h0_mean_x_mean_u: 0.466767311096 valid_h0_mean_x_min_u: 0.0859551951289 valid_h0_min_x_max_u: 0.352124005556 valid_h0_min_x_mean_u: 0.0299664791673 valid_h0_min_x_min_u: 2.06817705323e-11 valid_h0_row_norms_max: 6.40747022629 valid_h0_row_norms_mean: 3.15389037132 valid_h0_row_norms_min: 0.123720750213 valid_objective: 0.0733132436872 valid_y_col_norms_max: 9.64226436615 valid_y_col_norms_mean: 8.3891248703 valid_y_col_norms_min: 6.57722139359 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.976790785789 valid_y_min_max_class: 0.291347831488 valid_y_misclass: 0.0217000003904 valid_y_nll: 0.0733132436872 valid_y_row_norms_max: 3.5816681385 valid_y_row_norms_mean: 1.07294213772 valid_y_row_norms_min: 0.162085324526 Time this epoch: 35.082858 seconds Monitoring step: Epochs seen: 30 Batches seen: 150 Examples seen: 1500000 ave_grad_mult: 3.39051413536 ave_grad_size: 0.0302216522396 ave_step_size: 0.0965146124363 test_h0_col_norms_max: 6.44657659531 test_h0_col_norms_mean: 4.02922582626 test_h0_col_norms_min: 2.14045143127 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.963280200958 test_h0_max_x_min_u: 0.548351347446 test_h0_mean_x_max_u: 0.940559446812 test_h0_mean_x_mean_u: 0.466341674328 test_h0_mean_x_min_u: 0.0799687504768 test_h0_min_x_max_u: 0.357499152422 test_h0_min_x_mean_u: 0.0293492469937 test_h0_min_x_min_u: 2.20450845079e-11 test_h0_row_norms_max: 6.4218788147 test_h0_row_norms_mean: 3.16045355797 test_h0_row_norms_min: 0.124831520021 test_objective: 0.0672078579664 test_y_col_norms_max: 9.82299423218 test_y_col_norms_mean: 8.56633377075 test_y_col_norms_min: 6.71553707123 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.977503836155 test_y_min_max_class: 0.255842655897 test_y_misclass: 0.0198999978602 test_y_nll: 0.0672078579664 test_y_row_norms_max: 3.65550899506 test_y_row_norms_mean: 1.09536457062 test_y_row_norms_min: 0.163716614246 train_h0_col_norms_max: 6.44657611847 train_h0_col_norms_mean: 4.02922534943 train_h0_col_norms_min: 2.14045143127 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.964230835438 train_h0_max_x_min_u: 0.55356913805 train_h0_mean_x_max_u: 0.934283614159 train_h0_mean_x_mean_u: 0.466563820839 train_h0_mean_x_min_u: 0.0839123427868 train_h0_min_x_max_u: 0.358219176531 train_h0_min_x_mean_u: 0.0287974383682 train_h0_min_x_min_u: 2.7636832059e-11 train_h0_row_norms_max: 6.42187833786 train_h0_row_norms_mean: 3.16045331955 train_h0_row_norms_min: 0.12483151257 train_objective: 0.0282621402293 train_y_col_norms_max: 9.82299423218 train_y_col_norms_mean: 8.56633377075 train_y_col_norms_min: 6.71553659439 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.981470048428 train_y_min_max_class: 0.304827183485 train_y_misclass: 0.00591999944299 train_y_nll: 0.0282621402293 train_y_row_norms_max: 3.65550875664 train_y_row_norms_mean: 1.09536445141 train_y_row_norms_min: 0.163716599345 valid_h0_col_norms_max: 6.44657659531 valid_h0_col_norms_mean: 4.02922582626 valid_h0_col_norms_min: 2.14045143127 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.963276088238 valid_h0_max_x_min_u: 0.567677497864 valid_h0_mean_x_max_u: 0.936259627342 valid_h0_mean_x_mean_u: 0.466583073139 valid_h0_mean_x_min_u: 0.0857717692852 valid_h0_min_x_max_u: 0.344594448805 valid_h0_min_x_mean_u: 0.0292918123305 valid_h0_min_x_min_u: 2.15348190669e-11 valid_h0_row_norms_max: 6.4218788147 valid_h0_row_norms_mean: 3.16045355797 valid_h0_row_norms_min: 0.124831520021 valid_objective: 0.0722089111805 valid_y_col_norms_max: 9.82299423218 valid_y_col_norms_mean: 8.56633377075 valid_y_col_norms_min: 6.71553707123 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.977750241756 valid_y_min_max_class: 0.297483742237 valid_y_misclass: 0.021099999547 valid_y_nll: 0.0722089111805 valid_y_row_norms_max: 3.65550899506 valid_y_row_norms_mean: 1.09536457062 valid_y_row_norms_min: 0.163716614246 Time this epoch: 35.012923 seconds Monitoring step: Epochs seen: 31 Batches seen: 155 Examples seen: 1550000 ave_grad_mult: 3.48831152916 ave_grad_size: 0.0287408661097 ave_step_size: 0.0940494984388 test_h0_col_norms_max: 6.45725250244 test_h0_col_norms_mean: 4.03711128235 test_h0_col_norms_min: 2.14304447174 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.963537156582 test_h0_max_x_min_u: 0.54846316576 test_h0_mean_x_max_u: 0.940077364445 test_h0_mean_x_mean_u: 0.466366380453 test_h0_mean_x_min_u: 0.0763043165207 test_h0_min_x_max_u: 0.358212590218 test_h0_min_x_mean_u: 0.0289474800229 test_h0_min_x_min_u: 2.33178042847e-11 test_h0_row_norms_max: 6.44061088562 test_h0_row_norms_mean: 3.16697835922 test_h0_row_norms_min: 0.125991553068 test_objective: 0.0662763118744 test_y_col_norms_max: 10.0062093735 test_y_col_norms_mean: 8.73837566376 test_y_col_norms_min: 6.84926891327 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.978273510933 test_y_min_max_class: 0.285628795624 test_y_misclass: 0.0193999987096 test_y_nll: 0.0662763118744 test_y_row_norms_max: 3.72132134438 test_y_row_norms_mean: 1.11716985703 test_y_row_norms_min: 0.168285727501 train_h0_col_norms_max: 6.45725250244 train_h0_col_norms_mean: 4.03711175919 train_h0_col_norms_min: 2.14304423332 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.964494287968 train_h0_max_x_min_u: 0.551624715328 train_h0_mean_x_max_u: 0.933692634106 train_h0_mean_x_mean_u: 0.466590344906 train_h0_mean_x_min_u: 0.0803528800607 train_h0_min_x_max_u: 0.359871029854 train_h0_min_x_mean_u: 0.0284454971552 train_h0_min_x_min_u: 3.10430431361e-11 train_h0_row_norms_max: 6.44061088562 train_h0_row_norms_mean: 3.16697835922 train_h0_row_norms_min: 0.125991553068 train_objective: 0.0254283007234 train_y_col_norms_max: 10.0062084198 train_y_col_norms_mean: 8.73837471008 train_y_col_norms_min: 6.84926795959 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.982455551624 train_y_min_max_class: 0.295360028744 train_y_misclass: 0.00481999944896 train_y_nll: 0.0254283007234 train_y_row_norms_max: 3.72132158279 train_y_row_norms_mean: 1.11716985703 train_y_row_norms_min: 0.168285742402 valid_h0_col_norms_max: 6.45725250244 valid_h0_col_norms_mean: 4.03711128235 valid_h0_col_norms_min: 2.14304447174 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.96359193325 valid_h0_max_x_min_u: 0.566685140133 valid_h0_mean_x_max_u: 0.935809135437 valid_h0_mean_x_mean_u: 0.466593444347 valid_h0_mean_x_min_u: 0.0823497697711 valid_h0_min_x_max_u: 0.342845439911 valid_h0_min_x_mean_u: 0.0289610140026 valid_h0_min_x_min_u: 2.35798464088e-11 valid_h0_row_norms_max: 6.44061088562 valid_h0_row_norms_mean: 3.16697835922 valid_h0_row_norms_min: 0.125991553068 valid_objective: 0.0720023438334 valid_y_col_norms_max: 10.0062093735 valid_y_col_norms_mean: 8.73837566376 valid_y_col_norms_min: 6.84926891327 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.978184223175 valid_y_min_max_class: 0.33930772543 valid_y_misclass: 0.0208999998868 valid_y_nll: 0.0720023438334 valid_y_row_norms_max: 3.72132134438 valid_y_row_norms_mean: 1.11716985703 valid_y_row_norms_min: 0.168285727501 Time this epoch: 35.375439 seconds Monitoring step: Epochs seen: 32 Batches seen: 160 Examples seen: 1600000 ave_grad_mult: 3.63046574593 ave_grad_size: 0.0268990695477 ave_step_size: 0.091931194067 test_h0_col_norms_max: 6.46750497818 test_h0_col_norms_mean: 4.04480981827 test_h0_col_norms_min: 2.1463572979 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.96414077282 test_h0_max_x_min_u: 0.551202893257 test_h0_mean_x_max_u: 0.938820004463 test_h0_mean_x_mean_u: 0.466768145561 test_h0_mean_x_min_u: 0.0774406716228 test_h0_min_x_max_u: 0.35613399744 test_h0_min_x_mean_u: 0.0284414924681 test_h0_min_x_min_u: 2.09623499114e-11 test_h0_row_norms_max: 6.45651197433 test_h0_row_norms_mean: 3.17332720757 test_h0_row_norms_min: 0.126796171069 test_objective: 0.0663162916899 test_y_col_norms_max: 10.1816034317 test_y_col_norms_mean: 8.90622425079 test_y_col_norms_min: 6.98152685165 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.978770077229 test_y_min_max_class: 0.294499635696 test_y_misclass: 0.0207000002265 test_y_nll: 0.0663162916899 test_y_row_norms_max: 3.78270602226 test_y_row_norms_mean: 1.13857710361 test_y_row_norms_min: 0.170636937022 train_h0_col_norms_max: 6.46750450134 train_h0_col_norms_mean: 4.04480981827 train_h0_col_norms_min: 2.1463572979 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.965125858784 train_h0_max_x_min_u: 0.553418159485 train_h0_mean_x_max_u: 0.932287812233 train_h0_mean_x_mean_u: 0.466983139515 train_h0_mean_x_min_u: 0.0815795511007 train_h0_min_x_max_u: 0.351988613605 train_h0_min_x_mean_u: 0.0279593002051 train_h0_min_x_min_u: 2.79454966112e-11 train_h0_row_norms_max: 6.4565114975 train_h0_row_norms_mean: 3.17332744598 train_h0_row_norms_min: 0.126796171069 train_objective: 0.0232511665672 train_y_col_norms_max: 10.1816034317 train_y_col_norms_mean: 8.90622425079 train_y_col_norms_min: 6.98152732849 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.98363161087 train_y_min_max_class: 0.305530905724 train_y_misclass: 0.00421999953687 train_y_nll: 0.0232511665672 train_y_row_norms_max: 3.78270626068 train_y_row_norms_mean: 1.13857698441 train_y_row_norms_min: 0.170636937022 valid_h0_col_norms_max: 6.46750497818 valid_h0_col_norms_mean: 4.04480981827 valid_h0_col_norms_min: 2.1463572979 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.964271306992 valid_h0_max_x_min_u: 0.568406701088 valid_h0_mean_x_max_u: 0.934549808502 valid_h0_mean_x_mean_u: 0.466980487108 valid_h0_mean_x_min_u: 0.0836460664868 valid_h0_min_x_max_u: 0.336699128151 valid_h0_min_x_mean_u: 0.028556285426 valid_h0_min_x_min_u: 2.1281049839e-11 valid_h0_row_norms_max: 6.45651197433 valid_h0_row_norms_mean: 3.17332720757 valid_h0_row_norms_min: 0.126796171069 valid_objective: 0.0705031752586 valid_y_col_norms_max: 10.1816034317 valid_y_col_norms_mean: 8.90622425079 valid_y_col_norms_min: 6.98152685165 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.978758752346 valid_y_min_max_class: 0.291737556458 valid_y_misclass: 0.0208999998868 valid_y_nll: 0.0705031752586 valid_y_row_norms_max: 3.78270602226 valid_y_row_norms_mean: 1.13857710361 valid_y_row_norms_min: 0.170636937022 Time this epoch: 35.379330 seconds Monitoring step: Epochs seen: 33 Batches seen: 165 Examples seen: 1650000 ave_grad_mult: 3.85002589226 ave_grad_size: 0.0255950912833 ave_step_size: 0.0920957773924 test_h0_col_norms_max: 6.47780418396 test_h0_col_norms_mean: 4.05291509628 test_h0_col_norms_min: 2.14965701103 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.965208768845 test_h0_max_x_min_u: 0.553891956806 test_h0_mean_x_max_u: 0.941352784634 test_h0_mean_x_mean_u: 0.467216670513 test_h0_mean_x_min_u: 0.0769760459661 test_h0_min_x_max_u: 0.357422113419 test_h0_min_x_mean_u: 0.027681870386 test_h0_min_x_min_u: 1.6821729773e-11 test_h0_row_norms_max: 6.47196292877 test_h0_row_norms_mean: 3.18000507355 test_h0_row_norms_min: 0.127480790019 test_objective: 0.0658261179924 test_y_col_norms_max: 10.3589458466 test_y_col_norms_mean: 9.08145141602 test_y_col_norms_min: 7.12754154205 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.980045855045 test_y_min_max_class: 0.275538861752 test_y_misclass: 0.019999999553 test_y_nll: 0.0658261179924 test_y_row_norms_max: 3.84528589249 test_y_row_norms_mean: 1.16080152988 test_y_row_norms_min: 0.173613965511 train_h0_col_norms_max: 6.47780418396 train_h0_col_norms_mean: 4.05291461945 train_h0_col_norms_min: 2.14965701103 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.966172575951 train_h0_max_x_min_u: 0.551838994026 train_h0_mean_x_max_u: 0.935074448586 train_h0_mean_x_mean_u: 0.46742233634 train_h0_mean_x_min_u: 0.0811282843351 train_h0_min_x_max_u: 0.346001476049 train_h0_min_x_mean_u: 0.0272373519838 train_h0_min_x_min_u: 2.30680283902e-11 train_h0_row_norms_max: 6.47196340561 train_h0_row_norms_mean: 3.18000459671 train_h0_row_norms_min: 0.127480790019 train_objective: 0.02110886015 train_y_col_norms_max: 10.3589458466 train_y_col_norms_mean: 9.08145141602 train_y_col_norms_min: 7.12754058838 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.984871923923 train_y_min_max_class: 0.292335510254 train_y_misclass: 0.00331999990158 train_y_nll: 0.02110886015 train_y_row_norms_max: 3.84528613091 train_y_row_norms_mean: 1.16080152988 train_y_row_norms_min: 0.173613965511 valid_h0_col_norms_max: 6.47780418396 valid_h0_col_norms_mean: 4.05291509628 valid_h0_col_norms_min: 2.14965701103 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.965333819389 valid_h0_max_x_min_u: 0.567090988159 valid_h0_mean_x_max_u: 0.937143027782 valid_h0_mean_x_mean_u: 0.467420905828 valid_h0_mean_x_min_u: 0.0815225914121 valid_h0_min_x_max_u: 0.317524284124 valid_h0_min_x_mean_u: 0.0278288982809 valid_h0_min_x_min_u: 1.7383304865e-11 valid_h0_row_norms_max: 6.47196292877 valid_h0_row_norms_mean: 3.18000507355 valid_h0_row_norms_min: 0.127480790019 valid_objective: 0.0706555917859 valid_y_col_norms_max: 10.3589458466 valid_y_col_norms_mean: 9.08145141602 valid_y_col_norms_min: 7.12754154205 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.979981780052 valid_y_min_max_class: 0.314534544945 valid_y_misclass: 0.0206000003964 valid_y_nll: 0.0706555917859 valid_y_row_norms_max: 3.84528589249 valid_y_row_norms_mean: 1.16080152988 valid_y_row_norms_min: 0.173613965511 Time this epoch: 35.182908 seconds Monitoring step: Epochs seen: 34 Batches seen: 170 Examples seen: 1700000 ave_grad_mult: 4.07905960083 ave_grad_size: 0.0242200661451 ave_step_size: 0.0924715399742 test_h0_col_norms_max: 6.48900747299 test_h0_col_norms_mean: 4.06139850616 test_h0_col_norms_min: 2.1522192955 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.965900540352 test_h0_max_x_min_u: 0.551373183727 test_h0_mean_x_max_u: 0.942069590092 test_h0_mean_x_mean_u: 0.467438340187 test_h0_mean_x_min_u: 0.0787537544966 test_h0_min_x_max_u: 0.359593838453 test_h0_min_x_mean_u: 0.0271878745407 test_h0_min_x_min_u: 1.29720010775e-11 test_h0_row_norms_max: 6.49045753479 test_h0_row_norms_mean: 3.18700146675 test_h0_row_norms_min: 0.128459200263 test_objective: 0.0644877254963 test_y_col_norms_max: 10.5396261215 test_y_col_norms_mean: 9.26142787933 test_y_col_norms_min: 7.278901577 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.98046040535 test_y_min_max_class: 0.25162255764 test_y_misclass: 0.0206000003964 test_y_nll: 0.0644877254963 test_y_row_norms_max: 3.90689897537 test_y_row_norms_mean: 1.18369758129 test_y_row_norms_min: 0.177592679858 train_h0_col_norms_max: 6.48900747299 train_h0_col_norms_mean: 4.06139850616 train_h0_col_norms_min: 2.15221905708 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.966917276382 train_h0_max_x_min_u: 0.551108419895 train_h0_mean_x_max_u: 0.935860812664 train_h0_mean_x_mean_u: 0.467652916908 train_h0_mean_x_min_u: 0.0830486863852 train_h0_min_x_max_u: 0.34869286418 train_h0_min_x_mean_u: 0.0267758108675 train_h0_min_x_min_u: 1.72074004351e-11 train_h0_row_norms_max: 6.49045705795 train_h0_row_norms_mean: 3.18700098991 train_h0_row_norms_min: 0.128459185362 train_objective: 0.0193602163345 train_y_col_norms_max: 10.5396251678 train_y_col_norms_mean: 9.26142692566 train_y_col_norms_min: 7.27890205383 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.985767424107 train_y_min_max_class: 0.336476325989 train_y_misclass: 0.00289999973029 train_y_nll: 0.0193602163345 train_y_row_norms_max: 3.90689897537 train_y_row_norms_mean: 1.18369758129 train_y_row_norms_min: 0.177592664957 valid_h0_col_norms_max: 6.48900747299 valid_h0_col_norms_mean: 4.06139850616 valid_h0_col_norms_min: 2.1522192955 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.966026246548 valid_h0_max_x_min_u: 0.568945586681 valid_h0_mean_x_max_u: 0.937908649445 valid_h0_mean_x_mean_u: 0.467648357153 valid_h0_mean_x_min_u: 0.0835038796067 valid_h0_min_x_max_u: 0.32678771019 valid_h0_min_x_mean_u: 0.0274151265621 valid_h0_min_x_min_u: 1.33190177637e-11 valid_h0_row_norms_max: 6.49045753479 valid_h0_row_norms_mean: 3.18700146675 valid_h0_row_norms_min: 0.128459200263 valid_objective: 0.0707407668233 valid_y_col_norms_max: 10.5396261215 valid_y_col_norms_mean: 9.26142787933 valid_y_col_norms_min: 7.278901577 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.980556607246 valid_y_min_max_class: 0.298538506031 valid_y_misclass: 0.0219000000507 valid_y_nll: 0.0707407668233 valid_y_row_norms_max: 3.90689897537 valid_y_row_norms_mean: 1.18369758129 valid_y_row_norms_min: 0.177592679858 Time this epoch: 35.400439 seconds Monitoring step: Epochs seen: 35 Batches seen: 175 Examples seen: 1750000 ave_grad_mult: 4.3184633255 ave_grad_size: 0.022776318714 ave_step_size: 0.0920493155718 test_h0_col_norms_max: 6.49970197678 test_h0_col_norms_mean: 4.06945180893 test_h0_col_norms_min: 2.15374016762 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.966324806213 test_h0_max_x_min_u: 0.55324202776 test_h0_mean_x_max_u: 0.94215297699 test_h0_mean_x_mean_u: 0.466998904943 test_h0_mean_x_min_u: 0.076045922935 test_h0_min_x_max_u: 0.358031690121 test_h0_min_x_mean_u: 0.0268286950886 test_h0_min_x_min_u: 1.09078423377e-11 test_h0_row_norms_max: 6.50839042664 test_h0_row_norms_mean: 3.19361257553 test_h0_row_norms_min: 0.129599049687 test_objective: 0.0635969266295 test_y_col_norms_max: 10.7156534195 test_y_col_norms_mean: 9.42882728577 test_y_col_norms_min: 7.41432905197 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.980757176876 test_y_min_max_class: 0.248226299882 test_y_misclass: 0.0193999987096 test_y_nll: 0.0635969266295 test_y_row_norms_max: 3.96717524529 test_y_row_norms_mean: 1.20508480072 test_y_row_norms_min: 0.180099412799 train_h0_col_norms_max: 6.4997010231 train_h0_col_norms_mean: 4.06945180893 train_h0_col_norms_min: 2.1537399292 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.967268288136 train_h0_max_x_min_u: 0.551735162735 train_h0_mean_x_max_u: 0.935940921307 train_h0_mean_x_mean_u: 0.467215240002 train_h0_mean_x_min_u: 0.0804425179958 train_h0_min_x_max_u: 0.343524694443 train_h0_min_x_mean_u: 0.0264016315341 train_h0_min_x_min_u: 1.46074211754e-11 train_h0_row_norms_max: 6.50839090347 train_h0_row_norms_mean: 3.19361257553 train_h0_row_norms_min: 0.129599064589 train_objective: 0.0171565413475 train_y_col_norms_max: 10.7156524658 train_y_col_norms_mean: 9.42882633209 train_y_col_norms_min: 7.41432905197 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.986733615398 train_y_min_max_class: 0.337424963713 train_y_misclass: 0.00196000002325 train_y_nll: 0.0171565413475 train_y_row_norms_max: 3.96717500687 train_y_row_norms_mean: 1.20508468151 train_y_row_norms_min: 0.1800994277 valid_h0_col_norms_max: 6.49970197678 valid_h0_col_norms_mean: 4.06945180893 valid_h0_col_norms_min: 2.15374016762 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.966409444809 valid_h0_max_x_min_u: 0.564903616905 valid_h0_mean_x_max_u: 0.938026130199 valid_h0_mean_x_mean_u: 0.467201501131 valid_h0_mean_x_min_u: 0.0823206305504 valid_h0_min_x_max_u: 0.31958258152 valid_h0_min_x_mean_u: 0.0270514041185 valid_h0_min_x_min_u: 1.14466379084e-11 valid_h0_row_norms_max: 6.50839042664 valid_h0_row_norms_mean: 3.19361257553 valid_h0_row_norms_min: 0.129599049687 valid_objective: 0.0689148977399 valid_y_col_norms_max: 10.7156534195 valid_y_col_norms_mean: 9.42882728577 valid_y_col_norms_min: 7.41432905197 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.980640649796 valid_y_min_max_class: 0.264637023211 valid_y_misclass: 0.021099999547 valid_y_nll: 0.0689148977399 valid_y_row_norms_max: 3.96717524529 valid_y_row_norms_mean: 1.20508480072 valid_y_row_norms_min: 0.180099412799 Time this epoch: 35.392445 seconds Monitoring step: Epochs seen: 36 Batches seen: 180 Examples seen: 1800000 ave_grad_mult: 4.55049180984 ave_grad_size: 0.0215135067701 ave_step_size: 0.0914682373405 test_h0_col_norms_max: 6.5103468895 test_h0_col_norms_mean: 4.07773399353 test_h0_col_norms_min: 2.15385961533 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.967212915421 test_h0_max_x_min_u: 0.559629559517 test_h0_mean_x_max_u: 0.943103969097 test_h0_mean_x_mean_u: 0.466701477766 test_h0_mean_x_min_u: 0.0809521302581 test_h0_min_x_max_u: 0.355958789587 test_h0_min_x_mean_u: 0.0261587612331 test_h0_min_x_min_u: 8.34188967902e-12 test_h0_row_norms_max: 6.52430438995 test_h0_row_norms_mean: 3.20039582253 test_h0_row_norms_min: 0.130876362324 test_objective: 0.0621786899865 test_y_col_norms_max: 10.8925733566 test_y_col_norms_mean: 9.60350131989 test_y_col_norms_min: 7.55749177933 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.981753587723 test_y_min_max_class: 0.330662488937 test_y_misclass: 0.0190999973565 test_y_nll: 0.0621786899865 test_y_row_norms_max: 4.02781057358 test_y_row_norms_mean: 1.22741234303 test_y_row_norms_min: 0.181874185801 train_h0_col_norms_max: 6.51034593582 train_h0_col_norms_mean: 4.07773399353 train_h0_col_norms_min: 2.15385937691 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.968181371689 train_h0_max_x_min_u: 0.555752873421 train_h0_mean_x_max_u: 0.936968684196 train_h0_mean_x_mean_u: 0.466913819313 train_h0_mean_x_min_u: 0.0854497775435 train_h0_min_x_max_u: 0.338039875031 train_h0_min_x_mean_u: 0.0257331542671 train_h0_min_x_min_u: 1.09208597027e-11 train_h0_row_norms_max: 6.52430438995 train_h0_row_norms_mean: 3.20039534569 train_h0_row_norms_min: 0.130876347423 train_objective: 0.0157043337822 train_y_col_norms_max: 10.892572403 train_y_col_norms_mean: 9.60350131989 train_y_col_norms_min: 7.55749130249 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.987937808037 train_y_min_max_class: 0.323045521975 train_y_misclass: 0.00203999993391 train_y_nll: 0.0157043337822 train_y_row_norms_max: 4.02781057358 train_y_row_norms_mean: 1.22741222382 train_y_row_norms_min: 0.181874185801 valid_h0_col_norms_max: 6.5103468895 valid_h0_col_norms_mean: 4.07773399353 valid_h0_col_norms_min: 2.15385961533 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.967329084873 valid_h0_max_x_min_u: 0.567208707333 valid_h0_mean_x_max_u: 0.939025402069 valid_h0_mean_x_mean_u: 0.466897398233 valid_h0_mean_x_min_u: 0.0875384286046 valid_h0_min_x_max_u: 0.311605006456 valid_h0_min_x_mean_u: 0.0264036990702 valid_h0_min_x_min_u: 8.81185142215e-12 valid_h0_row_norms_max: 6.52430438995 valid_h0_row_norms_mean: 3.20039582253 valid_h0_row_norms_min: 0.130876362324 valid_objective: 0.0682094246149 valid_y_col_norms_max: 10.8925733566 valid_y_col_norms_mean: 9.60350131989 valid_y_col_norms_min: 7.55749177933 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.982199847698 valid_y_min_max_class: 0.324392050505 valid_y_misclass: 0.0208000000566 valid_y_nll: 0.0682094246149 valid_y_row_norms_max: 4.02781057358 valid_y_row_norms_mean: 1.22741234303 valid_y_row_norms_min: 0.181874185801 Time this epoch: 34.710048 seconds Monitoring step: Epochs seen: 37 Batches seen: 185 Examples seen: 1850000 ave_grad_mult: 4.72839355469 ave_grad_size: 0.0204669237137 ave_step_size: 0.0900116711855 test_h0_col_norms_max: 6.52083969116 test_h0_col_norms_mean: 4.08554124832 test_h0_col_norms_min: 2.15409636497 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.967603981495 test_h0_max_x_min_u: 0.557713389397 test_h0_mean_x_max_u: 0.943118810654 test_h0_mean_x_mean_u: 0.466896891594 test_h0_mean_x_min_u: 0.0787230879068 test_h0_min_x_max_u: 0.356404840946 test_h0_min_x_mean_u: 0.0257684588432 test_h0_min_x_min_u: 9.0589402299e-12 test_h0_row_norms_max: 6.53848934174 test_h0_row_norms_mean: 3.206792593 test_h0_row_norms_min: 0.131754085422 test_objective: 0.0623081922531 test_y_col_norms_max: 11.052611351 test_y_col_norms_mean: 9.76351451874 test_y_col_norms_min: 7.68663883209 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.982231199741 test_y_min_max_class: 0.287253022194 test_y_misclass: 0.0188999995589 test_y_nll: 0.0623081922531 test_y_row_norms_max: 4.08399629593 test_y_row_norms_mean: 1.24770605564 test_y_row_norms_min: 0.185720145702 train_h0_col_norms_max: 6.52083921432 train_h0_col_norms_mean: 4.08554124832 train_h0_col_norms_min: 2.15409636497 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.968553900719 train_h0_max_x_min_u: 0.553573608398 train_h0_mean_x_max_u: 0.937007129192 train_h0_mean_x_mean_u: 0.467112243176 train_h0_mean_x_min_u: 0.0832107812166 train_h0_min_x_max_u: 0.334134042263 train_h0_min_x_mean_u: 0.0253749713302 train_h0_min_x_min_u: 1.24377470129e-11 train_h0_row_norms_max: 6.5384888649 train_h0_row_norms_mean: 3.20679235458 train_h0_row_norms_min: 0.131754085422 train_objective: 0.0140552837402 train_y_col_norms_max: 11.0526103973 train_y_col_norms_mean: 9.76351451874 train_y_col_norms_min: 7.68663883209 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.988768100739 train_y_min_max_class: 0.329038023949 train_y_misclass: 0.00163999991491 train_y_nll: 0.0140552837402 train_y_row_norms_max: 4.08399581909 train_y_row_norms_mean: 1.24770605564 train_y_row_norms_min: 0.1857201159 valid_h0_col_norms_max: 6.52083969116 valid_h0_col_norms_mean: 4.08554124832 valid_h0_col_norms_min: 2.15409636497 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.967728018761 valid_h0_max_x_min_u: 0.569734930992 valid_h0_mean_x_max_u: 0.939063310623 valid_h0_mean_x_mean_u: 0.467084676027 valid_h0_mean_x_min_u: 0.0852277651429 valid_h0_min_x_max_u: 0.310160905123 valid_h0_min_x_mean_u: 0.0260545928031 valid_h0_min_x_min_u: 9.77232097327e-12 valid_h0_row_norms_max: 6.53848934174 valid_h0_row_norms_mean: 3.206792593 valid_h0_row_norms_min: 0.131754085422 valid_objective: 0.0679266303778 valid_y_col_norms_max: 11.052611351 valid_y_col_norms_mean: 9.76351451874 valid_y_col_norms_min: 7.68663883209 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.982333242893 valid_y_min_max_class: 0.319318085909 valid_y_misclass: 0.0204000007361 valid_y_nll: 0.0679266303778 valid_y_row_norms_max: 4.08399629593 valid_y_row_norms_mean: 1.24770605564 valid_y_row_norms_min: 0.185720145702 Time this epoch: 35.364850 seconds Monitoring step: Epochs seen: 38 Batches seen: 190 Examples seen: 1900000 ave_grad_mult: 5.14290428162 ave_grad_size: 0.0190559756011 ave_step_size: 0.0913925841451 test_h0_col_norms_max: 6.53183841705 test_h0_col_norms_mean: 4.09429168701 test_h0_col_norms_min: 2.1546792984 test_h0_max_x_max_u: 1.0 test_h0_max_x_mean_u: 0.968341529369 test_h0_max_x_min_u: 0.560349822044 test_h0_mean_x_max_u: 0.94111353159 test_h0_mean_x_mean_u: 0.466603428125 test_h0_mean_x_min_u: 0.0797407329082 test_h0_min_x_max_u: 0.351446330547 test_h0_min_x_mean_u: 0.0251497104764 test_h0_min_x_min_u: 7.31677132076e-12 test_h0_row_norms_max: 6.55737257004 test_h0_row_norms_mean: 3.21393060684 test_h0_row_norms_min: 0.132736563683 test_objective: 0.0633104071021 test_y_col_norms_max: 11.2310876846 test_y_col_norms_mean: 9.94289398193 test_y_col_norms_min: 7.82843732834 test_y_max_max_class: 1.0 test_y_mean_max_class: 0.982944607735 test_y_min_max_class: 0.318380922079 test_y_misclass: 0.0193999987096 test_y_nll: 0.0633104071021 test_y_row_norms_max: 4.14330053329 test_y_row_norms_mean: 1.27068781853 test_y_row_norms_min: 0.189937055111 train_h0_col_norms_max: 6.53183746338 train_h0_col_norms_mean: 4.09429121017 train_h0_col_norms_min: 2.15467905998 train_h0_max_x_max_u: 0.999999940395 train_h0_max_x_mean_u: 0.969276428223 train_h0_max_x_min_u: 0.554496645927 train_h0_mean_x_max_u: 0.934813499451 train_h0_mean_x_mean_u: 0.466816186905 train_h0_mean_x_min_u: 0.0843253731728 train_h0_min_x_max_u: 0.332267045975 train_h0_min_x_mean_u: 0.0247781910002 train_h0_min_x_min_u: 9.73409200467e-12 train_h0_row_norms_max: 6.5573720932 train_h0_row_norms_mean: 3.21393036842 train_h0_row_norms_min: 0.132736548781 train_objective: 0.0125638237223 train_y_col_norms_max: 11.231086731 train_y_col_norms_mean: 9.94289398193 train_y_col_norms_min: 7.82843637466 train_y_max_max_class: 0.999999940395 train_y_mean_max_class: 0.989765167236 train_y_min_max_class: 0.37343031168 train_y_misclass: 0.00133999995887 train_y_nll: 0.0125638237223 train_y_row_norms_max: 4.14330005646 train_y_row_norms_mean: 1.27068758011 train_y_row_norms_min: 0.189937055111 valid_h0_col_norms_max: 6.53183841705 valid_h0_col_norms_mean: 4.09429168701 valid_h0_col_norms_min: 2.1546792984 valid_h0_max_x_max_u: 1.0 valid_h0_max_x_mean_u: 0.968490362167 valid_h0_max_x_min_u: 0.566503345966 valid_h0_mean_x_max_u: 0.93706715107 valid_h0_mean_x_mean_u: 0.466789364815 valid_h0_mean_x_min_u: 0.0863413140178 valid_h0_min_x_max_u: 0.307645887136 valid_h0_min_x_mean_u: 0.0253705345094 valid_h0_min_x_min_u: 7.80784985277e-12 valid_h0_row_norms_max: 6.55737257004 valid_h0_row_norms_mean: 3.21393060684 valid_h0_row_norms_min: 0.132736563683 valid_objective: 0.0684154629707 valid_y_col_norms_max: 11.2310876846 valid_y_col_norms_mean: 9.94289398193 valid_y_col_norms_min: 7.82843732834 valid_y_max_max_class: 1.0 valid_y_mean_max_class: 0.983206391335 valid_y_min_max_class: 0.354223191738 valid_y_misclass: 0.0201999973506 valid_y_nll: 0.0684154629707 valid_y_row_norms_max: 4.14330053329 valid_y_row_norms_mean: 1.27068781853 valid_y_row_norms_min: 0.189937055111
As the model trained, it should have printed out progress messages. Most of these are the values of the various channels being monitored throughout training.
We can use the print_monitor script to print the last monitoring entry of a saved model. By running it on "mlp_best.pkl", we can see the performance of the model at the point where it did the best on the validation set.
!print_monitor.py mlp_best.pkl | grep test_y_misclass
Using gpu device 2: GeForce GTX 285 /u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit. warnings.warn("MLP changing the recursion limit.") test_y_misclass : 0.0193999987096
The test set error has dropped to 1.94%! This is a big improvement over softmax regression.
Another common way of analyzing trained models is to look at their weights. Here we use the show_weights script to visualize $W$:
!show_weights.py mlp_best.pkl
Using gpu device 0: GeForce GTX 285 making weights report loading model loading done loading dataset... ...done smallest enc weight magnitude: 0.0 mean enc weight magnitude: 0.0409141770966 max enc weight magnitude: 4.76068 min norm: 2.15468 mean norm: 4.09429199219 max norm: 6.53184
So far in these tutorials, there has not been much benefit to using pylearn2, rather than some other machine learning library, or even just an implementation of softmax regression or an MLP without an accompanying library.
Now it's time to see some of why pylearn2 is useful. We're going to make several changes to our experimental setup, while still re-using most of the code. The beauty of pylearn2 is that it is built from interchangeable parts, so that if you want to create a new machine learning experiment, you don't need to rewrite the whole experiment from scratch.
We're going to take the MLP example above and change it in three major ways:
-Instead of training just a two layer MLP, we'll train a three layer MLP. We can do this just by putting one more layer in the "layers" list. We don't need to change the training algorithm or the main MLP model.
-Instead of using the Sigmoid Layer class, we'll use a different kind of layer, called a rectified linear layer. The rectified linear layer uses the usual affine function $z = x^T W + b$ to compute the presynaptic inputs, then passes each element of $z$ through the function $g(z) = \mathbb{I}_{z > 0} z$. In other words, values greater than 0 are left unchanged, while negative values are replaced with zeros. In pylearn2, we can do this just by loading a different class in the layers list. We don't need to change the training algorithm or the main MLP model.
-Instead of optimizing the log likelihood using the nonlinear conjugate gradient descent algorithm, we will optimize it using a minibatch version of stochastic gradient descent. We can do this just by passing in a different TrainingAlgorithm object. No changes to the model or the code for the cost are needed.
Here is the updated YAML description of the experiment:
import os
import pylearn2
path = os.path.join(pylearn2.__path__[0], 'scripts', 'tutorials', 'multilayer_perceptron', 'mlp_tutorial_part_3.yaml')
with open(path, 'r') as f:
train_2 = f.read()
hyper_params = {'train_stop' : 50000,
'valid_stop' : 60000,
'dim_h0' : 500,
'dim_h1' : 1000,
'sparse_init_h1' : 15,
'max_epochs' : 10000,
'save_path' : '.'}
train_2 = train_2 % (hyper_params)
print train_2
!obj:pylearn2.train.Train { dataset: &train !obj:pylearn2.datasets.mnist.MNIST { which_set: 'train', start: 0, stop: 50000 }, model: !obj:pylearn2.models.mlp.MLP { layers: [ !obj:pylearn2.models.mlp.RectifiedLinear { layer_name: 'h0', dim: 500, sparse_init: 15 }, !obj:pylearn2.models.mlp.RectifiedLinear { layer_name: 'h1', dim: 1000, sparse_init: 15 }, !obj:pylearn2.models.mlp.Softmax { layer_name: 'y', n_classes: 10, irange: 0. } ], nvis: 784, }, algorithm: !obj:pylearn2.training_algorithms.sgd.SGD { batch_size: 100, learning_rate: .01, monitoring_dataset: { 'train' : *train, 'valid' : !obj:pylearn2.datasets.mnist.MNIST { which_set: 'train', start: 50000, stop: 60000 }, 'test' : !obj:pylearn2.datasets.mnist.MNIST { which_set: 'test', } }, learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum { init_momentum: .5 }, termination_criterion: !obj:pylearn2.termination_criteria.And { criteria: [ !obj:pylearn2.termination_criteria.MonitorBased { channel_name: "valid_y_misclass", prop_decrease: 0., N: 10 }, !obj:pylearn2.termination_criteria.EpochCounter { max_epochs: 10000 } ] } }, extensions: [ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest { channel_name: 'valid_y_misclass', save_path: "mlp_2_best.pkl" }, !obj:pylearn2.training_algorithms.learning_rule.MomentumAdjustor { start: 1, saturate: 10, final_momentum: .99 } ] }
This YAML config file also introduces another use of extensions to the Train object. Here, we add the MomentumAdjustor. It uses a callback to adjust the momentum setting of the SGD algorithm at the end of each epoch. Here, we configure it to start increasing the momentum after 1 epoch, and to continue increasing it until it reaches a value of .99 at the end of the tenth epoch. See the docstring for the SGD class for more information on what this momentum setting does.
from pylearn2.config import yaml_parse
train_2 = yaml_parse.load(train_2)
train_2.main_loop()
Parameter and initial learning rate summary: h0_W: 0.00999999977648 h0_b: 0.00999999977648 h1_W: 0.00999999977648 h1_b: 0.00999999977648 softmax_b: 0.00999999977648 softmax_W: 0.00999999977648 Compiling sgd_update... Compiling sgd_update done. Time elapsed: 2.516152 seconds compiling begin_record_entry... compiling begin_record_entry done. Time elapsed: 0.395491 seconds Monitored channels: learning_rate momentum test_h0_col_norms_max test_h0_col_norms_mean test_h0_col_norms_min test_h0_row_norms_max test_h0_row_norms_mean test_h0_row_norms_min test_h1_col_norms_max test_h1_col_norms_mean test_h1_col_norms_min test_h1_row_norms_max test_h1_row_norms_mean test_h1_row_norms_min test_objective test_y_col_norms_max test_y_col_norms_mean test_y_col_norms_min test_y_max_max_class test_y_mean_max_class test_y_min_max_class test_y_misclass test_y_nll test_y_row_norms_max test_y_row_norms_mean test_y_row_norms_min train_h0_col_norms_max train_h0_col_norms_mean train_h0_col_norms_min train_h0_row_norms_max train_h0_row_norms_mean train_h0_row_norms_min train_h1_col_norms_max train_h1_col_norms_mean train_h1_col_norms_min train_h1_row_norms_max train_h1_row_norms_mean train_h1_row_norms_min train_objective train_y_col_norms_max train_y_col_norms_mean train_y_col_norms_min train_y_max_max_class train_y_mean_max_class train_y_min_max_class train_y_misclass train_y_nll train_y_row_norms_max train_y_row_norms_mean train_y_row_norms_min valid_h0_col_norms_max valid_h0_col_norms_mean valid_h0_col_norms_min valid_h0_row_norms_max valid_h0_row_norms_mean valid_h0_row_norms_min valid_h1_col_norms_max valid_h1_col_norms_mean valid_h1_col_norms_min valid_h1_row_norms_max valid_h1_row_norms_mean valid_h1_row_norms_min valid_objective valid_y_col_norms_max valid_y_col_norms_mean valid_y_col_norms_min valid_y_max_max_class valid_y_mean_max_class valid_y_min_max_class valid_y_misclass valid_y_nll valid_y_row_norms_max valid_y_row_norms_mean valid_y_row_norms_min Compiling accum... graph size: 165 graph size: 163 graph size: 163 Compiling accum done. Time elapsed: 11.563393 seconds Monitoring step: Epochs seen: 0 Batches seen: 0 Examples seen: 0 learning_rate: 0.00999999046326 momentum: 0.499999672174 test_h0_col_norms_max: 6.23503017426 test_h0_col_norms_mean: 3.82356023788 test_h0_col_norms_min: 2.06193947792 test_h0_row_norms_max: 5.89326524734 test_h0_row_norms_mean: 2.98549389839 test_h0_row_norms_min: 0.0 test_h1_col_norms_max: 5.99438333511 test_h1_col_norms_mean: 3.80721712112 test_h1_col_norms_min: 1.71524214745 test_h1_row_norms_max: 7.80886650085 test_h1_row_norms_mean: 5.40815734863 test_h1_row_norms_min: 2.97773504257 test_objective: 2.30258488655 test_y_col_norms_max: 0.0 test_y_col_norms_mean: 0.0 test_y_col_norms_min: 0.0 test_y_max_max_class: 0.100000023842 test_y_mean_max_class: 0.100000031292 test_y_min_max_class: 0.100000023842 test_y_misclass: 0.901999890804 test_y_nll: 2.30258488655 test_y_row_norms_max: 0.0 test_y_row_norms_mean: 0.0 test_y_row_norms_min: 0.0 train_h0_col_norms_max: 6.23505115509 train_h0_col_norms_mean: 3.82354259491 train_h0_col_norms_min: 2.0619494915 train_h0_row_norms_max: 5.89324569702 train_h0_row_norms_mean: 2.98548007011 train_h0_row_norms_min: 0.0 train_h1_col_norms_max: 5.99438095093 train_h1_col_norms_mean: 3.80721092224 train_h1_col_norms_min: 1.71524274349 train_h1_row_norms_max: 7.80887794495 train_h1_row_norms_mean: 5.40813541412 train_h1_row_norms_min: 2.97772955894 train_objective: 2.30257916451 train_y_col_norms_max: 0.0 train_y_col_norms_mean: 0.0 train_y_col_norms_min: 0.0 train_y_max_max_class: 0.100000545382 train_y_mean_max_class: 0.100000545382 train_y_min_max_class: 0.100000545382 train_y_misclass: 0.901360213757 train_y_nll: 2.30257916451 train_y_row_norms_max: 0.0 train_y_row_norms_mean: 0.0 train_y_row_norms_min: 0.0 valid_h0_col_norms_max: 6.23503017426 valid_h0_col_norms_mean: 3.82356023788 valid_h0_col_norms_min: 2.06193947792 valid_h0_row_norms_max: 5.89326524734 valid_h0_row_norms_mean: 2.98549389839 valid_h0_row_norms_min: 0.0 valid_h1_col_norms_max: 5.99438333511 valid_h1_col_norms_mean: 3.80721712112 valid_h1_col_norms_min: 1.71524214745 valid_h1_row_norms_max: 7.80886650085 valid_h1_row_norms_mean: 5.40815734863 valid_h1_row_norms_min: 2.97773504257 valid_objective: 2.30258488655 valid_y_col_norms_max: 0.0 valid_y_col_norms_mean: 0.0 valid_y_col_norms_min: 0.0 valid_y_max_max_class: 0.100000023842 valid_y_mean_max_class: 0.100000031292 valid_y_min_max_class: 0.100000023842 valid_y_misclass: 0.90089994669 valid_y_nll: 2.30258488655 valid_y_row_norms_max: 0.0 valid_y_row_norms_mean: 0.0 valid_y_row_norms_min: 0.0 Time this epoch: 3.343442 seconds Monitoring step: Epochs seen: 1 Batches seen: 500 Examples seen: 50000 learning_rate: 0.00999999046326 momentum: 0.499999672174 test_h0_col_norms_max: 6.23488473892 test_h0_col_norms_mean: 3.82359194756 test_h0_col_norms_min: 2.06265735626 test_h0_row_norms_max: 5.89264249802 test_h0_row_norms_mean: 2.98556685448 test_h0_row_norms_min: 0.00163861282635 test_h1_col_norms_max: 5.99485731125 test_h1_col_norms_mean: 3.80723309517 test_h1_col_norms_min: 1.71526324749 test_h1_row_norms_max: 7.80893564224 test_h1_row_norms_mean: 5.40817546844 test_h1_row_norms_min: 2.97778272629 test_objective: 0.268750548363 test_y_col_norms_max: 0.645500898361 test_y_col_norms_mean: 0.596350252628 test_y_col_norms_min: 0.520334303379 test_y_max_max_class: 0.999946475029 test_y_mean_max_class: 0.904475390911 test_y_min_max_class: 0.38064879179 test_y_misclass: 0.0812000110745 test_y_nll: 0.268750548363 test_y_row_norms_max: 0.17966529727 test_y_row_norms_mean: 0.0518538914621 test_y_row_norms_min: 0.000149252169649 train_h0_col_norms_max: 6.23488473892 train_h0_col_norms_mean: 3.82361268997 train_h0_col_norms_min: 2.06266713142 train_h0_row_norms_max: 5.89267301559 train_h0_row_norms_mean: 2.98556661606 train_h0_row_norms_min: 0.001638607122 train_h1_col_norms_max: 5.99485683441 train_h1_col_norms_mean: 3.80721235275 train_h1_col_norms_min: 1.71525621414 train_h1_row_norms_max: 7.80892753601 train_h1_row_norms_mean: 5.4081993103 train_h1_row_norms_min: 2.97776818275 train_objective: 0.264730095863 train_y_col_norms_max: 0.645499527454 train_y_col_norms_mean: 0.596347033978 train_y_col_norms_min: 0.520334303379 train_y_max_max_class: 0.999963521957 train_y_mean_max_class: 0.899078428745 train_y_min_max_class: 0.361695259809 train_y_misclass: 0.0793600603938 train_y_nll: 0.264730095863 train_y_row_norms_max: 0.179665282369 train_y_row_norms_mean: 0.051854070276 train_y_row_norms_min: 0.000149251762195 valid_h0_col_norms_max: 6.23488473892 valid_h0_col_norms_mean: 3.82359194756 valid_h0_col_norms_min: 2.06265735626 valid_h0_row_norms_max: 5.89264249802 valid_h0_row_norms_mean: 2.98556685448 valid_h0_row_norms_min: 0.00163861282635 valid_h1_col_norms_max: 5.99485731125 valid_h1_col_norms_mean: 3.80723309517 valid_h1_col_norms_min: 1.71526324749 valid_h1_row_norms_max: 7.80893564224 valid_h1_row_norms_mean: 5.40817546844 valid_h1_row_norms_min: 2.97778272629 valid_objective: 0.252131432295 valid_y_col_norms_max: 0.645500898361 valid_y_col_norms_mean: 0.596350252628 valid_y_col_norms_min: 0.520334303379 valid_y_max_max_class: 0.999965012074 valid_y_mean_max_class: 0.907301902771 valid_y_min_max_class: 0.362495720387 valid_y_misclass: 0.0754000097513 valid_y_nll: 0.252131432295 valid_y_row_norms_max: 0.17966529727 valid_y_row_norms_mean: 0.0518538914621 valid_y_row_norms_min: 0.000149252169649 Time this epoch: 3.325040 seconds Monitoring step: Epochs seen: 2 Batches seen: 1000 Examples seen: 100000 learning_rate: 0.00999999046326 momentum: 0.554444551468 test_h0_col_norms_max: 6.2346944809 test_h0_col_norms_mean: 3.82387781143 test_h0_col_norms_min: 2.06334352493 test_h0_row_norms_max: 5.89264249802 test_h0_row_norms_mean: 2.98581314087 test_h0_row_norms_min: 0.00337248062715 test_h1_col_norms_max: 5.99546384811 test_h1_col_norms_mean: 3.80735421181 test_h1_col_norms_min: 1.71530222893 test_h1_row_norms_max: 7.80887699127 test_h1_row_norms_mean: 5.40835094452 test_h1_row_norms_min: 2.97777676582 test_objective: 0.209201917052 test_y_col_norms_max: 0.849824726582 test_y_col_norms_mean: 0.752399742603 test_y_col_norms_min: 0.648707330227 test_y_max_max_class: 0.999981224537 test_y_mean_max_class: 0.928354024887 test_y_min_max_class: 0.417280673981 test_y_misclass: 0.0621000118554 test_y_nll: 0.209201917052 test_y_row_norms_max: 0.202846974134 test_y_row_norms_mean: 0.0668164640665 test_y_row_norms_min: 0.000276584294625 train_h0_col_norms_max: 6.23466491699 train_h0_col_norms_mean: 3.82387685776 train_h0_col_norms_min: 2.06333851814 train_h0_row_norms_max: 5.89267301559 train_h0_row_norms_mean: 2.98582696915 train_h0_row_norms_min: 0.00337246293202 train_h1_col_norms_max: 5.99549293518 train_h1_col_norms_mean: 3.80733585358 train_h1_col_norms_min: 1.71530234814 train_h1_row_norms_max: 7.80891132355 train_h1_row_norms_mean: 5.4083533287 train_h1_row_norms_min: 2.97776651382 train_objective: 0.192548781633 train_y_col_norms_max: 0.849820315838 train_y_col_norms_mean: 0.752397358418 train_y_col_norms_min: 0.648707211018 train_y_max_max_class: 0.999981343746 train_y_mean_max_class: 0.925991177559 train_y_min_max_class: 0.379428476095 train_y_misclass: 0.0572400614619 train_y_nll: 0.192548781633 train_y_row_norms_max: 0.202847748995 train_y_row_norms_mean: 0.0668167173862 train_y_row_norms_min: 0.000276583392406 valid_h0_col_norms_max: 6.2346944809 valid_h0_col_norms_mean: 3.82387781143 valid_h0_col_norms_min: 2.06334352493 valid_h0_row_norms_max: 5.89264249802 valid_h0_row_norms_mean: 2.98581314087 valid_h0_row_norms_min: 0.00337248062715 valid_h1_col_norms_max: 5.99546384811 valid_h1_col_norms_mean: 3.80735421181 valid_h1_col_norms_min: 1.71530222893 valid_h1_row_norms_max: 7.80887699127 valid_h1_row_norms_mean: 5.40835094452 valid_h1_row_norms_min: 2.97777676582 valid_objective: 0.201314240694 valid_y_col_norms_max: 0.849824726582 valid_y_col_norms_mean: 0.752399742603 valid_y_col_norms_min: 0.648707330227 valid_y_max_max_class: 0.999982595444 valid_y_mean_max_class: 0.93180680275 valid_y_min_max_class: 0.40289413929 valid_y_misclass: 0.0579000003636 valid_y_nll: 0.201314240694 valid_y_row_norms_max: 0.202846974134 valid_y_row_norms_mean: 0.0668164640665 valid_y_row_norms_min: 0.000276584294625 Time this epoch: 3.321143 seconds Monitoring step: Epochs seen: 3 Batches seen: 1500 Examples seen: 150000 learning_rate: 0.00999999046326 momentum: 0.608888924122 test_h0_col_norms_max: 6.23464679718 test_h0_col_norms_mean: 3.82416844368 test_h0_col_norms_min: 2.06404829025 test_h0_row_norms_max: 5.89243221283 test_h0_row_norms_mean: 2.98607397079 test_h0_row_norms_min: 0.00511313043535 test_h1_col_norms_max: 5.99604940414 test_h1_col_norms_mean: 3.80747485161 test_h1_col_norms_min: 1.71535277367 test_h1_row_norms_max: 7.80883836746 test_h1_row_norms_mean: 5.40852594376 test_h1_row_norms_min: 2.97782230377 test_objective: 0.18524043262 test_y_col_norms_max: 1.00719892979 test_y_col_norms_mean: 0.879001736641 test_y_col_norms_min: 0.748181402683 test_y_max_max_class: 0.999993741512 test_y_mean_max_class: 0.939781844616 test_y_min_max_class: 0.445061296225 test_y_misclass: 0.0548000186682 test_y_nll: 0.18524043262 test_y_row_norms_max: 0.216917276382 test_y_row_norms_mean: 0.0788432434201 test_y_row_norms_min: 0.000395227049012 train_h0_col_norms_max: 6.23464632034 train_h0_col_norms_mean: 3.82414579391 train_h0_col_norms_min: 2.06404733658 train_h0_row_norms_max: 5.89245033264 train_h0_row_norms_mean: 2.98607373238 train_h0_row_norms_min: 0.00511312671006 train_h1_col_norms_max: 5.99604892731 train_h1_col_norms_mean: 3.80745625496 train_h1_col_norms_min: 1.71535873413 train_h1_row_norms_max: 7.80887460709 train_h1_row_norms_mean: 5.40852594376 train_h1_row_norms_min: 2.9778380394 train_objective: 0.161898091435 train_y_col_norms_max: 1.00719916821 train_y_col_norms_mean: 0.87899774313 train_y_col_norms_min: 0.748184919357 train_y_max_max_class: 0.999991238117 train_y_mean_max_class: 0.93733805418 train_y_min_max_class: 0.405598640442 train_y_misclass: 0.0483000576496 train_y_nll: 0.161898091435 train_y_row_norms_max: 0.216916337609 train_y_row_norms_mean: 0.0788431763649 train_y_row_norms_min: 0.000395228940761 valid_h0_col_norms_max: 6.23464679718 valid_h0_col_norms_mean: 3.82416844368 valid_h0_col_norms_min: 2.06404829025 valid_h0_row_norms_max: 5.89243221283 valid_h0_row_norms_mean: 2.98607397079 valid_h0_row_norms_min: 0.00511313043535 valid_h1_col_norms_max: 5.99604940414 valid_h1_col_norms_mean: 3.80747485161 valid_h1_col_norms_min: 1.71535277367 valid_h1_row_norms_max: 7.80883836746 valid_h1_row_norms_mean: 5.40852594376 valid_h1_row_norms_min: 2.97782230377 valid_objective: 0.174453571439 valid_y_col_norms_max: 1.00719892979 valid_y_col_norms_mean: 0.879001736641 valid_y_col_norms_min: 0.748181402683 valid_y_max_max_class: 0.999995052814 valid_y_mean_max_class: 0.94245827198 valid_y_min_max_class: 0.418575078249 valid_y_misclass: 0.0514000207186 valid_y_nll: 0.174453571439 valid_y_row_norms_max: 0.216917276382 valid_y_row_norms_mean: 0.0788432434201 valid_y_row_norms_min: 0.000395227049012 Time this epoch: 3.407873 seconds Monitoring step: Epochs seen: 4 Batches seen: 2000 Examples seen: 200000 learning_rate: 0.00999999046326 momentum: 0.663333714008 test_h0_col_norms_max: 6.23483276367 test_h0_col_norms_mean: 3.82449483871 test_h0_col_norms_min: 2.06498026848 test_h0_row_norms_max: 5.89247989655 test_h0_row_norms_mean: 2.98636126518 test_h0_row_norms_min: 0.00637936964631 test_h1_col_norms_max: 5.99670314789 test_h1_col_norms_mean: 3.80761146545 test_h1_col_norms_min: 1.71540987492 test_h1_row_norms_max: 7.80886650085 test_h1_row_norms_mean: 5.40871572495 test_h1_row_norms_min: 2.97799134254 test_objective: 0.167924150825 test_y_col_norms_max: 1.14452064037 test_y_col_norms_mean: 0.995063841343 test_y_col_norms_min: 0.840617954731 test_y_max_max_class: 0.99999588728 test_y_mean_max_class: 0.946992635727 test_y_min_max_class: 0.455186247826 test_y_misclass: 0.0552000291646 test_y_nll: 0.167924150825 test_y_row_norms_max: 0.23083357513 test_y_row_norms_mean: 0.08986672014 test_y_row_norms_min: 0.000483248528326 train_h0_col_norms_max: 6.2348651886 train_h0_col_norms_mean: 3.82447862625 train_h0_col_norms_min: 2.06498932838 train_h0_row_norms_max: 5.89249992371 train_h0_row_norms_mean: 2.98634982109 train_h0_row_norms_min: 0.00637934077531 train_h1_col_norms_max: 5.99670362473 train_h1_col_norms_mean: 3.80763316154 train_h1_col_norms_min: 1.71541762352 train_h1_row_norms_max: 7.80887794495 train_h1_row_norms_mean: 5.40874290466 train_h1_row_norms_min: 2.97797679901 train_objective: 0.138446286321 train_y_col_norms_max: 1.1445235014 train_y_col_norms_mean: 0.995067954063 train_y_col_norms_min: 0.840613126755 train_y_max_max_class: 0.999992251396 train_y_mean_max_class: 0.945943057537 train_y_min_max_class: 0.423846125603 train_y_misclass: 0.0430600605905 train_y_nll: 0.138446286321 train_y_row_norms_max: 0.230833858252 train_y_row_norms_mean: 0.0898664072156 train_y_row_norms_min: 0.000483250943944 valid_h0_col_norms_max: 6.23483276367 valid_h0_col_norms_mean: 3.82449483871 valid_h0_col_norms_min: 2.06498026848 valid_h0_row_norms_max: 5.89247989655 valid_h0_row_norms_mean: 2.98636126518 valid_h0_row_norms_min: 0.00637936964631 valid_h1_col_norms_max: 5.99670314789 valid_h1_col_norms_mean: 3.80761146545 valid_h1_col_norms_min: 1.71540987492 valid_h1_row_norms_max: 7.80886650085 valid_h1_row_norms_mean: 5.40871572495 valid_h1_row_norms_min: 2.97799134254 valid_objective: 0.157675400376 valid_y_col_norms_max: 1.14452064037 valid_y_col_norms_mean: 0.995063841343 valid_y_col_norms_min: 0.840617954731 valid_y_max_max_class: 0.999996602535 valid_y_mean_max_class: 0.949966013432 valid_y_min_max_class: 0.442742049694 valid_y_misclass: 0.046300008893 valid_y_nll: 0.157675400376 valid_y_row_norms_max: 0.23083357513 valid_y_row_norms_mean: 0.08986672014 valid_y_row_norms_min: 0.000483248528326 Time this epoch: 3.220654 seconds Monitoring step: Epochs seen: 5 Batches seen: 2500 Examples seen: 250000 learning_rate: 0.00999999046326 momentum: 0.717777192593 test_h0_col_norms_max: 6.23521852493 test_h0_col_norms_mean: 3.82483482361 test_h0_col_norms_min: 2.06603121758 test_h0_row_norms_max: 5.89207363129 test_h0_row_norms_mean: 2.98667144775 test_h0_row_norms_min: 0.00797319039702 test_h1_col_norms_max: 5.99737501144 test_h1_col_norms_mean: 3.80774116516 test_h1_col_norms_min: 1.71550190449 test_h1_row_norms_max: 7.80892467499 test_h1_row_norms_mean: 5.40890693665 test_h1_row_norms_min: 2.97820734978 test_objective: 0.13814201951 test_y_col_norms_max: 1.26785862446 test_y_col_norms_mean: 1.10942089558 test_y_col_norms_min: 0.9239538908 test_y_max_max_class: 0.999995410442 test_y_mean_max_class: 0.953776538372 test_y_min_max_class: 0.461881011724 test_y_misclass: 0.0431000031531 test_y_nll: 0.13814201951 test_y_row_norms_max: 0.258687496185 test_y_row_norms_mean: 0.10072222352 test_y_row_norms_min: 0.000603844528086 train_h0_col_norms_max: 6.23519468307 train_h0_col_norms_mean: 3.82483053207 train_h0_col_norms_min: 2.06602716446 train_h0_row_norms_max: 5.89205408096 train_h0_row_norms_mean: 2.98667001724 train_h0_row_norms_min: 0.0079732267186 train_h1_col_norms_max: 5.99740314484 train_h1_col_norms_mean: 3.80775809288 train_h1_col_norms_min: 1.71549510956 train_h1_row_norms_max: 7.80892419815 train_h1_row_norms_mean: 5.40891933441 train_h1_row_norms_min: 2.97820615768 train_objective: 0.104295127094 train_y_col_norms_max: 1.26785480976 train_y_col_norms_mean: 1.109421134 train_y_col_norms_min: 0.923955321312 train_y_max_max_class: 0.999992787838 train_y_mean_max_class: 0.954641282558 train_y_min_max_class: 0.442351669073 train_y_misclass: 0.0312000326812 train_y_nll: 0.104295127094 train_y_row_norms_max: 0.258685946465 train_y_row_norms_mean: 0.100721813738 train_y_row_norms_min: 0.000603846099693 valid_h0_col_norms_max: 6.23521852493 valid_h0_col_norms_mean: 3.82483482361 valid_h0_col_norms_min: 2.06603121758 valid_h0_row_norms_max: 5.89207363129 valid_h0_row_norms_mean: 2.98667144775 valid_h0_row_norms_min: 0.00797319039702 valid_h1_col_norms_max: 5.99737501144 valid_h1_col_norms_mean: 3.80774116516 valid_h1_col_norms_min: 1.71550190449 valid_h1_row_norms_max: 7.80892467499 valid_h1_row_norms_mean: 5.40890693665 valid_h1_row_norms_min: 2.97820734978 valid_objective: 0.136576414108 valid_y_col_norms_max: 1.26785862446 valid_y_col_norms_mean: 1.10942089558 valid_y_col_norms_min: 0.9239538908 valid_y_max_max_class: 0.999996840954 valid_y_mean_max_class: 0.956140458584 valid_y_min_max_class: 0.448911756277 valid_y_misclass: 0.0386999994516 valid_y_nll: 0.136576414108 valid_y_row_norms_max: 0.258687496185 valid_y_row_norms_mean: 0.10072222352 valid_y_row_norms_min: 0.000603844528086 Time this epoch: 3.204515 seconds Monitoring step: Epochs seen: 6 Batches seen: 3000 Examples seen: 300000 learning_rate: 0.00999999046326 momentum: 0.772221684456 test_h0_col_norms_max: 6.23541164398 test_h0_col_norms_mean: 3.82526040077 test_h0_col_norms_min: 2.0674469471 test_h0_row_norms_max: 5.89197492599 test_h0_row_norms_mean: 2.98706746101 test_h0_row_norms_min: 0.00963484868407 test_h1_col_norms_max: 5.9978518486 test_h1_col_norms_mean: 3.80790233612 test_h1_col_norms_min: 1.71558940411 test_h1_row_norms_max: 7.80901002884 test_h1_row_norms_mean: 5.40913200378 test_h1_row_norms_min: 2.97820520401 test_objective: 0.12612003088 test_y_col_norms_max: 1.39495909214 test_y_col_norms_mean: 1.23315572739 test_y_col_norms_min: 1.02864944935 test_y_max_max_class: 0.999998807907 test_y_mean_max_class: 0.961598396301 test_y_min_max_class: 0.503333091736 test_y_misclass: 0.040100004524 test_y_nll: 0.12612003088 test_y_row_norms_max: 0.288501292467 test_y_row_norms_mean: 0.112407810986 test_y_row_norms_min: 0.000765459961258 train_h0_col_norms_max: 6.23538017273 train_h0_col_norms_mean: 3.82528162003 train_h0_col_norms_min: 2.0674469471 train_h0_row_norms_max: 5.89197683334 train_h0_row_norms_mean: 2.98705887794 train_h0_row_norms_min: 0.00963485334069 train_h1_col_norms_max: 5.99787998199 train_h1_col_norms_mean: 3.80790233612 train_h1_col_norms_min: 1.7155970335 train_h1_row_norms_max: 7.80897331238 train_h1_row_norms_mean: 5.40915393829 train_h1_row_norms_min: 2.97820544243 train_objective: 0.0812869444489 train_y_col_norms_max: 1.39496576786 train_y_col_norms_mean: 1.23315918446 train_y_col_norms_min: 1.02865147591 train_y_max_max_class: 0.99999409914 train_y_mean_max_class: 0.963725090027 train_y_min_max_class: 0.476592302322 train_y_misclass: 0.0230800136924 train_y_nll: 0.0812869444489 train_y_row_norms_max: 0.288501352072 train_y_row_norms_mean: 0.112407691777 train_y_row_norms_min: 0.00076545990305 valid_h0_col_norms_max: 6.23541164398 valid_h0_col_norms_mean: 3.82526040077 valid_h0_col_norms_min: 2.0674469471 valid_h0_row_norms_max: 5.89197492599 valid_h0_row_norms_mean: 2.98706746101 valid_h0_row_norms_min: 0.00963484868407 valid_h1_col_norms_max: 5.9978518486 valid_h1_col_norms_mean: 3.80790233612 valid_h1_col_norms_min: 1.71558940411 valid_h1_row_norms_max: 7.80901002884 valid_h1_row_norms_mean: 5.40913200378 valid_h1_row_norms_min: 2.97820520401 valid_objective: 0.127863824368 valid_y_col_norms_max: 1.39495909214 valid_y_col_norms_mean: 1.23315572739 valid_y_col_norms_min: 1.02864944935 valid_y_max_max_class: 0.999999046326 valid_y_mean_max_class: 0.964188098907 valid_y_min_max_class: 0.480807334185 valid_y_misclass: 0.0376999974251 valid_y_nll: 0.127863824368 valid_y_row_norms_max: 0.288501292467 valid_y_row_norms_mean: 0.112407810986 valid_y_row_norms_min: 0.000765459961258 Time this epoch: 3.235264 seconds Monitoring step: Epochs seen: 7 Batches seen: 3500 Examples seen: 350000 learning_rate: 0.00999999046326 momentum: 0.826667308807 test_h0_col_norms_max: 6.23617553711 test_h0_col_norms_mean: 3.82576131821 test_h0_col_norms_min: 2.06955361366 test_h0_row_norms_max: 5.8926115036 test_h0_row_norms_mean: 2.98752951622 test_h0_row_norms_min: 0.011014319025 test_h1_col_norms_max: 5.99838781357 test_h1_col_norms_mean: 3.8080675602 test_h1_col_norms_min: 1.71574032307 test_h1_row_norms_max: 7.80883789062 test_h1_row_norms_mean: 5.40936756134 test_h1_row_norms_min: 2.97880935669 test_objective: 0.127731248736 test_y_col_norms_max: 1.54538154602 test_y_col_norms_mean: 1.37167823315 test_y_col_norms_min: 1.13854420185 test_y_max_max_class: 0.999999046326 test_y_mean_max_class: 0.9629342556 test_y_min_max_class: 0.519809484482 test_y_misclass: 0.0402000173926 test_y_nll: 0.127731248736 test_y_row_norms_max: 0.32344275713 test_y_row_norms_mean: 0.125407382846 test_y_row_norms_min: 0.000886962865479 train_h0_col_norms_max: 6.23615169525 train_h0_col_norms_mean: 3.82577753067 train_h0_col_norms_min: 2.06954622269 train_h0_row_norms_max: 5.89259195328 train_h0_row_norms_mean: 2.98751401901 train_h0_row_norms_min: 0.0110142948106 train_h1_col_norms_max: 5.99837732315 train_h1_col_norms_mean: 3.80804681778 train_h1_col_norms_min: 1.71573352814 train_h1_row_norms_max: 7.80887413025 train_h1_row_norms_mean: 5.4093914032 train_h1_row_norms_min: 2.97880387306 train_objective: 0.0784979835153 train_y_col_norms_max: 1.54537415504 train_y_col_norms_mean: 1.37168061733 train_y_col_norms_min: 1.13854324818 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.965206980705 train_y_min_max_class: 0.486533343792 train_y_misclass: 0.0245000198483 train_y_nll: 0.0784979835153 train_y_row_norms_max: 0.323444247246 train_y_row_norms_mean: 0.125407934189 train_y_row_norms_min: 0.000886966707185 valid_h0_col_norms_max: 6.23617553711 valid_h0_col_norms_mean: 3.82576131821 valid_h0_col_norms_min: 2.06955361366 valid_h0_row_norms_max: 5.8926115036 valid_h0_row_norms_mean: 2.98752951622 valid_h0_row_norms_min: 0.011014319025 valid_h1_col_norms_max: 5.99838781357 valid_h1_col_norms_mean: 3.8080675602 valid_h1_col_norms_min: 1.71574032307 valid_h1_row_norms_max: 7.80883789062 valid_h1_row_norms_mean: 5.40936756134 valid_h1_row_norms_min: 2.97880935669 valid_objective: 0.126347467303 valid_y_col_norms_max: 1.54538154602 valid_y_col_norms_mean: 1.37167823315 valid_y_col_norms_min: 1.13854420185 valid_y_max_max_class: 0.999999165535 valid_y_mean_max_class: 0.966301620007 valid_y_min_max_class: 0.483229219913 valid_y_misclass: 0.0362999886274 valid_y_nll: 0.126347467303 valid_y_row_norms_max: 0.32344275713 valid_y_row_norms_mean: 0.125407382846 valid_y_row_norms_min: 0.000886962865479 Time this epoch: 3.324166 seconds Monitoring step: Epochs seen: 8 Batches seen: 4000 Examples seen: 400000 learning_rate: 0.00999999046326 momentum: 0.881111502647 test_h0_col_norms_max: 6.23693847656 test_h0_col_norms_mean: 3.8264799118 test_h0_col_norms_min: 2.07238268852 test_h0_row_norms_max: 5.89200305939 test_h0_row_norms_mean: 2.98819732666 test_h0_row_norms_min: 0.0122548062354 test_h1_col_norms_max: 5.99879837036 test_h1_col_norms_mean: 3.80823135376 test_h1_col_norms_min: 1.71583795547 test_h1_row_norms_max: 7.80892133713 test_h1_row_norms_mean: 5.40960502625 test_h1_row_norms_min: 2.97916102409 test_objective: 0.121290750802 test_y_col_norms_max: 1.74212527275 test_y_col_norms_mean: 1.55456089973 test_y_col_norms_min: 1.29530310631 test_y_max_max_class: 0.999999284744 test_y_mean_max_class: 0.970344901085 test_y_min_max_class: 0.541184604168 test_y_misclass: 0.0355000011623 test_y_nll: 0.121290750802 test_y_row_norms_max: 0.393140137196 test_y_row_norms_mean: 0.142595127225 test_y_row_norms_min: 0.00119761796668 train_h0_col_norms_max: 6.23696804047 train_h0_col_norms_mean: 3.82649302483 train_h0_col_norms_min: 2.07238888741 train_h0_row_norms_max: 5.89202260971 train_h0_row_norms_mean: 2.98821163177 train_h0_row_norms_min: 0.0122548071668 train_h1_col_norms_max: 5.99882984161 train_h1_col_norms_mean: 3.80823636055 train_h1_col_norms_min: 1.71583855152 train_h1_row_norms_max: 7.80892324448 train_h1_row_norms_mean: 5.40962982178 train_h1_row_norms_min: 2.979159832 train_objective: 0.0608208738267 train_y_col_norms_max: 1.7421246767 train_y_col_norms_mean: 1.55455350876 train_y_col_norms_min: 1.29530549049 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.97307318449 train_y_min_max_class: 0.52649885416 train_y_misclass: 0.018220026046 train_y_nll: 0.0608208738267 train_y_row_norms_max: 0.393138289452 train_y_row_norms_mean: 0.142595857382 train_y_row_norms_min: 0.00119762122631 valid_h0_col_norms_max: 6.23693847656 valid_h0_col_norms_mean: 3.8264799118 valid_h0_col_norms_min: 2.07238268852 valid_h0_row_norms_max: 5.89200305939 valid_h0_row_norms_mean: 2.98819732666 valid_h0_row_norms_min: 0.0122548062354 valid_h1_col_norms_max: 5.99879837036 valid_h1_col_norms_mean: 3.80823135376 valid_h1_col_norms_min: 1.71583795547 valid_h1_row_norms_max: 7.80892133713 valid_h1_row_norms_mean: 5.40960502625 valid_h1_row_norms_min: 2.97916102409 valid_objective: 0.120653524995 valid_y_col_norms_max: 1.74212527275 valid_y_col_norms_mean: 1.55456089973 valid_y_col_norms_min: 1.29530310631 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.971736133099 valid_y_min_max_class: 0.502751410007 valid_y_misclass: 0.0357999950647 valid_y_nll: 0.120653524995 valid_y_row_norms_max: 0.393140137196 valid_y_row_norms_mean: 0.142595127225 valid_y_row_norms_min: 0.00119761796668 Time this epoch: 3.219467 seconds Monitoring step: Epochs seen: 9 Batches seen: 4500 Examples seen: 450000 learning_rate: 0.00999999046326 momentum: 0.935554862022 test_h0_col_norms_max: 6.23974847794 test_h0_col_norms_mean: 3.82828760147 test_h0_col_norms_min: 2.07858109474 test_h0_row_norms_max: 5.89074993134 test_h0_row_norms_mean: 2.98990464211 test_h0_row_norms_min: 0.0139329638332 test_h1_col_norms_max: 6.00128126144 test_h1_col_norms_mean: 3.80823659897 test_h1_col_norms_min: 1.71664977074 test_h1_row_norms_max: 7.80959177017 test_h1_row_norms_mean: 5.40965270996 test_h1_row_norms_min: 2.98309516907 test_objective: 0.133454963565 test_y_col_norms_max: 2.09113478661 test_y_col_norms_mean: 1.89531803131 test_y_col_norms_min: 1.55502259731 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.972993254662 test_y_min_max_class: 0.555838704109 test_y_misclass: 0.03900000453 test_y_nll: 0.133454963565 test_y_row_norms_max: 0.505987465382 test_y_row_norms_mean: 0.174324646592 test_y_row_norms_min: 0.00215850048698 train_h0_col_norms_max: 6.23972511292 train_h0_col_norms_mean: 3.82828736305 train_h0_col_norms_min: 2.07858753204 train_h0_row_norms_max: 5.89076900482 train_h0_row_norms_mean: 2.98989081383 train_h0_row_norms_min: 0.0139330253005 train_h1_col_norms_max: 6.00125265121 train_h1_col_norms_mean: 3.80825352669 train_h1_col_norms_min: 1.7166570425 train_h1_row_norms_max: 7.80962467194 train_h1_row_norms_mean: 5.40965032578 train_h1_row_norms_min: 2.98309373856 train_objective: 0.0678227543831 train_y_col_norms_max: 2.09112644196 train_y_col_norms_mean: 1.8953114748 train_y_col_norms_min: 1.55502521992 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.976901352406 train_y_min_max_class: 0.541133284569 train_y_misclass: 0.0215600207448 train_y_nll: 0.0678227543831 train_y_row_norms_max: 0.505986630917 train_y_row_norms_mean: 0.174323886633 train_y_row_norms_min: 0.00215849909 valid_h0_col_norms_max: 6.23974847794 valid_h0_col_norms_mean: 3.82828760147 valid_h0_col_norms_min: 2.07858109474 valid_h0_row_norms_max: 5.89074993134 valid_h0_row_norms_mean: 2.98990464211 valid_h0_row_norms_min: 0.0139329638332 valid_h1_col_norms_max: 6.00128126144 valid_h1_col_norms_mean: 3.80823659897 valid_h1_col_norms_min: 1.71664977074 valid_h1_row_norms_max: 7.80959177017 valid_h1_row_norms_mean: 5.40965270996 valid_h1_row_norms_min: 2.98309516907 valid_objective: 0.14155356586 valid_y_col_norms_max: 2.09113478661 valid_y_col_norms_mean: 1.89531803131 valid_y_col_norms_min: 1.55502259731 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.975651443005 valid_y_min_max_class: 0.524011075497 valid_y_misclass: 0.0348999835551 valid_y_nll: 0.14155356586 valid_y_row_norms_max: 0.505987465382 valid_y_row_norms_mean: 0.174324646592 valid_y_row_norms_min: 0.00215850048698 Time this epoch: 3.242812 seconds Monitoring step: Epochs seen: 10 Batches seen: 5000 Examples seen: 500000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.33813095093 test_h0_col_norms_mean: 4.00221395493 test_h0_col_norms_min: 2.23122644424 test_h0_row_norms_max: 6.13888168335 test_h0_row_norms_mean: 3.13162064552 test_h0_row_norms_min: 0.0540144480765 test_h1_col_norms_max: 5.99460268021 test_h1_col_norms_mean: 3.81764769554 test_h1_col_norms_min: 1.72675585747 test_h1_row_norms_max: 7.80806827545 test_h1_row_norms_mean: 5.42556667328 test_h1_row_norms_min: 3.22008705139 test_objective: 0.242982923985 test_y_col_norms_max: 4.86701011658 test_y_col_norms_mean: 4.50406503677 test_y_col_norms_min: 3.79116678238 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.970284223557 test_y_min_max_class: 0.494895517826 test_y_misclass: 0.0614000074565 test_y_nll: 0.242982923985 test_y_row_norms_max: 1.25091540813 test_y_row_norms_mean: 0.422105878592 test_y_row_norms_min: 0.00902531389147 train_h0_col_norms_max: 6.33812093735 train_h0_col_norms_mean: 4.00221395493 train_h0_col_norms_min: 2.23123693466 train_h0_row_norms_max: 6.13886117935 train_h0_row_norms_mean: 3.13162612915 train_h0_row_norms_min: 0.0540147125721 train_h1_col_norms_max: 5.99457454681 train_h1_col_norms_mean: 3.81765389442 train_h1_col_norms_min: 1.726749897 train_h1_row_norms_max: 7.80803012848 train_h1_row_norms_mean: 5.42554092407 train_h1_row_norms_min: 3.2200820446 train_objective: 0.216101527214 train_y_col_norms_max: 4.86700248718 train_y_col_norms_mean: 4.50406646729 train_y_col_norms_min: 3.7911875248 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.971834897995 train_y_min_max_class: 0.494699120522 train_y_misclass: 0.0546000786126 train_y_nll: 0.216101527214 train_y_row_norms_max: 1.25092113018 train_y_row_norms_mean: 0.422105282545 train_y_row_norms_min: 0.00902529340237 valid_h0_col_norms_max: 6.33813095093 valid_h0_col_norms_mean: 4.00221395493 valid_h0_col_norms_min: 2.23122644424 valid_h0_row_norms_max: 6.13888168335 valid_h0_row_norms_mean: 3.13162064552 valid_h0_row_norms_min: 0.0540144480765 valid_h1_col_norms_max: 5.99460268021 valid_h1_col_norms_mean: 3.81764769554 valid_h1_col_norms_min: 1.72675585747 valid_h1_row_norms_max: 7.80806827545 valid_h1_row_norms_mean: 5.42556667328 valid_h1_row_norms_min: 3.22008705139 valid_objective: 0.262977838516 valid_y_col_norms_max: 4.86701011658 valid_y_col_norms_mean: 4.50406503677 valid_y_col_norms_min: 3.79116678238 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.972873926163 valid_y_min_max_class: 0.484322339296 valid_y_misclass: 0.0602999925613 valid_y_nll: 0.262977838516 valid_y_row_norms_max: 1.25091540813 valid_y_row_norms_mean: 0.422105878592 valid_y_row_norms_min: 0.00902531389147 Time this epoch: 3.246498 seconds Monitoring step: Epochs seen: 11 Batches seen: 5500 Examples seen: 550000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34423732758 test_h0_col_norms_mean: 4.09757995605 test_h0_col_norms_min: 2.23610663414 test_h0_row_norms_max: 6.29168701172 test_h0_row_norms_mean: 3.20701622963 test_h0_row_norms_min: 0.0794842615724 test_h1_col_norms_max: 5.99344968796 test_h1_col_norms_mean: 3.83266830444 test_h1_col_norms_min: 1.72617077827 test_h1_row_norms_max: 7.81531667709 test_h1_row_norms_mean: 5.44732666016 test_h1_row_norms_min: 3.22785973549 test_objective: 0.149660229683 test_y_col_norms_max: 5.28322935104 test_y_col_norms_mean: 4.86907577515 test_y_col_norms_min: 4.24763870239 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.971002280712 test_y_min_max_class: 0.500847101212 test_y_misclass: 0.0448000095785 test_y_nll: 0.149660229683 test_y_row_norms_max: 1.53015840054 test_y_row_norms_mean: 0.458264380693 test_y_row_norms_min: 0.0079955086112 train_h0_col_norms_max: 6.34425830841 train_h0_col_norms_mean: 4.09757804871 train_h0_col_norms_min: 2.23611760139 train_h0_row_norms_max: 6.29168462753 train_h0_row_norms_mean: 3.20700359344 train_h0_row_norms_min: 0.0794841647148 train_h1_col_norms_max: 5.99343013763 train_h1_col_norms_mean: 3.83267450333 train_h1_col_norms_min: 1.72616374493 train_h1_row_norms_max: 7.81534910202 train_h1_row_norms_mean: 5.44732189178 train_h1_row_norms_min: 3.22785782814 train_objective: 0.115495532751 train_y_col_norms_max: 5.2832069397 train_y_col_norms_mean: 4.86906385422 train_y_col_norms_min: 4.24765825272 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.974365890026 train_y_min_max_class: 0.503006339073 train_y_misclass: 0.0362600125372 train_y_nll: 0.115495532751 train_y_row_norms_max: 1.53016579151 train_y_row_norms_mean: 0.458266496658 train_y_row_norms_min: 0.00799546111375 valid_h0_col_norms_max: 6.34423732758 valid_h0_col_norms_mean: 4.09757995605 valid_h0_col_norms_min: 2.23610663414 valid_h0_row_norms_max: 6.29168701172 valid_h0_row_norms_mean: 3.20701622963 valid_h0_row_norms_min: 0.0794842615724 valid_h1_col_norms_max: 5.99344968796 valid_h1_col_norms_mean: 3.83266830444 valid_h1_col_norms_min: 1.72617077827 valid_h1_row_norms_max: 7.81531667709 valid_h1_row_norms_mean: 5.44732666016 valid_h1_row_norms_min: 3.22785973549 valid_objective: 0.1691185534 valid_y_col_norms_max: 5.28322935104 valid_y_col_norms_mean: 4.86907577515 valid_y_col_norms_min: 4.24763870239 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.974966526031 valid_y_min_max_class: 0.529185950756 valid_y_misclass: 0.0438999943435 valid_y_nll: 0.1691185534 valid_y_row_norms_max: 1.53015840054 valid_y_row_norms_mean: 0.458264380693 valid_y_row_norms_min: 0.0079955086112 Time this epoch: 3.224365 seconds Monitoring step: Epochs seen: 12 Batches seen: 6000 Examples seen: 600000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34843397141 test_h0_col_norms_mean: 4.13394451141 test_h0_col_norms_min: 2.23612523079 test_h0_row_norms_max: 6.36067008972 test_h0_row_norms_mean: 3.23545217514 test_h0_row_norms_min: 0.111102260649 test_h1_col_norms_max: 5.99360513687 test_h1_col_norms_mean: 3.8399875164 test_h1_col_norms_min: 1.72649633884 test_h1_row_norms_max: 7.9447259903 test_h1_row_norms_mean: 5.45721006393 test_h1_row_norms_min: 3.23267006874 test_objective: 0.138930052519 test_y_col_norms_max: 5.38853263855 test_y_col_norms_mean: 4.97749423981 test_y_col_norms_min: 4.37515115738 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.976369380951 test_y_min_max_class: 0.539442539215 test_y_misclass: 0.0395999997854 test_y_nll: 0.138930052519 test_y_row_norms_max: 1.5151270628 test_y_row_norms_mean: 0.468785196543 test_y_row_norms_min: 0.00989222805947 train_h0_col_norms_max: 6.34842920303 train_h0_col_norms_mean: 4.13394021988 train_h0_col_norms_min: 2.23612689972 train_h0_row_norms_max: 6.36069536209 train_h0_row_norms_mean: 3.23545718193 train_h0_row_norms_min: 0.111102797091 train_h1_col_norms_max: 5.99360513687 train_h1_col_norms_mean: 3.83997154236 train_h1_col_norms_min: 1.72650408745 train_h1_row_norms_max: 7.94476556778 train_h1_row_norms_mean: 5.45718336105 train_h1_row_norms_min: 3.23268294334 train_objective: 0.0762413665652 train_y_col_norms_max: 5.38851499557 train_y_col_norms_mean: 4.97747087479 train_y_col_norms_min: 4.37513685226 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.980703771114 train_y_min_max_class: 0.538486421108 train_y_misclass: 0.0236600115895 train_y_nll: 0.0762413665652 train_y_row_norms_max: 1.51513409615 train_y_row_norms_mean: 0.46878734231 train_y_row_norms_min: 0.00989221502095 valid_h0_col_norms_max: 6.34843397141 valid_h0_col_norms_mean: 4.13394451141 valid_h0_col_norms_min: 2.23612523079 valid_h0_row_norms_max: 6.36067008972 valid_h0_row_norms_mean: 3.23545217514 valid_h0_row_norms_min: 0.111102260649 valid_h1_col_norms_max: 5.99360513687 valid_h1_col_norms_mean: 3.8399875164 valid_h1_col_norms_min: 1.72649633884 valid_h1_row_norms_max: 7.9447259903 valid_h1_row_norms_mean: 5.45721006393 valid_h1_row_norms_min: 3.23267006874 valid_objective: 0.158047273755 valid_y_col_norms_max: 5.38853263855 valid_y_col_norms_mean: 4.97749423981 valid_y_col_norms_min: 4.37515115738 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.978621006012 valid_y_min_max_class: 0.533575236797 valid_y_misclass: 0.0357999950647 valid_y_nll: 0.158047273755 valid_y_row_norms_max: 1.5151270628 valid_y_row_norms_mean: 0.468785196543 valid_y_row_norms_min: 0.00989222805947 Time this epoch: 3.233253 seconds Monitoring step: Epochs seen: 13 Batches seen: 6500 Examples seen: 650000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34697389603 test_h0_col_norms_mean: 4.15769052505 test_h0_col_norms_min: 2.23618888855 test_h0_row_norms_max: 6.40273475647 test_h0_row_norms_mean: 3.25405025482 test_h0_row_norms_min: 0.113349400461 test_h1_col_norms_max: 5.99226903915 test_h1_col_norms_mean: 3.84424233437 test_h1_col_norms_min: 1.7265651226 test_h1_row_norms_max: 8.25644397736 test_h1_row_norms_mean: 5.46302652359 test_h1_row_norms_min: 3.24811220169 test_objective: 0.126156955957 test_y_col_norms_max: 5.49813652039 test_y_col_norms_mean: 5.06592178345 test_y_col_norms_min: 4.50360441208 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.983133792877 test_y_min_max_class: 0.586351394653 test_y_misclass: 0.0298999845982 test_y_nll: 0.126156955957 test_y_row_norms_max: 1.57926058769 test_y_row_norms_mean: 0.477343022823 test_y_row_norms_min: 0.0155787682161 train_h0_col_norms_max: 6.34694576263 train_h0_col_norms_mean: 4.15768814087 train_h0_col_norms_min: 2.23619389534 train_h0_row_norms_max: 6.40273189545 train_h0_row_norms_mean: 3.25405526161 train_h0_row_norms_min: 0.113349400461 train_h1_col_norms_max: 5.99224901199 train_h1_col_norms_mean: 3.84425520897 train_h1_col_norms_min: 1.72656738758 train_h1_row_norms_max: 8.25643634796 train_h1_row_norms_mean: 5.46304035187 train_h1_row_norms_min: 3.24809789658 train_objective: 0.0474301576614 train_y_col_norms_max: 5.49815416336 train_y_col_norms_mean: 5.06591844559 train_y_col_norms_min: 4.5035943985 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.986293017864 train_y_min_max_class: 0.579336941242 train_y_misclass: 0.0156600344926 train_y_nll: 0.0474301576614 train_y_row_norms_max: 1.57926678658 train_y_row_norms_mean: 0.477343559265 train_y_row_norms_min: 0.0155787058175 valid_h0_col_norms_max: 6.34697389603 valid_h0_col_norms_mean: 4.15769052505 valid_h0_col_norms_min: 2.23618888855 valid_h0_row_norms_max: 6.40273475647 valid_h0_row_norms_mean: 3.25405025482 valid_h0_row_norms_min: 0.113349400461 valid_h1_col_norms_max: 5.99226903915 valid_h1_col_norms_mean: 3.84424233437 valid_h1_col_norms_min: 1.7265651226 valid_h1_row_norms_max: 8.25644397736 valid_h1_row_norms_mean: 5.46302652359 valid_h1_row_norms_min: 3.24811220169 valid_objective: 0.136303275824 valid_y_col_norms_max: 5.49813652039 valid_y_col_norms_mean: 5.06592178345 valid_y_col_norms_min: 4.50360441208 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.983997404575 valid_y_min_max_class: 0.568609714508 valid_y_misclass: 0.0302999857813 valid_y_nll: 0.136303275824 valid_y_row_norms_max: 1.57926058769 valid_y_row_norms_mean: 0.477343022823 valid_y_row_norms_min: 0.0155787682161 Time this epoch: 3.243910 seconds Monitoring step: Epochs seen: 14 Batches seen: 7000 Examples seen: 700000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.3465590477 test_h0_col_norms_mean: 4.17470979691 test_h0_col_norms_min: 2.23621320724 test_h0_row_norms_max: 6.44742536545 test_h0_row_norms_mean: 3.26748609543 test_h0_row_norms_min: 0.117137983441 test_h1_col_norms_max: 5.99374818802 test_h1_col_norms_mean: 3.84760499001 test_h1_col_norms_min: 1.7263559103 test_h1_row_norms_max: 8.39470767975 test_h1_row_norms_mean: 5.46778011322 test_h1_row_norms_min: 3.26342630386 test_objective: 0.107709117234 test_y_col_norms_max: 5.54377269745 test_y_col_norms_mean: 5.13376808167 test_y_col_norms_min: 4.58503246307 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.985660552979 test_y_min_max_class: 0.598270595074 test_y_misclass: 0.0245999917388 test_y_nll: 0.107709117234 test_y_row_norms_max: 1.54413878918 test_y_row_norms_mean: 0.484508126974 test_y_row_norms_min: 0.0144754517823 train_h0_col_norms_max: 6.346534729 train_h0_col_norms_mean: 4.17470979691 train_h0_col_norms_min: 2.23620676994 train_h0_row_norms_max: 6.44738912582 train_h0_row_norms_mean: 3.26749873161 train_h0_row_norms_min: 0.117138013244 train_h1_col_norms_max: 5.99376821518 train_h1_col_norms_mean: 3.84760093689 train_h1_col_norms_min: 1.72634637356 train_h1_row_norms_max: 8.39471530914 train_h1_row_norms_mean: 5.46780490875 train_h1_row_norms_min: 3.26344394684 train_objective: 0.0289139077067 train_y_col_norms_max: 5.54377365112 train_y_col_norms_mean: 5.13377904892 train_y_col_norms_min: 4.58502912521 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.990230798721 train_y_min_max_class: 0.636234402657 train_y_misclass: 0.0095800133422 train_y_nll: 0.0289139077067 train_y_row_norms_max: 1.54413354397 train_y_row_norms_mean: 0.484510302544 train_y_row_norms_min: 0.0144755160436 valid_h0_col_norms_max: 6.3465590477 valid_h0_col_norms_mean: 4.17470979691 valid_h0_col_norms_min: 2.23621320724 valid_h0_row_norms_max: 6.44742536545 valid_h0_row_norms_mean: 3.26748609543 valid_h0_row_norms_min: 0.117137983441 valid_h1_col_norms_max: 5.99374818802 valid_h1_col_norms_mean: 3.84760499001 valid_h1_col_norms_min: 1.7263559103 valid_h1_row_norms_max: 8.39470767975 valid_h1_row_norms_mean: 5.46778011322 valid_h1_row_norms_min: 3.26342630386 valid_objective: 0.118425898254 valid_y_col_norms_max: 5.54377269745 valid_y_col_norms_mean: 5.13376808167 valid_y_col_norms_min: 4.58503246307 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.987525939941 valid_y_min_max_class: 0.608628451824 valid_y_misclass: 0.0258999839425 valid_y_nll: 0.118425898254 valid_y_row_norms_max: 1.54413878918 valid_y_row_norms_mean: 0.484508126974 valid_y_row_norms_min: 0.0144754517823 Time this epoch: 3.231089 seconds Monitoring step: Epochs seen: 15 Batches seen: 7500 Examples seen: 750000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34646034241 test_h0_col_norms_mean: 4.18901968002 test_h0_col_norms_min: 2.23616552353 test_h0_row_norms_max: 6.49371194839 test_h0_row_norms_mean: 3.27877855301 test_h0_row_norms_min: 0.122728899121 test_h1_col_norms_max: 5.9948592186 test_h1_col_norms_mean: 3.85039448738 test_h1_col_norms_min: 1.72630560398 test_h1_row_norms_max: 8.49246692657 test_h1_row_norms_mean: 5.47177028656 test_h1_row_norms_min: 3.27335119247 test_objective: 0.120883144438 test_y_col_norms_max: 5.61785268784 test_y_col_norms_mean: 5.21456623077 test_y_col_norms_min: 4.61228704453 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.985354423523 test_y_min_max_class: 0.593527436256 test_y_misclass: 0.0276999864727 test_y_nll: 0.120883144438 test_y_row_norms_max: 1.59560739994 test_y_row_norms_mean: 0.492057174444 test_y_row_norms_min: 0.0153611358255 train_h0_col_norms_max: 6.34646320343 train_h0_col_norms_mean: 4.18901586533 train_h0_col_norms_min: 2.23617053032 train_h0_row_norms_max: 6.49373817444 train_h0_row_norms_mean: 3.27876186371 train_h0_row_norms_min: 0.122729450464 train_h1_col_norms_max: 5.99485731125 train_h1_col_norms_mean: 3.8503715992 train_h1_col_norms_min: 1.72631311417 train_h1_row_norms_max: 8.49246883392 train_h1_row_norms_mean: 5.47177219391 train_h1_row_norms_min: 3.27336573601 train_objective: 0.0283282585442 train_y_col_norms_max: 5.61785554886 train_y_col_norms_mean: 5.21454381943 train_y_col_norms_min: 4.61229038239 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.990546524525 train_y_min_max_class: 0.649133205414 train_y_misclass: 0.00910001061857 train_y_nll: 0.0283282585442 train_y_row_norms_max: 1.59561276436 train_y_row_norms_mean: 0.492054820061 train_y_row_norms_min: 0.0153612047434 valid_h0_col_norms_max: 6.34646034241 valid_h0_col_norms_mean: 4.18901968002 valid_h0_col_norms_min: 2.23616552353 valid_h0_row_norms_max: 6.49371194839 valid_h0_row_norms_mean: 3.27877855301 valid_h0_row_norms_min: 0.122728899121 valid_h1_col_norms_max: 5.9948592186 valid_h1_col_norms_mean: 3.85039448738 valid_h1_col_norms_min: 1.72630560398 valid_h1_row_norms_max: 8.49246692657 valid_h1_row_norms_mean: 5.47177028656 valid_h1_row_norms_min: 3.27335119247 valid_objective: 0.126225486398 valid_y_col_norms_max: 5.61785268784 valid_y_col_norms_mean: 5.21456623077 valid_y_col_norms_min: 4.61228704453 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.987080872059 valid_y_min_max_class: 0.583371043205 valid_y_misclass: 0.0265999827534 valid_y_nll: 0.126225486398 valid_y_row_norms_max: 1.59560739994 valid_y_row_norms_mean: 0.492057174444 valid_y_row_norms_min: 0.0153611358255 Time this epoch: 3.249726 seconds Monitoring step: Epochs seen: 16 Batches seen: 8000 Examples seen: 800000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34652328491 test_h0_col_norms_mean: 4.20311164856 test_h0_col_norms_min: 2.23617291451 test_h0_row_norms_max: 6.52294111252 test_h0_row_norms_mean: 3.28994369507 test_h0_row_norms_min: 0.123597666621 test_h1_col_norms_max: 5.99608755112 test_h1_col_norms_mean: 3.85341596603 test_h1_col_norms_min: 1.72634136677 test_h1_row_norms_max: 8.52042388916 test_h1_row_norms_mean: 5.47620201111 test_h1_row_norms_min: 3.27072739601 test_objective: 0.140668272972 test_y_col_norms_max: 5.69501256943 test_y_col_norms_mean: 5.31268548965 test_y_col_norms_min: 4.74868249893 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989543557167 test_y_min_max_class: 0.626976370811 test_y_misclass: 0.0258999932557 test_y_nll: 0.140668272972 test_y_row_norms_max: 1.60322284698 test_y_row_norms_mean: 0.500980615616 test_y_row_norms_min: 0.0168750006706 train_h0_col_norms_max: 6.34652090073 train_h0_col_norms_mean: 4.20310306549 train_h0_col_norms_min: 2.23617100716 train_h0_row_norms_max: 6.52291107178 train_h0_row_norms_mean: 3.28993988037 train_h0_row_norms_min: 0.123597674072 train_h1_col_norms_max: 5.99606466293 train_h1_col_norms_mean: 3.85343289375 train_h1_col_norms_min: 1.72633349895 train_h1_row_norms_max: 8.52042198181 train_h1_row_norms_mean: 5.47621965408 train_h1_row_norms_min: 3.27073836327 train_objective: 0.0259083565325 train_y_col_norms_max: 5.69503641129 train_y_col_norms_mean: 5.31268262863 train_y_col_norms_min: 4.74867868423 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.993756473064 train_y_min_max_class: 0.699871182442 train_y_misclass: 0.0079400036484 train_y_nll: 0.0259083565325 train_y_row_norms_max: 1.6032307148 train_y_row_norms_mean: 0.500979840755 train_y_row_norms_min: 0.0168750379235 valid_h0_col_norms_max: 6.34652328491 valid_h0_col_norms_mean: 4.20311164856 valid_h0_col_norms_min: 2.23617291451 valid_h0_row_norms_max: 6.52294111252 valid_h0_row_norms_mean: 3.28994369507 valid_h0_row_norms_min: 0.123597666621 valid_h1_col_norms_max: 5.99608755112 valid_h1_col_norms_mean: 3.85341596603 valid_h1_col_norms_min: 1.72634136677 valid_h1_row_norms_max: 8.52042388916 valid_h1_row_norms_mean: 5.47620201111 valid_h1_row_norms_min: 3.27072739601 valid_objective: 0.140435069799 valid_y_col_norms_max: 5.69501256943 valid_y_col_norms_mean: 5.31268548965 valid_y_col_norms_min: 4.74868249893 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.990495383739 valid_y_min_max_class: 0.633842229843 valid_y_misclass: 0.0265999827534 valid_y_nll: 0.140435069799 valid_y_row_norms_max: 1.60322284698 valid_y_row_norms_mean: 0.500980615616 valid_y_row_norms_min: 0.0168750006706 Time this epoch: 3.211907 seconds Monitoring step: Epochs seen: 17 Batches seen: 8500 Examples seen: 850000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34764194489 test_h0_col_norms_mean: 4.21877479553 test_h0_col_norms_min: 2.23619961739 test_h0_row_norms_max: 6.5714468956 test_h0_row_norms_mean: 3.30228757858 test_h0_row_norms_min: 0.13643656671 test_h1_col_norms_max: 5.99594020844 test_h1_col_norms_mean: 3.85699319839 test_h1_col_norms_min: 1.72638630867 test_h1_row_norms_max: 8.61135101318 test_h1_row_norms_mean: 5.48117828369 test_h1_row_norms_min: 3.27077460289 test_objective: 0.152983635664 test_y_col_norms_max: 5.81860494614 test_y_col_norms_mean: 5.40938711166 test_y_col_norms_min: 4.81085681915 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.990412473679 test_y_min_max_class: 0.641472399235 test_y_misclass: 0.0277999881655 test_y_nll: 0.152983635664 test_y_row_norms_max: 1.66027259827 test_y_row_norms_mean: 0.509944438934 test_y_row_norms_min: 0.0174780637026 train_h0_col_norms_max: 6.3476524353 train_h0_col_norms_mean: 4.21879482269 train_h0_col_norms_min: 2.23619699478 train_h0_row_norms_max: 6.57147264481 train_h0_row_norms_mean: 3.30230164528 train_h0_row_norms_min: 0.136435881257 train_h1_col_norms_max: 5.9959692955 train_h1_col_norms_mean: 3.85701036453 train_h1_col_norms_min: 1.72638809681 train_h1_row_norms_max: 8.61137866974 train_h1_row_norms_mean: 5.48117685318 train_h1_row_norms_min: 3.27077269554 train_objective: 0.0280419886112 train_y_col_norms_max: 5.81860494614 train_y_col_norms_mean: 5.40940761566 train_y_col_norms_min: 4.81083345413 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.993593096733 train_y_min_max_class: 0.69740664959 train_y_misclass: 0.00842001195997 train_y_nll: 0.0280419886112 train_y_row_norms_max: 1.66027379036 train_y_row_norms_mean: 0.509946644306 train_y_row_norms_min: 0.0174781102687 valid_h0_col_norms_max: 6.34764194489 valid_h0_col_norms_mean: 4.21877479553 valid_h0_col_norms_min: 2.23619961739 valid_h0_row_norms_max: 6.5714468956 valid_h0_row_norms_mean: 3.30228757858 valid_h0_row_norms_min: 0.13643656671 valid_h1_col_norms_max: 5.99594020844 valid_h1_col_norms_mean: 3.85699319839 valid_h1_col_norms_min: 1.72638630867 valid_h1_row_norms_max: 8.61135101318 valid_h1_row_norms_mean: 5.48117828369 valid_h1_row_norms_min: 3.27077460289 valid_objective: 0.156515717506 valid_y_col_norms_max: 5.81860494614 valid_y_col_norms_mean: 5.40938711166 valid_y_col_norms_min: 4.81085681915 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.991046726704 valid_y_min_max_class: 0.649928092957 valid_y_misclass: 0.0286999810487 valid_y_nll: 0.156515717506 valid_y_row_norms_max: 1.66027259827 valid_y_row_norms_mean: 0.509944438934 valid_y_row_norms_min: 0.0174780637026 Time this epoch: 3.213883 seconds Monitoring step: Epochs seen: 18 Batches seen: 9000 Examples seen: 900000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.34813308716 test_h0_col_norms_mean: 4.2329621315 test_h0_col_norms_min: 2.23619866371 test_h0_row_norms_max: 6.60563611984 test_h0_row_norms_mean: 3.31354284286 test_h0_row_norms_min: 0.142215177417 test_h1_col_norms_max: 5.99625921249 test_h1_col_norms_mean: 3.86032938957 test_h1_col_norms_min: 1.72629284859 test_h1_row_norms_max: 8.70863246918 test_h1_row_norms_mean: 5.48595952988 test_h1_row_norms_min: 3.27107739449 test_objective: 0.1266990453 test_y_col_norms_max: 5.94182395935 test_y_col_norms_mean: 5.48706197739 test_y_col_norms_min: 4.85955810547 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.991642594337 test_y_min_max_class: 0.680797755718 test_y_misclass: 0.0225999932736 test_y_nll: 0.1266990453 test_y_row_norms_max: 1.65575671196 test_y_row_norms_mean: 0.517508506775 test_y_row_norms_min: 0.0219007991254 train_h0_col_norms_max: 6.34813261032 train_h0_col_norms_mean: 4.23297452927 train_h0_col_norms_min: 2.23619627953 train_h0_row_norms_max: 6.6056265831 train_h0_row_norms_mean: 3.31354165077 train_h0_row_norms_min: 0.142215907574 train_h1_col_norms_max: 5.99623250961 train_h1_col_norms_mean: 3.86034679413 train_h1_col_norms_min: 1.72628378868 train_h1_row_norms_max: 8.70865249634 train_h1_row_norms_mean: 5.48598957062 train_h1_row_norms_min: 3.27109384537 train_objective: 0.0143134472892 train_y_col_norms_max: 5.94185161591 train_y_col_norms_mean: 5.48704767227 train_y_col_norms_min: 4.85954427719 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996097743511 train_y_min_max_class: 0.768372476101 train_y_misclass: 0.00466000149027 train_y_nll: 0.0143134472892 train_y_row_norms_max: 1.6557571888 train_y_row_norms_mean: 0.517506301403 train_y_row_norms_min: 0.0219009146094 valid_h0_col_norms_max: 6.34813308716 valid_h0_col_norms_mean: 4.2329621315 valid_h0_col_norms_min: 2.23619866371 valid_h0_row_norms_max: 6.60563611984 valid_h0_row_norms_mean: 3.31354284286 valid_h0_row_norms_min: 0.142215177417 valid_h1_col_norms_max: 5.99625921249 valid_h1_col_norms_mean: 3.86032938957 valid_h1_col_norms_min: 1.72629284859 valid_h1_row_norms_max: 8.70863246918 valid_h1_row_norms_mean: 5.48595952988 valid_h1_row_norms_min: 3.27107739449 valid_objective: 0.158007115126 valid_y_col_norms_max: 5.94182395935 valid_y_col_norms_mean: 5.48706197739 valid_y_col_norms_min: 4.85955810547 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.99270170927 valid_y_min_max_class: 0.685421526432 valid_y_misclass: 0.0256999861449 valid_y_nll: 0.158007115126 valid_y_row_norms_max: 1.65575671196 valid_y_row_norms_mean: 0.517508506775 valid_y_row_norms_min: 0.0219007991254 Time this epoch: 3.216884 seconds Monitoring step: Epochs seen: 19 Batches seen: 9500 Examples seen: 950000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.35075473785 test_h0_col_norms_mean: 4.24279737473 test_h0_col_norms_min: 2.23619127274 test_h0_row_norms_max: 6.61569023132 test_h0_row_norms_mean: 3.32117795944 test_h0_row_norms_min: 0.160097524524 test_h1_col_norms_max: 5.99536848068 test_h1_col_norms_mean: 3.86252450943 test_h1_col_norms_min: 1.72685301304 test_h1_row_norms_max: 8.74706554413 test_h1_row_norms_mean: 5.48911523819 test_h1_row_norms_min: 3.27158546448 test_objective: 0.128275766969 test_y_col_norms_max: 6.00630426407 test_y_col_norms_mean: 5.54901790619 test_y_col_norms_min: 4.95159053802 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.992789447308 test_y_min_max_class: 0.689596951008 test_y_misclass: 0.0208999924362 test_y_nll: 0.128275766969 test_y_row_norms_max: 1.56810212135 test_y_row_norms_mean: 0.523332059383 test_y_row_norms_min: 0.0221748072654 train_h0_col_norms_max: 6.35075521469 train_h0_col_norms_mean: 4.24279022217 train_h0_col_norms_min: 2.23619437218 train_h0_row_norms_max: 6.61565685272 train_h0_row_norms_mean: 3.32117271423 train_h0_row_norms_min: 0.160097926855 train_h1_col_norms_max: 5.99534845352 train_h1_col_norms_mean: 3.86250782013 train_h1_col_norms_min: 1.7268614769 train_h1_row_norms_max: 8.74709033966 train_h1_row_norms_mean: 5.48911237717 train_h1_row_norms_min: 3.27158045769 train_objective: 0.0107667120174 train_y_col_norms_max: 6.00630140305 train_y_col_norms_mean: 5.54901885986 train_y_col_norms_min: 4.95157289505 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996949017048 train_y_min_max_class: 0.813183248043 train_y_misclass: 0.00347999692895 train_y_nll: 0.0107667120174 train_y_row_norms_max: 1.56809437275 train_y_row_norms_mean: 0.523329675198 train_y_row_norms_min: 0.0221747960895 valid_h0_col_norms_max: 6.35075473785 valid_h0_col_norms_mean: 4.24279737473 valid_h0_col_norms_min: 2.23619127274 valid_h0_row_norms_max: 6.61569023132 valid_h0_row_norms_mean: 3.32117795944 valid_h0_row_norms_min: 0.160097524524 valid_h1_col_norms_max: 5.99536848068 valid_h1_col_norms_mean: 3.86252450943 valid_h1_col_norms_min: 1.72685301304 valid_h1_row_norms_max: 8.74706554413 valid_h1_row_norms_mean: 5.48911523819 valid_h1_row_norms_min: 3.27158546448 valid_objective: 0.152880609035 valid_y_col_norms_max: 6.00630426407 valid_y_col_norms_mean: 5.54901790619 valid_y_col_norms_min: 4.95159053802 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.992956161499 valid_y_min_max_class: 0.687586247921 valid_y_misclass: 0.0238999892026 valid_y_nll: 0.152880609035 valid_y_row_norms_max: 1.56810212135 valid_y_row_norms_mean: 0.523332059383 valid_y_row_norms_min: 0.0221748072654 Time this epoch: 3.381361 seconds Monitoring step: Epochs seen: 20 Batches seen: 10000 Examples seen: 1000000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.36764955521 test_h0_col_norms_mean: 4.25145339966 test_h0_col_norms_min: 2.23608016968 test_h0_row_norms_max: 6.65068340302 test_h0_row_norms_mean: 3.32794356346 test_h0_row_norms_min: 0.160930916667 test_h1_col_norms_max: 5.99686193466 test_h1_col_norms_mean: 3.86456871033 test_h1_col_norms_min: 1.72680532932 test_h1_row_norms_max: 8.77167224884 test_h1_row_norms_mean: 5.49206733704 test_h1_row_norms_min: 3.27174091339 test_objective: 0.135456323624 test_y_col_norms_max: 6.06686162949 test_y_col_norms_mean: 5.60846662521 test_y_col_norms_min: 5.02197170258 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.992590069771 test_y_min_max_class: 0.687869131565 test_y_misclass: 0.0230999924242 test_y_nll: 0.135456323624 test_y_row_norms_max: 1.63880228996 test_y_row_norms_mean: 0.528553962708 test_y_row_norms_min: 0.0211624447256 train_h0_col_norms_max: 6.36767864227 train_h0_col_norms_mean: 4.25147294998 train_h0_col_norms_min: 2.23607754707 train_h0_row_norms_max: 6.65068531036 train_h0_row_norms_mean: 3.32795858383 train_h0_row_norms_min: 0.160931810737 train_h1_col_norms_max: 5.9968791008 train_h1_col_norms_mean: 3.86455130577 train_h1_col_norms_min: 1.72680592537 train_h1_row_norms_max: 8.77168178558 train_h1_row_norms_mean: 5.49205350876 train_h1_row_norms_min: 3.27172803879 train_objective: 0.0139410560951 train_y_col_norms_max: 6.06685829163 train_y_col_norms_mean: 5.60847902298 train_y_col_norms_min: 5.02197313309 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.997198402882 train_y_min_max_class: 0.82025551796 train_y_misclass: 0.00411999737844 train_y_nll: 0.0139410560951 train_y_row_norms_max: 1.63881158829 train_y_row_norms_mean: 0.528555572033 train_y_row_norms_min: 0.0211624447256 valid_h0_col_norms_max: 6.36764955521 valid_h0_col_norms_mean: 4.25145339966 valid_h0_col_norms_min: 2.23608016968 valid_h0_row_norms_max: 6.65068340302 valid_h0_row_norms_mean: 3.32794356346 valid_h0_row_norms_min: 0.160930916667 valid_h1_col_norms_max: 5.99686193466 valid_h1_col_norms_mean: 3.86456871033 valid_h1_col_norms_min: 1.72680532932 valid_h1_row_norms_max: 8.77167224884 valid_h1_row_norms_mean: 5.49206733704 valid_h1_row_norms_min: 3.27174091339 valid_objective: 0.154028758407 valid_y_col_norms_max: 6.06686162949 valid_y_col_norms_mean: 5.60846662521 valid_y_col_norms_min: 5.02197170258 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.993701696396 valid_y_min_max_class: 0.705734312534 valid_y_misclass: 0.0234999880195 valid_y_nll: 0.154028758407 valid_y_row_norms_max: 1.63880228996 valid_y_row_norms_mean: 0.528553962708 valid_y_row_norms_min: 0.0211624447256 Time this epoch: 3.224501 seconds Monitoring step: Epochs seen: 21 Batches seen: 10500 Examples seen: 1050000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.36559724808 test_h0_col_norms_mean: 4.25865936279 test_h0_col_norms_min: 2.23606491089 test_h0_row_norms_max: 6.65287876129 test_h0_row_norms_mean: 3.33374094963 test_h0_row_norms_min: 0.160923495889 test_h1_col_norms_max: 5.9981341362 test_h1_col_norms_mean: 3.866314888 test_h1_col_norms_min: 1.72683930397 test_h1_row_norms_max: 8.78785800934 test_h1_row_norms_mean: 5.49455070496 test_h1_row_norms_min: 3.27166962624 test_objective: 0.132553175092 test_y_col_norms_max: 6.10146903992 test_y_col_norms_mean: 5.65123224258 test_y_col_norms_min: 5.06749105453 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.992962539196 test_y_min_max_class: 0.700105249882 test_y_misclass: 0.0216999929398 test_y_nll: 0.132553175092 test_y_row_norms_max: 1.6686950922 test_y_row_norms_mean: 0.532644450665 test_y_row_norms_min: 0.0201863590628 train_h0_col_norms_max: 6.36559391022 train_h0_col_norms_mean: 4.25863981247 train_h0_col_norms_min: 2.23606181145 train_h0_row_norms_max: 6.6528468132 train_h0_row_norms_mean: 3.33372306824 train_h0_row_norms_min: 0.160924375057 train_h1_col_norms_max: 5.9981341362 train_h1_col_norms_mean: 3.86631464958 train_h1_col_norms_min: 1.7268487215 train_h1_row_norms_max: 8.78784656525 train_h1_row_norms_mean: 5.494576931 train_h1_row_norms_min: 3.27167153358 train_objective: 0.00576127693057 train_y_col_norms_max: 6.10144424438 train_y_col_norms_mean: 5.65123510361 train_y_col_norms_min: 5.06749773026 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998003363609 train_y_min_max_class: 0.865698575974 train_y_misclass: 0.00196000072174 train_y_nll: 0.00576127693057 train_y_row_norms_max: 1.66869258881 train_y_row_norms_mean: 0.532643556595 train_y_row_norms_min: 0.0201863981783 valid_h0_col_norms_max: 6.36559724808 valid_h0_col_norms_mean: 4.25865936279 valid_h0_col_norms_min: 2.23606491089 valid_h0_row_norms_max: 6.65287876129 valid_h0_row_norms_mean: 3.33374094963 valid_h0_row_norms_min: 0.160923495889 valid_h1_col_norms_max: 5.9981341362 valid_h1_col_norms_mean: 3.866314888 valid_h1_col_norms_min: 1.72683930397 valid_h1_row_norms_max: 8.78785800934 valid_h1_row_norms_mean: 5.49455070496 valid_h1_row_norms_min: 3.27166962624 valid_objective: 0.149952054024 valid_y_col_norms_max: 6.10146903992 valid_y_col_norms_mean: 5.65123224258 valid_y_col_norms_min: 5.06749105453 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.993861615658 valid_y_min_max_class: 0.696114599705 valid_y_misclass: 0.0218999926001 valid_y_nll: 0.149952054024 valid_y_row_norms_max: 1.6686950922 valid_y_row_norms_mean: 0.532644450665 valid_y_row_norms_min: 0.0201863590628 Time this epoch: 3.191485 seconds Monitoring step: Epochs seen: 22 Batches seen: 11000 Examples seen: 1100000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.37540435791 test_h0_col_norms_mean: 4.26554250717 test_h0_col_norms_min: 2.23606491089 test_h0_row_norms_max: 6.68969488144 test_h0_row_norms_mean: 3.33926701546 test_h0_row_norms_min: 0.15927760303 test_h1_col_norms_max: 5.99918460846 test_h1_col_norms_mean: 3.86793661118 test_h1_col_norms_min: 1.72684121132 test_h1_row_norms_max: 8.80519104004 test_h1_row_norms_mean: 5.49690055847 test_h1_row_norms_min: 3.27168059349 test_objective: 0.129877910018 test_y_col_norms_max: 6.1563615799 test_y_col_norms_mean: 5.69634532928 test_y_col_norms_min: 5.04322528839 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.993051469326 test_y_min_max_class: 0.701347351074 test_y_misclass: 0.0222999919206 test_y_nll: 0.129877910018 test_y_row_norms_max: 1.71757590771 test_y_row_norms_mean: 0.536909937859 test_y_row_norms_min: 0.019919058308 train_h0_col_norms_max: 6.37537336349 train_h0_col_norms_mean: 4.26554632187 train_h0_col_norms_min: 2.23606181145 train_h0_row_norms_max: 6.68972921371 train_h0_row_norms_mean: 3.33928227425 train_h0_row_norms_min: 0.159278333187 train_h1_col_norms_max: 5.99916362762 train_h1_col_norms_mean: 3.86795496941 train_h1_col_norms_min: 1.72684931755 train_h1_row_norms_max: 8.80523300171 train_h1_row_norms_mean: 5.4969124794 train_h1_row_norms_min: 3.27169203758 train_objective: 0.00547823868692 train_y_col_norms_max: 6.15638256073 train_y_col_norms_mean: 5.69631719589 train_y_col_norms_min: 5.04325008392 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998302519321 train_y_min_max_class: 0.880661785603 train_y_misclass: 0.00176000059582 train_y_nll: 0.00547823868692 train_y_row_norms_max: 1.71756851673 train_y_row_norms_mean: 0.53690803051 train_y_row_norms_min: 0.0199191085994 valid_h0_col_norms_max: 6.37540435791 valid_h0_col_norms_mean: 4.26554250717 valid_h0_col_norms_min: 2.23606491089 valid_h0_row_norms_max: 6.68969488144 valid_h0_row_norms_mean: 3.33926701546 valid_h0_row_norms_min: 0.15927760303 valid_h1_col_norms_max: 5.99918460846 valid_h1_col_norms_mean: 3.86793661118 valid_h1_col_norms_min: 1.72684121132 valid_h1_row_norms_max: 8.80519104004 valid_h1_row_norms_mean: 5.49690055847 valid_h1_row_norms_min: 3.27168059349 valid_objective: 0.151706501842 valid_y_col_norms_max: 6.1563615799 valid_y_col_norms_mean: 5.69634532928 valid_y_col_norms_min: 5.04322528839 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.99382263422 valid_y_min_max_class: 0.683702290058 valid_y_misclass: 0.0223999917507 valid_y_nll: 0.151706501842 valid_y_row_norms_max: 1.71757590771 valid_y_row_norms_mean: 0.536909937859 valid_y_row_norms_min: 0.019919058308 Time this epoch: 3.206554 seconds Monitoring step: Epochs seen: 23 Batches seen: 11500 Examples seen: 1150000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.38593149185 test_h0_col_norms_mean: 4.27048826218 test_h0_col_norms_min: 2.23606491089 test_h0_row_norms_max: 6.67957162857 test_h0_row_norms_mean: 3.34312343597 test_h0_row_norms_min: 0.159357041121 test_h1_col_norms_max: 5.99570322037 test_h1_col_norms_mean: 3.8691701889 test_h1_col_norms_min: 1.72683918476 test_h1_row_norms_max: 8.81508731842 test_h1_row_norms_mean: 5.49859952927 test_h1_row_norms_min: 3.27181625366 test_objective: 0.123887695372 test_y_col_norms_max: 6.22244215012 test_y_col_norms_mean: 5.73378896713 test_y_col_norms_min: 5.06025886536 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.993882775307 test_y_min_max_class: 0.726153492928 test_y_misclass: 0.0201999936253 test_y_nll: 0.123887695372 test_y_row_norms_max: 1.69931674004 test_y_row_norms_mean: 0.540387809277 test_y_row_norms_min: 0.020066447556 train_h0_col_norms_max: 6.38596725464 train_h0_col_norms_mean: 4.27046966553 train_h0_col_norms_min: 2.23606181145 train_h0_row_norms_max: 6.67954874039 train_h0_row_norms_mean: 3.34311199188 train_h0_row_norms_min: 0.159357577562 train_h1_col_norms_max: 5.9957318306 train_h1_col_norms_mean: 3.86917424202 train_h1_col_norms_min: 1.72684860229 train_h1_row_norms_max: 8.81507587433 train_h1_row_norms_mean: 5.49862718582 train_h1_row_norms_min: 3.27181768417 train_objective: 0.00308265769854 train_y_col_norms_max: 6.22247123718 train_y_col_norms_mean: 5.73379087448 train_y_col_norms_min: 5.06026697159 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998538374901 train_y_min_max_class: 0.894672214985 train_y_misclass: 0.000919999612961 train_y_nll: 0.00308265769854 train_y_row_norms_max: 1.69932484627 train_y_row_norms_mean: 0.540388822556 train_y_row_norms_min: 0.0200663488358 valid_h0_col_norms_max: 6.38593149185 valid_h0_col_norms_mean: 4.27048826218 valid_h0_col_norms_min: 2.23606491089 valid_h0_row_norms_max: 6.67957162857 valid_h0_row_norms_mean: 3.34312343597 valid_h0_row_norms_min: 0.159357041121 valid_h1_col_norms_max: 5.99570322037 valid_h1_col_norms_mean: 3.8691701889 valid_h1_col_norms_min: 1.72683918476 valid_h1_row_norms_max: 8.81508731842 valid_h1_row_norms_mean: 5.49859952927 valid_h1_row_norms_min: 3.27181625366 valid_objective: 0.14809820056 valid_y_col_norms_max: 6.22244215012 valid_y_col_norms_mean: 5.73378896713 valid_y_col_norms_min: 5.06025886536 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.993686497211 valid_y_min_max_class: 0.684677302837 valid_y_misclass: 0.0215999912471 valid_y_nll: 0.14809820056 valid_y_row_norms_max: 1.69931674004 valid_y_row_norms_mean: 0.540387809277 valid_y_row_norms_min: 0.020066447556 Time this epoch: 3.230241 seconds Monitoring step: Epochs seen: 24 Batches seen: 12000 Examples seen: 1200000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.392578125 test_h0_col_norms_mean: 4.27436256409 test_h0_col_norms_min: 2.23606491089 test_h0_row_norms_max: 6.68859195709 test_h0_row_norms_mean: 3.34622907639 test_h0_row_norms_min: 0.159570723772 test_h1_col_norms_max: 5.99895811081 test_h1_col_norms_mean: 3.87013435364 test_h1_col_norms_min: 1.72682142258 test_h1_row_norms_max: 8.82981967926 test_h1_row_norms_mean: 5.49993467331 test_h1_row_norms_min: 3.27214646339 test_objective: 0.123282536864 test_y_col_norms_max: 6.26617622375 test_y_col_norms_mean: 5.76239967346 test_y_col_norms_min: 5.08875703812 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994056642056 test_y_min_max_class: 0.738864719868 test_y_misclass: 0.0195999927819 test_y_nll: 0.123282536864 test_y_row_norms_max: 1.71165382862 test_y_row_norms_mean: 0.542974531651 test_y_row_norms_min: 0.0200950335711 train_h0_col_norms_max: 6.39255237579 train_h0_col_norms_mean: 4.27436685562 train_h0_col_norms_min: 2.23606181145 train_h0_row_norms_max: 6.68859434128 train_h0_row_norms_mean: 3.34621357918 train_h0_row_norms_min: 0.159569814801 train_h1_col_norms_max: 5.99892854691 train_h1_col_norms_mean: 3.8701300621 train_h1_col_norms_min: 1.72681927681 train_h1_row_norms_max: 8.82980918884 train_h1_row_norms_mean: 5.49992132187 train_h1_row_norms_min: 3.27214837074 train_objective: 0.00190907681827 train_y_col_norms_max: 6.26617431641 train_y_col_norms_mean: 5.76240110397 train_y_col_norms_min: 5.08878278732 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998930156231 train_y_min_max_class: 0.918284237385 train_y_misclass: 0.000559999898542 train_y_nll: 0.00190907681827 train_y_row_norms_max: 1.71166217327 train_y_row_norms_mean: 0.542971789837 train_y_row_norms_min: 0.0200951248407 valid_h0_col_norms_max: 6.392578125 valid_h0_col_norms_mean: 4.27436256409 valid_h0_col_norms_min: 2.23606491089 valid_h0_row_norms_max: 6.68859195709 valid_h0_row_norms_mean: 3.34622907639 valid_h0_row_norms_min: 0.159570723772 valid_h1_col_norms_max: 5.99895811081 valid_h1_col_norms_mean: 3.87013435364 valid_h1_col_norms_min: 1.72682142258 valid_h1_row_norms_max: 8.82981967926 valid_h1_row_norms_mean: 5.49993467331 valid_h1_row_norms_min: 3.27214646339 valid_objective: 0.146879151464 valid_y_col_norms_max: 6.26617622375 valid_y_col_norms_mean: 5.76239967346 valid_y_col_norms_min: 5.08875703812 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.994406104088 valid_y_min_max_class: 0.706291854382 valid_y_misclass: 0.0211999956518 valid_y_nll: 0.146879151464 valid_y_row_norms_max: 1.71165382862 valid_y_row_norms_mean: 0.542974531651 valid_y_row_norms_min: 0.0200950335711 Time this epoch: 3.222738 seconds Monitoring step: Epochs seen: 25 Batches seen: 12500 Examples seen: 1250000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.4013338089 test_h0_col_norms_mean: 4.27870225906 test_h0_col_norms_min: 2.2360560894 test_h0_row_norms_max: 6.69665718079 test_h0_row_norms_mean: 3.34976291656 test_h0_row_norms_min: 0.160002231598 test_h1_col_norms_max: 6.00171422958 test_h1_col_norms_mean: 3.87124419212 test_h1_col_norms_min: 1.72680687904 test_h1_row_norms_max: 8.85285282135 test_h1_row_norms_mean: 5.50152254105 test_h1_row_norms_min: 3.27291631699 test_objective: 0.121946468949 test_y_col_norms_max: 6.27880191803 test_y_col_norms_mean: 5.80026340485 test_y_col_norms_min: 5.12123060226 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.993581533432 test_y_min_max_class: 0.695117354393 test_y_misclass: 0.0199999921024 test_y_nll: 0.121946468949 test_y_row_norms_max: 1.76450884342 test_y_row_norms_mean: 0.546270668507 test_y_row_norms_min: 0.0209660548717 train_h0_col_norms_max: 6.40130519867 train_h0_col_norms_mean: 4.2786822319 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.69668722153 train_h0_row_norms_mean: 3.34977436066 train_h0_row_norms_min: 0.160001769662 train_h1_col_norms_max: 6.00171136856 train_h1_col_norms_mean: 3.87122607231 train_h1_col_norms_min: 1.726806283 train_h1_row_norms_max: 8.85290527344 train_h1_row_norms_mean: 5.5015130043 train_h1_row_norms_min: 3.27290010452 train_objective: 0.0036760433577 train_y_col_norms_max: 6.27877187729 train_y_col_norms_mean: 5.80025196075 train_y_col_norms_min: 5.12123060226 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998859524727 train_y_min_max_class: 0.912291646004 train_y_misclass: 0.00121999997646 train_y_nll: 0.0036760433577 train_y_row_norms_max: 1.76451909542 train_y_row_norms_mean: 0.546273350716 train_y_row_norms_min: 0.0209659561515 valid_h0_col_norms_max: 6.4013338089 valid_h0_col_norms_mean: 4.27870225906 valid_h0_col_norms_min: 2.2360560894 valid_h0_row_norms_max: 6.69665718079 valid_h0_row_norms_mean: 3.34976291656 valid_h0_row_norms_min: 0.160002231598 valid_h1_col_norms_max: 6.00171422958 valid_h1_col_norms_mean: 3.87124419212 valid_h1_col_norms_min: 1.72680687904 valid_h1_row_norms_max: 8.85285282135 valid_h1_row_norms_mean: 5.50152254105 valid_h1_row_norms_min: 3.27291631699 valid_objective: 0.137758076191 valid_y_col_norms_max: 6.27880191803 valid_y_col_norms_mean: 5.80026340485 valid_y_col_norms_min: 5.12123060226 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.994390308857 valid_y_min_max_class: 0.728678107262 valid_y_misclass: 0.019999993965 valid_y_nll: 0.137758076191 valid_y_row_norms_max: 1.76450884342 valid_y_row_norms_mean: 0.546270668507 valid_y_row_norms_min: 0.0209660548717 Time this epoch: 3.272793 seconds Monitoring step: Epochs seen: 26 Batches seen: 13000 Examples seen: 1300000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.4121389389 test_h0_col_norms_mean: 4.28374528885 test_h0_col_norms_min: 2.2360560894 test_h0_row_norms_max: 6.71324443817 test_h0_row_norms_mean: 3.35392951965 test_h0_row_norms_min: 0.1600792557 test_h1_col_norms_max: 6.00099658966 test_h1_col_norms_mean: 3.87249565125 test_h1_col_norms_min: 1.72674298286 test_h1_row_norms_max: 8.85911655426 test_h1_row_norms_mean: 5.50325918198 test_h1_row_norms_min: 3.27451777458 test_objective: 0.148935392499 test_y_col_norms_max: 6.33092308044 test_y_col_norms_mean: 5.83676052094 test_y_col_norms_min: 5.21046447754 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.993810713291 test_y_min_max_class: 0.718041598797 test_y_misclass: 0.0221999920905 test_y_nll: 0.148935392499 test_y_row_norms_max: 1.7590252161 test_y_row_norms_mean: 0.54948079586 test_y_row_norms_min: 0.020847639069 train_h0_col_norms_max: 6.41210317612 train_h0_col_norms_mean: 4.2837562561 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.71320962906 train_h0_row_norms_mean: 3.35394501686 train_h0_row_norms_min: 0.16007861495 train_h1_col_norms_max: 6.00099611282 train_h1_col_norms_mean: 3.87251186371 train_h1_col_norms_min: 1.72674548626 train_h1_row_norms_max: 8.85914611816 train_h1_row_norms_mean: 5.50324678421 train_h1_row_norms_min: 3.27453041077 train_objective: 0.00680599268526 train_y_col_norms_max: 6.33095264435 train_y_col_norms_mean: 5.83674097061 train_y_col_norms_min: 5.21046924591 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.99815505743 train_y_min_max_class: 0.87034368515 train_y_misclass: 0.00213999999687 train_y_nll: 0.00680599268526 train_y_row_norms_max: 1.75903534889 train_y_row_norms_mean: 0.549481749535 train_y_row_norms_min: 0.0208476502448 valid_h0_col_norms_max: 6.4121389389 valid_h0_col_norms_mean: 4.28374528885 valid_h0_col_norms_min: 2.2360560894 valid_h0_row_norms_max: 6.71324443817 valid_h0_row_norms_mean: 3.35392951965 valid_h0_row_norms_min: 0.1600792557 valid_h1_col_norms_max: 6.00099658966 valid_h1_col_norms_mean: 3.87249565125 valid_h1_col_norms_min: 1.72674298286 valid_h1_row_norms_max: 8.85911655426 valid_h1_row_norms_mean: 5.50325918198 valid_h1_row_norms_min: 3.27451777458 valid_objective: 0.157335549593 valid_y_col_norms_max: 6.33092308044 valid_y_col_norms_mean: 5.83676052094 valid_y_col_norms_min: 5.21046447754 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.994030356407 valid_y_min_max_class: 0.726344525814 valid_y_misclass: 0.0226999893785 valid_y_nll: 0.157335549593 valid_y_row_norms_max: 1.7590252161 valid_y_row_norms_mean: 0.54948079586 valid_y_row_norms_min: 0.020847639069 Time this epoch: 3.208633 seconds Monitoring step: Epochs seen: 27 Batches seen: 13500 Examples seen: 1350000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.41813564301 test_h0_col_norms_mean: 4.28969669342 test_h0_col_norms_min: 2.2360560894 test_h0_row_norms_max: 6.7286157608 test_h0_row_norms_mean: 3.35873889923 test_h0_row_norms_min: 0.160087496042 test_h1_col_norms_max: 6.00020074844 test_h1_col_norms_mean: 3.87404108047 test_h1_col_norms_min: 1.72669911385 test_h1_row_norms_max: 8.87103843689 test_h1_row_norms_mean: 5.50552749634 test_h1_row_norms_min: 3.27386808395 test_objective: 0.143524944782 test_y_col_norms_max: 6.35547590256 test_y_col_norms_mean: 5.87758922577 test_y_col_norms_min: 5.21483325958 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994994282722 test_y_min_max_class: 0.740391731262 test_y_misclass: 0.0209999959916 test_y_nll: 0.143524944782 test_y_row_norms_max: 1.73408651352 test_y_row_norms_mean: 0.5533670187 test_y_row_norms_min: 0.0205177664757 train_h0_col_norms_max: 6.41816806793 train_h0_col_norms_mean: 4.28971195221 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.72864484787 train_h0_row_norms_mean: 3.35872411728 train_h0_row_norms_min: 0.160087764263 train_h1_col_norms_max: 6.00021934509 train_h1_col_norms_mean: 3.87405753136 train_h1_col_norms_min: 1.72669124603 train_h1_row_norms_max: 8.87106800079 train_h1_row_norms_mean: 5.50554513931 train_h1_row_norms_min: 3.27385210991 train_objective: 0.00366839556955 train_y_col_norms_max: 6.3555059433 train_y_col_norms_mean: 5.87757110596 train_y_col_norms_min: 5.21484279633 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998865902424 train_y_min_max_class: 0.908865869045 train_y_misclass: 0.00126000004821 train_y_nll: 0.00366839556955 train_y_row_norms_max: 1.73407900333 train_y_row_norms_mean: 0.553368866444 train_y_row_norms_min: 0.0205178782344 valid_h0_col_norms_max: 6.41813564301 valid_h0_col_norms_mean: 4.28969669342 valid_h0_col_norms_min: 2.2360560894 valid_h0_row_norms_max: 6.7286157608 valid_h0_row_norms_mean: 3.35873889923 valid_h0_row_norms_min: 0.160087496042 valid_h1_col_norms_max: 6.00020074844 valid_h1_col_norms_mean: 3.87404108047 valid_h1_col_norms_min: 1.72669911385 valid_h1_row_norms_max: 8.87103843689 valid_h1_row_norms_mean: 5.50552749634 valid_h1_row_norms_min: 3.27386808395 valid_objective: 0.155297890306 valid_y_col_norms_max: 6.35547590256 valid_y_col_norms_mean: 5.87758922577 valid_y_col_norms_min: 5.21483325958 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.994454801083 valid_y_min_max_class: 0.73979562521 valid_y_misclass: 0.0205999910831 valid_y_nll: 0.155297890306 valid_y_row_norms_max: 1.73408651352 valid_y_row_norms_mean: 0.5533670187 valid_y_row_norms_min: 0.0205177664757 Time this epoch: 3.239587 seconds Monitoring step: Epochs seen: 28 Batches seen: 14000 Examples seen: 1400000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.42320108414 test_h0_col_norms_mean: 4.29595088959 test_h0_col_norms_min: 2.2360560894 test_h0_row_norms_max: 6.73588323593 test_h0_row_norms_mean: 3.36365532875 test_h0_row_norms_min: 0.160095050931 test_h1_col_norms_max: 6.00109481812 test_h1_col_norms_mean: 3.87561798096 test_h1_col_norms_min: 1.72673380375 test_h1_row_norms_max: 8.90102100372 test_h1_row_norms_mean: 5.50783443451 test_h1_row_norms_min: 3.27579259872 test_objective: 0.176090538502 test_y_col_norms_max: 6.37317848206 test_y_col_norms_mean: 5.91372203827 test_y_col_norms_min: 5.26935434341 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994395077229 test_y_min_max_class: 0.743200361729 test_y_misclass: 0.0221999939531 test_y_nll: 0.176090538502 test_y_row_norms_max: 1.72095572948 test_y_row_norms_mean: 0.556442499161 test_y_row_norms_min: 0.0208181608468 train_h0_col_norms_max: 6.4232301712 train_h0_col_norms_mean: 4.29595375061 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.73585557938 train_h0_row_norms_mean: 3.36363792419 train_h0_row_norms_min: 0.160095304251 train_h1_col_norms_max: 6.00107383728 train_h1_col_norms_mean: 3.87561368942 train_h1_col_norms_min: 1.72674226761 train_h1_row_norms_max: 8.90106678009 train_h1_row_norms_mean: 5.50785970688 train_h1_row_norms_min: 3.27577996254 train_objective: 0.00485403602943 train_y_col_norms_max: 6.37316846848 train_y_col_norms_mean: 5.91373300552 train_y_col_norms_min: 5.26935815811 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998713254929 train_y_min_max_class: 0.896820962429 train_y_misclass: 0.00136000022758 train_y_nll: 0.00485403602943 train_y_row_norms_max: 1.72096157074 train_y_row_norms_mean: 0.556439995766 train_y_row_norms_min: 0.0208181329072 valid_h0_col_norms_max: 6.42320108414 valid_h0_col_norms_mean: 4.29595088959 valid_h0_col_norms_min: 2.2360560894 valid_h0_row_norms_max: 6.73588323593 valid_h0_row_norms_mean: 3.36365532875 valid_h0_row_norms_min: 0.160095050931 valid_h1_col_norms_max: 6.00109481812 valid_h1_col_norms_mean: 3.87561798096 valid_h1_col_norms_min: 1.72673380375 valid_h1_row_norms_max: 8.90102100372 valid_h1_row_norms_mean: 5.50783443451 valid_h1_row_norms_min: 3.27579259872 valid_objective: 0.183195546269 valid_y_col_norms_max: 6.37317848206 valid_y_col_norms_mean: 5.91372203827 valid_y_col_norms_min: 5.26935434341 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.994852602482 valid_y_min_max_class: 0.74536216259 valid_y_misclass: 0.0237999893725 valid_y_nll: 0.183195546269 valid_y_row_norms_max: 1.72095572948 valid_y_row_norms_mean: 0.556442499161 valid_y_row_norms_min: 0.0208181608468 Time this epoch: 3.306142 seconds Monitoring step: Epochs seen: 29 Batches seen: 14500 Examples seen: 1450000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.45381164551 test_h0_col_norms_mean: 4.30269384384 test_h0_col_norms_min: 2.2360560894 test_h0_row_norms_max: 6.74906110764 test_h0_row_norms_mean: 3.36890244484 test_h0_row_norms_min: 0.159244820476 test_h1_col_norms_max: 6.00183820724 test_h1_col_norms_mean: 3.87737250328 test_h1_col_norms_min: 1.7269256115 test_h1_row_norms_max: 8.89922237396 test_h1_row_norms_mean: 5.51038217545 test_h1_row_norms_min: 3.27727627754 test_objective: 0.158995479345 test_y_col_norms_max: 6.38246154785 test_y_col_norms_mean: 5.95248889923 test_y_col_norms_min: 5.29096841812 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994712769985 test_y_min_max_class: 0.747330605984 test_y_misclass: 0.0207999963313 test_y_nll: 0.158995479345 test_y_row_norms_max: 1.74560809135 test_y_row_norms_mean: 0.559956371784 test_y_row_norms_min: 0.0206812545657 train_h0_col_norms_max: 6.45380783081 train_h0_col_norms_mean: 4.3027176857 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.74909591675 train_h0_row_norms_mean: 3.36888813972 train_h0_row_norms_min: 0.159244179726 train_h1_col_norms_max: 6.00187015533 train_h1_col_norms_mean: 3.87737822533 train_h1_col_norms_min: 1.72692549229 train_h1_row_norms_max: 8.89921569824 train_h1_row_norms_mean: 5.51035165787 train_h1_row_norms_min: 3.27729272842 train_objective: 0.00499874725938 train_y_col_norms_max: 6.38246393204 train_y_col_norms_mean: 5.9525179863 train_y_col_norms_min: 5.29098033905 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998843669891 train_y_min_max_class: 0.907702028751 train_y_misclass: 0.00154000031762 train_y_nll: 0.00499874725938 train_y_row_norms_max: 1.7455984354 train_y_row_norms_mean: 0.559955894947 train_y_row_norms_min: 0.0206812303513 valid_h0_col_norms_max: 6.45381164551 valid_h0_col_norms_mean: 4.30269384384 valid_h0_col_norms_min: 2.2360560894 valid_h0_row_norms_max: 6.74906110764 valid_h0_row_norms_mean: 3.36890244484 valid_h0_row_norms_min: 0.159244820476 valid_h1_col_norms_max: 6.00183820724 valid_h1_col_norms_mean: 3.87737250328 valid_h1_col_norms_min: 1.7269256115 valid_h1_row_norms_max: 8.89922237396 valid_h1_row_norms_mean: 5.51038217545 valid_h1_row_norms_min: 3.27727627754 valid_objective: 0.161353841424 valid_y_col_norms_max: 6.38246154785 valid_y_col_norms_mean: 5.95248889923 valid_y_col_norms_min: 5.29096841812 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995362341404 valid_y_min_max_class: 0.764035582542 valid_y_misclass: 0.0211999919266 valid_y_nll: 0.161353841424 valid_y_row_norms_max: 1.74560809135 valid_y_row_norms_mean: 0.559956371784 valid_y_row_norms_min: 0.0206812545657 Time this epoch: 3.264931 seconds Monitoring step: Epochs seen: 30 Batches seen: 15000 Examples seen: 1500000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.45126152039 test_h0_col_norms_mean: 4.30855321884 test_h0_col_norms_min: 2.2360560894 test_h0_row_norms_max: 6.77185153961 test_h0_row_norms_mean: 3.37364006042 test_h0_row_norms_min: 0.159440949559 test_h1_col_norms_max: 6.00142860413 test_h1_col_norms_mean: 3.8789036274 test_h1_col_norms_min: 1.72696387768 test_h1_row_norms_max: 8.92525005341 test_h1_row_norms_mean: 5.5125746727 test_h1_row_norms_min: 3.27923321724 test_objective: 0.159945309162 test_y_col_norms_max: 6.50855636597 test_y_col_norms_mean: 5.9870095253 test_y_col_norms_min: 5.30891561508 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995383441448 test_y_min_max_class: 0.755910158157 test_y_misclass: 0.0218999926001 test_y_nll: 0.159945309162 test_y_row_norms_max: 1.7809484005 test_y_row_norms_mean: 0.563234627247 test_y_row_norms_min: 0.0199234094471 train_h0_col_norms_max: 6.45129537582 train_h0_col_norms_mean: 4.30855226517 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.77182006836 train_h0_row_norms_mean: 3.37362527847 train_h0_row_norms_min: 0.159441739321 train_h1_col_norms_max: 6.00145721436 train_h1_col_norms_mean: 3.87892222404 train_h1_col_norms_min: 1.72697114944 train_h1_row_norms_max: 8.92523765564 train_h1_row_norms_mean: 5.5125579834 train_h1_row_norms_min: 3.27921772003 train_objective: 0.0052194846794 train_y_col_norms_max: 6.50858449936 train_y_col_norms_mean: 5.98699235916 train_y_col_norms_min: 5.3088889122 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998774111271 train_y_min_max_class: 0.904954195023 train_y_misclass: 0.00160000030883 train_y_nll: 0.0052194846794 train_y_row_norms_max: 1.78094053268 train_y_row_norms_mean: 0.56323415041 train_y_row_norms_min: 0.0199234373868 valid_h0_col_norms_max: 6.45126152039 valid_h0_col_norms_mean: 4.30855321884 valid_h0_col_norms_min: 2.2360560894 valid_h0_row_norms_max: 6.77185153961 valid_h0_row_norms_mean: 3.37364006042 valid_h0_row_norms_min: 0.159440949559 valid_h1_col_norms_max: 6.00142860413 valid_h1_col_norms_mean: 3.8789036274 valid_h1_col_norms_min: 1.72696387768 valid_h1_row_norms_max: 8.92525005341 valid_h1_row_norms_mean: 5.5125746727 valid_h1_row_norms_min: 3.27923321724 valid_objective: 0.172797784209 valid_y_col_norms_max: 6.50855636597 valid_y_col_norms_mean: 5.9870095253 valid_y_col_norms_min: 5.30891561508 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995368361473 valid_y_min_max_class: 0.741488099098 valid_y_misclass: 0.0201999936253 valid_y_nll: 0.172797784209 valid_y_row_norms_max: 1.7809484005 valid_y_row_norms_mean: 0.563234627247 valid_y_row_norms_min: 0.0199234094471 Time this epoch: 3.279603 seconds Monitoring step: Epochs seen: 31 Batches seen: 15500 Examples seen: 1550000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.45977544785 test_h0_col_norms_mean: 4.31496477127 test_h0_col_norms_min: 2.23605561256 test_h0_row_norms_max: 6.77787017822 test_h0_row_norms_mean: 3.37872552872 test_h0_row_norms_min: 0.167061835527 test_h1_col_norms_max: 6.00070905685 test_h1_col_norms_mean: 3.88056731224 test_h1_col_norms_min: 1.7269756794 test_h1_row_norms_max: 8.94437408447 test_h1_row_norms_mean: 5.51490449905 test_h1_row_norms_min: 3.27992272377 test_objective: 0.131766811013 test_y_col_norms_max: 6.49069547653 test_y_col_norms_mean: 6.01968860626 test_y_col_norms_min: 5.32379293442 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.99406349659 test_y_min_max_class: 0.709186255932 test_y_misclass: 0.0214999895543 test_y_nll: 0.131766811013 test_y_row_norms_max: 1.75881135464 test_y_row_norms_mean: 0.566387176514 test_y_row_norms_min: 0.0195109490305 train_h0_col_norms_max: 6.45976924896 train_h0_col_norms_mean: 4.31498289108 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.77783346176 train_h0_row_norms_mean: 3.37874174118 train_h0_row_norms_min: 0.167062133551 train_h1_col_norms_max: 6.00073814392 train_h1_col_norms_mean: 3.88058972359 train_h1_col_norms_min: 1.72698163986 train_h1_row_norms_max: 8.94434833527 train_h1_row_norms_mean: 5.51487779617 train_h1_row_norms_min: 3.27992391586 train_objective: 0.00692026689649 train_y_col_norms_max: 6.49070358276 train_y_col_norms_mean: 6.01966762543 train_y_col_norms_min: 5.323802948 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.99833124876 train_y_min_max_class: 0.877075016499 train_y_misclass: 0.00206000055186 train_y_nll: 0.00692026689649 train_y_row_norms_max: 1.7588135004 train_y_row_norms_mean: 0.566390037537 train_y_row_norms_min: 0.0195109229535 valid_h0_col_norms_max: 6.45977544785 valid_h0_col_norms_mean: 4.31496477127 valid_h0_col_norms_min: 2.23605561256 valid_h0_row_norms_max: 6.77787017822 valid_h0_row_norms_mean: 3.37872552872 valid_h0_row_norms_min: 0.167061835527 valid_h1_col_norms_max: 6.00070905685 valid_h1_col_norms_mean: 3.88056731224 valid_h1_col_norms_min: 1.7269756794 valid_h1_row_norms_max: 8.94437408447 valid_h1_row_norms_mean: 5.51490449905 valid_h1_row_norms_min: 3.27992272377 valid_objective: 0.161748409271 valid_y_col_norms_max: 6.49069547653 valid_y_col_norms_mean: 6.01968860626 valid_y_col_norms_min: 5.32379293442 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.994541585445 valid_y_min_max_class: 0.741445958614 valid_y_misclass: 0.0221999902278 valid_y_nll: 0.161748409271 valid_y_row_norms_max: 1.75881135464 valid_y_row_norms_mean: 0.566387176514 valid_y_row_norms_min: 0.0195109490305 Time this epoch: 3.251266 seconds Monitoring step: Epochs seen: 32 Batches seen: 16000 Examples seen: 1600000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.45842170715 test_h0_col_norms_mean: 4.32045173645 test_h0_col_norms_min: 2.23605561256 test_h0_row_norms_max: 6.78466415405 test_h0_row_norms_mean: 3.38315415382 test_h0_row_norms_min: 0.16744081676 test_h1_col_norms_max: 6.00018596649 test_h1_col_norms_mean: 3.88205099106 test_h1_col_norms_min: 1.72608160973 test_h1_row_norms_max: 8.9562330246 test_h1_row_norms_mean: 5.51705217361 test_h1_row_norms_min: 3.28056788445 test_objective: 0.156137660146 test_y_col_norms_max: 6.55750894547 test_y_col_norms_mean: 6.04845666885 test_y_col_norms_min: 5.33018064499 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995250225067 test_y_min_max_class: 0.769113063812 test_y_misclass: 0.0210999939591 test_y_nll: 0.156137660146 test_y_row_norms_max: 1.7797113657 test_y_row_norms_mean: 0.568675458431 test_y_row_norms_min: 0.0223224461079 train_h0_col_norms_max: 6.45844841003 train_h0_col_norms_mean: 4.32046604156 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.7846736908 train_h0_row_norms_mean: 3.3831589222 train_h0_row_norms_min: 0.167441576719 train_h1_col_norms_max: 6.00020599365 train_h1_col_norms_mean: 3.88205075264 train_h1_col_norms_min: 1.7260876894 train_h1_row_norms_max: 8.9562292099 train_h1_row_norms_mean: 5.51702356339 train_h1_row_norms_min: 3.280554533 train_objective: 0.00448899809271 train_y_col_norms_max: 6.55747938156 train_y_col_norms_mean: 6.0484457016 train_y_col_norms_min: 5.33017015457 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998964965343 train_y_min_max_class: 0.915260314941 train_y_misclass: 0.0013400001917 train_y_nll: 0.00448899809271 train_y_row_norms_max: 1.77971935272 train_y_row_norms_mean: 0.568675458431 train_y_row_norms_min: 0.0223225466907 valid_h0_col_norms_max: 6.45842170715 valid_h0_col_norms_mean: 4.32045173645 valid_h0_col_norms_min: 2.23605561256 valid_h0_row_norms_max: 6.78466415405 valid_h0_row_norms_mean: 3.38315415382 valid_h0_row_norms_min: 0.16744081676 valid_h1_col_norms_max: 6.00018596649 valid_h1_col_norms_mean: 3.88205099106 valid_h1_col_norms_min: 1.72608160973 valid_h1_row_norms_max: 8.9562330246 valid_h1_row_norms_mean: 5.51705217361 valid_h1_row_norms_min: 3.28056788445 valid_objective: 0.185146003962 valid_y_col_norms_max: 6.55750894547 valid_y_col_norms_mean: 6.04845666885 valid_y_col_norms_min: 5.33018064499 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995594918728 valid_y_min_max_class: 0.771956503391 valid_y_misclass: 0.0222999881953 valid_y_nll: 0.185146003962 valid_y_row_norms_max: 1.7797113657 valid_y_row_norms_mean: 0.568675458431 valid_y_row_norms_min: 0.0223224461079 Time this epoch: 3.265816 seconds Monitoring step: Epochs seen: 33 Batches seen: 16500 Examples seen: 1650000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.48042154312 test_h0_col_norms_mean: 4.32581949234 test_h0_col_norms_min: 2.23605561256 test_h0_row_norms_max: 6.79249668121 test_h0_row_norms_mean: 3.38737988472 test_h0_row_norms_min: 0.167504921556 test_h1_col_norms_max: 6.0035238266 test_h1_col_norms_mean: 3.88333916664 test_h1_col_norms_min: 1.72610199451 test_h1_row_norms_max: 8.94651126862 test_h1_row_norms_mean: 5.51890897751 test_h1_row_norms_min: 3.28360319138 test_objective: 0.142962425947 test_y_col_norms_max: 6.59494447708 test_y_col_norms_mean: 6.06826543808 test_y_col_norms_min: 5.36811923981 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995299935341 test_y_min_max_class: 0.757121562958 test_y_misclass: 0.0198999904096 test_y_nll: 0.142962425947 test_y_row_norms_max: 1.8589527607 test_y_row_norms_mean: 0.570496380329 test_y_row_norms_min: 0.0232647489756 train_h0_col_norms_max: 6.48045063019 train_h0_col_norms_mean: 4.32584047318 train_h0_col_norms_min: 2.23605871201 train_h0_row_norms_max: 6.79252815247 train_h0_row_norms_mean: 3.38736534119 train_h0_row_norms_min: 0.167504131794 train_h1_col_norms_max: 6.00354385376 train_h1_col_norms_mean: 3.88335561752 train_h1_col_norms_min: 1.72609436512 train_h1_row_norms_max: 8.94646167755 train_h1_row_norms_mean: 5.51891183853 train_h1_row_norms_min: 3.28361749649 train_objective: 0.00277355127037 train_y_col_norms_max: 6.59491348267 train_y_col_norms_mean: 6.06824493408 train_y_col_norms_min: 5.36814403534 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999049842358 train_y_min_max_class: 0.921933472157 train_y_misclass: 0.000999999581836 train_y_nll: 0.00277355127037 train_y_row_norms_max: 1.85895049572 train_y_row_norms_mean: 0.570495426655 train_y_row_norms_min: 0.0232646763325 valid_h0_col_norms_max: 6.48042154312 valid_h0_col_norms_mean: 4.32581949234 valid_h0_col_norms_min: 2.23605561256 valid_h0_row_norms_max: 6.79249668121 valid_h0_row_norms_mean: 3.38737988472 valid_h0_row_norms_min: 0.167504921556 valid_h1_col_norms_max: 6.0035238266 valid_h1_col_norms_mean: 3.88333916664 valid_h1_col_norms_min: 1.72610199451 valid_h1_row_norms_max: 8.94651126862 valid_h1_row_norms_mean: 5.51890897751 valid_h1_row_norms_min: 3.28360319138 valid_objective: 0.179574415088 valid_y_col_norms_max: 6.59494447708 valid_y_col_norms_mean: 6.06826543808 valid_y_col_norms_min: 5.36811923981 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995453417301 valid_y_min_max_class: 0.75123167038 valid_y_misclass: 0.0197999905795 valid_y_nll: 0.179574415088 valid_y_row_norms_max: 1.8589527607 valid_y_row_norms_mean: 0.570496380329 valid_y_row_norms_min: 0.0232647489756 Time this epoch: 3.231476 seconds Monitoring step: Epochs seen: 34 Batches seen: 17000 Examples seen: 1700000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.4917049408 test_h0_col_norms_mean: 4.33141994476 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.79732465744 test_h0_row_norms_mean: 3.39186024666 test_h0_row_norms_min: 0.171120882034 test_h1_col_norms_max: 6.00534772873 test_h1_col_norms_mean: 3.88460206985 test_h1_col_norms_min: 1.72610270977 test_h1_row_norms_max: 8.96625423431 test_h1_row_norms_mean: 5.52066421509 test_h1_row_norms_min: 3.28276824951 test_objective: 0.141110450029 test_y_col_norms_max: 6.61644887924 test_y_col_norms_mean: 6.09203910828 test_y_col_norms_min: 5.40572547913 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994820356369 test_y_min_max_class: 0.726503133774 test_y_misclass: 0.0200999956578 test_y_nll: 0.141110450029 test_y_row_norms_max: 1.85092616081 test_y_row_norms_mean: 0.572713196278 test_y_row_norms_min: 0.0240506455302 train_h0_col_norms_max: 6.49167537689 train_h0_col_norms_mean: 4.33143472672 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.79731702805 train_h0_row_norms_mean: 3.39186143875 train_h0_row_norms_min: 0.171120166779 train_h1_col_norms_max: 6.00534725189 train_h1_col_norms_mean: 3.88458299637 train_h1_col_norms_min: 1.72609496117 train_h1_row_norms_max: 8.9662437439 train_h1_row_norms_mean: 5.52065134048 train_h1_row_norms_min: 3.28278303146 train_objective: 0.00290546845645 train_y_col_norms_max: 6.61642169952 train_y_col_norms_mean: 6.09206676483 train_y_col_norms_min: 5.40573072433 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999073982239 train_y_min_max_class: 0.924475312233 train_y_misclass: 0.000939999590628 train_y_nll: 0.00290546845645 train_y_row_norms_max: 1.85091614723 train_y_row_norms_mean: 0.572711467743 train_y_row_norms_min: 0.0240505319089 valid_h0_col_norms_max: 6.4917049408 valid_h0_col_norms_mean: 4.33141994476 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.79732465744 valid_h0_row_norms_mean: 3.39186024666 valid_h0_row_norms_min: 0.171120882034 valid_h1_col_norms_max: 6.00534772873 valid_h1_col_norms_mean: 3.88460206985 valid_h1_col_norms_min: 1.72610270977 valid_h1_row_norms_max: 8.96625423431 valid_h1_row_norms_mean: 5.52066421509 valid_h1_row_norms_min: 3.28276824951 valid_objective: 0.162981122732 valid_y_col_norms_max: 6.61644887924 valid_y_col_norms_mean: 6.09203910828 valid_y_col_norms_min: 5.40572547913 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995312690735 valid_y_min_max_class: 0.743762373924 valid_y_misclass: 0.0194999910891 valid_y_nll: 0.162981122732 valid_y_row_norms_max: 1.85092616081 valid_y_row_norms_mean: 0.572713196278 valid_y_row_norms_min: 0.0240506455302 Time this epoch: 3.214131 seconds Monitoring step: Epochs seen: 35 Batches seen: 17500 Examples seen: 1750000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.49574804306 test_h0_col_norms_mean: 4.3364033699 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.8160161972 test_h0_row_norms_mean: 3.39588427544 test_h0_row_norms_min: 0.171171665192 test_h1_col_norms_max: 6.00441598892 test_h1_col_norms_mean: 3.88574457169 test_h1_col_norms_min: 1.72610199451 test_h1_row_norms_max: 8.98808574677 test_h1_row_norms_mean: 5.52225542068 test_h1_row_norms_min: 3.28273797035 test_objective: 0.170048907399 test_y_col_norms_max: 6.62913417816 test_y_col_norms_mean: 6.11489725113 test_y_col_norms_min: 5.41416931152 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994616866112 test_y_min_max_class: 0.73312073946 test_y_misclass: 0.0217999909073 test_y_nll: 0.170048907399 test_y_row_norms_max: 1.85863983631 test_y_row_norms_mean: 0.574832618237 test_y_row_norms_min: 0.0238261986524 train_h0_col_norms_max: 6.49571895599 train_h0_col_norms_mean: 4.33637952805 train_h0_col_norms_min: 2.23605918884 train_h0_row_norms_max: 6.81597948074 train_h0_row_norms_mean: 3.39588832855 train_h0_row_norms_min: 0.171171709895 train_h1_col_norms_max: 6.00439691544 train_h1_col_norms_mean: 3.88574552536 train_h1_col_norms_min: 1.72609436512 train_h1_row_norms_max: 8.98807621002 train_h1_row_norms_mean: 5.52225255966 train_h1_row_norms_min: 3.28275132179 train_objective: 0.00725457724184 train_y_col_norms_max: 6.62916135788 train_y_col_norms_mean: 6.11490011215 train_y_col_norms_min: 5.41417980194 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998784661293 train_y_min_max_class: 0.90457379818 train_y_misclass: 0.00184000050649 train_y_nll: 0.00725457724184 train_y_row_norms_max: 1.85864841938 train_y_row_norms_mean: 0.574829816818 train_y_row_norms_min: 0.0238260868937 valid_h0_col_norms_max: 6.49574804306 valid_h0_col_norms_mean: 4.3364033699 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.8160161972 valid_h0_row_norms_mean: 3.39588427544 valid_h0_row_norms_min: 0.171171665192 valid_h1_col_norms_max: 6.00441598892 valid_h1_col_norms_mean: 3.88574457169 valid_h1_col_norms_min: 1.72610199451 valid_h1_row_norms_max: 8.98808574677 valid_h1_row_norms_mean: 5.52225542068 valid_h1_row_norms_min: 3.28273797035 valid_objective: 0.188135892153 valid_y_col_norms_max: 6.62913417816 valid_y_col_norms_mean: 6.11489725113 valid_y_col_norms_min: 5.41416931152 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.99530762434 valid_y_min_max_class: 0.754189014435 valid_y_misclass: 0.0216999892145 valid_y_nll: 0.188135892153 valid_y_row_norms_max: 1.85863983631 valid_y_row_norms_mean: 0.574832618237 valid_y_row_norms_min: 0.0238261986524 Time this epoch: 3.284179 seconds Monitoring step: Epochs seen: 36 Batches seen: 18000 Examples seen: 1800000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.50911712646 test_h0_col_norms_mean: 4.34344434738 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.84686803818 test_h0_row_norms_mean: 3.40156388283 test_h0_row_norms_min: 0.171174243093 test_h1_col_norms_max: 6.00547456741 test_h1_col_norms_mean: 3.88733744621 test_h1_col_norms_min: 1.72609961033 test_h1_row_norms_max: 9.016705513 test_h1_row_norms_mean: 5.5244436264 test_h1_row_norms_min: 3.28328037262 test_objective: 0.147451668978 test_y_col_norms_max: 6.66465806961 test_y_col_norms_mean: 6.14104557037 test_y_col_norms_min: 5.43022489548 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.994985222816 test_y_min_max_class: 0.730050563812 test_y_misclass: 0.0206999927759 test_y_nll: 0.147451668978 test_y_row_norms_max: 1.78328752518 test_y_row_norms_mean: 0.577396690845 test_y_row_norms_min: 0.025094171986 train_h0_col_norms_max: 6.50908374786 train_h0_col_norms_mean: 4.34342718124 train_h0_col_norms_min: 2.23605918884 train_h0_row_norms_max: 6.8468914032 train_h0_row_norms_mean: 3.40154623985 train_h0_row_norms_min: 0.171174883842 train_h1_col_norms_max: 6.00550603867 train_h1_col_norms_mean: 3.8873193264 train_h1_col_norms_min: 1.72609198093 train_h1_row_norms_max: 9.0167131424 train_h1_row_norms_mean: 5.52441453934 train_h1_row_norms_min: 3.28326916695 train_objective: 0.00539966486394 train_y_col_norms_max: 6.66468572617 train_y_col_norms_mean: 6.14102125168 train_y_col_norms_min: 5.43022203445 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.99889010191 train_y_min_max_class: 0.915294647217 train_y_misclass: 0.00152000051457 train_y_nll: 0.00539966486394 train_y_row_norms_max: 1.78329563141 train_y_row_norms_mean: 0.57739341259 train_y_row_norms_min: 0.0250942651182 valid_h0_col_norms_max: 6.50911712646 valid_h0_col_norms_mean: 4.34344434738 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.84686803818 valid_h0_row_norms_mean: 3.40156388283 valid_h0_row_norms_min: 0.171174243093 valid_h1_col_norms_max: 6.00547456741 valid_h1_col_norms_mean: 3.88733744621 valid_h1_col_norms_min: 1.72609961033 valid_h1_row_norms_max: 9.016705513 valid_h1_row_norms_mean: 5.5244436264 valid_h1_row_norms_min: 3.28328037262 valid_objective: 0.161581993103 valid_y_col_norms_max: 6.66465806961 valid_y_col_norms_mean: 6.14104557037 valid_y_col_norms_min: 5.43022489548 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995217263699 valid_y_min_max_class: 0.752208411694 valid_y_misclass: 0.0202999934554 valid_y_nll: 0.161581993103 valid_y_row_norms_max: 1.78328752518 valid_y_row_norms_mean: 0.577396690845 valid_y_row_norms_min: 0.025094171986 Time this epoch: 3.277391 seconds Monitoring step: Epochs seen: 37 Batches seen: 18500 Examples seen: 1850000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.51882982254 test_h0_col_norms_mean: 4.34898805618 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.86316585541 test_h0_row_norms_mean: 3.40600013733 test_h0_row_norms_min: 0.171176031232 test_h1_col_norms_max: 6.00360631943 test_h1_col_norms_mean: 3.88884663582 test_h1_col_norms_min: 1.72619795799 test_h1_row_norms_max: 9.0371131897 test_h1_row_norms_mean: 5.526512146 test_h1_row_norms_min: 3.28363656998 test_objective: 0.174357533455 test_y_col_norms_max: 6.70250511169 test_y_col_norms_mean: 6.17451667786 test_y_col_norms_min: 5.43355512619 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995144307613 test_y_min_max_class: 0.754079401493 test_y_misclass: 0.0214999951422 test_y_nll: 0.174357533455 test_y_row_norms_max: 1.83495354652 test_y_row_norms_mean: 0.580399692059 test_y_row_norms_min: 0.0246269144118 train_h0_col_norms_max: 6.51883935928 train_h0_col_norms_mean: 4.34899139404 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.86313438416 train_h0_row_norms_mean: 3.40601491928 train_h0_row_norms_min: 0.171176567674 train_h1_col_norms_max: 6.00361680984 train_h1_col_norms_mean: 3.88884592056 train_h1_col_norms_min: 1.72620582581 train_h1_row_norms_max: 9.03706741333 train_h1_row_norms_mean: 5.52653741837 train_h1_row_norms_min: 3.28362202644 train_objective: 0.00331209623255 train_y_col_norms_max: 6.70247983932 train_y_col_norms_mean: 6.17454624176 train_y_col_norms_min: 5.43355798721 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999139487743 train_y_min_max_class: 0.931698381901 train_y_misclass: 0.000979999545962 train_y_nll: 0.00331209623255 train_y_row_norms_max: 1.83494448662 train_y_row_norms_mean: 0.580400049686 train_y_row_norms_min: 0.0246269479394 valid_h0_col_norms_max: 6.51882982254 valid_h0_col_norms_mean: 4.34898805618 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.86316585541 valid_h0_row_norms_mean: 3.40600013733 valid_h0_row_norms_min: 0.171176031232 valid_h1_col_norms_max: 6.00360631943 valid_h1_col_norms_mean: 3.88884663582 valid_h1_col_norms_min: 1.72619795799 valid_h1_row_norms_max: 9.0371131897 valid_h1_row_norms_mean: 5.526512146 valid_h1_row_norms_min: 3.28363656998 valid_objective: 0.164556577802 valid_y_col_norms_max: 6.70250511169 valid_y_col_norms_mean: 6.17451667786 valid_y_col_norms_min: 5.43355512619 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995738983154 valid_y_min_max_class: 0.76286149025 valid_y_misclass: 0.0205999910831 valid_y_nll: 0.164556577802 valid_y_row_norms_max: 1.83495354652 valid_y_row_norms_mean: 0.580399692059 valid_y_row_norms_min: 0.0246269144118 Time this epoch: 3.300500 seconds Monitoring step: Epochs seen: 38 Batches seen: 19000 Examples seen: 1900000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.52381372452 test_h0_col_norms_mean: 4.35216140747 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.87646770477 test_h0_row_norms_mean: 3.40848636627 test_h0_row_norms_min: 0.171177119017 test_h1_col_norms_max: 6.00470304489 test_h1_col_norms_mean: 3.88970422745 test_h1_col_norms_min: 1.72622287273 test_h1_row_norms_max: 9.0545091629 test_h1_row_norms_mean: 5.52772140503 test_h1_row_norms_min: 3.28486537933 test_objective: 0.16956473887 test_y_col_norms_max: 6.70925521851 test_y_col_norms_mean: 6.2000246048 test_y_col_norms_min: 5.47072219849 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996227920055 test_y_min_max_class: 0.774923741817 test_y_misclass: 0.0192999932915 test_y_nll: 0.16956473887 test_y_row_norms_max: 1.87937033176 test_y_row_norms_mean: 0.582399070263 test_y_row_norms_min: 0.0244527608156 train_h0_col_norms_max: 6.52384281158 train_h0_col_norms_mean: 4.35217618942 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87646818161 train_h0_row_norms_mean: 3.40846681595 train_h0_row_norms_min: 0.171177104115 train_h1_col_norms_max: 6.00473213196 train_h1_col_norms_mean: 3.88968753815 train_h1_col_norms_min: 1.72621440887 train_h1_row_norms_max: 9.05449295044 train_h1_row_norms_mean: 5.52773332596 train_h1_row_norms_min: 3.28484797478 train_objective: 0.0016456496669 train_y_col_norms_max: 6.70928049088 train_y_col_norms_mean: 6.20005607605 train_y_col_norms_min: 5.47074699402 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999532461166 train_y_min_max_class: 0.958807349205 train_y_misclass: 0.000520000001416 train_y_nll: 0.0016456496669 train_y_row_norms_max: 1.87937915325 train_y_row_norms_mean: 0.582397639751 train_y_row_norms_min: 0.0244527999312 valid_h0_col_norms_max: 6.52381372452 valid_h0_col_norms_mean: 4.35216140747 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.87646770477 valid_h0_row_norms_mean: 3.40848636627 valid_h0_row_norms_min: 0.171177119017 valid_h1_col_norms_max: 6.00470304489 valid_h1_col_norms_mean: 3.88970422745 valid_h1_col_norms_min: 1.72622287273 valid_h1_row_norms_max: 9.0545091629 valid_h1_row_norms_mean: 5.52772140503 valid_h1_row_norms_min: 3.28486537933 valid_objective: 0.174608826637 valid_y_col_norms_max: 6.70925521851 valid_y_col_norms_mean: 6.2000246048 valid_y_col_norms_min: 5.47072219849 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996570110321 valid_y_min_max_class: 0.792669534683 valid_y_misclass: 0.0185999963433 valid_y_nll: 0.174608826637 valid_y_row_norms_max: 1.87937033176 valid_y_row_norms_mean: 0.582399070263 valid_y_row_norms_min: 0.0244527608156 Time this epoch: 3.301847 seconds Monitoring step: Epochs seen: 39 Batches seen: 19500 Examples seen: 1950000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.52631568909 test_h0_col_norms_mean: 4.35376691818 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.87409830093 test_h0_row_norms_mean: 3.40977239609 test_h0_row_norms_min: 0.171177133918 test_h1_col_norms_max: 6.00363349915 test_h1_col_norms_mean: 3.89011406898 test_h1_col_norms_min: 1.72623074055 test_h1_row_norms_max: 9.06535053253 test_h1_row_norms_mean: 5.52831077576 test_h1_row_norms_min: 3.28474617004 test_objective: 0.158702552319 test_y_col_norms_max: 6.72936153412 test_y_col_norms_mean: 6.2109913826 test_y_col_norms_min: 5.48157644272 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995932340622 test_y_min_max_class: 0.764656722546 test_y_misclass: 0.0200999919325 test_y_nll: 0.158702552319 test_y_row_norms_max: 1.87921774387 test_y_row_norms_mean: 0.583407759666 test_y_row_norms_min: 0.024447273463 train_h0_col_norms_max: 6.52629041672 train_h0_col_norms_mean: 4.35376310349 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87408828735 train_h0_row_norms_mean: 3.40976953506 train_h0_row_norms_min: 0.171177104115 train_h1_col_norms_max: 6.00362253189 train_h1_col_norms_mean: 3.8901321888 train_h1_col_norms_min: 1.72622382641 train_h1_row_norms_max: 9.06535148621 train_h1_row_norms_mean: 5.52829360962 train_h1_row_norms_min: 3.28472876549 train_objective: 0.00152394291945 train_y_col_norms_max: 6.7293639183 train_y_col_norms_mean: 6.21102333069 train_y_col_norms_min: 5.48156309128 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999768614769 train_y_min_max_class: 0.979155957699 train_y_misclass: 0.000379999983124 train_y_nll: 0.00152394291945 train_y_row_norms_max: 1.87921559811 train_y_row_norms_mean: 0.583409488201 train_y_row_norms_min: 0.0244472324848 valid_h0_col_norms_max: 6.52631568909 valid_h0_col_norms_mean: 4.35376691818 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.87409830093 valid_h0_row_norms_mean: 3.40977239609 valid_h0_row_norms_min: 0.171177133918 valid_h1_col_norms_max: 6.00363349915 valid_h1_col_norms_mean: 3.89011406898 valid_h1_col_norms_min: 1.72623074055 valid_h1_row_norms_max: 9.06535053253 valid_h1_row_norms_mean: 5.52831077576 valid_h1_row_norms_min: 3.28474617004 valid_objective: 0.17522443831 valid_y_col_norms_max: 6.72936153412 valid_y_col_norms_mean: 6.2109913826 valid_y_col_norms_min: 5.48157644272 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996479153633 valid_y_min_max_class: 0.788241684437 valid_y_misclass: 0.0187999941409 valid_y_nll: 0.17522443831 valid_y_row_norms_max: 1.87921774387 valid_y_row_norms_mean: 0.583407759666 valid_y_row_norms_min: 0.024447273463 Time this epoch: 3.268098 seconds Monitoring step: Epochs seen: 40 Batches seen: 20000 Examples seen: 2000000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.53570699692 test_h0_col_norms_mean: 4.35643339157 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.86570596695 test_h0_row_norms_mean: 3.41193628311 test_h0_row_norms_min: 0.171177208424 test_h1_col_norms_max: 6.00472784042 test_h1_col_norms_mean: 3.89065885544 test_h1_col_norms_min: 1.72635400295 test_h1_row_norms_max: 9.0626745224 test_h1_row_norms_mean: 5.52905321121 test_h1_row_norms_min: 3.28488898277 test_objective: 0.16143476963 test_y_col_norms_max: 6.73923158646 test_y_col_norms_mean: 6.22264146805 test_y_col_norms_min: 5.52369451523 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995911836624 test_y_min_max_class: 0.785954415798 test_y_misclass: 0.019199995324 test_y_nll: 0.16143476963 test_y_row_norms_max: 1.85353505611 test_y_row_norms_mean: 0.584432959557 test_y_row_norms_min: 0.0243270788342 train_h0_col_norms_max: 6.5357131958 train_h0_col_norms_mean: 4.35641145706 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.86573553085 train_h0_row_norms_mean: 3.411921978 train_h0_row_norms_min: 0.171177133918 train_h1_col_norms_max: 6.00474691391 train_h1_col_norms_mean: 3.89064121246 train_h1_col_norms_min: 1.72634625435 train_h1_row_norms_max: 9.06272411346 train_h1_row_norms_mean: 5.52907943726 train_h1_row_norms_min: 3.28490185738 train_objective: 0.00306967948563 train_y_col_norms_max: 6.73919677734 train_y_col_norms_mean: 6.22266340256 train_y_col_norms_min: 5.52368307114 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999223351479 train_y_min_max_class: 0.938300907612 train_y_misclass: 0.00105999980588 train_y_nll: 0.00306967948563 train_y_row_norms_max: 1.85352873802 train_y_row_norms_mean: 0.584433317184 train_y_row_norms_min: 0.0243270788342 valid_h0_col_norms_max: 6.53570699692 valid_h0_col_norms_mean: 4.35643339157 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.86570596695 valid_h0_row_norms_mean: 3.41193628311 valid_h0_row_norms_min: 0.171177208424 valid_h1_col_norms_max: 6.00472784042 valid_h1_col_norms_mean: 3.89065885544 valid_h1_col_norms_min: 1.72635400295 valid_h1_row_norms_max: 9.0626745224 valid_h1_row_norms_mean: 5.52905321121 valid_h1_row_norms_min: 3.28488898277 valid_objective: 0.182417109609 valid_y_col_norms_max: 6.73923158646 valid_y_col_norms_mean: 6.22264146805 valid_y_col_norms_min: 5.52369451523 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996435403824 valid_y_min_max_class: 0.793238520622 valid_y_misclass: 0.0203999951482 valid_y_nll: 0.182417109609 valid_y_row_norms_max: 1.85353505611 valid_y_row_norms_mean: 0.584432959557 valid_y_row_norms_min: 0.0243270788342 Time this epoch: 3.294892 seconds Monitoring step: Epochs seen: 41 Batches seen: 20500 Examples seen: 2050000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.5425863266 test_h0_col_norms_mean: 4.35914468765 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.87823629379 test_h0_row_norms_mean: 3.41422724724 test_h0_row_norms_min: 0.171178132296 test_h1_col_norms_max: 6.00586032867 test_h1_col_norms_mean: 3.89139056206 test_h1_col_norms_min: 1.72638916969 test_h1_row_norms_max: 9.06592273712 test_h1_row_norms_mean: 5.53010177612 test_h1_row_norms_min: 3.28573608398 test_objective: 0.158061608672 test_y_col_norms_max: 6.74868965149 test_y_col_norms_mean: 6.23669672012 test_y_col_norms_min: 5.50828027725 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995399415493 test_y_min_max_class: 0.739345610142 test_y_misclass: 0.0194999948144 test_y_nll: 0.158061608672 test_y_row_norms_max: 1.86322903633 test_y_row_norms_mean: 0.585780024529 test_y_row_norms_min: 0.0242832899094 train_h0_col_norms_max: 6.54261350632 train_h0_col_norms_mean: 4.3591375351 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87820577621 train_h0_row_norms_mean: 3.41424489021 train_h0_row_norms_min: 0.171177610755 train_h1_col_norms_max: 6.00583934784 train_h1_col_norms_mean: 3.89137220383 train_h1_col_norms_min: 1.72638905048 train_h1_row_norms_max: 9.06593418121 train_h1_row_norms_mean: 5.53011369705 train_h1_row_norms_min: 3.28573846817 train_objective: 0.00130198767874 train_y_col_norms_max: 6.74868011475 train_y_col_norms_mean: 6.23671960831 train_y_col_norms_min: 5.50826644897 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999629914761 train_y_min_max_class: 0.966445803642 train_y_misclass: 0.000419999967562 train_y_nll: 0.00130198767874 train_y_row_norms_max: 1.86323726177 train_y_row_norms_mean: 0.585781753063 train_y_row_norms_min: 0.0242832992226 valid_h0_col_norms_max: 6.5425863266 valid_h0_col_norms_mean: 4.35914468765 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.87823629379 valid_h0_row_norms_mean: 3.41422724724 valid_h0_row_norms_min: 0.171178132296 valid_h1_col_norms_max: 6.00586032867 valid_h1_col_norms_mean: 3.89139056206 valid_h1_col_norms_min: 1.72638916969 valid_h1_row_norms_max: 9.06592273712 valid_h1_row_norms_mean: 5.53010177612 valid_h1_row_norms_min: 3.28573608398 valid_objective: 0.168345704675 valid_y_col_norms_max: 6.74868965149 valid_y_col_norms_mean: 6.23669672012 valid_y_col_norms_min: 5.50828027725 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995861887932 valid_y_min_max_class: 0.767153561115 valid_y_misclass: 0.0193999931216 valid_y_nll: 0.168345704675 valid_y_row_norms_max: 1.86322903633 valid_y_row_norms_mean: 0.585780024529 valid_y_row_norms_min: 0.0242832899094 Time this epoch: 3.283051 seconds Monitoring step: Epochs seen: 42 Batches seen: 21000 Examples seen: 2100000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.54826259613 test_h0_col_norms_mean: 4.36148118973 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.87971019745 test_h0_row_norms_mean: 3.41604399681 test_h0_row_norms_min: 0.171194016933 test_h1_col_norms_max: 6.00196123123 test_h1_col_norms_mean: 3.89196276665 test_h1_col_norms_min: 1.72636771202 test_h1_row_norms_max: 9.06931400299 test_h1_row_norms_mean: 5.53089809418 test_h1_row_norms_min: 3.28621292114 test_objective: 0.152915328741 test_y_col_norms_max: 6.76382827759 test_y_col_norms_mean: 6.25065279007 test_y_col_norms_min: 5.53469228745 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995481073856 test_y_min_max_class: 0.755885243416 test_y_misclass: 0.0184999946505 test_y_nll: 0.152915328741 test_y_row_norms_max: 1.87736725807 test_y_row_norms_mean: 0.586914539337 test_y_row_norms_min: 0.0246897321194 train_h0_col_norms_max: 6.54823541641 train_h0_col_norms_mean: 4.36147928238 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87974071503 train_h0_row_norms_mean: 3.4160592556 train_h0_row_norms_min: 0.171194061637 train_h1_col_norms_max: 6.00195074081 train_h1_col_norms_mean: 3.8919467926 train_h1_col_norms_min: 1.72637498379 train_h1_row_norms_max: 9.06928443909 train_h1_row_norms_mean: 5.53089904785 train_h1_row_norms_min: 3.28621530533 train_objective: 0.00141110678669 train_y_col_norms_max: 6.76386547089 train_y_col_norms_mean: 6.25062465668 train_y_col_norms_min: 5.5346660614 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999635100365 train_y_min_max_class: 0.967330634594 train_y_misclass: 0.000319999962812 train_y_nll: 0.00141110678669 train_y_row_norms_max: 1.87736737728 train_y_row_norms_mean: 0.58691483736 train_y_row_norms_min: 0.0246896371245 valid_h0_col_norms_max: 6.54826259613 valid_h0_col_norms_mean: 4.36148118973 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.87971019745 valid_h0_row_norms_mean: 3.41604399681 valid_h0_row_norms_min: 0.171194016933 valid_h1_col_norms_max: 6.00196123123 valid_h1_col_norms_mean: 3.89196276665 valid_h1_col_norms_min: 1.72636771202 valid_h1_row_norms_max: 9.06931400299 valid_h1_row_norms_mean: 5.53089809418 valid_h1_row_norms_min: 3.28621292114 valid_objective: 0.164742320776 valid_y_col_norms_max: 6.76382827759 valid_y_col_norms_mean: 6.25065279007 valid_y_col_norms_min: 5.53469228745 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996081888676 valid_y_min_max_class: 0.769794583321 valid_y_misclass: 0.0189999956638 valid_y_nll: 0.164742320776 valid_y_row_norms_max: 1.87736725807 valid_y_row_norms_mean: 0.586914539337 valid_y_row_norms_min: 0.0246897321194 Time this epoch: 3.293110 seconds Monitoring step: Epochs seen: 43 Batches seen: 21500 Examples seen: 2150000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.55259990692 test_h0_col_norms_mean: 4.36336374283 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.8740735054 test_h0_row_norms_mean: 3.41756176949 test_h0_row_norms_min: 0.17119500041 test_h1_col_norms_max: 6.0039639473 test_h1_col_norms_mean: 3.89240264893 test_h1_col_norms_min: 1.72636425495 test_h1_row_norms_max: 9.07901191711 test_h1_row_norms_mean: 5.53150510788 test_h1_row_norms_min: 3.28636312485 test_objective: 0.136897221208 test_y_col_norms_max: 6.77247095108 test_y_col_norms_mean: 6.26096439362 test_y_col_norms_min: 5.51252508163 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995941281319 test_y_min_max_class: 0.776527881622 test_y_misclass: 0.0178999938071 test_y_nll: 0.136897221208 test_y_row_norms_max: 1.87920343876 test_y_row_norms_mean: 0.587858736515 test_y_row_norms_min: 0.0247891973704 train_h0_col_norms_max: 6.55262708664 train_h0_col_norms_mean: 4.36338043213 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.8740811348 train_h0_row_norms_mean: 3.41757678986 train_h0_row_norms_min: 0.171194568276 train_h1_col_norms_max: 6.00393533707 train_h1_col_norms_mean: 3.89241600037 train_h1_col_norms_min: 1.72637200356 train_h1_row_norms_max: 9.07898330688 train_h1_row_norms_mean: 5.53153181076 train_h1_row_norms_min: 3.28636193275 train_objective: 0.00148291292135 train_y_col_norms_max: 6.77250146866 train_y_col_norms_mean: 6.26093387604 train_y_col_norms_min: 5.51249742508 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999666690826 train_y_min_max_class: 0.971082031727 train_y_misclass: 0.000460000039311 train_y_nll: 0.00148291292135 train_y_row_norms_max: 1.87921154499 train_y_row_norms_mean: 0.58786034584 train_y_row_norms_min: 0.0247890818864 valid_h0_col_norms_max: 6.55259990692 valid_h0_col_norms_mean: 4.36336374283 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.8740735054 valid_h0_row_norms_mean: 3.41756176949 valid_h0_row_norms_min: 0.17119500041 valid_h1_col_norms_max: 6.0039639473 valid_h1_col_norms_mean: 3.89240264893 valid_h1_col_norms_min: 1.72636425495 valid_h1_row_norms_max: 9.07901191711 valid_h1_row_norms_mean: 5.53150510788 valid_h1_row_norms_min: 3.28636312485 valid_objective: 0.161794766784 valid_y_col_norms_max: 6.77247095108 valid_y_col_norms_mean: 6.26096439362 valid_y_col_norms_min: 5.51252508163 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995917260647 valid_y_min_max_class: 0.753068387508 valid_y_misclass: 0.0201999936253 valid_y_nll: 0.161794766784 valid_y_row_norms_max: 1.87920343876 valid_y_row_norms_mean: 0.587858736515 valid_y_row_norms_min: 0.0247891973704 Time this epoch: 3.359274 seconds Monitoring step: Epochs seen: 44 Batches seen: 22000 Examples seen: 2200000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.55098342896 test_h0_col_norms_mean: 4.36544847488 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.87497997284 test_h0_row_norms_mean: 3.41930341721 test_h0_row_norms_min: 0.171195015311 test_h1_col_norms_max: 6.00462388992 test_h1_col_norms_mean: 3.89291667938 test_h1_col_norms_min: 1.72640001774 test_h1_row_norms_max: 9.07387065887 test_h1_row_norms_mean: 5.53226518631 test_h1_row_norms_min: 3.28615379333 test_objective: 0.140558704734 test_y_col_norms_max: 6.78662919998 test_y_col_norms_mean: 6.26840209961 test_y_col_norms_min: 5.52506113052 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996500074863 test_y_min_max_class: 0.794377505779 test_y_misclass: 0.0174999963492 test_y_nll: 0.140558704734 test_y_row_norms_max: 1.89163661003 test_y_row_norms_mean: 0.58866494894 test_y_row_norms_min: 0.0248291995376 train_h0_col_norms_max: 6.5510134697 train_h0_col_norms_mean: 4.36544466019 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87498617172 train_h0_row_norms_mean: 3.41928720474 train_h0_row_norms_min: 0.171194568276 train_h1_col_norms_max: 6.00459194183 train_h1_col_norms_mean: 3.89290046692 train_h1_col_norms_min: 1.72639226913 train_h1_row_norms_max: 9.07389450073 train_h1_row_norms_mean: 5.53226518631 train_h1_row_norms_min: 3.28615093231 train_objective: 0.000438805494923 train_y_col_norms_max: 6.7865986824 train_y_col_norms_mean: 6.2684264183 train_y_col_norms_min: 5.52509069443 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999836444855 train_y_min_max_class: 0.985349237919 train_y_misclass: 0.000159999981406 train_y_nll: 0.000438805494923 train_y_row_norms_max: 1.89162778854 train_y_row_norms_mean: 0.588665127754 train_y_row_norms_min: 0.0248291157186 valid_h0_col_norms_max: 6.55098342896 valid_h0_col_norms_mean: 4.36544847488 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.87497997284 valid_h0_row_norms_mean: 3.41930341721 valid_h0_row_norms_min: 0.171195015311 valid_h1_col_norms_max: 6.00462388992 valid_h1_col_norms_mean: 3.89291667938 valid_h1_col_norms_min: 1.72640001774 valid_h1_row_norms_max: 9.07387065887 valid_h1_row_norms_mean: 5.53226518631 valid_h1_row_norms_min: 3.28615379333 valid_objective: 0.157897502184 valid_y_col_norms_max: 6.78662919998 valid_y_col_norms_mean: 6.26840209961 valid_y_col_norms_min: 5.52506113052 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995646238327 valid_y_min_max_class: 0.742088675499 valid_y_misclass: 0.0179999954998 valid_y_nll: 0.157897502184 valid_y_row_norms_max: 1.89163661003 valid_y_row_norms_mean: 0.58866494894 valid_y_row_norms_min: 0.0248291995376 Time this epoch: 3.258919 seconds Monitoring step: Epochs seen: 45 Batches seen: 22500 Examples seen: 2250000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.55773639679 test_h0_col_norms_mean: 4.36757230759 test_h0_col_norms_min: 2.23605656624 test_h0_row_norms_max: 6.88162136078 test_h0_row_norms_mean: 3.42107534409 test_h0_row_norms_min: 0.171194553375 test_h1_col_norms_max: 6.00479459763 test_h1_col_norms_mean: 3.89360809326 test_h1_col_norms_min: 1.72638893127 test_h1_row_norms_max: 9.08829307556 test_h1_row_norms_mean: 5.53330039978 test_h1_row_norms_min: 3.28643465042 test_objective: 0.172753751278 test_y_col_norms_max: 6.79764652252 test_y_col_norms_mean: 6.28485965729 test_y_col_norms_min: 5.55204916 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.99615073204 test_y_min_max_class: 0.77610886097 test_y_misclass: 0.020499991253 test_y_nll: 0.172753751278 test_y_row_norms_max: 1.87029504776 test_y_row_norms_mean: 0.590070128441 test_y_row_norms_min: 0.0248381886631 train_h0_col_norms_max: 6.55770730972 train_h0_col_norms_mean: 4.36758470535 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.8816576004 train_h0_row_norms_mean: 3.4210703373 train_h0_row_norms_min: 0.171194195747 train_h1_col_norms_max: 6.00480556488 train_h1_col_norms_mean: 3.89361214638 train_h1_col_norms_min: 1.72638893127 train_h1_row_norms_max: 9.08830928802 train_h1_row_norms_mean: 5.53328752518 train_h1_row_norms_min: 3.28645133972 train_objective: 0.00231568375602 train_y_col_norms_max: 6.79761791229 train_y_col_norms_mean: 6.2848906517 train_y_col_norms_min: 5.55205202103 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999481022358 train_y_min_max_class: 0.955595433712 train_y_misclass: 0.000539999979082 train_y_nll: 0.00231568375602 train_y_row_norms_max: 1.87028670311 train_y_row_norms_mean: 0.590068638325 train_y_row_norms_min: 0.0248383041471 valid_h0_col_norms_max: 6.55773639679 valid_h0_col_norms_mean: 4.36757230759 valid_h0_col_norms_min: 2.23605656624 valid_h0_row_norms_max: 6.88162136078 valid_h0_row_norms_mean: 3.42107534409 valid_h0_row_norms_min: 0.171194553375 valid_h1_col_norms_max: 6.00479459763 valid_h1_col_norms_mean: 3.89360809326 valid_h1_col_norms_min: 1.72638893127 valid_h1_row_norms_max: 9.08829307556 valid_h1_row_norms_mean: 5.53330039978 valid_h1_row_norms_min: 3.28643465042 valid_objective: 0.17547737062 valid_y_col_norms_max: 6.79764652252 valid_y_col_norms_mean: 6.28485965729 valid_y_col_norms_min: 5.55204916 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995465695858 valid_y_min_max_class: 0.7330275774 valid_y_misclass: 0.020799998194 valid_y_nll: 0.17547737062 valid_y_row_norms_max: 1.87029504776 valid_y_row_norms_mean: 0.590070128441 valid_y_row_norms_min: 0.0248381886631 Time this epoch: 3.263050 seconds Monitoring step: Epochs seen: 46 Batches seen: 23000 Examples seen: 2300000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.59422492981 test_h0_col_norms_mean: 4.37052488327 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.8726644516 test_h0_row_norms_mean: 3.42359375954 test_h0_row_norms_min: 0.171194955707 test_h1_col_norms_max: 6.00406217575 test_h1_col_norms_mean: 3.89443039894 test_h1_col_norms_min: 1.72642493248 test_h1_row_norms_max: 9.08111953735 test_h1_row_norms_mean: 5.53444480896 test_h1_row_norms_min: 3.28672647476 test_objective: 0.176214575768 test_y_col_norms_max: 6.79807567596 test_y_col_norms_mean: 6.29843473434 test_y_col_norms_min: 5.56106996536 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996127128601 test_y_min_max_class: 0.767001569271 test_y_misclass: 0.0188999976963 test_y_nll: 0.176214575768 test_y_row_norms_max: 1.88375401497 test_y_row_norms_mean: 0.591480791569 test_y_row_norms_min: 0.0244950912893 train_h0_col_norms_max: 6.59419536591 train_h0_col_norms_mean: 4.37053442001 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87265539169 train_h0_row_norms_mean: 3.42357826233 train_h0_row_norms_min: 0.171194553375 train_h1_col_norms_max: 6.00408983231 train_h1_col_norms_mean: 3.89444637299 train_h1_col_norms_min: 1.72643446922 train_h1_row_norms_max: 9.08109664917 train_h1_row_norms_mean: 5.53442716599 train_h1_row_norms_min: 3.28672146797 train_objective: 0.00163910887204 train_y_col_norms_max: 6.79804325104 train_y_col_norms_mean: 6.29846715927 train_y_col_norms_min: 5.56109666824 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999451816082 train_y_min_max_class: 0.952225148678 train_y_misclass: 0.00061999988975 train_y_nll: 0.00163910887204 train_y_row_norms_max: 1.88374614716 train_y_row_norms_mean: 0.591483712196 train_y_row_norms_min: 0.0244949962944 valid_h0_col_norms_max: 6.59422492981 valid_h0_col_norms_mean: 4.37052488327 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.8726644516 valid_h0_row_norms_mean: 3.42359375954 valid_h0_row_norms_min: 0.171194955707 valid_h1_col_norms_max: 6.00406217575 valid_h1_col_norms_mean: 3.89443039894 valid_h1_col_norms_min: 1.72642493248 valid_h1_row_norms_max: 9.08111953735 valid_h1_row_norms_mean: 5.53444480896 valid_h1_row_norms_min: 3.28672647476 valid_objective: 0.186354964972 valid_y_col_norms_max: 6.79807567596 valid_y_col_norms_mean: 6.29843473434 valid_y_col_norms_min: 5.56106996536 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.99598556757 valid_y_min_max_class: 0.759403705597 valid_y_misclass: 0.0206999909133 valid_y_nll: 0.186354964972 valid_y_row_norms_max: 1.88375401497 valid_y_row_norms_mean: 0.591480791569 valid_y_row_norms_min: 0.0244950912893 Time this epoch: 3.263870 seconds Monitoring step: Epochs seen: 47 Batches seen: 23500 Examples seen: 2350000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.61253595352 test_h0_col_norms_mean: 4.37271356583 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.87648153305 test_h0_row_norms_mean: 3.425365448 test_h0_row_norms_min: 0.17119538784 test_h1_col_norms_max: 6.00418663025 test_h1_col_norms_mean: 3.8950676918 test_h1_col_norms_min: 1.72642803192 test_h1_row_norms_max: 9.10107326508 test_h1_row_norms_mean: 5.53528594971 test_h1_row_norms_min: 3.28674340248 test_objective: 0.160995185375 test_y_col_norms_max: 6.79669380188 test_y_col_norms_mean: 6.31103897095 test_y_col_norms_min: 5.58734273911 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995669007301 test_y_min_max_class: 0.77006238699 test_y_misclass: 0.0187999941409 test_y_nll: 0.160995185375 test_y_row_norms_max: 1.89035248756 test_y_row_norms_mean: 0.592762053013 test_y_row_norms_min: 0.0258602239192 train_h0_col_norms_max: 6.61253833771 train_h0_col_norms_mean: 4.37270545959 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.87647247314 train_h0_row_norms_mean: 3.42536139488 train_h0_row_norms_min: 0.171194672585 train_h1_col_norms_max: 6.00415945053 train_h1_col_norms_mean: 3.89505052567 train_h1_col_norms_min: 1.72643530369 train_h1_row_norms_max: 9.10108375549 train_h1_row_norms_mean: 5.53529548645 train_h1_row_norms_min: 3.28672790527 train_objective: 0.0021845579613 train_y_col_norms_max: 6.79666471481 train_y_col_norms_mean: 6.31101417542 train_y_col_norms_min: 5.58734083176 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999446511269 train_y_min_max_class: 0.954655766487 train_y_misclass: 0.000679999880958 train_y_nll: 0.0021845579613 train_y_row_norms_max: 1.89036035538 train_y_row_norms_mean: 0.59276509285 train_y_row_norms_min: 0.0258601009846 valid_h0_col_norms_max: 6.61253595352 valid_h0_col_norms_mean: 4.37271356583 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.87648153305 valid_h0_row_norms_mean: 3.425365448 valid_h0_row_norms_min: 0.17119538784 valid_h1_col_norms_max: 6.00418663025 valid_h1_col_norms_mean: 3.8950676918 valid_h1_col_norms_min: 1.72642803192 valid_h1_row_norms_max: 9.10107326508 valid_h1_row_norms_mean: 5.53528594971 valid_h1_row_norms_min: 3.28674340248 valid_objective: 0.158408492804 valid_y_col_norms_max: 6.79669380188 valid_y_col_norms_mean: 6.31103897095 valid_y_col_norms_min: 5.58734273911 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995150506496 valid_y_min_max_class: 0.737409770489 valid_y_misclass: 0.0196999944746 valid_y_nll: 0.158408492804 valid_y_row_norms_max: 1.89035248756 valid_y_row_norms_mean: 0.592762053013 valid_y_row_norms_min: 0.0258602239192 Time this epoch: 3.246123 seconds Monitoring step: Epochs seen: 48 Batches seen: 24000 Examples seen: 2400000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.60942840576 test_h0_col_norms_mean: 4.37416505814 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.88673448563 test_h0_row_norms_mean: 3.42650437355 test_h0_row_norms_min: 0.171197414398 test_h1_col_norms_max: 6.00638771057 test_h1_col_norms_mean: 3.89544963837 test_h1_col_norms_min: 1.72642791271 test_h1_row_norms_max: 9.09871959686 test_h1_row_norms_mean: 5.53591918945 test_h1_row_norms_min: 3.2867603302 test_objective: 0.175691723824 test_y_col_norms_max: 6.78237819672 test_y_col_norms_mean: 6.3183298111 test_y_col_norms_min: 5.59972047806 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996128737926 test_y_min_max_class: 0.775415062904 test_y_misclass: 0.0199999921024 test_y_nll: 0.175691723824 test_y_row_norms_max: 1.88972866535 test_y_row_norms_mean: 0.593335032463 test_y_row_norms_min: 0.0257790517062 train_h0_col_norms_max: 6.60943603516 train_h0_col_norms_mean: 4.37415838242 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.88672494888 train_h0_row_norms_mean: 3.4265191555 train_h0_row_norms_min: 0.171197369695 train_h1_col_norms_max: 6.00641536713 train_h1_col_norms_mean: 3.8954308033 train_h1_col_norms_min: 1.72643482685 train_h1_row_norms_max: 9.09872722626 train_h1_row_norms_mean: 5.53590679169 train_h1_row_norms_min: 3.28674817085 train_objective: 0.000762883864809 train_y_col_norms_max: 6.78235006332 train_y_col_norms_mean: 6.31833028793 train_y_col_norms_min: 5.59973287582 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.99974834919 train_y_min_max_class: 0.978080093861 train_y_misclass: 0.000239999950281 train_y_nll: 0.000762883864809 train_y_row_norms_max: 1.88971841335 train_y_row_norms_mean: 0.593334615231 train_y_row_norms_min: 0.0257790144533 valid_h0_col_norms_max: 6.60942840576 valid_h0_col_norms_mean: 4.37416505814 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.88673448563 valid_h0_row_norms_mean: 3.42650437355 valid_h0_row_norms_min: 0.171197414398 valid_h1_col_norms_max: 6.00638771057 valid_h1_col_norms_mean: 3.89544963837 valid_h1_col_norms_min: 1.72642791271 valid_h1_row_norms_max: 9.09871959686 valid_h1_row_norms_mean: 5.53591918945 valid_h1_row_norms_min: 3.2867603302 valid_objective: 0.178655579686 valid_y_col_norms_max: 6.78237819672 valid_y_col_norms_mean: 6.3183298111 valid_y_col_norms_min: 5.59972047806 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996117174625 valid_y_min_max_class: 0.773514211178 valid_y_misclass: 0.0190999954939 valid_y_nll: 0.178655579686 valid_y_row_norms_max: 1.88972866535 valid_y_row_norms_mean: 0.593335032463 valid_y_row_norms_min: 0.0257790517062 Time this epoch: 3.274107 seconds Monitoring step: Epochs seen: 49 Batches seen: 24500 Examples seen: 2450000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.61158514023 test_h0_col_norms_mean: 4.37596178055 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.89095163345 test_h0_row_norms_mean: 3.42802858353 test_h0_row_norms_min: 0.171208888292 test_h1_col_norms_max: 6.00729322433 test_h1_col_norms_mean: 3.89590859413 test_h1_col_norms_min: 1.72634100914 test_h1_row_norms_max: 9.11584568024 test_h1_row_norms_mean: 5.53655290604 test_h1_row_norms_min: 3.28674292564 test_objective: 0.173922881484 test_y_col_norms_max: 6.80417919159 test_y_col_norms_mean: 6.32538461685 test_y_col_norms_min: 5.59382343292 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.995864152908 test_y_min_max_class: 0.774653494358 test_y_misclass: 0.0195999965072 test_y_nll: 0.173922881484 test_y_row_norms_max: 1.90011572838 test_y_row_norms_mean: 0.593958258629 test_y_row_norms_min: 0.0259114392102 train_h0_col_norms_max: 6.61158514023 train_h0_col_norms_mean: 4.37594175339 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.89095973969 train_h0_row_norms_mean: 3.42801046371 train_h0_row_norms_min: 0.171208947897 train_h1_col_norms_max: 6.00726985931 train_h1_col_norms_mean: 3.89590215683 train_h1_col_norms_min: 1.7263327837 train_h1_row_norms_max: 9.11585617065 train_h1_row_norms_mean: 5.53655576706 train_h1_row_norms_min: 3.28672790527 train_objective: 0.00186892366037 train_y_col_norms_max: 6.80421066284 train_y_col_norms_mean: 6.32541131973 train_y_col_norms_min: 5.59379386902 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999438583851 train_y_min_max_class: 0.950516223907 train_y_misclass: 0.000539999979082 train_y_nll: 0.00186892366037 train_y_row_norms_max: 1.90012443066 train_y_row_norms_mean: 0.59395968914 train_y_row_norms_min: 0.025911314413 valid_h0_col_norms_max: 6.61158514023 valid_h0_col_norms_mean: 4.37596178055 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.89095163345 valid_h0_row_norms_mean: 3.42802858353 valid_h0_row_norms_min: 0.171208888292 valid_h1_col_norms_max: 6.00729322433 valid_h1_col_norms_mean: 3.89590859413 valid_h1_col_norms_min: 1.72634100914 valid_h1_row_norms_max: 9.11584568024 valid_h1_row_norms_mean: 5.53655290604 valid_h1_row_norms_min: 3.28674292564 valid_objective: 0.167324125767 valid_y_col_norms_max: 6.80417919159 valid_y_col_norms_mean: 6.32538461685 valid_y_col_norms_min: 5.59382343292 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.995694279671 valid_y_min_max_class: 0.751230061054 valid_y_misclass: 0.0211999900639 valid_y_nll: 0.167324125767 valid_y_row_norms_max: 1.90011572838 valid_y_row_norms_mean: 0.593958258629 valid_y_row_norms_min: 0.0259114392102 Time this epoch: 3.276921 seconds Monitoring step: Epochs seen: 50 Batches seen: 25000 Examples seen: 2500000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.61482906342 test_h0_col_norms_mean: 4.37731599808 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.90527057648 test_h0_row_norms_mean: 3.42910242081 test_h0_row_norms_min: 0.171211406589 test_h1_col_norms_max: 6.01254796982 test_h1_col_norms_mean: 3.89633321762 test_h1_col_norms_min: 1.72635316849 test_h1_row_norms_max: 9.1068277359 test_h1_row_norms_mean: 5.53717756271 test_h1_row_norms_min: 3.28689336777 test_objective: 0.178737580776 test_y_col_norms_max: 6.81128787994 test_y_col_norms_mean: 6.33507156372 test_y_col_norms_min: 5.58309650421 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.99630522728 test_y_min_max_class: 0.788845181465 test_y_misclass: 0.0197999924421 test_y_nll: 0.178737580776 test_y_row_norms_max: 1.93474268913 test_y_row_norms_mean: 0.594771564007 test_y_row_norms_min: 0.0260054916143 train_h0_col_norms_max: 6.6148557663 train_h0_col_norms_mean: 4.37733840942 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.90530347824 train_h0_row_norms_mean: 3.42908787727 train_h0_row_norms_min: 0.171211794019 train_h1_col_norms_max: 6.01251840591 train_h1_col_norms_mean: 3.89634943008 train_h1_col_norms_min: 1.72634553909 train_h1_row_norms_max: 9.10683345795 train_h1_row_norms_mean: 5.53719091415 train_h1_row_norms_min: 3.28687477112 train_objective: 0.00155572697986 train_y_col_norms_max: 6.81132364273 train_y_col_norms_mean: 6.33503913879 train_y_col_norms_min: 5.58306837082 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999542534351 train_y_min_max_class: 0.959059894085 train_y_misclass: 0.000539999920875 train_y_nll: 0.00155572697986 train_y_row_norms_max: 1.93475210667 train_y_row_norms_mean: 0.594768106937 train_y_row_norms_min: 0.0260053742677 valid_h0_col_norms_max: 6.61482906342 valid_h0_col_norms_mean: 4.37731599808 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.90527057648 valid_h0_row_norms_mean: 3.42910242081 valid_h0_row_norms_min: 0.171211406589 valid_h1_col_norms_max: 6.01254796982 valid_h1_col_norms_mean: 3.89633321762 valid_h1_col_norms_min: 1.72635316849 valid_h1_row_norms_max: 9.1068277359 valid_h1_row_norms_mean: 5.53717756271 valid_h1_row_norms_min: 3.28689336777 valid_objective: 0.168507456779 valid_y_col_norms_max: 6.81128787994 valid_y_col_norms_mean: 6.33507156372 valid_y_col_norms_min: 5.58309650421 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996147751808 valid_y_min_max_class: 0.789148688316 valid_y_misclass: 0.0197999924421 valid_y_nll: 0.168507456779 valid_y_row_norms_max: 1.93474268913 valid_y_row_norms_mean: 0.594771564007 valid_y_row_norms_min: 0.0260054916143 Time this epoch: 3.200028 seconds Monitoring step: Epochs seen: 51 Batches seen: 25500 Examples seen: 2550000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.6147813797 test_h0_col_norms_mean: 4.37828779221 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.90187358856 test_h0_row_norms_mean: 3.42986750603 test_h0_row_norms_min: 0.171211406589 test_h1_col_norms_max: 6.0115852356 test_h1_col_norms_mean: 3.89659976959 test_h1_col_norms_min: 1.72635293007 test_h1_row_norms_max: 9.1057882309 test_h1_row_norms_mean: 5.53757143021 test_h1_row_norms_min: 3.28781795502 test_objective: 0.172010108829 test_y_col_norms_max: 6.82110786438 test_y_col_norms_mean: 6.33999061584 test_y_col_norms_min: 5.59692811966 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996686458588 test_y_min_max_class: 0.808368086815 test_y_misclass: 0.0198999941349 test_y_nll: 0.172010108829 test_y_row_norms_max: 1.93597054482 test_y_row_norms_mean: 0.595229923725 test_y_row_norms_min: 0.0260351337492 train_h0_col_norms_max: 6.61475372314 train_h0_col_norms_mean: 4.37829828262 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.9019112587 train_h0_row_norms_mean: 3.42988300323 train_h0_row_norms_min: 0.171211794019 train_h1_col_norms_max: 6.01156425476 train_h1_col_norms_mean: 3.89659571648 train_h1_col_norms_min: 1.72634553909 train_h1_row_norms_max: 9.10573482513 train_h1_row_norms_mean: 5.53757476807 train_h1_row_norms_min: 3.28780126572 train_objective: 0.00100987718906 train_y_col_norms_max: 6.8211388588 train_y_col_norms_mean: 6.34001255035 train_y_col_norms_min: 5.59689760208 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999795496464 train_y_min_max_class: 0.98121035099 train_y_misclass: 0.000219999958063 train_y_nll: 0.00100987718906 train_y_row_norms_max: 1.93596208096 train_y_row_norms_mean: 0.59522998333 train_y_row_norms_min: 0.0260351262987 valid_h0_col_norms_max: 6.6147813797 valid_h0_col_norms_mean: 4.37828779221 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.90187358856 valid_h0_row_norms_mean: 3.42986750603 valid_h0_row_norms_min: 0.171211406589 valid_h1_col_norms_max: 6.0115852356 valid_h1_col_norms_mean: 3.89659976959 valid_h1_col_norms_min: 1.72635293007 valid_h1_row_norms_max: 9.1057882309 valid_h1_row_norms_mean: 5.53757143021 valid_h1_row_norms_min: 3.28781795502 valid_objective: 0.175070494413 valid_y_col_norms_max: 6.82110786438 valid_y_col_norms_mean: 6.33999061584 valid_y_col_norms_min: 5.59692811966 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996282696724 valid_y_min_max_class: 0.780044913292 valid_y_misclass: 0.0196999944746 valid_y_nll: 0.175070494413 valid_y_row_norms_max: 1.93597054482 valid_y_row_norms_mean: 0.595229923725 valid_y_row_norms_min: 0.0260351337492 Time this epoch: 3.239240 seconds Monitoring step: Epochs seen: 52 Batches seen: 26000 Examples seen: 2600000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.61600589752 test_h0_col_norms_mean: 4.37898588181 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.90484142303 test_h0_row_norms_mean: 3.43044447899 test_h0_row_norms_min: 0.171211466193 test_h1_col_norms_max: 6.01123189926 test_h1_col_norms_mean: 3.8968091011 test_h1_col_norms_min: 1.72635233402 test_h1_row_norms_max: 9.11070537567 test_h1_row_norms_mean: 5.53788709641 test_h1_row_norms_min: 3.28822088242 test_objective: 0.160425424576 test_y_col_norms_max: 6.80656099319 test_y_col_norms_mean: 6.34523868561 test_y_col_norms_min: 5.59447908401 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996790707111 test_y_min_max_class: 0.826726913452 test_y_misclass: 0.0184999965131 test_y_nll: 0.160425424576 test_y_row_norms_max: 1.94036662579 test_y_row_norms_mean: 0.595700562 test_y_row_norms_min: 0.0263105537742 train_h0_col_norms_max: 6.61604356766 train_h0_col_norms_mean: 4.37900781631 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.9048409462 train_h0_row_norms_mean: 3.43045806885 train_h0_row_norms_min: 0.171212136745 train_h1_col_norms_max: 6.01124334335 train_h1_col_norms_mean: 3.89682626724 train_h1_col_norms_min: 1.72634339333 train_h1_row_norms_max: 9.11072158813 train_h1_row_norms_mean: 5.53790187836 train_h1_row_norms_min: 3.28823471069 train_objective: 0.000158851937158 train_y_col_norms_max: 6.8065943718 train_y_col_norms_mean: 6.34526729584 train_y_col_norms_min: 5.59449052811 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999920845032 train_y_min_max_class: 0.992827177048 train_y_misclass: 7.9999997979e-05 train_y_nll: 0.000158851937158 train_y_row_norms_max: 1.94036877155 train_y_row_norms_mean: 0.595700562 train_y_row_norms_min: 0.0263105835766 valid_h0_col_norms_max: 6.61600589752 valid_h0_col_norms_mean: 4.37898588181 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.90484142303 valid_h0_row_norms_mean: 3.43044447899 valid_h0_row_norms_min: 0.171211466193 valid_h1_col_norms_max: 6.01123189926 valid_h1_col_norms_mean: 3.8968091011 valid_h1_col_norms_min: 1.72635233402 valid_h1_row_norms_max: 9.11070537567 valid_h1_row_norms_mean: 5.53788709641 valid_h1_row_norms_min: 3.28822088242 valid_objective: 0.169489264488 valid_y_col_norms_max: 6.80656099319 valid_y_col_norms_mean: 6.34523868561 valid_y_col_norms_min: 5.59447908401 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996821761131 valid_y_min_max_class: 0.813496589661 valid_y_misclass: 0.0197999961674 valid_y_nll: 0.169489264488 valid_y_row_norms_max: 1.94036662579 valid_y_row_norms_mean: 0.595700562 valid_y_row_norms_min: 0.0263105537742 Time this epoch: 3.259741 seconds Monitoring step: Epochs seen: 53 Batches seen: 26500 Examples seen: 2650000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.6160197258 test_h0_col_norms_mean: 4.3792681694 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.90263414383 test_h0_row_norms_mean: 3.43065404892 test_h0_row_norms_min: 0.171211466193 test_h1_col_norms_max: 6.01123523712 test_h1_col_norms_mean: 3.89689803123 test_h1_col_norms_min: 1.72635245323 test_h1_row_norms_max: 9.11182498932 test_h1_row_norms_mean: 5.53798723221 test_h1_row_norms_min: 3.2883348465 test_objective: 0.151598215103 test_y_col_norms_max: 6.82387685776 test_y_col_norms_mean: 6.34804153442 test_y_col_norms_min: 5.58777189255 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.99653673172 test_y_min_max_class: 0.809294104576 test_y_misclass: 0.0181999951601 test_y_nll: 0.151598215103 test_y_row_norms_max: 1.94308698177 test_y_row_norms_mean: 0.595957636833 test_y_row_norms_min: 0.0262990482152 train_h0_col_norms_max: 6.61604738235 train_h0_col_norms_mean: 4.3792719841 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.9026427269 train_h0_row_norms_mean: 3.43063807487 train_h0_row_norms_min: 0.171212136745 train_h1_col_norms_max: 6.01124382019 train_h1_col_norms_mean: 3.89692115784 train_h1_col_norms_min: 1.72634339333 train_h1_row_norms_max: 9.11186790466 train_h1_row_norms_mean: 5.53798723221 train_h1_row_norms_min: 3.28835225105 train_objective: 0.000210020065424 train_y_col_norms_max: 6.82384681702 train_y_col_norms_mean: 6.348072052 train_y_col_norms_min: 5.58779907227 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999953627586 train_y_min_max_class: 0.996000170708 train_y_misclass: 5.99999984843e-05 train_y_nll: 0.000210020065424 train_y_row_norms_max: 1.94309675694 train_y_row_norms_mean: 0.595957219601 train_y_row_norms_min: 0.0262989122421 valid_h0_col_norms_max: 6.6160197258 valid_h0_col_norms_mean: 4.3792681694 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.90263414383 valid_h0_row_norms_mean: 3.43065404892 valid_h0_row_norms_min: 0.171211466193 valid_h1_col_norms_max: 6.01123523712 valid_h1_col_norms_mean: 3.89689803123 valid_h1_col_norms_min: 1.72635245323 valid_h1_row_norms_max: 9.11182498932 valid_h1_row_norms_mean: 5.53798723221 valid_h1_row_norms_min: 3.2883348465 valid_objective: 0.163225889206 valid_y_col_norms_max: 6.82387685776 valid_y_col_norms_mean: 6.34804153442 valid_y_col_norms_min: 5.58777189255 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.996884763241 valid_y_min_max_class: 0.818299531937 valid_y_misclass: 0.0186999943107 valid_y_nll: 0.163225889206 valid_y_row_norms_max: 1.94308698177 valid_y_row_norms_mean: 0.595957636833 valid_y_row_norms_min: 0.0262990482152 Time this epoch: 3.224364 seconds Monitoring step: Epochs seen: 54 Batches seen: 27000 Examples seen: 2700000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 6.61593580246 test_h0_col_norms_mean: 4.37931966782 test_h0_col_norms_min: 2.23605632782 test_h0_row_norms_max: 6.90164899826 test_h0_row_norms_mean: 3.43070554733 test_h0_row_norms_min: 0.171211466193 test_h1_col_norms_max: 6.01123189926 test_h1_col_norms_mean: 3.89690995216 test_h1_col_norms_min: 1.72635293007 test_h1_row_norms_max: 9.10886287689 test_h1_row_norms_mean: 5.53801727295 test_h1_row_norms_min: 3.28835773468 test_objective: 0.156616300344 test_y_col_norms_max: 6.82711935043 test_y_col_norms_mean: 6.34913825989 test_y_col_norms_min: 5.58710432053 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.996679246426 test_y_min_max_class: 0.821564376354 test_y_misclass: 0.0183999948204 test_y_nll: 0.156616300344 test_y_row_norms_max: 1.94326412678 test_y_row_norms_mean: 0.596061944962 test_y_row_norms_min: 0.0262778773904 train_h0_col_norms_max: 6.61593151093 train_h0_col_norms_mean: 4.37929821014 train_h0_col_norms_min: 2.23605895042 train_h0_row_norms_max: 6.90168523788 train_h0_row_norms_mean: 3.43071746826 train_h0_row_norms_min: 0.171212136745 train_h1_col_norms_max: 6.01124286652 train_h1_col_norms_mean: 3.89692831039 train_h1_col_norms_min: 1.72634553909 train_h1_row_norms_max: 9.10885238647 train_h1_row_norms_mean: 5.53799200058 train_h1_row_norms_min: 3.28836083412 train_objective: 1.18671387099e-05 train_y_col_norms_max: 6.82711696625 train_y_col_norms_mean: 6.34910583496 train_y_col_norms_min: 5.5871014595 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.999985456467 train_y_min_max_class: 0.999015629292 train_y_misclass: 0.0 train_y_nll: 1.18671387099e-05 train_y_row_norms_max: 1.94327509403 train_y_row_norms_mean: 0.596063792706 train_y_row_norms_min: 0.0262779761106 valid_h0_col_norms_max: 6.61593580246 valid_h0_col_norms_mean: 4.37931966782 valid_h0_col_norms_min: 2.23605632782 valid_h0_row_norms_max: 6.90164899826 valid_h0_row_norms_mean: 3.43070554733 valid_h0_row_norms_min: 0.171211466193 valid_h1_col_norms_max: 6.01123189926 valid_h1_col_norms_mean: 3.89690995216 valid_h1_col_norms_min: 1.72635293007 valid_h1_row_norms_max: 9.10886287689 valid_h1_row_norms_mean: 5.53801727295 valid_h1_row_norms_min: 3.28835773468 valid_objective: 0.168142601848 valid_y_col_norms_max: 6.82711935043 valid_y_col_norms_mean: 6.34913825989 valid_y_col_norms_min: 5.58710432053 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.997302770615 valid_y_min_max_class: 0.829375386238 valid_y_misclass: 0.0184999927878 valid_y_nll: 0.168142601848 valid_y_row_norms_max: 1.94326412678 valid_y_row_norms_mean: 0.596061944962 valid_y_row_norms_min: 0.0262778773904
!print_monitor.py mlp_2_best.pkl | grep test_y_misclass
Using gpu device 2: GeForce GTX 285 /u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit. warnings.warn("MLP changing the recursion limit.") test_y_misclass : 0.0174999963492
Using the deeper architecture, rectifier units, and SGD brought the test error rate down from 1.94% to 1.75%.
In softmax_regression.ipynb, we discussed the problem of overfitting, and how early stopping guided by validation set performance can result in better test set performance. Another way to prevent overfitting is to explicitly change the cost function to discourage overfitting.
The best way to prevent overfitting is to use Bayesian inference to predict labels on the new data. Suppose we have been given a dataset $\mathcal{D}$, and we want to classify a new point $x'$. Call its uknown label $y'$. Suppose that we also have a probability distribution over all possible model parameters, and that we call the set of all parameters $\theta$. Then
$$p(y' \mid x', \mathcal{D} ) = \int p(y', \theta \mid x', \mathcal{D}) d \theta $$$$ = \int p( y' \mid x' , \theta ) p( \theta \mid \mathcal{D} ) d \theta $$$$ \propto \int p( y' \mid x' , \theta ) p( \mathcal{D} \mid \theta ) p(\theta) d \theta $$(On the last line, we only worry about computing the distribution over $y'$ up to a constant, because we can easily find this constant by summing over the $k$ possible values of $y'$)
In other words, the right thing to do is to have all of the infinitely many possible values of $\theta$ vote on how to classify $x'$, with each value of $\theta$'s vote weighted by $p(\theta) p(\mathcal{D} \mid \theta)$.
Unfortunately, while conceptually straight forward, there is not an obvious way to evaluate this integral for a large multilayer perceptron. Instead, we assume that the distribution $p(\theta) p(\mathcal{D} \mid \theta)$ is very peaked, so that we can get a good prediction by using the single most likely value of $\theta$.
This suggests that we should maximize $p(\theta) p(\mathcal{D} \mid \theta)$, rather than maximizing $p(\mathcal{D} \mid \theta)$ as we have so far. Note that in log space, this is $\log p(\theta) + \log p( \mathcal{D} \mid \theta)$. We can thus add regularization to our training procedure by adding a term for $\log p(\theta)$ to our objective function.
This is very easy to do in pylearn2 using the SumOfCosts class. The following YAML string sets up the same experiment as before, but using SumOfCosts to add a regularization term. Before, we did not specify the "cost" argument to the training algorithm. The model provided the training algorithm with a default cost. Now, we specify that the cost should be the sum of two different costs. The first is the Default cost, which just asks the output layer what cost to use. This is the same cost we have implicitly been using all along, because models.mlp.MLP.get_default_cost() returns costs.mlp.Default(). The second term of our new cost function is called WeightDecay, and it implements a prior on our model parameters $\theta$.
import os
import pylearn2
path = os.path.join(pylearn2.__path__[0], 'scripts', 'tutorials', 'multilayer_perceptron', 'mlp_tutorial_part_4.yaml')
with open(path, 'r') as f:
train_3 = f.read()
hyper_params = {'train_stop' : 50000,
'valid_stop' : 60000,
'dim_h0' : 500,
'dim_h1' : 1000,
'sparse_init_h1' : 15,
'max_epochs' : 10000,
'save_path' : '.'}
train_3 = train_3 % (hyper_params)
print train_3
!obj:pylearn2.train.Train { dataset: &train !obj:pylearn2.datasets.mnist.MNIST { which_set: 'train', start: 0, stop: 50000 }, model: !obj:pylearn2.models.mlp.MLP { layers: [ !obj:pylearn2.models.mlp.RectifiedLinear { layer_name: 'h0', dim: 500, sparse_init: 15 }, !obj:pylearn2.models.mlp.RectifiedLinear { layer_name: 'h1', dim: 500, sparse_init: 15 }, !obj:pylearn2.models.mlp.Softmax { layer_name: 'y', n_classes: 10, irange: 0. } ], nvis: 784, }, algorithm: !obj:pylearn2.training_algorithms.sgd.SGD { batch_size: 100, learning_rate: .01, monitoring_dataset: { 'train' : *train, 'valid' : !obj:pylearn2.datasets.mnist.MNIST { which_set: 'train', start: 50000, stop: 60000 }, 'test' : !obj:pylearn2.datasets.mnist.MNIST { which_set: 'test', } }, cost: !obj:pylearn2.costs.cost.SumOfCosts { costs: [ !obj:pylearn2.costs.mlp.Default { }, !obj:pylearn2.costs.mlp.WeightDecay { coeffs: [ .00005, .00005, .00005 ] } ] }, learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum { init_momentum: .5 }, termination_criterion: !obj:pylearn2.termination_criteria.And { criteria: [ !obj:pylearn2.termination_criteria.MonitorBased { channel_name: "valid_y_misclass", prop_decrease: 0., N: 10 }, !obj:pylearn2.termination_criteria.EpochCounter { max_epochs: 10000 } ] } }, extensions: [ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest { channel_name: 'valid_y_misclass', save_path: "mlp_3_best.pkl" }, !obj:pylearn2.training_algorithms.learning_rule.MomentumAdjustor { start: 1, saturate: 10, final_momentum: .99 } ] }
The WeightDecay class adds a cost based on the sum of the squares of the elements of $W$ for the different layers, multiplying each by a different coefficient. This corresponds to $p(\theta)$ being Gaussian distribution on $W$, with a diagonal covariance matrix. (We don't regularize $b$, which is a bit of a hack, but can be thought of as putting extremely high variance on $b$ in the prior) In other words, our prior belief about $\theta$ is that the weights should be small. This basically says that, all else being equal, the different units in our network shouldn't interact with each other. Compared to the unregularized network, a network trained with weight decay wants to see more evidence that two units should interact before it allows them to do so.
Note that the SumOfCosts class doesn't explicitly have anything to do with the MLP. There is no requirement that the cost function be closely tied to the code for a particular model in pylearn2. This gives you great flexibility in the kind of experiments pylearn2 can run. The SumOfCosts class allows you to combine several pre-existing building blocks in pylearn2. By implementing your own cost classes, you can get even greater flexibility.
Of course, some costs are tightly integrated with a specific kind of model. The costs.mlp.Default cost expects to be able to ask a model for its last layer, and ask that layer what kind of cost to apply to the target values $y$ and an estimate of them produced by calling the model's fprop method. This implies that the cost can really only be used with MLP subclasses. Likewise, the WeightDecay cost depends on the assumption that the model is organized into layers and each layer has a single weight matrix. This means that it can only be used with an MLP,and even then only with layers that are governed by a weight matrix. It's OK to make a Cost that is this tightly integrated with a specific kind of model. Doing so is inevitable. Usually in pylearn2 we put the costs for a specific model family in their own submodule of pylearn2 so it's easy to tell what models they can be used with.
We now show what happens when you train the regularized MLP:
from pylearn2.config import yaml_parse
train_3 = yaml_parse.load(train_3)
train_3.main_loop()
Parameter and initial learning rate summary: h0_W: 0.00999999977648 h0_b: 0.00999999977648 h1_W: 0.00999999977648 h1_b: 0.00999999977648 softmax_b: 0.00999999977648 softmax_W: 0.00999999977648 Compiling sgd_update... Compiling sgd_update done. Time elapsed: 2.973035 seconds compiling begin_record_entry... compiling begin_record_entry done. Time elapsed: 0.457965 seconds Monitored channels: learning_rate momentum test_h0_col_norms_max test_h0_col_norms_mean test_h0_col_norms_min test_h0_row_norms_max test_h0_row_norms_mean test_h0_row_norms_min test_h1_col_norms_max test_h1_col_norms_mean test_h1_col_norms_min test_h1_row_norms_max test_h1_row_norms_mean test_h1_row_norms_min test_objective test_term_0 test_term_1_weight_decay test_y_col_norms_max test_y_col_norms_mean test_y_col_norms_min test_y_max_max_class test_y_mean_max_class test_y_min_max_class test_y_misclass test_y_nll test_y_row_norms_max test_y_row_norms_mean test_y_row_norms_min train_h0_col_norms_max train_h0_col_norms_mean train_h0_col_norms_min train_h0_row_norms_max train_h0_row_norms_mean train_h0_row_norms_min train_h1_col_norms_max train_h1_col_norms_mean train_h1_col_norms_min train_h1_row_norms_max train_h1_row_norms_mean train_h1_row_norms_min train_objective train_term_0 train_term_1_weight_decay train_y_col_norms_max train_y_col_norms_mean train_y_col_norms_min train_y_max_max_class train_y_mean_max_class train_y_min_max_class train_y_misclass train_y_nll train_y_row_norms_max train_y_row_norms_mean train_y_row_norms_min valid_h0_col_norms_max valid_h0_col_norms_mean valid_h0_col_norms_min valid_h0_row_norms_max valid_h0_row_norms_mean valid_h0_row_norms_min valid_h1_col_norms_max valid_h1_col_norms_mean valid_h1_col_norms_min valid_h1_row_norms_max valid_h1_row_norms_mean valid_h1_row_norms_min valid_objective valid_term_0 valid_term_1_weight_decay valid_y_col_norms_max valid_y_col_norms_mean valid_y_col_norms_min valid_y_max_max_class valid_y_mean_max_class valid_y_min_max_class valid_y_misclass valid_y_nll valid_y_row_norms_max valid_y_row_norms_mean valid_y_row_norms_min Compiling accum... graph size: 171 graph size: 169 graph size: 169 Compiling accum done. Time elapsed: 13.418733 seconds Monitoring step: Epochs seen: 0 Batches seen: 0 Examples seen: 0 learning_rate: 0.00999999046326 momentum: 0.499999672174 test_h0_col_norms_max: 6.23503017426 test_h0_col_norms_mean: 3.82356023788 test_h0_col_norms_min: 2.06193947792 test_h0_row_norms_max: 5.89326524734 test_h0_row_norms_mean: 2.98549389839 test_h0_row_norms_min: 0.0 test_h1_col_norms_max: 5.99438333511 test_h1_col_norms_mean: 3.80721712112 test_h1_col_norms_min: 1.71524214745 test_h1_row_norms_max: 7.80886650085 test_h1_row_norms_mean: 5.40815734863 test_h1_row_norms_min: 2.97773504257 test_objective: 3.4297709465 test_term_0: 2.30258488655 test_term_1_weight_decay: 1.12718772888 test_y_col_norms_max: 0.0 test_y_col_norms_mean: 0.0 test_y_col_norms_min: 0.0 test_y_max_max_class: 0.100000023842 test_y_mean_max_class: 0.100000031292 test_y_min_max_class: 0.100000023842 test_y_misclass: 0.901999890804 test_y_nll: 2.30258488655 test_y_row_norms_max: 0.0 test_y_row_norms_mean: 0.0 test_y_row_norms_min: 0.0 train_h0_col_norms_max: 6.23505115509 train_h0_col_norms_mean: 3.82354259491 train_h0_col_norms_min: 2.0619494915 train_h0_row_norms_max: 5.89324569702 train_h0_row_norms_mean: 2.98548007011 train_h0_row_norms_min: 0.0 train_h1_col_norms_max: 5.99438095093 train_h1_col_norms_mean: 3.80721092224 train_h1_col_norms_min: 1.71524274349 train_h1_row_norms_max: 7.80887794495 train_h1_row_norms_mean: 5.40813541412 train_h1_row_norms_min: 2.97772955894 train_objective: 3.42977070808 train_term_0: 2.30257916451 train_term_1_weight_decay: 1.12718474865 train_y_col_norms_max: 0.0 train_y_col_norms_mean: 0.0 train_y_col_norms_min: 0.0 train_y_max_max_class: 0.100000545382 train_y_mean_max_class: 0.100000545382 train_y_min_max_class: 0.100000545382 train_y_misclass: 0.901360213757 train_y_nll: 2.30257916451 train_y_row_norms_max: 0.0 train_y_row_norms_mean: 0.0 train_y_row_norms_min: 0.0 valid_h0_col_norms_max: 6.23503017426 valid_h0_col_norms_mean: 3.82356023788 valid_h0_col_norms_min: 2.06193947792 valid_h0_row_norms_max: 5.89326524734 valid_h0_row_norms_mean: 2.98549389839 valid_h0_row_norms_min: 0.0 valid_h1_col_norms_max: 5.99438333511 valid_h1_col_norms_mean: 3.80721712112 valid_h1_col_norms_min: 1.71524214745 valid_h1_row_norms_max: 7.80886650085 valid_h1_row_norms_mean: 5.40815734863 valid_h1_row_norms_min: 2.97773504257 valid_objective: 3.4297709465 valid_term_0: 2.30258488655 valid_term_1_weight_decay: 1.12718772888 valid_y_col_norms_max: 0.0 valid_y_col_norms_mean: 0.0 valid_y_col_norms_min: 0.0 valid_y_max_max_class: 0.100000023842 valid_y_mean_max_class: 0.100000031292 valid_y_min_max_class: 0.100000023842 valid_y_misclass: 0.90089994669 valid_y_nll: 2.30258488655 valid_y_row_norms_max: 0.0 valid_y_row_norms_mean: 0.0 valid_y_row_norms_min: 0.0 Time this epoch: 3.310886 seconds Monitoring step: Epochs seen: 1 Batches seen: 500 Examples seen: 50000 learning_rate: 0.00999999046326 momentum: 0.499999672174 test_h0_col_norms_max: 6.22863864899 test_h0_col_norms_mean: 3.81978034973 test_h0_col_norms_min: 2.06060481071 test_h0_row_norms_max: 5.88668251038 test_h0_row_norms_mean: 2.98259210587 test_h0_row_norms_min: 0.00163801340386 test_h1_col_norms_max: 5.98888349533 test_h1_col_norms_mean: 3.80343770981 test_h1_col_norms_min: 1.71354997158 test_h1_row_norms_max: 7.80116271973 test_h1_row_norms_mean: 5.40278577805 test_h1_row_norms_min: 2.97481369972 test_objective: 1.39391481876 test_term_0: 0.268794178963 test_term_1_weight_decay: 1.12512099743 test_y_col_norms_max: 0.645387113094 test_y_col_norms_mean: 0.59630638361 test_y_col_norms_min: 0.520404875278 test_y_max_max_class: 0.999945759773 test_y_mean_max_class: 0.904323577881 test_y_min_max_class: 0.380515068769 test_y_misclass: 0.0813000127673 test_y_nll: 0.268794178963 test_y_row_norms_max: 0.179665878415 test_y_row_norms_mean: 0.0518467575312 test_y_row_norms_min: 0.000148977691424 train_h0_col_norms_max: 6.2286696434 train_h0_col_norms_mean: 3.81979823112 train_h0_col_norms_min: 2.06059765816 train_h0_row_norms_max: 5.88671255112 train_h0_row_norms_mean: 2.9826066494 train_h0_row_norms_min: 0.00163802062161 train_h1_col_norms_max: 5.9888548851 train_h1_col_norms_mean: 3.80346035957 train_h1_col_norms_min: 1.71355748177 train_h1_row_norms_max: 7.80111694336 train_h1_row_norms_mean: 5.40279817581 train_h1_row_norms_min: 2.97482800484 train_objective: 1.38994812965 train_term_0: 0.264828205109 train_term_1_weight_decay: 1.12512207031 train_y_col_norms_max: 0.645388245583 train_y_col_norms_mean: 0.596305251122 train_y_col_norms_min: 0.520407259464 train_y_max_max_class: 0.99996304512 train_y_mean_max_class: 0.898920297623 train_y_min_max_class: 0.361467987299 train_y_misclass: 0.0793600603938 train_y_nll: 0.264828205109 train_y_row_norms_max: 0.179665371776 train_y_row_norms_mean: 0.0518467389047 train_y_row_norms_min: 0.000148977618665 valid_h0_col_norms_max: 6.22863864899 valid_h0_col_norms_mean: 3.81978034973 valid_h0_col_norms_min: 2.06060481071 valid_h0_row_norms_max: 5.88668251038 valid_h0_row_norms_mean: 2.98259210587 valid_h0_row_norms_min: 0.00163801340386 valid_h1_col_norms_max: 5.98888349533 valid_h1_col_norms_mean: 3.80343770981 valid_h1_col_norms_min: 1.71354997158 valid_h1_row_norms_max: 7.80116271973 valid_h1_row_norms_mean: 5.40278577805 valid_h1_row_norms_min: 2.97481369972 valid_objective: 1.37731289864 valid_term_0: 0.252192467451 valid_term_1_weight_decay: 1.12512099743 valid_y_col_norms_max: 0.645387113094 valid_y_col_norms_mean: 0.59630638361 valid_y_col_norms_min: 0.520404875278 valid_y_max_max_class: 0.999964594841 valid_y_mean_max_class: 0.907153248787 valid_y_min_max_class: 0.362326830626 valid_y_misclass: 0.0756999999285 valid_y_nll: 0.252192467451 valid_y_row_norms_max: 0.179665878415 valid_y_row_norms_mean: 0.0518467575312 valid_y_row_norms_min: 0.000148977691424 Time this epoch: 3.343837 seconds Monitoring step: Epochs seen: 2 Batches seen: 1000 Examples seen: 100000 learning_rate: 0.00999999046326 momentum: 0.554444551468 test_h0_col_norms_max: 6.22144937515 test_h0_col_norms_mean: 3.81579256058 test_h0_col_norms_min: 2.05898046494 test_h0_row_norms_max: 5.88006973267 test_h0_row_norms_mean: 2.9794948101 test_h0_row_norms_min: 0.00336797139607 test_h1_col_norms_max: 5.98277664185 test_h1_col_norms_mean: 3.79929542542 test_h1_col_norms_min: 1.71166646481 test_h1_row_norms_max: 7.79234170914 test_h1_row_norms_mean: 5.3969039917 test_h1_row_norms_min: 2.97146487236 test_objective: 1.3320376873 test_term_0: 0.209235101938 test_term_1_weight_decay: 1.12280321121 test_y_col_norms_max: 0.849509298801 test_y_col_norms_mean: 0.752226889133 test_y_col_norms_min: 0.648749351501 test_y_max_max_class: 0.999980688095 test_y_mean_max_class: 0.928127348423 test_y_min_max_class: 0.417017698288 test_y_misclass: 0.0624000132084 test_y_nll: 0.209235101938 test_y_row_norms_max: 0.202931031585 test_y_row_norms_mean: 0.0667919442058 test_y_row_norms_min: 0.00027507453342 train_h0_col_norms_max: 6.22147130966 train_h0_col_norms_mean: 3.81577634811 train_h0_col_norms_min: 2.0589826107 train_h0_row_norms_max: 5.8800983429 train_h0_row_norms_mean: 2.9795088768 train_h0_row_norms_min: 0.00336798490025 train_h1_col_norms_max: 5.98279714584 train_h1_col_norms_mean: 3.7993118763 train_h1_col_norms_min: 1.71166646481 train_h1_row_norms_max: 7.79229545593 train_h1_row_norms_mean: 5.39690923691 train_h1_row_norms_min: 2.97145032883 train_objective: 1.31553328037 train_term_0: 0.192730411887 train_term_1_weight_decay: 1.12280583382 train_y_col_norms_max: 0.849513113499 train_y_col_norms_mean: 0.752230584621 train_y_col_norms_min: 0.648747861385 train_y_max_max_class: 0.999980807304 train_y_mean_max_class: 0.925747811794 train_y_min_max_class: 0.379059791565 train_y_misclass: 0.0572400614619 train_y_nll: 0.192730411887 train_y_row_norms_max: 0.202931344509 train_y_row_norms_mean: 0.0667921230197 train_y_row_norms_min: 0.00027507476625 valid_h0_col_norms_max: 6.22144937515 valid_h0_col_norms_mean: 3.81579256058 valid_h0_col_norms_min: 2.05898046494 valid_h0_row_norms_max: 5.88006973267 valid_h0_row_norms_mean: 2.9794948101 valid_h0_row_norms_min: 0.00336797139607 valid_h1_col_norms_max: 5.98277664185 valid_h1_col_norms_mean: 3.79929542542 valid_h1_col_norms_min: 1.71166646481 valid_h1_row_norms_max: 7.79234170914 valid_h1_row_norms_mean: 5.3969039917 valid_h1_row_norms_min: 2.97146487236 valid_objective: 1.32417428493 valid_term_0: 0.201371654868 valid_term_1_weight_decay: 1.12280321121 valid_y_col_norms_max: 0.849509298801 valid_y_col_norms_mean: 0.752226889133 valid_y_col_norms_min: 0.648749351501 valid_y_max_max_class: 0.999982237816 valid_y_mean_max_class: 0.931577861309 valid_y_min_max_class: 0.40255895257 valid_y_misclass: 0.0578999966383 valid_y_nll: 0.201371654868 valid_y_row_norms_max: 0.202931031585 valid_y_row_norms_mean: 0.0667919442058 valid_y_row_norms_min: 0.00027507453342 Time this epoch: 3.283221 seconds Monitoring step: Epochs seen: 3 Batches seen: 1500 Examples seen: 150000 learning_rate: 0.00999999046326 momentum: 0.608888924122 test_h0_col_norms_max: 6.21347379684 test_h0_col_norms_mean: 3.81121587753 test_h0_col_norms_min: 2.05705142021 test_h0_row_norms_max: 5.87235736847 test_h0_row_norms_mean: 2.97595834732 test_h0_row_norms_min: 0.00510276248679 test_h1_col_norms_max: 5.97572278976 test_h1_col_norms_mean: 3.79457330704 test_h1_col_norms_min: 1.70953249931 test_h1_row_norms_max: 7.78235435486 test_h1_row_norms_mean: 5.39019727707 test_h1_row_norms_min: 2.96771478653 test_objective: 1.30544030666 test_term_0: 0.185299769044 test_term_1_weight_decay: 1.12013947964 test_y_col_norms_max: 1.00650155544 test_y_col_norms_mean: 0.878560483456 test_y_col_norms_min: 0.748090326786 test_y_max_max_class: 0.999993503094 test_y_mean_max_class: 0.939459979534 test_y_min_max_class: 0.444366723299 test_y_misclass: 0.0547000169754 test_y_nll: 0.185299769044 test_y_row_norms_max: 0.217191457748 test_y_row_norms_mean: 0.0787876471877 test_y_row_norms_min: 0.000392778747482 train_h0_col_norms_max: 6.21344470978 train_h0_col_norms_mean: 3.81123256683 train_h0_col_norms_min: 2.05706167221 train_h0_row_norms_max: 5.87232971191 train_h0_row_norms_mean: 2.97594833374 train_h0_row_norms_min: 0.00510273734108 train_h1_col_norms_max: 5.97572278976 train_h1_col_norms_mean: 3.79455709457 train_h1_col_norms_min: 1.70952439308 train_h1_row_norms_max: 7.78239917755 train_h1_row_norms_mean: 5.39017248154 train_h1_row_norms_min: 2.96771502495 train_objective: 1.2823060751 train_term_0: 0.162165120244 train_term_1_weight_decay: 1.12014472485 train_y_col_norms_max: 1.00650632381 train_y_col_norms_mean: 0.878564417362 train_y_col_norms_min: 0.748090386391 train_y_max_max_class: 0.999991178513 train_y_mean_max_class: 0.93700414896 train_y_min_max_class: 0.404900848866 train_y_misclass: 0.0482200570405 train_y_nll: 0.162165120244 train_y_row_norms_max: 0.21719174087 train_y_row_norms_mean: 0.0787875503302 train_y_row_norms_min: 0.000392780813854 valid_h0_col_norms_max: 6.21347379684 valid_h0_col_norms_mean: 3.81121587753 valid_h0_col_norms_min: 2.05705142021 valid_h0_row_norms_max: 5.87235736847 valid_h0_row_norms_mean: 2.97595834732 valid_h0_row_norms_min: 0.00510276248679 valid_h1_col_norms_max: 5.97572278976 valid_h1_col_norms_mean: 3.79457330704 valid_h1_col_norms_min: 1.70953249931 valid_h1_row_norms_max: 7.78235435486 valid_h1_row_norms_mean: 5.39019727707 valid_h1_row_norms_min: 2.96771478653 valid_objective: 1.29470717907 valid_term_0: 0.174566537142 valid_term_1_weight_decay: 1.12013947964 valid_y_col_norms_max: 1.00650155544 valid_y_col_norms_mean: 0.878560483456 valid_y_col_norms_min: 0.748090326786 valid_y_max_max_class: 0.999994695187 valid_y_mean_max_class: 0.942149102688 valid_y_min_max_class: 0.417711257935 valid_y_misclass: 0.051200017333 valid_y_nll: 0.174566537142 valid_y_row_norms_max: 0.217191457748 valid_y_row_norms_mean: 0.0787876471877 valid_y_row_norms_min: 0.000392778747482 Time this epoch: 3.301401 seconds Monitoring step: Epochs seen: 4 Batches seen: 2000 Examples seen: 200000 learning_rate: 0.00999999046326 momentum: 0.663333714008 test_h0_col_norms_max: 6.20446586609 test_h0_col_norms_mean: 3.80589365959 test_h0_col_norms_min: 2.05491876602 test_h0_row_norms_max: 5.86368274689 test_h0_row_norms_mean: 2.97183966637 test_h0_row_norms_min: 0.00636858073995 test_h1_col_norms_max: 5.96751737595 test_h1_col_norms_mean: 3.78907322884 test_h1_col_norms_min: 1.70705342293 test_h1_row_norms_max: 7.77082681656 test_h1_row_norms_mean: 5.38239336014 test_h1_row_norms_min: 2.96349358559 test_objective: 1.28483641148 test_term_0: 0.167798668146 test_term_1_weight_decay: 1.11703836918 test_y_col_norms_max: 1.14337170124 test_y_col_norms_mean: 0.994192421436 test_y_col_norms_min: 0.840292572975 test_y_max_max_class: 0.999995589256 test_y_mean_max_class: 0.946651279926 test_y_min_max_class: 0.454940706491 test_y_misclass: 0.0549000278115 test_y_nll: 0.167798668146 test_y_row_norms_max: 0.231142029166 test_y_row_norms_mean: 0.089763648808 test_y_row_norms_min: 0.000477136200061 train_h0_col_norms_max: 6.20444250107 train_h0_col_norms_mean: 3.80587768555 train_h0_col_norms_min: 2.05492663383 train_h0_row_norms_max: 5.86367082596 train_h0_row_norms_mean: 2.97184991837 train_h0_row_norms_min: 0.00636860262603 train_h1_col_norms_max: 5.96753835678 train_h1_col_norms_mean: 3.7890689373 train_h1_col_norms_min: 1.70706069469 train_h1_row_norms_max: 7.77079200745 train_h1_row_norms_mean: 5.38238239288 train_h1_row_norms_min: 2.96350455284 train_objective: 1.25564575195 train_term_0: 0.138607770205 train_term_1_weight_decay: 1.11704432964 train_y_col_norms_max: 1.14337110519 train_y_col_norms_mean: 0.994198083878 train_y_col_norms_min: 0.840297460556 train_y_max_max_class: 0.999992132187 train_y_mean_max_class: 0.945581674576 train_y_min_max_class: 0.42304289341 train_y_misclass: 0.0431200563908 train_y_nll: 0.138607770205 train_y_row_norms_max: 0.231140971184 train_y_row_norms_mean: 0.0897636190057 train_y_row_norms_min: 0.000477139052236 valid_h0_col_norms_max: 6.20446586609 valid_h0_col_norms_mean: 3.80589365959 valid_h0_col_norms_min: 2.05491876602 valid_h0_row_norms_max: 5.86368274689 valid_h0_row_norms_mean: 2.97183966637 valid_h0_row_norms_min: 0.00636858073995 valid_h1_col_norms_max: 5.96751737595 valid_h1_col_norms_mean: 3.78907322884 valid_h1_col_norms_min: 1.70705342293 valid_h1_row_norms_max: 7.77082681656 valid_h1_row_norms_mean: 5.38239336014 valid_h1_row_norms_min: 2.96349358559 valid_objective: 1.27460837364 valid_term_0: 0.157571211457 valid_term_1_weight_decay: 1.11703836918 valid_y_col_norms_max: 1.14337170124 valid_y_col_norms_mean: 0.994192421436 valid_y_col_norms_min: 0.840292572975 valid_y_max_max_class: 0.999996304512 valid_y_mean_max_class: 0.949614882469 valid_y_min_max_class: 0.442067503929 valid_y_misclass: 0.0465000085533 valid_y_nll: 0.157571211457 valid_y_row_norms_max: 0.231142029166 valid_y_row_norms_mean: 0.089763648808 valid_y_row_norms_min: 0.000477136200061 Time this epoch: 3.266055 seconds Monitoring step: Epochs seen: 5 Batches seen: 2500 Examples seen: 250000 learning_rate: 0.00999999046326 momentum: 0.717777192593 test_h0_col_norms_max: 6.19388818741 test_h0_col_norms_mean: 3.79951477051 test_h0_col_norms_min: 2.05230784416 test_h0_row_norms_max: 5.85298204422 test_h0_row_norms_mean: 2.96690416336 test_h0_row_norms_min: 0.00795079302043 test_h1_col_norms_max: 5.95764780045 test_h1_col_norms_mean: 3.78251552582 test_h1_col_norms_min: 1.70412421227 test_h1_row_norms_max: 7.7571387291 test_h1_row_norms_mean: 5.37306642532 test_h1_row_norms_min: 2.95853662491 test_objective: 1.25132834911 test_term_0: 0.138001933694 test_term_1_weight_decay: 1.11332631111 test_y_col_norms_max: 1.26581287384 test_y_col_norms_mean: 1.10778701305 test_y_col_norms_min: 0.922472834587 test_y_max_max_class: 0.999994754791 test_y_mean_max_class: 0.953354179859 test_y_min_max_class: 0.460847198963 test_y_misclass: 0.0430000051856 test_y_nll: 0.138001933694 test_y_row_norms_max: 0.258754551411 test_y_row_norms_mean: 0.100538700819 test_y_row_norms_min: 0.000593058066443 train_h0_col_norms_max: 6.19387769699 train_h0_col_norms_mean: 3.79953241348 train_h0_col_norms_min: 2.05230736732 train_h0_row_norms_max: 5.85295391083 train_h0_row_norms_mean: 2.96689033508 train_h0_row_norms_min: 0.0079507548362 train_h1_col_norms_max: 5.95761966705 train_h1_col_norms_mean: 3.78251123428 train_h1_col_norms_min: 1.70413899422 train_h1_row_norms_max: 7.75717258453 train_h1_row_norms_mean: 5.37306880951 train_h1_row_norms_min: 2.95853638649 train_objective: 1.21803998947 train_term_0: 0.104714490473 train_term_1_weight_decay: 1.11332845688 train_y_col_norms_max: 1.26581907272 train_y_col_norms_mean: 1.1077862978 train_y_col_norms_min: 0.922471702099 train_y_max_max_class: 0.999992728233 train_y_mean_max_class: 0.954178750515 train_y_min_max_class: 0.440906405449 train_y_misclass: 0.0312400292605 train_y_nll: 0.104714490473 train_y_row_norms_max: 0.258753240108 train_y_row_norms_mean: 0.100538358092 train_y_row_norms_min: 0.000593057950027 valid_h0_col_norms_max: 6.19388818741 valid_h0_col_norms_mean: 3.79951477051 valid_h0_col_norms_min: 2.05230784416 valid_h0_row_norms_max: 5.85298204422 valid_h0_row_norms_mean: 2.96690416336 valid_h0_row_norms_min: 0.00795079302043 valid_h1_col_norms_max: 5.95764780045 valid_h1_col_norms_mean: 3.78251552582 valid_h1_col_norms_min: 1.70412421227 valid_h1_row_norms_max: 7.7571387291 valid_h1_row_norms_mean: 5.37306642532 valid_h1_row_norms_min: 2.95853662491 valid_objective: 1.24973428249 valid_term_0: 0.136407867074 valid_term_1_weight_decay: 1.11332631111 valid_y_col_norms_max: 1.26581287384 valid_y_col_norms_mean: 1.10778701305 valid_y_col_norms_min: 0.922472834587 valid_y_max_max_class: 0.999996542931 valid_y_mean_max_class: 0.955720424652 valid_y_min_max_class: 0.447657436132 valid_y_misclass: 0.0386000014842 valid_y_nll: 0.136407867074 valid_y_row_norms_max: 0.258754551411 valid_y_row_norms_mean: 0.100538700819 valid_y_row_norms_min: 0.000593058066443 Time this epoch: 3.281634 seconds Monitoring step: Epochs seen: 6 Batches seen: 3000 Examples seen: 300000 learning_rate: 0.00999999046326 momentum: 0.772221684456 test_h0_col_norms_max: 6.18053913116 test_h0_col_norms_mean: 3.79164195061 test_h0_col_norms_min: 2.04931807518 test_h0_row_norms_max: 5.84014606476 test_h0_row_norms_mean: 2.96080875397 test_h0_row_norms_min: 0.00960826966912 test_h1_col_norms_max: 5.94511365891 test_h1_col_norms_mean: 3.77440404892 test_h1_col_norms_min: 1.70048546791 test_h1_row_norms_max: 7.74020195007 test_h1_row_norms_mean: 5.3615436554 test_h1_row_norms_min: 2.95202755928 test_objective: 1.23484170437 test_term_0: 0.126101091504 test_term_1_weight_decay: 1.10874140263 test_y_col_norms_max: 1.39184403419 test_y_col_norms_mean: 1.23041391373 test_y_col_norms_min: 1.02565836906 test_y_max_max_class: 0.999998748302 test_y_mean_max_class: 0.961094081402 test_y_min_max_class: 0.502607226372 test_y_misclass: 0.0397000052035 test_y_nll: 0.126101091504 test_y_row_norms_max: 0.288574844599 test_y_row_norms_mean: 0.112107351422 test_y_row_norms_min: 0.000744926044717 train_h0_col_norms_max: 6.18052864075 train_h0_col_norms_mean: 3.79166030884 train_h0_col_norms_min: 2.04932594299 train_h0_row_norms_max: 5.84012889862 train_h0_row_norms_mean: 2.96080327034 train_h0_row_norms_min: 0.0096082771197 train_h1_col_norms_max: 5.94514036179 train_h1_col_norms_mean: 3.77440428734 train_h1_col_norms_min: 1.7004776001 train_h1_row_norms_max: 7.74021291733 train_h1_row_norms_mean: 5.36155557632 train_h1_row_norms_min: 2.9520175457 train_objective: 1.19061946869 train_term_0: 0.0818792134523 train_term_1_weight_decay: 1.10874009132 train_y_col_norms_max: 1.39184439182 train_y_col_norms_mean: 1.2304173708 train_y_col_norms_min: 1.02565360069 train_y_max_max_class: 0.999994039536 train_y_mean_max_class: 0.963193774223 train_y_min_max_class: 0.475303918123 train_y_misclass: 0.0230600107461 train_y_nll: 0.0818792134523 train_y_row_norms_max: 0.288575559855 train_y_row_norms_mean: 0.112107902765 train_y_row_norms_min: 0.000744922028389 valid_h0_col_norms_max: 6.18053913116 valid_h0_col_norms_mean: 3.79164195061 valid_h0_col_norms_min: 2.04931807518 valid_h0_row_norms_max: 5.84014606476 valid_h0_row_norms_mean: 2.96080875397 valid_h0_row_norms_min: 0.00960826966912 valid_h1_col_norms_max: 5.94511365891 valid_h1_col_norms_mean: 3.77440404892 valid_h1_col_norms_min: 1.70048546791 valid_h1_row_norms_max: 7.74020195007 valid_h1_row_norms_mean: 5.3615436554 valid_h1_row_norms_min: 2.95202755928 valid_objective: 1.23645818233 valid_term_0: 0.127717524767 valid_term_1_weight_decay: 1.10874140263 valid_y_col_norms_max: 1.39184403419 valid_y_col_norms_mean: 1.23041391373 valid_y_col_norms_min: 1.02565836906 valid_y_max_max_class: 0.999998986721 valid_y_mean_max_class: 0.963711440563 valid_y_min_max_class: 0.479158580303 valid_y_misclass: 0.0373999997973 valid_y_nll: 0.127717524767 valid_y_row_norms_max: 0.288574844599 valid_y_row_norms_mean: 0.112107351422 valid_y_row_norms_min: 0.000744926044717 Time this epoch: 3.285549 seconds Monitoring step: Epochs seen: 7 Batches seen: 3500 Examples seen: 350000 learning_rate: 0.00999999046326 momentum: 0.826667308807 test_h0_col_norms_max: 6.16351413727 test_h0_col_norms_mean: 3.78127264977 test_h0_col_norms_min: 2.04552721977 test_h0_row_norms_max: 5.82413673401 test_h0_row_norms_mean: 2.95279192924 test_h0_row_norms_min: 0.0109715117142 test_h1_col_norms_max: 5.92860794067 test_h1_col_norms_mean: 3.7637283802 test_h1_col_norms_min: 1.69574940205 test_h1_row_norms_max: 7.71776247025 test_h1_row_norms_mean: 5.34639310837 test_h1_row_norms_min: 2.94415974617 test_objective: 1.2293548584 test_term_0: 0.126640558243 test_term_1_weight_decay: 1.1027148962 test_y_col_norms_max: 1.53999233246 test_y_col_norms_mean: 1.36674308777 test_y_col_norms_min: 1.134085536 test_y_max_max_class: 0.999998986721 test_y_mean_max_class: 0.962450027466 test_y_min_max_class: 0.520037055016 test_y_misclass: 0.0400000177324 test_y_nll: 0.126640558243 test_y_row_norms_max: 0.323384702206 test_y_row_norms_mean: 0.124884955585 test_y_row_norms_min: 0.000862787244841 train_h0_col_norms_max: 6.1635351181 train_h0_col_norms_mean: 3.78129315376 train_h0_col_norms_min: 2.04552340508 train_h0_row_norms_max: 5.82410860062 train_h0_row_norms_mean: 2.95280575752 train_h0_row_norms_min: 0.0109714772552 train_h1_col_norms_max: 5.92858171463 train_h1_col_norms_mean: 3.76370692253 train_h1_col_norms_min: 1.69575130939 train_h1_row_norms_max: 7.71779823303 train_h1_row_norms_mean: 5.34638214111 train_h1_row_norms_min: 2.94414997101 train_objective: 1.18144452572 train_term_0: 0.0787304490805 train_term_1_weight_decay: 1.10271286964 train_y_col_norms_max: 1.54000031948 train_y_col_norms_mean: 1.36673867702 train_y_col_norms_min: 1.13409137726 train_y_max_max_class: 0.999994158745 train_y_mean_max_class: 0.964662730694 train_y_min_max_class: 0.485619604588 train_y_misclass: 0.0242600161582 train_y_nll: 0.0787304490805 train_y_row_norms_max: 0.323384910822 train_y_row_norms_mean: 0.124885700643 train_y_row_norms_min: 0.000862783577759 valid_h0_col_norms_max: 6.16351413727 valid_h0_col_norms_mean: 3.78127264977 valid_h0_col_norms_min: 2.04552721977 valid_h0_row_norms_max: 5.82413673401 valid_h0_row_norms_mean: 2.95279192924 valid_h0_row_norms_min: 0.0109715117142 valid_h1_col_norms_max: 5.92860794067 valid_h1_col_norms_mean: 3.7637283802 valid_h1_col_norms_min: 1.69574940205 valid_h1_row_norms_max: 7.71776247025 valid_h1_row_norms_mean: 5.34639310837 valid_h1_row_norms_min: 2.94415974617 valid_objective: 1.22817146778 valid_term_0: 0.125456944108 valid_term_1_weight_decay: 1.1027148962 valid_y_col_norms_max: 1.53999233246 valid_y_col_norms_mean: 1.36674308777 valid_y_col_norms_min: 1.134085536 valid_y_max_max_class: 0.99999910593 valid_y_mean_max_class: 0.965774953365 valid_y_min_max_class: 0.481605708599 valid_y_misclass: 0.0360999889672 valid_y_nll: 0.125456944108 valid_y_row_norms_max: 0.323384702206 valid_y_row_norms_mean: 0.124884955585 valid_y_row_norms_min: 0.000862787244841 Time this epoch: 3.275973 seconds Monitoring step: Epochs seen: 8 Batches seen: 4000 Examples seen: 400000 learning_rate: 0.00999999046326 momentum: 0.881111502647 test_h0_col_norms_max: 6.13874149323 test_h0_col_norms_mean: 3.76625037193 test_h0_col_norms_min: 2.03984022141 test_h0_row_norms_max: 5.79944992065 test_h0_row_norms_mean: 2.94116210938 test_h0_row_norms_min: 0.0121828410774 test_h1_col_norms_max: 5.90430831909 test_h1_col_norms_mean: 3.74820208549 test_h1_col_norms_min: 1.68876981735 test_h1_row_norms_max: 7.68556308746 test_h1_row_norms_mean: 5.32432985306 test_h1_row_norms_min: 2.93232631683 test_objective: 1.21413767338 test_term_0: 0.12014952302 test_term_1_weight_decay: 1.09398806095 test_y_col_norms_max: 1.73185801506 test_y_col_norms_mean: 1.54484415054 test_y_col_norms_min: 1.28760778904 test_y_max_max_class: 0.999999284744 test_y_mean_max_class: 0.969546198845 test_y_min_max_class: 0.53670758009 test_y_misclass: 0.0355999991298 test_y_nll: 0.12014952302 test_y_row_norms_max: 0.390541791916 test_y_row_norms_mean: 0.141607090831 test_y_row_norms_min: 0.00119230698328 train_h0_col_norms_max: 6.13874340057 train_h0_col_norms_mean: 3.76626849174 train_h0_col_norms_min: 2.03984594345 train_h0_row_norms_max: 5.79946660995 train_h0_row_norms_mean: 2.94116210938 train_h0_row_norms_min: 0.0121827786788 train_h1_col_norms_max: 5.90427827835 train_h1_col_norms_mean: 3.74818611145 train_h1_col_norms_min: 1.68877720833 train_h1_row_norms_max: 7.68560028076 train_h1_row_norms_mean: 5.32434654236 train_h1_row_norms_min: 2.93231272697 train_objective: 1.15510380268 train_term_0: 0.0611156411469 train_term_1_weight_decay: 1.09398496151 train_y_col_norms_max: 1.73185968399 train_y_col_norms_mean: 1.54483699799 train_y_col_norms_min: 1.28760266304 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.97226446867 train_y_min_max_class: 0.523442387581 train_y_misclass: 0.0181600283831 train_y_nll: 0.0611156411469 train_y_row_norms_max: 0.390543580055 train_y_row_norms_mean: 0.141606390476 train_y_row_norms_min: 0.00119230675045 valid_h0_col_norms_max: 6.13874149323 valid_h0_col_norms_mean: 3.76625037193 valid_h0_col_norms_min: 2.03984022141 valid_h0_row_norms_max: 5.79944992065 valid_h0_row_norms_mean: 2.94116210938 valid_h0_row_norms_min: 0.0121828410774 valid_h1_col_norms_max: 5.90430831909 valid_h1_col_norms_mean: 3.74820208549 valid_h1_col_norms_min: 1.68876981735 valid_h1_row_norms_max: 7.68556308746 valid_h1_row_norms_mean: 5.32432985306 valid_h1_row_norms_min: 2.93232631683 valid_objective: 1.2128187418 valid_term_0: 0.118830725551 valid_term_1_weight_decay: 1.09398806095 valid_y_col_norms_max: 1.73185801506 valid_y_col_norms_mean: 1.54484415054 valid_y_col_norms_min: 1.28760778904 valid_y_max_max_class: 0.999999284744 valid_y_mean_max_class: 0.971059143543 valid_y_min_max_class: 0.500100016594 valid_y_misclass: 0.0353999920189 valid_y_nll: 0.118830725551 valid_y_row_norms_max: 0.390541791916 valid_y_row_norms_mean: 0.141607090831 valid_y_row_norms_min: 0.00119230698328 Time this epoch: 3.273986 seconds Monitoring step: Epochs seen: 9 Batches seen: 4500 Examples seen: 450000 learning_rate: 0.00999999046326 momentum: 0.935554862022 test_h0_col_norms_max: 6.09445524216 test_h0_col_norms_mean: 3.73940348625 test_h0_col_norms_min: 2.03072142601 test_h0_row_norms_max: 5.75560235977 test_h0_row_norms_mean: 2.92046833038 test_h0_row_norms_min: 0.014029703103 test_h1_col_norms_max: 5.86166810989 test_h1_col_norms_mean: 3.71971082687 test_h1_col_norms_min: 1.67665565014 test_h1_row_norms_max: 7.62777662277 test_h1_row_norms_mean: 5.2838845253 test_h1_row_norms_min: 2.91292881966 test_objective: 1.20774161816 test_term_0: 0.129474073648 test_term_1_weight_decay: 1.0782674551 test_y_col_norms_max: 2.063549757 test_y_col_norms_mean: 1.8654705286 test_y_col_norms_min: 1.53516829014 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.971782028675 test_y_min_max_class: 0.541796386242 test_y_misclass: 0.0371000058949 test_y_nll: 0.129474073648 test_y_row_norms_max: 0.496850013733 test_y_row_norms_mean: 0.171486049891 test_y_row_norms_min: 0.00181403872557 train_h0_col_norms_max: 6.09445524216 train_h0_col_norms_mean: 3.73938298225 train_h0_col_norms_min: 2.03072929382 train_h0_row_norms_max: 5.75560045242 train_h0_row_norms_mean: 2.92047595978 train_h0_row_norms_min: 0.0140297813341 train_h1_col_norms_max: 5.86169338226 train_h1_col_norms_mean: 3.71969389915 train_h1_col_norms_min: 1.67666423321 train_h1_row_norms_max: 7.62774133682 train_h1_row_norms_mean: 5.2838549614 train_h1_row_norms_min: 2.91291928291 train_objective: 1.14386320114 train_term_0: 0.0655960813165 train_term_1_weight_decay: 1.07827007771 train_y_col_norms_max: 2.06355881691 train_y_col_norms_mean: 1.86546158791 train_y_col_norms_min: 1.53517353535 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.976020038128 train_y_min_max_class: 0.536927580833 train_y_misclass: 0.0207600202411 train_y_nll: 0.0655960813165 train_y_row_norms_max: 0.4968495965 train_y_row_norms_mean: 0.17148527503 train_y_row_norms_min: 0.00181404093746 valid_h0_col_norms_max: 6.09445524216 valid_h0_col_norms_mean: 3.73940348625 valid_h0_col_norms_min: 2.03072142601 valid_h0_row_norms_max: 5.75560235977 valid_h0_row_norms_mean: 2.92046833038 valid_h0_row_norms_min: 0.014029703103 valid_h1_col_norms_max: 5.86166810989 valid_h1_col_norms_mean: 3.71971082687 valid_h1_col_norms_min: 1.67665565014 valid_h1_row_norms_max: 7.62777662277 valid_h1_row_norms_mean: 5.2838845253 valid_h1_row_norms_min: 2.91292881966 valid_objective: 1.21526145935 valid_term_0: 0.136994019151 valid_term_1_weight_decay: 1.0782674551 valid_y_col_norms_max: 2.063549757 valid_y_col_norms_mean: 1.8654705286 valid_y_col_norms_min: 1.53516829014 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.974301934242 valid_y_min_max_class: 0.516560852528 valid_y_misclass: 0.0349999815226 valid_y_nll: 0.136994019151 valid_y_row_norms_max: 0.496850013733 valid_y_row_norms_mean: 0.171486049891 valid_y_row_norms_min: 0.00181403872557 Time this epoch: 3.317775 seconds Monitoring step: Epochs seen: 10 Batches seen: 5000 Examples seen: 500000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 5.92522621155 test_h0_col_norms_mean: 3.73818850517 test_h0_col_norms_min: 2.15961098671 test_h0_row_norms_max: 5.7353053093 test_h0_row_norms_mean: 2.92477583885 test_h0_row_norms_min: 0.0341217853129 test_h1_col_norms_max: 5.61352205276 test_h1_col_norms_mean: 3.57546806335 test_h1_col_norms_min: 1.61370325089 test_h1_row_norms_max: 7.31059169769 test_h1_row_norms_mean: 5.08152914047 test_h1_row_norms_min: 2.99987840652 test_objective: 1.26450061798 test_term_0: 0.236082434654 test_term_1_weight_decay: 1.02841842175 test_y_col_norms_max: 4.73058700562 test_y_col_norms_mean: 4.18089103699 test_y_col_norms_min: 3.5669798851 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.968040525913 test_y_min_max_class: 0.498749941587 test_y_misclass: 0.059400010854 test_y_nll: 0.236082434654 test_y_row_norms_max: 0.891591668129 test_y_row_norms_mean: 0.392109334469 test_y_row_norms_min: 0.0124359438196 train_h0_col_norms_max: 5.92519760132 train_h0_col_norms_mean: 3.73817253113 train_h0_col_norms_min: 2.15960621834 train_h0_row_norms_max: 5.73533010483 train_h0_row_norms_mean: 2.92479014397 train_h0_row_norms_min: 0.0341217927635 train_h1_col_norms_max: 5.61354923248 train_h1_col_norms_mean: 3.57545208931 train_h1_col_norms_min: 1.61369478703 train_h1_row_norms_max: 7.31061649323 train_h1_row_norms_mean: 5.08150863647 train_h1_row_norms_min: 2.99989366531 train_objective: 1.2140481472 train_term_0: 0.185629963875 train_term_1_weight_decay: 1.02841842175 train_y_col_norms_max: 4.73060131073 train_y_col_norms_mean: 4.18090629578 train_y_col_norms_min: 3.56698012352 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.968826234341 train_y_min_max_class: 0.484800755978 train_y_misclass: 0.0509400516748 train_y_nll: 0.185629963875 train_y_row_norms_max: 0.891595542431 train_y_row_norms_mean: 0.392109185457 train_y_row_norms_min: 0.0124359484762 valid_h0_col_norms_max: 5.92522621155 valid_h0_col_norms_mean: 3.73818850517 valid_h0_col_norms_min: 2.15961098671 valid_h0_row_norms_max: 5.7353053093 valid_h0_row_norms_mean: 2.92477583885 valid_h0_row_norms_min: 0.0341217853129 valid_h1_col_norms_max: 5.61352205276 valid_h1_col_norms_mean: 3.57546806335 valid_h1_col_norms_min: 1.61370325089 valid_h1_row_norms_max: 7.31059169769 valid_h1_row_norms_mean: 5.08152914047 valid_h1_row_norms_min: 2.99987840652 valid_objective: 1.27066576481 valid_term_0: 0.242247447371 valid_term_1_weight_decay: 1.02841842175 valid_y_col_norms_max: 4.73058700562 valid_y_col_norms_mean: 4.18089103699 valid_y_col_norms_min: 3.5669798851 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.969310641289 valid_y_min_max_class: 0.485083043575 valid_y_misclass: 0.0584000013769 valid_y_nll: 0.242247447371 valid_y_row_norms_max: 0.891591668129 valid_y_row_norms_mean: 0.392109334469 valid_y_row_norms_min: 0.0124359438196 Time this epoch: 3.378083 seconds Monitoring step: Epochs seen: 11 Batches seen: 5500 Examples seen: 550000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 5.70130395889 test_h0_col_norms_mean: 3.63394594193 test_h0_col_norms_min: 2.06413507462 test_h0_row_norms_max: 5.63436841965 test_h0_row_norms_mean: 2.84383249283 test_h0_row_norms_min: 0.0585759952664 test_h1_col_norms_max: 5.33032464981 test_h1_col_norms_mean: 3.41074442863 test_h1_col_norms_min: 1.54273200035 test_h1_row_norms_max: 6.95094776154 test_h1_row_norms_mean: 4.84889364243 test_h1_row_norms_min: 2.85255265236 test_objective: 1.09497404099 test_term_0: 0.145816907287 test_term_1_weight_decay: 0.94915664196 test_y_col_norms_max: 4.7894949913 test_y_col_norms_mean: 4.32798671722 test_y_col_norms_min: 3.85334467888 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.976011812687 test_y_min_max_class: 0.533329129219 test_y_misclass: 0.0402000099421 test_y_nll: 0.145816907287 test_y_row_norms_max: 1.16048634052 test_y_row_norms_mean: 0.407433569431 test_y_row_norms_min: 0.0134850135073 train_h0_col_norms_max: 5.70130395889 train_h0_col_norms_mean: 3.63395118713 train_h0_col_norms_min: 2.06412315369 train_h0_row_norms_max: 5.63437128067 train_h0_row_norms_mean: 2.84384655952 train_h0_row_norms_min: 0.0585760846734 train_h1_col_norms_max: 5.33032464981 train_h1_col_norms_mean: 3.4107298851 train_h1_col_norms_min: 1.54273247719 train_h1_row_norms_max: 6.95091104507 train_h1_row_norms_mean: 4.84888315201 train_h1_row_norms_min: 2.8525583744 train_objective: 1.04947304726 train_term_0: 0.10031542182 train_term_1_weight_decay: 0.949151813984 train_y_col_norms_max: 4.78949642181 train_y_col_norms_mean: 4.32800579071 train_y_col_norms_min: 3.85332846642 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.977987766266 train_y_min_max_class: 0.520123898983 train_y_misclass: 0.0307400058955 train_y_nll: 0.10031542182 train_y_row_norms_max: 1.16048312187 train_y_row_norms_mean: 0.407431900501 train_y_row_norms_min: 0.0134850600734 valid_h0_col_norms_max: 5.70130395889 valid_h0_col_norms_mean: 3.63394594193 valid_h0_col_norms_min: 2.06413507462 valid_h0_row_norms_max: 5.63436841965 valid_h0_row_norms_mean: 2.84383249283 valid_h0_row_norms_min: 0.0585759952664 valid_h1_col_norms_max: 5.33032464981 valid_h1_col_norms_mean: 3.41074442863 valid_h1_col_norms_min: 1.54273200035 valid_h1_row_norms_max: 6.95094776154 valid_h1_row_norms_mean: 4.84889364243 valid_h1_row_norms_min: 2.85255265236 valid_objective: 1.09732854366 valid_term_0: 0.148171290755 valid_term_1_weight_decay: 0.94915664196 valid_y_col_norms_max: 4.7894949913 valid_y_col_norms_mean: 4.32798671722 valid_y_col_norms_min: 3.85334467888 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.977250099182 valid_y_min_max_class: 0.51136559248 valid_y_misclass: 0.0399999879301 valid_y_nll: 0.148171290755 valid_y_row_norms_max: 1.16048634052 valid_y_row_norms_mean: 0.407433569431 valid_y_row_norms_min: 0.0134850135073 Time this epoch: 3.333940 seconds Monitoring step: Epochs seen: 12 Batches seen: 6000 Examples seen: 600000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 5.4267373085 test_h0_col_norms_mean: 3.48842096329 test_h0_col_norms_min: 1.96254551411 test_h0_row_norms_max: 5.41478538513 test_h0_row_norms_mean: 2.7299387455 test_h0_row_norms_min: 0.0784849375486 test_h1_col_norms_max: 5.06470775604 test_h1_col_norms_mean: 3.24842214584 test_h1_col_norms_min: 1.46678352356 test_h1_row_norms_max: 6.60853338242 test_h1_row_norms_mean: 4.6184220314 test_h1_row_norms_min: 2.71205830574 test_objective: 0.985752701759 test_term_0: 0.119499914348 test_term_1_weight_decay: 0.866252303123 test_y_col_norms_max: 4.76992559433 test_y_col_norms_mean: 4.27050018311 test_y_col_norms_min: 3.78093886375 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.979267477989 test_y_min_max_class: 0.543375730515 test_y_misclass: 0.0315999910235 test_y_nll: 0.119499914348 test_y_row_norms_max: 0.912945210934 test_y_row_norms_mean: 0.402993023396 test_y_row_norms_min: 0.0216930937022 train_h0_col_norms_max: 5.42672777176 train_h0_col_norms_mean: 3.48842120171 train_h0_col_norms_min: 1.96254348755 train_h0_row_norms_max: 5.41479063034 train_h0_row_norms_mean: 2.72992825508 train_h0_row_norms_min: 0.0784846991301 train_h1_col_norms_max: 5.06472110748 train_h1_col_norms_mean: 3.24842524529 train_h1_col_norms_min: 1.46679055691 train_h1_row_norms_max: 6.60850334167 train_h1_row_norms_mean: 4.61840820312 train_h1_row_norms_min: 2.71205258369 train_objective: 0.922969102859 train_term_0: 0.0567165091634 train_term_1_weight_decay: 0.866256058216 train_y_col_norms_max: 4.76994085312 train_y_col_norms_mean: 4.27052545547 train_y_col_norms_min: 3.7809548378 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.982750058174 train_y_min_max_class: 0.558061778545 train_y_misclass: 0.018240025267 train_y_nll: 0.0567165091634 train_y_row_norms_max: 0.912941157818 train_y_row_norms_mean: 0.402991384268 train_y_row_norms_min: 0.0216932129115 valid_h0_col_norms_max: 5.4267373085 valid_h0_col_norms_mean: 3.48842096329 valid_h0_col_norms_min: 1.96254551411 valid_h0_row_norms_max: 5.41478538513 valid_h0_row_norms_mean: 2.7299387455 valid_h0_row_norms_min: 0.0784849375486 valid_h1_col_norms_max: 5.06470775604 valid_h1_col_norms_mean: 3.24842214584 valid_h1_col_norms_min: 1.46678352356 valid_h1_row_norms_max: 6.60853338242 valid_h1_row_norms_mean: 4.6184220314 valid_h1_row_norms_min: 2.71205830574 valid_objective: 0.983159482479 valid_term_0: 0.11690659076 valid_term_1_weight_decay: 0.866252303123 valid_y_col_norms_max: 4.76992559433 valid_y_col_norms_mean: 4.27050018311 valid_y_col_norms_min: 3.78093886375 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.981662929058 valid_y_min_max_class: 0.533038794994 valid_y_misclass: 0.0296999812126 valid_y_nll: 0.11690659076 valid_y_row_norms_max: 0.912945210934 valid_y_row_norms_mean: 0.402993023396 valid_y_row_norms_min: 0.0216930937022 Time this epoch: 3.286931 seconds Monitoring step: Epochs seen: 13 Batches seen: 6500 Examples seen: 650000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 5.16044139862 test_h0_col_norms_mean: 3.34162855148 test_h0_col_norms_min: 1.86588740349 test_h0_row_norms_max: 5.20599794388 test_h0_row_norms_mean: 2.61515665054 test_h0_row_norms_min: 0.0764672607183 test_h1_col_norms_max: 4.8263502121 test_h1_col_norms_mean: 3.09343934059 test_h1_col_norms_min: 1.39710497856 test_h1_row_norms_max: 6.2830324173 test_h1_row_norms_mean: 4.39829969406 test_h1_row_norms_min: 2.57847547531 test_objective: 0.8922701478 test_term_0: 0.102732278407 test_term_1_weight_decay: 0.789536893368 test_y_col_norms_max: 4.68681240082 test_y_col_norms_mean: 4.23078680038 test_y_col_norms_min: 3.78408479691 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.983542442322 test_y_min_max_class: 0.593747019768 test_y_misclass: 0.0273999907076 test_y_nll: 0.102732278407 test_y_row_norms_max: 0.966835141182 test_y_row_norms_mean: 0.398707449436 test_y_row_norms_min: 0.0218474734575 train_h0_col_norms_max: 5.16042280197 train_h0_col_norms_mean: 3.34164571762 train_h0_col_norms_min: 1.86587870121 train_h0_row_norms_max: 5.2060174942 train_h0_row_norms_mean: 2.61516785622 train_h0_row_norms_min: 0.0764675214887 train_h1_col_norms_max: 4.82632637024 train_h1_col_norms_mean: 3.09345006943 train_h1_col_norms_min: 1.39711165428 train_h1_row_norms_max: 6.28304100037 train_h1_row_norms_mean: 4.39832401276 train_h1_row_norms_min: 2.5784881115 train_objective: 0.829474568367 train_term_0: 0.0399360619485 train_term_1_weight_decay: 0.789532542229 train_y_col_norms_max: 4.6868262291 train_y_col_norms_mean: 4.23076534271 train_y_col_norms_min: 3.78406834602 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.987834095955 train_y_min_max_class: 0.609752178192 train_y_misclass: 0.0125200273469 train_y_nll: 0.0399360619485 train_y_row_norms_max: 0.96683973074 train_y_row_norms_mean: 0.398709416389 train_y_row_norms_min: 0.0218475684524 valid_h0_col_norms_max: 5.16044139862 valid_h0_col_norms_mean: 3.34162855148 valid_h0_col_norms_min: 1.86588740349 valid_h0_row_norms_max: 5.20599794388 valid_h0_row_norms_mean: 2.61515665054 valid_h0_row_norms_min: 0.0764672607183 valid_h1_col_norms_max: 4.8263502121 valid_h1_col_norms_mean: 3.09343934059 valid_h1_col_norms_min: 1.39710497856 valid_h1_row_norms_max: 6.2830324173 valid_h1_row_norms_mean: 4.39829969406 valid_h1_row_norms_min: 2.57847547531 valid_objective: 0.903808116913 valid_term_0: 0.114270374179 valid_term_1_weight_decay: 0.789536893368 valid_y_col_norms_max: 4.68681240082 valid_y_col_norms_mean: 4.23078680038 valid_y_col_norms_min: 3.78408479691 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.984707713127 valid_y_min_max_class: 0.566586375237 valid_y_misclass: 0.028899980709 valid_y_nll: 0.114270374179 valid_y_row_norms_max: 0.966835141182 valid_y_row_norms_mean: 0.398707449436 valid_y_row_norms_min: 0.0218474734575 Time this epoch: 3.373106 seconds Monitoring step: Epochs seen: 14 Batches seen: 7000 Examples seen: 700000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 4.90614843369 test_h0_col_norms_mean: 3.19672346115 test_h0_col_norms_min: 1.77398645878 test_h0_row_norms_max: 4.96580123901 test_h0_row_norms_mean: 2.50189256668 test_h0_row_norms_min: 0.0782802626491 test_h1_col_norms_max: 4.59008312225 test_h1_col_norms_mean: 2.94533538818 test_h1_col_norms_min: 1.32907187939 test_h1_row_norms_max: 5.97338581085 test_h1_row_norms_mean: 4.18786859512 test_h1_row_norms_min: 2.45148181915 test_objective: 0.819695711136 test_term_0: 0.100928872824 test_term_1_weight_decay: 0.718766570091 test_y_col_norms_max: 4.5962023735 test_y_col_norms_mean: 4.16727113724 test_y_col_norms_min: 3.66778349876 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.987643420696 test_y_min_max_class: 0.622340202332 test_y_misclass: 0.0252999924123 test_y_nll: 0.100928872824 test_y_row_norms_max: 0.958882212639 test_y_row_norms_mean: 0.392418205738 test_y_row_norms_min: 0.0207168832421 train_h0_col_norms_max: 4.90617132187 train_h0_col_norms_mean: 3.19671821594 train_h0_col_norms_min: 1.77399635315 train_h0_row_norms_max: 4.96578741074 train_h0_row_norms_mean: 2.50188994408 train_h0_row_norms_min: 0.0782798752189 train_h1_col_norms_max: 4.5900592804 train_h1_col_norms_mean: 2.94533443451 train_h1_col_norms_min: 1.32906579971 train_h1_row_norms_max: 5.97335529327 train_h1_row_norms_mean: 4.18784570694 train_h1_row_norms_min: 2.45149064064 train_objective: 0.745359420776 train_term_0: 0.0265923049301 train_term_1_weight_decay: 0.718770325184 train_y_col_norms_max: 4.5962138176 train_y_col_norms_mean: 4.16729164124 train_y_col_norms_min: 3.66780090332 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.990943193436 train_y_min_max_class: 0.66484606266 train_y_misclass: 0.00918001402169 train_y_nll: 0.0265923049301 train_y_row_norms_max: 0.95888376236 train_y_row_norms_mean: 0.392418503761 train_y_row_norms_min: 0.0207169353962 valid_h0_col_norms_max: 4.90614843369 valid_h0_col_norms_mean: 3.19672346115 valid_h0_col_norms_min: 1.77398645878 valid_h0_row_norms_max: 4.96580123901 valid_h0_row_norms_mean: 2.50189256668 valid_h0_row_norms_min: 0.0782802626491 valid_h1_col_norms_max: 4.59008312225 valid_h1_col_norms_mean: 2.94533538818 valid_h1_col_norms_min: 1.32907187939 valid_h1_row_norms_max: 5.97338581085 valid_h1_row_norms_mean: 4.18786859512 valid_h1_row_norms_min: 2.45148181915 valid_objective: 0.827313005924 valid_term_0: 0.108545988798 valid_term_1_weight_decay: 0.718766570091 valid_y_col_norms_max: 4.5962023735 valid_y_col_norms_mean: 4.16727113724 valid_y_col_norms_min: 3.66778349876 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.987360954285 valid_y_min_max_class: 0.601630806923 valid_y_misclass: 0.0260999873281 valid_y_nll: 0.108545988798 valid_y_row_norms_max: 0.958882212639 valid_y_row_norms_mean: 0.392418205738 valid_y_row_norms_min: 0.0207168832421 Time this epoch: 3.270202 seconds Monitoring step: Epochs seen: 15 Batches seen: 7500 Examples seen: 750000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 4.66921758652 test_h0_col_norms_mean: 3.05542969704 test_h0_col_norms_min: 1.68660902977 test_h0_row_norms_max: 4.75306463242 test_h0_row_norms_mean: 2.39132237434 test_h0_row_norms_min: 0.0765107423067 test_h1_col_norms_max: 4.3710064888 test_h1_col_norms_mean: 2.8040626049 test_h1_col_norms_min: 1.26379609108 test_h1_row_norms_max: 5.67917490005 test_h1_row_norms_mean: 3.98712182045 test_h1_row_norms_min: 2.33073425293 test_objective: 0.737741053104 test_term_0: 0.083799123764 test_term_1_weight_decay: 0.653942167759 test_y_col_norms_max: 4.54380941391 test_y_col_norms_mean: 4.11158180237 test_y_col_norms_min: 3.66944622993 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.987535178661 test_y_min_max_class: 0.639883577824 test_y_misclass: 0.0226999949664 test_y_nll: 0.083799123764 test_y_row_norms_max: 0.992673635483 test_y_row_norms_mean: 0.386759877205 test_y_row_norms_min: 0.0214904490858 train_h0_col_norms_max: 4.66919612885 train_h0_col_norms_mean: 3.05544400215 train_h0_col_norms_min: 1.68661606312 train_h0_row_norms_max: 4.75308895111 train_h0_row_norms_mean: 2.39132881165 train_h0_row_norms_min: 0.0765103250742 train_h1_col_norms_max: 4.37101602554 train_h1_col_norms_mean: 2.80404901505 train_h1_col_norms_min: 1.26379692554 train_h1_row_norms_max: 5.67914772034 train_h1_row_norms_mean: 3.98710203171 train_h1_row_norms_min: 2.33073854446 train_objective: 0.66691416502 train_term_0: 0.0129722505808 train_term_1_weight_decay: 0.653943121433 train_y_col_norms_max: 4.54378795624 train_y_col_norms_mean: 4.11155748367 train_y_col_norms_min: 3.66946792603 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.993395149708 train_y_min_max_class: 0.715351760387 train_y_misclass: 0.00405999692157 train_y_nll: 0.0129722505808 train_y_row_norms_max: 0.992678523064 train_y_row_norms_mean: 0.386759728193 train_y_row_norms_min: 0.0214903373271 valid_h0_col_norms_max: 4.66921758652 valid_h0_col_norms_mean: 3.05542969704 valid_h0_col_norms_min: 1.68660902977 valid_h0_row_norms_max: 4.75306463242 valid_h0_row_norms_mean: 2.39132237434 valid_h0_row_norms_min: 0.0765107423067 valid_h1_col_norms_max: 4.3710064888 valid_h1_col_norms_mean: 2.8040626049 valid_h1_col_norms_min: 1.26379609108 valid_h1_row_norms_max: 5.67917490005 valid_h1_row_norms_mean: 3.98712182045 valid_h1_row_norms_min: 2.33073425293 valid_objective: 0.734254300594 valid_term_0: 0.0803121104836 valid_term_1_weight_decay: 0.653942167759 valid_y_col_norms_max: 4.54380941391 valid_y_col_norms_mean: 4.11158180237 valid_y_col_norms_min: 3.66944622993 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.987278044224 valid_y_min_max_class: 0.594924449921 valid_y_misclass: 0.0219999905676 valid_y_nll: 0.0803121104836 valid_y_row_norms_max: 0.992673635483 valid_y_row_norms_mean: 0.386759877205 valid_y_row_norms_min: 0.0214904490858 Time this epoch: 3.281498 seconds Monitoring step: Epochs seen: 16 Batches seen: 8000 Examples seen: 800000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 4.46374130249 test_h0_col_norms_mean: 2.9173309803 test_h0_col_norms_min: 1.60354030132 test_h0_row_norms_max: 4.55304861069 test_h0_row_norms_mean: 2.28333234787 test_h0_row_norms_min: 0.0760971903801 test_h1_col_norms_max: 4.15992879868 test_h1_col_norms_mean: 2.66938233376 test_h1_col_norms_min: 1.20336544514 test_h1_row_norms_max: 5.3994641304 test_h1_row_norms_mean: 3.79574894905 test_h1_row_norms_min: 2.21593642235 test_objective: 0.674262106419 test_term_0: 0.0797682702541 test_term_1_weight_decay: 0.594494223595 test_y_col_norms_max: 4.41636514664 test_y_col_norms_mean: 4.05042076111 test_y_col_norms_min: 3.58171629906 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.986650168896 test_y_min_max_class: 0.618766665459 test_y_misclass: 0.0221999883652 test_y_nll: 0.0797682702541 test_y_row_norms_max: 0.987005531788 test_y_row_norms_mean: 0.380280554295 test_y_row_norms_min: 0.0215586218983 train_h0_col_norms_max: 4.46375417709 train_h0_col_norms_mean: 2.91732239723 train_h0_col_norms_min: 1.60354280472 train_h0_row_norms_max: 4.55305957794 train_h0_row_norms_mean: 2.2833340168 train_h0_row_norms_min: 0.0760968104005 train_h1_col_norms_max: 4.159927845 train_h1_col_norms_mean: 2.66937685013 train_h1_col_norms_min: 1.20335972309 train_h1_row_norms_max: 5.39946508408 train_h1_row_norms_mean: 3.79574465752 train_h1_row_norms_min: 2.21593785286 train_objective: 0.604817152023 train_term_0: 0.0103233894333 train_term_1_weight_decay: 0.594495952129 train_y_col_norms_max: 4.41635942459 train_y_col_norms_mean: 4.05040311813 train_y_col_norms_min: 3.58173537254 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.993671536446 train_y_min_max_class: 0.732953190804 train_y_misclass: 0.00291999848559 train_y_nll: 0.0103233894333 train_y_row_norms_max: 0.987000524998 train_y_row_norms_mean: 0.380278617144 train_y_row_norms_min: 0.0215585716069 valid_h0_col_norms_max: 4.46374130249 valid_h0_col_norms_mean: 2.9173309803 valid_h0_col_norms_min: 1.60354030132 valid_h0_row_norms_max: 4.55304861069 valid_h0_row_norms_mean: 2.28333234787 valid_h0_row_norms_min: 0.0760971903801 valid_h1_col_norms_max: 4.15992879868 valid_h1_col_norms_mean: 2.66938233376 valid_h1_col_norms_min: 1.20336544514 valid_h1_row_norms_max: 5.3994641304 valid_h1_row_norms_mean: 3.79574894905 valid_h1_row_norms_min: 2.21593642235 valid_objective: 0.67914390564 valid_term_0: 0.0846498459578 valid_term_1_weight_decay: 0.594494223595 valid_y_col_norms_max: 4.41636514664 valid_y_col_norms_mean: 4.05042076111 valid_y_col_norms_min: 3.58171629906 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.987098455429 valid_y_min_max_class: 0.597771346569 valid_y_misclass: 0.0203999932855 valid_y_nll: 0.0846498459578 valid_y_row_norms_max: 0.987005531788 valid_y_row_norms_mean: 0.380280554295 valid_y_row_norms_min: 0.0215586218983 Time this epoch: 3.317685 seconds Monitoring step: Epochs seen: 17 Batches seen: 8500 Examples seen: 850000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 4.26407384872 test_h0_col_norms_mean: 2.78463411331 test_h0_col_norms_min: 1.52455866337 test_h0_row_norms_max: 4.34796571732 test_h0_row_norms_mean: 2.17960953712 test_h0_row_norms_min: 0.0764012187719 test_h1_col_norms_max: 3.95875430107 test_h1_col_norms_mean: 2.54128909111 test_h1_col_norms_min: 1.14473068714 test_h1_row_norms_max: 5.13351964951 test_h1_row_norms_mean: 3.61375975609 test_h1_row_norms_min: 2.1067969799 test_objective: 0.610892295837 test_term_0: 0.0704278945923 test_term_1_weight_decay: 0.540464937687 test_y_col_norms_max: 4.3217663765 test_y_col_norms_mean: 4.00428628922 test_y_col_norms_min: 3.53744649887 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.988494455814 test_y_min_max_class: 0.640034735203 test_y_misclass: 0.0186999924481 test_y_nll: 0.0704278945923 test_y_row_norms_max: 0.994636058807 test_y_row_norms_mean: 0.3746727705 test_y_row_norms_min: 0.0214648172259 train_h0_col_norms_max: 4.26409387589 train_h0_col_norms_mean: 2.78462028503 train_h0_col_norms_min: 1.52456390858 train_h0_row_norms_max: 4.34795570374 train_h0_row_norms_mean: 2.17961931229 train_h0_row_norms_min: 0.0764016136527 train_h1_col_norms_max: 3.95873188972 train_h1_col_norms_mean: 2.54130077362 train_h1_col_norms_min: 1.14473164082 train_h1_row_norms_max: 5.13353729248 train_h1_row_norms_mean: 3.61374282837 train_h1_row_norms_min: 2.10678911209 train_objective: 0.5477257967 train_term_0: 0.007261632476 train_term_1_weight_decay: 0.540465056896 train_y_col_norms_max: 4.32178735733 train_y_col_norms_mean: 4.00426435471 train_y_col_norms_min: 3.5374417305 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.995666265488 train_y_min_max_class: 0.789768993855 train_y_misclass: 0.00194000091869 train_y_nll: 0.007261632476 train_y_row_norms_max: 0.994630157948 train_y_row_norms_mean: 0.374674469233 train_y_row_norms_min: 0.0214648637921 valid_h0_col_norms_max: 4.26407384872 valid_h0_col_norms_mean: 2.78463411331 valid_h0_col_norms_min: 1.52455866337 valid_h0_row_norms_max: 4.34796571732 valid_h0_row_norms_mean: 2.17960953712 valid_h0_row_norms_min: 0.0764012187719 valid_h1_col_norms_max: 3.95875430107 valid_h1_col_norms_mean: 2.54128909111 valid_h1_col_norms_min: 1.14473068714 valid_h1_row_norms_max: 5.13351964951 valid_h1_row_norms_mean: 3.61375975609 valid_h1_row_norms_min: 2.1067969799 valid_objective: 0.617604732513 valid_term_0: 0.0771402940154 valid_term_1_weight_decay: 0.540464937687 valid_y_col_norms_max: 4.3217663765 valid_y_col_norms_mean: 4.00428628922 valid_y_col_norms_min: 3.53744649887 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989329993725 valid_y_min_max_class: 0.605314671993 valid_y_misclass: 0.0208999905735 valid_y_nll: 0.0771402940154 valid_y_row_norms_max: 0.994636058807 valid_y_row_norms_mean: 0.3746727705 valid_y_row_norms_min: 0.0214648172259 Time this epoch: 3.269546 seconds Monitoring step: Epochs seen: 18 Batches seen: 9000 Examples seen: 900000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 4.07507371902 test_h0_col_norms_mean: 2.65655350685 test_h0_col_norms_min: 1.44946885109 test_h0_row_norms_max: 4.15038585663 test_h0_row_norms_mean: 2.07941555977 test_h0_row_norms_min: 0.0857979208231 test_h1_col_norms_max: 3.77103662491 test_h1_col_norms_mean: 2.41892313957 test_h1_col_norms_min: 1.08750927448 test_h1_row_norms_max: 4.88069534302 test_h1_row_norms_mean: 3.43979096413 test_h1_row_norms_min: 2.00302839279 test_objective: 0.562726557255 test_term_0: 0.0717355385423 test_term_1_weight_decay: 0.490990847349 test_y_col_norms_max: 4.28208780289 test_y_col_norms_mean: 3.93249392509 test_y_col_norms_min: 3.48496580124 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.990153551102 test_y_min_max_class: 0.659395575523 test_y_misclass: 0.0185999944806 test_y_nll: 0.0717355385423 test_y_row_norms_max: 0.942749202251 test_y_row_norms_mean: 0.367405802011 test_y_row_norms_min: 0.019349604845 train_h0_col_norms_max: 4.07505750656 train_h0_col_norms_mean: 2.65656781197 train_h0_col_norms_min: 1.44946610928 train_h0_row_norms_max: 4.15039014816 train_h0_row_norms_mean: 2.07942199707 train_h0_row_norms_min: 0.0857974886894 train_h1_col_norms_max: 3.77104258537 train_h1_col_norms_mean: 2.41892194748 train_h1_col_norms_min: 1.08750891685 train_h1_row_norms_max: 4.88067293167 train_h1_row_norms_mean: 3.43978619576 train_h1_row_norms_min: 2.00302529335 train_objective: 0.496095150709 train_term_0: 0.00510408030823 train_term_1_weight_decay: 0.490992516279 train_y_col_norms_max: 4.28208827972 train_y_col_norms_mean: 3.93247795105 train_y_col_norms_min: 3.48496460915 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996478378773 train_y_min_max_class: 0.825236082077 train_y_misclass: 0.00105999980588 train_y_nll: 0.00510408030823 train_y_row_norms_max: 0.942744672298 train_y_row_norms_mean: 0.367404073477 train_y_row_norms_min: 0.0193495322019 valid_h0_col_norms_max: 4.07507371902 valid_h0_col_norms_mean: 2.65655350685 valid_h0_col_norms_min: 1.44946885109 valid_h0_row_norms_max: 4.15038585663 valid_h0_row_norms_mean: 2.07941555977 valid_h0_row_norms_min: 0.0857979208231 valid_h1_col_norms_max: 3.77103662491 valid_h1_col_norms_mean: 2.41892313957 valid_h1_col_norms_min: 1.08750927448 valid_h1_row_norms_max: 4.88069534302 valid_h1_row_norms_mean: 3.43979096413 valid_h1_row_norms_min: 2.00302839279 valid_objective: 0.568551659584 valid_term_0: 0.0775607377291 valid_term_1_weight_decay: 0.490990847349 valid_y_col_norms_max: 4.28208780289 valid_y_col_norms_mean: 3.93249392509 valid_y_col_norms_min: 3.48496580124 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989968895912 valid_y_min_max_class: 0.620238602161 valid_y_misclass: 0.0208999887109 valid_y_nll: 0.0775607377291 valid_y_row_norms_max: 0.942749202251 valid_y_row_norms_mean: 0.367405802011 valid_y_row_norms_min: 0.019349604845 Time this epoch: 3.304628 seconds Monitoring step: Epochs seen: 19 Batches seen: 9500 Examples seen: 950000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 3.88326621056 test_h0_col_norms_mean: 2.53280711174 test_h0_col_norms_min: 1.37807917595 test_h0_row_norms_max: 3.96461653709 test_h0_row_norms_mean: 1.98260319233 test_h0_row_norms_min: 0.09099239856 test_h1_col_norms_max: 3.58734297752 test_h1_col_norms_mean: 2.30225491524 test_h1_col_norms_min: 1.03453934193 test_h1_row_norms_max: 4.64029741287 test_h1_row_norms_mean: 3.27396249771 test_h1_row_norms_min: 1.90437150002 test_objective: 0.510573804379 test_term_0: 0.0647685080767 test_term_1_weight_decay: 0.445804834366 test_y_col_norms_max: 4.17118215561 test_y_col_norms_mean: 3.85513329506 test_y_col_norms_min: 3.38714289665 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989724636078 test_y_min_max_class: 0.680286705494 test_y_misclass: 0.0178999938071 test_y_nll: 0.0647685080767 test_y_row_norms_max: 0.922487914562 test_y_row_norms_mean: 0.359323531389 test_y_row_norms_min: 0.0180249232799 train_h0_col_norms_max: 3.883248806 train_h0_col_norms_mean: 2.53280425072 train_h0_col_norms_min: 1.37807655334 train_h0_row_norms_max: 3.96463823318 train_h0_row_norms_mean: 1.982614398 train_h0_row_norms_min: 0.090992718935 train_h1_col_norms_max: 3.58735847473 train_h1_col_norms_mean: 2.30224943161 train_h1_col_norms_min: 1.03453481197 train_h1_row_norms_max: 4.64032030106 train_h1_row_norms_mean: 3.2739636898 train_h1_row_norms_min: 1.90436935425 train_objective: 0.449831366539 train_term_0: 0.00402596499771 train_term_1_weight_decay: 0.445802211761 train_y_col_norms_max: 4.17116117477 train_y_col_norms_mean: 3.85511660576 train_y_col_norms_min: 3.3871281147 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996860563755 train_y_min_max_class: 0.843070626259 train_y_misclass: 0.000819999666419 train_y_nll: 0.00402596499771 train_y_row_norms_max: 0.922493100166 train_y_row_norms_mean: 0.359325319529 train_y_row_norms_min: 0.0180248413235 valid_h0_col_norms_max: 3.88326621056 valid_h0_col_norms_mean: 2.53280711174 valid_h0_col_norms_min: 1.37807917595 valid_h0_row_norms_max: 3.96461653709 valid_h0_row_norms_mean: 1.98260319233 valid_h0_row_norms_min: 0.09099239856 valid_h1_col_norms_max: 3.58734297752 valid_h1_col_norms_mean: 2.30225491524 valid_h1_col_norms_min: 1.03453934193 valid_h1_row_norms_max: 4.64029741287 valid_h1_row_norms_mean: 3.27396249771 valid_h1_row_norms_min: 1.90437150002 valid_objective: 0.517447412014 valid_term_0: 0.0716420337558 valid_term_1_weight_decay: 0.445804834366 valid_y_col_norms_max: 4.17118215561 valid_y_col_norms_mean: 3.85513329506 valid_y_col_norms_min: 3.38714289665 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.990103065968 valid_y_min_max_class: 0.65864700079 valid_y_misclass: 0.019399991259 valid_y_nll: 0.0716420337558 valid_y_row_norms_max: 0.922487914562 valid_y_row_norms_mean: 0.359323531389 valid_y_row_norms_min: 0.0180249232799 Time this epoch: 3.352973 seconds Monitoring step: Epochs seen: 20 Batches seen: 10000 Examples seen: 1000000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 3.69830107689 test_h0_col_norms_mean: 2.41490340233 test_h0_col_norms_min: 1.31020438671 test_h0_row_norms_max: 3.78174734116 test_h0_row_norms_mean: 1.89037334919 test_h0_row_norms_min: 0.0872991830111 test_h1_col_norms_max: 3.41410470009 test_h1_col_norms_mean: 2.19139242172 test_h1_col_norms_min: 0.983708977699 test_h1_row_norms_max: 4.41174936295 test_h1_row_norms_mean: 3.11639881134 test_h1_row_norms_min: 1.81057536602 test_objective: 0.4696611166 test_term_0: 0.064779728651 test_term_1_weight_decay: 0.404881685972 test_y_col_norms_max: 4.09565019608 test_y_col_norms_mean: 3.78634428978 test_y_col_norms_min: 3.3164498806 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.98881983757 test_y_min_max_class: 0.653255581856 test_y_misclass: 0.017599998042 test_y_nll: 0.064779728651 test_y_row_norms_max: 0.925839364529 test_y_row_norms_mean: 0.351857930422 test_y_row_norms_min: 0.0175057649612 train_h0_col_norms_max: 3.69831848145 train_h0_col_norms_mean: 2.41490244865 train_h0_col_norms_min: 1.31020689011 train_h0_row_norms_max: 3.78176569939 train_h0_row_norms_mean: 1.89036512375 train_h0_row_norms_min: 0.0872987210751 train_h1_col_norms_max: 3.41411972046 train_h1_col_norms_mean: 2.19141077995 train_h1_col_norms_min: 0.983713150024 train_h1_row_norms_max: 4.41172409058 train_h1_row_norms_mean: 3.11639785767 train_h1_row_norms_min: 1.81056690216 train_objective: 0.409165471792 train_term_0: 0.00428416905925 train_term_1_weight_decay: 0.404880315065 train_y_col_norms_max: 4.09566497803 train_y_col_norms_mean: 3.78633141518 train_y_col_norms_min: 3.31643605232 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996947467327 train_y_min_max_class: 0.855619430542 train_y_misclass: 0.000879999657627 train_y_nll: 0.00428416905925 train_y_row_norms_max: 0.925839066505 train_y_row_norms_mean: 0.351859807968 train_y_row_norms_min: 0.0175057388842 valid_h0_col_norms_max: 3.69830107689 valid_h0_col_norms_mean: 2.41490340233 valid_h0_col_norms_min: 1.31020438671 valid_h0_row_norms_max: 3.78174734116 valid_h0_row_norms_mean: 1.89037334919 valid_h0_row_norms_min: 0.0872991830111 valid_h1_col_norms_max: 3.41410470009 valid_h1_col_norms_mean: 2.19139242172 valid_h1_col_norms_min: 0.983708977699 valid_h1_row_norms_max: 4.41174936295 valid_h1_row_norms_mean: 3.11639881134 valid_h1_row_norms_min: 1.81057536602 valid_objective: 0.475686132908 valid_term_0: 0.0708047524095 valid_term_1_weight_decay: 0.404881685972 valid_y_col_norms_max: 4.09565019608 valid_y_col_norms_mean: 3.78634428978 valid_y_col_norms_min: 3.3164498806 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989249825478 valid_y_min_max_class: 0.616850614548 valid_y_misclass: 0.0192999914289 valid_y_nll: 0.0708047524095 valid_y_row_norms_max: 0.925839364529 valid_y_row_norms_mean: 0.351857930422 valid_y_row_norms_min: 0.0175057649612 Time this epoch: 3.278321 seconds Monitoring step: Epochs seen: 21 Batches seen: 10500 Examples seen: 1050000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 3.52711653709 test_h0_col_norms_mean: 2.30246567726 test_h0_col_norms_min: 1.2456703186 test_h0_row_norms_max: 3.60426926613 test_h0_row_norms_mean: 1.80239653587 test_h0_row_norms_min: 0.0854785442352 test_h1_col_norms_max: 3.24995541573 test_h1_col_norms_mean: 2.08604121208 test_h1_col_norms_min: 0.935500979424 test_h1_row_norms_max: 4.19445514679 test_h1_row_norms_mean: 2.96661686897 test_h1_row_norms_min: 1.72139751911 test_objective: 0.435198038816 test_term_0: 0.0674102455378 test_term_1_weight_decay: 0.367787539959 test_y_col_norms_max: 4.01598834991 test_y_col_norms_mean: 3.72248363495 test_y_col_norms_min: 3.24742627144 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.988867938519 test_y_min_max_class: 0.670085370541 test_y_misclass: 0.0185999963433 test_y_nll: 0.0674102455378 test_y_row_norms_max: 0.902759611607 test_y_row_norms_mean: 0.344950795174 test_y_row_norms_min: 0.0167198460549 train_h0_col_norms_max: 3.52713274956 train_h0_col_norms_mean: 2.30245995522 train_h0_col_norms_min: 1.24567604065 train_h0_row_norms_max: 3.60428500175 train_h0_row_norms_mean: 1.80238819122 train_h0_row_norms_min: 0.0854781419039 train_h1_col_norms_max: 3.24996852875 train_h1_col_norms_mean: 2.08604311943 train_h1_col_norms_min: 0.935496866703 train_h1_row_norms_max: 4.19447517395 train_h1_row_norms_mean: 2.96663236618 train_h1_row_norms_min: 1.72139537334 train_objective: 0.372235387564 train_term_0: 0.00444769486785 train_term_1_weight_decay: 0.367789417505 train_y_col_norms_max: 4.01601171494 train_y_col_norms_mean: 3.7224612236 train_y_col_norms_min: 3.24741005898 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996587693691 train_y_min_max_class: 0.846141993999 train_y_misclass: 0.000799999630544 train_y_nll: 0.00444769486785 train_y_row_norms_max: 0.902763843536 train_y_row_norms_mean: 0.344951033592 train_y_row_norms_min: 0.0167199298739 valid_h0_col_norms_max: 3.52711653709 valid_h0_col_norms_mean: 2.30246567726 valid_h0_col_norms_min: 1.2456703186 valid_h0_row_norms_max: 3.60426926613 valid_h0_row_norms_mean: 1.80239653587 valid_h0_row_norms_min: 0.0854785442352 valid_h1_col_norms_max: 3.24995541573 valid_h1_col_norms_mean: 2.08604121208 valid_h1_col_norms_min: 0.935500979424 valid_h1_row_norms_max: 4.19445514679 valid_h1_row_norms_mean: 2.96661686897 valid_h1_row_norms_min: 1.72139751911 valid_objective: 0.439748078585 valid_term_0: 0.0719603598118 valid_term_1_weight_decay: 0.367787539959 valid_y_col_norms_max: 4.01598834991 valid_y_col_norms_mean: 3.72248363495 valid_y_col_norms_min: 3.24742627144 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989000380039 valid_y_min_max_class: 0.610602736473 valid_y_misclass: 0.0192999914289 valid_y_nll: 0.0719603598118 valid_y_row_norms_max: 0.902759611607 valid_y_row_norms_mean: 0.344950795174 valid_y_row_norms_min: 0.0167198460549 Time this epoch: 3.292022 seconds Monitoring step: Epochs seen: 22 Batches seen: 11000 Examples seen: 1100000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 3.36697125435 test_h0_col_norms_mean: 2.19646573067 test_h0_col_norms_min: 1.18431913853 test_h0_row_norms_max: 3.44316983223 test_h0_row_norms_mean: 1.71948647499 test_h0_row_norms_min: 0.0820252001286 test_h1_col_norms_max: 3.09565114975 test_h1_col_norms_mean: 1.98627471924 test_h1_col_norms_min: 0.889862000942 test_h1_row_norms_max: 3.98788499832 test_h1_row_norms_mean: 2.82480931282 test_h1_row_norms_min: 1.63661336899 test_objective: 0.394841223955 test_term_0: 0.0604076348245 test_term_1_weight_decay: 0.334433555603 test_y_col_norms_max: 4.01139307022 test_y_col_norms_mean: 3.67662405968 test_y_col_norms_min: 3.18460655212 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989187180996 test_y_min_max_class: 0.643190681934 test_y_misclass: 0.0180999971926 test_y_nll: 0.0604076348245 test_y_row_norms_max: 0.901866018772 test_y_row_norms_mean: 0.339687138796 test_y_row_norms_min: 0.01714236103 train_h0_col_norms_max: 3.36695551872 train_h0_col_norms_mean: 2.19647264481 train_h0_col_norms_min: 1.18431949615 train_h0_row_norms_max: 3.44315481186 train_h0_row_norms_mean: 1.7194788456 train_h0_row_norms_min: 0.0820248499513 train_h1_col_norms_max: 3.09563612938 train_h1_col_norms_mean: 1.98626804352 train_h1_col_norms_min: 0.889865934849 train_h1_row_norms_max: 3.98790216446 train_h1_row_norms_mean: 2.82479405403 train_h1_row_norms_min: 1.63662087917 train_objective: 0.337020277977 train_term_0: 0.00258666928858 train_term_1_weight_decay: 0.334432244301 train_y_col_norms_max: 4.01139307022 train_y_col_norms_mean: 3.67662858963 train_y_col_norms_min: 3.18459391594 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.997638344765 train_y_min_max_class: 0.894991517067 train_y_misclass: 0.000179999973625 train_y_nll: 0.00258666928858 train_y_row_norms_max: 0.901870131493 train_y_row_norms_mean: 0.339686959982 train_y_row_norms_min: 0.0171423424035 valid_h0_col_norms_max: 3.36697125435 valid_h0_col_norms_mean: 2.19646573067 valid_h0_col_norms_min: 1.18431913853 valid_h0_row_norms_max: 3.44316983223 valid_h0_row_norms_mean: 1.71948647499 valid_h0_row_norms_min: 0.0820252001286 valid_h1_col_norms_max: 3.09565114975 valid_h1_col_norms_mean: 1.98627471924 valid_h1_col_norms_min: 0.889862000942 valid_h1_row_norms_max: 3.98788499832 valid_h1_row_norms_mean: 2.82480931282 valid_h1_row_norms_min: 1.63661336899 valid_objective: 0.399684429169 valid_term_0: 0.065250813961 valid_term_1_weight_decay: 0.334433555603 valid_y_col_norms_max: 4.01139307022 valid_y_col_norms_mean: 3.67662405968 valid_y_col_norms_min: 3.18460655212 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989832878113 valid_y_min_max_class: 0.622855961323 valid_y_misclass: 0.0177999921143 valid_y_nll: 0.065250813961 valid_y_row_norms_max: 0.901866018772 valid_y_row_norms_mean: 0.339687138796 valid_y_row_norms_min: 0.01714236103 Time this epoch: 3.278771 seconds Monitoring step: Epochs seen: 23 Batches seen: 11500 Examples seen: 1150000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 3.2081348896 test_h0_col_norms_mean: 2.09370923042 test_h0_col_norms_min: 1.12598621845 test_h0_row_norms_max: 3.28446102142 test_h0_row_norms_mean: 1.63902020454 test_h0_row_norms_min: 0.0778670459986 test_h1_col_norms_max: 2.94689941406 test_h1_col_norms_mean: 1.89091038704 test_h1_col_norms_min: 0.846523106098 test_h1_row_norms_max: 3.79147481918 test_h1_row_norms_mean: 2.68924212456 test_h1_row_norms_min: 1.55600595474 test_objective: 0.359032511711 test_term_0: 0.0552162267268 test_term_1_weight_decay: 0.303816497326 test_y_col_norms_max: 3.92041349411 test_y_col_norms_mean: 3.61436057091 test_y_col_norms_min: 3.12086963654 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989588081837 test_y_min_max_class: 0.669209182262 test_y_misclass: 0.016299996525 test_y_nll: 0.0552162267268 test_y_row_norms_max: 0.909698069096 test_y_row_norms_mean: 0.333013266325 test_y_row_norms_min: 0.0162505507469 train_h0_col_norms_max: 3.20815110207 train_h0_col_norms_mean: 2.09371638298 train_h0_col_norms_min: 1.12598991394 train_h0_row_norms_max: 3.28445625305 train_h0_row_norms_mean: 1.63901221752 train_h0_row_norms_min: 0.0778671503067 train_h1_col_norms_max: 2.94688630104 train_h1_col_norms_mean: 1.89090168476 train_h1_col_norms_min: 0.846526682377 train_h1_row_norms_max: 3.79145789146 train_h1_row_norms_mean: 2.68924236298 train_h1_row_norms_min: 1.55599808693 train_objective: 0.306422680616 train_term_0: 0.00260642380454 train_term_1_weight_decay: 0.303818255663 train_y_col_norms_max: 3.92042994499 train_y_col_norms_mean: 3.61433935165 train_y_col_norms_min: 3.12087059021 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.997653722763 train_y_min_max_class: 0.897151112556 train_y_misclass: 0.000119999996969 train_y_nll: 0.00260642380454 train_y_row_norms_max: 0.909693837166 train_y_row_norms_mean: 0.333012223244 train_y_row_norms_min: 0.0162506196648 valid_h0_col_norms_max: 3.2081348896 valid_h0_col_norms_mean: 2.09370923042 valid_h0_col_norms_min: 1.12598621845 valid_h0_row_norms_max: 3.28446102142 valid_h0_row_norms_mean: 1.63902020454 valid_h0_row_norms_min: 0.0778670459986 valid_h1_col_norms_max: 2.94689941406 valid_h1_col_norms_mean: 1.89091038704 valid_h1_col_norms_min: 0.846523106098 valid_h1_row_norms_max: 3.79147481918 valid_h1_row_norms_mean: 2.68924212456 valid_h1_row_norms_min: 1.55600595474 valid_objective: 0.370760649443 valid_term_0: 0.0669444948435 valid_term_1_weight_decay: 0.303816497326 valid_y_col_norms_max: 3.92041349411 valid_y_col_norms_mean: 3.61436057091 valid_y_col_norms_min: 3.12086963654 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989882349968 valid_y_min_max_class: 0.639726042747 valid_y_misclass: 0.0178999956697 valid_y_nll: 0.0669444948435 valid_y_row_norms_max: 0.909698069096 valid_y_row_norms_mean: 0.333013266325 valid_y_row_norms_min: 0.0162505507469 Time this epoch: 3.283699 seconds Monitoring step: Epochs seen: 24 Batches seen: 12000 Examples seen: 1200000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 3.0553176403 test_h0_col_norms_mean: 1.99559020996 test_h0_col_norms_min: 1.07052779198 test_h0_row_norms_max: 3.1363966465 test_h0_row_norms_mean: 1.56222510338 test_h0_row_norms_min: 0.0742355883121 test_h1_col_norms_max: 2.80656790733 test_h1_col_norms_mean: 1.80023908615 test_h1_col_norms_min: 0.805699706078 test_h1_row_norms_max: 3.60472822189 test_h1_row_norms_mean: 2.5603313446 test_h1_row_norms_min: 1.47936725616 test_objective: 0.333813428879 test_term_0: 0.0577764734626 test_term_1_weight_decay: 0.276036947966 test_y_col_norms_max: 3.85618805885 test_y_col_norms_mean: 3.55425548553 test_y_col_norms_min: 3.04648113251 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989487826824 test_y_min_max_class: 0.649445652962 test_y_misclass: 0.0160999950022 test_y_nll: 0.0577764734626 test_y_row_norms_max: 0.918738484383 test_y_row_norms_mean: 0.326478481293 test_y_row_norms_min: 0.0155664272606 train_h0_col_norms_max: 3.055331707 train_h0_col_norms_mean: 1.99557840824 train_h0_col_norms_min: 1.07052719593 train_h0_row_norms_max: 3.1363966465 train_h0_row_norms_mean: 1.56223297119 train_h0_row_norms_min: 0.0742354020476 train_h1_col_norms_max: 2.80655431747 train_h1_col_norms_mean: 1.80023896694 train_h1_col_norms_min: 0.805703580379 train_h1_row_norms_max: 3.60474324226 train_h1_row_norms_mean: 2.56033945084 train_h1_row_norms_min: 1.47937119007 train_objective: 0.278264194727 train_term_0: 0.00222728447989 train_term_1_weight_decay: 0.276037305593 train_y_col_norms_max: 3.85618400574 train_y_col_norms_mean: 3.55425071716 train_y_col_norms_min: 3.04648280144 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.997917592525 train_y_min_max_class: 0.919997572899 train_y_misclass: 7.9999997979e-05 train_y_nll: 0.00222728447989 train_y_row_norms_max: 0.918739795685 train_y_row_norms_mean: 0.326479077339 train_y_row_norms_min: 0.0155664980412 valid_h0_col_norms_max: 3.0553176403 valid_h0_col_norms_mean: 1.99559020996 valid_h0_col_norms_min: 1.07052779198 valid_h0_row_norms_max: 3.1363966465 valid_h0_row_norms_mean: 1.56222510338 valid_h0_row_norms_min: 0.0742355883121 valid_h1_col_norms_max: 2.80656790733 valid_h1_col_norms_mean: 1.80023908615 valid_h1_col_norms_min: 0.805699706078 valid_h1_row_norms_max: 3.60472822189 valid_h1_row_norms_mean: 2.5603313446 valid_h1_row_norms_min: 1.47936725616 valid_objective: 0.338611006737 valid_term_0: 0.062574096024 valid_term_1_weight_decay: 0.276036947966 valid_y_col_norms_max: 3.85618805885 valid_y_col_norms_mean: 3.55425548553 valid_y_col_norms_min: 3.04648113251 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.98962688446 valid_y_min_max_class: 0.629032611847 valid_y_misclass: 0.0176999941468 valid_y_nll: 0.062574096024 valid_y_row_norms_max: 0.918738484383 valid_y_row_norms_mean: 0.326478481293 valid_y_row_norms_min: 0.0155664272606 Time this epoch: 3.319413 seconds Monitoring step: Epochs seen: 25 Batches seen: 12500 Examples seen: 1250000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.91628909111 test_h0_col_norms_mean: 1.90174818039 test_h0_col_norms_min: 1.01780152321 test_h0_row_norms_max: 2.98749780655 test_h0_row_norms_mean: 1.48878085613 test_h0_row_norms_min: 0.0709456577897 test_h1_col_norms_max: 2.6732199192 test_h1_col_norms_mean: 1.71399199963 test_h1_col_norms_min: 0.766523241997 test_h1_row_norms_max: 3.42718911171 test_h1_row_norms_mean: 2.43771839142 test_h1_row_norms_min: 1.40650296211 test_objective: 0.305504858494 test_term_0: 0.0546990483999 test_term_1_weight_decay: 0.2508058846 test_y_col_norms_max: 3.78892588615 test_y_col_norms_mean: 3.49624156952 test_y_col_norms_min: 3.00044965744 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989326953888 test_y_min_max_class: 0.664230465889 test_y_misclass: 0.0153999980539 test_y_nll: 0.0546990483999 test_y_row_norms_max: 0.914359211922 test_y_row_norms_mean: 0.320046216249 test_y_row_norms_min: 0.0148301701993 train_h0_col_norms_max: 2.9162979126 train_h0_col_norms_mean: 1.90174603462 train_h0_col_norms_min: 1.01780331135 train_h0_row_norms_max: 2.98750209808 train_h0_row_norms_mean: 1.48878598213 train_h0_row_norms_min: 0.0709457397461 train_h1_col_norms_max: 2.67322587967 train_h1_col_norms_mean: 1.71399140358 train_h1_col_norms_min: 0.766523063183 train_h1_row_norms_max: 3.42717552185 train_h1_row_norms_mean: 2.4377117157 train_h1_row_norms_min: 1.40649604797 train_objective: 0.252682715654 train_term_0: 0.00187695620116 train_term_1_weight_decay: 0.250807106495 train_y_col_norms_max: 3.78894209862 train_y_col_norms_mean: 3.49625706673 train_y_col_norms_min: 3.00046014786 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.998175680637 train_y_min_max_class: 0.935915350914 train_y_misclass: 0.0 train_y_nll: 0.00187695620116 train_y_row_norms_max: 0.914363384247 train_y_row_norms_mean: 0.320046842098 train_y_row_norms_min: 0.0148302586749 valid_h0_col_norms_max: 2.91628909111 valid_h0_col_norms_mean: 1.90174818039 valid_h0_col_norms_min: 1.01780152321 valid_h0_row_norms_max: 2.98749780655 valid_h0_row_norms_mean: 1.48878085613 valid_h0_row_norms_min: 0.0709456577897 valid_h1_col_norms_max: 2.6732199192 valid_h1_col_norms_mean: 1.71399199963 valid_h1_col_norms_min: 0.766523241997 valid_h1_row_norms_max: 3.42718911171 valid_h1_row_norms_mean: 2.43771839142 valid_h1_row_norms_min: 1.40650296211 valid_objective: 0.310465872288 valid_term_0: 0.0596601851285 valid_term_1_weight_decay: 0.2508058846 valid_y_col_norms_max: 3.78892588615 valid_y_col_norms_mean: 3.49624156952 valid_y_col_norms_min: 3.00044965744 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989892303944 valid_y_min_max_class: 0.631849765778 valid_y_misclass: 0.0163999944925 valid_y_nll: 0.0596601851285 valid_y_row_norms_max: 0.914359211922 valid_y_row_norms_mean: 0.320046216249 valid_y_row_norms_min: 0.0148301701993 Time this epoch: 3.332601 seconds Monitoring step: Epochs seen: 26 Batches seen: 13000 Examples seen: 1300000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.78018331528 test_h0_col_norms_mean: 1.81279492378 test_h0_col_norms_min: 0.967669785023 test_h0_row_norms_max: 2.84347510338 test_h0_row_norms_mean: 1.41918671131 test_h0_row_norms_min: 0.0677683353424 test_h1_col_norms_max: 2.54604840279 test_h1_col_norms_mean: 1.63219892979 test_h1_col_norms_min: 0.7294241786 test_h1_row_norms_max: 3.26415896416 test_h1_row_norms_mean: 2.32143187523 test_h1_row_norms_min: 1.33722984791 test_objective: 0.282132327557 test_term_0: 0.0540966317058 test_term_1_weight_decay: 0.228035539389 test_y_col_norms_max: 3.75175428391 test_y_col_norms_mean: 3.44715094566 test_y_col_norms_min: 2.93773794174 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.989186286926 test_y_min_max_class: 0.657333433628 test_y_misclass: 0.0153999971226 test_y_nll: 0.0540966317058 test_y_row_norms_max: 0.922168970108 test_y_row_norms_mean: 0.314452946186 test_y_row_norms_min: 0.0141052464023 train_h0_col_norms_max: 2.7801964283 train_h0_col_norms_mean: 1.81280255318 train_h0_col_norms_min: 0.967675149441 train_h0_row_norms_max: 2.84348917007 train_h0_row_norms_mean: 1.41918671131 train_h0_row_norms_min: 0.0677684471011 train_h1_col_norms_max: 2.54606103897 train_h1_col_norms_mean: 1.63219988346 train_h1_col_norms_min: 0.729421555996 train_h1_row_norms_max: 3.26416134834 train_h1_row_norms_mean: 2.3214328289 train_h1_row_norms_min: 1.33723008633 train_objective: 0.230251327157 train_term_0: 0.00221558846533 train_term_1_weight_decay: 0.228034421802 train_y_col_norms_max: 3.75175452232 train_y_col_norms_mean: 3.44716668129 train_y_col_norms_min: 2.93775320053 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.997844338417 train_y_min_max_class: 0.929183900356 train_y_misclass: 0.0 train_y_nll: 0.00221558846533 train_y_row_norms_max: 0.922172784805 train_y_row_norms_mean: 0.314453363419 train_y_row_norms_min: 0.0141053134575 valid_h0_col_norms_max: 2.78018331528 valid_h0_col_norms_mean: 1.81279492378 valid_h0_col_norms_min: 0.967669785023 valid_h0_row_norms_max: 2.84347510338 valid_h0_row_norms_mean: 1.41918671131 valid_h0_row_norms_min: 0.0677683353424 valid_h1_col_norms_max: 2.54604840279 valid_h1_col_norms_mean: 1.63219892979 valid_h1_col_norms_min: 0.7294241786 valid_h1_row_norms_max: 3.26415896416 valid_h1_row_norms_mean: 2.32143187523 valid_h1_row_norms_min: 1.33722984791 valid_objective: 0.287775874138 valid_term_0: 0.0597402378917 valid_term_1_weight_decay: 0.228035539389 valid_y_col_norms_max: 3.75175428391 valid_y_col_norms_mean: 3.44715094566 valid_y_col_norms_min: 2.93773794174 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989176094532 valid_y_min_max_class: 0.624788343906 valid_y_misclass: 0.0166999921203 valid_y_nll: 0.0597402378917 valid_y_row_norms_max: 0.922168970108 valid_y_row_norms_mean: 0.314452946186 valid_y_row_norms_min: 0.0141052464023 Time this epoch: 3.284030 seconds Monitoring step: Epochs seen: 27 Batches seen: 13500 Examples seen: 1350000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.65619587898 test_h0_col_norms_mean: 1.72903525829 test_h0_col_norms_min: 0.920009553432 test_h0_row_norms_max: 2.71527957916 test_h0_row_norms_mean: 1.3536605835 test_h0_row_norms_min: 0.0648870319128 test_h1_col_norms_max: 2.42550611496 test_h1_col_norms_mean: 1.55485773087 test_h1_col_norms_min: 0.694191396236 test_h1_row_norms_max: 3.11506414413 test_h1_row_norms_mean: 2.21147465706 test_h1_row_norms_min: 1.27136671543 test_objective: 0.261537849903 test_term_0: 0.0539344884455 test_term_1_weight_decay: 0.207603096962 test_y_col_norms_max: 3.72367501259 test_y_col_norms_mean: 3.41610884666 test_y_col_norms_min: 2.90950012207 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.98896753788 test_y_min_max_class: 0.650016546249 test_y_misclass: 0.0154999988154 test_y_nll: 0.0539344884455 test_y_row_norms_max: 0.928149938583 test_y_row_norms_mean: 0.310383200645 test_y_row_norms_min: 0.0134189818054 train_h0_col_norms_max: 2.6562101841 train_h0_col_norms_mean: 1.72902774811 train_h0_col_norms_min: 0.920005261898 train_h0_row_norms_max: 2.71527171135 train_h0_row_norms_mean: 1.35366332531 train_h0_row_norms_min: 0.0648870840669 train_h1_col_norms_max: 2.42551159859 train_h1_col_norms_mean: 1.5548504591 train_h1_col_norms_min: 0.694188058376 train_h1_row_norms_max: 3.1150598526 train_h1_row_norms_mean: 2.21146249771 train_h1_row_norms_min: 1.27136695385 train_objective: 0.20999661088 train_term_0: 0.002393146744 train_term_1_weight_decay: 0.207603022456 train_y_col_norms_max: 3.7236533165 train_y_col_norms_mean: 3.41609406471 train_y_col_norms_min: 2.90950846672 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.997686505318 train_y_min_max_class: 0.923012793064 train_y_misclass: 1.99999994948e-05 train_y_nll: 0.002393146744 train_y_row_norms_max: 0.928141772747 train_y_row_norms_mean: 0.310382217169 train_y_row_norms_min: 0.0134189641103 valid_h0_col_norms_max: 2.65619587898 valid_h0_col_norms_mean: 1.72903525829 valid_h0_col_norms_min: 0.920009553432 valid_h0_row_norms_max: 2.71527957916 valid_h0_row_norms_mean: 1.3536605835 valid_h0_row_norms_min: 0.0648870319128 valid_h1_col_norms_max: 2.42550611496 valid_h1_col_norms_mean: 1.55485773087 valid_h1_col_norms_min: 0.694191396236 valid_h1_row_norms_max: 3.11506414413 valid_h1_row_norms_mean: 2.21147465706 valid_h1_row_norms_min: 1.27136671543 valid_objective: 0.268219769001 valid_term_0: 0.0606164671481 valid_term_1_weight_decay: 0.207603096962 valid_y_col_norms_max: 3.72367501259 valid_y_col_norms_mean: 3.41610884666 valid_y_col_norms_min: 2.90950012207 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989122092724 valid_y_min_max_class: 0.619553089142 valid_y_misclass: 0.0165999922901 valid_y_nll: 0.0606164671481 valid_y_row_norms_max: 0.928149938583 valid_y_row_norms_mean: 0.310383200645 valid_y_row_norms_min: 0.0134189818054 Time this epoch: 3.261071 seconds Monitoring step: Epochs seen: 28 Batches seen: 14000 Examples seen: 1400000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.53800392151 test_h0_col_norms_mean: 1.65007030964 test_h0_col_norms_min: 0.874696552753 test_h0_row_norms_max: 2.59223008156 test_h0_row_norms_mean: 1.29187560081 test_h0_row_norms_min: 0.0620772130787 test_h1_col_norms_max: 2.31092762947 test_h1_col_norms_mean: 1.48166322708 test_h1_col_norms_min: 0.660871863365 test_h1_row_norms_max: 2.96952915192 test_h1_row_norms_mean: 2.10743236542 test_h1_row_norms_min: 1.20874655247 test_objective: 0.244267836213 test_term_0: 0.0550341755152 test_term_1_weight_decay: 0.189233824611 test_y_col_norms_max: 3.7156329155 test_y_col_norms_mean: 3.3953909874 test_y_col_norms_min: 2.88775634766 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.988679349422 test_y_min_max_class: 0.66010850668 test_y_misclass: 0.015999995172 test_y_nll: 0.0550341755152 test_y_row_norms_max: 0.940686166286 test_y_row_norms_mean: 0.307195395231 test_y_row_norms_min: 0.0127685274929 train_h0_col_norms_max: 2.53800177574 train_h0_col_norms_mean: 1.65007722378 train_h0_col_norms_min: 0.874697685242 train_h0_row_norms_max: 2.59221696854 train_h0_row_norms_mean: 1.29187226295 train_h0_row_norms_min: 0.0620768405497 train_h1_col_norms_max: 2.3109228611 train_h1_col_norms_mean: 1.48165631294 train_h1_col_norms_min: 0.660868704319 train_h1_row_norms_max: 2.96951341629 train_h1_row_norms_mean: 2.10743737221 train_h1_row_norms_min: 1.208745718 train_objective: 0.19228720665 train_term_0: 0.00305363954976 train_term_1_weight_decay: 0.18923483789 train_y_col_norms_max: 3.71563386917 train_y_col_norms_mean: 3.39540982246 train_y_col_norms_min: 2.88775682449 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.99712729454 train_y_min_max_class: 0.899743020535 train_y_misclass: 9.99999974738e-05 train_y_nll: 0.00305363954976 train_y_row_norms_max: 0.940682113171 train_y_row_norms_mean: 0.307196319103 train_y_row_norms_min: 0.0127684678882 valid_h0_col_norms_max: 2.53800392151 valid_h0_col_norms_mean: 1.65007030964 valid_h0_col_norms_min: 0.874696552753 valid_h0_row_norms_max: 2.59223008156 valid_h0_row_norms_mean: 1.29187560081 valid_h0_row_norms_min: 0.0620772130787 valid_h1_col_norms_max: 2.31092762947 valid_h1_col_norms_mean: 1.48166322708 valid_h1_col_norms_min: 0.660871863365 valid_h1_row_norms_max: 2.96952915192 valid_h1_row_norms_mean: 2.10743236542 valid_h1_row_norms_min: 1.20874655247 valid_objective: 0.250370264053 valid_term_0: 0.0611366219819 valid_term_1_weight_decay: 0.189233824611 valid_y_col_norms_max: 3.7156329155 valid_y_col_norms_mean: 3.3953909874 valid_y_col_norms_min: 2.88775634766 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.98891800642 valid_y_min_max_class: 0.613278985023 valid_y_misclass: 0.0181999895722 valid_y_nll: 0.0611366219819 valid_y_row_norms_max: 0.940686166286 valid_y_row_norms_mean: 0.307195395231 valid_y_row_norms_min: 0.0127685274929 Time this epoch: 3.281839 seconds Monitoring step: Epochs seen: 29 Batches seen: 14500 Examples seen: 1450000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.42693471909 test_h0_col_norms_mean: 1.57674181461 test_h0_col_norms_min: 0.831614494324 test_h0_row_norms_max: 2.47894763947 test_h0_row_norms_mean: 1.23453938961 test_h0_row_norms_min: 0.0597868897021 test_h1_col_norms_max: 2.20548892021 test_h1_col_norms_mean: 1.41274940968 test_h1_col_norms_min: 0.629146695137 test_h1_row_norms_max: 2.83711600304 test_h1_row_norms_mean: 2.00946569443 test_h1_row_norms_min: 1.14921236038 test_objective: 0.231429338455 test_term_0: 0.0585358664393 test_term_1_weight_decay: 0.17289365828 test_y_col_norms_max: 3.72800803185 test_y_col_norms_mean: 3.39454960823 test_y_col_norms_min: 2.87621188164 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.988135576248 test_y_min_max_class: 0.641624808311 test_y_misclass: 0.0174999944866 test_y_nll: 0.0585358664393 test_y_row_norms_max: 0.94206482172 test_y_row_norms_mean: 0.305722147226 test_y_row_norms_min: 0.0122694317251 train_h0_col_norms_max: 2.42693591118 train_h0_col_norms_mean: 1.57673621178 train_h0_col_norms_min: 0.831611156464 train_h0_row_norms_max: 2.47895431519 train_h0_row_norms_mean: 1.23453593254 train_h0_row_norms_min: 0.0597870908678 train_h1_col_norms_max: 2.20549035072 train_h1_col_norms_mean: 1.41274940968 train_h1_col_norms_min: 0.629147231579 train_h1_row_norms_max: 2.83710312843 train_h1_row_norms_mean: 2.0094628334 train_h1_row_norms_min: 1.14921247959 train_objective: 0.176759794354 train_term_0: 0.00386637193151 train_term_1_weight_decay: 0.172893241048 train_y_col_norms_max: 3.72802448273 train_y_col_norms_mean: 3.39456868172 train_y_col_norms_min: 2.87620210648 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996502935886 train_y_min_max_class: 0.867810547352 train_y_misclass: 0.000219999958063 train_y_nll: 0.00386637193151 train_y_row_norms_max: 0.942059278488 train_y_row_norms_mean: 0.305722266436 train_y_row_norms_min: 0.012269385159 valid_h0_col_norms_max: 2.42693471909 valid_h0_col_norms_mean: 1.57674181461 valid_h0_col_norms_min: 0.831614494324 valid_h0_row_norms_max: 2.47894763947 valid_h0_row_norms_mean: 1.23453938961 valid_h0_row_norms_min: 0.0597868897021 valid_h1_col_norms_max: 2.20548892021 valid_h1_col_norms_mean: 1.41274940968 valid_h1_col_norms_min: 0.629146695137 valid_h1_row_norms_max: 2.83711600304 valid_h1_row_norms_mean: 2.00946569443 valid_h1_row_norms_min: 1.14921236038 valid_objective: 0.233349367976 valid_term_0: 0.0604558549821 valid_term_1_weight_decay: 0.17289365828 valid_y_col_norms_max: 3.72800803185 valid_y_col_norms_mean: 3.39454960823 valid_y_col_norms_min: 2.87621188164 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.988323628902 valid_y_min_max_class: 0.623263895512 valid_y_misclass: 0.0180999934673 valid_y_nll: 0.0604558549821 valid_y_row_norms_max: 0.94206482172 valid_y_row_norms_mean: 0.305722147226 valid_y_row_norms_min: 0.0122694317251 Time this epoch: 3.305740 seconds Monitoring step: Epochs seen: 30 Batches seen: 15000 Examples seen: 1500000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.32734417915 test_h0_col_norms_mean: 1.50859749317 test_h0_col_norms_min: 0.790655136108 test_h0_row_norms_max: 2.3708486557 test_h0_row_norms_mean: 1.18130934238 test_h0_row_norms_min: 0.0573941357434 test_h1_col_norms_max: 2.10413718224 test_h1_col_norms_mean: 1.34777581692 test_h1_col_norms_min: 0.599398136139 test_h1_row_norms_max: 2.71429800987 test_h1_row_norms_mean: 1.91710066795 test_h1_row_norms_min: 1.09260857105 test_objective: 0.213433161378 test_term_0: 0.0551191605628 test_term_1_weight_decay: 0.158313959837 test_y_col_norms_max: 3.74245023727 test_y_col_norms_mean: 3.40365672112 test_y_col_norms_min: 2.87184524536 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.988063156605 test_y_min_max_class: 0.63681012392 test_y_misclass: 0.0161999966949 test_y_nll: 0.0551191605628 test_y_row_norms_max: 0.970815420151 test_y_row_norms_mean: 0.305047929287 test_y_row_norms_min: 0.0117727546021 train_h0_col_norms_max: 2.32733273506 train_h0_col_norms_mean: 1.50859475136 train_h0_col_norms_min: 0.790655434132 train_h0_row_norms_max: 2.37083745003 train_h0_row_norms_mean: 1.18130576611 train_h0_row_norms_min: 0.0573940649629 train_h1_col_norms_max: 2.10414910316 train_h1_col_norms_mean: 1.34777891636 train_h1_col_norms_min: 0.599396586418 train_h1_row_norms_max: 2.71428489685 train_h1_row_norms_mean: 1.91711127758 train_h1_row_norms_min: 1.09261226654 train_objective: 0.161947190762 train_term_0: 0.00363326980732 train_term_1_weight_decay: 0.158314481378 train_y_col_norms_max: 3.74245476723 train_y_col_norms_mean: 3.40366172791 train_y_col_norms_min: 2.87186050415 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996790409088 train_y_min_max_class: 0.880268752575 train_y_misclass: 0.0002599999425 train_y_nll: 0.00363326980732 train_y_row_norms_max: 0.970812141895 train_y_row_norms_mean: 0.305047690868 train_y_row_norms_min: 0.0117728123441 valid_h0_col_norms_max: 2.32734417915 valid_h0_col_norms_mean: 1.50859749317 valid_h0_col_norms_min: 0.790655136108 valid_h0_row_norms_max: 2.3708486557 valid_h0_row_norms_mean: 1.18130934238 valid_h0_row_norms_min: 0.0573941357434 valid_h1_col_norms_max: 2.10413718224 valid_h1_col_norms_mean: 1.34777581692 valid_h1_col_norms_min: 0.599398136139 valid_h1_row_norms_max: 2.71429800987 valid_h1_row_norms_mean: 1.91710066795 valid_h1_row_norms_min: 1.09260857105 valid_objective: 0.221332803369 valid_term_0: 0.0630188435316 valid_term_1_weight_decay: 0.158313959837 valid_y_col_norms_max: 3.74245023727 valid_y_col_norms_mean: 3.40365672112 valid_y_col_norms_min: 2.87184524536 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.988815426826 valid_y_min_max_class: 0.629059970379 valid_y_misclass: 0.0181999914348 valid_y_nll: 0.0630188435316 valid_y_row_norms_max: 0.970815420151 valid_y_row_norms_mean: 0.305047929287 valid_y_row_norms_min: 0.0117727546021 Time this epoch: 3.277658 seconds Monitoring step: Epochs seen: 31 Batches seen: 15500 Examples seen: 1550000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.22986006737 test_h0_col_norms_mean: 1.44450759888 test_h0_col_norms_min: 0.751711428165 test_h0_row_norms_max: 2.27136349678 test_h0_row_norms_mean: 1.13126826286 test_h0_row_norms_min: 0.055449090898 test_h1_col_norms_max: 2.01123976707 test_h1_col_norms_mean: 1.28632330894 test_h1_col_norms_min: 0.570827186108 test_h1_row_norms_max: 2.59384894371 test_h1_row_norms_mean: 1.82977592945 test_h1_row_norms_min: 1.03879511356 test_objective: 0.199405178428 test_term_0: 0.0542121008039 test_term_1_weight_decay: 0.145193070173 test_y_col_norms_max: 3.75564265251 test_y_col_norms_mean: 3.41556763649 test_y_col_norms_min: 2.89411330223 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.98805475235 test_y_min_max_class: 0.651372611523 test_y_misclass: 0.0173999965191 test_y_nll: 0.0542121008039 test_y_row_norms_max: 0.984490036964 test_y_row_norms_mean: 0.30471482873 test_y_row_norms_min: 0.0115118613467 train_h0_col_norms_max: 2.22986245155 train_h0_col_norms_mean: 1.44451439381 train_h0_col_norms_min: 0.751711428165 train_h0_row_norms_max: 2.27136206627 train_h0_row_norms_mean: 1.13127017021 train_h0_row_norms_min: 0.0554490871727 train_h1_col_norms_max: 2.01124000549 train_h1_col_norms_mean: 1.28632640839 train_h1_col_norms_min: 0.570830464363 train_h1_row_norms_max: 2.59386348724 train_h1_row_norms_mean: 1.8297867775 train_h1_row_norms_min: 1.03879117966 train_objective: 0.148557990789 train_term_0: 0.00336487870663 train_term_1_weight_decay: 0.145192667842 train_y_col_norms_max: 3.75566291809 train_y_col_norms_mean: 3.41558241844 train_y_col_norms_min: 2.89412260056 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.996838271618 train_y_min_max_class: 0.885646343231 train_y_misclass: 7.9999997979e-05 train_y_nll: 0.00336487870663 train_y_row_norms_max: 0.98448997736 train_y_row_norms_mean: 0.304714143276 train_y_row_norms_min: 0.0115119209513 valid_h0_col_norms_max: 2.22986006737 valid_h0_col_norms_mean: 1.44450759888 valid_h0_col_norms_min: 0.751711428165 valid_h0_row_norms_max: 2.27136349678 valid_h0_row_norms_mean: 1.13126826286 valid_h0_row_norms_min: 0.055449090898 valid_h1_col_norms_max: 2.01123976707 valid_h1_col_norms_mean: 1.28632330894 valid_h1_col_norms_min: 0.570827186108 valid_h1_row_norms_max: 2.59384894371 valid_h1_row_norms_mean: 1.82977592945 valid_h1_row_norms_min: 1.03879511356 valid_objective: 0.204451009631 valid_term_0: 0.0592579171062 valid_term_1_weight_decay: 0.145193070173 valid_y_col_norms_max: 3.75564265251 valid_y_col_norms_mean: 3.41556763649 valid_y_col_norms_min: 2.89411330223 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.989104747772 valid_y_min_max_class: 0.617950022221 valid_y_misclass: 0.0168999936432 valid_y_nll: 0.0592579171062 valid_y_row_norms_max: 0.984490036964 valid_y_row_norms_mean: 0.30471482873 valid_y_row_norms_min: 0.0115118613467 Time this epoch: 3.299106 seconds Monitoring step: Epochs seen: 32 Batches seen: 16000 Examples seen: 1600000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.14349389076 test_h0_col_norms_mean: 1.38710844517 test_h0_col_norms_min: 0.714687824249 test_h0_row_norms_max: 2.18458104134 test_h0_row_norms_mean: 1.08642613888 test_h0_row_norms_min: 0.0531989820302 test_h1_col_norms_max: 1.92048859596 test_h1_col_norms_mean: 1.22870218754 test_h1_col_norms_min: 0.54377913475 test_h1_row_norms_max: 2.48094320297 test_h1_row_norms_mean: 1.74789762497 test_h1_row_norms_min: 0.987630963326 test_objective: 0.198874086142 test_term_0: 0.0651714801788 test_term_1_weight_decay: 0.13370269537 test_y_col_norms_max: 3.78167033195 test_y_col_norms_mean: 3.43723106384 test_y_col_norms_min: 2.910176754 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.986017048359 test_y_min_max_class: 0.594191014767 test_y_misclass: 0.0199999921024 test_y_nll: 0.0651714801788 test_y_row_norms_max: 0.997191548347 test_y_row_norms_mean: 0.305199384689 test_y_row_norms_min: 0.0112372441217 train_h0_col_norms_max: 2.14350128174 train_h0_col_norms_mean: 1.38711500168 train_h0_col_norms_min: 0.714688956738 train_h0_row_norms_max: 2.18457770348 train_h0_row_norms_mean: 1.08642041683 train_h0_row_norms_min: 0.0531990006566 train_h1_col_norms_max: 1.92047715187 train_h1_col_norms_mean: 1.22869598866 train_h1_col_norms_min: 0.543776392937 train_h1_row_norms_max: 2.4809448719 train_h1_row_norms_mean: 1.74790513515 train_h1_row_norms_min: 0.987626254559 train_objective: 0.142097592354 train_term_0: 0.00839497055858 train_term_1_weight_decay: 0.133703291416 train_y_col_norms_max: 3.78167462349 train_y_col_norms_mean: 3.43724656105 train_y_col_norms_min: 2.91016840935 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.993861615658 train_y_min_max_class: 0.764626443386 train_y_misclass: 0.00168000021949 train_y_nll: 0.00839497055858 train_y_row_norms_max: 0.997187256813 train_y_row_norms_mean: 0.305199593306 train_y_row_norms_min: 0.0112373000011 valid_h0_col_norms_max: 2.14349389076 valid_h0_col_norms_mean: 1.38710844517 valid_h0_col_norms_min: 0.714687824249 valid_h0_row_norms_max: 2.18458104134 valid_h0_row_norms_mean: 1.08642613888 valid_h0_row_norms_min: 0.0531989820302 valid_h1_col_norms_max: 1.92048859596 valid_h1_col_norms_mean: 1.22870218754 valid_h1_col_norms_min: 0.54377913475 valid_h1_row_norms_max: 2.48094320297 valid_h1_row_norms_mean: 1.74789762497 valid_h1_row_norms_min: 0.987630963326 valid_objective: 0.206293180585 valid_term_0: 0.0725905746222 valid_term_1_weight_decay: 0.13370269537 valid_y_col_norms_max: 3.78167033195 valid_y_col_norms_mean: 3.43723106384 valid_y_col_norms_min: 2.910176754 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.98618721962 valid_y_min_max_class: 0.584758162498 valid_y_misclass: 0.0210999920964 valid_y_nll: 0.0725905746222 valid_y_row_norms_max: 0.997191548347 valid_y_row_norms_mean: 0.305199384689 valid_y_row_norms_min: 0.0112372441217 Time this epoch: 3.316685 seconds Monitoring step: Epochs seen: 33 Batches seen: 16500 Examples seen: 1650000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.07966470718 test_h0_col_norms_mean: 1.34304857254 test_h0_col_norms_min: 0.67948693037 test_h0_row_norms_max: 2.12437868118 test_h0_row_norms_mean: 1.05228435993 test_h0_row_norms_min: 0.0516484305263 test_h1_col_norms_max: 1.84021937847 test_h1_col_norms_mean: 1.17601656914 test_h1_col_norms_min: 0.518966257572 test_h1_row_norms_max: 2.38222265244 test_h1_row_norms_mean: 1.67307877541 test_h1_row_norms_min: 0.938986539841 test_objective: 0.1945425421 test_term_0: 0.0701323673129 test_term_1_weight_decay: 0.124409988523 test_y_col_norms_max: 3.7988409996 test_y_col_norms_mean: 3.48081755638 test_y_col_norms_min: 2.99444794655 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.985891222954 test_y_min_max_class: 0.610743761063 test_y_misclass: 0.021299995482 test_y_nll: 0.0701323673129 test_y_row_norms_max: 1.03870010376 test_y_row_norms_mean: 0.307827204466 test_y_row_norms_min: 0.0110923619941 train_h0_col_norms_max: 2.07966327667 train_h0_col_norms_mean: 1.34305346012 train_h0_col_norms_min: 0.679490327835 train_h0_row_norms_max: 2.12437534332 train_h0_row_norms_mean: 1.05228734016 train_h0_row_norms_min: 0.0516487248242 train_h1_col_norms_max: 1.84022164345 train_h1_col_norms_mean: 1.17601895332 train_h1_col_norms_min: 0.518965959549 train_h1_row_norms_max: 2.3822286129 train_h1_row_norms_mean: 1.67308568954 train_h1_row_norms_min: 0.93898332119 train_objective: 0.135569825768 train_term_0: 0.0111596826464 train_term_1_weight_decay: 0.124409854412 train_y_col_norms_max: 3.79884195328 train_y_col_norms_mean: 3.48080062866 train_y_col_norms_min: 2.99444794655 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.992844820023 train_y_min_max_class: 0.72961461544 train_y_misclass: 0.00289999856614 train_y_nll: 0.0111596826464 train_y_row_norms_max: 1.03869926929 train_y_row_norms_mean: 0.307827889919 train_y_row_norms_min: 0.0110923871398 valid_h0_col_norms_max: 2.07966470718 valid_h0_col_norms_mean: 1.34304857254 valid_h0_col_norms_min: 0.67948693037 valid_h0_row_norms_max: 2.12437868118 valid_h0_row_norms_mean: 1.05228435993 valid_h0_row_norms_min: 0.0516484305263 valid_h1_col_norms_max: 1.84021937847 valid_h1_col_norms_mean: 1.17601656914 valid_h1_col_norms_min: 0.518966257572 valid_h1_row_norms_max: 2.38222265244 valid_h1_row_norms_mean: 1.67307877541 valid_h1_row_norms_min: 0.938986539841 valid_objective: 0.197794348001 valid_term_0: 0.0733841732144 valid_term_1_weight_decay: 0.124409988523 valid_y_col_norms_max: 3.7988409996 valid_y_col_norms_mean: 3.48081755638 valid_y_col_norms_min: 2.99444794655 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.986917078495 valid_y_min_max_class: 0.61943089962 valid_y_misclass: 0.0203999932855 valid_y_nll: 0.0733841732144 valid_y_row_norms_max: 1.03870010376 valid_y_row_norms_mean: 0.307827204466 valid_y_row_norms_min: 0.0110923619941 Time this epoch: 3.390813 seconds Monitoring step: Epochs seen: 34 Batches seen: 17000 Examples seen: 1700000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.09944272041 test_h0_col_norms_mean: 1.31166350842 test_h0_col_norms_min: 0.646019160748 test_h0_row_norms_max: 2.08779644966 test_h0_row_norms_mean: 1.02819681168 test_h0_row_norms_min: 0.0544924363494 test_h1_col_norms_max: 1.76557374001 test_h1_col_norms_mean: 1.12808454037 test_h1_col_norms_min: 0.495299696922 test_h1_row_norms_max: 2.29631304741 test_h1_row_norms_mean: 1.60503029823 test_h1_row_norms_min: 0.892738819122 test_objective: 0.189725786448 test_term_0: 0.0726886093616 test_term_1_weight_decay: 0.117037259042 test_y_col_norms_max: 3.87525558472 test_y_col_norms_mean: 3.52211046219 test_y_col_norms_min: 3.00069046021 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.983940660954 test_y_min_max_class: 0.607474207878 test_y_misclass: 0.0210999920964 test_y_nll: 0.0726886093616 test_y_row_norms_max: 1.04711163044 test_y_row_norms_mean: 0.310424894094 test_y_row_norms_min: 0.0110520040616 train_h0_col_norms_max: 2.09944820404 train_h0_col_norms_mean: 1.31166100502 train_h0_col_norms_min: 0.646022617817 train_h0_row_norms_max: 2.08778810501 train_h0_row_norms_mean: 1.02820193768 train_h0_row_norms_min: 0.0544921904802 train_h1_col_norms_max: 1.76556527615 train_h1_col_norms_mean: 1.12807917595 train_h1_col_norms_min: 0.495299696922 train_h1_row_norms_max: 2.29631876945 train_h1_row_norms_mean: 1.60503292084 train_h1_row_norms_min: 0.892735242844 train_objective: 0.138252094388 train_term_0: 0.0212149638683 train_term_1_weight_decay: 0.117037393153 train_y_col_norms_max: 3.87525558472 train_y_col_norms_mean: 3.5221259594 train_y_col_norms_min: 3.00070405006 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.989364624023 train_y_min_max_class: 0.659551143646 train_y_misclass: 0.00660000368953 train_y_nll: 0.0212149638683 train_y_row_norms_max: 1.04711127281 train_y_row_norms_mean: 0.310425490141 train_y_row_norms_min: 0.0110520040616 valid_h0_col_norms_max: 2.09944272041 valid_h0_col_norms_mean: 1.31166350842 valid_h0_col_norms_min: 0.646019160748 valid_h0_row_norms_max: 2.08779644966 valid_h0_row_norms_mean: 1.02819681168 valid_h0_row_norms_min: 0.0544924363494 valid_h1_col_norms_max: 1.76557374001 valid_h1_col_norms_mean: 1.12808454037 valid_h1_col_norms_min: 0.495299696922 valid_h1_row_norms_max: 2.29631304741 valid_h1_row_norms_mean: 1.60503029823 valid_h1_row_norms_min: 0.892738819122 valid_objective: 0.204115614295 valid_term_0: 0.0870784968138 valid_term_1_weight_decay: 0.117037259042 valid_y_col_norms_max: 3.87525558472 valid_y_col_norms_mean: 3.52211046219 valid_y_col_norms_min: 3.00069046021 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.984562575817 valid_y_min_max_class: 0.597014904022 valid_y_misclass: 0.0247999858111 valid_y_nll: 0.0870784968138 valid_y_row_norms_max: 1.04711163044 valid_y_row_norms_mean: 0.310424894094 valid_y_row_norms_min: 0.0110520040616 Time this epoch: 3.325348 seconds Monitoring step: Epochs seen: 35 Batches seen: 17500 Examples seen: 1750000 learning_rate: 0.00999999046326 momentum: 0.989998817444 test_h0_col_norms_max: 2.33471369743 test_h0_col_norms_mean: 1.30764365196 test_h0_col_norms_min: 0.614201545715 test_h0_row_norms_max: 2.11369776726 test_h0_row_norms_mean: 1.02605807781 test_h0_row_norms_min: 0.0789580345154 test_h1_col_norms_max: 1.72045576572 test_h1_col_norms_mean: 1.08744347095 test_h1_col_norms_min: 0.471859395504 test_h1_row_norms_max: 2.22104668617 test_h1_row_norms_mean: 1.54732775688 test_h1_row_norms_min: 0.848768413067 test_objective: 0.186729609966 test_term_0: 0.0738602727652 test_term_1_weight_decay: 0.112869426608 test_y_col_norms_max: 3.81233644485 test_y_col_norms_mean: 3.53644061089 test_y_col_norms_min: 3.07366251945 test_y_max_max_class: 0.999999344349 test_y_mean_max_class: 0.982964873314 test_y_min_max_class: 0.570746660233 test_y_misclass: 0.0224999897182 test_y_nll: 0.0738602727652 test_y_row_norms_max: 1.04587638378 test_y_row_norms_mean: 0.311368614435 test_y_row_norms_min: 0.0108088394627 train_h0_col_norms_max: 2.33471369743 train_h0_col_norms_mean: 1.30764782429 train_h0_col_norms_min: 0.614198505878 train_h0_row_norms_max: 2.11369967461 train_h0_row_norms_mean: 1.02606165409 train_h0_row_norms_min: 0.0789578035474 train_h1_col_norms_max: 1.72044575214 train_h1_col_norms_mean: 1.08744776249 train_h1_col_norms_min: 0.471859931946 train_h1_row_norms_max: 2.2210419178 train_h1_row_norms_mean: 1.54733288288 train_h1_row_norms_min: 0.848769664764 train_objective: 0.133081272244 train_term_0: 0.020211936906 train_term_1_weight_decay: 0.112869039178 train_y_col_norms_max: 3.81231951714 train_y_col_norms_mean: 3.53645634651 train_y_col_norms_min: 3.07366323471 train_y_max_max_class: 0.999994218349 train_y_mean_max_class: 0.989763617516 train_y_min_max_class: 0.656112134457 train_y_misclass: 0.00610000034794 train_y_nll: 0.020211936906 train_y_row_norms_max: 1.04588091373 train_y_row_norms_mean: 0.311368972063 train_y_row_norms_min: 0.0108088953421 valid_h0_col_norms_max: 2.33471369743 valid_h0_col_norms_mean: 1.30764365196 valid_h0_col_norms_min: 0.614201545715 valid_h0_row_norms_max: 2.11369776726 valid_h0_row_norms_mean: 1.02605807781 valid_h0_row_norms_min: 0.0789580345154 valid_h1_col_norms_max: 1.72045576572 valid_h1_col_norms_mean: 1.08744347095 valid_h1_col_norms_min: 0.471859395504 valid_h1_row_norms_max: 2.22104668617 valid_h1_row_norms_mean: 1.54732775688 valid_h1_row_norms_min: 0.848768413067 valid_objective: 0.191735550761 valid_term_0: 0.0788661986589 valid_term_1_weight_decay: 0.112869426608 valid_y_col_norms_max: 3.81233644485 valid_y_col_norms_mean: 3.53644061089 valid_y_col_norms_min: 3.07366251945 valid_y_max_max_class: 0.999999344349 valid_y_mean_max_class: 0.985468804836 valid_y_min_max_class: 0.593561589718 valid_y_misclass: 0.0224999915808 valid_y_nll: 0.0788661986589 valid_y_row_norms_max: 1.04587638378 valid_y_row_norms_mean: 0.311368614435 valid_y_row_norms_min: 0.0108088394627
!print_monitor.py mlp_3_best.pkl | grep test_y_misclass
Using gpu device 2: GeForce GTX 285 /u/goodfeli/pylearn2/models/mlp.py:36: UserWarning: MLP changing the recursion limit. warnings.warn("MLP changing the recursion limit.") test_y_misclass : 0.0153999980539
Using a simple form of regularization thus brought the test error rate for this MLP down from 1.75% to 1.54%.
You can find more information on MLPs from the following sources:
LISA lab's Deep Learning Tutorials: Multilayer Perception
This is by no means a complete list.