Mercurial > ift6266
diff writeup/nips2010_submission.tex @ 514:920a38715c90
merge
author | Yoshua Bengio <bengioy@iro.umontreal.ca> |
---|---|
date | Tue, 01 Jun 2010 14:05:21 -0400 |
parents | 66a905508e34 d057941417ed |
children | 092dae9a5040 |
line wrap: on
line diff
--- a/writeup/nips2010_submission.tex Tue Jun 01 14:05:02 2010 -0400 +++ b/writeup/nips2010_submission.tex Tue Jun 01 14:05:21 2010 -0400 @@ -20,7 +20,7 @@ Recent theoretical and empirical work in statistical machine learning has demonstrated the importance of learning algorithms for deep architectures, i.e., function classes obtained by composing multiple - non-linear transformations. The self-taught learning (exploiting unlabeled + non-linear transformations. Self-taught learning (exploiting unlabeled examples or examples from other distributions) has already been applied to deep learners, but mostly to show the advantage of unlabeled examples. Here we explore the advantage brought by {\em out-of-distribution @@ -74,8 +74,8 @@ performed similarly or better than previously proposed Restricted Boltzmann Machines in terms of unsupervised extraction of a hierarchy of features useful for classification. The principle is that each layer starting from -the bottom is trained to encode their input (the output of the previous -layer) and try to reconstruct it from a corrupted version of it. After this +the bottom is trained to encode its input (the output of the previous +layer) and to reconstruct it from a corrupted version of it. After this unsupervised initialization, the stack of denoising auto-encoders can be converted into a deep supervised feedforward neural network and fine-tuned by stochastic gradient descent. @@ -95,6 +95,8 @@ between different regions in input space or different tasks, as discussed in the conclusion. +% TODO: why we care to evaluate this relative advantage + In this paper we ask the following questions: %\begin{enumerate} @@ -119,7 +121,7 @@ a corresponding shallow and purely supervised architecture? %\end{enumerate} -The experimental results presented here provide positive evidence towards all of these questions. +Our experimental results provide positive evidence towards all of these questions. \vspace*{-1mm} \section{Perturbation and Transformation of Character Images}