Mercurial > ift6266

annotate writeup/techreport.tex @ 396:116b2de2c0a4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
utils for the amazon MT code
author goldfinger
date Tue, 27 Apr 2010 13:47:33 -0400
parents 4c840798d290
children fe2e2964e7a3
rev   line source
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
1 \documentclass[12pt,letterpaper]{article}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
2 \usepackage[utf8]{inputenc}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
3 \usepackage{graphicx}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
4 \usepackage{times}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
5 \usepackage{mlapa}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
6
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
7 \begin{document}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
8 \title{Generating and Exploiting Perturbed Training Data for Deep Architectures}
381
0a91fc69ff90 authors
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 379
diff changeset
9 \author{The IFT6266 Gang}
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
10 \date{April 2010, Technical Report, Dept. IRO, U. Montreal}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
11
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
12 \maketitle
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
13
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
14 \begin{abstract}
392
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
15 Recent theoretical and empirical work in statistical machine learning has
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
16 demonstrated the importance of learning algorithms for deep
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
17 architectures, i.e., function classes obtained by composing multiple
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
18 non-linear transformations. In the area of handwriting recognition,
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
19 deep learning algorithms
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
20 had been evaluated on rather small datasets with a few tens of thousands
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
21 of examples. Here we propose a powerful generator of variations
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
22 of examples for character images based on a pipeline of stochastic
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
23 transformations that include not only the usual affine transformations
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
24 but also the addition of slant, local elastic deformations, changes
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
25 in thickness, background images, color, contrast, occlusion, and
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
26 various types of pixel and spatially correlated noise.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
27 We evaluate a deep learning algorithm (Stacked Denoising Autoencoders)
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
28 on the task of learning to classify digits and letters transformed
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
29 with this pipeline, using the hundreds of millions of generated examples
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
30 and testing on the full NIST test set.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
31 We find that the SDA outperforms its
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
32 shallow counterpart, an ordinary Multi-Layer Perceptron,
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
33 and that it is better able to take advantage of the additional
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
34 generated data.
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
35 \end{abstract}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
36
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
37 \section{Introduction}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
38
392
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
39 Deep Learning has emerged as a promising new area of research in
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
40 statistical machine learning (see~\emcite{Bengio-2009} for a review).
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
41 Learning algorithms for deep architectures are centered on the learning
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
42 of useful representations of data, which are better suited to the task at hand.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
43 This is in great part inspired by observations of the mammalian visual cortex,
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
44 which consists of a chain of processing elements, each of which is associated with a
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
45 different representation. In fact,
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
46 it was found recently that the features learnt in deep architectures resemble
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
47 those observed in the first two of these stages (in areas V1 and V2
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
48 of visual cortex)~\cite{HonglakL2008}.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
49 Processing images typically involves transforming the raw pixel data into
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
50 new {\bf representations} that can be used for analysis or classification.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
51 For example, a principal component analysis representation linearly projects
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
52 the input image into a lower-dimensional feature space.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
53 Why learn a representation? Current practice in the computer vision
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
54 literature converts the raw pixels into a hand-crafted representation
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
55 (e.g.\ SIFT features~\cite{Lowe04}), but deep learning algorithms
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
56 tend to discover similar features in their first few
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
57 levels~\cite{HonglakL2008,ranzato-08,Koray-08,VincentPLarochelleH2008-very-small}.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
58 Learning increases the
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
59 ease and practicality of developing representations that are at once
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
60 tailored to specific tasks, yet are able to borrow statistical strength
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
61 from other related tasks (e.g., modeling different kinds of objects). Finally, learning the
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
62 feature representation can lead to higher-level (more abstract, more
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
63 general) features that are more robust to unanticipated sources of
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
64 variance extant in real data.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
65
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
66 Whereas a deep architecture can in principle be more powerful than a shallow
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
67 one in terms of representation, depth appears to render the training problem
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
68 more difficult in terms of optimization and local minima.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
69 It is also only recently that
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
70 successful algorithms were proposed to overcome some of these
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
71 difficulties.
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
72
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
73 \section{Perturbation and Transformation of Character Images}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
74
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
75 \subsection{Affine Transformations}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
76 \subsection{Adding Slant}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
77 \subsection{Local Elastic Deformations}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
78 \subsection{Changing Thickness}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
79 \subsection{Occlusion}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
80 \subsection{Background Images}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
81 \subsection{Salt and Pepper Noise}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
82 \subsection{Spatially Gaussian Noise}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
83 \subsection{Color and Contrast Changes}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
84
393
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
85 \begin{figure}[h]
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
86 \resizebox{.99\textwidth}{!}{\includegraphics{images/example_t.png}}\\
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
87 \caption{Illustration of the pipeline of stochastic
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
88 transformations applied to the image of a lower-case t
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
89 (the upper left image). Each image in the pipeline (going from
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
90 left to right, first top line, then bottom line) shows the result
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
91 of applying one of the modules in the pipeline. The last image
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
92 (bottom right) is used as training example.}
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
93 \label{fig:pipeline}
4c840798d290 added examples of figure and table of results
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 392
diff changeset
94 \end{figure}
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
95
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
96 \section{Learning Algorithms for Deep Architectures}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
97
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
98 \section{Experimental Setup}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
99
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
100 \subsection{Training Datasets}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
101
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
102 \subsubsection{Data Sources}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
103
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
104 \begin{itemize}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
105 \item {\bf NIST}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
106 \item {\bf Fonts}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
107 \item {\bf Captchas}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
108 \item {\bf OCR data}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
109 \end{itemize}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
110
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
111 \subsubsection{Data Sets}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
112 \begin{itemize}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
113 \item {\bf NIST}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
114 \item {\bf P07}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
115 \item {\bf NISTP} {\em ne pas utiliser PNIST mais NISTP, pour rester politically correct...}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
116 \end{itemize}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
117
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
118 \subsection{Models and their Hyperparameters}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
119
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
120 \subsubsection{Multi-Layer Perceptrons (MLP)}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
121
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
122 \subsubsection{Stacked Denoising Auto-Encoders (SDAE)}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
123
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
124 \section{Experimental Results}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
125
392
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
126 \subsection{SDA vs MLP}
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
127
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
128 \begin{center}
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
129 \begin{tabular}{lcc}
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
130 & train w/ & train w/ \\
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
131 & NIST & P07 + NIST \\ \hline
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
132 SDA & & \\ \hline
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
133 MLP & & \\ \hline
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
134 \end{tabular}
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
135 \end{center}
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
136
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
137 \subsection{Perturbed Training Data More Helpful for SDAE}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
138
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
139 \subsection{Training with More Classes than Necessary}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
140
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
141 \section{Conclusions}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
142
392
5f8fffd7347f possible image for illustrating perturbations
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 381
diff changeset
143 \bibliography{strings,ml,aigaion}
379
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
144 \bibliographystyle{mlapa}
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
145
a21a174c1c18 added writeup skeleton
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents:
diff changeset
146 \end{document}