ift6266: writeup/mlj_submission.tex comparison

comparison writeup/mlj_submission.tex @ 590:4672fb6b4385

Changé mlj_submission.tex pour merge

author	fsavard
date	Thu, 30 Sep 2010 17:54:56 -0400
parents	9a6abcf143e8
children	8bd4ff0c5c05

comparison

equal deleted inserted replaced

-:9a6abcf143e8
+:4672fb6b4385
 It is also only recently that successful algorithms were proposed to
 overcome some of these difficulties.  All are based on unsupervised
 learning, often in an greedy layer-wise ``unsupervised pre-training''
 stage~\citep{Bengio-2009}.  One of these layer initialization techniques,
 applied here, is the Denoising
-Auto-encoder~(DA)~\citep{VincentPLarochelleH2008} (see Figure~\ref{fig:da}),
+Auto-encoder~(DA)~\citep{VincentPLarochelleH2008-very-small} (see Figure~\ref{fig:da}),
 which
 performed similarly or better than previously proposed Restricted Boltzmann
 Machines in terms of unsupervised extraction of a hierarchy of features
 useful for classification. Each layer is trained to denoise its
 input, creating a layer of features that can be used as input for the next layer.
 \begin{wrapfigure}[8]{l}{0.15\textwidth}
 %\begin{minipage}[b]{0.14\linewidth}
 %\vspace*{-5mm}
 \begin{center}
-\includegraphics[scale=.4]{images/Original.png}\\
+\includegraphics[scale=.4]{Original.png}\\
 {\bf Original}
 \end{center}
 \end{wrapfigure}
 %%\vspace{0.7cm}
 %\end{minipage}%
 %\begin{wrapfigure}[7]{l}{0.15\textwidth}
 \begin{minipage}[b]{0.14\linewidth}
 %\centering
 \begin{center}
 \vspace*{-5mm}
-\includegraphics[scale=.4]{images/Thick_only.png}\\
+\includegraphics[scale=.4]{Thick_only.png}\\
 %{\bf Thickness}
 \end{center}
 \vspace{.6cm}
 \end{minipage}%
 \hspace{0.3cm}\begin{minipage}[b]{0.86\linewidth}
 \subsubsection*{Slant}
 \vspace*{2mm}
 \begin{minipage}[b]{0.14\linewidth}
 \centering
-\includegraphics[scale=.4]{images/Slant_only.png}\\
+\includegraphics[scale=.4]{Slant_only.png}\\
 %{\bf Slant}
 \end{minipage}%
 \hspace{0.3cm}
 \begin{minipage}[b]{0.83\linewidth}
 %\centering
 \begin{minipage}[b]{0.14\linewidth}
 %\centering
 %\begin{wrapfigure}[8]{l}{0.15\textwidth}
 \begin{center}
-\includegraphics[scale=.4]{images/Affine_only.png}
+\includegraphics[scale=.4]{Affine_only.png}
 \vspace*{6mm}
 %{\small {\bf Affine \mbox{Transformation}}}
 \end{center}
 %\end{wrapfigure}
 \end{minipage}%
 %\hspace*{-8mm}
 \begin{minipage}[b]{0.14\linewidth}
 %\centering
 \begin{center}
 \vspace*{5mm}
-\includegraphics[scale=.4]{images/Localelasticdistorsions_only.png}
+\includegraphics[scale=.4]{Localelasticdistorsions_only.png}
 %{\bf Local Elastic Deformation}
 \end{center}
 %\end{wrapfigure}
 \end{minipage}%
 \hspace{3mm}
 \begin{minipage}[b]{0.14\linewidth}
 %\centering
 %\begin{wrapfigure}[7]{l}{0.15\textwidth}
 %\vspace*{-5mm}
 \begin{center}
-\includegraphics[scale=.4]{images/Pinch_only.png}\\
+\includegraphics[scale=.4]{Pinch_only.png}\\
 \vspace*{15mm}
 %{\bf Pinch}
 \end{center}
 %\end{wrapfigure}
 %%\vspace{.6cm}
 %%\vspace*{-.2cm}
 \begin{minipage}[t]{0.14\linewidth}
 \centering
 \vspace*{0mm}
-\includegraphics[scale=.4]{images/Motionblur_only.png}
+\includegraphics[scale=.4]{Motionblur_only.png}
 %{\bf Motion Blur}
 \end{minipage}%
 \hspace{0.3cm}\begin{minipage}[t]{0.83\linewidth}
 %%\vspace*{.5mm}
 \vspace*{2mm}
 \subsubsection*{Occlusion}
 \begin{minipage}[t]{0.14\linewidth}
 \centering
 \vspace*{3mm}
-\includegraphics[scale=.4]{images/occlusion_only.png}\\
+\includegraphics[scale=.4]{occlusion_only.png}\\
 %{\bf Occlusion}
 %%\vspace{.5cm}
 \end{minipage}%
 \hspace{0.3cm}\begin{minipage}[t]{0.83\linewidth}
 %\vspace*{-18mm}
 %\vspace*{-6mm}
 \begin{minipage}[t]{0.14\linewidth}
 \begin{center}
 %\centering
 \vspace*{6mm}
-\includegraphics[scale=.4]{images/Bruitgauss_only.png}
+\includegraphics[scale=.4]{Bruitgauss_only.png}
 %{\bf Gaussian Smoothing}
 \end{center}
 %\end{wrapfigure}
 %%\vspace{.5cm}
 \end{minipage}%
 \begin{minipage}[t]{0.14\textwidth}
 %\begin{wrapfigure}[7]{l}{
 %\vspace*{-5mm}
 \begin{center}
 \vspace*{1mm}
-\includegraphics[scale=.4]{images/Permutpixel_only.png}
+\includegraphics[scale=.4]{Permutpixel_only.png}
 %{\small\bf Permute Pixels}
 \end{center}
 %\end{wrapfigure}
 \end{minipage}%
 \hspace{3mm}\begin{minipage}[t]{0.86\linewidth}
 %%\vspace*{-3mm}
 \begin{center}
 %\hspace*{-3mm}\begin{minipage}[t]{0.18\linewidth}
 %\centering
 \vspace*{0mm}
-\includegraphics[scale=.4]{images/Distorsiongauss_only.png}
+\includegraphics[scale=.4]{Distorsiongauss_only.png}
 %{\small \bf Gauss. Noise}
 \end{center}
 %\end{wrapfigure}
 \end{minipage}%
 \hspace{0.3cm}\begin{minipage}[t]{0.86\linewidth}
 \begin{minipage}[t]{\linewidth}
 \begin{minipage}[t]{0.14\linewidth}
 \centering
 \vspace*{0mm}
-\includegraphics[scale=.4]{images/background_other_only.png}
+\includegraphics[scale=.4]{background_other_only.png}
 %{\small \bf Bg Image}
 \end{minipage}%
 \hspace{0.3cm}\begin{minipage}[t]{0.83\linewidth}
 \vspace*{1mm}
 Following~\citet{Larochelle-jmlr-2009}, the {\bf background image} module adds a random
 \subsubsection*{Salt and Pepper Noise}
 \begin{minipage}[t]{0.14\linewidth}
 \centering
 \vspace*{0mm}
-\includegraphics[scale=.4]{images/Poivresel_only.png}
+\includegraphics[scale=.4]{Poivresel_only.png}
 %{\small \bf Salt \& Pepper}
 \end{minipage}%
 \hspace{0.3cm}\begin{minipage}[t]{0.83\linewidth}
 \vspace*{1mm}
 The {\bf salt and pepper noise} module adds noise $\sim U[0,1]$ to random subsets of pixels.
 %\begin{minipage}[t]{0.14\linewidth}
 %\centering
 \begin{center}
 \vspace*{4mm}
 %\hspace*{-1mm}
-\includegraphics[scale=.4]{images/Rature_only.png}\\
+\includegraphics[scale=.4]{Rature_only.png}\\
 %{\bf Scratches}
 \end{center}
 \end{minipage}%
 %\end{wrapfigure}
 \hspace{0.3cm}\begin{minipage}[t]{0.86\linewidth}
 \subsubsection*{Grey Level and Contrast Changes}
 \begin{minipage}[t]{0.15\linewidth}
 \centering
 \vspace*{0mm}
-\includegraphics[scale=.4]{images/Contrast_only.png}
+\includegraphics[scale=.4]{Contrast_only.png}
 %{\bf Grey Level \& Contrast}
 \end{minipage}%
 \hspace{3mm}\begin{minipage}[t]{0.85\linewidth}
 \vspace*{1mm}
 The {\bf grey level and contrast} module changes the contrast by changing grey levels, and may invert the image polarity (white
 deep architecture (whereby complex concepts are expressed as
 compositions of simpler ones through a deep hierarchy).
 \begin{figure}[ht]
 %\vspace*{-2mm}
-\centerline{\resizebox{0.8\textwidth}{!}{\includegraphics{images/denoising_autoencoder_small.pdf}}}
+\centerline{\resizebox{0.8\textwidth}{!}{\includegraphics{denoising_autoencoder_small.pdf}}}
 %\vspace*{-2mm}
 \caption{Illustration of the computations and training criterion for the denoising
 auto-encoder used to pre-train each layer of the deep architecture. Input $x$ of
 the layer (i.e. raw input or output of previous layer)
 s corrupted into $\tilde{x}$ and encoded into code $y$ by the encoder $f_\theta(\cdot)$.
 %\vspace*{-1mm}
 \begin{figure}[ht]
 %\vspace*{-2mm}
-\centerline{\resizebox{.99\textwidth}{!}{\includegraphics{images/error_rates_charts.pdf}}}
+\centerline{\resizebox{.99\textwidth}{!}{\includegraphics{error_rates_charts.pdf}}}
 %\vspace*{-3mm}
 \caption{SDAx are the {\bf deep} models. Error bars indicate a 95\% confidence interval. 0 indicates that the model was trained
 on NIST, 1 on NISTP, and 2 on P07. Left: overall results
 of all models, on NIST and NISTP test sets.
 Right: error rates on NIST test digits only, along with the previous results from
 \end{figure}
 \begin{figure}[ht]
 %\vspace*{-3mm}
-\centerline{\resizebox{.99\textwidth}{!}{\includegraphics{images/improvements_charts.pdf}}}
+\centerline{\resizebox{.99\textwidth}{!}{\includegraphics{improvements_charts.pdf}}}
 %\vspace*{-3mm}
 \caption{Relative improvement in error rate due to self-taught learning.
 Left: Improvement (or loss, when negative)
 induced by out-of-distribution examples (perturbed data).
 Right: Improvement (or loss, when negative) induced by multi-task

Mercurial > ift6266

comparison writeup/mlj_submission.tex @ 590:4672fb6b4385