Mercurial > ift6266
diff writeup/nips2010_submission.tex @ 516:092dae9a5040
make the reference more compact.
author | Frederic Bastien <nouiz@nouiz.org> |
---|---|
date | Tue, 01 Jun 2010 14:08:44 -0400 |
parents | 920a38715c90 |
children | 460a4e78c9a4 |
line wrap: on
line diff
--- a/writeup/nips2010_submission.tex Tue Jun 01 14:08:14 2010 -0400 +++ b/writeup/nips2010_submission.tex Tue Jun 01 14:08:44 2010 -0400 @@ -323,7 +323,7 @@ \vspace*{-1mm} Whereas much previous work on deep learning algorithms had been performed on -the MNIST digits classification task~\citep{Hinton06,ranzato-07,Bengio-nips-2006,Salakhutdinov+Hinton-2009}, +the MNIST digits classification task~\citep{Hinton06,ranzato-07-small,Bengio-nips-2006,Salakhutdinov+Hinton-2009}, with 60~000 examples, and variants involving 10~000 examples~\citep{Larochelle-jmlr-toappear-2008,VincentPLarochelleH2008}, we want to focus here on the case of much larger training sets, from 10 times to @@ -359,12 +359,12 @@ {\bf NIST.} Our main source of characters is the NIST Special Database 19~\citep{Grother-1995}, widely used for training and testing character -recognition systems~\citep{Granger+al-2007,Cortes+al-2000,Oliveira+al-2002,Milgram+al-2005}. +recognition systems~\citep{Granger+al-2007,Cortes+al-2000,Oliveira+al-2002-short,Milgram+al-2005}. The dataset is composed with 814255 digits and characters (upper and lower cases), with hand checked classifications, extracted from handwritten sample forms of 3600 writers. The characters are labelled by one of the 62 classes corresponding to "0"-"9","A"-"Z" and "a"-"z". The dataset contains 8 series of different complexity. The fourth series, $hsf_4$, experimentally recognized to be the most difficult one is recommended -by NIST as testing set and is used in our work and some previous work~\citep{Granger+al-2007,Cortes+al-2000,Oliveira+al-2002,Milgram+al-2005} +by NIST as testing set and is used in our work and some previous work~\citep{Granger+al-2007,Cortes+al-2000,Oliveira+al-2002-short,Milgram+al-2005} for that purpose. We randomly split the remainder into a training set and a validation set for model selection. The sizes of these data sets are: 651668 for training, 80000 for validation, and 82587 for testing. @@ -453,7 +453,7 @@ {\bf Stacked Denoising Auto-Encoders (SDA).} Various auto-encoder variants and Restricted Boltzmann Machines (RBMs) can be used to initialize the weights of each layer of a deep MLP (with many hidden -layers)~\citep{Hinton06,ranzato-07,Bengio-nips-2006} +layers)~\citep{Hinton06,ranzato-07-small,Bengio-nips-2006} enabling better generalization, apparently setting parameters in the basin of attraction of supervised gradient descent yielding better generalization~\citep{Erhan+al-2010}. It is hypothesized that the @@ -501,7 +501,7 @@ SDA2), along with the previous results on the digits NIST special database 19 test set from the literature respectively based on ARTMAP neural networks ~\citep{Granger+al-2007}, fast nearest-neighbor search -~\citep{Cortes+al-2000}, MLPs ~\citep{Oliveira+al-2002}, and SVMs +~\citep{Cortes+al-2000}, MLPs ~\citep{Oliveira+al-2002-short}, and SVMs ~\citep{Milgram+al-2005}. More detailed and complete numerical results (figures and tables, including standard errors on the error rates) can be found in the supplementary material. The 3 kinds of model differ in the @@ -546,7 +546,7 @@ of all models, on 3 different test sets corresponding to the three datasets. Right: error rates on NIST test digits only, along with the previous results from -literature~\citep{Granger+al-2007,Cortes+al-2000,Oliveira+al-2002,Milgram+al-2005} +literature~\citep{Granger+al-2007,Cortes+al-2000,Oliveira+al-2002-short,Milgram+al-2005} respectively based on ART, nearest neighbors, MLPs, and SVMs.} \label{fig:error-rates-charts}