annotate writeup/aigaion-shorter.bib @ 640:9910b921c9a6

added rules depency again *.bib file.
author Frederic Bastien <nouiz@nouiz.org>
date Mon, 21 Mar 2011 11:26:56 -0400
parents 51213beaed8b
children
rev   line source
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1 %Aigaion2 BibTeX export from LISA - Publications
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2 %Tuesday 02 November 2010 04:10:50 PM
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3 @MASTERSTHESIS{,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4 author = {Breuleux, Olivier},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
5 title = {{\'{E}}chantillonnage dynamique de champs markoviens},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
6 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
7 school = {Universit{\'{e}} de Montr{\'{e}}al}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
8 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
9
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
10 @PHDTHESIS{,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
11 author = {Rivest, Fran{\c c}ois},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
12 title = {Mod{\`{e}}le informatique du coapprentissage des ganglions de la base et du cortex : L’apprentissage par renforcement et le d{\'{e}}veloppement de repr{\'{e}}sentations},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
13 year = {2009},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
14 school = {Universit{\'{e}} de Montr{\'{e}}al, D{\'{e}}partement d’informatique et de recherche op{\'{e}}rationnelle},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
15 abstract = {English follow:
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
16
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
17 Tout au long de la vie, le cerveau d{\'{e}}veloppe des repr{\'{e}}sentations de son
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
18 environnement permettant {\`{a}} l’individu d’en tirer meilleur profit. Comment ces
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
19 repr{\'{e}}sentations se d{\'{e}}veloppent-elles pendant la qu{\^{e}}te de r{\'{e}}compenses demeure un
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
20 myst{\`{e}}re. Il est raisonnable de penser que le cortex est le si{\`{e}}ge de ces repr{\'{e}}sentations
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
21 et que les ganglions de la base jouent un r{\^{o}}le important dans la maximisation des
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
22 r{\'{e}}compenses. En particulier, les neurones dopaminergiques semblent coder un signal
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
23 d’erreur de pr{\'{e}}diction de r{\'{e}}compense. Cette th{\`{e}}se {\'{e}}tudie le probl{\`{e}}me en construisant,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
24 {\`{a}} l’aide de l’apprentissage machine, un mod{\`{e}}le informatique int{\'{e}}grant de nombreuses
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
25 {\'{e}}vidences neurologiques.
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
26 Apr{\`{e}}s une introduction au cadre math{\'{e}}matique et {\`{a}} quelques algorithmes de
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
27 l’apprentissage machine, un survol de l’apprentissage en psychologie et en
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
28 neuroscience et une revue des mod{\`{e}}les de l’apprentissage dans les ganglions de la
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
29 base, la th{\`{e}}se comporte trois articles. Le premier montre qu’il est possible
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
30 d’apprendre {\`{a}} maximiser ses r{\'{e}}compenses tout en d{\'{e}}veloppant de meilleures
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
31 repr{\'{e}}sentations des entr{\'{e}}es. Le second article porte sur l'important probl{\`{e}}me toujours
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
32 non r{\'{e}}solu de la repr{\'{e}}sentation du temps. Il d{\'{e}}montre qu’une repr{\'{e}}sentation du temps
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
33 peut {\^{e}}tre acquise automatiquement dans un r{\'{e}}seau de neurones artificiels faisant
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
34 office de m{\'{e}}moire de travail. La repr{\'{e}}sentation d{\'{e}}velopp{\'{e}}e par le mod{\`{e}}le ressemble
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
35 beaucoup {\`{a}} l’activit{\'{e}} de neurones corticaux dans des t{\^{a}}ches similaires. De plus, le
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
36 mod{\`{e}}le montre que l’utilisation du signal d’erreur de r{\'{e}}compense peut acc{\'{e}}l{\'{e}}rer la
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
37 construction de ces repr{\'{e}}sentations temporelles. Finalement, il montre qu’une telle
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
38 repr{\'{e}}sentation acquise automatiquement dans le cortex peut fournir l’information
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
39 n{\'{e}}cessaire aux ganglions de la base pour expliquer le signal dopaminergique. Enfin,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
40 le troisi{\`{e}}me article {\'{e}}value le pouvoir explicatif et pr{\'{e}}dictif du mod{\`{e}}le sur diff{\'{e}}rentes
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
41 situations comme la pr{\'{e}}sence ou l’absence d’un stimulus (conditionnement classique
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
42 ou de trace) pendant l’attente de la r{\'{e}}compense. En plus de faire des pr{\'{e}}dictions tr{\`{e}}s
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
43 int{\'{e}}ressantes en lien avec la litt{\'{e}}rature sur les intervalles de temps, l’article r{\'{e}}v{\`{e}}le
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
44 certaines lacunes du mod{\`{e}}le qui devront {\^{e}}tre am{\'{e}}lior{\'{e}}es.
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
45 Bref, cette th{\`{e}}se {\'{e}}tend les mod{\`{e}}les actuels de l’apprentissage des ganglions de
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
46 la base et du syst{\`{e}}me dopaminergique au d{\'{e}}veloppement concurrent de
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
47 repr{\'{e}}sentations temporelles dans le cortex et aux interactions de ces deux structures.
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
48
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
49 Throughout lifetime, the brain develops abstract representations of its
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
50 environment that allow the individual to maximize his benefits. How these
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
51 representations are developed while trying to acquire rewards remains a mystery. It is
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
52 reasonable to assume that these representations arise in the cortex and that the basal
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
53 ganglia are playing an important role in reward maximization. In particular,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
54 dopaminergic neurons appear to code a reward prediction error signal. This thesis
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
55 studies the problem by constructing, using machine learning tools, a computational
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
56 model that incorporates a number of relevant neurophysiological findings.
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
57 After an introduction to the machine learning framework and to some of its
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
58 algorithms, an overview of learning in psychology and neuroscience, and a review of
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
59 models of learning in the basal ganglia, the thesis comprises three papers. The first
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
60 article shows that it is possible to learn a better representation of the inputs while
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
61 learning to maximize reward. The second paper addresses the important and still
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
62 unresolved problem of the representation of time in the brain. The paper shows that a
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
63 time representation can be acquired automatically in an artificial neural network
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
64 acting like a working memory. The representation learned by the model closely
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
65 resembles the activity of cortical neurons in similar tasks. Moreover, the model shows
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
66 that the reward prediction error signal could accelerate the development of the
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
67 temporal representation. Finally, it shows that if such a learned representation exists
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
68 in the cortex, it could provide the necessary information to the basal ganglia to
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
69 explain the dopaminergic signal. The third article evaluates the explanatory and
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
70 predictive power of the model on the effects of differences in task conditions such as
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
71 the presence or absence of a stimulus (classical versus trace conditioning) while
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
72 waiting for the reward. Beyond making interesting predictions relevant to the timing
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
73 literature, the paper reveals some shortcomings of the model that will need to be
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
74 resolved.
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
75 In summary, this thesis extends current models of reinforcement learning of
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
76 the basal ganglia and the dopaminergic system to the concurrent development of
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
77 representation in the cortex and to the interactions between these two regions.}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
78 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
79
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
80 @MASTERSTHESIS{,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
81 author = {Wood, Sean},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
82 title = {Non-negative matrix decomposition approaches to frequency domain analysis of music audio signals},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
83 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
84 school = {Universit{\'{e}} de Montr{\'{e}}al}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
85 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
86
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
87 @TECHREPORT{ARXIV-2010,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
88 author = {Bastien, Fr{\'{e}}d{\'{e}}ric and Bengio, Yoshua and Bergeron, Arnaud and Boulanger-Lewandowski, Nicolas and Breuel, Thomas and Chherawala, Youssouf and Cisse, Moustapha and C{\^{o}}t{\'{e}}, Myriam and Erhan, Dumitru and Eustache, Jeremy and Glorot, Xavier and Muller, Xavier and Pannetier Lebeuf, Sylvain and Pascanu, Razvan and Rifai, Salah and Savard, Fran{\c c}ois and Sicard, Guillaume},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
89 keywords = {Computer Vision and Pattern Recognition, Learning, Neural and Evolutionary Computing},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
90 title = {Deep Self-Taught Learning for Handwritten Character Recognition},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
91 number = {1353},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
92 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
93 institution = {University of Montr{\'{e}}al},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
94 abstract = {Recent theoretical and empirical work in statistical machine learning has demonstrated the importance of learning algorithms for deep architectures, i.e., function classes obtained by composing multiple non-linear transformations. Self-taught learning (exploiting unlabeled examples or examples from other distributions) has already been applied to deep learners, but mostly to show the advantage of unlabeled examples. Here we explore the advantage brought by {\em out-of-distribution examples}. For this purpose we developed a powerful generator of stochastic variations and noise processes for character images, including not only affine transformations but also slant, local elastic deformations, changes in thickness, background images, grey level changes, contrast, occlusion, and various types of noise. The out-of-distribution examples are obtained from these highly distorted images or by including examples of object classes different from those in the target test set. We show that {\em deep learners benefit more from out-of-distribution examples than a corresponding shallow learner}, at least in the area of handwritten character recognition. In fact, we show that they beat previously published results and reach human-level performance on both handwritten digit classification and 62-class handwritten character recognition.}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
95 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
96
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
97 @INPROCEEDINGS{Attardi+al-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
98 author = {Attardi, Giuseppe and Dell'Orletta, Felice and Simi, Maria and Turian, Joseph},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
99 keywords = {classifier, dependency parsing, natural language, parser, perceptron},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
100 title = {Accurate Dependency Parsing with a Stacked Multilayer Perceptron},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
101 booktitle = {Proceeding of Evalita 2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
102 series = {LNCS},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
103 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
104 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
105 abstract = {Abstract. DeSR is a statistical transition-based dependency parser which learns from annotated corpora which actions to perform for building parse trees while scanning a sentence. We describe recent improvements to the parser, in particular stacked parsing, exploiting a beam search strategy and using a Multilayer Perceptron classifier. For the Evalita 2009 Dependency Parsing task DesR was configured to use a combination of stacked parsers. The stacked combination achieved the best accuracy scores in both the main and pilot subtasks. The contribution to the result of various choices is analyzed, in particular for taking advantage of the peculiar features of the TUT Treebank.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
106 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
107
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
108 @INPROCEEDINGS{Bengio+al-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
109 author = {Bengio, Yoshua and Louradour, Jerome and Collobert, Ronan and Weston, Jason},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
110 title = {Curriculum Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
111 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
112 crossref = {ICML09-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
113 abstract = {Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them 'curriculum learning'. In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved by using a particular curriculum, i.e., the selection and order of training examples. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
114 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
115
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
116 @TECHREPORT{Bengio+al-2009-TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
117 author = {Bengio, Yoshua and Louradour, Jerome and Collobert, Ronan and Weston, Jason},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
118 title = {Curriculum Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
119 number = {1330},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
120 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
121 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
122 abstract = {Humans and animals learn much better when the examples are not randomly presented but organized in a meaningful order which illustrates gradually more concepts, and gradually more complex ones. Here, we formalize such training strategies in the context of machine learning, and call them 'curriculum learning'. In the context of recent research studying the difficulty of training in the presence of non-convex training criteria (for deep deterministic and stochastic neural networks), we explore curriculum learning in various set-ups. The experiments show that significant improvements in generalization can be achieved. We hypothesize that curriculum learning has both an effect on the speed of convergence of the training process to a minimum and, in the case of non-convex criteria, on the quality of the local minima obtained: curriculum learning can be seen as a particular form of continuation method (a general strategy for global optimization of non-convex functions).}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
123 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
124
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
125 @MISC{Bengio+al-patent-2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
126 author = {Bengio, Yoshua and Bottou, {L{\'{e}}on} and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
127 title = {Module for constructing trainable modular network in which each module outputs and inputs data structured as a graph},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
128 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
129 howpublished = {U.S. Patent 6,128,606, October 3}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
130 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
131
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
132 @MISC{Bengio+al-patent-2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
133 author = {Bengio, Yoshua and Bottou, {L{\'{e}}on} and G. Howard, Paul},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
134 title = {Z-Coder : a fast adaptive binary arithmetic coder},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
135 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
136 howpublished = {U.S. Patent 6,188,334, February 13, 2001, along with patents 6,225,925, 6,281,817, and 6,476,740}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
137 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
138
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
139 @MISC{Bengio+al-patent-94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
140 author = {Bengio, Yoshua and {LeCun}, Yann and Nohl, Craig and Burges, Chris},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
141 title = {Visitor Registration System Using Automatic Handwriting Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
142 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
143 howpublished = {Patent submitted in the U.S.A. in October 1994, submission number 1-16-18-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
144 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
145
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
146 @INCOLLECTION{Bengio+al-spectral-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
147 author = {Bengio, Yoshua and Delalleau, Olivier and Le Roux, Nicolas and Paiement, Jean-Fran{\c c}ois and Vincent, Pascal and Ouimet, Marie},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
148 editor = {Guyon, Isabelle and Gunn, Steve and Nikravesh, Masoud and Zadeh, Lofti},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
149 title = {Spectral Dimensionality Reduction},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
150 booktitle = {Feature Extraction, Foundations and Applications},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
151 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
152 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
153 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/eigenfn_chapter.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
154 abstract = {In this chapter, we study and put under a common framework a number
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
155 of non-linear dimensionality reduction methods, such as Locally Linear Embedding,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
156 Isomap, Laplacian eigenmaps and kernel {PCA}, which are based
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
157 on performing an eigen-decomposition (hence the name "spectral"). That
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
158 framework also includes classical methods such as {PCA} and metric multidimensional
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
159 scaling ({MDS}). It also includes the data transformation step used
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
160 in spectral clustering. We show that in all of these cases the learning algorithm
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
161 estimates the principal eigenfunctions of an operator that depends on
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
162 the unknown data density and on a kernel that is not necessarily positive
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
163 semi-definite. This helps to generalize some of these algorithms so as to predict
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
164 an embedding for out-of-sample examples without having to retrain the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
165 model. It also makes it more transparent what these algorithm are minimizing
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
166 on the empirical data and gives a corresponding notion of generalization
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
167 error.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
168 cat={B},topics={HighDimensional,Kernel,Unsupervised},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
169 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
170
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
171 @INCOLLECTION{Bengio+al-ssl-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
172 author = {Bengio, Yoshua and Delalleau, Olivier and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
173 editor = {Chapelle, Olivier and {Sch{\"{o}}lkopf}, Bernhard and Zien, Alexander},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
174 title = {Label Propagation and Quadratic Criterion},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
175 booktitle = {Semi-Supervised Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
176 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
177 pages = {193--216},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
178 publisher = {{MIT} Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
179 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_ssl.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
180 abstract = {Various graph-based algorithms for semi-supervised learning have been proposed in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
181 the recent literature. They rely on the idea of building a graph whose nodes are
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
182 data points (labeled and unlabeled) and edges represent similarities between points.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
183 Known labels are used to propagate information through the graph in order to label
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
184 all nodes. In this chapter, we show how these different algorithms can be cast into
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
185 a common framework where one minimizes a quadratic cost criterion whose closed-form solution is found by solving a linear system of size n (total number of data
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
186 points). The cost criterion naturally leads to an extension of such algorithms to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
187 the inductive setting, where one obtains test samples one at a time: the derived
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
188 induction formula can be evaluated in O(n) time, which is much more efficient
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
189 than solving again exactly the linear system (which in general costs O(kn2) time
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
190 for a sparse graph where each data point has k neighbors). We also use this inductive
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
191 formula to show that when the similarity between points satisfies a locality property,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
192 then the algorithms are plagued by the curse of dimensionality, with respect to the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
193 dimensionality of an underlying manifold.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
194 cat={B},topics={Unsupervised},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
195 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
196
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
197 @TECHREPORT{Bengio+al-treecurse-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
198 author = {Bengio, Yoshua and Delalleau, Olivier and Simard, Clarence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
199 title = {Decision Trees do not Generalize to New Variations},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
200 number = {1304},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
201 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
202 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
203 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio+al-tr1304.pdf}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
204 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
205
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
206 @INPROCEEDINGS{Bengio+Bengio96,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
207 author = {Bengio, Samy and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
208 editor = {Xu, L.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
209 title = {An {EM} Algorithm for Asynchronous Input/Output Hidden {M}arkov Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
210 booktitle = {International Conference On Neural Information Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
211 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
212 pages = {328--334},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
213 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/iconip96.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
214 abstract = {In learning tasks in which input sequences are mapped to output sequences, it is often the case that the input and output sequences are not synchronous. For example, in speech recognition, acoustic sequences are longer than phoneme sequences. Input/Output Hidden {Markov} Models have already been proposed to represent the distribution of an output sequence given an input sequence of the same length. We extend here this model to the case of asynchronous sequences_ and show an Expectation-Maximization algorithm for training such models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
215 topics={Markov},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
216 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
217
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
218 @INCOLLECTION{Bengio+chapter2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
219 author = {Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
220 editor = {Bottou, {L{\'{e}}on} and Chapelle, Olivier and DeCoste, D. and Weston, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
221 title = {Scaling Learning Algorithms towards {AI}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
222 booktitle = {Large Scale Kernel Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
223 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
224 publisher = {MIT Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
225 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio+lecun_chapter2007.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
226 abstract = {One long-term goal of machine learning research is to produce methods that
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
227 are applicable to highly complex tasks, such as perception (vision, audition), reasoning,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
228 intelligent control, and other artificially intelligent behaviors. We argue
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
229 that in order to progress toward this goal, the Machine Learning community must
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
230 endeavor to discover algorithms that can learn highly complex functions, with minimal
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
231 need for prior knowledge, and with minimal human intervention. We present
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
232 mathematical and empirical evidence suggesting that many popular approaches
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
233 to non-parametric learning, particularly kernel methods, are fundamentally limited
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
234 in their ability to learn complex high-dimensional functions. Our analysis
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
235 focuses on two problems. First, kernel machines are shallow architectures, in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
236 which one large layer of simple template matchers is followed by a single layer
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
237 of trainable coefficients. We argue that shallow architectures can be very inefficient
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
238 in terms of required number of computational elements and examples. Second,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
239 we analyze a limitation of kernel machines with a local kernel, linked to the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
240 curse of dimensionality, that applies to supervised, unsupervised (manifold learning)
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
241 and semi-supervised kernel machines. Using empirical results on invariant
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
242 image recognition tasks, kernel methods are compared with deep architectures, in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
243 which lower-level features or concepts are progressively combined into more abstract
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
244 and higher-level representations. We argue that deep architectures have the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
245 potential to generalize in non-local ways, i.e., beyond immediate neighbors, and
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
246 that this is crucial in order to make progress on the kind of complex tasks required
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
247 for artificial intelligence.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
248 cat={B},topics={HighDimensional},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
249 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
250
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
251 @ARTICLE{Bengio+Delalleau-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
252 author = {Bengio, Yoshua and Delalleau, Olivier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
253 title = {Justifying and Generalizing Contrastive Divergence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
254 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
255 volume = {21},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
256 number = {6},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
257 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
258 pages = {1601--1621},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
259 abstract = {We study an expansion of the log-likelihood in undirected graphical models such as the Restricted {Boltzmann} Machine (RBM), where each term in the expansion is associated with a sample in a Gibbs chain alternating between two random variables (the visible vector and the hidden vector, in RBMs). We are particularly interested in estimators of the gradient of the log-likelihood obtained through this expansion. We show that its residual term converges to zero, justifying the use of a truncation, i.e. running only a short Gibbs chain, which is the main idea behind the Contrastive Divergence (CD) estimator of the log-likelihood gradient. By truncating even more, we obtain a stochastic reconstruction error, related through a mean-field approximation to the reconstruction error often used to train autoassociators and stacked auto-associators. The derivation is not specific to the particular parametric forms used in RBMs, and only requires convergence of the Gibbs chain. We present theoretical and empirical evidence linking the number of Gibbs steps $k$ and the magnitude of the RBM parameters to the bias in the CD estimator. These experiments also suggest that the sign of the CD estimator is correct most of the time, even when the bias is large, so that CD-$k$ is a good descent direction even for small $k$.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
260 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
261
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
262 @TECHREPORT{Bengio+Delalleau-TR2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
263 author = {Bengio, Yoshua and Delalleau, Olivier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
264 keywords = {Contrastive Divergence, Restricted {Boltzmann} Machine},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
265 title = {Justifying and Generalizing Contrastive Divergence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
266 number = {1311},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
267 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
268 institution = {D{\'{e}}partement d'Informatique et Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
269 abstract = {We study an expansion of the log-likelihood in undirected graphical models such as the Restricted {Boltzmann} Machine (RBM), where each term in the expansion is associated with a sample in a Gibbs chain alternating between two random variables (the visible vector and the hidden vector, in RBMs). We are particularly interested in estimators of the gradient of the log-likelihood obtained through this expansion. We show that its terms converge to zero, justifying the use of a truncation, i.e. running only a short Gibbs chain, which is the main idea behind the Contrastive Divergence approximation of the log-likelihood gradient. By truncating even more, we obtain a stochastic reconstruction error, related through a mean-field approximation to the reconstruction error often used to train autoassociators and stacked auto-associators. The derivation is not specific to the particular parametric forms used in RBMs, and only requires convergence of the Gibbs chain.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
270 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
271
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
272 @INPROCEEDINGS{Bengio+DeMori88,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
273 author = {Bengio, Yoshua and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
274 title = {Use of neural networks for the recognition of place of articulation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
275 booktitle = {International Conference on Acoustics, Speech and Signal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
276 year = {1988},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
277 pages = {103--106},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
278 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
279 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
280
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
281 @INPROCEEDINGS{Bengio+DeMori89,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
282 author = {Bengio, Yoshua and Cardin, Regis and Cosi, Piero and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
283 title = {Speech coding with multi-layer networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
284 booktitle = {International Conference on Acoustics, Speech and Signal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
285 year = {1989},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
286 pages = {164--167},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
287 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
288 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
289
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
290 @INCOLLECTION{Bengio+DeMori90a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
291 author = {Bengio, Yoshua and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
292 editor = {Sethi, I. K. and Jain, A. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
293 title = {Connectionist models and their application to automatic speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
294 booktitle = {Artificial Neural Networks and Statistical Pattern Recognition: Old and New Connections},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
295 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
296 pages = {175--192},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
297 publisher = {Elsevier, Machine Intelligence and Pattern Recognition Series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
298 topics={Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
299 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
300
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
301 @ARTICLE{Bengio+Frasconi-jair95,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
302 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
303 title = {Diffusion of Context and Credit Information in {M}arkovian Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
304 journal = {Journal of Artificial Intelligence Research},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
305 volume = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
306 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
307 pages = {249--270},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
308 abstract = {This paper studies the problem of ergodicity of transition probability matrices in {Markovian} models, such as hidden {Markov} models ({HMM}s), and how it makes very difficult the task of learning to represent long-term context for sequential data. This phenomenon hurts the forward propagation of long-term context information, as well as learning a hidden state representation to represent long-term context, which depends on propagating credit information backwards in time. Using results from {Markov} chain theory, we show that this problem of diffusion of context and credit is reduced when the transition probabilities approach 0 or 1, i.e., the transition probability matrices are sparse and the model essentially deterministic. The results found in this paper apply to learning approaches based on continuous optimization, such as gradient descent and the Baum-Welch algorithm.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
309 topics={Markov,LongTerm},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
310 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
311
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
312 @INPROCEEDINGS{Bengio+Frasconi-nips7-diffuse,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
313 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
314 title = {Diffusion of Credit in {M}arkovian Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
315 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
316 pages = {553--560},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
317 crossref = {NIPS7-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
318 abstract = {This paper studies the problem of diffusion in {Markovian} models, such as hidden {Markov} models ({HMM}s) and how it makes very difficult the task of learning of long-term dependencies in sequences. Using results from {Markov} chain theory, we show that the problem of diffusion is reduced if the transition probabilities approach 0 or 1. Under this condition, standard {HMM}s have very limited modeling capabilities, but input/output {HMM}s can still perform interesting computations.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
319 topics={Markov},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
320 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
321
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
322 @INPROCEEDINGS{Bengio+Frasconi-nips7-iohmms,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
323 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
324 title = {An Input/Output {HMM} Architecture},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
325 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
326 pages = {427--434},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
327 crossref = {NIPS7-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
328 abstract = {We introduce a recurrent architecture having a modular structure and we formulate a training procedure based on the {EM} algorithm. The resulting model has similarities to hidden {Markov} models, but supports recurrent networks processing style and allows to exploit the supervised learning paradigm while using maximum likelihood estimation.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
329 topics={Markov},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
330 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
331
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
332 @INPROCEEDINGS{Bengio+Frasconi-nips94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
333 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
334 title = {Credit Assignment through Time: Alternatives to Backpropagation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
335 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
336 pages = {75--82},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
337 crossref = {NIPS6-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
338 abstract = {Learning to recognize or predict sequences using long-term context has many applications. However, practical and theoretical problems are found in training recurrent neural networks to perform tasks in which input/output dependencies span long intervals. Starting from a mathematical analysis of the problem, we consider and compare alternative algorithms and architectures on tasks for which the span of the input/output dependencies can be controlled. Results on the new algorithms show performance qualitatively superior to that obtained with backpropagation.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
339 topics={LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
340 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
341
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
342 @ARTICLE{Bengio+Pouliot90,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
343 author = {Bengio, Yoshua and Pouliot, Yannick},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
344 title = {Efficient recognition of immunoglobulin domains from amino-acid sequences using a neural network},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
345 journal = {Computer Applications in the Biosciences},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
346 volume = {6},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
347 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
348 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
349 pages = {319--324},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
350 topics={Bioinformatic,PriorKnowledge},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
351 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
352
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
353 @INPROCEEDINGS{Bengio+Senecal-2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
354 author = {Bengio, Yoshua and S{\'{e}}n{\'{e}}cal, Jean-S{\'{e}}bastien},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
355 title = {Quick Training of Probabilistic Neural Nets by Importance Sampling},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
356 booktitle = {Proceedings of the conference on Artificial Intelligence and Statistics (AISTATS)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
357 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
358 abstract = {Our previous work on statistical language modeling introduced the use of probabilistic feedforward neural networks to help dealing with the curse of dimensionality. Training this model by maximum likelihood however requires for each example to perform as many network passes as there are words in the vocabulary. Inspired by the contrastive divergence model, we propose and evaluate sampling-based methods which require network passes only for the observed "positive example'' and a few sampled negative example words. A very significant speed-up is obtained with an adaptive importance sampling.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
359 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
360
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
361 @ARTICLE{Bengio+Senecal-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
362 author = {Bengio, Yoshua and S{\'{e}}n{\'{e}}cal, Jean-S{\'{e}}bastien},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
363 keywords = {Energy-based models, fast training, importance sampling, language modeling, Monte Carlo methods, probabilistic neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
364 title = {Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
365 journal = {IEEE Transactions on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
366 volume = {19},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
367 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
368 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
369 pages = {713--722},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
370 abstract = {Previous work on statistical language modeling has shown that it is possible to train a feedforward neural network to approximate probabilities over sequences of words, resulting in significant error reduction when compared to standard baseline models based on -grams. However, training the neural network model with the maximum-likelihood criterion requires computations proportional to the number of words in the vocabulary. In this paper, we introduce adaptive importance sampling as a way to accelerate training of the model. The idea is to use an adaptive n-gram model to track the conditional distributions produced by the neural network. We show that a very significant speedup can be obtained on standard problems.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
371 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
372
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
373 @INCOLLECTION{Bengio-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
374 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
375 editor = {Cisek, Paul and Kalaska, John and Drew, Trevor},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
376 title = {On the Challenge of Learning Complex Functions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
377 booktitle = {Computational Neuroscience: Theoretical Insights into Brain Function},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
378 series = {Progress in Brain Research},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
379 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
380 publisher = {Elsevier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
381 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/PBR_chapter.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
382 abstract = {A common goal of computational neuroscience and of artificial intelligence
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
383 research based on statistical learning algorithms is the discovery and
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
384 understanding of computational principles that could explain what we
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
385 consider adaptive intelligence, in animals as well as in machines. This
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
386 chapter focuses on what is required for the learning of complex behaviors. We
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
387 believe it involves the learning of highly varying functions, in a
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
388 mathematical sense. We bring forward two types of arguments which convey
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
389 the message that many currently popular machine learning approaches to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
390 learning flexible functions have fundamental limitations that render them
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
391 inappropriate for learning highly varying functions. The first issue
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
392 concerns the representation of such functions with what we call shallow model
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
393 architectures. We discuss limitations of shallow architectures, such as
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
394 so-called kernel machines, boosting algorithms, and one-hidden-layer artificial neural
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
395 networks. The second issue is more focused and concerns kernel machines
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
396 with a local kernel (the type used most often in practice),
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
397 that act like a collection of template matching units. We present
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
398 mathematical results on such computational architectures showing that they
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
399 have a limitation similar to those already proved for older non-parametric
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
400 methods, and connected to the so-called curse of dimensionality. Though it has long
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
401 been believed that efficient learning in deep architectures is difficult,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
402 recently proposed computational principles for learning in deep architectures
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
403 may offer a breakthrough.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
404 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
405
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
406 @ARTICLE{Bengio-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
407 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
408 title = {Learning deep architectures for {AI}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
409 journal = {Foundations and Trends in Machine Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
410 volume = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
411 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
412 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
413 pages = {1--127},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
414 note = {Also published as a book. Now Publishers, 2009.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
415 abstract = {Theoretical results suggest that in order to learn the kind of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
416 complicated functions that can represent high-level abstractions (e.g. in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
417 vision, language, and other AI-level tasks), one may need {\insist deep
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
418 architectures}. Deep architectures are composed of multiple levels of non-linear
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
419 operations, such as in neural nets with many hidden layers or in complicated
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
420 propositional formulae re-using many sub-formulae. Searching the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
421 parameter space of deep architectures is a difficult task, but
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
422 learning algorithms such as those for Deep Belief Networks have recently been proposed
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
423 to tackle this problem with notable success, beating the state-of-the-art
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
424 in certain areas. This paper discusses the motivations and principles regarding
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
425 learning algorithms for deep architectures, in particular those exploiting as
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
426 building blocks unsupervised learning of single-layer models such as Restricted {Boltzmann} Machines,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
427 used to construct deeper models such as Deep Belief Networks.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
428 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
429
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
430 @TECHREPORT{Bengio-96-TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
431 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
432 title = {Using a Financial Training Criterion Rather than a Prediction Criterion},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
433 number = {\#1019},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
434 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
435 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
436 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengioy_TR1019.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
437 abstract = {The application of this work is to decision taking with financial time-series, using learning algorithms. The traditional approach is to train a model using a rediction criterion, such as minimizing the squared error between predictions and actual values of a dependent variable, or maximizing the likelihood of a conditional model of the dependent variable. We find here with noisy time-series that better results can be obtained when the model is directly trained in order to optimize the financial criterion of interest. Experiments were performed on portfolio selection with 35 Canadian stocks.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
438 topics={Finance,Discriminant},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
439 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
440
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
441 @BOOK{bengio-book96,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
442 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
443 title = {Neural Networks for Speech and Sequence Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
444 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
445 publisher = {International Thompson Computer Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
446 topics={Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
447 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
448
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
449 @TECHREPORT{Bengio-convex-05,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
450 author = {Bengio, Yoshua and Le Roux, Nicolas and Vincent, Pascal and Delalleau, Olivier and Marcotte, Patrice},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
451 title = {Convex neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
452 number = {1263},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
453 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
454 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
455 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1263.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
456 abstract = {Convexity has recently received a lot of attention in the machine learning community, and the lack of convexity has been seen as a major disadvantage of many learning algorithms, such as multi-layer artificial neural networks. We how that training multi-layer neural networks in which the number of hidden units is learned can be viewed as a convex optimization problem. This problem involves an infinite number of variables, but can be solved by incrementally inserting a hidden unit at a time, each time finding a linear classifiers that minimizes a weighted sum of errors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
457 topics={Boosting},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
458 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
459
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
460 @ARTICLE{Bengio-decision-trees10,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
461 author = {Bengio, Yoshua and Delalleau, Olivier and Simard, Clarence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
462 title = {Decision Trees do not Generalize to New Variations},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
463 journal = {Computational Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
464 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
465 note = {To appear}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
466 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
467
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
468 @ARTICLE{bengio-demori89,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
469 author = {Bengio, Yoshua and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
470 title = {Use of multilayer networks for the recognition of phonetic features and phonemes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
471 journal = {Computational Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
472 volume = {5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
473 year = {1989},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
474 pages = {134--141},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
475 topics={Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
476 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
477
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
478 @ARTICLE{Bengio-eigen-NC2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
479 author = {Bengio, Yoshua and Delalleau, Olivier and Le Roux, Nicolas and Paiement, Jean-Fran{\c c}ois and Vincent, Pascal and Ouimet, Marie},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
480 title = {Learning eigenfunctions links spectral embedding and kernel {PCA}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
481 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
482 volume = {16},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
483 number = {10},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
484 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
485 pages = {2197--2219},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
486 abstract = {In this paper, we show a direct relation between spectral embedding methods and kernel {PCA}, and how both are special cases of a more general learning problem, that of learning the principal eigenfunctions of an operator defined from a kernel and the unknown data generating density. Whereas spectral embedding methods only provided coordinates for the training points, the analysis justifies a simple extension to out-of-sample examples (the Nystr{\"{o}}m formula) for Multi-Dimensional Scaling, spectral clustering, Laplacian eigenmaps, Locally Linear Embedding ({LLE}) and Isomap. The analysis provides, for all such spectral embedding methods, the definition of a loss function, whose empirical average is minimized by the traditional algorithms. The asymptotic expected value of that loss defines a generalization performance and clarifies what these algorithms are trying to learn. Experiments with {LLE}, Isomap, spectral clustering and {MDS} show that this out-of-sample embedding formula generalizes well, with a level of error comparable to the effect of small perturbations of the training set on the embedding.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
487 topics={HighDimensional,Kernel,Unsupervised},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
488 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
489
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
490 @INPROCEEDINGS{Bengio-Gingras-nips8,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
491 author = {Bengio, Yoshua and Gingras, Fran{\c c}ois},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
492 title = {Recurrent Neural Networks for Missing or Asynchronous Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
493 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
494 pages = {395--401},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
495 crossref = {NIPS8-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
496 abstract = {In this paper we propose recurrent neural networks with feedback into the input units for handling two types of data analysis problems. On the one hand, this scheme can be used for static data when some of the input variables are missing. On the other hand, it can also be used for sequential data, when some of the input variables are missing or are available at different frequencies. Unlike in the case of probabilistic models (e.g. Gaussian) of the missing variables, the network does not attempt to model the distribution of the missing variables given the observed variables. Instead it is a more discriminant approach that fills in the missing variables for the sole purpose of minimizing a learning criterion (e.g., to minimize an output error).},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
497 topics={Finance,Missing},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
498 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
499
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
500 @ARTICLE{Bengio-Grandvalet-JMLR-04,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
501 author = {Bengio, Yoshua and Grandvalet, Yves},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
502 title = {No Unbiased Estimator of the Variance of K-Fold Cross-Validation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
503 volume = {5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
504 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
505 pages = {1089--1105},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
506 crossref = {JMLR-shorter},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
507 abstract = {Most machine learning researchers perform quantitative experiments to estimate generalization error and compare the performance of different algorithms (in particular, their proposed algorithm). In order to be able to draw statistically convincing conclusions, it is important to estimate the uncertainty of such estimates. This paper studies the very commonly used K-fold cross-validation estimator of generalization performance. The main theorem shows that there exists no universal (valid under all distributions) unbiased estimator of the variance of K-fold cross-validation. The analysis that accompanies this result is based on the eigen-decomposition of the covariance matrix of errors, which has only three different eigenvalues corresponding to three degrees of freedom of the matrix and three components of the total variance. This analysis helps to better understand the nature of the problem and how it can make naive estimators (that don’t take into account the error correlations due to the overlap between training and test sets) grossly underestimate variance. This is confirmed by numerical experiments in which the three components of the variance are compared when the difficulty of the learning problem and the number of folds are varied.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
508 topics={Comparative},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
509 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
510
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
511 @TECHREPORT{bengio-hyper-TR99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
512 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
513 title = {Continuous Optimization of Hyper-Parameters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
514 number = {1144},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
515 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
516 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
517 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/hyperTR.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
518 abstract = {Many machine learning algorithms can be formulated as the minimization of a training criterion which involves (1) “training errors” on each training example and (2) some hyper-parameters, which are kept fixed during this minimization. When there is only a single hyper-parameter one can easily explore how its value aects a model selection criterion (that is not the same as the training criterion, and is used to select hyper-parameters). In this paper we present a methodology to select many hyper-parameters that is based on the computation of the gradient of a model selection criterion with respect to the hyper-parameters. We first consider the case of a training criterion that is quadratic in the parameters. In that case, the gradient of the selection criterion with respect to the hyper-parameters is efficiently computed by back-propagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyper-parameter gradient, but this formula requires the computation of second derivatives of the training criterion},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
519 topics={ModelSelection},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
520 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
521
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
522 @INPROCEEDINGS{Bengio-icnn93,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
523 author = {Bengio, Yoshua and Frasconi, Paolo and Simard, Patrice},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
524 title = {The problem of learning long-term dependencies in recurrent networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
525 booktitle = {IEEE International Conference on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
526 year = {1993},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
527 pages = {1183--1195},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
528 publisher = {IEEE Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
529 note = {(invited paper)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
530 topics={LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
531 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
532
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
533 @ARTICLE{Bengio-ijprai93,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
534 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
535 title = {A Connectionist Approach to Speech Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
536 journal = {International Journal on Pattern Recognition and Artificial Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
537 volume = {7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
538 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
539 year = {1993},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
540 pages = {647--668},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
541 abstract = {The task discussed in this paper is that of learning to map input sequences to output sequences. In particular, problems of phoneme recognition in continuous speech are considered, but most of the discussed techniques could be applied to other tasks, such as the recognition of sequences of handwritten characters. The systems considered in this paper are based on connectionist models, or artificial neural networks, sometimes combined with statistical techniques for recognition of sequences of patterns, stressing the integration of prior knowledge and learning. Different architectures for sequence and speech recognition are reviewed, including recurrent networks as well as hybrid systems involving hidden {Markov} models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
542 topics={PriorKnowledge,Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
543 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
544
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
545 @TECHREPORT{Bengio-iohmms-TR99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
546 author = {Bengio, Yoshua and Lauzon, Vincent-Philippe and Ducharme, R{\'{e}}jean},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
547 title = {Experiments on the Application of {IOHMM}s to Model Financial Returns Series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
548 number = {1146},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
549 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
550 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
551 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/iohmms-returnsTR.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
552 abstract = {Input/Output Hidden {Markov} Models ({IOHMM}s) are conditional hidden {Markov} models in which the emission (and possibly the transition) probabilities can be conditionned on an input sequence. For example, these conditional distributions can be linear, logistic, or non-linear (using for example multi-layer neural networks). We compare the generalization performance of several models which are special cases of Input/Output Hidden {Markov} Models on financial time-series prediction tasks: an unconditional Gaussian, a conditional linear Gaussian, a mixture of Gaussians, a mixture of conditional linear Gaussians, a hidden {Markov} model, and various {IOHMM}s. The experiments are performed on modeling the returns of market and sector indices. Note that the unconditional Gaussian estimates the first moment with the historical average. The results show that, although for the first moment the historical average gives the best results, for the higher moments, the {IOHMM}s yielded significantly better performance, as measured by the out-of-sample likelihood.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
553 topics={Markov},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
554 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
555
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
556 @ARTICLE{bengio-lauzon-ducharme:2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
557 author = {Bengio, Yoshua and Lauzon, Vincent-Philippe and Ducharme, R{\'{e}}jean},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
558 title = {Experiments on the Application of {IOHMM}s to Model Financial Returns Series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
559 journal = {IEEE Transaction on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
560 volume = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
561 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
562 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
563 pages = {113--123},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
564 abstract = {Input/Output Hidden {Markov} Models ({IOHMM}s) are conditional hidden {Markov} models in which the emission (and possibly the transition) probabilities can be conditioned on an input sequence. For example, these conditional distributions can be logistic, or non-linear (using for example multi-layer neural networks). We compare generalization performance of several models which are special cases of Input/Output Hidden {Markov} Models on financial time-series prediction tasks: an unconditional Gaussian, a conditional linear Gaussian, a mixture of Gaussians, a mixture of conditional linear Gaussians, a hidden {Markov} model, and various {IOHMM}s. The experiments compare these models on predicting the conditional density of returns of market sector indices. Note that the unconditional Gaussian estimates the first moment the historical average. The results show that_ although for the first moment the historical average gives the best results, for the higher moments, the {IOHMM}s significantly better performance, as estimated by the out-of-sample likelihood.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
565 topics={Markov,Finance},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
566 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
567
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
568 @INPROCEEDINGS{bengio-lecun-94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
569 author = {Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
570 title = {Word normalization for on-line handwritten word recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
571 booktitle = {Proc. of the International Conference on Pattern Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
572 volume = {II},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
573 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
574 pages = {409--413},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
575 publisher = {IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
576 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/icpr-norm.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
577 abstract = {We introduce a new approach to normalizing words written with an electronic stylus that applies to all styles of handwriting (upper case, lower case, printed, cursive, or mixed). A geometrical model of the word spatial structure is fitted to the pen trajectory using the {EM} algorithm. The fitting process maximizes the likelihood of the trajectory given the model and a set a priors on its parameters. The method was evaluated and integrated to a recognition system that combines neural networks and hidden {Markov} models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
578 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
579 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
580
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
581 @TECHREPORT{Bengio-localfailure-TR-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
582 author = {Bengio, Yoshua and Delalleau, Olivier and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
583 title = {The Curse of Dimensionality for Local Kernel Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
584 number = {1258},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
585 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
586 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
587 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1258.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
588 abstract = {We present a series of theoretical arguments supporting the claim that a large class of modern learning algorithms based on local kernels are sensitive to the curse of dimensionality. These include local manifold learning algorithms such as Isomap and {LLE}, support vector classifiers with Gaussian or other local kernels, and graph-based semisupervised learning algorithms using a local similarity function. These algorithms are shown to be local in the sense that crucial properties of the learned function at x depend mostly on the neighbors of x in the training set. This makes them sensitive to the curse of dimensionality, well studied for classical non-parametric statistical learning. There
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
589 is a large class of data distributions for which non-local solutions could be expressed compactly and potentially be learned with few examples, but which will require a large number of local bases and therefore a large number of training examples when using a local learning algorithm.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
590 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
591 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
592
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
593 @INPROCEEDINGS{Bengio-nips-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
594 author = {Bengio, Yoshua and Lamblin, Pascal and Popovici, Dan and Larochelle, Hugo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
595 title = {Greedy Layer-Wise Training of Deep Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
596 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
597 pages = {153--160},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
598 crossref = {NIPS19-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
599 abstract = {Complexity theory of circuits strongly suggests that deep architectures can be
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
600 much more efficient (sometimes exponentially) than shallow architectures,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
601 in terms of computational elements required to represent some functions.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
602 Deep multi-layer neural networks have many levels of non-linearities
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
603 allowing them to compactly represent highly non-linear and
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
604 highly-varying functions. However, until recently it was not clear how
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
605 to train such deep networks, since gradient-based
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
606 optimization starting from random initialization appears to often get stuck
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
607 in poor solutions. Hinton et al. recently introduced
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
608 a greedy layer-wise unsupervised learning algorithm for Deep Belief
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
609 Networks (DBN), a generative model with many layers of hidden causal
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
610 variables. In the context of the above optimization problem,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
611 we study this algorithm empirically and explore variants to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
612 better understand its success and extend it to cases where the inputs are
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
613 continuous or where the structure of the input distribution is not
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
614 revealing enough about the variable to be predicted in a supervised task.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
615 Our experiments also confirm the hypothesis that the greedy
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
616 layer-wise unsupervised training strategy mostly helps the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
617 optimization, by initializing weights in a region near a
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
618 good local minimum, giving rise to internal distributed representations
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
619 that are high-level abstractions of the input, bringing better generalization.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
620 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
621
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
622 @INPROCEEDINGS{Bengio-nips10,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
623 author = {Bengio, Yoshua and Bengio, Samy and Isabelle, Jean-Fran{\c c}ois and Singer, Yoram},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
624 title = {Shared Context Probabilistic Transducers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
625 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
626 crossref = {NIPS10-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
627 abstract = {Recently, a model for supervised learning of probabilistic transducers represented by suffix trees was introduced. However, this algorithm tends to build very large trees, requiring very large amounts of computer memory. In this paper, we propose a new, more compact, transducer model in which one shares the parameters of distributions associated to contexts yielding similar conditional output distributions. We illustrate the advantages of the proposed algorithm with comparative experiments on inducing a noun phrase recognizer.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
628 topics={HighDimensional},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
629 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
630
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
631 @TECHREPORT{Bengio-NLMP-TR-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
632 author = {Bengio, Yoshua and Larochelle, Hugo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
633 title = {Non-Local Manifold Parzen Windows},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
634 number = {1264},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
635 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
636 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
637 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/NLMP-techreport.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
638 abstract = {In order to escape from the curse of dimensionality, we claim that one can learn non-local functions, in the sense that the value and shape of the learned function at x must be inferred using examples that may be far from x. With this objective, we present a non-local non-parametric density estimator. It builds upon previously proposed Gaussian mixture models with regularized covariance matrices to take into account the local shape of the manifold. It also builds upon recent work on non-local estimators of the tangent plane of a manifold, which are able to generalize in places with little training data, unlike traditional, local, non-parametric models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
639 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
640 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
641
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
642 @INPROCEEDINGS{Bengio-nncm96,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
643 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
644 editor = {Weigend, A.S. and Abu-Mostafa, Y.S. and Refenes, A. -P. N.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
645 title = {Training A Neural Network with a Financial Criterion Rather than a Prediction Criterion},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
646 booktitle = {Proceedings of the Fourth International Conference on Neural Networks in the Capital Markets ({NNCM}-96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
647 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
648 pages = {433--443},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
649 publisher = {World Scientific},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
650 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/nncm.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
651 abstract = {A common approach to quantitative decision taking with financial time-series is to train a model using a prediction criterion (e.g., squared error). We find on a portfolio selection problem that better results can be obtained when the model is directly trained in order to optimize the financial criterion of interest, with a differentiable decision module.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
652 topics={Finance,PriorKnowledge,Discriminant},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
653 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
654
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
655 @TECHREPORT{Bengio-NonStat-Hyper-TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
656 author = {Bengio, Yoshua and Dugas, Charles},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
657 title = {Learning Simple Non-Stationarities with Hyper-Parameters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
658 number = {1145},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
659 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
660 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
661 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/nonstatTR.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
662 abstract = {We consider sequential data that is sampled from an unknown process, so that the data are not necessarily i.i.d.. Most approaches to machine learning assume that data points are i.i.d.. Instead we consider a measure of generalization that does not make this assumption, and we consider in this context a recently proposed approach to optimizing hyper-parameters, based on the computation of the gradient of a model selection criterion with respect to hyper-parameters. Here we use hyper-parameters that control a function that gives different weights to different time steps in the historical data sequence. The approach is successfully applied to modeling thev olatility of stock returns one month ahead. Comparative experiments with more traditional methods are presented.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
663 topics={ModelSelection,Finance},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
664 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
665
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
666 @ARTICLE{Bengio-scholarpedia-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
667 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
668 title = {Neural net language models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
669 journal = {Scholarpedia},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
670 volume = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
671 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
672 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
673 pages = {3881},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
674 abstract = {A language model is a function, or an algorithm for learning such a function, that captures the salient statistical characteristics of the distribution of sequences of words in a natural language, typically allowing one to make probabilistic predictions of the next word given preceding ones.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
675
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
676 A neural network language model is a language model based on Neural Networks , exploiting their ability to learn distributed representations to reduce the impact of the curse of dimensionality.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
677
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
678 In the context of learning algorithms, the curse of dimensionality refers to the need for huge numbers of training examples when learning highly complex functions. When the number of input variables increases, the number of required examples can grow exponentially. The curse of dimensionality arises when a huge number of different combinations of values of the input variables must be discriminated from each other, and the learning algorithm needs at least one example per relevant combination of values. In the context of language models, the problem comes from the huge number of possible sequences of words, e.g., with a sequence of 10 words taken from a vocabulary of 100,000 there are 10^{50} possible sequences...
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
679
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
680 A distributed representation of a symbol is a tuple (or vector) of features which characterize the meaning of the symbol, and are not mutually exclusive. If a human were to choose the features of a word, he might pick grammatical features like gender or plurality, as well as semantic features like animate" or invisible. With a neural network language model, one relies on the learning algorithm to discover these features, and the features are continuous-valued (making the optimization problem involved in learning much simpler).
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
681
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
682 The basic idea is to learn to associate each word in the dictionary with a continuous-valued vector representation. Each word corresponds to a point in a feature space. One can imagine that each dimension of that space corresponds to a semantic or grammatical characteristic of words. The hope is that functionally similar words get to be closer to each other in that space, at least along some directions. A sequence of words can thus be transformed into a sequence of these learned feature vectors. The neural network learns to map that sequence of feature vectors to a prediction of interest, such as the probability distribution over the next word in the sequence. What pushes the learned word features to correspond to a form of semantic and grammatical similarity is that when two words are functionally similar, they can be replaced by one another in the same context, helping the neural network to compactly represent a function that makes good predictions on the training set, the set of word sequences used to train the model.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
683
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
684 The advantage of this distributed representation approach is that it allows the model to generalize well to sequences that are not in the set of training word sequences, but that are similar in terms of their features, i.e., their distributed representation. Because neural networks tend to map nearby inputs to nearby outputs, the predictions corresponding to word sequences with similar features are mapped to similar predictions. Because many different combinations of feature values are possible, a very large set of possible meanings can be represented compactly, allowing a model with a comparatively small number of parameters to fit a large training set.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
685 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
686
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
687 @TECHREPORT{Bengio-TR1312,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
688 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
689 title = {Learning deep architectures for AI},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
690 number = {1312},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
691 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
692 institution = {Dept. IRO, Universite de Montreal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
693 note = {Preliminary version of journal article with the same title appearing in Foundations and Trends in Machine Learning (2009)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
694 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1312.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
695 abstract = {Theoretical results strongly suggest that in order to learn the kind of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
696 complicated functions that can represent high-level abstractions (e.g. in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
697 vision, language, and other AI-level tasks), one may need deep
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
698 architectures. Deep architectures are composed of multiple levels of non-linear
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
699 operations, such as in neural nets with many hidden layers. Searching the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
700 parameter space of deep architectures is a difficult optimization task, but
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
701 learning algorithms such as those for Deep Belief Networks have recently been proposed
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
702 to tackle this problem with notable success, beating the state-of-the-art
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
703 in certain areas. This paper discusses the motivations and principles regarding
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
704 learning algorithms for deep architectures and in particular for those based
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
705 on unsupervised learning such as Deep Belief Networks, using as building
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
706 blocks single-layer models such as Restricted {Boltzmann} Machines.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
707 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
708
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
709 @ARTICLE{Bengio-trnn94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
710 author = {Bengio, Yoshua and Simard, Patrice and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
711 title = {Learning Long-Term Dependencies with Gradient Descent is Difficult},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
712 journal = {IEEE Transactions on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
713 volume = {5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
714 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
715 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
716 pages = {157--166},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
717 abstract = {Recurrent neural networks can be used to map input sequences to output sequences, such as for recognition, production or prediction problems. However, practical difficulties have been reported in training recurrent neural networks to perform tasks in which the temporal contingencies present in the input/output sequences span long intervals. We show why gradient based learning algorithms face an increasingly difficult problem as the duration of the dependencies to be captures increases. These results expose a trade-off between efficient learning by gradient descent and latching on information for long periods. Based on an understanding of this problem, alternatives to standard gradient descent are considered.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
718 optnote={(Special Issue on Recurrent Neural Networks)},topics={LongTerm},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
719 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
720
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
721 @INPROCEEDINGS{Bengio-wirn93,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
722 author = {Bengio, Yoshua and Frasconi, Paolo and Gori, Marco and Soda, G.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
723 editor = {Caianello, E.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
724 title = {Recurrent Neural Networks for Adaptive Temporal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
725 booktitle = {Proc. of the 6th Italian Workshop on Neural Networks, WIRN-93},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
726 year = {1993},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
727 pages = {1183--1195},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
728 publisher = {World Scientific Publ.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
729 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/rnn_review93.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
730 topics={LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
731 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
732
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
733 @ARTICLE{Bengio2000c,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
734 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
735 title = {Gradient-Based Optimization of Hyperparameters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
736 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
737 volume = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
738 number = {8},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
739 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
740 pages = {1889--1900},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
741 abstract = {Many machine learning algorithms can be formulated as the minimization of a training criterion which involves a hyper-parameter. This hyper-parameter is usually chosen by trial and error with a model selection criterion. In this paper we present a methodology to optimize several hyper-parameters, based on the computation of the gradient of a model selection criterion with respect to the hyper-parameters. In the case of a quadratic training criterion, the gradient of the selection criterion with respect to the hyper-parameters is efficiently computed by back-propagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyper-parameter gradient involving second derivatives of the training criterion.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
742 topics={ModelSelection},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
743 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
744
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
745 @ARTICLE{Bengio89a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
746 author = {Bengio, Yoshua and Cardin, Regis and De Mori, Renato and Merlo, Ettore},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
747 title = {Programmable execution of multi-layered networks for automatic speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
748 journal = {Communications of the Association for Computing Machinery},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
749 volume = {32},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
750 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
751 year = {1989},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
752 pages = {195--199},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
753 topics={Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
754 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
755
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
756 @INPROCEEDINGS{Bengio89c,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
757 author = {Bengio, Yoshua and Cosi, Piero and Cardin, Regis and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
758 title = {Use of multi-layered networks for coding speech with phonetic features},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
759 year = {1989},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
760 pages = {224--231},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
761 crossref = {NIPS1-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
762 abstract = {Preliminary results on speaker-independant speech recognition are reported. A method that combines expertise on neural networks with expertise on speech recognition is used to build the recognition systems. For transient sounds, event-driven property extractors with variable resolution in the time and frequency domains are used. For sonorant speech, a model of the human auditory system is preferred to FFT as a front-end module.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
763 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
764 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
765
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
766 @INPROCEEDINGS{Bengio89d,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
767 author = {De Mori, Renato and Bengio, Yoshua and Cosi, Piero},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
768 title = {On the generalization capability of multilayered networks in the extraction of speech properties},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
769 booktitle = {Proceedings of the International Joint Conference on Artificial Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
770 year = {1989},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
771 pages = {1531--1536},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
772 publisher = {IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
773 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
774 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
775
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
776 @INPROCEEDINGS{Bengio90,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
777 author = {Bengio, Yoshua and Cardin, Regis and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
778 title = {Speaker Independent Speech Recognition with Neural Networks and Speech Knowledge},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
779 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
780 pages = {218--225},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
781 crossref = {NIPS2-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
782 abstract = {We attempt to combine neural networks with knowledge from speech science to build a speaker independent speech recognition system. This knowledge is utilized in designing the preprocessing, input coding, output coding, output supervision and architectural constraints. To handle the temporal aspect of speech we combine delays, copies of activations of hidden and output units at the input level, and Back-Propagation for Sequences (BPS), a learning algorithm for networks with local self-loops. This strategy is demonstrated in several experiments, in particular a nasal discrimination task for which the application of a speech theory hypothesis dramatically improved generalization.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
783 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
784 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
785
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
786 @INCOLLECTION{Bengio90b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
787 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
788 title = {Radial Basis Functions for speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
789 booktitle = {Speech Recognition and Understanding: Recent Advances, Trends and Applications},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
790 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
791 pages = {293--298},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
792 publisher = {NATO Advanced Study Institute Series F: Computer and Systems Sciences},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
793 topics={Kernel,Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
794 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
795
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
796 @INCOLLECTION{Bengio90c,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
797 author = {Bengio, Yoshua and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
798 editor = {{Fogelman Soulie}, F. and Herault, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
799 title = {Speech coding with multilayer networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
800 booktitle = {Neurocomputing: Algorithms, Architectures and Applications},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
801 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
802 pages = {207--216},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
803 publisher = {NATO Advanced Study Institute Series F: Computer and Systems Sciences},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
804 topics={Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
805 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
806
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
807 @INPROCEEDINGS{Bengio90e,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
808 author = {Bengio, Yoshua and Pouliot, Yannick and Bengio, Samy and Agin, Patrick},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
809 title = {A neural network to detect homologies in proteins},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
810 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
811 pages = {423--430},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
812 crossref = {NIPS2-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
813 abstract = {In order to detect the presence and location of immunoglobulin (Ig) domains from amino acid sequences we built a system based on a neural network with one hidden layer trained with back propagation. The program was designed to efficiently identify proteins exhibiting such domains, characterized by a few localized conserved regions and a low overall homology. When the National Biomedical Research Foundation (NBRF) NEW protein sequence database was scanned to evaluate the program's performance, we obtained very low rates of false negatives coupled with a moderate rate of false positives.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
814 topics={Bioinformatic,PriorKnowledge},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
815 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
816
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
817 @INPROCEEDINGS{Bengio90z,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
818 author = {Bengio, Yoshua and De Mori, Renato and Gori, Marco},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
819 editor = {Caianello, E.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
820 title = {Experiments on automatic speech recognition using BPS},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
821 booktitle = {Parallel Architectures and Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
822 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
823 pages = {223--232},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
824 publisher = {World Scientific Publ.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
825 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
826 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
827
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
828 @INPROCEEDINGS{Bengio91a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
829 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
830 title = {A comparative study of hybrid acoustic phonetic decoders based on artificial neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
831 booktitle = {Proceedings of EuroSpeech'91},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
832 year = {1991},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
833 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
834 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
835
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
836 @INPROCEEDINGS{Bengio91b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
837 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
838 title = {Global Optimization of a Neural Network - Hidden {M}arkov Model Hybrid},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
839 booktitle = {Proceedings of EuroSpeech'91},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
840 year = {1991},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
841 topics={Markov},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
842 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
843
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
844 @INPROCEEDINGS{Bengio91z,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
845 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
846 title = {Phonetically motivated acoustic parameters for continuous speech recognition using artificial neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
847 booktitle = {Proceedings of EuroSpeech'91},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
848 year = {1991},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
849 cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
850 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
851
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
852 @ARTICLE{Bengio92b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
853 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
854 title = {Phonetically motivated acoustic parameters for continuous speech recognition using artificial neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
855 journal = {Speech Communication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
856 volume = {11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
857 number = {2--3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
858 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
859 pages = {261--271},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
860 note = {Special issue on neurospeech},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
861 topics={PriorKnowledge,Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
862 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
863
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
864 @INPROCEEDINGS{Bengio92c,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
865 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
866 title = {Neural Network - Gaussian Mixture Hybrid for Speech Recognition or Density Estimation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
867 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
868 pages = {175--182},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
869 crossref = {NIPS4-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
870 abstract = {The subject of this paper is the integration of multi-layered Artificial Neural Networks ({ANN}) with probability density functions such as Gaussian mixtures found in continuous density hlidden {Markov} Models ({HMM}). In the first part of this paper we present an {ANN}/HMM hybrid in which all the parameters or the the system are simultaneously optimized with respect to a single criterion. In the second part of this paper, we study the relationship between the density of the inputs of the network and the density of the outputs of the networks. A rew experiments are presented to explore how to perform density estimation with {ANN}s.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
871 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
872 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
873
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
874 @INPROCEEDINGS{Bengio94d,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
875 author = {Frasconi, Paolo and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
876 title = {An {EM} Approach to Grammatical Inference: Input/Output {HMMs}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
877 booktitle = {International Conference on Pattern Recognition (ICPR'94)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
878 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
879 pages = {289--294},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
880 topics={Markov,LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
881 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
882
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
883 @ARTICLE{Bengio96,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
884 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
885 title = {Input/{O}utput {HMM}s for Sequence Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
886 journal = {IEEE Transactions on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
887 volume = {7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
888 number = {5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
889 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
890 pages = {1231--1249},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
891 abstract = {We consider problems of sequence processing and propose a solution based on a discrete state model in order to represent past context. We introduce a recurrent connectionist architecture having a modular structure that associates a subnetwork to each state. The model has a statistical interpretation we call Input/Output Hidden {Markov} Model ({IOHMM}). It can be trained by the {EM} or {GEM} algorithms, considering state trajectories as missing data, which decouples temporal credit assignment and actual parameter estimation.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
892 The model presents similarities to hidden {Markov} models ({HMM}s), but allows us to map input sequences to output sequences, using the same processing style as recurrent neural networks. {IOHMM}s are trained using a more discriminant learning paradigm than {HMM}s, while potentially taking advantage of the {EM} algorithm.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
893 We demonstrate that {IOHMM}s are well suited for solving grammatical inference problems on a benchmark problem. Experimental results are presented for the seven Tomita grammars, showing that these adaptive models can attain excellent generalization.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
894 topics={Markov},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
895 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
896
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
897 @TECHREPORT{Bengio96-hmmsTR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
898 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
899 title = {Markovian Models for Sequential Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
900 number = {1049},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
901 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
902 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
903 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/hmmsTR.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
904 abstract = {Hidden {Markov} Models ({HMM}s) are statistical models of sequential data that have been used successfully in many applications, especially for speech recognition. We first summarize the basics of {HMM}s, and then review several recent related learning algorithms and extensions of {HMM}s, including hybrids of {HMM}s with artificial neural networks, Input-Output {HMM}s, weighted transducers, variable-length {Markov} models and {Markov} switching state-space models. Finally, we discuss some of the challenges of future research in this area.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
905 topics={Markov},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
906 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
907
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
908 @ARTICLE{Bengio97,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
909 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
910 title = {Using a Financial Training Criterion Rather than a Prediction Criterion},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
911 journal = {International Journal of Neural Systems},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
912 volume = {8},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
913 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
914 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
915 pages = {433--443},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
916 note = {Special issue on noisy time-series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
917 abstract = {The application of this work is to decision taking with financial time-series, using learning algorithms. The traditional approach is to train a model using a prediction criterion, such as minimizing the squared error between predictions and actual values of a dependent variable, or maximizing the likelihood of a conditional model of the dependent variable. We find here with noisy time-series that better results can be obtained when the model is directly trained in order to maximize the financial criterion of interest, here gains and losses (including those due to transactions) incurred during trading. Experiments were performed on portfolio selection with 35 Canadian stocks.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
918 topics={Finance,PriorKnowledge,Discriminant},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
919 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
920
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
921 @ARTICLE{Bengio99a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
922 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
923 title = {Markovian Models for Sequential Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
924 journal = {Neural Computing Surveys},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
925 volume = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
926 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
927 pages = {129--162},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
928 abstract = {Hidden {Markov} Models ({HMM}s) are statistical models of sequential data that have been used successfully in many machine learning applications, especially for speech recognition. Furthermore? in the last few years, many new and promising probabilistic models related to {HMM}s have been proposed. We first summarize the basics of {HMM}s, arid then review several recent related learning algorithms and extensions of {HMM}s, including in particular hybrids of {HMM}s with artificial neural networks, Input-Output {HMM}s (which are conditional {HMM}s using neural networks to compute probabilities), weighted transducers, variable-length {Markov} models and {Markov} switching state-space models. Finally, we discuss some of the challenges of future research in this very active area.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
929 topics={Markov},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
930 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
931
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
932 @ARTICLE{Bengio99b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
933 author = {Bengio, Samy and Bengio, Yoshua and Robert, Jacques and B{\'{e}}langer, Gilles},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
934 title = {Stochastic Learning of Strategic Equilibria for Auctions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
935 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
936 volume = {11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
937 number = {5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
938 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
939 pages = {1199--1209},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
940 abstract = {This paper presents a new application of stochastic adaptive learning algorithms to the computation of strategic equilibria in auctions. The proposed approach addresses the problems of tracking a moving target and balancing exploration (of action space) versus exploitation (of better modeled regions of action space). Neural networks are used to represent a stochastic decision model for each bidder. Experiments confirm the correctness and usefulness of the approach.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
941 topics={Auction},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
942 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
943
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
944 @TECHREPORT{bengio:1990,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
945 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
946 title = {Learning a Synaptic Learning Rule},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
947 number = {751},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
948 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
949 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
950 topics={BioRules},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
951 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
952
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
953 @INPROCEEDINGS{bengio:1990:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
954 author = {Bengio, Yoshua and R., De Mori},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
955 title = {Recurrent networks with Radial Basis Functions for speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
956 booktitle = {1990 Neural Networks for Computing Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
957 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
958 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
959 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
960
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
961 @INPROCEEDINGS{bengio:1991:ijcnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
962 author = {Bengio, Yoshua and Bengio, Samy and Cloutier, Jocelyn},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
963 title = {Learning a Synaptic Learning Rule},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
964 booktitle = {Proceedings of the International Joint Conference on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
965 year = {1991},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
966 pages = {II--A969},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
967 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1991_ijcnn.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
968 abstract = {This paper presents an original approach to neural modeling based on the idea of searching, with learning methods, for a synaptic learning rule which is biologically plausible, and yields networks that are able to learn to perform difficult tasks. The proposed method of automatically finding the learning rule relies on the idea of considering the synaptic modification rule as a parametric function. This function has local inputs and is the same in many neurons. The parameters that define this function can be estimated with known learning methods. For this optimization, we give particular attention to gradient descent and genetic algorithms. In both cases, estimation of this function consists of a joint global optimization of (a) the synaptic modification function, and (b) the networks that are learning to perform some tasks. The proposed methodology can be used as a tool to explore the missing pieces of the puzzle of neural networks learning. Both network architecture, and the learning function can be designed within constraints derived from biological knowledge.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
969 addressfr={Seattle, USA},topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
970 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
971
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
972 @INPROCEEDINGS{bengio:1991:nnc,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
973 author = {Bengio, Yoshua and Bengio, Samy and Cloutier, Jocelyn},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
974 title = {Learning Synaptic Learning Rules},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
975 booktitle = {Neural Networks for Computing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
976 year = {1991},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
977 addressfr={Snowbird, Utah, USA},topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
978 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
979
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
980 @INPROCEEDINGS{bengio:1991:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
981 author = {Bengio, Yoshua and Bengio, Samy and Cloutier, Jocelyn},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
982 title = {Learning a Synaptic Learning Rule},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
983 booktitle = {1991 Neural Networks for Computing Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
984 year = {1991},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
985 topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
986 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
987
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
988 @INPROCEEDINGS{bengio:1992:nn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
989 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn and Gecsei, Jan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
990 title = {Aspects th{\'{e}}oriques de l'optimisation d'une r{\`{e}}gle d'apprentissage},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
991 booktitle = {Actes de la conf{\'{e}}rence Neuro-N{\^{\i}}mes 1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
992 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
993 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1992_nn.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
994 abstract = {Ayant expos{\'{e}} dans de pr{\'{e}}c{\'{e}}dentes publications (voir [Beng90, Beng92] notamment) l’id{\'{e}}e que l’on pouvait optimiser des r{\`{e}}gles d’apprentissage param{\'{e}}triques pour r{\'{e}}seaux de neurones, nous montrons dans cet article comment d{\'{e}}velopper, par la m{\'{e}}thode du Lagrangien, le gradient n{\'{e}}cessaire {\`{a}} l’optimisation d’une r{\`{e}}gle d’apprentissage par descente du gradient. Nous pr{\'{e}}sentons aussi les bases th{\'{e}}oriques qui permettent d’{\'{e}}tudier la g{\'{e}}n{\'{e}}ralisation {\`{a}} de nouvelles t{\^{a}}ches d’une r{\`{e}}gle d’apprentissage dont les param{\`{e}}tres ont {\'{e}}t{\'{e}} estim{\'{e}}s {\`{a}} partir d’un certain ensemble de t{\^{a}}ches. Enfin, nous exposons bri{\`{e}}vement les r{\'{e}}sultats d’une exp{\'{e}}rience consistant {\`{a}} trouver, par descente du gradient, une r{\`{e}}gle d’apprentissage pouvant r{\'{e}}soudre plusieurs t{\^{a}}ches bool{\'{e}}ennes lin{\'{e}}airement et non lin{\'{e}}airement s{\'{e}}parables.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
995 addressfr={N{\^i}es, France},topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
996 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
997
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
998 @INPROCEEDINGS{bengio:1992:oban,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
999 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn and Gecsei, Jan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1000 title = {On the Optimization of a Synaptic Learning rule},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1001 booktitle = {Conference on Optimality in Biological and Artificial Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1002 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1003 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1992_oban.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1004 abstract = {This paper presents a new approach to neural modeling based on the idea of using an automated method to optimize the parameters of a synaptic learning rule. The synaptic modification rule is considered as a parametric function. This function has local inputs and is the same in many neurons. We can use standard optimization methods to select appropriate parameters for a given type of task. We also present a theoretical analysis permitting to study the generalization property of such parametric learning rules. By generalization, we mean the possibility for the learning rule to learn to solve new tasks. Experiments were performed on three types of problems: a biologically inspired circuit (for conditioning in Aplysia). Boolean functions (linearly separable as well as non linearly separable) and classification tasks. The neural network architecture as well as the form and initial parameter values of the synaptic learning function can be designed using a priori knowledge.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1005 addressfr={Dallas, USA},topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1006 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1007
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1008 @INPROCEEDINGS{bengio:1992:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1009 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1010 title = {Representations Based on Articulatory Dynamics for Speech Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1011 booktitle = {1992 Neural Networks for Computing Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1012 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1013 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1014 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1015
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1016 @INPROCEEDINGS{bengio:1993:icann,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1017 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn and Gecsei, Jan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1018 editor = {Gielen, S. and Kappen, B.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1019 title = {Generalization of a Parametric Learning Rule},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1020 booktitle = {{ICANN} '93: Proceedings of the International Conference on Artificial Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1021 year = {1993},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1022 pages = {502},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1023 publisher = {Springer-Verlag},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1024 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1993_icann.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1025 abstract = {In previous work ([4,2,1]) we discussed the subject of parametric learning rules for neural networks. In this article, we present a theoretical basis permitting to study the generalization property of a learning rule whose parameters are estimated from a set of learning tasks. By generalization, we mean the possibility of using the learning rule to learn solve new tasks. Finally, we describe simple experiments on two-dimensional categorization tasks and show how they corroborate the theoretical results.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1026 addressfr={Amsterdam, Pays-Bas},topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1027 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1028
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1029 @INPROCEEDINGS{bengio:1993:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1030 author = {Bengio, Yoshua and Simard, Patrice and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1031 title = {The Problem of Learning Long-Term Dependencies in Recurrent Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1032 booktitle = {1993 Neural Networks for Computing Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1033 year = {1993},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1034 topics={LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1035 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1036
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1037 @TECHREPORT{bengio:1994,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1038 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1039 title = {An {EM} Approach to Learning Sequential Behavior},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1040 number = {DSI 11-94},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1041 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1042 institution = {Universita di Firenze, Dipartimento di Sistemi e Informatica},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1043 topics={LongTerm},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1044 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1045
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1046 @INPROCEEDINGS{bengio:1994:acfas,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1047 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn and Gecsei, Jan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1048 title = {Optimisation d'une r{\`{e}}gle d'apprentissage pour r{\'{e}}seaux de neurones artificiels},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1049 booktitle = {Actes du soixante-deuxi{\`{e}}me congr{\`{e}}s de l'Association Canadienne Fran{\c c}aise pour l'Avancement des Sciences, colloque sur l'apprentissage et les r{\'{e}}seaux de neurones artificiels},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1050 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1051 topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1052 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1053
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1054 @INPROCEEDINGS{bengio:1994:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1055 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1056 title = {An {EM} Algorithm for Target Propagation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1057 booktitle = {1994 Neural Networks for Computing Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1058 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1059 topics={LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1060 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1061
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1062 @INPROCEEDINGS{bengio:1994:wcci,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1063 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1064 title = {Use of Genetic Programming for the Search of a New Learning Rule for Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1065 booktitle = {Proceedings of the First Conference on Evolutionary Computation, {IEEE} World Congress on Computational Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1066 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1067 pages = {324--327},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1068 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1994_wcci.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1069 abstract = {In previous work ([1,2,3]), we explained how to use standard optimization methods such as simulated annealing, gradient descent and genetic algorithms to optimize a parametric function which could be used as a learning rule for neural networks. To use these methods, we had to choose a fixed number of parameters and a rigid form for the learning rule. In this article, we propose to use genetic programming to find not only the values of rule parameters but also the optimal number of parameters and the form of the rule. Experiments on classification tasks suggest genetic programming finds better learning rules than other optimization methods. Furthermore, the best rule found with genetic programming outperformed the well-known backpropagation algorithm for a given set of tasks.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1070 topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1071 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1072
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1073 @INPROCEEDINGS{bengio:1994b:acfas,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1074 author = {Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1075 title = {R{\'{e}}seaux de neurones {M}arkoviens pour l'inf{\'{e}}rence grammaticale},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1076 booktitle = {Actes du soixante-deuxi{\`{e}}me congr{\`{e}}s de l'Association Canadienne Fran{\c c}aise pour l'Avancement des Sciences, colloque sur l'apprentissage et les r{\'{e}}seaux de neurones artificiels},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1077 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1078 topics={Markov,Language},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1079 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1080
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1081 @ARTICLE{bengio:1995:npl,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1082 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1083 title = {On the Search for New Learning Rules for {ANN}s},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1084 journal = {Neural Processing Letters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1085 volume = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1086 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1087 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1088 pages = {26--30},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1089 abstract = {In this paper, we present a framework where a learning rule can be optimized within a parametric learning rule space. We define what we call parametric learning rules and present a theoretical study of their generalization properties when estimated from a set of learning tasks and tested over another set of tasks. We corroborate the results of this study with practical experiments.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1090 topics={BioRules},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1091 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1092
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1093 @INCOLLECTION{bengio:1995:oban,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1094 author = {Bengio, Samy and Bengio, Yoshua and Cloutier, Jocelyn and Gecsei, Jan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1095 editor = {Levine, D. S. and Elsberry, W. R.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1096 title = {{O}n the Optimization of a Synaptic Learning Rule},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1097 booktitle = {Optimality in Biological and Artificial Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1098 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1099 publisher = {Lawrence Erlbaum},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1100 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1995_oban.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1101 abstract = {This paper presents a new approach to neural modeling based on the idea of using an automated method to optimize the parameters of a synaptic learning rule. The synaptic modification rule is considered as a parametric function. This function has local inputs and is the same in many neurons. We can use standard optimization methods to select appropriate parameters for a given type of task. We also present a theoretical analysis permitting to study the generalization property of such parametric learning rules. By generalization, we mean the possibility for the learning rule to learn to solve new tasks. Experiments were performed on three types of problems: a biologically inspired circuit (for conditioning in Aplysia), Boolean functions (linearly separable as well as non linearly separable) and classification tasks. The neural network architecture as well as the form and initial parameter values of the synaptic learning function can be designed using a priori knowledge.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1102 topics={BioRules},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1103 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1104
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1105 @TECHREPORT{bengio:1996:udem,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1106 author = {Bengio, Yoshua and Bengio, Samy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1107 title = {Training Asynchronous Input/Output Hidden {M}arkov Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1108 number = {1013},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1109 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1110 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}}de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1111 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1996_udem.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1112 topics={Markov},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1113 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1114
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1115 @INPROCEEDINGS{bengio:1997:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1116 author = {Bengio, Yoshua and Bengio, Samy and Singer, Yoram and Isabelle, Jean-Fran{\c c}ois},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1117 title = {On the Clusterization of Probabilistic Transducers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1118 booktitle = {1997 Neural Networks for Computing Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1119 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1120 topics={HighDimensional},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1121 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1122
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1123 @INPROCEEDINGS{bengio:1998:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1124 author = {Bengio, Samy and Bengio, Yoshua and Robert, Jacques and B{\'{e}}langer, Gilles},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1125 title = {Stochastic Learning of Strategic Equilibria for Auctions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1126 booktitle = {Learning Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1127 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1128 topics={Auction},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1129 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1130
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1131 @TECHREPORT{bengio:1998:udem,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1132 author = {Bengio, Samy and Bengio, Yoshua and Robert, Jacques and B{\'{e}}langer, Gilles},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1133 title = {Stochastic Learning of Strategic Equilibria for Auctions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1134 number = {1119},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1135 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1136 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}}de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1137 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bengio_1998_udem.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1138 abstract = {This paper presents a new application of stochastic adaptive learning algorithms to the computation of strategic equilibria in auctions. The proposed approach addresses the problems of tracking a moving target and balancing exploration (of action space) versus exploitation (of better modeled regions of action space). Neural networks are used to represent a stochastic decision model for each bidder. Experiments confirm the correctness and usefulness of the approach.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1139 topics={Auction},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1140 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1141
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1142 @INPROCEEDINGS{bengio:1999:snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1143 author = {Bengio, Yoshua and Latendresse, Simon and Dugas, Charles},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1144 title = {Gradient-Based Learning of Hyper-Parameters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1145 booktitle = {Learning Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1146 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1147 topics={ModelSelection},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1148 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1149
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1150 @INPROCEEDINGS{bengio:1999:titration,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1151 author = {Bengio, Yoshua and Brault, J-J. and Major, Fran{\c c}ois and Neal, R. and Pigeon, Steven},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1152 title = {Learning Algorithms for Sorting Compounds from Titration Curves},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1153 booktitle = {Symposium on New Perspectives for Computer-Aided Drug Design},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1154 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1155 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1156 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1157
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1158 @ARTICLE{bengio:2000:ieeetrnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1159 author = {Bengio, Samy and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1160 title = {Taking on the Curse of Dimensionality in Joint Distributions Using Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1161 journal = {IEEE Transaction on Neural Networks special issue on data mining and knowledge discovery},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1162 volume = {11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1163 number = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1164 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1165 pages = {550--557},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1166 abstract = {The curse of dimensionality is severe when modeling high-dimensional discrete data: the number of possible combinations of the variables explodes exponentially. In this paper we propose a new architecture for modeling high-dimensional data that requires resources (parameters and computations) that grow at most as the square of the number of variables, using a multi_layer neural network to represent the joint distribution of the variables as the product of conditional distributions. The neural network can be interpreted as a graphical model without hidden random variables, but in which the conditional distributions are tied through the hidden units. The connectivity of the neural network can be pruned by using dependency tests between the variables (thus reducing significantly the number of parameters). Experiments on modeling the distribution of several discrete data sets show statistically significant improvements over other methods such as naive Bayes and comparable Bayesian networks, and show that significant improvements can be obtained by pruning the network.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1167 topics={HighDimensional,Unsupervised,Mining},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1168 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1169
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1170 @INPROCEEDINGS{bengio:2000:nips,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1171 author = {Bengio, Yoshua and Bengio, Samy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1172 title = {Modeling High-Dimensional Discrete Data with Multi-Layer Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1173 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1174 pages = {400--406},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1175 crossref = {NIPS12-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1176 abstract = {The curse of dimensionality is severe when modeling high-dimensional discrete data: the number of possible combinations of the variables explodes exponentially. In this paper we propose a new architecture for modeling high-dimensional data that requires resources (parameters and computations) that grow only at most as the square of the number of variables, using a multi-layer neural network to represent the joint distribution of the variables as the product of conditional distributions. The neural network can be interpreted as a graphical model without hidden random variables, but in which the conditional distributions are tied through the hidden units. The connectivity of the neural network can be pruned by using dependency tests between the variables. Experiments on modeling the distribution of several discrete data sets show statistically significant improvements over other methods such as naive Bayes and comparable Bayesian networks, and show that significant improvements can be obtained by pruning the network.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1177 topics={HighDimensional,Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1178 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1179
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1180 @ARTICLE{bengio:2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1181 author = {Bengio, Yoshua and Ducharme, R{\'{e}}jean and Vincent, Pascal and Jauvin, Christian},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1182 title = {A Neural Probabilistic Language Model},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1183 volume = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1184 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1185 pages = {1137--1155},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1186 crossref = {JMLR-shorter},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1187 abstract = {A goal of statistical language modeling is to learn the joint probability function of sequences of words in a language. This is intrinsically difficult because of the curse of dimensionality: a word sequence on which the model will be tested is likely to be different from all the word sequences seen during training. Traditional but very successful approaches based on n-grams obtain generalization by concatenating very short overlapping sequences seen in the training set. We propose to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. The model learns simultaneously (1) a distributed representation for each word along with (2) the probability function for word sequences, expressed in terms of these representations. Generalization is obtained because a sequence of words that has never been seen before gets high probability if it is made of words that are similar (in the sense of having a nearby representation) to words forming an already seen sentence. Training such large models (with millions of parameters) within a reasonable time is itself a significant challenge. We report on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach significantly improves on state-of-the-art n-gram models, and that the proposed approach allows to take advantage of longer contexts.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1188 topics={Markov,Unsupervised,Language},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1189 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1190
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1191 @TECHREPORT{bengio:socs-1990,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1192 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1193 title = {Global Optimization of a Neural Network - Hidden {M}arkov Model Hybrid},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1194 number = {TR-SOCS-90.22},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1195 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1196 institution = {School of Computer Science, McGill University},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1197 topics={Markov},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1198 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1199
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1200 @INPROCEEDINGS{bengioc:1994:acfas,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1201 author = {Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1202 title = {Reconnaissance de mots manuscrits avec r{\'{e}}seaux de neurones et mod{\`{e}}les de {M}arkov},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1203 booktitle = {Actes du soixante-deuxi{\`{e}}me congr{\`{e}}s de l'Association Canadienne Fran{\c c}aise pour l'Avancement des Sciences, colloque sur l'apprentissage et les r{\'{e}}seaux de neurones artificiels},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1204 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1205 topics={Markov,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1206 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1207
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1208 @TECHREPORT{Bengio_Bottou92,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1209 author = {Bengio, Yoshua and Bottou, {L{\'{e}}on}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1210 title = {A New Approach to Estimating Probability Density Functions with Artificial Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1211 number = {TR-92.02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1212 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1213 institution = {Massachusetts Institute of Technology, Dept. Brain and Cognitive Sciences},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1214 topics={HighDimensional},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1215 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1216
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1217 @INCOLLECTION{bengio_extension_nips_2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1218 author = {Bengio, Yoshua and Paiement, Jean-Fran{\c c}ois and Vincent, Pascal and Delalleau, Olivier and Le Roux, Nicolas and Ouimet, Marie},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1219 keywords = {dimensionality reduction, eigenfunctions learning, Isomap, kernel {PCA}, locally linear embedding, Nystrom formula, spectral methods},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1220 title = {Out-of-Sample Extensions for {LLE}, Isomap, {MDS}, Eigenmaps, and Spectral Clustering},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1221 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1222 crossref = {NIPS16-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1223 abstract = {Several unsupervised learning algorithms based on an eigendecomposition provide either an embedding or a clustering only for given training points, with no straightforward extension for out-of-sample examples short of recomputing eigenvectors. This paper provides a unified framework for extending Local Linear Embedding ({LLE}), Isomap, Laplacian Eigenmaps, Multi-Dimensional Scaling (for dimensionality reduction) as well as for Spectral Clustering. This framework is based on seeing these algorithms as learning eigenfunctions of a data-dependent kernel. Numerical experiments show that the generalizations performed have a level of error comparable to the variability of the embedding algorithms due to the choice of training data.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1224 topics={HighDimensional,Kernel,Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1225 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1226
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1227 @ARTICLE{Bengio_Gingras98a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1228 author = {Bengio, Yoshua and Gingras, Fran{\c c}ois and Goulard, Bernard and Lina, Jean-Marc},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1229 title = {Gaussian Mixture Densities for Classification of Nuclear Power Plant Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1230 journal = {Computers and Artificial Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1231 volume = {17},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1232 number = {2-3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1233 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1234 pages = {189--209},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1235 abstract = {In this paper we are concerned with the application of learning algorithms to the classification of reactor states in nuclear plants. Two aspects must be considered, (1) some types of events (e.g., abnormal or rare) will not appear in the data set, but the system should be able to detect them, (2) not only classification of signals but also their interpretation are important for nuclear plant monitoring. We address both issues with a mixture of mixtures of Gaussians in which some parameters are shared to reflect the similar signals observed in different states of the reactor. An {EM} algorithm for these shared Gaussian mixtures is presented. Experimental results on nuclear plant data demonstrate the advantages of the proposed approach with respect to the above two points.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1236 topics={Mining},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1237 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1238
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1239 @ARTICLE{Bengio_Gingras98b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1240 author = {Gingras, Fran{\c c}ois and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1241 title = {Handling Asynchronous or Missing Financial Data with Recurrent Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1242 journal = {International Journal of Computational Intelligence and Organizations},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1243 volume = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1244 number = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1245 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1246 pages = {154--163},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1247 abstract = {An important issue with many sequential data analysis problems, such as those encountered in financial data sets, is that different variables are known at different frequencies, at different times (asynchronicity), or are sometimes missing. To address this issue we propose to use recurrent networks with feedback into the input units, based on two fundamental ideas. The first motivation is that the “filled-in” value of the missing variable may not only depend in complicated ways on the value of this variable in the past of the sequence but also on the current and past values of other variables. The second motivation is that, for the purpose of making predictions or taking decisions, it is not always necessary to fill in the best possible value of the missing variables. In fact, it is sufficient to fill in a value which helps the system make better predictions or decisions. The advantages of this approach are demonstrated through experiments on several tasks.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1248 topics={Finance,Missing},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1249 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1250
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1251 @INPROCEEDINGS{Bengio_icassp90,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1252 author = {Bengio, Yoshua and Cardin, Regis and De Mori, Renato and Normandin, Yves},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1253 title = {A Hybrid Coder for Hidden {M}arkov Models Using a Recurrent Neural Network},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1254 booktitle = {International Conference on Acoustics, Speech and Signal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1255 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1256 pages = {537--540},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1257 topics={Markov,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1258 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1259
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1260 @INPROCEEDINGS{Bengio_LeCun94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1261 author = {Bengio, Yoshua and {LeCun}, Yann and Henderson, Donnie},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1262 title = {Globally Trained Handwritten Word Recognizer using Spatial Representation, Space Displacement Neural Networks and Hidden {M}arkov Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1263 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1264 pages = {937--944},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1265 crossref = {NIPS6-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1266 abstract = {We introduce a new approach for on-line recognition of handwritten words written in unconstrained mixed style. The preprocessor performs a word-level normalization by fitting a model of the word structure using the {EM} algorithm. Words are then coded into low resolution “annotated images” where each pixel contains information about trajectory direction and curvature. The recognizer is a convolution network which can be spatially replicated. From the network output, a hidden {Markov} model produces word scores. The entire system is globally trained to minimize word-level errors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1267 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1268 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1269
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1270 @ARTICLE{Bengio_LeCun95,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1271 author = {Bengio, Yoshua and {LeCun}, Yann and Nohl, Craig and Burges, Chris},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1272 title = {LeRec: A {NN}/{HMM} Hybrid for On-Line Handwriting Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1273 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1274 volume = {7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1275 number = {6},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1276 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1277 pages = {1289--1303},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1278 abstract = {We introduce a new approach for on-line recognition of handwritten words written in unconstrained mixed style. The preprocessor performs a word-level normalization by fitting a model of the word structure using the {EM} algorithm. Words are then coded into low resolution “annotated images” where each pixel contains information about trajectory direction and curvature. The recognizer is a convolution network which can be spatially replicated. From the network output, a hidden {Markov} model produces word scores. The entire system is globally trained to minimize word-level errors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1279 topics={PriorKnowledge,Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1280 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1281
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1282 @ARTICLE{Bengio_prel92,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1283 author = {Bengio, Yoshua and Gori, Marco and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1284 title = {Learning the Dynamic Nature of Speech with Back-propagation for Sequences},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1285 journal = {Pattern Recognition Letters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1286 volume = {13},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1287 number = {5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1288 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1289 pages = {375--385},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1290 note = {(Special issue on Artificial Neural Networks)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1291 topics={Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1292 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1293
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1294 @ARTICLE{Bengio_trnn92,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1295 author = {Bengio, Yoshua and De Mori, Renato and Flammia, Giovanni and Kompe, Ralf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1296 title = {Global Optimization of a Neural Network-Hidden {M}arkov Model Hybrid},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1297 journal = {IEEE Transactions on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1298 volume = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1299 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1300 year = {1992},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1301 pages = {252--259},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1302 topics={Markov},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1303 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1304
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1305 @TECHREPORT{Bergstra+2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1306 author = {Bergstra, James and Desjardins, Guillaume and Lamblin, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1307 title = {Quadratic Polynomials Learn Better Image Features},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1308 number = {1337},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1309 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1310 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1311 abstract = {The affine-sigmoidal hidden unit (of the form $\sigma(ax+b)$)
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1312 is a crude predictor of neuron response in visual area V1.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1313 More descriptive models of V1 have been advanced that are no more computationally expensive,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1314 yet artificial neural network research continues to focus on networks of affine-sigmoidal models.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1315 This paper identifies two qualitative differences between the affine-sigmoidal hidden unit
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1316 and a particular recent model of V1 response:
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1317 a) the presence of a low-rank quadratic term in the argument to $\sigma$,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1318 and b) the use of a gentler non-linearity than the $\tanh$ or logistic sigmoid.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1319 We evaluate these model ingredients by training single-layer
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1320 neural networks to solve three image classification tasks.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1321 We experimented with fully-connected hidden units,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1322 as well as locally-connected units and convolutional units
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1323 that more closely mimic the function and connectivity of the visual system.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1324 On all three tasks, both the quadratic interactions and the gentler non-linearity
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1325 lead to significantly better generalization.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1326 The advantage of quadratic units was strongest in conjunction with sparse and convolutional hidden units.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1327 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1328
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1329 @ARTICLE{Bergstra+al-2010,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1330 author = {Bergstra, James and Bengio, Yoshua and Louradour, Jerome},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1331 title = {Suitability of V1 Energy Models for Object Classification},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1332 journal = {Neural Computation},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1333 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1334 note = {to appear}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1335 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1336
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1337 @INPROCEEDINGS{bergstra+al:2010-scipy,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1338 author = {Bergstra, James and Breuleux, Olivier and Bastien, Fr{\'{e}}d{\'{e}}ric and Lamblin, Pascal and Pascanu, Razvan and Desjardins, Guillaume and Turian, Joseph and Bengio, Yoshua},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1339 title = {Theano: a {CPU} and {GPU} Math Expression Compiler},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1340 booktitle = {Proceedings of the Python for Scientific Computing Conference ({SciPy})},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1341 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1342 note = {Oral}
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1343 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1344
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1345 @MISC{bergstra+al:2010-sharcnet,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1346 author = {Bergstra, James and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1347 title = {{GPU} Programming with Theano},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1348 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1349 howpublished = {{SHARCNET} Research Day},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1350 note = {Oral}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1351 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1352
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1353 @MISC{bergstra+al:2010snowbird,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1354 author = {Bergstra, James and Breuleux, Olivier and Bastien, Fr{\'{e}}d{\'{e}}ric and Lamblin, Pascal and Turian, Joseph and Desjardins, Guillaume and Pascanu, Razvan and Erhan, Dumitru and Delalleau, Olivier and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1355 title = {Deep Learning on {GPU}s with Theano},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1356 booktitle = {The Learning Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1357 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1358 note = {Oral}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1359 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1360
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1361 @INPROCEEDINGS{Bergstra+Bengio-2009,
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1362 author = {Bergstra, James and Bengio, Yoshua},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1363 title = {Slow, Decorrelated Features for Pretraining Complex Cell-like Networks},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1364 year = {2009},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1365 pages = {99--107},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1366 publisher = {MIT Press},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1367 url = {http://books.nips.cc/papers/files/nips22/NIPS2009_0933.pdf},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1368 crossref = {NIPS22}
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1369 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1370
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1371 @ARTICLE{bergstra+casagrande+erhan+eck+kegl:2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1372 author = {Bergstra, James and Casagrande, Norman and Erhan, Dumitru and Eck, Douglas and K{\'{e}}gl, Bal{\'{a}}zs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1373 title = {Aggregate Features and AdaBoost for Music Classification},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1374 journal = {Machine Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1375 volume = {65},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1376 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1377 pages = {473--484},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1378 issn = {0885-6125},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1379 abstract = {We present an algorithm that predicts musical genre and artist from an audio waveform. Our method uses the ensemble learner ADABOOST to select from a set of audio features that have been extracted from segmented audio and then aggregated. Our classifier proved to be the most effective method for genre classification at the recent MIREX 2005 international contests in music information extraction, and the second-best method for recognizing artists. This paper describes our method in detail, from feature extraction to song classification, and presents an evaluation of our method on three genre databases and two artist-recognition databases. Furthermore, we present evidence collected from a variety of popular features and classifiers that the technique of classifying features aggregated over segments of audio is better than classifying either entire songs or individual short-timescale features.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1380 PDF = {papers/2006_ml_draft.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1381 SOURCE = {OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1382 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1383
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1384 @INPROCEEDINGS{bergstra+lacoste+eck:2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1385 author = {Bergstra, James and Lacoste, Alexandre and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1386 title = {Predicting Genre Labels for Artists using FreeDB},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1387 booktitle = {Proc. 7th International Conference on Music Information Retrieval ({ISMIR})},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1388 year = {2006},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1389 pages = {85--88},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1390 publisher = {University of Victoria},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1391 SOURCE = {OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1392 PDF = {papers/2006_ismir_freedb.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1393 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1394
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1395 @INPROCEEDINGS{bergstra+mandel+eck:2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1396 author = {Bergstra, James and Mandel, Michael and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1397 title = {Scalable Genre and Tag Prediction with Spectral Covariance},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1398 booktitle = {{ISMIR}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1399 year = {2010},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1400 pages = {507--512},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1401 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1402
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1403 @MASTERSTHESIS{Bergstra-Msc-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1404 author = {Bergstra, James},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1405 keywords = {apprentissage statistique, classification de musique par genre, extraction de caract{\'{e}}ristiques sonores, recherche d'information musicale},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1406 title = {Algorithms for Classifying Recorded Music by Genre},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1407 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1408 school = {Universit{\'{e}} de Montreal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1409 abstract = {Ce m{\'{e}}moire traite le probl{\`{e}}me de la classification automatique de signaux musicaux par genre. Dans un premier temps, je pr{\'{e}}sente une technique utilisant l'apprentissage machine pour classifier des statistiques extraites sur des segments du signal sonore. Malgr{\'{e}} le fait que cette technique a d{\'{e}}j{\`{a}} {\'{e}}t{\'{e}} explor{\'{e}}e, mon m{\'{e}}moire est le premier {\`{a}} investiguer l'influence de la longueur et de la quantit{\'{e}} de ces segments sur le taux de classification. J'explore {\'{e}}galement l'importance d'avoir des segments contigus dans le temps. Les segments d'une {\`{a}} trois secondes apportent une meilleure performance, mais pour ce faire, ils doivent {\^{e}}tre suffisamment nombreux. Il peut m{\^{e}}me {\^{e}}tre utile d'augmenter la quantit{\'{e}} de segments jusqu'{\`{a}} ce qu'ils se chevauchent. Dans les m{\^{e}}mes exp{\'{e}}riences, je pr{\'{e}}sente une formulation alternative des descripteurs d'audio nomm{\'{e}}e Melfrequency Cepstral Coefficient (MFCC) qui am{\`{e}}ne un taux de classification de 81 \% sur un jeux de donn{\'{e}}es pour lequel la meilleure performance publi{\'{e}}e est de 71 \%. Cette m{\'{e}}thode de segmentation des chansons, ainsi que cette formulation alternative, ont pour but d'am{\'{e}}liorer l'algorithme gagnant du concours de classification de genre de MIREX 2005, d{\'{e}}velopp{\'{e}} par Norman Casagrande et moi. Ces exp{\'{e}}riences sont un approfondissement du travail entam{\'{e}} par Bergstra et al. [2006a], qui d{\'{e}}crit l'algorithme gagnant de ce concours.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1410 Dans un deuxi{\`{e}}me temps, je pr{\'{e}}sent une m{\'{e}}thode qui utilise FreeDB, une base de donn{\'{e}}es d'information sur les albums, pour attribuer {\`{a}} un artiste une distribution de probabilit{\'{e}} sur son genre. Avec une petite base de donn{\'{e}}es, faite {\`{a}} la main, je montre qu'il y a une haute corr{\'{e}}lation entre cette distribution et l'{\'{e}}tiquette de genre traditionnel. Bien qu'il reste {\`{a}} d{\'{e}}montrer que cette m{\'{e}}thode est utile pour organiser une collection de musique, ce r{\'{e}}sultat sugg{\`{e}}re qu'on peut maintenant {\'{e}}tiqueter de grandes bases de musique automatiquement {\`{a}} un faible co{\^{u}}t, et par cons{\'{e}}quent de poursuivre plus facilement la recherche en classification {\`{a}} grande {\'{e}}chelle. Ce travail sera publi{\'{e}} comme Bergstra et al. [2006b] {\`{a}} ISMIR 2006.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1411 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1412
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1413 @INPROCEEDINGS{bergstra:2010cosyne,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1414 author = {Bergstra, James and Bengio, Yoshua and Lamblin, Pascal and Desjardins, Guillaume and Louradour, Jerome},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1415 title = {Image classification with complex cell neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1416 booktitle = {Computational and systems neuroscience (COSYNE)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1417 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1418 note = {Poster},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1419 url = {http://www.frontiersin.org/conferences/individual_abstract_listing.php?conferid=770&pap=3626&ind_abs=1&pg=335},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1420 doi = {10.3389/conf.fnins.2010.03.00334}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1421 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1422
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1423 @INPROCEEDINGS{biaslearn:2000:ijcnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1424 author = {Ghosn, Joumana and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1425 title = {Bias Learning, Knowledge Sharing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1426 booktitle = {International Joint Conference on Neural Networks 2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1427 volume = {I},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1428 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1429 pages = {9--14},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1430 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/ijcnn_manifold.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1431 abstract = {Biasing the hypothesis space of a learner has been shown to improve generalisation performances. Methods for achieving this goal have been proposed, that range from deriving and introducing a bias into a learner to automatically learning the bias. In the latter case, most methods learn the bias by simultaneously training several related tasks derived from the same domain and imposing constraints on their parameters. We extend some of the ideas presented in this field and describe a new model that parameterizes the parameters of each task as a function of an affine manifold defined in parameter space and a point lying on the manifold. An analysis of variance on a class of learning tasks is performed that shows some significantly improved performances when using the model.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1432 topics={MultiTask},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1433 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1434
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1435 @ARTICLE{biaslearn:2003:tnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1436 author = {Ghosn, Joumana and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1437 title = {Bias Learning, Knowledge Sharing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1438 journal = {IEEE Transaction on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1439 volume = {14},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1440 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1441 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1442 pages = {748--765},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1443 abstract = {Biasing properly the hypothesis space of a learner has been shown to improve generalization performance. Methods for achieving this goal have been proposed, that range from designing and introducing a bias into a learner to automatically learning the bias. Multitask learning methods fall into the latter category. When several related tasks derived from the same domain are available, these methods use the domain-related knowledge coded in the training examples of all the tasks as a source of bias. We extend some of the ideas presented in this field and describe a new approach that identifies a family of hypotheses, represented by a manifold in hypothesis space, that embodies domain-related knowledge. This family is learned using training examples sampled from a group of related tasks. Learning models trained on these tasks are only allowed to select hypotheses that belong to the family. We show that the new approach encompasses a large variety of families which can be learned. A statistical analysis on a class of related tasks is performed that shows significantly improved performances when using this approach.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1444 topics={MultiTask},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1445 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1446
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1447 @MASTERSTHESIS{Boisvert-Mcs-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1448 author = {Boisvert, Maryse},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1449 keywords = {Algorithme {EM} , D{\'{e}}composition en valeurs singuli{\`{e}}res , D{\'{e}}sambigu{\"{\i}}sation s{\'{e}}mantique , Mod{\`{e}}les graphiques, WordNet },
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1450 title = {R{\'{e}}duction de dimension pour mod{\`{e}}les graphiques probabilistes appliqu{\'{e}}s {\`{a}} la d{\'{e}}sambiguisation s{\'{e}}mantique},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1451 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1452 school = {Universit{\'{e}} de Montr{\'{e}}al}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1453 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1454
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1455 @INPROCEEDINGS{bonneville98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1456 author = {Bonneville, Martin and Meunier, Jean and Bengio, Yoshua and Soucy, Jean-Paul},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1457 title = {Support Vector Machines for Improving the classification of Brain Pet Images},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1458 booktitle = {SPIE Medical Imaging},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1459 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1460 topics={Kernel},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1461 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1462
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1463 @INPROCEEDINGS{Bottou+Bengio95,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1464 author = {Bottou, {L{\'{e}}on} and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1465 title = {Convergence Properties of the {K}-Means Algorithm},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1466 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1467 pages = {585--592},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1468 crossref = {NIPS7-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1469 abstract = {This paper studies the convergence properties of the well known K-Means clustering algorithm. The K-Means algorithm can be described either as a gradient descent algorithm or by slightly extending the mathematics of the {EM} algorithm to this hard threshold case. We show that the K-Means algorithm actually minimizes the quantization error using the very fast Newton algorithm.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1470 topics={Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1471 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1472
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1473 @ARTICLE{bottou-98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1474 author = {Bottou, {L{\'{e}}on} and Haffner, Patrick and G. Howard, Paul and Simard, Patrice and Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1475 title = {High Quality Document Image Compression with {DjVu}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1476 journal = {Journal of Electronic Imaging},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1477 volume = {7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1478 number = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1479 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1480 pages = {410--425},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1481 topics={Compression},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1482 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1483
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1484 @INPROCEEDINGS{Bottou-dcc98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1485 author = {Bottou, {L{\'{e}}on} and G. Howard, Paul and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1486 editor = {Society, {IEEE} Computer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1487 title = {The Z-Coder Adaptive Binary Coder},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1488 booktitle = {Data Compression Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1489 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1490 url = {http://leon.bottou.org/papers/bottou-howard-bengio-98},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1491 topics={Compression},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1492 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1493
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1494 @INPROCEEDINGS{bottou-lecun-bengio-97,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1495 author = {Bottou, {L{\'{e}}on} and {LeCun}, Yann and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1496 title = {Global Training of Document Processing Systems using Graph Transformer Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1497 booktitle = {Proc. of Computer Vision and Pattern Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1498 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1499 pages = {490--494},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1500 publisher = {IEEE},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1501 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/bottou-lecun-bengio-97.pdf},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1502 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1503 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1504
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1505 @TECHREPORT{bottou96TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1506 author = {Bottou, {L{\'{e}}on} and Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1507 title = {Document analysis with transducers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1508 number = {Technical Memorandum HA615600-960701-01TM},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1509 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1510 institution = {AT\&T Labs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1511 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/transducer-tm.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1512 topics={HighDimensional},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1513 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1514
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1515 @TECHREPORT{bottou97TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1516 author = {Bottou, {L{\'{e}}on} and Bengio, Yoshua and G. Howard, Paul},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1517 title = {Z-Coder: A Fast Adaptive Binary Arithmetic Coder},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1518 number = {Technical Memorandum HA615600-970721-02TM},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1519 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1520 institution = {AT\&T Labs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1521 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/zcoder-tm.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1522 topics={Compression},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1523 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1524
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1525 @MASTERSTHESIS{Bouchard-Msc-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1526 author = {Bouchard, Lysiane},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1527 keywords = {auditory cortex, fMRI, linear classifier, logistic regression, na{\"{\i}}ve bayesian gaussian model, neuroimaging, spectro-temporal modulation, support vectors machine},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1528 title = {Analyse par apprentissage automatique des r{\'{e}}ponses fMRI du cortex auditif {\`{a}} des modulations spectro-temporelles.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1529 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1530 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1531 abstract = {The application of linear machine learning classifiers to the analysis of brain imaging data (fMRI) has led to several interesting breakthroughs in recent years. These classifiers combine the responses of the voxels to detect and categorize different brain states. They allow a more agnostic analysis than conventional fMRI analysis that systematically treats weak and distributed patterns as unwanted noise. In this project, we use such classifiers to validate an hypothesis concerning the encoding of sounds in the human brain. More precisely, we attempt to locate neurons tuned to spectral and temporal modulations in sound. We use fMRI recordings of brain responses of subjects listening to 49 different spectro-temporal modulations. The analysis of fMRI data through linear classifiers is not yet a standard procedure in this field. Thus, an important objective of this project, in the long term, is the development of new machine learning algorithms specialized for neuroimaging data. For these reasons, an important part of the experiments is dedicated to studying the behaviour of the classifiers. We are mainly interested in 3 standard linear classifiers, namely the support vectors machine algorithm (linear), the logistic regression algorithm (regularized) and the na{\"{\i}}ve bayesian gaussian model (shared variances).}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1532 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1533
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1534 @PHDTHESIS{Boufaden-Phd-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1535 author = {Boufaden, Narj{\`{e}}s},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1536 title = {Extraction d’information {\`{a}} partir de transcriptions de conversations t{\'{e}}l{\'{e}}phoniques sp{\'{e}}cialis{\'{e}}es},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1537 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1538 school = {Universit{\'{e}} de Montr{\'{e}}al, D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnel}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1539 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1540
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1541 @TECHREPORT{Breuleux+al-TR-2010,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1542 author = {Breuleux, Olivier and Bengio, Yoshua and Vincent, Pascal},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1543 title = {Unlearning for Better Mixing},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1544 number = {1349},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1545 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1546 institution = {Universit{\'{e}} de Montr{\'{e}}al/DIRO},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1547 abstract = {Two learning algorithms were recently proposed – Herding and Fast Persistent Contrastive Divergence (FPCD) – which share the following interesting characteristic: they exploit changes in the model parameters while sampling in order to escape modes and mix better, during the sampling process that is part of the learning algorithm. We first justify such approaches as ways to escape modes while approximately keeping the same asymptotic distribution of the {Markov} chain. We then extend FPCD using an idea borrowed from Herding in order to obtain a pure sampling algorithm and show empirically that this FPCD-sampler yields substantially better samples than Gibbs sampling. Because these algorithms entangle the model and the sampling algorithm and we want to evaluate both (but particularly how well the sampling schemes mix), it is not always easy to evaluate them, so we propose a “black-box” approach based on how well and how quickly the samples generated by a model “cover” the test set examples. We empirically study these algorithms and variations with this perspective and these new evaluation tools in order to better understand their strengths and limitations.}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1548 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1549
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1550 @INPROCEEDINGS{Carreau+Bengio-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1551 author = {Carreau, Julie and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1552 title = {A Hybrid {Pareto} Model for Conditional Density Estimation of Asymmetric Fat-Tail Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1553 booktitle = {Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1554 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1555 publisher = {Omnipress},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1556 abstract = {We propose an estimator for the conditional density p(Y|X) that can adapt for asymmetric heavy tails which might depend on X. Such estimators have important applications in finance and insurance. We draw from Extreme Value Theory the tools to build a hybrid unimodal density having a parameter controlling the heaviness of the upper tail. This hybrid is a Gaussian whose upper tail has been replaced by a generalized {Pareto} tail. We use this hybrid in a multi-modal mixture in order to obtain a nonparametric density estimator that can easily adapt for heavy tailed data. To obtain a conditional density estimator, the parameters of the mixture estimator can be seen as
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1557 functions of X and these functions learned. We show experimentally that this approach better models the conditional density in terms of likelihood than compared competing algorithms : conditional mixture models with other types of components and multivariate nonparametric models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1558 date={21-24}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1559 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1560
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1561 @ARTICLE{Carreau+Bengio-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1562 author = {Carreau, Julie and Bengio, Yoshua},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1563 title = {A Hybrid {Pareto} Mixture for Conditional Asymmetric Fat-Tailed Distribution},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1564 journal = {IEEE Transactions on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1565 volume = {20},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1566 number = {7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1567 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1568 pages = {1087--1101},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1569 issn = {1045-9227},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1570 abstract = {In many cases, we observe some variables X that contain predictive information over a scalar variable of interest Y, with (X,Y) pairs observed in a training set. We can take advantage of this information to estimate the conditional density P(Y\X = x). In this paper, we propose a conditional mixture model with hybrid {Pareto} components to estimate P(Y\X = x).The hybrid {Pareto} is a Gaussian whose upper tail has been replaced by a generalized {Pareto} tail. A third parameter, in addition to the location and spread parameters of the Gaussian, controls the heaviness of the upper tail. Using the hybrid {Pareto} in a mixture model results in a nonparametric estimator that can adapt to multimodality, asymmetry, and heavy tails. A conditional density estimator is built by modeling the parameters of the mixture estimator as functions of X. We use a neural network to implement these functions. Such conditional density estimators have important applications in many domains such as finance and insurance. We show experimentally that this novel approach better models the conditional density in terms of likelihood, compared to competing algorithms: conditional mixture models with other types of components and a classical kernel-based nonparametric model.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1571 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1572
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1573 @ARTICLE{Carreau+Bengio-extreme-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1574 author = {Carreau, Julie and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1575 title = {A Hybrid {Pareto} Model for Asymmetric Fat-Tailed Data: the univariate case},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1576 journal = {Extremes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1577 volume = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1578 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1579 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1580 pages = {53--76},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1581 abstract = {Density estimators that can adapt to asymmetric heavy tails are required in many applications such as finance and insurance. Extreme Value Theory (EVT) has developped principled methods based on asymptotic results to estimate the tails of most distributions. However, the finite sample approximation might introduce a severe bias in many cases. Moreover, the full range of the distribution is often needed, not only the tail area. On the other hand, non-parametric methods, while being powerful where data are abundant, fail to extrapolate properly in the tail area. We put forward a non-parametric density estimator that brings together the strengths of non-parametric density estimation and of EVT. A hybrid {Pareto} distribution that can be used in a mixture model is proposed to extend the generalized {Pareto} (GP) to the whole real axis. Experiments on simulated data show the following. On one hand, the mixture of hybrid {Pareto}s converges faster in terms of log-likelihood and provides good estimates of the tail of the distributions when compared with other density estimators including the GP distribution. On the other hand, the mixture of hybrid {Pareto}s offers an alternate way to estimate the tail index which is comparable to the one estimated with the standard GP methodology. The mixture of hybrids is also evaluated on the Danish fire insurance data set.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1582 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1583
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1584 @PHDTHESIS{Carreau-PhD-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1585 author = {Carreau, Julie},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1586 keywords = {density estimation, extreme values, generalized {Pareto} distribution, heavy-tailed distribution, mixture of distributions, neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1587 title = {Mod{\`{e}}les {Pareto} hybrides pour distributions asym{\'{e}}triques et {\`{a}} queues lourdes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1588 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1589 school = {UdeM},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1590 abstract = {We put forward a class of density estimators that can adapt to asymmetric, multi-modal and heavy-tailed distributions. Such distributions occur in many application domains such as finance and insurance. Mixture of gaussians are flexible non-parametric density estimators that have good approximation properties when the number of components is well chosen with respect to the training set size. However, those models are performing poorly on heavy-tailed data because few observations occur in the tail area. To solve this problem, we resort to extreme value theory where methods based on sound parametric assumptions have been developped to enable extrapolation beyond the range of the observations. More precisely, we build on the PoT method that was developped in hydrology where PoT stands for "Peaks-over-Threshold". The observations exceeding a given threshold are modeled by the generalized {Pareto} distribution. This distribution can approximate arbitrarily well the tail of most distributions. We build a new distribution, the hybrid {Pareto}, by stitching together a truncated Normal and a generalized {Pareto} distribution. We impose continuity constraints at the junction point. The hybrid {Pareto} is thus a smooth distribution that can be used in a mixture model. The behavior of the upper tail of the hybrid is similar to the behavior of the generalized {Pareto} tail. Moreover, the threshold inherent in the the PoT methodology can now be defined implicitly as the junction point of the component with the heaviest tail. This component also determines the tail index of the mixture. Hence, the hybrid {Pareto} mixture offers an alternate way to estimate the tail index associated with heavy-tailed data. In several applications, information that has predictive power on the variable of interest is available. In that case, we want to model the conditional density of Y given X, the vector containing predictive information. When the distribution of Y given X is asymmetric, multi-modal and heavy-tailed, we propose to use a mixure of hybrid {Pareto}s whose parameters are functions of X. Those functions are implemented by means of a neural network with one hidden layer. Neural neworks are non-parametric models that can, in principle, approximate any continuous function. Experiments on artificial and real data sets show that the hybrid {Pareto} mixture, unconditional and conditional, outperforms other density estimators in terms of log-likelihood.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1591 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1592
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1593 @INPROCEEDINGS{casagrande+eck+kegl:icmc2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1594 author = {Casagrande, Norman and Eck, Douglas and K{\'{e}}gl, Bal{\'{a}}zs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1595 title = {Geometry in Sound: A Speech/Music Audio Classifier Inspired by an Image Classifier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1596 booktitle = {{Proceedings of the International Computer Music Conference (ICMC)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1597 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1598 pages = {207--210},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1599 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2005_icmc_casagrande.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1600 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1601 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1602 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1603
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1604 @INPROCEEDINGS{casagrande+eck+kegl:ismir2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1605 author = {Casagrande, Norman and Eck, Douglas and K{\'{e}}gl, Bal{\'{a}}zs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1606 title = {Frame-Level Audio Feature Extraction using {A}da{B}oost},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1607 booktitle = {{Proceedings of the 6th International Conference on Music Information Retrieval ({ISMIR} 2005)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1608 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1609 pages = {345--350},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1610 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2005_ismir_casagrande.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1611 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1612 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1613 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1614
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1615 @PROCEEDINGS{ccai2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1616 editor = {Lamontagne, Luc and Marchand, Mario},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1617 title = {Advances in Artificial Intelligence, 19th Conference of the Canadian Society for Computational Studies of Intelligence, Canadian AI 2006, Qu{\'{e}}bec City, Qu{\'{e}}bec, Canada, June 7-9, 2006, Proceedings},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1618 booktitle = {Canadian Conference on AI},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1619 series = {Lecture Notes in Computer Science},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1620 volume = {4013},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1621 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1622 publisher = {Springer}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1623 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1624
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1625 @INPROCEEDINGS{Chapados+Bengio-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1626 author = {Chapados, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1627 title = {The K Best-Paths Approach to Approximate Dynamic Programming with Application to Portfolio Optimization},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1628 booktitle = {AI06},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1629 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1630 pages = {491-502}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1631 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1632
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1633 @INPROCEEDINGS{Chapados+Bengio-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1634 author = {Chapados, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1635 title = {Forecasting Commodity Contract Spreads with Gaussian Process},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1636 booktitle = {13th Intarnational Conference on Computing in Economics and Finance},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1637 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1638 abstract = {We introduce a functional representation of time series which allows forecasts to be performed over an unspecified horizon with progressively-revealed information sets. By virtue of using Gaussian processes, a complete covariance matrix between forecasts at several time-steps is available. This information is put to use in an application to actively trade price spreads between commodity futures contracts. The approach delivers impressive out-of-sample risk-adjusted returns after transaction costs on a portfolio of 30 spreads.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1639 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1640
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1641 @ARTICLE{Chapados+Bengio-2008-JOC,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1642 author = {Chapados, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1643 title = {Noisy K Best-Paths for Approximate Dynamic Programming with Application to Portfolio Optimization},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1644 journal = {Journal of Computers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1645 volume = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1646 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1647 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1648 pages = {12--19},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1649 abstract = {We describe a general method to transform a non-Markovian sequential decision problem into a supervised learning problem using a K-bestpaths algorithm. We consider an application in financial portfolio management where we can train a controller to directly optimize a Sharpe Ratio (or other risk-averse non-additive) utility function. We illustrate the approach by demonstrating experimental results using a kernel-based controller architecture that would not normally be considered in traditional
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1650 reinforcement learning or approximate dynamic programming.We further show that using a non-additive criterion (incremental Sharpe Ratio) yields a noisy K-best-paths extraction problem, that can give substantially improved performance.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1651 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1652
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1653 @MASTERSTHESIS{Chapados-Msc-2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1654 author = {Chapados, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1655 title = {Crit{\`{e}}res d'optimisation d'algorithmes d'apprentissage en gestion de portefeuille},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1656 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1657 school = {Universit{\'{e}} de Montr{\'{e}}al}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1658 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1659
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1660 @INPROCEEDINGS{chapados2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1661 author = {Chapados, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1662 title = {Cost Functions and Model Combination for {VaR}-Based Asset Allocation Using Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1663 booktitle = {Computational Finance 2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1664 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1665 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/compfin2000_final.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1666 abstract = {We introduce an asset-allocation framework based on the active control of the value-at-risk of the portfolio. Within this framework, we compare two paradigms for making the allocation using neural networks. The first one uses the network to make a forecast of asset behavior, in conjunction with a traditional mean-variance allocator for constructing the portfolio. The second paradigm uses the network to directly make the portfolio allocation decisions. We consider a method for performing soft input variable selection, and show its considerable utility. We use model combination (committee) methods to systematize the choice of hyperparemeters during training. We show that committees using both paradigms are significantly outperforming the benchmark market performance.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1667 topics={Finance},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1668 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1669
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1670 @ARTICLE{chapados:2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1671 author = {Chapados, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1672 title = {Cost Functions and Model Combination for VaR--based Asset Allocation using Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1673 journal = {IEEE Transactions on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1674 volume = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1675 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1676 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1677 pages = {890--906},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1678 abstract = {We introduce an asset-allocation framework based on the active control of the value-at-risk of the portfolio. Within this framework, we
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1679 compare two paradigms for making the allocation using neural networks. The first one uses the network to make a forecast of asset behavior, in conjunction with a traditional mean-variance allocator for constructing the portfolio. The second paradigm uses the network to directly make the portfolio allocation decisions. We consider a method for performing soft input variable selection, and show its considerable utility. We use model combination (committee) methods to systematize the choice of hyperparemeters during training. We show that committees
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1680 using both paradigms are significantly outperforming the benchmark market performance.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1681 topics={Finance},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1682 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1683
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1684 @ARTICLE{chapados:2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1685 author = {Bengio, Yoshua and Chapados, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1686 title = {Extensions to Metric-Based Model Selection},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1687 year = {2003},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1688 crossref = {JMLR-shorter},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1689 abstract = {Metric-based methods have recently been introduced for model selection and regularization, often yielding very significant improvements over the alternatives tried (including cross-validation). All these methods require unlabeled data over which to compare functions and detect gross differences in behavior away from the training points. We introduce three new extensions of the metric model selection methods and apply them to feature selection. The first extension takes advantage of the particular case of time-series data in which the task involves prediction with a horizon h. The idea is to use at t the h unlabeled examples that precede t for model selection. The second extension takes advantage of the different error distributions of cross-validation and the metric methods: cross-validation tends to have a larger variance and is unbiased. A hybrid combining the two model selection methods is rarely beaten by any of the two methods. The third extension deals with the case when unlabeled data is not available at all, using an estimated input density. Experiments are described to study these extensions in the context of capacity control and feature subset selection.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1690 topics={ModelSelection,Finance},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1691 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1692
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1693 @ARTICLE{chapelle:2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1694 author = {Chapelle, Olivier and Vapnik, Vladimir and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1695 title = {Model Selection for Small Sample Regression},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1696 journal = {Machine Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1697 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1698 abstract = {Model selection is an important ingredient of many machine learning algorithms, in particular when the sample size in small, in order to strike the right trade-off between overfitting and underfitting. Previous classical results for linear regression are based on an asymptotic analysis. We present a new penalization method for performing model selection for regression that is appropriate even for small samples. Our penalization is based on an accurate estimator of the ratio of the expected training error and the expected generalization error, in terms of the expected eigenvalues of the input covariance matrix.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1699 topics={ModelSelection},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1700 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1701
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1702 @INCOLLECTION{chapter-eval-longterm-2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1703 author = {Schmidhuber, Juergen and Hochreiter, Sepp and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1704 editor = {Kolen, J. and Kremer, S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1705 title = {Evaluating Benchmark Problems by Random Guessing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1706 booktitle = {Field Guide to Dynamical Recurrent Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1707 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1708 publisher = {IEEE Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1709 topics={LongTerm},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1710 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1711
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1712 @INCOLLECTION{chapter-gradient-document-2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1713 author = {{LeCun}, Yann and Bottou, {L{\'{e}}on} and Bengio, Yoshua and Haffner, Patrick},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1714 editor = {Haykin, S. and Kosko, B.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1715 title = {Gradient-Based Learning Applied to Document Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1716 booktitle = {Intelligent Signal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1717 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1718 pages = {306--351},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1719 publisher = {IEEE Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1720 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/lecun-01a.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1721 abstract = {Multilayer Neural Networks trained with a backprppagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1722 Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to monimize an overall peformance measure.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1723 Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1724 A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with a global training technique to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1725 topics={PriorKnowledge,Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1726 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1727
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1728 @INCOLLECTION{chapter-gradient-flow-2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1729 author = {Hochreiter, Sepp and Bengio, Yoshua and Frasconi, Paolo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1730 editor = {Kolen, J. and Kremer, S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1731 title = {Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1732 booktitle = {Field Guide to Dynamical Recurrent Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1733 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1734 publisher = {IEEE Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1735 topics={LongTerm},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1736 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1737
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1738 @INPROCEEDINGS{chemero+eck:1999,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1739 author = {Chemero, T. and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1740 title = {An Exploration of Representational Complexity via Coupled Oscillators},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1741 booktitle = {{Proceedings of the Tenth Midwest Artificial Intelligence and Cognitive Science Society}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1742 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1743 publisher = {MIT Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1744 url = {http://www.iro.umontreal.ca/~eckdoug/papers/1999_chemero.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1745 abstract = {We note some inconsistencies in a view of representation which takes {\it decoupling} to be of key importance. We explore these inconsistencies using examples of representational vehicles taken from coupled oscillator theory and suggest a new way to reconcile {\it coupling} with {\it absence}. Finally, we tie these views to a teleological definition of representation.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1746 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1747 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1748 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1749
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1750 @ARTICLE{ChemInfModel2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1751 author = {Erhan, Dumitru and {L'Heureux}, Pierre-Jean and Yue, Shi Yi and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1752 title = {Collaborative Filtering on a Family of Biological Targets},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1753 journal = {J. Chem. Inf. Model.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1754 volume = {46},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1755 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1756 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1757 pages = {626--635},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1758 abstract = {Building a QSAR model of a new biological target for which few screening data are available is a statistical
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1759 challenge. However, the new target may be part of a bigger family, for which we have more screening data.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1760 Collaborative filtering or, more generally, multi-task learning, is a machine learning approach that improves
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1761 the generalization performance of an algorithm by using information from related tasks as an inductive
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1762 bias. We use collaborative filtering techniques for building predictive models that link multiple targets to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1763 multiple examples. The more commonalities between the targets, the better the multi-target model that can
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1764 be built. We show an example of a multi-target neural network that can use family information to produce
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1765 a predictive model of an undersampled target. We evaluate JRank, a kernel-based method designed for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1766 collaborative filtering. We show their performance on compound prioritization for an HTS campaign and
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1767 the underlying shared representation between targets. JRank outperformed the neural network both in the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1768 single- and multi-target models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1769 topics={Bioinformatic,MultiTask},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1770 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1771
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1772 @TECHREPORT{collobert:2001:rr01-12,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1773 author = {Collobert, Ronan and Bengio, Samy and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1774 title = {A Parallel Mixture of {SVM}s for Very Large Scale Problems},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1775 number = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1776 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1777 institution = {IDIAP},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1778 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/IDIAP-RR-01-12.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1779 abstract = {Support Vector Machines ({SVM}s) are currently the state-of-the-art models for many classification problems but they suffer from the complexity of their training algorithm which is at least quadratic with respect to the number of examples. Hence, it is hopeless to try to solve real-life problems having more than a few hundreds of thousands examples with {SVM}s. The present paper proposes a new mixture of {SVM}s that can be easily implemented in parallel and where each {SVM} is trained on a small subset of the whole dataset. Experiments on a large benchmark dataset (Forest) yielded significant time improvement (time complexity appears empirically to locally grow linearly with the number of examples). In addition, and that is a surprise, a significant improvement in generalization was observed.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1780 topics={Kernel},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1781 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1782
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1783 @ARTICLE{collobert:2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1784 author = {Collobert, Ronan and Bengio, Samy and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1785 title = {Parallel Mixture of {SVM}s for Very Large Scale Problem},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1786 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1787 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1788 abstract = {Support Vector Machines ({SVM}s) are currently the state-of-the-art models for many classification problems but they suffer from the complexity of their training algorithm which is at least quadratic with respect to the number of examples. Hence, it is hopeless to try to solve real-life problems having more than a few hundreds of thousands examples with {SVM}s. The present paper proposes a new mixture of {SVM}s that can be easily implemented in parallel and where each {SVM} is trained on a small subset of the whole dataset. Experiments on a large benchmark dataset (Forest) yielded significant time improvement (time complexity appears empirically to locally grow linearly with the number of examples). In addition, and that is a surprise, a significant improvement in generalization was observed.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1789 topics={HighDimensional,Kernel},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1790 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1791
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1792 @BOOK{collobert:2002:book,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1793 author = {Collobert, Ronan and Bengio, Yoshua and Bengio, Samy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1794 editor = {Lee, S. W. and Verri, A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1795 title = {Scaling Large Learning Problems with Hard Parallel Mixtures},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1796 booktitle = {Pattern Recognition with Support Vector Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1797 series = {Lecture Notes in Computer Science},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1798 volume = {2388},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1799 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1800 publisher = {Springer-Verlag},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1801 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/2002_mixtures_svm.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1802 abstract = {A challenge for statistical learning is to deal with large data sets, e.g. in data mining. Popular learning algorithms such as Support Vector Machines have training time at least quadratic in the number of examples: they are hopeless to solve prolems with a million examples. We propose a "hard parallelizable mixture" methodology which yields significantly reduced training time through modularization and parallelization: the training data is iteratively partitioned by a "gater" model in such a way that it becoms easy to learn an "expert" model separately in each region of the parition. A probabilistic extension and the use of a set of generative models allows representing a gater so that all pieces of the model are locally trained. For {SVM}s, time complexity appears empirically to locally grow linearly with the number of examples, while generalization performance can be enhanced. For the probabilistic version of the algorithm, the iterative algorithm provably goes down in a cost function that is an upper bound on the negative log-likelihood.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1803 topics={Kernel},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1804 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1805
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1806 @MISC{copyright-CTAI,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1807 author = {Bengio, Yoshua and Ducharme, R{\'{e}}jean and Dorion, Christian},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1808 title = {Commodity Trading Advisor Index},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1809 year = {2004-2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1810 howpublished = {copyright, and commercialized software license.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1811 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1812
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1813 @MISC{copyright-PLearn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1814 author = {Vincent, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1815 title = {{PLearn}, a {C++} Machine Learning Library},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1816 year = {1998-2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1817 howpublished = {copyright, public domain license.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1818 url = {www.plearn.org}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1819 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1820
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1821 @ARTICLE{Cosi90,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1822 author = {Cosi, Piero and Bengio, Yoshua and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1823 title = {Phonetically-based multi-layered networks for acoustic property extraction and automatic speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1824 journal = {Speech Communication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1825 volume = {9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1826 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1827 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1828 pages = {15--30},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1829 topics={PriorKnowledge,Speech},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1830 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1831
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1832 @INCOLLECTION{courville+eck+bengio:nips2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1833 author = {Courville, Aaron and Eck, Douglas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1834 editor = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1835 title = {An Infinite Factor Model Hierarchy Via a Noisy-Or Mechanism},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1836 booktitle = {Neural Information Processing Systems Conference (NIPS) 22},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1837 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1838 pages = {405--413},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1839 publisher = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1840 url = {http://books.nips.cc/papers/files/nips22/NIPS2009_1100.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1841 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1842 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1843 pdf={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1844 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1845
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1846 @INPROCEEDINGS{davies+plumbley+eck:waspaa2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1847 author = {Davies, M. and Plumbley, M. and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1848 title = {Towards a musical beat emphasis function},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1849 booktitle = {Proceedings of IEEE WASPAA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1850 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1851 organization = {IEEE Workshop on Applications of Signal Processing to Audio and Acoustics},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1852 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1853 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1854 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1855
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1856 @INPROCEEDINGS{Delalleau+al-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1857 author = {Delalleau, Olivier and Bengio, Yoshua and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1858 editor = {Cowell, Robert G. and Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1859 title = {Efficient Non-Parametric Function Induction in Semi-Supervised Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1860 booktitle = {Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics (AISTATS'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1861 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1862 pages = {96--103},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1863 publisher = {Society for Artificial Intelligence and Statistics},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1864 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/semisup_aistats2005.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1865 abstract = {There has been an increase of interest for semi-supervised learning recently, because of the many datasets with large amounts of unlabeled examples and only a few labeled ones. This paper follows up on proposed nonparametric algorithms which provide an estimated continuous label for the given unlabeled examples. First, it extends them to function induction algorithms that minimize a regularization criterion applied to an out-of-sample example, and happen to have the form of Parzen windows regressors. This allows to predict test labels without solving again a linear system of dimension n (the number of unlabeled and labeled training examples), which can cost O(n^3). Second, this function induction procedure gives rise to an efficient approximation of the training process, reducing the linear system to be solved to m << n unknowns, using only a subset of m examples. An improvement of O(n^2/m^2) in time can thus be obtained. Comparative experiments are presented, showing the good performance of the induction formula and approximation algorithm.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1866 topics={Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1867 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1868
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1869 @INCOLLECTION{Delalleau+al-ssl-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1870 author = {Delalleau, Olivier and Bengio, Yoshua and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1871 editor = {Chapelle, Olivier and {Sch{\"{o}}lkopf}, Bernhard and Zien, Alexander},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1872 title = {Large-Scale Algorithms},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1873 booktitle = {Semi-Supervised Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1874 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1875 pages = {333--341},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1876 publisher = {{MIT} Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1877 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/delalleau_ssl.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1878 abstract = {In Chapter 11, it is shown how a number of graph-based semi-supervised learning
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1879 algorithms can be seen as the minimization of a specific cost function, leading to a
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1880 linear system with n equations and unknowns (with n the total number of labeled
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1881 and unlabeled examples). Solving such a linear system will in general require on the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1882 order of O(kn2) time and O(kn) memory (for a sparse graph where each data point
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1883 has k neighbors), which can be prohibitive on large datasets (especially if k = n,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1884 i.e. the graph is dense). We present in this chapter a subset selection method that
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1885 can be used to reduce the original system to one of size m << n. The idea is to solve
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1886 for the labels of a subset S of X of only m points, while still retaining information
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1887 from the rest of the data by approximating their label with a linear combination of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1888 the labels in S (using the induction formula presented in Chapter 11). This leads
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1889 to an algorithm whose computational requirements scale as O(m2n) and memory
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1890 requirements as O(m2), thus allowing one to take advantage of significantly bigger
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1891 unlabeled datasets than with the original algorithms.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1892 cat={B},topics={Unsupervised},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1893 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1894
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1895 @INCOLLECTION{DeMori90a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1896 author = {De Mori, Renato and Bengio, Yoshua and Cosi, Piero},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1897 editor = {Mohr, R. and Pavlidis, T. and Sanfelin, A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1898 title = {On the use of an ear model and multi-layer networks for automatic speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1899 booktitle = {Structural Pattern Analysis},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1900 year = {1990},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1901 publisher = {World Scientific},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1902 topics={PriorKnowledge,Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1903 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1904
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1905 @INPROCEEDINGS{Desjardins+al-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1906 author = {Desjardins, Guillaume and Courville, Aaron and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1907 title = {Tempered {Markov} Chain Monte Carlo for training of Restricted {Boltzmann} Machine},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1908 booktitle = {JMLR W\&CP: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010)},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1909 volume = {9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1910 year = {2010},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
1911 pages = {145--152},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1912 abstract = {Alternating Gibbs sampling is the most common scheme used for sampling from Restricted {Boltzmann} Machines (RBM), a crucial component in deep architectures such as Deep Belief Networks. However, we find that it often does a very poor job of rendering the diversity of modes captured by the trained model. We suspect that this hinders the advantage that could in principle be brought by training algorithms relying on Gibbs sampling for uncovering spurious modes, such as the Persistent Contrastive Divergence algorithm. To alleviate this problem, we explore the use of tempered {Markov} Chain Monte-Carlo for sampling in RBMs. We find both through visualization of samples and measures of likelihood on a toy dataset that it helps both sampling and learning.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1913 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1914
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1915 @TECHREPORT{Desjardins-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1916 author = {Desjardins, Guillaume and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1917 keywords = {Convolutional Architectures, Deep Networks, RBM, Vision},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1918 title = {Empirical Evaluation of Convolutional RBMs for Vision},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1919 number = {1327},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1920 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1921 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1922 abstract = {Convolutional Neural Networks ({CNN}) have had great success in machine learning tasks involving vision and represent one of the early successes of deep networks. Local receptive fields and weight
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1923 sharing make their architecture ideally suited for vision tasks by helping to enforce a prior based on our knowledge of natural images. This same prior could also be applied to recent developments in the field of deep networks, in order to tailor these new architectures for artificial vision. In this context, we show how the Restricted {Boltzmann} Machine (RBM), the building block of Deep Belief Networks (DBN), can be adapted to operate in a convolutional manner. We compare their performance to standard fully-connected RBMs on a simple visual learning task and show that the convolutional RBMs (CRBMs) converge to smaller values of the negative likelihood function. Our experiments also indicate that CRBMs are more efficient than standard RBMs trained on small image patches, with the CRBMs having faster convergence.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1924 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1925
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1926 @TECHREPORT{Desjardins-tech-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1927 author = {Desjardins, Guillaume and Courville, Aaron and Bengio, Yoshua and Vincent, Pascal and Delalleau, Olivier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1928 keywords = {CD, PCD, RBM, simulated tempering, tempered MCMC, unsupervised learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1929 title = {Tempered {Markov} Chain Monte Carlo for training of Restricted {Boltzmann} Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1930 number = {1345},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1931 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1932 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1933 abstract = {Alternating Gibbs sampling is the most common scheme used for sampling from Restricted {Boltzmann} Machines (RBM), a crucial component in deep architectures such as Deep Belief Networks. However, we find that it often does a very poor job of rendering the diversity of modes captured by the trained model. We suspect that this hinders the advantage that could in principle be brought by training algorithms relying on Gibbs sampling for uncovering spurious modes, such as the Persistent Contrastive Divergence algorithm. To alleviate this problem, we
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1934 explore the use of tempered {Markov} Chain Monte-Carlo for sampling in RBMs. We find both through visualization of samples and measures of likelihood that it helps both sampling and learning.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1935 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1936
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1937 @ARTICLE{Dugas+Bengio-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1938 author = {Dugas, Charles and Bengio, Yoshua and Belisle, Francois and Nadeau, Claude and Garcia, Rene},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1939 title = {Incorporating Functional Knowledge in Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1940 journal = {The Journal of Machine Learning Research},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1941 volume = {10},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1942 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1943 pages = {1239--1262},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1944 abstract = {Incorporating prior knowledge of a particular task into the architecture of a learning algorithm can greatly improve generalization performance. We study here a case where we know that the function to be learned is non-decreasing in its two arguments and convex in one of them. For this purpose we propose a class of functions similar to multi-layer neural networks but (1) that has those properties, (2) is a universal approximator of Lipschitz functions with these and other properties. We apply this new class of functions to the task of modelling the price of call options. Experiments show improvements on regressing the price of call options using the new types of function classes that incorporate the a priori constraints.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1945 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1946
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1947 @PHDTHESIS{Dugas-Phd-2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1948 author = {Dugas, Charles},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1949 title = {Les algorithmes d'apprentissage appliqu{\'{e}}s aux risques financiers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1950 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1951 school = {Universit{\'{e}} de Montr{\'{e}}al}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1952 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1953
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1954 @ARTICLE{dugas:2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1955 author = {Dugas, Charles and Bengio, Yoshua and Chapados, Nicolas and Vincent, Pascal and Denoncourt, Germain and Fournier, Christian},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1956 title = {Statistical Learning Algorithms Applied to Automobile Insurance Ratemaking},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1957 journal = {CAS Forum},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1958 volume = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1959 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1960 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1961 pages = {179--214},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1962 abstract = {We recently conducted a research project for a large North American automobile insurer. This study was the most exhaustive ever undertaken by this particular insurer and lasted over an entire year. We analyzed the discriminating power of each variable used for ratemaking. We analyzed the performance of several models within five broad categories: linear regressions, generalized linear models, decision trees, neural networks and support vector machines. In this paper, we present the main results of this study. We qualitatively compare models and show how neural networks can represent high-order nonlinear dependencies with a small number of parameters, each of which is estimated on a large proportion of the data, thus yielding low variance. We thoroughly explain the purpose of the nonlinear sigmoidal transforms which are at the very heart of neural networks' performances. The main numerical result is a statistically significant reduction in the out-of-sample mean-squared error using the neural network model and our ability to substantially reduce the median premium by charging more to the highest risks. This in turn can translate into substantial savings and financial benefits for an insurer. We hope this paper goes a long way towards convincing actuaries to include neural networks within their set of modeling tools for ratemaking.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1963 topics={Finance,Mining},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1964 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1965
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1966 @INPROCEEDINGS{eck+bertinmahieux+lamere+green:nips2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1967 author = {Eck, Douglas and Lamere, Paul and Bertin-Mahieux, Thierry and Green, Stephen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1968 editor = {Platt, John and Kolen, J. and Singer, Yoram and Roweis, S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1969 title = {Automatic Generation of Social Tags for Music Recommendation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1970 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1971 crossref = {NIPS20-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1972 source = "OwnPublication"
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1973 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1974
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1975 @INPROCEEDINGS{eck+bertinmahieux+lamere:ismir2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1976 author = {Eck, Douglas and Bertin-Mahieux, Thierry and Lamere, Paul},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1977 title = {Autotagging music using supervised machine learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1978 booktitle = {{Proceedings of the 8th International Conference on Music Information Retrieval ({ISMIR} 2007)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1979 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1980 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1981 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1982
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1983 @INPROCEEDINGS{eck+casagrande:ismir2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1984 author = {Eck, Douglas and Casagrande, Norman},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1985 title = {Finding Meter in Music Using an Autocorrelation Phase Matrix and Shannon Entropy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1986 booktitle = {{Proceedings of the 6th International Conference on Music Information Retrieval ({ISMIR} 2005)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1987 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1988 pages = {504--509},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1989 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2005_ismir.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1990 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1991 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1992 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1993
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1994 @INCOLLECTION{eck+gasser+port:2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1995 author = {Eck, Douglas and Gasser, M. and Port, Robert},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1996 editor = {Desain, P. and Windsor, L.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1997 title = {Dynamics and Embodiment in Beat Induction},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1998 booktitle = {{Rhythm Perception and Production}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1999 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2000 pages = {157--170},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2001 publisher = {Swets and Zeitlinger},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2002 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2000_rppw.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2003 abstract = {We provide an argument for using dynamical systems theory in the domain of beat induction. We motivate the study of beat induction and to relate beat induction to the more general study of human rhythm cognition. In doing so we compare a dynamical, embodied approach to a symbolic (traditional AI) one, paying particular attention to how the modeling approach brings with it tacit assumptions about what is being modeled. Please note that this is a philosophy paper about research that was, at the time of writing, very much in progress.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2004 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2005 sourcetype={Chapter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2006 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2007
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2008 @INPROCEEDINGS{eck+gasser:1996,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2009 author = {Eck, Douglas and Gasser, M.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2010 editor = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2011 title = {Perception of Simple Rhythmic Patterns in a Network of Oscillators},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2012 booktitle = {{The Proceedings of the Eighteenth Annual Conference of the Cognitive Science Society}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2013 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2014 publisher = {Lawrence Erlbaum Associates},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2015 abstract = {This paper is concerned with the complex capacity to recognize and reproduce rhythmic patterns. While this capacity has not been well investigated, in broad qualitative terms it is clear that people can learn to identify and produce recurring patterns defined in terms of sequences of beats of varying intensity and rests: the rhythms behind waltzes, reels, sambas, etc. Our short term goal is a model which is "hard-wired" with knowledge of a set of such patterns. Presented with a portion of one of the patterns or a label for a pattern, the model should reproduce the pattern and continue to do so when the input is turned off. Our long-term goal is a model which can learn to adjust the connection strengths which implement particular patterns as it is exposed to input patterns.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2016 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2017 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2018 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2019
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2020 @TECHREPORT{eck+graves+schmidhuber:tr-speech2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2021 author = {Eck, Douglas and Graves, A. and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2022 title = {A New Approach to Continuous Speech Recognition Using {LSTM} Recurrent Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2023 number = {IDSIA-14-03},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2024 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2025 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2026 abstract = {This paper presents an algorithm for continuous speech recognition built from two Long Short-Term Memory ({LSTM}) recurrent neural networks. A first {LSTM} network performs frame-level phone probability estimation. A second network maps these phone predictions onto words. In contrast to {HMM}s, this allows greater exploitation of long-timescale correlations. Simulation results are presented for a hand-segmented subset of the "Numbers-95" database. These results include isolated phone prediction, continuous frame-level phone prediction and continuous word prediction. We conclude that despite its early stage of development, our new model is already competitive with existing approaches on certain aspects of speech recognition and promising on others, warranting further research.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2027 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2028 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2029 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2030
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2031 @TECHREPORT{eck+lapalme:2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2032 author = {Eck, Douglas and Lapalme, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2033 title = {Learning Musical Structure Directly from Sequences of Music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2034 number = {1300},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2035 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2036 institution = {Universit{\'{e}} de Montr{\'{e}}al DIRO},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2037 url = {http://www.iro.umontreal.ca/~eckdoug/papers/tr1300.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2038 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2039 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2040 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2041
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2042 @INPROCEEDINGS{eck+schmidhuber:icann2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2043 author = {Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2044 editor = {Dorronsoro, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2045 title = {Learning The Long-Term Structure of the Blues},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2046 booktitle = {{Artificial Neural Networks -- ICANN 2002 (Proceedings)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2047 volume = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2048 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2049 pages = {284--289},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2050 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2051 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2002_icannMusic.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2052 abstract = {In general music composed by recurrent neural networks ({RNN}s) suffers from a lack of global structure. Though networks can learn note-by-note transition probabilities and even reproduce phrases, they have been unable to learn an entire musical form and use that knowledge to guide composition. In this study, we describe model details and present experimental results showing that {LSTM} successfully learns a form of blues music and is able to compose novel (and some listeners believe pleasing) melodies in that style. Remarkably, once the network has found the relevant structure it does not drift from it: {LSTM} is able to play the blues with good timing and proper structure as long as one is willing to listen.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2053 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2054 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2055 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2056
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2057 @INPROCEEDINGS{eck+schmidhuber:ieee2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2058 author = {Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2059 editor = {Bourlard, H.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2060 title = {Finding Temporal Structure in Music: Blues Improvisation with {LSTM} Recurrent Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2061 booktitle = {Neural Networks for Signal Processing XII, Proceedings of the 2002 IEEE Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2062 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2063 pages = {747--756},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2064 publisher = {IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2065 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2002_ieee.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2066 abstract = {Few types of signal streams are as ubiquitous as music. Here we consider the problem of extracting essential ingredients of music signals, such as well-defined global temporal structure in the form of nested periodicities (or {\em meter}). Can we construct an adaptive signal processing device that learns by example how to generate new instances of a given musical style? Because recurrent neural networks can in principle learn the temporal structure of a signal, they are good candidates for such a task. Unfortunately, music composed by standard recurrent neural networks ({RNN}s) often lacks global coherence. The reason for this failure seems to be that {RNN}s cannot keep track of temporally distant events that indicate global music structure. Long Short-Term Memory ({LSTM}) has succeeded in similar domains where other {RNN}s have failed, such as timing \& counting and learning of context sensitive languages. In the current study we show that {LSTM} is also a good mechanism for learning to compose music. We present experimental results showing that {LSTM} successfully learns a form of blues music and is able to compose novel (and we believe pleasing) melodies in that style. Remarkably, once the network has found the relevant structure it does not drift from it: {LSTM} is able to play the blues with good timing and proper structure as long as one is willing to listen.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2067 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2068 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2069 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2070
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2071 @ARTICLE{eck+scott:2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2072 author = {Eck, Douglas and Scott, S. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2073 title = {Editorial: New Research in Rhythm Perception and Production},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2074 journal = {Music Perception},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2075 volume = {22},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2076 number = {3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2077 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2078 pages = {371-388},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2079 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2080 sourcetype={Other},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2081 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2082
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2083 @MISC{eck+scott:editor2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2084 author = {Eck, Douglas and Scott, S. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2085 title = {Music Perception},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2086 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2087 note = {Guest Editor, Special Issue on Rhythm Perception and Production, 22(3)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2088 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2089 sourcetype={Other},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2090 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2091
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2092 @INPROCEEDINGS{eck:1999,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2093 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2094 editor = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2095 title = {Learning Simple Metrical Preferences in a Network of {F}itzhugh-{N}agumo Oscillators},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2096 booktitle = {{The Proceedings of the Twenty-First Annual Conference of the Cognitive Science Society}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2097 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2098 publisher = {Lawrence Erlbaum Associates},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2099 abstract = {Hebbian learning is used to train a network of oscillators to prefer periodic signals of pulses over aperiodic signals. Target signals consisted of metronome-like voltage pulses with varying amounts of inter-onset noise injected. (with 0\% noise yielding a periodic signal and more noise yielding more and more aperiodic signals.) The oscillators---piecewise-linear approximations (Abbott, 1990) to Fitzhugh-Nagumo oscillators---are trained using mean phase coherence as an objective function. Before training a network is shown to readily synchronize with signals having wide range of noise. After training on a series of noise-free signals, a network is shown to only synchronize with signals having little or no noise. This represents a bias towards periodicity and is explained by strong positive coupling connections between oscillators having harmonically-related periods.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2100 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2101 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2102 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2103
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2104 @UNPUBLISHED{eck:bramsworkshop2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2105 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2106 title = {Challenges for Machine Learning in the Domain of Music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2107 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2108 note = {BRAMS Workshop on Brain and Music, Montreal Neurological Institute},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2109 abstract = {Slides and musical examples available on request.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2110 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2111 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2112 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2113 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2114 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2115 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2116
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2117 @PHDTHESIS{eck:diss,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2118 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2119 title = {{Meter Through Synchrony: Processing Rhythmical Patterns with Relaxation Oscillators}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2120 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2121 school = {Indiana University, Bloomington, IN, www.idsia.ch/\-\~{}doug/\-publications.html},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2122 abstract = {This dissertation uses a network of relaxation oscillators to beat along with temporal signals. Relaxation oscillators exhibit interspersed slow-fast movement and model a wide array of biological oscillations. The model is built up gradually: first a single relaxation oscillator is exposed to rhythms and shown to be good at finding downbeats in them. Then large networks of oscillators are mutually coupled in an exploration of their internal synchronization behavior. It is demonstrated that appropriate weights on coupling connections cause a network to form multiple pools of oscillators having stable phase relationships. This is a promising first step towards networks that can recreate a rhythmical pattern from memory. In the full model, a coupled network of relaxation oscillators is exposed to rhythmical patterns. It is shown that the network finds downbeats in patterns while continuing to exhibit good internal stability. A novel non-dynamical model of downbeat induction called the Normalized Positive (NP) clock model is proposed, analyzed, and used to generate comparison predictions for the oscillator model. The oscillator model compares favorably to other dynamical approaches to beat induction such as adaptive oscillators. However, the relaxation oscillator model takes advantage of intrinsic synchronization stability to allow the creation of large coupled networks. This research lays the groundwork for a long-term research goal, a robotic arm that responds to rhythmical signals by tapping along. It also opens the door to future work in connectionist learning of long rhythmical patterns.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2123 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2124 sourcetype={Thesis},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2125 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2126
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2127 @INPROCEEDINGS{eck:icann2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2128 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2129 editor = {Dorffner, Georg},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2130 title = {A Network of Relaxation Oscillators that Finds Downbeats in Rhythms},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2131 booktitle = {{Artificial Neural Networks -- ICANN 2001 (Proceedings)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2132 volume = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2133 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2134 pages = {1239--1247},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2135 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2136 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2001_icann.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2137 abstract = {A network of relaxation oscillators is used to find downbeats in rhythmical patterns. In this study, a novel model is described in detail. Its behavior is tested by exposing it to patterns having various levels of rhythmic complexity. We analyze the performance of the model and relate its success to previous work dealing with fast synchrony in coupled oscillators.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2138 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2139 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2140 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2141
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2142 @INPROCEEDINGS{eck:icassp2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2143 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2144 editor = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2145 title = {Beat Tracking Using an Autocorrelation Phase Matrix},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2146 booktitle = {{Proceedings of the 2007 International Conference on Acoustics, Speech and Signal Processing (ICASSP)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2147 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2148 pages = {1313--1316},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2149 publisher = {IEEE Signal Processing Society},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2150 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2007_icassp.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2151 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2152 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2153 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2154
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2155 @INPROCEEDINGS{eck:icmpc2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2156 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2157 editor = {Lipscomb, S. D. and Ashley, R. and Gjerdingen, R. O. and Webster, P.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2158 title = {A Machine-Learning Approach to Musical Sequence Induction That Uses Autocorrelation to Bridge Long Timelags},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2159 booktitle = {{The Proceedings of the Eighth International Conference on Music Perception and Cognition ({ICMPC}8)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2160 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2161 pages = {542-543},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2162 publisher = {Causal Productions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2163 abstract = {One major challenge in using statistical sequence learning methods in the domain of music lies in bridging the long timelags that separate important musical events. Consider, for example, the chord changes that convey the basic structure of a pop song. A sequence learner that cannot predict chord changes will almost certainly not be able to generate new examples in a musical style or to categorize songs by style. Yet, it is surprisingly difficult for a sequence learner to bridge the long timelags necessary to identify when a chord change will occur and what its new value will be. This is the case because chord changes can be separated by dozens or hundreds of intervening notes. One could solve this problem by treating chords as being special (as did Mozer, NIPS 1991). But this is impractical---it requires chords to be labeled specially in the dataset, limiting the applicability of the model to non-labeled examples---and furthermore does not address the general issue of nested temporal structure in music. I will briefly describe this temporal structure (known commonly as "meter") and present a model that uses to its advantage an assumption that sequences are metrical. The model consists of an autocorrelation-based filtration that estimates online the most likely metrical tree (i.e. the frequency and phase of beat, measure, phrase &etc.) and uses that to generate a series of sequences varying at different rates. These sequences correspond to each level in the hierarchy. Multiple learners can be used to treat each series separately and their predictions can be combined to perform composition and categorization. I will present preliminary results that demonstrate the usefulness of this approach. Time permitting I will also compare the model to alternate approaches.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2164 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2165 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2166 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2167
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2168 @INPROCEEDINGS{eck:icmpc2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2169 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2170 editor = {Baroni, M. and Addessi, A. R. and Caterina, R. and Costa, M.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2171 title = {Beat Induction Using an Autocorrelation Phase Matrix},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2172 booktitle = {The Proceedings of the 9th International Conference on Music Perception and Cognition ({ICMPC9})},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2173 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2174 pages = {931-932},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2175 publisher = {Causal Productions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2176 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2177 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2178 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2179
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2180 @UNPUBLISHED{eck:irisworkshop2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2181 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2182 title = {Using Autocorrelation to Bridge Long Timelags when Learning Sequences of Music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2183 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2184 note = {IRIS 2004 Machine Learning Workshop, Ottawa, Canada},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2185 abstract = {Slides and musical examples available on request.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2186 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2187 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2188 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2189 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2190 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2191 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2192
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2193 @ARTICLE{eck:jnmr2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2194 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2195 title = {A Positive-Evidence Model for Rhythmical Beat Induction},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2196 journal = {Journal of New Music Research},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2197 volume = {30},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2198 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2199 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2200 pages = {187--200},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2201 abstract = {The Normalized Positive (NPOS) model is a rule-based model that predicts downbeat location and pattern complexity in rhythmical patterns. Though derived from several existing models, the NPOS model is particularly effective at making correct predictions while at the same time having low complexity. In this paper, the details of the model are explored and a comparison is made to existing models. Several datasets are used to examine the complexity predictions of the model. Special attention is paid to the model's ability to account for the effects of musical experience on beat induction.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2202 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2203 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2204 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2205
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2206 @UNPUBLISHED{eck:mipsworkshop2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2207 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2208 title = {Bridging Long Timelags in Music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2209 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2210 note = {NIPS 2004 Workshop on Music and Machine Learning (MIPS), Whistler, British Columbia},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2211 abstract = {Slides and musical examples available on request.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2212 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2213 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2214 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2215 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2216 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2217 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2218
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2219 @ARTICLE{eck:mp2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2220 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2221 title = {Finding Long-Timescale Musical Structure with an Autocorrelation Phase Matrix},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2222 journal = {Music Perception},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2223 volume = {24},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2224 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2225 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2226 pages = {167--176},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2227 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2228 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2229 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2230
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2231 @UNPUBLISHED{eck:nipsworkshop2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2232 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2233 title = {Time-warped hierarchical structure in music and speech: A sequence prediction challenge},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2234 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2235 note = {NIPS 2003 Workshop on Recurrent Neural Networks, Whistler, British Columbia},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2236 abstract = {Slides and musical examples available on request.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2237 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2238 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2239 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2240 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2241 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2242 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2243
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2244 @UNPUBLISHED{eck:nipsworkshop2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2245 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2246 title = {Generating music sequences with an echo state network},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2247 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2248 note = {NIPS 2006 Workshop on Echo State Networks and Liquid State Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2249 abstract = {Slides and musical examples available on request.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2250 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2251 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2252 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2253 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2254 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2255 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2256
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2257 @UNPUBLISHED{eck:nipsworkshop2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2258 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2259 title = {Measuring and modeling musical expression},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2260 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2261 note = {NIPS 2007 Workshop on Music, Brain and Cognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2262 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2263 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2264 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2265 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2266 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2267 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2268
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2269 @ARTICLE{eck:psyres2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2270 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2271 title = {Finding Downbeats with a Relaxation Oscillator},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2272 journal = {Psychol. Research},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2273 volume = {66},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2274 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2275 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2276 pages = {18--25},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2277 abstract = {A relaxation oscillator model of neural spiking dynamics is applied to the task of finding downbeats in rhythmical patterns. The importance of downbeat discovery or {\em beat induction} is discussed, and the relaxation oscillator model is compared to other oscillator models. In a set of computer simulations the model is tested on 35 rhythmical patterns from Povel \& Essens (1985). The model performs well, making good predictions in 34 of 35 cases. In an analysis we identify some shortcomings of the model and relate model behavior to dynamical properties of relaxation oscillators.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2278 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2279 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2280 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2281
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2282 @UNPUBLISHED{eck:rppw2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2283 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2284 title = {Meter and Autocorrelation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2285 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2286 note = {{10th Rhythm Perception and Production Workshop (RPPW), Alden Biesen, Belgium}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2287 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2288 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2289 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2290
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2291 @TECHREPORT{eck:tr-music2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2292 author = {Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2293 title = {A First Look at Music Composition using {LSTM} Recurrent Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2294 number = {IDSIA-07-02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2295 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2296 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2297 abstract = {In general music composed by recurrent neural networks ({RNN}s) suffers from a lack of global structure. Though networks can learn note-by-note transition probabilities and even reproduce phrases, attempts at learning an entire musical form and using that knowledge to guide composition have been unsuccessful. The reason for this failure seems to be that {RNN}s cannot keep track of temporally distant events that indicate global music structure. Long Short-Term Memory ({LSTM}) has succeeded in similar domains where other {RNN}s have failed, such as timing \& counting and CSL learning. In the current study I show that {LSTM} is also a good mechanism for learning to compose music. I compare this approach to previous attempts, with particular focus on issues of data representation. I present experimental results showing that {LSTM} successfully learns a form of blues music and is able to compose novel (and I believe pleasing) melodies in that style. Remarkably, once the network has found the relevant structure it does not drift from it: {LSTM} is able to play the blues with good timing and proper structure as long as one is willing to listen. {\em Note: This is a more complete version of the 2002 ICANN submission Learning the Long-Term Structure of the Blues.}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2298 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2299 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2300 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2301
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2302 @TECHREPORT{eck:tr-npos2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2303 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2304 title = {A Positive-Evidence Model for Classifying Rhythmical Patterns},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2305 number = {IDSIA-09-00},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2306 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2307 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2308 abstract = {The Normalized Positive (NPOS) model is a novel matching model that predicts downbeat location and pattern complexity in rhythmical patterns. Though similar models report success, the NPOS model is particularly effective at making these predictions while at the same time being theoretically and mathematically simple. In this paper, the details of the model are explored and a comparison is made to existing models. Several datasets are used to examine the complexity predictions of the model. Special attention is paid to the model's ability to account for the effects of musical experience on rhythm perception.\\ {\em Note: See the 2001 Journal of New Music Research paper "A Positive-Evidence Model for Rhythmical Beat Induction" for a newer version of this paper.}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2309 ps={ftp://ftp.idsia.ch/pub/techrep/IDSIA-09-00.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2310 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2311 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2312 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2313
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2314 @TECHREPORT{eck:tr-oscnet2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2315 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2316 title = {A Network of Relaxation Oscillators that Finds Downbeats in Rhythms},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2317 number = {IDSIA-06-01},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2318 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2319 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2320 abstract = {A network of relaxation oscillators is used to find downbeats in rhythmical patterns. In this study, a novel model is described in detail. Its behavior is tested by exposing it to patterns having various levels of rhythmic complexity. We analyze the performance of the model and relate its success to previous work dealing with fast synchrony in coupled oscillators. \\ {\em Note: See the 2001 ICANN conference proceeding by the same title for a newer version of this paper.}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2321 ps={ftp://ftp.idsia.ch/pub/techrep/IDSIA-06-01.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2322 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2323 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2324 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2325
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2326 @TECHREPORT{eck:tr-tracking2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2327 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2328 title = {Tracking Rhythms with a Relaxation Oscillator},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2329 number = {IDSIA-10-00},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2330 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2331 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2332 abstract = {A number of biological and mechanical processes are typified by a continued slow accrual and fast release of energy. A nonlinear oscillator exhibiting this slow-fast behavior is called a relaxation oscillator and is used to model, for example, human heartbeat pacemaking and neural action potential. Similar limit cycle oscillators are used to model a wider range of behaviors including predator-prey relationships and synchrony in animal populations such as fireflies. Though nonlinear limit-cycle oscillators have been successfully applied to beat induction, relaxation oscillators have received less attention. In this work we offer a novel and effective relaxation oscillator model of beat induction. We outline the model in detail and provide a perturbation analysis of its response to external stimuli. In a series of simulations we expose the model to patterns from Experiment 1 of Povel \& Essens (1985). We then examine the beat assignments of the model. Although the overall performance of the model is very good, there are shortcomings. We believe that a network of mutually-coupled oscillators will address many of these shortcomings, and we suggest an appropriate course for future research.\\ {\em Note: See the 2001 {\em Psychological Research} article "Finding Downbeats with a Relaxation Oscillator" for a revised but less detailed version of this paper.}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2333 ps={ftp://ftp.idsia.ch/pub/techrep/IDSIA-10-00.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2334 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2335 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2336 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2337
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2338 @TECHREPORT{eck:tr-tracking2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2339 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2340 title = {Real-Time Musical Beat Induction with Spiking Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2341 number = {IDSIA-22-02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2342 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2343 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2344 abstract = {Beat induction is best described by analogy to the activities of hand clapping or foot tapping, and involves finding important metrical components in an auditory signal, usually music. Though beat induction is intuitively easy to understand it is difficult to define and still more difficult to perform automatically. We will present a model of beat induction that uses a spiking neural network as the underlying synchronization mechanism. This approach has some advantages over existing methods; it runs online, responds at many levels in the metrical hierarchy, and produces good results on performed music (Beatles piano performances encoded as MIDI). In this paper the model is described in some detail and simulation results are discussed.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2345 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2346 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2347 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2348
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2349 @UNPUBLISHED{eck:verita2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2350 author = {Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2351 title = {Real Time Beat Induction with Spiking Neurons},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2352 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2353 note = {{Music, Motor Control and the Mind: Symposium at Monte Verita, May}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2354 abstract = {Beat induction is best described by analogy to the activites of hand clapping or foot tapping, and involves finding important metrical components in an auditory signal, usually music. Though beat induction is intuitively easy to understand it is difficult to define and still more difficult to model. I will discuss an approach to beat induction that uses a network of spiking neurons to synchronize with periodic components in a signal at many timescales. Through a competitive process, groups of oscillators embodying a particular metrical interpretation (e.g. \"4/4\") are selected from the network and used to track the pattern. I will compare this model to other approaches including a traditional symbolic AI system (Dixon 2001), and one based on Bayesian statistics (Cemgil et al, 2001). Finally I will present performance results of the network on a set of MIDI-recorded piano performances of Beatles songs collected by the Music, Mind, Machine Group, NICI, University of Nijmegen (see Cemgil et al, 2001 for more details or http://www.nici.kun.nl/mmm).},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2355 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2356 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2357 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2358
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2359 @INPROCEEDINGS{ElHihi+Bengio-nips8,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2360 author = {El Hihi, Salah and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2361 title = {Hierarchical Recurrent Neural Networks for Long-Term Dependencies},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2362 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2363 crossref = {NIPS8-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2364 abstract = {We have already shown that extracting lone-term dependencies from sequential data is difficult, both for deterministic dynamical systems such as recurrent networks, and probabilistic models such as hidden {Markov} models ({HMM}s) or input/output hidden {Markov} models ({IOHMM}s). In practice, to avoid this problem, researchers have used domain specific a-priori knowledge to give meaning to the hidden or state variables representing past context. In this paper we propose to use a more general type of a-priori knowledge, namely that the temporal dependencies are structured hierarchically. This implies that long-term dependencies are represented by variables with a long time scale. This principle is applied to a recurrent network which includes delays and multiple time scales. Experiments confirm the advantages of such structures. A similar approach is proposed for {HMM}s and {IOHMM}s.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2365 topics={LongTerm},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2366 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2367
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2368 @ARTICLE{Erhan+al-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2369 author = {Erhan, Dumitru and Bengio, Yoshua and Courville, Aaron and Manzagol, Pierre-Antoine and Vincent, Pascal and Bengio, Samy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2370 title = {Why Does Unsupervised Pre-training Help Deep Learning?},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2371 volume = {11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2372 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2373 pages = {625--660},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2374 crossref = {JMLR-shorter},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2375 abstract = {Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of auto-encoder variants, with impressive results obtained in several areas, mostly on vision and language datasets. The best results obtained on supervised learning tasks involve an unsupervised learning component, usually in an unsupervised pre-training phase. Even though these new algorithms have enabled training deep models, many questions remain as to the nature of this difficult learning problem. The main question investigated here is the following: why does unsupervised pre-training work and why does it work so well? Answering these questions is important if learning in deep architectures is to be further improved. We propose several explanatory hypotheses and test them through extensive simulations. We empirically show the influence of pre-training with respect to architecture depth, model capacity, and number of training examples. The experiments confirm and clarify the advantage of unsupervised pre-training. The results suggest that unsupervised pre-training guides the learning towards basins of attraction of minima that are better in terms of the underlying data distribution; the evidence from these results supports a regularization explanation for the effect of pre-training.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2376 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2377
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2378 @INPROCEEDINGS{Erhan-aistats-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2379 author = {Erhan, Dumitru and Courville, Aaron and Bengio, Yoshua and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2380 title = {Why Does Unsupervised Pre-training Help Deep Learning?},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2381 booktitle = {Proceedings of AISTATS 2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2382 volume = {9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2383 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2384 pages = {201-208},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2385 abstract = {Much recent research has been devoted to learning algorithms for deep architectures such as Deep Belief Networks and stacks of auto-encoder variants with impressive results being obtained in several areas, mostly on vision and language datasets. The best results obtained on supervised learning tasks often involve an unsupervised learning component, usually in an unsupervised pre-training phase. The main question investigated here is the following: why does unsupervised pre-training work so well? Through extensive experimentation, we explore several possible explanations discussed in the literature including its action as a regularizer (Erhan et al. 2009) and as an aid to optimization (Bengio et al. 2007). Our results build on the work of Erhan et al. 2009, showing that unsupervised pre-training appears to play predominantly a regularization role in subsequent supervised training. However our results in an online setting, with a virtually unlimited data stream, point to a somewhat more nuanced interpretation of the roles of optimization and regularization in the unsupervised pre-training effect.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2386 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2387
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2388 @MASTERSTHESIS{Erhan-MSc,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2389 author = {Erhan, Dumitru},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2390 keywords = {Apprentisage multit{\^{a}}che, Filtrage collaboratif, M{\'{e}}thodes {\`{a}} noyaux, QSAR, R{\'{e}}seaux de neurones},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2391 title = {Collaborative filtering techniques for drug discovery},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2392 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2393 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2394 abstract = {Cette th{\`{e}}se examine le probl{\`{e}}me d'apprendre plusieurs t{\^{a}}ches simultan{\'{e}}ment,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2395 afin de transf{\'{e}}rer les connaissances apprises {\`{a}} une nouvelle t{\^{a}}che. Si
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2396 on suppose que les t{\^{a}}ches partagent une repr{\'{e}}sentation et qu'il est possible de
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2397 d{\'{e}}couvrir cette repr{\'{e}}sentation efficacement, cela peut nous servir {\`{a}} construire un
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2398 meilleur mod{\`{e}}le de la nouvelle t{\^{a}}che. Il existe plusieurs variantes de
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2399 cette m{\'{e}}thode: transfert inductif, apprentisage multit{\^{a}}che, filtrage
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2400 collaboratif etc. Nous avons {\'{e}}valu{\'{e}} plusieurs algorithmes d'apprentisage
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2401 supervis{\'{e}} pour d{\'{e}}couvrir des repr{\'{e}}sentations partag{\'{e}}es parmi les
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2402 t{\^{a}}ches d{\'{e}}finies dans un probl{\`{e}}me de chimie computationelle. Nous avons
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2403 formul{\'{e}} le probl{\`{e}}me dans un cadre d'apprentisage automatique,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2404 fait l'analogie avec les algorithmes standards de filtrage collaboratif et construit les
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2405 hypoth{\`{e}}ses g{\'{e}}n{\'{e}}rales qui devraient {\^{e}}tre test{\'{e}}es pour valider l'utilitisation des
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2406 algorithmes multit{\^{a}}che. Nous avons aussi {\'{e}}valu{\'{e}} la performance des algorithmes
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2407 d'apprentisage utilis{\'{e}}s et d{\'{e}}montrons qu'il est, en effet, possible de trouver une
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2408 repr{\'{e}}sentation partag{\'{e}}e pour le probl{\`{e}}me consider{\'{e}}. Du point de vue
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2409 th{\'{e}}orique, notre apport est une modification d'un algorithme
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2410 standard---les machines {\`{a}} vecteurs de support--qui produit des r{\'{e}}sultats
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2411 comparables aux meilleurs algorithmes disponsibles et qui utilise {\`{a}} fond les
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2412 concepts de l'apprentisage multit{\^{a}}che. Du point de vue pratique, notre
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2413 apport est l'utilisation de notre algorithme par les compagnies
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2414 pharmaceutiques dans leur d{\'{e}}couverte de nouveaux m{\'{e}}dicaments.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2415 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2416
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2417 @TECHREPORT{Erhan-vis-techreport-2010,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2418 author = {Erhan, Dumitru and Courville, Aaron and Bengio, Yoshua},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2419 title = {Understanding Representations Learned in Deep Architectures},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2420 number = {1355},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2421 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2422 institution = {Universit{\'{e}} de Montr{\'{e}}al/DIRO},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2423 abstract = {Deep architectures have demonstrated state-of-the-art performance in a variety of
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2424 settings, especially with vision datasets. Deep learning algorithms are based on learning
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2425 several levels of representation of the input. Beyond test-set performance, there
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2426 is a need for qualitative comparisons of the solutions learned by various deep architectures,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2427 focused on those learned representations. One of the goals of our research
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2428 is to improve tools for finding good qualitative interpretations of high level features
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2429 learned by such models. We also seek to gain insight into the invariances learned by
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2430 deep networks. To this end, we contrast and compare several techniques for finding
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2431 such interpretations. We applied our techniques on Stacked Denoising Auto-Encoders
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2432 and Deep Belief Networks, trained on several vision datasets. We show that consistent
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2433 filter-like interpretation is possible and simple to accomplish at the unit level. The tools
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2434 developed make it possible to analyze deep models in more depth and accomplish the
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2435 tracing of invariance manifolds for each of the hidden units. We hope that such techniques
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2436 will allow researchers in deep architectures to understand more of how and why
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2437 deep architectures work.}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2438 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2439
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2440 @INPROCEEDINGS{Erhan2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2441 author = {Erhan, Dumitru and Manzagol, Pierre-Antoine and Bengio, Yoshua and Bengio, Samy and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2442 keywords = {Deep Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2443 title = {The Difficulty of Training Deep Architectures and the effect of Unsupervised Pre-Training},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2444 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2445 pages = {153--160},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2446 crossref = {xAISTATS2009-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2447 abstract = {Whereas theoretical work suggests that deep architectures might be more efficient at representing highly-varying functions, training deep architectures was unsuccessful until the recent advent of algorithms based on unsupervised pretraining. Even though these new algorithms have enabled training deep models, many questions remain as to the nature of this difficult learning problem. Answering these questions is important if learning in deep architectures is to be further improved. We attempt to shed some light on these questions through extensive simulations. The experiments confirm and clarify the advantage of unsupervised pre-training. They demonstrate the robustness of the training procedure with respect to the random initialization, the positive effect of pre-training in terms of optimization and its role as a regularizer. We empirically show the influence of pre-training with respect to architecture depth, model capacity, and number of training examples.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2448 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2449
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2450 @ARTICLE{gasser+eck+port:1999,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2451 author = {Gasser, M. and Eck, Douglas and Port, Robert},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2452 title = {Meter as Mechanism: A Neural Network Model that Learns Metrical patterns},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2453 journal = {Connection Science},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2454 volume = {11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2455 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2456 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2457 pages = {187--216},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2458 abstract = {One kind of prosodic structure that apparently underlies both music and some examples of speech production is meter. Yet detailed measurements of the timing of both music and speech show that the nested periodicities that define metrical structure can be quite noisy in time. What kind of system could produce or perceive such variable metrical timing patterns? And what would it take to be able to store and reproduce particular metrical patterns from long-term memory? We have developed a network of coupled oscillators that both produces and perceives patterns of pulses that conform to particular meters. In addition, beginning with an initial state with no biases, it can learn to prefer the particular meter that it has been previously exposed to.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2459 own={Have},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2460 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2461 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2462 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2463
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2464 @TECHREPORT{gasser+eck+port:tr-1996,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2465 author = {Gasser, M. and Eck, Douglas and Port, Robert},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2466 title = {Meter as Mechanism A Neural Network that Learns Metrical Patterns},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2467 number = {180},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2468 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2469 institution = {Indiana University Cognitive Science Program},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2470 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2471 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2472 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2473
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2474 @INPROCEEDINGS{gasser+eck:1996,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2475 author = {Gasser, M. and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2476 editor = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2477 title = {Representing Rhythmic Patterns in a Network of Oscillators},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2478 booktitle = {{The Proceedings of the International Conference on Music Perception and Cognition}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2479 number = {4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2480 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2481 pages = {361--366},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2482 publisher = {Lawrence Erlbaum Associates},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2483 url = {http://www.iro.umontreal.ca/~eckdoug/papers/1996_gasser_icmpc.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2484 abstract = {This paper describes an evolving computational model of the perception and pro-duction of simple rhythmic patterns. The model consists of a network of oscillators of different resting frequencies which couple with input patterns and with each other. Os-cillators whose frequencies match periodicities in the input tend to become activated. Metrical structure is represented explicitly in the network in the form of clusters of os-cillators whose frequencies and phase angles are constrained to maintain the harmonic relationships that characterize meter. Rests in rhythmic patterns are represented by ex-plicit rest oscillators in the network, which become activated when an expected beat in the pattern fails to appear. The model makes predictions about the relative difficulty of patterns and the effect of deviations from periodicity in the input.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2485 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2486 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2487 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2488
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2489 @INPROCEEDINGS{gers+eck+schmidhuber:icann2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2490 author = {Gers, F. A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2491 editor = {Dorffner, Georg},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2492 title = {Applying {LSTM} to Time Series Predictable Through Time-Window Approaches},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2493 booktitle = {{Artificial Neural Networks -- ICANN 2001 (Proceedings)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2494 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2495 pages = {669--676},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2496 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2497 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2001_gers_icann.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2498 abstract = {Long Short-Term Memory ({LSTM}) is able to solve many time series tasks unsolvable by feed-forward networks using fixed size time windows. Here we find that {LSTM}'s superiority does {\em not} carry over to certain simpler time series tasks solvable by time window approaches: the Mackey-Glass series and the Santa Fe {FIR} laser emission series (Set A). This suggests t use {LSTM} only when simpler traditional approaches fail.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2499 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2500 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2501 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2502
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2503 @TECHREPORT{gers+eck+schmidhuber:tr-2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2504 author = {Gers, F. A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2505 title = {Applying {LSTM} to Time Series Predictable Through Time-Window Approaches},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2506 number = {IDSIA-22-00},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2507 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2508 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2509 abstract = {Long Short-Term Memory ({LSTM}) is able to solve many time series tasks unsolvable by feed-forward networks using fixed size time windows. Here we find that {LSTM}'s superiority does {\em not} carry over to certain simpler time series tasks solvable by time window approaches: the Mackey-Glass series and the Santa Fe {FIR} laser emission series (Set A). This suggests t use {LSTM} only when simpler traditional approaches fail.\\ {\em Note: See the 2001 ICANN conference proceeding by the same title for a newer version of this paper.}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2510 ps={ftp://ftp.idsia.ch/pub/techrep/IDSIA-22-00.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2511 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2512 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2513 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2514
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2515 @INPROCEEDINGS{gers+perez+eck+schmidhuber:esann2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2516 author = {Gers, F. A. and Perez-Ortiz, J. A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2517 title = {{DEKF-LSTM}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2518 booktitle = {Proceedings of the 10th European Symposium on Artificial Neural Networks, ESANN 2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2519 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2520 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2521 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2522 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2523
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2524 @INPROCEEDINGS{gers+perez+eck+schmidhuber:icannA2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2525 author = {Gers, F. A. and Perez-Ortiz, J. A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2526 editor = {Dorronsoro, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2527 title = {Learning Context Sensitive Languages with {LSTM} Trained with {Kalman} Filters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2528 booktitle = {{Artificial Neural Networks -- ICANN 2002 (Proceedings)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2529 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2530 pages = {655--660},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2531 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2532 abstract = {Unlike traditional recurrent neural networks, the Long Short-Term Memory ({LSTM}) model generalizes well when presented with training sequences derived from regular and also simple nonregular languages. Our novel combination of {LSTM} and the decoupled extended Kalman filter, however, learns even faster and generalizes even better, requiring only the 10 shortest exemplars n <= 10 of the context sensitive language a^nb^nc^n to deal correctly with values of n up to 1000 and more. Even when we consider the relatively high update complexity per timestep, in many cases the hybrid offers faster learning than {LSTM} by itself.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2533 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2534 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2535 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2536
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2537 @PHDTHESIS{Ghosn-Phd-2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2538 author = {Ghosn, Joumana},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2539 title = {Apprentissage multi-t{\^{a}}ches et partage de connaissances},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2540 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2541 school = {Universit{\'{e}} de Montr{\'{e}}al}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2542 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2543
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2544 @INPROCEEDINGS{ghosn97,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2545 author = {Ghosn, Joumana and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2546 title = {Multi-Task Learning for Stock Selection},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2547 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2548 pages = {946--952},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2549 publisher = {MIT Press, Cambridge, MA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2550 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/multitask-nips97.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2551 crossref = {NIPS9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2552 abstract = {Artificial Neural Networks can be used to predict future returns of stocks in order to take financial decisions. Should one build a separate network for each stock or share the same network for all the stocks. In this paper we also explore other alternatives, in which some layers are shared and others are not shared. When the prediction of future returns for different stocks are viewed as different tasks, sharing some parameters across stocks is a form of multi-task learning. In a series of experiments with Canadian stocks, we obtain yearly returns that are more than 14\% above various benchmarks.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2553 topics={MultiTask,Finance},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2554 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2555
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2556 @TECHREPORT{Gingras-asynchronous-TR96,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2557 author = {Gingras, Fran{\c c}ois and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2558 title = {Handling asynchronous or missing financial data with recurrent networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2559 number = {1020},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2560 year = {1996},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2561 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2562 topics={Finance,Missing},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2563 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2564
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2565 @TECHREPORT{Gingras-financial-TR99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2566 author = {Gingras, Fran{\c c}ois and Bengio, Yoshua and Nadeau, Claude},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2567 title = {On Out-of-Sample Statistics for Financial Time-Series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2568 number = {2585},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2569 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2570 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2571 topics={Comparative,Finance},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2572 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2573
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2574 @INPROCEEDINGS{gingras2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2575 author = {Gingras, Fran{\c c}ois and Bengio, Yoshua and Nadeau, Claude},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2576 title = {On Out-of-Sample Statistics for Time-Series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2577 booktitle = {Computational Finance 2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2578 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2579 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/out-err-cf2000.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2580 abstract = {This paper studies an out-of-sample statistic for time-series prediction that is analogous to the widely used R2 in-sample statistic. We propose and study methods to estimate the variance of this out-of-sample statistic. We suggest that the out-of-sample statistic is more robust to distributional and asymptotic assumptions behind many tests for in-sample statistics. Furthermore we argue that it may be more important in some cases to choose a model that generalizes as well as possible rather than choose the parameters that are closest to the true parameters. Comparative experiments are performed on a financial time-series (daily and monthly returns of the TSE300 index). The experiments are performed or varying prediction horizons and we study the relation between predictibility (out-of-sample R2), variability of the out-of-sample R2 statistic, and the prediction horizon.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2581 topics={Comparative,Finance},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2582 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2583
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2584 @INPROCEEDINGS{GlorotAISTATS2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2585 author = {Bengio, Yoshua and Glorot, Xavier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2586 title = {Understanding the difficulty of training deep feedforward neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2587 booktitle = {Proceedings of AISTATS 2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2588 volume = {9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2589 year = {2010},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2590 pages = {249-256},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2591 abstract = {Whereas before 2006 it appears that deep multi-layer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the superiority of deeper vs less deep architectures. All these experimental results were obtained with new initialization or training mechanisms. Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. We first observe the influence of the non-linear activations functions. We find that the logistic sigmoid activation is unsuited for deep networks with random initialization because of its mean value, which can drive especially the top hidden layer into saturation. Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks. We find that a new non-linearity that saturates less can often be beneficial. Finally, we study how activations and gradients vary across layers and during training, with the idea that training may be more difficult when the singular values of the Jacobian associated with each layer are far from 1. Based on these considerations, we propose a new initialization scheme that brings substantially faster convergence.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2592 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2593
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2594 @INPROCEEDINGS{Gori89,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2595 author = {Gori, Marco and Bengio, Yoshua and De Mori, Renato},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2596 title = {BPS: a learning algorithm for capturing the dynamic nature of speech},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2597 booktitle = {International Joint Conference on Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2598 volume = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2599 year = {1989},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2600 pages = {417--424},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2601 publisher = {IEEE, New York},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2602 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2603 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2604
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2605 @INCOLLECTION{Grandvalet+Bengio-ssl-2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2606 author = {Grandvalet, Yves and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2607 editor = {Chapelle, Olivier and {Sch{\"{o}}lkopf}, Bernhard and Zien, Alexander},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2608 title = {Entropy Regularization},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2609 booktitle = {Semi-Supervised Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2610 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2611 pages = {151--168},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2612 publisher = {{MIT} Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2613 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/entropy_regularization_2006.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2614 abstract = {The problem of semi-supervised induction consists in learning a decision rule from
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2615 labeled and unlabeled data. This task can be undertaken by discriminative methods,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2616 provided that learning criteria are adapted consequently. In this chapter, we motivate the use of entropy regularization as a means to benefit from unlabeled data in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2617 the framework of maximum a posteriori estimation. The learning criterion is derived
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2618 from clearly stated assumptions and can be applied to any smoothly parametrized
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2619 model of posterior probabilities. The regularization scheme favors low density separation, without any modeling of the density of input features. The contribution
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2620 of unlabeled data to the learning criterion induces local optima, but this problem
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2621 can be alleviated by deterministic annealing. For well-behaved models of posterior
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2622 probabilities, deterministic annealing {EM} provides a decomposition of the learning
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2623 problem in a series of concave subproblems. Other approaches to the semi-supervised
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2624 problem are shown to be close relatives or limiting cases of entropy regularization.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2625 A series of experiments illustrates the good behavior of the algorithm in terms of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2626 performance and robustness with respect to the violation of the postulated low density separation assumption. The minimum entropy solution benefits from unlabeled
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2627 data and is able to challenge mixture models and manifold learning in a number of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2628 situations.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2629 cat={B},topics={Unsupervised},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2630 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2631
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2632 @INPROCEEDINGS{graves+eck+schmidhuber:bio-adit2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2633 author = {Graves, A. and Eck, Douglas and Beringer, N. and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2634 title = {Biologically Plausible Speech Recognition with {LSTM} Neural Nets},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2635 booktitle = {Proceedings of the First Int'l Workshop on Biologically Inspired Approaches to Advanced Information Technology (Bio-ADIT)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2636 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2637 pages = {127-136},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2638 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2004_bioadit.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2639 abstract = {Long Short-Term Memory ({LSTM}) recurrent neural networks ({RNN}s) are local in space and time and closely related to a biological model of memory in the prefrontal cortex. Not only are they more biologically plausible than previous artificial {RNN}s, they also outperformed them on many artificially generated sequential processing tasks. This encouraged us to apply {LSTM} to more realistic problems, such as the recognition of spoken digits. Without any modification of the underlying algorithm, we achieved results comparable to state-of-the-art Hidden {Markov} Model ({HMM}) based recognisers on both the {TIDIGITS} and TI46 speech corpora. We conclude that {LSTM} should be further investigated as a biologically plausible basis for a bottom-up, neural net-based approach to speech recognition.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2640 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2641 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2642 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2643
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2644 @TECHREPORT{graves+eck+schmidhuber:tr-digits2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2645 author = {Graves, A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2646 title = {Comparing {LSTM} Recurrent Networks and Spiking Recurrent Networks on the Recognition of Spoken Digits},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2647 number = {IDSIA-13-03},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2648 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2649 institution = {IDSIA},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2650 abstract = {One advantage of spiking recurrent neural networks ({SNN}s) is an ability to categorise data using a synchrony-based latching mechnanism. This is particularly useful in problems where timewarping is encountered, such as speech recognition. Differentiable recurrent neural networks ({RNN}s) by contrast fail at tasks involving difficult timewarping, despite having sequence learning capabilities superior to {SNN}s. In this paper we demonstrate that Long Short-Term Memory ({LSTM}) is an {RNN} capable of robustly categorizing timewarped speech data, thus combining the most useful features of both paradigms. We compare its performance to {SNN}s on two variants of a spoken digit identification task, using data from an international competition. The first task (described in Nature (Nadis 2003)) required the categorisation of spoken digits with only a single training exemplar, and was specifically designed to test robustness to timewarping. Here {LSTM} performed better than all the {SNN}s in the competition. The second task was to predict spoken digits using a larger training set. Here {LSTM} greatly outperformed an {SNN}-like model found in the literature. These results suggest that {LSTM} has a place in domains that require the learning of large timewarped datasets, such as automatic speech recognition.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2651 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2652 sourcetype={TechReport},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2653 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2654
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2655 @INPROCEEDINGS{haffner-98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2656 author = {Haffner, Patrick and Bottou, {L{\'{e}}on} and G. Howard, Paul and Simard, Patrice and Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2657 title = {Browsing through High Quality Document Images with {DjVu}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2658 booktitle = {Proc. of Advances in Digital Libraries 98},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2659 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2660 pages = {309--318},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2661 publisher = {IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2662 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/haffner-98.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2663 topics={HighDimensional},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2664 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2665
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2666 @INPROCEEDINGS{Hamel+al-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2667 author = {Hamel, Philippe and Wood, Sean and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2668 title = {Automatic Identification of Instrument Classes in Polyphonic and Poly-Instrument Audio},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2669 booktitle = {10th International Society for Music Information Retrieval Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2670 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2671 pages = {399--404},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2672 url = {http://ismir2009.ismir.net/proceedings/PS3-2.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2673 abstract = {We present and compare several models for automatic identification of instrument classes in polyphonic and poly-instrument audio. The goal is to be able to identify which categories of instrument (Strings, Woodwind, Guitar, Piano, etc.) are present in a given audio example. We use a machine learning approach to solve this task. We constructed a system to generate a large database of musically relevant poly-instrument audio. Our database is generated from hundreds of instruments classified in 7 categories. Musical audio examples are generated by mixing multi-track MIDI files with thousands of instrument combinations. We compare three different classifiers : a Support Vector Machine ({SVM}), a Multilayer Perceptron (MLP) and a Deep Belief Network (DBN). We show that the DBN tends to outperform both the {SVM} and the MLP in most cases.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2674 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2675
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2676 @MISC{Hugo+al-snowbird-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2677 author = {Larochelle, Hugo and Bengio, Yoshua and Erhan, Dumitru},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2678 title = {Generalization to a zero-data task: an empirical study},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2679 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2680 howpublished = {Talk and poster presented at the Learning Workshop(Snowbird), San Juan, Puerto Rico, 2007}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2681 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2682
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2683 @INPROCEEDINGS{hyper:2000:ijcnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2684 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2685 title = {Continuous Optimization of Hyper-Parameters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2686 booktitle = {International Joint Conference on Neural Networks 2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2687 volume = {I},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2688 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2689 pages = {305--310},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2690 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/hyper-ijcnn2000.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2691 abstract = {Many machine learning algorithms can be formulated as the minimization of a training criterion which involves a hyper-parameter. This hyper-parameter is usually chosen by trial and error with a model selection criterion. In this paper we present a methodology to optimize several hyper-parameters, based on the computation of the gradient of a model selection criterion with respect to the hyper-parameters. In the case of a quadratic training criterion, the gradient of the selection criterion with respect to the hyper-parameters is efficiently computed by back-propagating through a Cholesky decomposition. In the more general case, we show that the implicit function theorem can be used to derive a formula for the hyper-parameter gradient involving second derivatives of the training criterion.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2692 topics={ModelSelection},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2693 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2694
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2695 @INPROCEEDINGS{ICML01,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2696 editor = {Brodley, Carla E. and Danyluk, Andrea Pohoreckyj},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2697 title = {Proceedings of the Eighteenth International Conference on Machine Learning (ICML'01)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2698 booktitle = {Proceedings of the Eighteenth International Conference on Machine Learning (ICML'01)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2699 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2700 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2701 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2702
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2703 @INPROCEEDINGS{ICML01-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2704 editor = {Brodley, Carla E. and Danyluk, Andrea Pohoreckyj},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2705 title = {Proceedings of the Eighteenth International Conference on Machine Learning (ICML'01)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2706 booktitle = {ICML'01},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2707 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2708 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2709 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2710
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2711
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2712 @INPROCEEDINGS{ICML02,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2713 editor = {Sammut, Claude and Hoffmann, Achim G.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2714 title = {Proceedings of the Nineteenth International Conference on Machine Learning (ICML'02)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2715 booktitle = {Proceedings of the Nineteenth International Conference on Machine Learning (ICML'02)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2716 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2717 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2718 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2719
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2720 @INPROCEEDINGS{ICML02-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2721 editor = {Sammut, Claude and Hoffmann, Achim G.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2722 title = {Proceedings of the Nineteenth International Conference on Machine Learning (ICML'02)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2723 booktitle = {ICML'02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2724 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2725 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2726 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2727
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2728
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2729 @INPROCEEDINGS{ICML03,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2730 editor = {Fawcett, Tom and Mishra, Nina},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2731 title = {Proceedings of the Twenty International Conference on Machine Learning (ICML'03)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2732 booktitle = {Proceedings of the Twenty International Conference on Machine Learning (ICML'03)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2733 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2734 publisher = {AAAI Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2735 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2736
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2737 @INPROCEEDINGS{ICML03-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2738 editor = {Fawcett, Tom and Mishra, Nina},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2739 title = {Proceedings of the Twenty International Conference on Machine Learning (ICML'03)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2740 booktitle = {ICML'03},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2741 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2742 publisher = {AAAI Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2743 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2744
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2745
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2746 @INPROCEEDINGS{ICML04,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2747 editor = {Brodley, Carla E.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2748 title = {Proceedings of the Twenty-first International Conference on Machine Learning (ICML'04)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2749 booktitle = {Proceedings of the Twenty-first International Conference on Machine Learning (ICML'04)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2750 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2751 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2752 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2753
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2754 @INPROCEEDINGS{ICML04-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2755 editor = {Brodley, Carla E.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2756 title = {Proceedings of the Twenty-first International Conference on Machine Learning (ICML'04)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2757 booktitle = {ICML'04},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2758 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2759 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2760 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2761
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2762
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2763 @INPROCEEDINGS{ICML05-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2764 editor = {Raedt, Luc De and Wrobel, Stefan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2765 title = {Proceedings of the Twenty-second International Conference on Machine Learning (ICML'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2766 booktitle = {ICML'05},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2767 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2768 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2769 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2770
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2771
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2772 @INPROCEEDINGS{ICML06-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2773 editor = {Cohen, William W. and Moore, Andrew},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2774 title = {Proceedings of the Twenty-three International Conference on Machine Learning (ICML'06)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2775 booktitle = {ICML'06},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2776 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2777 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2778 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2779
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2780
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2781 @INPROCEEDINGS{ICML07-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2782 editor = {Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2783 title = {Proceedings of the 24th International Conference on Machine Learning (ICML'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2784 booktitle = {ICML'07},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2785 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2786 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2787 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2788
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2789
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2790 @INPROCEEDINGS{ICML08-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2791 editor = {Cohen, William W. and McCallum, Andrew and Roweis, Sam T.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2792 title = {Proceedings of the Twenty-fifth International Conference on Machine Learning (ICML'08)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2793 booktitle = {ICML'08},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2794 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2795 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2796 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2797
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2798
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2799 @INPROCEEDINGS{ICML09-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2800 editor = {Bottou, {L{\'{e}}on} and Littman, Michael},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2801 title = {Proceedings of the Twenty-sixth International Conference on Machine Learning (ICML'09)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2802 booktitle = {ICML'09},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2803 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2804 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2805 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2806
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2807
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2808 @INPROCEEDINGS{ICML96,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2809 editor = {Saitta, L.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2810 title = {Proceedings of the Thirteenth International Conference on Machine Learning (ICML'96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2811 booktitle = {Proceedings of the Thirteenth International Conference on Machine Learning (ICML'96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2812 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2813 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2814 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2815
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2816 @INPROCEEDINGS{ICML96-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2817 editor = {Saitta, L.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2818 title = {Proceedings of the Thirteenth International Conference on Machine Learning (ICML'96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2819 booktitle = {ICML'96},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2820 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2821 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2822 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2823
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2824
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2825 @INPROCEEDINGS{ICML97,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2826 editor = {Fisher, Douglas H.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2827 title = {{}Proceedings of the Fourteenth International Conference on Machine Learning (ICML'97)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2828 booktitle = {Proceedings of the Fourteenth International Conference on Machine Learning (ICML'97)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2829 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2830 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2831 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2832
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2833 @INPROCEEDINGS{ICML97-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2834 editor = {Fisher, Douglas H.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2835 title = {{}Proceedings of the Fourteenth International Conference on Machine Learning (ICML'97)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2836 booktitle = {ICML'97},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2837 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2838 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2839 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2840
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2841
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2842 @INPROCEEDINGS{ICML98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2843 editor = {Shavlik, Jude W.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2844 title = {Proceedings of the Fifteenth International Conference on Machine Learning (ICML'98)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2845 booktitle = {Proceedings of the Fifteenth International Conference on Machine Learning (ICML'98)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2846 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2847 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2848 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2849
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2850 @INPROCEEDINGS{ICML98-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2851 editor = {Shavlik, Jude W.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2852 title = {Proceedings of the Fifteenth International Conference on Machine Learning (ICML'98)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2853 booktitle = {ICML'98},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2854 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2855 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2856 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2857
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2858
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2859 @INPROCEEDINGS{ICML99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2860 editor = {Bratko, Ivan and Dzeroski, Saso},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2861 title = {Proceedings of the Sixteenth International Conference on Machine Learning (ICML'99)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2862 booktitle = {Proceedings of the Sixteenth International Conference on Machine Learning (ICML'99)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2863 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2864 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2865 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2866
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2867 @INPROCEEDINGS{ICML99-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2868 editor = {Bratko, Ivan and Dzeroski, Saso},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2869 title = {Proceedings of the Sixteenth International Conference on Machine Learning (ICML'99)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2870 booktitle = {ICML'99},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2871 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2872 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2873 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2874
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2875
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2876 @INCOLLECTION{jaeger+eck:2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2877 author = {Jaeger, H. and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2878 title = {Can't get you out of my head: {A} connectionist model of cyclic rehearsal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2879 booktitle = {Modeling Communications with Robots and Virtual Humans},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2880 series = {{LNCS}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2881 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2882 publisher = {Springer-Verlag},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2883 url = {http://www.iro.umontreal.ca/~eckdoug/papers/2007_jaeger_eck.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2884 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2885 sourcetype={Chapter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2886 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2887
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2888 @MISC{James+al-snowbird-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2889 author = {Bergstra, James and Bengio, Yoshua and Louradour, Jerome},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2890 title = {Image Classification using Higher-Order Neural Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2891 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2892 howpublished = {The Learning Workshop (Snowbird, Utah)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2893 url = {http://snowbird.djvuzone.org/2007/abstracts/161.pdf}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2894 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2895
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2896 @ARTICLE{JMLR-short,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2897 journal = {JMLR},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2898 year = {-1}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2899 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2900
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2901
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2902 @INPROCEEDINGS{Kegl+Bertin+Eck-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2903 author = {K{\'{e}}gl, Bal{\'{a}}zs and Bertin-Mahieux, Thierry and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2904 title = {Metropolis-Hastings Sampling in a FilterBoost Music Classifier},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2905 booktitle = {Music and machine learning workshop (ICML08)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2906 year = {2008}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2907 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2908
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2909 @INPROCEEDINGS{kegl2005b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2910 author = {K{\'{e}}gl, Bal{\'{a}}zs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2911 title = {Generalization Error and Algorithmic Convergence of Median Boosting.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2912 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2913 crossref = {NIPS17-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2914 abstract = {We have recently proposed an extension of ADABOOST to regression that uses the median of the base regressors as the final regressor. In this paper we extend theoretical results obtained for ADABOOST to median boosting and to its localized variant. First, we extend recent results on efficient margin maximizing to show that the algorithm can converge to the maximum achievable margin within a preset precision in a finite number of steps. Then we provide confidence-interval-type bounds on the generalization error.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2915 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2916
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2917 @ARTICLE{lacoste+eck:eurasip,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2918 author = {Lacoste, Alexandre and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2919 title = {A Supervised Classification Algorithm For Note Onset Detection},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2920 journal = {EURASIP Journal on Applied Signal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2921 volume = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2922 number = {ID 43745},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2923 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2924 pages = {1--13},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2925 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2926 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2927 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2928
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2929 @MASTERSTHESIS{Lajoie2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2930 author = {Lajoie, Isabelle},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2931 keywords = {apprentissage non-supervis{\'{e}}, architecture profonde, auto-encodeur d{\'{e}}bruiteur, machine de {Boltzmann} restreinte, r{\'{e}}seau de neurones artificiel},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2932 title = {Apprentissage de repr{\'{e}}sentations sur-compl{\`{e}}tes par entra{\^{\i}}nement d’auto-encodeurs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2933 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2934 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2935 abstract = {Les avanc{\'{e}}s dans le domaine de l’intelligence artificielle, permettent {\`{a}} des syst{\`{e}}mes
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2936 informatiques de r{\'{e}}soudre des t{\^{a}}ches de plus en plus complexes li{\'{e}}es par exemple {\`{a}}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2937 la vision, {\`{a}} la compr{\'{e}}hension de signaux sonores ou au traitement de la langue. Parmi
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2938 les mod{\`{e}}les existants, on retrouve les R{\'{e}}seaux de Neurones Artificiels (RNA), dont la
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2939 popularit{\'{e}} a fait un grand bond en avant avec la d{\'{e}}couverte de Hinton et al. [22], soit
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2940 l’utilisation de Machines de {Boltzmann} Restreintes (RBM) pour un pr{\'{e}}-entra{\^{\i}}nement
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2941 non-supervis{\'{e}} couche apr{\`{e}}s couche, facilitant grandement l’entra{\^{\i}}nement supervis{\'{e}} du
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2942 r{\'{e}}seau {\`{a}} plusieurs couches cach{\'{e}}es (DBN), entra{\^{\i}}nement qui s’av{\'{e}}rait jusqu’alors tr{\`{e}}s
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2943 difficile {\`{a}} r{\'{e}}ussir. Depuis cette d{\'{e}}couverte, des chercheurs ont {\'{e}}tudi{\'{e}} l’efficacit{\'{e}} de nouvelles strat{\'{e}}gies de pr{\'{e}}-entra{\^{\i}}nement, telles que l’empilement d’auto-encodeurs traditionnels (SAE) [5, 38], et l’empilement d’auto-encodeur d{\'{e}}bruiteur (SDAE) [44].
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2944 C’est dans ce contexte qu’a d{\'{e}}but{\'{e}} la pr{\'{e}}sente {\'{e}}tude. Apr{\`{e}}s un bref passage en revue des notions de base du domaine de l’apprentissage machine et des m{\'{e}}thodes de
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2945 pr{\'{e}}-entra{\^{\i}}nement employ{\'{e}}es jusqu’{\`{a}} pr{\'{e}}sent avec les modules RBM, AE et DAE, nous
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2946 avons approfondi notre compr{\'{e}}hension du pr{\'{e}}-entra{\^{\i}}nement de type SDAE, explor{\'{e}} ses
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2947 diff{\'{e}}rentes propri{\'{e}}t{\'{e}}s et {\'{e}}tudi{\'{e}} des variantes de SDAE comme strat{\'{e}}gie d’initialisation
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2948 d’architecture profonde. Nous avons ainsi pu, entre autres choses, mettre en lumi{\`{e}}re
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2949 l’influence du niveau de bruit, du nombre de couches et du nombre d’unit{\'{e}}s cach{\'{e}}es
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2950 sur l’erreur de g{\'{e}}n{\'{e}}ralisation du SDAE. Nous avons constat{\'{e}} une am{\'{e}}lioration de la
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2951 performance sur la t{\^{a}}che supervis{\'{e}}e avec l’utilisation des bruits poivre et sel (PS) et
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2952 gaussien (GS), bruits s’av{\'{e}}rant mieux justifi{\'{e}}s que celui utilis{\'{e}} jusqu’{\`{a}} pr{\'{e}}sent, soit le
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2953 masque {\`{a}} z{\'{e}}ro (MN). De plus, nous avons d{\'{e}}montr{\'{e}} que la performance profitait d’une
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2954 emphase impos{\'{e}}e sur la reconstruction des donn{\'{e}}es corrompues durant l’entra{\^{\i}}nement
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2955 des diff{\'{e}}rents DAE. Nos travaux ont aussi permis de r{\'{e}}v{\'{e}}ler que le DAE {\'{e}}tait en mesure d’apprendre, sur des images naturelles, des filtres semblables {\`{a}} ceux retrouv{\'{e}}s dans
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2956 les cellules V1 du cortex visuel, soit des filtres d{\'{e}}tecteurs de bordures. Nous aurons par
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2957 ailleurs pu montrer que les repr{\'{e}}sentations apprises du SDAE, compos{\'{e}}es des caract{\'{e}}ristiques ainsi extraites, s’av{\'{e}}raient fort utiles {\`{a}} l’apprentissage d’une machine {\`{a}} vecteurs de
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2958 support ({SVM}) lin{\'{e}}aire ou {\`{a}} noyau gaussien, am{\'{e}}liorant grandement sa performance de
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2959 g{\'{e}}n{\'{e}}ralisation. Aussi, nous aurons observ{\'{e}} que similairement au DBN, et contrairement
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2960 au SAE, le SDAE poss{\'{e}}dait une bonne capacit{\'{e}} en tant que mod{\`{e}}le g{\'{e}}n{\'{e}}rateur. Nous
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2961 avons {\'{e}}galement ouvert la porte {\`{a}} de nouvelles strat{\'{e}}gies de pr{\'{e}}-entra{\^{\i}}nement et d{\'{e}}couvert le potentiel de l’une d’entre elles, soit l’empilement d’auto-encodeurs rebruiteurs
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2962 (SRAE).}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2963 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2964
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2965 @INPROCEEDINGS{lamere+eck:ismir2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2966 author = {Lamere, Paul and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2967 editor = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2968 title = {Using 3D Visualizations to Explore and Discover Music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2969 booktitle = {{Proceedings of the 8th International Conference on Music Information Retrieval ({ISMIR} 2007)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2970 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2971 publisher = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2972 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2973 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2974 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2975
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2976 @ARTICLE{Larochelle+al-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2977 author = {Larochelle, Hugo and Bengio, Yoshua and Turian, Joseph},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2978 title = {Tractable Multivariate Binary Density Estimation and the Restricted {Boltzmann} Forest},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2979 journal = {Neural Computation},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2980 volume = {22},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2981 number = {9},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2982 year = {2010},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
2983 pages = {2285--2307}
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2984 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2985
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2986 @INPROCEEDINGS{Larochelle+Bengio-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2987 author = {Larochelle, Hugo and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2988 title = {Classification using Discriminative Restricted {B}oltzmann Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2989 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2990 pages = {536--543},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2991 crossref = {ICML08-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2992 abstract = {Recently, many applications for Restricted {Boltzmann} Machines (RBMs) have been developed for a large variety of learning problems. However, RBMs are usually used as feature extractors for another learning algorithm or to provide a good initialization
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2993 for deep feed-forward neural network classifiers, and are not considered as a standalone solution to classification problems. In
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2994 this paper, we argue that RBMs provide a self-contained framework for deriving competitive non-linear classifiers. We present an evaluation of different learning algorithms for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2995 RBMs which aim at introducing a discriminative component to RBM training and improve their performance as classifiers. This
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2996 approach is simple in that RBMs are used directly to build a classifier, rather than as a stepping stone. Finally, we demonstrate how discriminative RBMs can also be successfully employed in a semi-supervised setting.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2997 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2998
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
2999 @INPROCEEDINGS{Larochelle-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3000 author = {Larochelle, Hugo and Erhan, Dumitru and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3001 title = {Deep Learning using Robust Interdependent Codes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3002 booktitle = {Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS 2009)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3003 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3004 pages = {312--319},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3005 date = "April 16-18, 2009",
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3006 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3007
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3008 @ARTICLE{Larochelle-jmlr-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3009 author = {Larochelle, Hugo and Bengio, Yoshua and Louradour, Jerome and Lamblin, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3010 title = {Exploring Strategies for Training Deep Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3011 volume = {10},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3012 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3013 pages = {1--40},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3014 crossref = {JMLR-shorter},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3015 abstract = {Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varying functions. However, until recently it was not clear how to train such deep networks, since gradient-based optimization starting from random initialization often appears to get stuck in poor solutions. Hinton et al. recently proposed a greedy layer-wise unsupervised learning procedure relying on the training algorithm of restricted {Boltzmann} machines (RBM) to initialize the parameters of a deep belief network (DBN), a generative model with many layers of hidden causal variables. This was followed by the proposal of another greedy layer-wise procedure, relying on the usage of autoassociator networks. In the context of the above optimization problem, we study these algorithms empirically to better understand their success. Our experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy helps the optimization by initializing weights in a region near a good local minimum, but also implicitly acts as a sort of regularization that brings better generalization and encourages internal distributed representations that are high-level abstractions of the input. We also present a series of experiments aimed at evaluating the link between the performance of deep neural networks and practical aspects of their topology, for example, demonstrating cases where the addition of more depth helps. Finally, we empirically explore simple variants of these training algorithms, such as the use of different RBM input unit distributions, a simple way of combining gradient estimators to improve performance, as well as on-line versions of those algorithms.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3016 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3017
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3018 @PHDTHESIS{Larochelle-PhD-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3019 author = {Larochelle, Hugo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3020 keywords = {apprentissage non-supervis{\'{e}}, architecture profonde, autoassociateur, autoencodeur, machine de {Boltzmann} restreinte, r{\'{e}}seau de neurones artificiel},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3021 title = {{\'{E}}tude de techniques d'apprentissage non-supervis{\'{e}} pour l'am{\'{e}}lioration de l'entra{\^{\i}}nement supervis{\'{e}} de mod{\`{e}}les connexionnistes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3022 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3023 school = {University of Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3024 abstract = {Le domaine de l'intelligence artificielle a pour objectif le d{\'{e}}veloppement de syst{\`{e}}mes informatiques capables de simuler des comportements normalement associ{\'{e}}s {\`{a}} l'intelligence humaine. On aimerait entre autres pouvoir construire une machine qui puisse
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3025 r{\'{e}}soudre des t{\^{a}}ches li{\'{e}}es {\`{a}} la vision (e.g., la reconnaissance d'objet), au traitement de la langue (e.g., l'identification du sujet d'un texte) ou au traitement de signaux sonores (e.g., la reconnaissance de la parole).
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3026 Une approche d{\'{e}}velopp{\'{e}}e afin de r{\'{e}}soudre ce genre de t{\^{a}}ches est bas{\'{e}}e sur l'apprentissage automatique de mod{\`{e}}les {\`{a}} partir de donn{\'{e}}es {\'{e}}tiquet{\'{e}}es refl{\'{e}}tant le comportement intelligent {\`{a}} {\'{e}}muler. Entre autre, il a {\'{e}}t{\'{e}} propos{\'{e}} de mod{\'{e}}liser le calcul n{\'{e}}cessaire {\`{a}} la
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3027 r{\'{e}}solution d'une t{\^{a}}che {\`{a}} l'aide d'un r{\'{e}}seau de neurones artificiel, dont il est possible d'adapter le comportement {\`{a}} l'aide de la r{\'{e}}tropropagation [99, 131] d'un gradient informatif sur les erreurs commises par le r{\'{e}}seau. Populaire durant les ann{\'{e}}es 80, cette
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3028 approche sp{\'{e}}cifique a depuis perdu partiellement de son attrait, suite au d{\'{e}}veloppement des m{\'{e}}thodes {\`{a}} noyau. Celles-ci sont souvent plus stables, plus faciles {\`{a}} utiliser et leur performance est souvent au moins aussi {\'{e}}lev{\'{e}}e pour une vaste gamme de probl{\`{e}}mes.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3029 Les m{\'{e}}thodes d'apprentissage automatique ont donc progress{\'{e}} dans leur fonctionnement, mais aussi dans la complexit{\'{e}} des probl{\`{e}}mes auxquels elles se sont attaqu{\'{e}}. Ainsi, plus r{\'{e}}cemment, des travaux [12, 15] ont commenc{\'{e}} {\`{a}} {\'{e}}mettre des doutes sur la capacit{\'{e}} des machines {\`{a}} noyau {\`{a}} pouvoir efficacement r{\'{e}}soudre des probl{\`{e}}mes de la complexit{\'{e}} requise par l'intelligence artificielle. Parall{\`{e}}lement, Hinton et al. [81] faisaient une perc{\'{e}}e dans l'apprentissage automatique de r{\'{e}}seaux de neurones, en proposant une proc{\'{e}}dure permettant l'entra{\^{\i}}nement de r{\'{e}}seaux de neurones d'une plus grande complexit{\'{e}} (i.e., avec plus de couches de neurones cach{\'{e}}es) qu'il n'{\'{e}}tait possible auparavant.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3030 C'est dans ce contexte qu'ont {\'{e}}t{\'{e}} conduits les travaux de cette th{\`{e}}se. Cette th{\`{e}}se d{\'{e}}bute par une exposition des principes de base de l'apprentissage automatique (chapitre 1) et une discussion des obstacles {\`{a}} l'obtention d'un mod{\`{e}}le ayant une bonne performance
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3031 de g{\'{e}}n{\'{e}}ralisation (chapitre 2). Puis, sont pr{\'{e}}sent{\'{e}}es les contributions apport{\'{e}}es dans le cadre de cinq articles, contributions qui sont toutes bas{\'{e}}es sur l'utilisation d'une certaine
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3032 forme d'apprentissage non-supervis{\'{e}}.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3033 Le premier article (chapitre 4) propose une m{\'{e}}thode d'entra{\^{\i}}nement pour un type sp{\'{e}}cifique de r{\'{e}}seau {\`{a}} une seule couche cach{\'{e}}e (la machine de {Boltzmann} restreinte) bas{\'{e}}e sur une combinaison des apprentissages supervis{\'{e}} et non-supervis{\'{e}}. Cette m{\'{e}}thode permet d'obtenir une meilleure performance de g{\'{e}}n{\'{e}}ralisation qu'un r{\'{e}}seau de neurones standard ou qu'une machine {\`{a}} vecteurs de support {\`{a}} noyau, et met en {\'{e}}vidence de fa{\c c}on
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3034 explicite les b{\'{e}}n{\'{e}}fices qu'apporte l'apprentissage non-supervis{\'{e}} {\`{a}} l'entra{\^{\i}}nement d'un r{\'{e}}seau de neurones.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3035 Ensuite, dans le second article (chapitre 6), on {\'{e}}tudie et {\'{e}}tend la proc{\'{e}}dure d'entra{\^{\i}}nement propos{\'{e}}e par Hinton et al. [81]. Plus sp{\'{e}}cifiquement, on y propose une approche diff{\'{e}}rente mais plus flexible pour initialiser un r{\'{e}}seau {\`{a}} plusieurs couches cach{\'{e}}es, bas{\'{e}}e sur un r{\'{e}}seau autoassociateur. On y explore aussi l'impact du nombre de couches et de neurones par couche sur la performance d'un r{\'{e}}seau et on y d{\'{e}}crit diff{\'{e}}rentes variantes mieux adapt{\'{e}}es {\`{a}} l'apprentissage en ligne ou pour donn{\'{e}}es {\`{a}} valeurs continues.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3036 Dans le troisi{\`{e}}me article (chapitre 8), on explore plut{\^{o}}t la performance de r{\'{e}}seaux profonds sur plusieurs probl{\`{e}}mes de classification diff{\'{e}}rents. Les probl{\`{e}}mes choisis ont la propri{\'{e}}t{\'{e}} d'avoir {\'{e}}t{\'{e}} g{\'{e}}n{\'{e}}r{\'{e}}s {\`{a}} partir de plusieurs facteurs de variation. Cette propri{\'{e}}t{\'{e}}, qui caract{\'{e}}rise les probl{\`{e}}mes li{\'{e}}s {\`{a}} l'intelligence artificielle, pose difficult{\'{e}} aux machines {\`{a}} noyau, tel que confirm{\'{e}} par les exp{\'{e}}riences de cet article.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3037 Le quatri{\`{e}}me article (chapitre 10) pr{\'{e}}sente une am{\'{e}}lioration de l'approche bas{\'{e}}e sur les r{\'{e}}seaux autoassociateurs. Cette am{\'{e}}lioration applique une modification simple {\`{a}} la proc{\'{e}}dure d'entra{\^{\i}}nement d'un r{\'{e}}seau autoassociateur, en « bruitant » les entr{\'{e}}es du r{\'{e}}seau afin que celui-ci soit forc{\'{e}} {\`{a}} la d{\'{e}}bruiter.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3038 Le cinqui{\`{e}}me et dernier article (chapitre 12) apporte une autre am{\'{e}}lioration aux r{\'{e}}seaux autoassociateurs, en permettant des interactions d'inhibition ou d'excitation entre les neurones cach{\'{e}}s de ces r{\'{e}}seaux. On y d{\'{e}}montre que de telles interactions peuvent
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3039 {\^{e}}tre apprises et sont b{\'{e}}n{\'{e}}fiques {\`{a}} la performance d'un r{\'{e}}seau profond.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3040 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3041
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3042 @INPROCEEDINGS{Larochelle2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3043 author = {Larochelle, Hugo and Erhan, Dumitru and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3044 title = {Zero-data Learning of New Tasks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3045 booktitle = {AAAI Conference on Artificial Intelligence},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3046 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3047 url = {http://www-etud.iro.umontreal.ca/~larocheh/publications/aaai2008_zero-data.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3048 abstract = {Recently, many applications for Restricted {Boltzmann} Machines (RBMs) have been developed for a large variety of learning problems. However, RBMs are usually used as feature extractors for another learning algorithm or to provide a good initialization
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3049 for deep feed-forward neural network classifiers, and are not considered as a standalone solution to classification problems. In
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3050 this paper, we argue that RBMs provide a self-contained framework for deriving competitive non-linear classifiers. We present an evaluation of different learning algorithms for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3051 RBMs which aim at introducing a discriminative component to RBM training and improve their performance as classifiers. This
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3052 approach is simple in that RBMs are used directly to build a classifier, rather than as a stepping stone. Finally, we demonstrate how discriminative RBMs can also be successfully employed in a semi-supervised setting.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3053 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3054
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3055 @INPROCEEDINGS{LarochelleH2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3056 author = {Larochelle, Hugo and Erhan, Dumitru and Courville, Aaron and Bergstra, James and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3057 title = {An Empirical Evaluation of Deep Architectures on Problems with Many Factors of Variation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3058 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3059 pages = {473--480},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3060 crossref = {ICML07-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3061 abstract = {Recently, several learning algorithms relying on models with deep architectures have been proposed. Though they have demonstrated impressive performance, to date, they have only been evaluated on relatively simple problems such as digit recognition in a controlled environment, for which many machine learning algorithms already report reasonable results. Here, we present a series of experiments which indicate that these models show promise in solving harder learning problems that exhibit many factors of variation. These models are compared with well-established algorithms such as Support Vector Machines and single hidden-layer feed-forward neural networks.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3062 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3063
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3064 @MASTERSTHESIS{Latendresse-MSc,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3065 author = {Latendresse, Simon},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3066 title = {L'utilisation d'hyper-param{\`{e}}tres pour la selection de variables},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3067 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3068 school = {Universit{\'{e}} de Montreal, Dept. IRO},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3069 note = {(in French)}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3070 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3071
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3072 @MASTERSTHESIS{Lauzon99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3073 author = {Lauzon, Vincent-Philippe},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3074 title = {Mod{\'{e}}les statistiques comme algorithmes d'apprentissage et {MMCC}s; pr{\'{e}}diction de s{\'{e}}ries financi{\`{e}}res},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3075 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3076 school = {D{\'{e}}epartement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3077 crossref = {DIRO}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3078 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3079
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3080 @INPROCEEDINGS{lecun-93,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3081 author = {{LeCun}, Yann and Bengio, Yoshua and Henderson, Donnie and Weisbuch, A. and Weissman, H. and L., Jackel},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3082 title = {On-line handwriting recognition with neural networks: spatial representation versus temporal representation.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3083 booktitle = {Proc. International Conference on handwriting and drawing.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3084 year = {1993},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3085 publisher = {Ecole Nationale Superieure des Telecommunications},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3086 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/lecun-93.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3087 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3088 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3089
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3090 @INPROCEEDINGS{lecun-99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3091 author = {{LeCun}, Yann and Haffner, Patrick and Bottou, {L{\'{e}}on} and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3092 editor = {Forsyth, D.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3093 title = {Object Recognition with Gradient-Based Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3094 booktitle = {Shape, Contour and Grouping in Computer Vision},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3095 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3096 pages = {319-345},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3097 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3098 url = {orig/lecun-99.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3099 topics={PriorKnowledge,Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3100 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3101
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3102 @TECHREPORT{lecun-99b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3103 author = {{LeCun}, Yann and Haffner, Patrick and Bottou, {L{\'{e}}on} and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3104 title = {Gradient-Based Learning for Object Detection, Segmentation and Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3105 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3106 institution = {AT\&T Labs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3107 url = {orig/lecun-99b.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3108 topics={Speech},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3109 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3110
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3111 @INPROCEEDINGS{lecun-bengio-94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3112 author = {{LeCun}, Yann and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3113 title = {Word-level training of a handwritten word recognizer based on convolutional neural networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3114 booktitle = {Proc. of the International Conference on Pattern Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3115 volume = {II},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3116 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3117 pages = {88--92},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3118 publisher = {IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3119 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/icpr-word.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3120 abstract = {We introduce a new approach for on-line recognition of handwritten words written in unconstrained mixed style. Words are represented by low resolution “annotated images” where each pixel contains information about trajectory direction and curvature. The recognizer is a convolution network which can be spatially replicated. From the network output, a hidden {Markov} model produces word scores. The entire system is globally trained to minimize word-level errors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3121 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3122 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3123
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3124 @INPROCEEDINGS{lecun-bengio-95a,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3125 author = {{LeCun}, Yann and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3126 editor = {Arbib, M. A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3127 title = {Convolutional Networks for Images, Speech, and Time-Series},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3128 booktitle = {The Handbook of Brain Theory and Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3129 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3130 pages = {255--257},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3131 publisher = {MIT Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3132 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/handbook-convo.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3133 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3134 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3135
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3136 @INCOLLECTION{lecun-bengio-95b,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3137 author = {{LeCun}, Yann and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3138 editor = {Arbib, M. A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3139 title = {Pattern Recognition and Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3140 booktitle = {The Handbook of Brain Theory and Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3141 year = {1995},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3142 pages = {711--714},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3143 publisher = {MIT Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3144 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/handbook-patrec.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3145 topics={PriorKnowledge,Speech},cat={B},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3146 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3147
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3148 @ARTICLE{LeCun98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3149 author = {{LeCun}, Yann and Bottou, {L{\'{e}}on} and Bengio, Yoshua and Haffner, Patrick},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3150 title = {Gradient-Based Learning Applied to Document Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3151 journal = {Proceedings of the IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3152 volume = {86},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3153 number = {11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3154 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3155 pages = {2278--2324},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3156 abstract = {Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3157 Real-life document recognition systems are composed or multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3158 Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3159 A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3160 topics={PriorKnowledge,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3161 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3162
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3163 @INPROCEEDINGS{Lecun_icassp97,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3164 author = {{LeCun}, Yann and Bottou, {L{\'{e}}on} and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3165 title = {Reading Checks with graph transformer networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3166 booktitle = {International Conference on Acoustics, Speech and Signal Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3167 volume = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3168 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3169 pages = {151--154},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3170 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/lecun-bottou-bengio-97.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3171 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3172 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3173
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3174 @ARTICLE{LeRoux+Bengio-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3175 author = {Le Roux, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3176 title = {Deep Belief Networks are Compact Universal Approximators},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3177 journal = {Neural Computation},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3178 volume = {22},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3179 number = {8},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3180 year = {2010},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3181 pages = {2192-2207},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3182 issn = {0899-7667},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
3183 abstract = {Deep Belief Networks (DBN) are generative models with many layers of hidden causal variables, recently introduced by Hinton et al. (2006), along with a greedy layer-wise unsupervised learning algorithm. Building on Le Roux and Bengio (2008) and Sutskever and Hinton (2008), we show that deep but narrow generative networks do not require more parameters than shallow ones to achieve universal approximation. Exploiting the proof technique, we prove that deep but narrow feed-forward neural networks with sigmoidal units can represent any Boolean expression.}
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3184 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3185
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3186 @TECHREPORT{LeRoux-Bengio-2007-TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3187 author = {Le Roux, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3188 title = {Representational Power of Restricted {B}oltzmann Machines and Deep Belief Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3189 number = {1294},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3190 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3191 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3192 abstract = {Deep Belief Networks (DBN) are generative neural network models with
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3193 many layers of hidden explanatory factors, recently introduced by Hinton et al.,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3194 along with a greedy layer-wise unsupervised learning algorithm. The building
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3195 block of a DBN is a probabilistic model called a Restricted {Boltzmann} Machine
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3196 (RBM), used to represent one layer of the model. Restricted {Boltzmann} Machines
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3197 are interesting because inference is easy in them, and because they have been
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3198 successfully used as building blocks for training deeper models.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3199 We first prove that adding hidden units yields strictly improved modeling
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3200 power, while a second theorem shows that RBMs are universal approximators of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3201 discrete distributions. We then study the question of whether DBNs with more
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3202 layers are strictly more powerful in terms of representational power. This
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3203 suggests a new and less greedy criterion for training RBMs within DBNs.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3204 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3205
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3206 @ARTICLE{LeRoux-Bengio-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3207 author = {Le Roux, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3208 title = {Representational Power of Restricted {B}oltzmann Machines and Deep Belief Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3209 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3210 volume = {20},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3211 number = {6},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3212 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3213 pages = {1631--1649},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3214 abstract = {Deep Belief Networks (DBN) are generative neural network models with many layers of hidden explanatory factors, recently introduced by Hinton et al., along with a greedy layer-wise unsupervised learning algorithm. The building block of a DBN is a probabilistic model called a Restricted {Boltzmann} Machine (RBM), used to represent one layer of the model. Restricted {Boltzmann} Machines are interesting because inference is easy in them, and because they have been successfully used as building blocks for training deeper models. We first prove that adding hidden units yields strictly improved modelling power, while a second theorem shows that RBMs are universal approximators of discrete distributions. We then study the question of whether DBNs with more layers are strictly more powerful in terms of representational power. This suggests a new and less greedy criterion for training RBMs within DBNs.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3215 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3216
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3217 @INPROCEEDINGS{LeRoux-continuous,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3218 author = {Le Roux, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3219 title = {Continuous Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3220 booktitle = {Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3221 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3222 publisher = {Omnipress},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3223 abstract = {This article extends neural networks to the case of an uncountable number of hidden units, in several ways. In the first approach proposed, a finite parametrization is possible, allowing gradient-based learning. While having the same number of parameters as an ordinary neural network, its internal structure suggests that it can represent some smooth functions much more compactly. Under mild assumptions, we also find better error bounds than with ordinary neural networks. Furthermore, this parametrization may help reducing the problem of saturation of the neurons. In a second approach, the input-to-hidden weights arefully non-parametric, yielding a kernel machine for which we demonstrate a simple kernel formula. Interestingly, the resulting kernel machine can be made hyperparameter-free and still generalizes in spite of an absence of explicit regularization.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3224 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3225
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3226 @PHDTHESIS{LeRoux-PhD-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3227 author = {Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3228 title = {Avanc{\'{e}}es th{\'{e}}oriques sur la repr{\'{e}}sentation et l'optimisation des r{\'{e}}seaux de neurones},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3229 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3230 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3231 abstract = {Les r{\'{e}}seaux de neurones artificiels ont {\'{e}}t{\'{e}} abondamment utilis{\'{e}}s dans la communaut{\'{e}} de l'apprentissage machine depuis les ann{\'{e}}es 80. Bien qu'ils aient {\'{e}}t{\'{e}} {\'{e}}tudi{\'{e}}s pour la premi{\`{e}}re fois il y a cinquante ans par Rosenblatt [68], ils ne furent r{\'{e}}ellement populaires qu'apr{\`{e}}s l'apparition de la r{\'{e}}tropropagation du gradient, en 1986 [71].
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3232 En 1989, il a {\'{e}}t{\'{e}} prouv{\'{e}} [44] qu'une classe sp{\'{e}}cifique de r{\'{e}}seaux de neurones (les r{\'{e}}seaux de neurones {\`{a}} une couche cach{\'{e}}e) {\'{e}}tait suffisamment puissante pour pouvoir approximer presque n'importe quelle fonction avec une pr{\'{e}}cision arbitraire : le th{\'{e}}or{\`{e}}me d'approximation universelle. Toutefois, bien que ce th{\'{e}}or{\`{e}}me e{\^{u}}t pour cons{\'{e}}quence un int{\'{e}}r{\^{e}}t accru pour les r{\'{e}}seaux de neurones, il semblerait qu'aucun effort n'ait {\'{e}}t{\'{e}} fait pour profiter de cette propri{\'{e}}t{\'{e}}.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3233 En outre, l'optimisation des r{\'{e}}seaux de neurones {\`{a}} une couche cach{\'{e}}e n'est pas convexe. Cela a d{\'{e}}tourn{\'{e}} une grande partie de la communaut{\'{e}} vers d'autres algorithmes, comme par exemple les machines {\`{a}} noyau (machines {\`{a}} vecteurs de support et r{\'{e}}gression
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3234 {\`{a}} noyau, entre autres).
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3235 La premi{\`{e}}re partie de cette th{\`{e}}se pr{\'{e}}sentera les concepts d'apprentissage machine g{\'{e}}n{\'{e}}raux n{\'{e}}cessaires {\`{a}} la compr{\'{e}}hension des algorithmes utilis{\'{e}}s. La deuxi{\`{e}}me partie se focalisera plus sp{\'{e}}cifiquement sur les m{\'{e}}thodes {\`{a}} noyau et les r{\'{e}}seaux de neurones. La troisi{\`{e}}me partie de ce travail visera ensuite {\`{a}} {\'{e}}tudier les limitations des machines {\`{a}} noyaux et {\`{a}} comprendre les raisons pour lesquelles elles sont inadapt{\'{e}}es {\`{a}} certains probl{\`{e}}mes que nous avons {\`{a}} traiter.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3236 La quatri{\`{e}}me partie pr{\'{e}}sente une technique permettant d'optimiser les r{\'{e}}seaux de neurones {\`{a}} une couche cach{\'{e}}e de mani{\`{e}}re convexe. Bien que cette technique s'av{\`{e}}re difficilement exploitable pour des probl{\`{e}}mes de grande taille, une version approch{\'{e}}e permet d'obtenir une bonne solution dans un temps raisonnable.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3237 La cinqui{\`{e}}me partie se concentre sur les r{\'{e}}seaux de neurones {\`{a}} une couche cach{\'{e}}e infinie. Cela leur permet th{\'{e}}oriquement d'exploiter la propri{\'{e}}t{\'{e}} d'approximation universelle et ainsi d'approcher facilement une plus grande classe de fonctions.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3238 Toutefois, si ces deux variations sur les r{\'{e}}seaux de neurones {\`{a}} une couche cach{\'{e}}e leur conf{\`{e}}rent des propri{\'{e}}t{\'{e}}s int{\'{e}}ressantes, ces derniers ne peuvent extraire plus que des concepts de bas niveau. Les m{\'{e}}thodes {\`{a}} noyau souffrant des m{\^{e}}mes limites, aucun de
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3239 ces deux types d'algorithmes ne peut appr{\'{e}}hender des probl{\`{e}}mes faisant appel {\`{a}} l'apprentissage de concepts de haut niveau.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3240 R{\'{e}}cemment sont apparus les Deep Belief Networks [39] qui sont des r{\'{e}}seaux de neurones {\`{a}} plusieurs couches cach{\'{e}}es entra{\^{\i}}n{\'{e}}s de mani{\`{e}}re efficace. Cette profondeur leur permet d'extraire des concepts de haut niveau et donc de r{\'{e}}aliser des t{\^{a}}ches hors
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3241 de port{\'{e}}e des algorithmes conventionnels. La sixi{\`{e}}me partie {\'{e}}tudie des propri{\'{e}}t{\'{e}}s de ces r{\'{e}}seaux profonds.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3242 Les probl{\`{e}}mes que l'on rencontre actuellement n{\'{e}}cessitent non seulement des algorithmes capables d'extraire des concepts de haut niveau, mais {\'{e}}galement des m{\'{e}}thodes d'optimisation capables de traiter l'immense quantit{\'{e}} de donn{\'{e}}es parfois disponibles, si possible en temps r{\'{e}}el. La septi{\`{e}}me partie est donc la pr{\'{e}}sentation d'une nouvelle technique permettant une optimisation plus rapide.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3243 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3244
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3245 @ARTICLE{lheureux-04,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3246 author = {{L'Heureux}, Pierre-Jean and Carreau, Julie and Bengio, Yoshua and Delalleau, Olivier and Yue, Shi Yi},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3247 title = {Locally Linear Embedding for dimensionality reduction in {QSAR}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3248 journal = {Journal of Computer-Aided Molecular Design},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3249 volume = {18},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3250 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3251 pages = {475--482},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3252 abstract = {Current practice in Quantitative Structure Activity Relationship (QSAR) methods usually involves generating a great number of chemical descriptors and then cutting them back with variable selection techniques. Variable selection is an effective method to reduce the dimensionality but may discard some valuable information. This paper introduces Locally Linear Embedding ({LLE}), a local non-linear dimensionality reduction technique, that can statistically discover a low-dimensional representation of the chemical data. {LLE} is shown to create more stable representations than other non-linear dimensionality
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3253 reduction algorithms, and to be capable of capturing non-linearity in chemical data.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3254 topics={Bioinformatic},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3255 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3256
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3257 @TECHREPORT{lm-TR00,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3258 author = {Bengio, Yoshua and Ducharme, R{\'{e}}jean and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3259 title = {A Neural Probabilistic Language Model},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3260 number = {1178},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3261 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3262 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3263 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1178.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3264 abstract = {A goal of statistical language modeling is to learn the joint probability function of sequences of words in a language. This is intrinsically difficult because of the curse of dimensionality: a word sequence on which the model will be tested is likely to be different from all the word sequences seen during training. Traditional but very successful approaches based on n-grams obtain generalization by concatenating very short overlapping sequences seen in the training set. We propose to fight the curse of dimensionality by learning a distributed representation for words which allows each training sentence to inform the model about an exponential number of semantically neighboring sentences. The model learns simultaneously (1) a distributed representation for each word along with (2) the probability function for word sequences, expressed in terms of these representations. Generalization is obtained because a sequence of words that has never been seen before gets high probability if it is made or words that are similar (in the sense of having a nearby representation) to words forming an already seen sentence. Training such large models (with millions of parameters) within a reasonable time is itself a significant challenge. We report on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach very significantly improves on a state-of-the-art trigram model, and that the proposed approach allows to take advantage of much longer contexts.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3265 topics={Markov,Unsupervised,Language},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3266 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3267
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3268 @INPROCEEDINGS{Maillet+al-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3269 author = {Maillet, Fran{\c c}ois and Eck, Douglas and Desjardins, Guillaume and Lamere, Paul},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3270 title = {Steerable Playlist Generation by Learning Song Similarity from Radio Station Playlists},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3271 booktitle = {Proceedings of the 10th International Conference on Music Information Retrieval},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3272 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3273 url = {http://www-etud.iro.umontreal.ca/~mailletf/papers/ismir09-playlist.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3274 abstract = {This paper presents an approach to generating steerable playlists. We first demonstrate a method for learning song transition probabilities from audio features extracted from songs played in professional radio station playlists. We then show that by using this learnt similarity function as a prior, we are able to generate steerable playlists by choosing the next song to play not simply based on that prior, but on a tag cloud that the user is able to manipulate to express the high-level characteristics of the music he wishes Last.fm, to listen to.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3275 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3276
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3277 @INPROCEEDINGS{manzagol+bertinmahieux+eck:ismir2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3278 author = {Manzagol, Pierre-Antoine and Bertin-Mahieux, Thierry and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3279 title = {On the Use of Sparse Time-Relative Auditory Codes for Music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3280 booktitle = {{Proceedings of the 9th International Conference on Music Information Retrieval ({ISMIR} 2008)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3281 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3282 abstract = {Many if not most audio features used in MIR research are inspired by work done in speech recognition and are variations on the spectrogram. Recently, much attention has been given to new representations of audio that are sparse and time-relative. These representations are efficient and able to avoid the time-frequency trade-off of a spectrogram. Yet little work with music streams has been conducted and these features remain mostly unused in the MIR community. In this paper we further explore the use of these features for musical signals. In particular, we investigate their use on realistic music examples (i.e. released commercial music) and their use as input features for supervised learning. Furthermore, we identify three specific issues related to these features which will need to be further addressed in order to obtain the full benefit for MIR applications.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3283 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3284 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3285 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3286
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3287 @MASTERSTHESIS{Manzagol-Msc-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3288 author = {Manzagol, Pierre-Antoine},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3289 key = {Algorithme d'apprentissage, méthode de second ordre, gradient naturel, approximation stochastique},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3290 title = {TONGA - Un algorithme de gradient naturel pour les probl{\`{e}}mes de grande taille},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3291 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3292 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3293 abstract = {Les syst{\`{e}}mes adaptatifs sont confront{\'{e}}s {\`{a}} des donn{\'{e}}es qui {\'{e}}voluent rapidement en quantit{\'{e}} et en complexit{\'{e}}. Les avanc{\'{e}}es mat{\'{e}}rielles de l'informatique ne susent pas {\`{a}} compenser cet essor. Une mise {\`{a}} l'{\'{e}}chelle des techniques d'apprentissage est n{\'{e}}cessaire. D'une part, les mod{\`{e}}les doivent gagner en capacit{\'{e}} de repr{\'{e}}sentation. De l'autre, les algorithmes d'apprentissage doivent devenir plus ecaces.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3294 Nos travaux se situent dans ce contexte des probl{\`{e}}mes de grande taille et portent sur l'am{\'{e}}lioration des algorithmes d'apprentissage. Deux {\'{e}}l{\'{e}}ments de r{\'{e}}ponse sont d{\'{e}}j{\`{a}} connus. Il s'agit des m{\'{e}}thodes de second ordre et de l'approximation stochastique. Or, les m{\'{e}}thodes de second ordre poss{\`{e}}dent des complexit{\'{e}}s en calculs et en m{\'{e}}moire qui sont prohibitives dans le cadre des probl{\`{e}}mes de grande taille. {\'{E}}galement, il est notoirement dicile de concilier ces m{\'{e}}thodes avec l'approximation stochastique. TONGA est un algorithme d'apprentissage con{\c c}u pour faire face {\`{a}} ces dicult{\'{e}}s. Il s'agit d'une implantation stochastique et adapt{\'{e}}e aux probl{\`{e}}mes de grande taille d'une m{\'{e}}thode de second ordre, le gradient naturel. Dans ce m{\'{e}}moire, nous examinons de pr{\`{e}}s ce nouvel algorithme d'apprentissage en le comparant sur plusieurs probl{\`{e}}mes au gradient stochastique, la technique d'optimisation commun{\'{e}}ment utilis{\'{e}}e dans le cadre des probl{\`{e}}mes de grande taille. Nos exp{\'{e}}riences montrent que TONGA est au moins tout aussi ecace que le gradient stochastique, ce qui est un accomplissement en soit. Dans certains cas, TONGA offre une convergence nettement sup{\'{e}}rieure {\`{a}} celle du gradient stochastique.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3295 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3296
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3297 @INPROCEEDINGS{matic-94,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3298 author = {Matic, N. and Henderson, Donnie and {LeCun}, Yann and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3299 title = {Pen-based visitor registration system (PENGUIN)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3300 booktitle = {Conference Record of the Twenty-Eighth Asilomar Conference on Signals, Systems and Computers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3301 year = {1994},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3302 publisher = {IEEE},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3303 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/matic-94.tiff},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3304 abstract = {We describe a new electronic pen-based visitors registration system (PENGUIN) whose goal is to expand and modernize the visitor sign-in procedure at Bell Laboratories. The system uses a pen-interface (i.e. tablet-display) in what is essentially a form filling application. Our pen-interface is coupled with a powerful and accurate on-line handwriting recognition module. A database of AT&T employees (the visitors' hosts) and country names is used to check the recognition module outputs, in order to find the best match. The system provides assistance to the guard at one of the guard stations in routing visitors to their hosts. All the entered data are stored electronically. Initial testing shows that PENGUIN system performs reliably and with high accuracy. It retrieves the correct host name with 97\% accuracy and the correct visitors citizenship with 99\% accuracy. The system is robust and easy to use for both visitors and guards},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3305 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3306 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3307
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3308 @UNPUBLISHED{mirex2005artist,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3309 author = {Bergstra, James and Casagrande, Norman and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3310 title = {Artist Recognition: A Timbre- and Rhythm-Based Multiresolution Approach},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3311 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3312 note = {{MIREX} artist recognition contest},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3313 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3314 sourcetype={Other},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3315 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3316
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3317 @UNPUBLISHED{mirex2005genre,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3318 author = {Bergstra, James and Casagrande, Norman and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3319 title = {Genre Classification: Timbre- and Rhythm-Based Multiresolution Audio Classification},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3320 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3321 note = {{MIREX} genre classification contest},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3322 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3323 sourcetype={Other},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3324 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3325
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3326 @UNPUBLISHED{mirex2005note,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3327 author = {Lacoste, Alexandre and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3328 title = {Onset Detection with Artificial Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3329 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3330 note = {{MIREX} note onset detection contest},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3331 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3332 sourcetype={Other},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3333 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3334
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3335 @UNPUBLISHED{mirex2005tempo,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3336 author = {Eck, Douglas and Casagrande, Norman},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3337 title = {A Tempo-Extraction Algorithm Using an Autocorrelation Phase Matrix and Shannon Entropy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3338 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3339 note = {{MIREX} tempo extraction contest (www.music-ir.org/\-evaluation/\-mirex-results)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3340 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3341 sourcetype={Other},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3342 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3343
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3344 @INPROCEEDINGS{mitacs-insurance01,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3345 author = {Bengio, Yoshua and Chapados, Nicolas and Dugas, Charles and Ghosn, Joumana and Takeuchi, Ichiro and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3346 title = {High-Dimensional Data Inference for Automobile Insurance Premia Estimation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3347 booktitle = {Presented at the 2001 MITACS Annual Meeting},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3348 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3349 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/mitacs_insurance.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3350 topics={HighDimensional,Mining},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3351 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3352
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3353 @INPROCEEDINGS{Morin+al-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3354 author = {Morin, Frederic and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3355 editor = {Cowell, Robert G. and Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3356 title = {Hierarchical Probabilistic Neural Network Language Model},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3357 booktitle = {Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3358 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3359 pages = {246--252},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3360 publisher = {Society for Artificial Intelligence and Statistics},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3361 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/hierarchical-nnlm-aistats05.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3362 abstract = {In recent years, variants of a neural network architecture for statistical language modeling have been proposed and successfully applied, e.g. in the language modeling component of speech recognizers. The main advantage of these architectures is that they learn an embedding for words (or other symbols) in a continuous space that helps to smooth the language model and provide good generalization even when the number of training examples is insufficient. However, these models are extremely slow in comparison to the more commonly used n-gram models, both for training and recognition. As an alternative to an importance sampling method proposed to speed-up training, we introduce a hierarchical decomposition of the conditional probabilities that yields a speed-up of about 200 both during training and recognition. The hierarchical decomposition is a binary hierarchical clustering constrained by the prior knowledge extracted from the WordNet semantic hierarchy.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3363 topics={Language},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3364 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3365
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3366 @TECHREPORT{Nadeau-inference-TR99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3367 author = {Nadeau, Claude and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3368 title = {Inference and the Generalization Error},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3369 number = {99s-45},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3370 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3371 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3372 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/techrep.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3373 abstract = {We perform a theoretical investigation of the variance of the cross-validation estimate of the generalization error that takes into account the variability due to the choice of training sets and test examples. This allows us to propose two new estimators of this variance. We show, via simulations, that these new statistics perform well relative to the statistics considered in (Dietterich, 1998). In particular, tests of hypothesis based on these don’t tend to be too liberal like other tests currently available, and have good power.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3374 topics={Comparative},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3375 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3376
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3377 @INPROCEEDINGS{nadeau:2000:nips,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3378 author = {Nadeau, Claude and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3379 title = {Inference for the Generalization Error},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3380 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3381 pages = {307--313},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3382 crossref = {NIPS12-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3383 abstract = {In order to to compare learning algorithms, experimental results reported in the machine learning litterature often use statistical tests of significance. Unfortunately, most of these tests do not take into account the variability due to the choice of training set. We perform a theoretical investigation of the variance of the cross-validation estimate of the generalization error that takes into account the variability due to the choice of training sets. This allows us to propose two new ways to estimate this variance. We show, via simulations, that these new statistics perform well relative to the statistics considered by Dietterich (Dietterich, 1998).},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3384 topics={Comparative},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3385 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3386
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3387 @ARTICLE{nadeau:2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3388 author = {Nadeau, Claude and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3389 title = {Inference for the Generalization Error},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3390 journal = {Machine Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3391 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3392 abstract = {In order to compare learning algorithms, experimental results reported in the machine learning literature often use statistical tests of significance to support the claim that a new learning algorithm generalizes better. Such tests should take into account the variability due to the choice of training set and not only that due to the test examples, as is often the case. This could lead to gross underestimation of the variance of the cross-validation estimator, and to the wrong conclusion that the new algorithm is significantly better when it is not. We perform a theoretical investigation of the variance of a cross-validation estimator of the generalization error that takes into account the variability due to the randomness of the training set as well as test examples. Our analysis shows that all the variance estimators that are based only on the results of the cross-validation experiment must be biased. This analysis allows us to propose new estimators of this variance. We show, via simulations, that tests of hypothesis about the generalization error using those new variance estimators have better properties than tests involving variance estimators currently in use and listed in (Dietterich, 1998). In particular, the new tests have correct size and good power. That is, the new tests do not reject the null hypothesis too often when the hypothesis is true, but they tend to frequently reject the null hypothesis when the latter is false.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3393 topics={Comparative},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3394 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3395
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3396 @ARTICLE{NC06,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3397 author = {Bengio, Yoshua and Monperrus, Martin and Larochelle, Hugo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3398 title = {Nonlocal Estimation of Manifold Structure},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3399 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3400 volume = {18},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3401 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3402 pages = {2509--2528},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3403 abstract = {We claim and present arguments to the effect that a large class of manifold
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3404 learning algorithms that are essentially local and can be framed as
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3405 kernel learning algorithms will suffer from the curse of dimensionality, at
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3406 the dimension of the true underlying manifold. This observation suggests
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3407 to explore non-local manifold learning algorithms which attempt to discover
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3408 shared structure in the tangent planes at different positions. A criterion for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3409 such an algorithm is proposed and experiments estimating a tangent plane
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3410 prediction function are presented, showing its advantages with respect to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3411 local manifold learning algorithms: it is able to generalize very far from
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3412 training data (on learning handwritten character image rotations), where a
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3413 local non-parametric method fails.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3414 topics={HighDimensional,Kernel,Unsupervised},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3415 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3416
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3417 @INPROCEEDINGS{NIPS1-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3418 editor = {Touretzky, D. S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3419 title = {Advances in Neural Information Processing Systems 1 (NIPS'88)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3420 booktitle = {NIPS 1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3421 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3422 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3423 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3424
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3425
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3426 @INPROCEEDINGS{NIPS10-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3427 editor = {Jordan, M.I. and Kearns, M.J. and Solla, S.A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3428 title = {Advances in Neural Information Processing Systems 10 (NIPS'97)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3429 booktitle = {NIPS 10},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3430 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3431 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3432 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3433
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3434
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3435 @INPROCEEDINGS{NIPS11,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3436 editor = {Kearns, M.J. and Solla, S.A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3437 title = {Advances in Neural Information Processing Systems 11 (NIPS'98)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3438 booktitle = {Advances in Neural Information Processing Systems 11 (NIPS'98)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3439 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3440 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3441 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3442
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3443 @INPROCEEDINGS{NIPS11-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3444 editor = {Kearns, M.J. and Solla, S.A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3445 title = {Advances in Neural Information Processing Systems 11 (NIPS'98)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3446 booktitle = {NIPS 11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3447 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3448 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3449 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3450
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3451
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3452 @INPROCEEDINGS{NIPS12-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3453 editor = {Solla, S.A. and Leen, T. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3454 title = {Advances in Neural Information Processing Systems 12 (NIPS'99)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3455 booktitle = {NIPS 12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3456 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3457 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3458 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3459
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3460
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3461 @INPROCEEDINGS{NIPS13-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3462 editor = {Leen, T. K. and Dietterich, T.G.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3463 title = {Advances in Neural Information Processing Systems 13 (NIPS'00)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3464 booktitle = {NIPS 13},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3465 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3466 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3467 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3468
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3469
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3470 @INPROCEEDINGS{NIPS14,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3471 editor = {Dietterich, T.G. and Becker, S. and Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3472 title = {Advances in Neural Information Processing Systems 14 (NIPS'01)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3473 booktitle = {Advances in Neural Information Processing Systems 14 (NIPS'01)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3474 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3475 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3476 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3477
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3478 @INPROCEEDINGS{NIPS14-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3479 editor = {Dietterich, T.G. and Becker, S. and Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3480 title = {Advances in Neural Information Processing Systems 14 (NIPS'01)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3481 booktitle = {NIPS 14},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3482 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3483 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3484 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3485
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3486
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3487 @INPROCEEDINGS{NIPS15-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3488 editor = {Becker, S. and Thrun, Sebastian},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3489 title = {Advances in Neural Information Processing Systems 15 (NIPS'02)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3490 booktitle = {NIPS 15},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3491 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3492 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3493 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3494
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3495
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3496 @INPROCEEDINGS{NIPS16-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3497 editor = {Becker, S. and Saul, L. and {Sch{\"{o}}lkopf}, Bernhard},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3498 title = {Advances in Neural Information Processing Systems 16 (NIPS'03)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3499 booktitle = {NIPS 16},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3500 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3501 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3502
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3503
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3504 @INPROCEEDINGS{NIPS17-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3505 editor = {Saul, Lawrence K. and Weiss, Yair and Bottou, {L{\'{e}}on}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3506 title = {Advances in Neural Information Processing Systems 17 (NIPS'04)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3507 booktitle = {NIPS 17},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3508 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3509 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3510
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3511
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3512 @INPROCEEDINGS{NIPS18-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3513 editor = {Weiss, Yair and {Sch{\"{o}}lkopf}, Bernhard and Platt, John},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3514 title = {Advances in Neural Information Processing Systems 18 (NIPS'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3515 booktitle = {NIPS 18},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3516 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3517 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3518 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3519
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3520
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3521 @INPROCEEDINGS{NIPS19-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3522 editor = {{Sch{\"{o}}lkopf}, Bernhard and Platt, John and Hoffman, Thomas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3523 title = {Advances in Neural Information Processing Systems 19 (NIPS'06)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3524 booktitle = {NIPS 19},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3525 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3526 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3527 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3528
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3529
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3530 @INPROCEEDINGS{NIPS2-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3531 editor = {Touretzky, D. S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3532 title = {Advances in Neural Information Processing Systems 2 (NIPS'89)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3533 booktitle = {NIPS 2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3534 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3535 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3536 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3537
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3538
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3539 @INPROCEEDINGS{NIPS20-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3540 editor = {Platt, John and Koller, D. and Singer, Yoram and Roweis, S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3541 title = {Advances in Neural Information Processing Systems 20 (NIPS'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3542 booktitle = {NIPS 20},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3543 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3544 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3545 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3546
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3547
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3548 @INPROCEEDINGS{NIPS2003_AA65,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3549 author = {Bengio, Yoshua and Grandvalet, Yves},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3550 keywords = {cross validation, error bars, generalization error inference, k-fold cross-validation, model selection, statistical comparison of algorithms, variance estimate},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3551 title = {No Unbiased Estimator of the Variance of K-Fold Cross-Validation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3552 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3553 publisher = {MIT Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3554 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/var-kfold-part1-nips.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3555 crossref = {NIPS16},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3556 abstract = {Most machine learning researchers perform quantitative experiments to estimate generalization error and compare algorithm performances. In order to draw statistically convincing conclusions, it is important to estimate the uncertainty of such estimates. This paper studies the estimation of uncertainty around the K-fold cross-validation estimator. The main theorem shows that there exists no universal unbiased estimator of the variance of K-fold cross-validation. An analysis based on the eigendecomposition of the covariance matrix of errors helps to better understand the nature of the problem and shows that naive estimators may grossly underestimate variance, as confirmed by numerical experiments.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3557 topics={Comparative},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3558 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3559
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3560 @INCOLLECTION{NIPS2005_424,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3561 author = {Bengio, Yoshua and Delalleau, Olivier and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3562 title = {The Curse of Highly Variable Functions for Local Kernel Machines},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3563 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3564 pages = {107--114},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3565 crossref = {NIPS18-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3566 abstract = {We present a series of theoretical arguments supporting the claim that a
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3567 large class of modern learning algorithms that rely solely on the smoothness
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3568 prior – with similarity between examples expressed with a local
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3569 kernel – are sensitive to the curse of dimensionality, or more precisely
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3570 to the variability of the target. Our discussion covers supervised, semisupervised
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3571 and unsupervised learning algorithms. These algorithms are
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3572 found to be local in the sense that crucial properties of the learned function
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3573 at x depend mostly on the neighbors of x in the training set. This
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3574 makes them sensitive to the curse of dimensionality, well studied for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3575 classical non-parametric statistical learning. We show in the case of the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3576 Gaussian kernel that when the function to be learned has many variations,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3577 these algorithms require a number of training examples proportional to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3578 the number of variations, which could be large even though there may exist
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3579 short descriptions of the target function, i.e. their Kolmogorov complexity
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3580 may be low. This suggests that there exist non-local learning
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3581 algorithms that at least have the potential to learn about such structured
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3582 but apparently complex functions (because locally they have many variations),
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3583 while not using very specific prior domain knowledge.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3584 topics={HighDimensional,Kernel,Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3585 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3586
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3587 @INPROCEEDINGS{NIPS2005_456,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3588 author = {K{\'{e}}gl, Bal{\'{a}}zs and Wang, Ligen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3589 title = {Boosting on Manifolds: Adaptive Regularization of Base Classifiers},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3590 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3591 pages = {665--672},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3592 crossref = {NIPS17-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3593 abstract = {In this paper we propose to combine two powerful ideas, boosting and manifold learning. On the one hand, we improve ADABOOST by incorporating knowledge on the structure of the data into base classifier design and selection. On the other hand, we use ADABOOST’s efficient learning mechanism to significantly improve supervised and semi-supervised algorithms proposed in the context of manifold learning. Beside the specific manifold-based penalization, the resulting algorithm also accommodates the boosting of a large family of regularized learning algorithms.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3594 topics={Boosting},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3595 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3596
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3597 @INCOLLECTION{NIPS2005_519,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3598 author = {Grandvalet, Yves and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3599 title = {Semi-supervised Learning by Entropy Minimization},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3600 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3601 pages = {529--236},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3602 crossref = {NIPS17-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3603 abstract = {We consider the semi-supervised learning problem, where a decision rule is to be learned from labeled and unlabeled data. In this framework, we motivate minimum entropy regularization, which enables to incorporate unlabeled data in the standard supervised learning. Our approach includes other approaches to the semi-supervised problem as particular or limiting cases. A series of experiments illustrates that the proposed solution benefits from unlabeled data. The method challenges mixture models when the data are sampled from the distribution class spanned by the generative model. The performances are definitely in favor of minimum entropy regularization when generative models are misspecified, and the weighting of unlabeled data provides robustness to the violation of the “cluster assumption”. Finally, we also illustrate that the method can also be far superior to manifold learning in high dimension spaces.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3604 topics={Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3605 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3606
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3607 @INPROCEEDINGS{NIPS2005_539,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3608 author = {Bengio, Yoshua and Larochelle, Hugo and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3609 title = {Non-Local Manifold Parzen Windows},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3610 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3611 crossref = {NIPS18-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3612 abstract = {To escape from the curse of dimensionality, we claim that one can learn
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3613 non-local functions, in the sense that the value and shape of the learned
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3614 function at x must be inferred using examples that may be far from x.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3615 With this objective, we present a non-local non-parametric density estimator.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3616 It builds upon previously proposed Gaussian mixture models with
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3617 regularized covariance matrices to take into account the local shape of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3618 the manifold. It also builds upon recent work on non-local estimators of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3619 the tangent plane of a manifold, which are able to generalize in places
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3620 with little training data, unlike traditional, local, non-parametric models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3621 topics={HighDimensional,Kernel,Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3622 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3623
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3624 @INPROCEEDINGS{NIPS2005_583,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3625 author = {Bengio, Yoshua and Le Roux, Nicolas and Vincent, Pascal and Delalleau, Olivier and Marcotte, Patrice},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3626 title = {Convex Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3627 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3628 pages = {123--130},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3629 crossref = {NIPS18-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3630 abstract = {Convexity has recently received a lot of attention in the machine learning
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3631 community, and the lack of convexity has been seen as a major disadvantage
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3632 of many learning algorithms, such as multi-layer artificial neural
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3633 networks. We show that training multi-layer neural networks in which the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3634 number of hidden units is learned can be viewed as a convex optimization
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3635 problem. This problem involves an infinite number of variables, but can be
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3636 solved by incrementally inserting a hidden unit at a time, each time finding
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3637 a linear classifier that minimizes a weighted sum of errors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3638 topics={Boosting},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3639 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3640
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3641 @INPROCEEDINGS{NIPS2005_663,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3642 author = {Rivest, Fran{\c c}ois and Bengio, Yoshua and Kalaska, John},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3643 title = {Brain Inspired Reinforcement Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3644 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3645 pages = {1129--1136},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3646 crossref = {NIPS17-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3647 abstract = {Successful application of reinforcement learning algorithms often involves considerable hand-crafting of the necessary non-linear features to reduce the complexity of the value functions and hence to promote convergence of the algorithm. In contrast, the human brain readily and autonomously finds the complex features when provided with sufficient training. Recent work in machine learning and neurophysiology has demonstrated the role of the basal ganglia and the frontal cortex in mammalian reinforcement learning. This paper develops and explores new reinforcement learning algorithms inspired by neurological evidence that provides potential new approaches to the feature construction problem. The algorithms are compared and evaluated on the Acrobot task.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3648 topics={BioRules},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3649 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3650
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3651 @INCOLLECTION{NIPS2005_691,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3652 author = {Bengio, Yoshua and Monperrus, Martin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3653 title = {Non-Local Manifold Tangent Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3654 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3655 pages = {129--136},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3656 crossref = {NIPS17-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3657 abstract = {We claim and present arguments to the effect that a large class of manifold learning algorithms that are essentially local and can be framed as kernel learning algorithms will suffer from the curse of dimensionality, at the dimension of the true underlying manifold. This observation suggests to explore non-local manifold learning algorithms which attempt to discover shared structure in the tangent planes at different positions. A criterion for such an algorithm is proposed and experiments estimating a tangent plane prediction function are presented, showing its advantages with respect to local manifold learning algorithms: it is able to generalize very far from training data (on learning handwritten character image rotations), where a local non-parametric method fails.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3658 topics={HighDimensional,Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3659 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3660
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3661 @INPROCEEDINGS{NIPS2005_874,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3662 author = {K{\'{e}}gl, Bal{\'{a}}zs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3663 title = {Generalization Error and Algorithmic Convergence of Median Boosting},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3664 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3665 pages = {657--664},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3666 crossref = {NIPS17-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3667 abstract = {We have recently proposed an extension of ADABOOST to regression that uses the median of the base regressors as the final regressor. In this paper we extend theoretical results obtained for ADABOOST to median boosting and to its localized variant. First, we extend recent results on efficient margin maximizing to show that the algorithm can converge to the maximum achievable margin within a preset precision in a finite number of steps. Then we provide confidence-interval-type bounds on the generalization error.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3668 topics={Boosting},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3669 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3670
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3671 @INPROCEEDINGS{NIPS2007-56,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3672 author = {Le Roux, Nicolas and Manzagol, Pierre-Antoine and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3673 title = {Topmoumoute online natural gradient algorithm},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3674 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3675 crossref = {NIPS20-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3676 abstract = {Guided by the goal of obtaining an optimization algorithm that is both fast and yielding good generalization, we study the descent direction maximizing the decrease in generalization error or the probability of not increasing generalization error. The surprising result is that from both the Bayesian and frequentist perspectives this can yield the natural gradient direction. Although that direction can be very expensive to compute we develop an efficient, general, online approximation to the natural gradient descent which is suited to large scale problems. We report experimental results showing much faster convergence in computation time and in number of iterations with TONGA (Topmoumoute Online natural Gradient Algorithm) than with stochastic gradient descent, even on very large datasets.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3677 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3678
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3679 @INPROCEEDINGS{NIPS2007-812,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3680 author = {Chapados, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3681 title = {Augmented Functional Time Series Representation and Forecasting with Gaussian Processes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3682 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3683 pages = {265--272},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3684 crossref = {NIPS20-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3685 abstract = {We introduce a functional representation of time series which allows forecasts to be performed over an unspecified horizon with progressively-revealed information sets. By virtue of using Gaussian processes, a complete covariance matrix between forecasts at several time-steps is available. This information is put to use in an application to actively trade price spreads between commodity futures contracts. The approach delivers impressive out-of-sample risk-adjusted returns after transaction costs on a portfolio of 30 spreads.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3686 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3687
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3688 @INPROCEEDINGS{NIPS2007-925,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3689 author = {Le Roux, Nicolas and Bengio, Yoshua and Lamblin, Pascal and Joliveau, Marc and K{\'{e}}gl, Bal{\'{a}}zs},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3690 title = {Learning the 2-D Topology of Images},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3691 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3692 pages = {841--848},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3693 crossref = {NIPS20-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3694 abstract = {We study the following question: is the two-dimensional structure of images a very strong prior or is it something that can be learned with a few examples of natural images? If someone gave us a learning task involving images for which the two-dimensional topology of pixels was not known, could we discover it automatically and exploit it? For example suppose that the pixels had been permuted in a fixed but unknown way, could we recover the relative two-dimensional location of pixels on images? The surprising result presented here is that not only the answer is yes but that about as few as a thousand images are enough to approximately recover the relative locations of about a thousand pixels. This is achieved using a manifold learning algorithm applied to pixels associated with a measure of distributional similarity between pixel intensities. We compare different topologyextraction approaches and show how having the two-dimensional topology can be exploited.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3695 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3696
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3697 @INPROCEEDINGS{NIPS21,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3698 editor = {Koller, D. and Schuurmans, Dale and Bengio, Yoshua and Bottou, {L{\'{e}}on}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3699 title = {Advances in Neural Information Processing Systems 21 (NIPS'08)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3700 booktitle = {Advances in Neural Information Processing Systems 21 (NIPS'08)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3701 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3702 publisher = {Nips Foundation (http://books.nips.cc)}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3703 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3704
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3705 @INPROCEEDINGS{NIPS21-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3706 editor = {Koller, D. and Schuurmans, Dale and Bengio, Yoshua and Bottou, {L{\'{e}}on}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3707 title = {Advances in Neural Information Processing Systems 21 (NIPS'08)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3708 booktitle = {NIPS 21},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3709 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3710 publisher = {Nips Foundation (http://books.nips.cc)}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3711 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3712
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3713
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3714 @INPROCEEDINGS{NIPS22-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3715 editor = {Bengio, Yoshua and Schuurmans, Dale and Williams, Christopher and Lafferty, John and Culotta, Aron},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3716 title = {Advances in Neural Information Processing Systems 22 (NIPS'09)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3717 booktitle = {NIPS 22},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3718 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3719 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3720
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3721
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3722 @INPROCEEDINGS{NIPS3,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3723 editor = {Lipmann, R. P. and Moody, J. E. and Touretzky, D. S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3724 title = {Advances in Neural Information Processing Systems 3 (NIPS'90)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3725 booktitle = {Advances in Neural Information Processing Systems 3 (NIPS'90)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3726 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3727 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3728 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3729
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3730 @INPROCEEDINGS{NIPS3-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3731 editor = {Lipmann, R. P. and Moody, J. E. and Touretzky, D. S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3732 title = {Advances in Neural Information Processing Systems 3 (NIPS'90)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3733 booktitle = {NIPS 3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3734 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3735 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3736 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3737
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3738
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3739 @INPROCEEDINGS{NIPS4-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3740 editor = {Moody, J. E. and Hanson, S. J. and Lipmann, R. P.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3741 title = {Advances in Neural Information Processing Systems 4 (NIPS'91)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3742 booktitle = {NIPS 4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3743 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3744 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3745 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3746
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3747
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3748 @INPROCEEDINGS{NIPS5,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3749 editor = {Giles, C.L. and Hanson, S. J. and Cowan, J. D.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3750 title = {Advances in Neural Information Processing Systems 5 (NIPS'92)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3751 booktitle = {Advances in Neural Information Processing Systems 5 (NIPS'92)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3752 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3753 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3754 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3755
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3756 @INPROCEEDINGS{NIPS5-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3757 editor = {Giles, C.L. and Hanson, S. J. and Cowan, J. D.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3758 title = {Advances in Neural Information Processing Systems 5 (NIPS'92)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3759 booktitle = {NIPS 5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3760 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3761 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3762 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3763
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3764
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3765 @INPROCEEDINGS{NIPS6-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3766 editor = {Cowan, J. D. and Tesauro, G. and Alspector, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3767 title = {Advances in Neural Information Processing Systems 6 (NIPS'93)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3768 booktitle = {NIPS 6},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3769 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3770 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3771 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3772
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3773
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3774 @INPROCEEDINGS{NIPS7-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3775 editor = {Tesauro, G. and Touretzky, D. S. and Leen, T. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3776 title = {Advances in Neural Information Processing Systems 7 (NIPS'94)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3777 booktitle = {NIPS 7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3778 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3779 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3780 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3781
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3782
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3783 @INPROCEEDINGS{NIPS8-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3784 editor = {Touretzky, D. S. and Mozer, M. and Hasselmo, M.E.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3785 title = {Advances in Neural Information Processing Systems 8 (NIPS'95)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3786 booktitle = {NIPS 8},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3787 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3788 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3789 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3790
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3791
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3792 @INPROCEEDINGS{NIPS9-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3793 editor = {Mozer, M. and Jordan, M.I. and Petsche, T.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3794 title = {Advances in Neural Information Processing Systems 9 (NIPS'96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3795 booktitle = {NIPS 9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3796 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3797 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3798 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3799
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3800
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3801 @INPROCEEDINGS{nnlm:2001:nips,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3802 author = {Bengio, Yoshua and Ducharme, R{\'{e}}jean and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3803 title = {A Neural Probabilistic Language Model},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3804 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3805 crossref = {NIPS13-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3806 abstract = {A goal of statistical language modeling is to learn the joint probability function of sequences of words. This is intrinsically difficult because of the curse of dimensionality: we propose to fight it with its own weapons. In the proposed approach one learns simultaneously (1) a distributed representation for each word (i.e. a similarity between words) along with (2) the probability function for word sequences, expressed with these representations. Generalization is obtained because a sequence of words that
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3807 has never been seen before gets high probability if it is made of words that are similar to words forming an already seen sentence. We report on experiments using neural networks for the probability function, showing on two text corpora that the proposed approach very significantly improves on a state-of-the-art trigram model.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3808 topics={Markov,Unsupervised,Language},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3809 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3810
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3811 @INPROCEEDINGS{nsvn:2000:ijcnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3812 author = {Vincent, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3813 title = {A Neural Support Vector Network Architecture with Adaptive Kernels},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3814 booktitle = {International Joint Conference on Neural Networks 2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3815 volume = {V},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3816 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3817 pages = {187--192},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3818 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/nsvn.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3819 abstract = {In the Support Vector Machines ({SVM}) framework, the positive-definite kernel can be seen as representing a fixed similarity measure between two patterns, and a discriminant function is obtained by taking a linear combination of the kernels computed at training examples called support vectors. Here we investigate learning architectures in which the kernel functions can be replaced by more general similarity measures that can have arbitrary internal parameters. The training criterion used in {SVM}s is not appropriate for this purpose so we adopt the simple criterion that is generally used when training neural networks for classification tasks. Several experiments are performed which show that such Neural Support Vector Networks perform similarly to {SVM}s while requiring significantly fewer support vectors, even when the similarity measure has no internal parameters.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3820 topics={Kernel},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3821 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3822
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3823 @INPROCEEDINGS{Ouimet+al-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3824 author = {Ouimet, Marie and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3825 editor = {Cowell, Robert G. and Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3826 title = {Greedy Spectral Embedding},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3827 booktitle = {Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3828 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3829 pages = {253--260},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3830 publisher = {Society for Artificial Intelligence and Statistics},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3831 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/greedy-kernel-aistats05.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3832 abstract = {Spectral dimensionality reduction methods and spectral clustering methods require computation of the principal eigenvectors of an n X n matrix where n is the number of examples. Following up on previously proposed techniques to speed-up kernel methods by focusing on a subset of m examples, we study a greedy selection procedure for this subset, based on the feature space distance between a candidate example and the span of the previously chosen ones. In the case of kernel {PCA} or spectral clustering this reduces computation to O(m^2 n). For the same computational complexity, we can also compute the feature space projection of the non-selected examples on the subspace spanned by the selected examples, to estimate the embedding function based on all the data, which yields considerably better estimation of the embedding function. This algorithm can be formulated in an online setting and we can bound the error on the approximation of the Gram matrix.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3833 topics={HighDimensional,kenel},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3834 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3835
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3836 @MASTERSTHESIS{Ouimet-Msc-2004,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3837 author = {Ouimet, Marie},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3838 keywords = {algorithmes voraces., apprentissage non-supervis{\'{e}}, m{\'{e}}thodes spectrales, noyaux, r{\'{e}}duction de dimensionnalit{\'{e}}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3839 title = {R{\'{e}}duction de dimensionnalit{\'{e}} non lin{\'{e}}aire et vorace},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3840 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3841 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3842 abstract = {Les m{\'{e}}thodes spectrales de r{\'{e}}duction de dimensionnalit{\'{e}} et les m{\'{e}}thodes de segmentation spectrale exigent le calcul des vecteurs propres principaux d'une matrice de taille n x n o{\`{u}} n est le nombre d'exemples. Des techniques ont {\'{e}}t{\'{e}} propos{\'{e}}es dans la litt{\'{e}}rature pour acc{\'{e}}l{\'{e}}rer les m{\'{e}}thodes {\`{a}} noyau en se concentrant sur un sous-ensemble de m exemples. Nous proposons une proc{\'{e}}dure vorace pour la s{\'{e}}lection de ce sous-ensemble, qui est bas{\'{e}}e sur la distance dans l'espace des caract{\`{e}}ristiques entre un exemple candidat et le sous-espace g{\'{e}}n{\'{e}}r{\'{e}} par les exemples pr{\'{e}}c{\'{e}}demment choisis. Dans le cas de l'ACP {\`{a}} noyau ou de la segmentation spectrale, nous obtenons un algorithme en O(m*m*n), o{\`{u}} m << n, qui, contrairement aux techniques pr{\'{e}}c{\'{e}}demment propos{\'{e}}es, peut se formuler de fa{\c c}on en-ligne. Pour la m{\^{e}}me complexit{\'{e}} en temps, nous pouvons {\'{e}}galement calculer la projection des exemples non choisis sur le sous-espace engendr{\'{e}} par les exemples choisis dans l'espace des caract{\'{e}}ristiques. En repr{\'{e}}sentant ainsi les exemples par leur projection nous obtenons une approximation de plus faible rang de la matrice de Gram sur toutes les donn{\'{e}}es. Nous pouvons {\'{e}}galement borner l'erreur correspondant {\`{a}} cette approximation de la matrice de Gram.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3843 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3844
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3845 @ARTICLE{paiement+bengio+eck:aij,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3846 author = {Paiement, Jean-Fran{\c c}ois and Bengio, Samy and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3847 title = {Probabilistic Models for Melodic Prediction},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3848 journal = {Artificial Intelligence Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3849 volume = {173},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3850 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3851 pages = {1266-1274},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3852 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3853 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3854 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3855
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3856 @INPROCEEDINGS{paiement+eck+bengio+barber:icml2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3857 author = {Paiement, Jean-Fran{\c c}ois and Eck, Douglas and Bengio, Samy and Barber, D.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3858 title = {A graphical model for chord progressions embedded in a psychoacoustic space},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3859 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3860 pages = {641--648},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3861 publisher = {ACM Press},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3862 crossref = {ICML05},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3863 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3864 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3865 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3866
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3867 @INPROCEEDINGS{paiement+eck+bengio:ccai2006,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3868 author = {Paiement, Jean-Fran{\c c}ois and Eck, Douglas and Bengio, Samy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3869 editor = {Lamontagne, Luc and Marchand, Mario},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3870 title = {Probabilistic Melodic Harmonization},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3871 booktitle = {Canadian Conference on AI},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3872 series = {Lecture Notes in Computer Science},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3873 volume = {4013},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3874 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3875 pages = {218-229},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3876 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3877 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3878 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3879 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3880
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3881 @INPROCEEDINGS{paiement+eck+bengio:ismir2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3882 author = {Paiement, Jean-Fran{\c c}ois and Eck, Douglas and Bengio, Samy},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3883 title = {A Probabilistic Model for Chord Progressions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3884 booktitle = {{Proceedings of the 6th International Conference on Music Information Retrieval ({ISMIR} 2005)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3885 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3886 pages = {312-319},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3887 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3888 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3889 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3890
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3891 @INPROCEEDINGS{paiement+grandvalet+bengio+eck:icml2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3892 author = {Paiement, Jean-Fran{\c c}ois and Grandvalet, Yves and Bengio, Samy and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3893 title = {A generative model for rhythms},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3894 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3895 pages = {},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3896 crossref = {ICML06-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3897 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3898 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3899 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3900
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3901 @UNPUBLISHED{paiement+grandvalet+bengio+eck:nipsworkshop2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3902 author = {Paiement, Jean-Fran{\c c}ois and Grandvalet, Yves and Bengio, Samy and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3903 title = {A generative model for rhythms},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3904 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3905 note = {NIPS 2007 Workshop on Music, Brain and Cognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3906 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3907 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3908 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3909 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3910 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3911 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3912
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3913 @MASTERSTHESIS{Paiement-Msc-2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3914 author = {Paiement, Jean-Fran{\c c}ois},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3915 keywords = {algorithmes, apprentissage, apprentissage non supervis{\'{e}}, forage de donn{\'{e}}es, noyaux, r{\'{e}}duction de dimensions, statistique, Statistiques},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3916 title = {G{\'{e}}n{\'{e}}ralisation d'algorithmes de r{\'{e}}duction de dimension},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3917 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3918 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3919 abstract = {On pr{\'{e}}sente tout d'abord la notion de vari{\'{e}}t{\'{e}} comme r{\'{e}}gion de faible dimension contenant des observations situ{\'{e}}es dans un espace de haute dimension. Cette d{\'{e}}finition justifie l'{\'{e}}laboration d'algorithmes permettant d'exprimer les donn{\'{e}}es dans un syst{\`{e}}me de coordonn{\'{e}}es de dimensions {\'{e}}gale {\`{a}} celle de la vari{\'{e}}t{\'{e}} sur laquelle les donn{\'{e}}es sont approximativement situ{\'{e}}es.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3920 La notion de noyau comme mesure de similarit{\'{e}} est par la suite formalis{\'{e}}e. On constate que l'application d'un noyau {\`{a}} deux observations correspond {\`{a}} l'{\'{e}}valuation d'un produit scalaire dans un espace de Hilbert appel{\'{e}} espace de caract{\'{e}}ristiques.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3921 Une m{\'{e}}thode de r{\'{e}}duction de dimension lin{\'{e}}raire est expos{\'{e}}e ainsi que ces limites. Des algorithmes non lin{\'{e}}raires de r{\'{e}}duction de dimension et de segmentation permettent de s'affranchir de ces limites. Ces derniers ne fournissent cependant pas d'extension directe {\`{a}} des points hors {\'{e}}chantillon.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3922 L'{\'{e}}tape fondamentale au sein des algorithmes pr{\'{e}}sent{\'{e}}s est la solution d'un syst{\`{e}}me de vecteurs propres d'une matrice sym{\'{e}}trique cr{\'{e}}{\'{e}}e {\`{a}} partir d'un noyau d{\'{e}}pendant des donn{\'{e}}es. On con{\c c}oit cd probl{\`{e}}me comme le fait de trouver les fonctions propres d'un op{\'{e}}rateur lin{\'{e}}aire d{\'{e}}fini {\`{a}} partir du m{\^{e}}me noyau. On utilise alors la formulation de Nystr{\"{o}}m, pr{\'{e}}sente dans l'espace en composantes principales {\`{a}} noyaux, afin de r{\'{e}}duire la dimension des points hors {\'{e}}chantillon sur la vase des plongements obtenus {\`{a}} l'aide des algorithmes d{\'{e}}j{\`{a}} mentionn{\'{e}}s.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3923 La qualit{\'{e}} de la projection g{\'{e}}n{\'{e}}r{\'{e}}e est compar{\'{e}}e {\`{a}} la perturbation intrins{\`{e}}que des algorithmes si on substitue certaine observations par d'autres tir{\'{e}}es de la m{\^{e}}me distribution.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3924 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3925
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3926 @ARTICLE{perez+gers+schmidhuber+eck:2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3927 author = {Perez-Ortiz, J. A. and Gers, F. A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3928 title = {{K}alman filters improve {LSTM} network performance in problems unsolvable by traditional recurrent nets},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3929 journal = {Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3930 volume = {16},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3931 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3932 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3933 abstract = {The Long Short-Term Memory ({LSTM}) network trained by gradient descent solves difficult problems which traditional recurrent neural networks in general cannot. We have recently observed that the decoupled extended Kalman filter training algorithm allows for even better performance, reducing significantly the number of training steps when compared to the original gradient descent training algorithm. In this paper we present a set of experiments which are unsolvable by classical recurrent networks but which are solved elegantly and robustly and quickly by {LSTM} combined with Kalman filters.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3934 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3935 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3936 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3937
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3938 @ARTICLE{perez+gers+schmidhuber+eck:2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3939 author = {Perez-Ortiz, J. A. and Gers, F. A. and Eck, Douglas and Schmidhuber, Juergen},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3940 title = {{K}alman filters improve {LSTM} network performance in problems unsolvable by traditional recurrent nets},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3941 journal = {Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3942 volume = {16},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3943 number = {2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3944 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3945 pages = {241--250},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3946 abstract = {The Long Short-Term Memory ({LSTM}) network trained by gradient descent solves difficult problems which traditional recurrent neural networks in general cannot. We have recently observed that the decoupled extended Kalman filter training algorithm allows for even better performance, reducing significantly the number of training steps when compared to the original gradient descent training algorithm. In this paper we present a set of experiments which are unsolvable by classical recurrent networks but which are solved elegantly and robustly and quickly by {LSTM} combined with Kalman filters.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3947 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3948 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3949 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3950
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3951 @INPROCEEDINGS{perez+schmidhuber+gers+eck:icannB2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3952 author = {Perez-Ortiz, J. A. and Schmidhuber, Juergen and Gers, F. A. and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3953 editor = {Dorronsoro, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3954 title = {Improving Long-Term Online Prediction with {Decoupled Extended Kalman Filters}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3955 booktitle = {{Artificial Neural Networks -- ICANN 2002 (Proceedings)}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3956 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3957 pages = {1055--1060},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3958 publisher = {Springer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3959 abstract = {Long Short-Term Memory ({LSTM}) recurrent neural networks ({RNN}s) outperform traditional {RNN}s when dealing with sequences involving not only short-term but also long-term dependencies. The decoupled extended Kalman filter learning algorithm ({DEKF}) works well in online environments and reduces significantly the number of training steps when compared to the standard gradient-descent algorithms. Previous work on {LSTM}, however, has always used a form of gradient descent and has not focused on true online situations. Here we combine {LSTM} with {DEKF} and show that this new hybrid improves upon the original learning algorithm when applied to online processing.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3960 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3961 sourcetype={Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3962 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3963
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3964 @TECHREPORT{Pigeon-Bengio-96-aH-TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3965 author = {Pigeon, Steven and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3966 title = {A Memory-Efficient Huffman Adaptive Coding Algorithm for Very Large Sets of Symbols},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3967 number = {\#1081},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3968 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3969 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3970 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/HuffAdapt.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3971 abstract = {The problem of computing the minimum redundancy codes as we observe symbols one by one has received a lot of attention. However, existing algorithm implicitly assumes that either we have a small alphabet — quite typically 256 symbols — or that we have an arbitrary amount of memory at our disposal for the creation of the tree. In real life applications one may need to encode symbols coming from a much larger alphabet, for e.g. coding integers. We now have to deal not with hundreds of symbols but possibly with millions of symbols. While other algorithms use a space proportional to the number of observed symbol, we here propose one that uses space proportional to the number of frequency classes, which is, quite interestingly, always smaller or equal to the number of observed symbols.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3972 topics={Compression},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3973 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3974
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3975 @INPROCEEDINGS{Pigeon-dcc98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3976 author = {Pigeon, Steven and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3977 editor = {Society, {IEEE} Computer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3978 title = {A Memory-Efficient Adaptive Huffman Coding Algorithm for Very Large Sets of Symbols},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3979 booktitle = {Data Compression Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3980 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3981 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/dcc98.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3982 abstract = {The problem of computing the minimum redundancy codes as we observe symbols one by one has received a lot of attention. However, existing algorithms implicitly assumes that either we have a small alphabet — quite typically 256 symbols — or that we have an arbitrary amount of memory at our disposal for the creation of the tree. In real life applications one may need to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3983 encode symbols coming from a much larger alphabet, for e.g. coding integers. We now have to deal not with hundreds of symbols but possibly with millions of symbols. While other algorithms use a space proportional to the number of observed symbols, we here propose one that uses space proportional to the number of frequency classes, which is, quite interestingly, always smaller or equal to the size of the alphabet.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3984 topics={Compression},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3985 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3986
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3987 @INPROCEEDINGS{Pigeon-dcc99,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3988 author = {Pigeon, Steven and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3989 editor = {Society, {IEEE} Computer},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3990 title = {Binary Pseudowavelets and Applications to Bilevel Image Processing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3991 booktitle = {Data Compression Conference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3992 year = {1999},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3993 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/dcc99.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3994 abstract = {This paper shows the existance of binary pseudowavelets, bases on the binary domain that exhibit some of the properties of wavelets, such as multiresolution reconstruction and compact support. The binary pseudowavelets are defined on _n (binary vectors of length n) and are operated upon with the binary operators logical and and exclusive or. The forward transform, or analysis, is the decomposition of a binary vector into its constituant binary pseudowavelets. Binary pseudowavelets allow multiresolution, progressive reconstruction of binary vectors by using progressively more coefficients in the inverse transform. Binary pseudowavelets bases, being sparse matrices, also provide for fast transforms; moreover pseudowavelets rely on hardware-friendly operations for efficient software and hardware implementation.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3995 topics={Compression},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3996 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3997
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3998 @TECHREPORT{Pigeon-Huffman-TR98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3999 author = {Pigeon, Steven and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4000 title = {A Memory-Efficient Adaptive Huffman Coding for Very Large Sets of Symbols revisited},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4001 number = {1095},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4002 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4003 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4004 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TechRep_AdaptativeHuffman2.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4005 abstract = {While algorithm M (presented in A Memory-Efficient Huffman Adaptive Coding Algorithm for Very Large Sets of Symbols, by Steven Pigeon & Yoshua Bengio, Universit{\'{e}} de Montr{\'{e}}al technical report #1081 [1]) converges to the entropy of the signal, it also assumes that the characteristics of the signal are stationary, that is, that they do not change over time and that successive adjustments, ever decreasing in their magnitude, will lead to a reasonable approximation of the entropy. While this is true for some data, it is clearly not true for some other. We present here a modification of the M algorithm that allows negative updates. Negative updates are used to maintain a window over the source. Symbols enter the window at its right and will leave it at its left, after w steps (the window width). The algorithm presented here allows us to update correctly the weights of the symbols in the symbol tree. Here, we will also have negative migration or demotion, while we only had positive migration or promotion in M. This algorithm will be called M+.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4006 topics={Compression},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4007 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4008
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4009 @PHDTHESIS{Pigeon-Phd-2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4010 author = {Pigeon, Steven},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4011 keywords = {algorithmes, codes adaptatifs, codes de Golomb, codes universels, Compression de donn{\'{e}}es, compression LZ78, LZW, ondelettes, pseudo-ondelettes},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4012 title = {Contributions {\`{a}} la compression de donn{\'{e}}es},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4013 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4014 school = {Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4015 abstract = {L'objectif de cette th{\`{e}}se est de pr{\'{e}}senter nos contributions {\`{a}} la compression de donn{\'{e}}es. Le texte entier n'est pas consacr{\'{e}} {\`{a}} nos seules contributions. Une large part est consacr{\'{e}}e au mat{\'{e}}riel introductif et {\`{a}} la recension de litt{\'{e}}rature sur les sujets qui sont pertinents {\`{a}} nos contributions. Le premier chapitre de contribution, le chapitre "Contribution au codage des entiers" se concentre sur le probl{\`{e}}me de la g{\'{e}}n{\'{e}}ration de codes efficaces pour les entiers. Le chapitre "Codage Huffman Adaptatif" pr{\'{e}}sente deux nouveaux algorithmes pour la g{\'{e}}n{\'{e}}ration dynamique de codes structur{\'{e}}s en arbre, c'est-{\`{a}}-dire des codes de type Huffman. Le chapitre "LZW avec une perte" explore le probl{\`{e}}me de la compression d'images comportant un petit nombre de couleurs distinctes et propose une extension avec perte d'un algorithme originalement sans perte, LZW. Enfin, le dernier chapitre de contribution, le chapitre "Les pseudo-ondelettes binaires" pr{\'{e}}sente une solution original au probl{\`{e}}me de l'analyse multir{\'{e}}solution des images monochromes, c'est-{\`{a}}-dire des images n'ayant que deux couleurs, conventionnellement noir et blanc. Ce type d'image correspond par exemple aux images textuelles telle que produites par un processus de transmission de type facsimil{\'{e}}.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4016 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4017
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4018 @ARTICLE{Pigeon98,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4019 author = {Pigeon, Steven and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4020 title = {Memory-Efficient Adaptive Huffman Coding},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4021 journal = {Dr. Dobb's Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4022 volume = {290},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4023 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4024 pages = {131--135},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4025 topics={Compression},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4026 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4027
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4028 @INPROCEEDINGS{probnn:2000:ijcnn,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4029 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4030 title = {Probabilistic Neural Network Models for Sequential Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4031 booktitle = {International Joint Conference on Neural Networks 2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4032 volume = {V},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4033 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4034 pages = {79--84},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4035 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/81_01.PDF},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4036 abstract = {It has already been shown how Artificial Neural Networks ({ANN}s) can be incorporated into probabilistic models.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4037 In this paper we review some of the approaches which have been proposed to incorporate them into probabilistic
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4038 models of sequential data, such as Hidden {Markov} Models ({HMM}s). We also discuss new developments and new
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4039 ideas in this area, in particular how {ANN}s can be used to model high-dimensional discrete and continuous data to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4040 deal with the curse of dimensionality, and how the ideas proposed in these models could be applied to statistical
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4041 language modeling to represent longer-term context than allowed by trigram models, while keeping word-order
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4042 information.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4043 topics={Markov},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4044 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4045
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4046 @UNPUBLISHED{pugin+burgoyne+eck+fujinaga:nipsworkshop2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4047 author = {Pugin, L. and Burgoyne, J. A. and Eck, Douglas and Fujinaga, I.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4048 title = {Book-adaptive and book-dependant models to accelerate digitalization of early music},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4049 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4050 note = {NIPS 2007 Workshop on Music, Brain and Cognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4051 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4052 sourcetype={Workshop},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4053 optkey={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4054 optmonth={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4055 optannote={""},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4056 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4057
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4058 @INPROCEEDINGS{Rahim-97,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4059 author = {Rahim, Mazin and Bengio, Yoshua and {LeCun}, Yann},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4060 title = {Discriminative feature and model design for automatic speech recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4061 booktitle = {Proceedings of Eurospeech 1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4062 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4063 pages = {75--78},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4064 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/rahim-bengio-lecun-97.ps.gz},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4065 abstract = {A system for discriminative feature and model design is presented for automatic speech recognition. Training based on minimum classification error with a single objective function is applied for designing a set of parallel networks performing feature transformation and a set of hidden {Markov} models performing speech recognition. This paper compares the use of linear and non-linear functional transformations when applied to conventional recognition features, such as spectrum or cepstrum. It also provides a framework for integrated feature and model training when using class-specific transformations. Experimental results on telephone-based connected digit recognition are presented.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4066 topics={Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4067 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4068
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4069 @ARTICLE{Rivest-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4070 author = {Rivest, Fran{\c c}ois and Kalaska, John and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4071 title = {Alternative Time Representations in Dopamine Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4072 journal = {Journal of Computational Neuroscience},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4073 volume = {28},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4074 number = {1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4075 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4076 pages = {107--130},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4077 abstract = {Dopaminergic neuron activity has been modeled during learning and appetitive behavior, most commonly using the temporal-difference (TD) algorithm. However, a proper representation of elapsed time and of the exact task is usually required for the model to work. Most models use timing elements such as delay-line representations of time that are not biologically realistic for intervals in the range of seconds. The interval-timing literature provides several alternatives. One of them is that timing could emerge from general network dynamics, instead of coming from a dedicated circuit. Here, we present a general rate-based learning model based on long short-term memory ({LSTM}) networks that learns a time representation when needed. Using a na{\"{\i}}ve network learning its environment in conjunction with TD, we reproduce dopamine activity in appetitive trace conditioning with a constant CS-US interval, including probe trials with unexpected delays. The proposed model learns a representation of the environment dynamics in an adaptive biologically plausible framework, without recourse to delay lines or other special-purpose circuits. Instead, the model predicts that the task-dependent representation of time is learned by experience, is encoded in ramp-like changes in single-neuron activity distributed across small neural networks, and reflects a temporal integration mechanism resulting from the inherent dynamics of recurrent loops within the network. The model also reproduces the known finding that trace conditioning is more difficult than delay conditioning and that the learned representation of the task can be highly dependent on the types of trials experienced during training. Finally, it suggests that the phasic dopaminergic signal could facilitate learning in the cortex.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4078 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4079
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4080 @ARTICLE{schmidhuber+gers+eck:2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4081 author = {Schmidhuber, Juergen and Gers, F. A. and Eck, Douglas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4082 title = {Learning Nonregular Languages: A Comparison of Simple Recurrent Networks and {LSTM}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4083 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4084 volume = {14},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4085 number = {9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4086 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4087 pages = {2039--2041},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4088 abstract = {In response to Rodriguez' recent article (Rodriguez 2001) we compare the performance of simple recurrent nets and {\em ``Long Short-Term Memory''} ({LSTM}) recurrent nets on context-free and context-sensitive languages.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4089 source={OwnPublication},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4090 sourcetype={Journal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4091 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4092
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4093 @TECHREPORT{Schwenk-Bengio-97-TR,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4094 author = {Schwenk, Holger and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4095 title = {Adaptive Boosting of Neural Networks for Character Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4096 number = {\#1072},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4097 year = {1997},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4098 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4099 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/AdaBoostTR.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4100 abstract = {”Boosting” is a general method for improving the performance of any learning algorithm that consistently generates classifiers which need to perform only slightly better than random guessing. A recently proposed and very promising boosting algorithm is AdaBoost [5]. It has been applied with great success to several benchmark machine learning problems using rather simple learning algorithms [4], in particular decision trees [1, 2, 6]. In this paper we use AdaBoost to improve the performances of neural networks applied to character recognition tasks. We compare training methods based on sampling the training set and weighting the cost function. Our system achieves about 1.4\% error on a data base of online handwritten digits from more than 200 writers. Adaptive boosting of a multi-layer network achieved 2\% error on the UCI Letters offline characters data set.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4101 topics={Boosting,Speech},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4102 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4103
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4104 @INPROCEEDINGS{Schwenk-nips10,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4105 author = {Schwenk, Holger and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4106 title = {Training Methods for Adaptive Boosting of Neural Networks for Character Recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4107 year = {1998},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4108 crossref = {NIPS10-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4109 abstract = {”Boosting” is a general method for improving the performance of any learning algorithm that consistently generates classifiers which need to perform only slightly better than random guessing. A recently proposed and very promising boosting algorithm is AdaBoost [5]. It has been applied with great success to several benchmark machine learning problems using rather simple learning algorithms [4], in particular decision trees [1, 2, 6]. In this paper we use AdaBoost to improve the performances of neural networks applied to character recognition tasks. We compare training methods based on sampling the training set and weighting the cost function. Our system achieves about 1.4\% error on a data base of online handwritten digits from more than 200 writers. Adaptive boosting of a multi-layer network achieved 2\% error on the UCI Letters offline characters data set.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4110 topics={Boosting,Speech},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4111 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4112
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4113 @ARTICLE{Schwenk2000,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4114 author = {Schwenk, Holger and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4115 title = {Boosting Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4116 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4117 volume = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4118 number = {8},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4119 year = {2000},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4120 pages = {1869--1887},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4121 abstract = {“Boosting” is a general method for improving the performance of learning algorithms. A recently proposed boosting algorithm is AdaBoost. It has been applied with great success to several benchmark machine learning problems using mainly decision trees as base classifiers. In this paper we investigate whether AdaBoost also works as well with neural networks, and we discuss the advantages and drawbacks of di_erent versions of the AdaBoost algorithm. In particular, we compare training methods based on sampling the training set and weighting the cost function. The results suggest that random resampling of the training data is not the main explanation of the success of the improvements brought by AdaBoost. This is in contrast to Bagging which directly aims at reducing variance and for which random resampling is essential to obtain the reduction in generalization error. Our system achieves about 1.4\% error on a data set of online handwritten digits from more than 200 writers. A boosted multi-layer network achieved 1.5\% error on the UCI Letters and 8.1\% error on the UCI satellite data set, which is significantly better than boosted decision trees.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4122 topics={Boosting},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4123 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4124
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4125 @INPROCEEDINGS{secondorder:2001:nips,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4126 author = {Dugas, Charles and Bengio, Yoshua and Belisle, Francois and Nadeau, Claude and Garcia, Rene},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4127 title = {Incorporating Second-Order Functional Knowledge for Better Option Pricing},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4128 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4129 crossref = {NIPS13-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4130 abstract = {Incorporating prior knowledge of a particular task into the architecture of a learning algorithm can greatly improve generalization performance. We study here a case where we know that the function to be learned is non-decreasing in two of its arguments and convex in one of them. For this purpose we propose a class of functions similar to multi-layer neural networks but (1) that has those properties, (2) is a universal approximator of continuous functions with these and other properties. We apply this new class of functions to the task of modeling the price of call options. Experiments show improvements on regressing the price of call options using the new types of function classes that incorporate the a priori constraints.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4131 topics={Finance},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4132 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4133
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4134 @ARTICLE{Sonnenburg+al-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4135 author = {Sonnenburg, Soeren and et al. and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4136 title = {The Need for Open Source Software in Machine Learning.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4137 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4138 note = {institution: Fraunhofer Publica [http://publica.fraunhofer.de/oai.har] (Germany)},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4139 crossref = {JMLR-shorter},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4140 abstract = {all authors: Sonnenburg, S. and Braun, M.L. and Ong, C.S. and Bengio, S. and Bottou, L. and Holmes, G. and {LeCun}, Y. and M{\~{A}}¼ller, K.-R. and Pereira, F. and Rasmussen, C.E. and R{\~{A}}¤tsch, G. and Sch{\~{A}}{\P}lkopf, B. and Smola, A. and Vincent, P. and Weston, J. and Williamson, R.C.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4141
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4142 Open source tools have recently reached a level of maturity which makes them suitable for building large-scale real-world systems. At the same time, the field of machine learning has developed a large body of powerful learning algorithms for diverse applications. However, the true potential of these methods is not used, since existing implementations are not openly shared, resulting in software with low usability, and weak interoperability. We argue that this situation can be significantly improved by increasing incentives for researchers to publish their software under an open source model. Additionally, we outline the problems authors are faced with when trying to publish algorithmic implementations of machine learning methods. We believe that a resource of peer reviewed software accompanied by short articles would be highly valuable to both the machine learning and the general scientific community.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4143 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4144
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4145 @ARTICLE{Takeuchi-Bengio-Kanamori-2002,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4146 author = {Takeuchi, Ichiro and Bengio, Yoshua and Kanamori, Takafumi},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4147 title = {Robust Regression with Asymmetric Heavy-Tail Noise Distributions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4148 journal = {Neural Computation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4149 volume = {14},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4150 number = {10},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4151 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4152 pages = {2469--2496},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4153 abstract = {In the presence of a heavy-tail noise distribution, regression becomes much more difficult. Traditional robust regression methods assume that the noise distribution is symmetric and they down-weight the influence of so-called outliers. When the noise distribution is assymetric these methods yield biased regression estimators. Motivated by data-mining problems for the insurance industry, we propose in this paper a new approach to robust regession that is tailored to deal with the case where the noise distribution is asymmetric. The main idea is to learn most of the parameters of the model using conditional quantile estimators (which are biased but robust etimators of the regression), and to lern a few remaining parameters to combbine and correct these stimators, to unbiasedly minimize the average squared error. Theoritical analysis and experiments show the clear advantages of the approach. Results are on artificial data as well as real insurance data, using both linear and neural-network predictors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4154 topics={Mining},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4155 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4156
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4157 @PHDTHESIS{ThesisChapados2010,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4158 author = {Chapados, Nicolas},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4159 title = {Sequential Machine learning Approaches for Portfolio Management},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4160 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4161 school = {Universit{\'{e}} de Montr{\'{e}}al},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4162 abstract = {[English follow]
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4163 Cette th{\`{e}}se envisage un ensemble de m{\'{e}}thodes permettant aux algorithmes d'apprentissage statistique de mieux traiter la nature s{\'{e}}quentielle des probl{\`{e}}mes de gestion de portefeuilles financiers. Nous d{\'{e}}butons par une consid{\'{e}}ration du probl{\`{e}}me g{\'{e}}n{\'{e}}ral de la composition d'algorithmes d'apprentissage devant g{\'{e}}rer des t{\^{a}}ches s{\'{e}}quentielles, en particulier celui de la mise-{\`{a}}-jour efficace des ensembles d'apprentissage dans un cadre de validation s{\'{e}}quentielle. Nous {\'{e}}num{\'{e}}rons les desiderata que des primitives de composition doivent satisfaire, et faisons ressortir la difficult{\'{e}} de les atteindre de fa{\c c}on rigoureuse et efficace. Nous poursuivons en pr{\'{e}}sentant un ensemble d'algorithmes qui atteignent ces objectifs et pr{\'{e}}sentons une {\'{e}}tude de cas d'un syst{\`{e}}me complexe de prise de d{\'{e}}cision financi{\`{e}}re utilisant ces techniques. Nous d{\'{e}}crivons ensuite une m{\'{e}}thode g{\'{e}}n{\'{e}}rale permettant de transformer un probl{\`{e}}me de d{\'{e}}cision s{\'{e}}quentielle non-Markovien en un probl{\`{e}}me d'apprentissage supervis{\'{e}} en employant un algorithme de recherche bas{\'{e}} sur les K meilleurs chemins. Nous traitons d'une application en gestion de portefeuille o{\`{u}} nous entra{\^{\i}}nons un algorithme d'apprentissage {\`{a}} optimiser directement un ratio de Sharpe (ou autre crit{\`{e}}re non-additif incorporant une aversion au risque). Nous illustrons l'approche par une {\'{e}}tude exp{\'{e}}rimentale approfondie, proposant une architecture de r{\'{e}}seaux de neurones sp{\'{e}}cialis{\'{e}}e {\`{a}} la gestion de portefeuille et la comparant {\`{a}} plusieurs alternatives. Finalement, nous introduisons une repr{\'{e}}sentation fonctionnelle de s{\'{e}}ries chronologiques permettant {\`{a}} des pr{\'{e}}visions d'{\^{e}}tre effectu{\'{e}}es sur un horizon variable, tout en utilisant un ensemble informationnel r{\'{e}}v{\'{e}}l{\'{e}} de mani{\`{e}}re progressive. L'approche est bas{\'{e}}e sur l'utilisation des processus Gaussiens, lesquels fournissent une matrice de covariance compl{\`{e}}te entre tous les points pour lesquels une pr{\'{e}}vision est demand{\'{e}}e. Cette information est utilis{\'{e}}e {\`{a}} bon escient par un algorithme qui transige activement des {\'{e}}carts de cours (price spreads) entre des contrats {\`{a}} terme sur commodit{\'{e}}s. L'approche propos{\'{e}}e produit, hors {\'{e}}chantillon, un rendement ajust{\'{e}} pour le risque significatif, apr{\`{e}}s frais de transactions, sur un portefeuille de 30 actifs.
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4164 This thesis considers a number of approaches to make machine learning algorithms better suited to the sequential nature of financial portfolio management tasks. We start by considering the problem of the general composition of learning algorithms that must handle temporal learning tasks, in particular that of creating and efficiently updating the training sets in a sequential simulation framework. We enumerate the desiderata that composition primitives should satisfy, and underscore the difficulty of rigorously and efficiently reaching them. We follow by introducing a set of algorithms that accomplish the desired objectives, presenting a case-study of a real-world complex learning system for financial decision-making that uses those techniques. We then describe a general method to transform a non-Markovian sequential decision problem into a supervised learning problem using a K-best paths search algorithm. We consider an application in financial portfolio management where we train a learning algorithm to directly optimize a Sharpe Ratio (or other risk-averse non-additive) utility function. We illustrate the approach by demonstrating extensive experimental results using a neural network architecture specialized for portfolio management and compare against well-known alternatives. Finally, we introduce a functional representation of time series which allows forecasts to be performed over an unspecified horizon with progressively-revealed information sets. By virtue of using Gaussian processes, a complete covariance matrix between forecasts at several time-steps is available. This information is put to use in an application to actively trade price spreads between commodity futures contracts. The approach delivers impressive out-of-sample risk-adjusted returns after transaction costs on a portfolio of 30 spreads.}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4165 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4166
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4167 @ARTICLE{Thierry+al-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4168 author = {Bertin-Mahieux, Thierry and Eck, Douglas and Maillet, Fran{\c c}ois and Lamere, Paul},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4169 title = {Autotagger: A Model For Predicting Social Tags from Acoustic Features on Large Music Databases},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4170 journal = {Journal of New Music Research},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4171 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4172 abstract = {Social tags are user-generated keywords associated with some resource on the Web. In the case of music, social tags have become an important component of "Web 2.0" recommender systems, allowing users to generate playlists based on use-dependent terms such as chill or jogging that have been applied to particular songs. In this paper, we propose a method for predicting these social tags directly from MP3 files. Using a set of 360 classifiers trained using the online ensemble learning algorithm FilterBoost, we map audio features onto social tags collected from the Web. The resulting automatic tags (or autotags) furnish information about music that is otherwise untagged or poorly tagged, allowing for insertion of previously unheard music into a social recommender. This avoids the “cold-start problem” common in such systems. Autotags can also be used to smooth the tag space from which similarities and
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4173 recommendations are made by providing a set of comparable baseline tags for all tracks in a recommender system. Because the words we learn are the same as those used by people who label their music collections, it is easy to integrate our predictions into existing similarity and prediction methods based on web data.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4174 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4175
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4176 @ARTICLE{Thivierge+al-2007,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4177 author = {Thivierge, J. -P. and Rivest, Fran{\c c}ois and Monchi, O},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4178 title = {Spiking Neurons, Dopamine, and Plasticity: Timing Is Everything, But Concentration Also Matters},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4179 journal = {Synapse},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4180 volume = {61},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4181 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4182 pages = {375-390},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4183 abstract = {While both dopamine (DA) fluctuations and spike-timing-dependent plasticity (STDP) are known to influence long-term corticostriatal plasticity, little attention has been devoted to the interaction between these two fundamental mechanisms. Here, a theoretical framework is proposed to account for experimental results specifying the role of presynaptic activation, postsynaptic activation, and concentrations of extracellular DA in synaptic plasticity. Our starting point was an explicitly-implemented multiplicative rule linking STDP to Michaelis-Menton equations that models the dynamics of extracellular DA fluctuations. This rule captures a wide range of results on conditions leading to long-term potentiation and depression in simulations that manipulate the frequency of induced corticostriatal stimulation and DA release. A well-documented biphasic function relating DA concentrations to synaptic plasticity emerges naturally from simulations involving a multiplicative rule linking DA and neural activity. This biphasic function is found consistently across different neural coding schemes employed (voltage-based vs. spike-based models). By comparison, an additive rule fails to capture these results. The proposed framework is the first to generate testable predictions on the dual influence of DA concentrations and STDP on long-term plasticity, suggesting a way in which the biphasic influence of DA concentrations can modulate the direction and magnitude of change induced by STDP, and raising the possibility that DA concentrations may inverse the LTP/LTD components of the STDP rule.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4184 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4185
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4186 @TECHREPORT{tonga-tr,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4187 author = {Le Roux, Nicolas and Manzagol, Pierre-Antoine and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4188 title = {Topmoumoute online natural gradient algorithm},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4189 number = {1299},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4190 year = {2007},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4191 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4192 abstract = {Guided by the goal of obtaining an optimization algorithm that is
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4193 both fast and yielding good generalization, we study the descent direction maximizing the decrease in generalization error or the probability of not increasing generalization error. The surprising result is that from both the Bayesian and frequentist perspectives this can yield the natural gradient direction. Although that direction can be very expensive to compute we develop an efficient, general, online approximation to the natural gradient descent which is suited to large scale problems. We report experimental results showing much faster convergence in computation time and in number of iterations with TONGA (Topmoumoute Online natural Gradient Algorithm) than with stochastic gradient descent, even on very large datasets.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4194 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4195
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4196 @TECHREPORT{TR1197,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4197 author = {Vincent, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4198 title = {K-Local Hyperplane and Convex Distance Nearest Neighbor Algorithms},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4199 number = {1197},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4200 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4201 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4202 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1197.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4203 abstract = {Guided by an initial idea of building a complex (non linear) decision surface with maximal local margin in input space, we give a possible geometrical intuition as to why K-Nearest Neighbor ({KNN}) algorithms often perform more poorly than {SVM}s on classification tasks. We then propose modified K-Nearest Neighbor algorithms to overcome the perceived problem. The approach is similar in spirit to Tangent Distance, but with invariances inferred from the local neighborhood rather than prior knowledge. Experimental results on real world classification tasks suggest that the modified {KNN} algorithms often give a dramatic improvement over standard {KNN} and perform as well or better than {SVM}s.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4204 topics={Kernel},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4205 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4206
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4207 @TECHREPORT{TR1198,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4208 author = {Takeuchi, Ichiro and Bengio, Yoshua and Kanamori, Takafumi},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4209 title = {Robust Regression with Asymmetric Heavy-Tail Noise},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4210 number = {1198},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4211 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4212 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4213 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1198.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4214 abstract = {In the presence of a heavy-tail noise distribution, regression becomes much more difficult. Traditional robust regression methods assume that the noise distribution is symmetric and they downweight the influence of so-called outliers. When the noise distribution is asymmetric these methods yield strongly biased regression estimators. Motivated by data-mining problems for the insurance industry, we propose in this paper a new approach to robust regression that is tailored to deal with the case where the noise distribution is asymmetric. The main idea is to learn most of the parameters of the model using conditional quantile estimators (which are biased but robust estimators of the regression), and to learn a few remaining parameters to combine and correct these estimators, to minimize the average squared error. Theoretical analysis and experiments show the clear advantages of the approach. Results are on artificial data as well as real insurance data, using both linear and neural-network predictors.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4215 topics={Mining},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4216 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4217
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4218 @TECHREPORT{TR1199,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4219 author = {Chapados, Nicolas and Bengio, Yoshua and Vincent, Pascal and Ghosn, Joumana and Dugas, Charles and Takeuchi, Ichiro and Meng, Linyan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4220 title = {Estimating Car Insurance Premia: a Case Study in High-Dimensional Data Inference},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4221 number = {1199},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4222 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4223 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4224 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1199.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4225 abstract = {Estimating insurance premia from data is a difficult regression problem for several reasons: the large number of variables, many of which are discrete, and the very peculiar shape of the noise distribution, asymmetric with fat tails, with a large majority zeros and a few unreliable and very large values. We introduce a methodology for estimating insurance premia that has been applied in the car insurance industry. It is based on mixtures of specialized neural networks, in order to reduce the effect of outliers on the estimation. Statistical comparisons with several different alternatives, including decision trees and generalized linear models show that the proposed method is significantly more precise, allowing to identify the least and most risky contracts, and reducing the median premium by charging more to the most risky customers.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4226 topics={HighDimensional,Mining},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4227 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4228
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4229 @TECHREPORT{TR1200,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4230 author = {Bengio, Yoshua and Chapados, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4231 title = {Extending Metric-Based Model Selection and Regularization in the Absence of Unlabeled Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4232 number = {1200},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4233 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4234 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4235 url = {http://www.iro.umontreal.ca/lisa/pointeurs/TR1200.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4236 abstract = {Metric-based methods have recently been introduced for model selection and regularization, often yielding very significant improvements over all the alternatives tried (including cross-validation). However, these methods require a large set of unlabeled data, which is not always available in many applications. In this paper we extend these methods (TRI, ADJ and ADA) to the case where no unlabeled data is available. The extended methods (xTRI, xADJ, xADA) use a model of the input density directly estimated from the training set. The intuition is that the main reason why the above methods work well is that they make sure that the learned function behaves similarly on the training points and on “neighboring” points. The experiments are based on estimating a simple non-parametric density model. They show that the extended methods perform comparably to the originals even though no unlabeled data is used.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4237 topics={ModelSelection,Finance},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4238 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4239
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4240 @TECHREPORT{TR1215,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4241 author = {Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4242 title = {New Distributed Probabilistic Language Models},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4243 number = {1215},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4244 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4245 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4246 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1215.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4247 abstract = {Our previous work on statistical modeling introduced the use of probabilistic feedforward neural networks with shared parameters in order to help dealing with the curse of dimensionality. This work started with the motivation to speed up the above model and to take advantage of prior knowledge e.g., in WordNet or in syntactically labeled data sets, and to better deal with polysemy. With the objective of reaching these goals, we present here a series of new statistical language models, most of which are yet untested.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4248 topics={Markov,Language,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4249 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4250
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4251 @TECHREPORT{TR1216,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4252 author = {Bengio, Yoshua and S{\'{e}}n{\'{e}}cal, Jean-S{\'{e}}bastien},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4253 title = {Quick Training of Probabilistic Neural Nets by Importance Sampling},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4254 number = {1216},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4255 year = {2002},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4256 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4257 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1216.ps},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4258 abstract = {Our previous work on statistical language modeling introduced the use of probabilistic feedforward neural networks to help dealing with the curse of dimensionality. Training this model by maximum likelihood however requires for each example to perform as many network passes as there are words in the vocabulary. Inspired by the contrastive divergence model, we proposed and evaluate sampling-based methods which require network passes only for the observed “positive example” and a few sampled negative example words. A very significant speed-up is obtained with an adaptive importance sampling.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4259 topics={Markov,Language,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4260 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4261
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4262 @TECHREPORT{TR1231,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4263 author = {Bengio, Yoshua and Kermorvant, Christopher},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4264 title = {Extracting Hidden Sense Probabilities from Bitexts},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4265 number = {1231},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4266 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4267 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4268 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1231.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4269 abstract = {We propose a probabilistic model that is inspired by Diab & Resnik’s algorithm to extract disambiguation information from aligned bilingual texts. Like Diab & Resnik’s, the proposed model uses WordNet and the fact that word ambiguities are not always the same in the two languages. The generative model introduces a dependency between two translated words through a common ancestor inWordNet’s ontology. Unlike Diab & Resnik’s algorithm it does not suppose that the translation in the source language has a single meaning.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4270 topics={Language},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4271 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4272
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4273 @TECHREPORT{TR1232,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4274 author = {Bengio, Yoshua and Vincent, Pascal and Paiement, Jean-Fran{\c c}ois},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4275 title = {Learning Eigenfunctions of Similarity: Linking Spectral Clustering and Kernel {PCA}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4276 number = {1232},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4277 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4278 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4279 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1232.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4280 abstract = {In this paper, we show a direct equivalence between spectral clustering and kernel {PCA}, and how both are special cases of a more general learning problem, that of learning the principal eigenfunctions of a kernel, when the functions are from a Hilbert space whose inner product is defined with respect to a density model. This suggests a new approach to unsupervised learning in which abstractions (such as manifolds and clusters) that represent the main features of the data density are extracted. Abstractions discovered at one level can be used to build higher-level abstractions. This paper also discusses how these abstractions can be used to recover a quantitative model of the input density, which is at least useful for evaluative and comparative purposes.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4281 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4282 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4283
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4284 @TECHREPORT{TR1234,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4285 author = {Bengio, Yoshua and Grandvalet, Yves},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4286 title = {No Unbiased Estimator of the Variance of K-Fold Cros-Validation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4287 number = {1234},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4288 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4289 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4290 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/TR1234.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4291 abstract = {Most machine learning researchers perform quantitative experiments to estimate generalization error and compare the performance of different algorithms (in particular, their proposed algorithm). In order to be able to draw statistically convincing conclusions, it is important for them to also estimate the uncertainty around the error (or error difference) estimate. This paper studies the very commonly used K-fold cross-validation estimator of generalization performance. The main theorem shows that there exists no universal (valid under all distributions) unbiased estimator of the variance of K-fold cross-validation. The analysis that accompanies this result is based on the eigen-decomposition of the covariance matrix of errors, which has only three different eigenvalues corresponding to three degrees of freedom of the matrix and three components of the total variance. This analysis helps to better understand the nature of the problem and how it can make na{\"{\i}}ve estimators (that don’t take into account the error correlations due to the overlap between training and test sets) grossly underestimate variance. This is confirmed by numerical experiments in which the three components of the variance are compared when the difficulty of the learning problem and the number of folds are varied.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4292 topics={Comparative},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4293 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4294
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4295 @TECHREPORT{tr1238,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4296 author = {Bengio, Yoshua and Paiement, Jean-Fran{\c c}ois and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4297 title = {Out-of-Sample Extensions for {LLE}, {I}somap, {MDS}, {E}igenmaps, and Spectral Clustering},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4298 number = {1238},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4299 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4300 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4301 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1238.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4302 abstract = {Several unsupervised learning algorithms based on an eigendecomposition provide either an embedding or a clustering only for given training points, with no straightforward extension for out-of-sample examples short of recomputing eigenvectors. This paper provides algorithms for such an extension for Local Linear Embedding ({LLE}), Isomap, Laplacian Eigenmaps, Multi-Dimensional Scaling (all algorithms which provide lower-dimensional embedding for dimensionality reduction) as well as for Spectral Clustering (which performs non-Gaussian clustering). These extensions stem from a unified framework in which these algorithms are seen as learning eigenfunctions of a kernel. {LLE} and Isomap pose special challenges as the kernel is training-data dependent. Numerical experiments on real data show that the generalizations performed have a level of error comparable to the variability of the embedding algorithms to the choice of training data.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4303 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4304 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4305
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4306 @TECHREPORT{tr1239,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4307 author = {Bengio, Yoshua and Vincent, Pascal and Paiement, Jean-Fran{\c c}ois and Delalleau, Olivier and Ouimet, Marie and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4308 title = {Spectral Clustering and Kernel {PCA} are Learning Eigenfunctions},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4309 number = {1239},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4310 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4311 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4312 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1239.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4313 abstract = {In this paper, we show a direct equivalence between spectral clustering and kernel {PCA}, and how both are special cases of a more general learning problem, that of learning the principal eigenfunctions of a kernel, when the functions are from a function space whose scalar product is defined with respect to a density model. This defines a natural mapping for new data points, for methods that only provided an embedding, such as spectral clustering and Laplacian eigenmaps. The analysis hinges on a notion of generalization for embedding algorithms based on the estimation of underlying eigenfunctions, and suggests ways to improve this generalization by smoothing the data empirical distribution.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4314 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4315 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4316
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4317 @TECHREPORT{tr1240,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4318 author = {Vincent, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4319 title = {Locally Weighted Full Covariance Gaussian Density Estimation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4320 number = {1240},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4321 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4322 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4323 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1240.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4324 abstract = {We describe an interesting application of the principle of local learning to density estimation. Locally weighted fitting of a Gaussian with a regularized full covariance matrix yields a density estimator which displays improved behavior in the case where much of the probability mass is concentrated along a low dimensional manifold. While the proposed estimator is not guaranteed to integrate to 1 with a finite sample size, we prove asymptotic convergence to the true density. Experimental results illustrating the advantages of this estimator over classic non-parametric estimators are presented.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4325 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4326 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4327
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4328 @TECHREPORT{tr1247,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4329 author = {Bengio, Yoshua and Delalleau, Olivier and Le Roux, Nicolas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4330 title = {Efficient Non-Parametric Function Induction in Semi-Supervised Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4331 number = {1247},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4332 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4333 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4334 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1247.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4335 abstract = {There has been an increase of interest for semi-supervised learning recently, because of the many datasets with large amounts of unlabeled examples and only a few labeled ones. This paper follows up on proposed non-parametric algorithms which provide an estimated continuous label for the given unlabeled examples. It extends them to function induction algorithms that correspond to the minimization of a regularization criterion applied to an out-of-sample example, and happens to have the form of a Parzen windows regressor. The advantage of the extension is that it allows predicting the label for a new example without having to solve again a linear system of dimension n (the number of unlabeled and labeled training examples), which can cost O(n^3). Experiments show that the extension works well, in the sense of predicting a label close to the one that would have been obtained if the test example had been included in the unlabeled set. This relatively efficient function induction procedure can also be used when n is large to approximate the solution by writing it only in terms of a kernel expansion with m << n terms, and reducing the linear system to m equations in m unknowns.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4336 topics={Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4337 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4338
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4339 @TECHREPORT{tr1250,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4340 author = {Bengio, Yoshua and Monperrus, Martin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4341 title = {Discovering shared structure in manifold learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4342 number = {1250},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4343 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4344 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4345 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr-tangent.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4346 abstract = {We claim and present arguments to the effect that a large class of manifold learning algorithms that are essentially local will suffer from at least four generic problems associated with (1) noise in the data, (2) curvature of the manifold, (3) dimensionality of the manifold, and (4) the presence of many manifolds with little data per manifold. This analysis suggests non-local manifold learning algorithms which attempt to discover shared structure in the tangent planes at different positions. A criterion for such an algorithm is proposed and experiments estimating a tangent plane prediction function are presented. The function has parameters that are shared across space rather than estimated based on the local neighborhood, as in current non-parametric manifold learning algorithms. The results show clearly the advantages of this approach with respect to local manifold learning algorithms.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4347 topics={HighDimensional,Kernel,Unsupervised},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4348 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4349
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4350 @TECHREPORT{tr1252,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4351 author = {Bengio, Yoshua and Larochelle, Hugo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4352 title = {Implantation et analyse d'un mod{\`{e}}le graphique {\`{a}} entra{\^{\i}}nement supervis{\'{e}}, semi-supervis{\'{e}} et non-supervis{\'{e}} pour la d{\'{e}}sambigu{\"{\i}}sation s{\'{e}}mantique},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4353 number = {1252},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4354 year = {2004},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4355 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4356 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/tr1252.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4357 abstract = {La d{\'{e}}sambigu{\"{\i}}sation s{\'{e}}mantique est un sujet qui suscite beaucoup d’int{\'{e}}r{\^{e}}t dans la communaut{\'{e}} scientifique en apprentissage automatique. Quoique cette t{\^{a}}che ait {\'{e}}t{\'{e}} abord{\'{e}}e depuis les d{\'{e}}buts du traitement automatique de la langue, peu de progr{\`{e}}s ont {\'{e}}t{\'{e}} accomplis jusqu’{\`{a}} maintenant. Nous pr{\'{e}}sentons ici une application de d{\'{e}}sambigu{\"{\i}}sation bas{\'{e}}e sur un mod{\`{e}}le graphique probabiliste. Ce mod{\`{e}}le a {\'{e}}t{\'{e}} appris sur des donn{\'{e}}es {\'{e}}tiquet{\'{e}}es, non-{\'{e}}tiquet{\'{e}}es, et sur la hi{\'{e}}rarchie WordNet. Avec peu d’examples d’apprentissage, ses performances sont comparables {\`{a}} celles de l’algorithme de Bayes na{\"{\i}}f. Il pourrait {\'{e}}ventuellement {\^{e}}tre adapt{\'{e}} {\`{a}} des corpus bi-textes.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4358 topics={Unsupervised,Language},cat={T},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4359 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4360
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4361 @TECHREPORT{tr1281,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4362 author = {Le Roux, Nicolas and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4363 title = {Continuous Neural Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4364 number = {1281},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4365 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4366 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4367 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/continuous_nnet_tr1281.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4368 abstract = {This article extends neural networks to the case of an uncountable number of hidden units, in several
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4369 ways. In the first approach proposed, a finite parametrization is possible, allowing gradient-based
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4370 learning. While having the same number of parameters as an ordinary neural network, its internal
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4371 structure suggests that it can represent some smooth functions much more compactly. Under mild
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4372 assumptions, we also find better error bounds than with ordinary neural networks. Furthermore, this
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4373 parametrization may help reducing the problem of saturation of the neurons. In a second approach, the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4374 input-to-hidden weights are fully non-parametric, yielding a kernel machine for which we demonstrate
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4375 a simple kernel formula. Interestingly, the resulting kernel machine can be made hyperparameter-free
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4376 and still generalizes in spite of an absence of explicit regularization.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4377 cat={T},topics={Kernel,HighDimensional},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4378 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4379
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4380 @TECHREPORT{tr1282,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4381 author = {Bengio, Yoshua and Lamblin, Pascal and Popovici, Dan and Larochelle, Hugo},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4382 title = {Greedy Layer-Wise Training of Deep Networks},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4383 number = {1282},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4384 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4385 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4386 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/dbn_supervised_tr1282.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4387 abstract = {Deep multi-layer neural networks have many levels of non-linearities, which allows them to potentially
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4388 represent very compactly highly non-linear and highly-varying functions. However, until recently it
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4389 was not clear how to train such deep networks, since gradient-based optimization starting from random
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4390 initialization appears to often get stuck in poor solutions. Hinton et al. recently introduced a greedy
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4391 layer-wise unsupervised learning algorithm for Deep Belief Networks (DBN), a generative model with
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4392 many layers of hidden causal variables. In the context of the above optimization problem, we study
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4393 this algorithm empirically and explore variants to better understand its success and extend it to cases
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4394 where the inputs are continuous or where the structure of the input distribution is not revealing enough
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4395 about the variable to be predicted in a supervised task.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4396 cat={T},topics={HighDimensional,Unsupervised},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4397 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4398
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4399 @TECHREPORT{tr1283,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4400 author = {Carreau, Julie and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4401 title = {A Hybrid {Pareto} Model for Asymmetric Fat-Tail Data},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4402 number = {1283},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4403 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4404 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4405 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/fat_tails_tr1283.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4406 abstract = {We propose an estimator for the conditional density p(Y |X) that can adapt for asymmetric heavy tails
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4407 which might depend on X. Such estimators have important applications in finance and insurance. We
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4408 draw from Extreme Value Theory the tools to build a hybrid unimodal density having a parameter
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4409 controlling the heaviness of the upper tail. This hybrid is a Gaussian whose upper tail has been
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4410 replaced by a generalized {Pareto} tail. We use this hybrid in a multi-modal mixture in order to obtain
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4411 a nonparametric density estimator that can easily adapt for heavy tailed data. To obtain a conditional
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4412 density estimator, the parameters of the mixture estimator can be seen as functions of X and these
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4413 functions learned. We show experimentally that this approach better models the conditional density in
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4414 terms of likelihood than compared competing algorithms: conditional mixture models with other types
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4415 of components and multivariate nonparametric models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4416 cat={T},topics={Unsupervised,Mining},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4417 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4418
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4419 @TECHREPORT{tr1284,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4420 author = {Larochelle, Hugo and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4421 title = {Distributed Representation Prediction for Generalization to New Words},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4422 number = {1284},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4423 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4424 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4425 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/dist_rep_pred_tr1284.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4426 abstract = {Learning distributed representations of symbols (e.g. words) has been used in several Natural Language Processing
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4427 systems. Such representations can capture semantic or syntactic similarities between words, which permit to fight
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4428 the curse of dimensionality when considering sequences of such words. Unfortunately, because these representations
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4429 are learned only for a previously determined vocabulary of words, it is not clear how to obtain representations
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4430 for new words. We present here an approach which gets around this problem by considering the distributed representations
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4431 as predictions from low-level or domain-knowledge features of words. We report experiments on a Part
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4432 Of Speech tagging task, which demonstrates the success of this approach in learning meaningful representations and
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4433 in providing improved accuracy, especially for new words.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4434 cat={T},topics={HighDimensional,Language},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4435 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4436
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4437 @TECHREPORT{tr1285,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4438 author = {Grandvalet, Yves and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4439 title = {Hypothesis Testing for Cross-Validation},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4440 number = {1285},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4441 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4442 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4443 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/xv_rho_stat_tr1285.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4444 abstract = {K-fold cross-validation produces variable estimates, whose variance cannot be estimated unbiasedly. However, in practice, one would like to
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4445 provide a figure related to the variability of this estimate. The first part
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4446 of this paper lists a series of restrictive assumptions (on the distribution of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4447 cross-validation residuals) that allow to derive unbiased estimates. We exhibit three such estimates, corresponding to differing assumptions. Their
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4448 similar expressions however entail almost identical empirical behaviors.
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4449 Then, we look for a conservative test for detecting significant differences
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4450 in performances between two algorithms. Our proposal is based on the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4451 derivation of the form of a t-statistic parametrized by the correlation of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4452 residuals between each validation set. Its calibration is compared to the
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4453 usual t-test. While the latter is overconfident in concluding that differences are indeed significant, our test is bound to be more skeptical, with
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4454 smaller type-I error.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4455 cat={T},topics={ModelSelection,Comparative},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4456 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4457
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4458 @TECHREPORT{tr1286,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4459 author = {Erhan, Dumitru and Bengio, Yoshua and {L'Heureux}, Pierre-Jean and Yue, Shi Yi},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4460 title = {Generalizing to a Zero-Data Task: a Computational Chemistry Case Study},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4461 number = {1286},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4462 year = {2006},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4463 institution = {D{\'{e}}partement d'informatique et recherche op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4464 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/mt_qsar_tr1286.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4465 abstract = {We investigate the problem of learning several tasks simultaneously in order to transfer the acquired
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4466 knowledge to a completely new task for which no training data are available. Assuming that the tasks
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4467 share some representation that we can discover efficiently, such a scenario should lead to a better model of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4468 the new task, as compared to the model that is learned by only using the knowledge of the new task. We
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4469 have evaluated several supervised learning algorithms in order to discover shared representations among
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4470 the tasks defined in a computational chemistry/drug discovery problem. We have cast the problem from
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4471 a statistical learning point of view and set up the general hypotheses that have to be tested in order
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4472 to validate the multi-task learning approach. We have then evaluated the performance of the learning
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4473 algorithms and showed that it is indeed possible to learn a shared representation of the tasks that allows
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4474 to generalize to a new task for which no training data are available. From a theoretical point of view,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4475 our contribution also comprises a modification to the Support Vector Machine algorithm, which can
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4476 produce state-of-the-art results using multi-task learning concepts at its core. From a practical point
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4477 of view, our contribution is that this algorithm can be readily used by pharmaceutical companies for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4478 virtual screening campaigns.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4479 cat={T},topics={MultiTask,Kernel,Bioinformatic},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4480 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4481
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4482 @INPROCEEDINGS{Turian+al-2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4483 author = {Turian, Joseph and Bergstra, James and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4484 title = {Quadratic Features and Deep Architectures for Chunking},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4485 booktitle = {North American Chapter of the Association for Computational Linguistics - Human Language Technologies (NAACL HLT)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4486 year = {2009},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4487 pages = {245--248},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4488 publisher = {Association for Computational Linguistics},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4489 url = {http://www.aclweb.org/anthology/N/N09/N09-2062},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4490 abstract = {We experiment with several chunking models. Deeper architectures achieve better generalization. Quadratic filters, a simplification of theoretical model of V1 complex cells, reliably increase accuracy. In fact, logistic regression with quadratic filters outperforms a standard single hidden layer neural network. Adding quadratic filters to logistic regression is almost as effective as feature engineering. Despite predicting each output label independently, our model is competitive with ones that use previous decisions.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4491 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4492
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4493 @INPROCEEDINGS{Turian+al-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4494 author = {Turian, Joseph and Ratinov, Lev and Bengio, Yoshua and Roth, Dan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4495 title = {A preliminary evaluation of word representations for named-entity recognition},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4496 booktitle = {NIPS Workshop on Grammar Induction, Representation of Language and Language Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4497 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4498 url = {http://www.iro.umontreal.ca/~lisa/pointeurs/wordrepresentations-ner.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4499 abstract = {We use different word representations as word features for a named-entity recognition (NER) system with a linear model. This work is part of a larger empirical survey, evaluating different word representations on different NLP tasks. We evaluate Brown clusters, Collobert and Weston (2008) embeddings, and HLBL (Mnih & Hinton, 2009) embeddings of words. All three representations improve accuracy on NER, with the Brown clusters providing a larger improvement than the two embeddings, and the HLBL embeddings more than the Collobert and Weston (2008) embeddings. We also discuss some of the practical issues in using embeddings as features. Brown clusters are simpler than embeddings because they require less hyperparameter tuning.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4500 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4501
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4502 @INPROCEEDINGS{Turian+Ratinov+Bengio-2010,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4503 author = {Turian, Joseph and Ratinov, Lev and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4504 title = {Word representations: A simple and general method for semi-supervised learning},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4505 booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics(ACL2010)},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4506 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4507 pages = {384--394},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4508 publisher = {Association for Computational Linguistics},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4509 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4510
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4511 @INPROCEEDINGS{Vincent-Bengio-2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4512 author = {Vincent, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4513 title = {Manifold Parzen Windows},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4514 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4515 pages = {825--832},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4516 crossref = {NIPS15-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4517 abstract = {The similarity between objects is a fundamental element of many learning algorithms. Most non-parametric methods take this similarity to be fixed, but much recent work has shown the advantages of learning it, in particular to exploit the local invariances in the data or to capture the possibly non-linear manifold on which most of the data lies. We propose a new non-parametric kernel density estimation method which captures the local structure of an underlying manifold through the leading eigenvectors of regularized local covariance matrices. Experiments in density estimation show significant improvements with respect to Parzen density estimators. The density estimators can also be used within Bayes classifiers, yielding classification rates similar to {SVM}s and much superior to the Parzen classifier.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4518 topics={HighDimensional,Kernel,Unsupervised},cat={C},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4519 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4520
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4521 @ARTICLE{Vincent-JMLR-2010,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4522 author = {Vincent, Pascal and Larochelle, Hugo and Lajoie, Isabelle and Bengio, Yoshua and Manzagol, Pierre-Antoine},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4523 title = {Stacked Denoising Autoencoders: learning useful representations in a deep network with a local denoising criterion},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4524 journal = {JMLR},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4525 year = {2010},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4526 note = {to appear}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4527 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4528
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4529 @TECHREPORT{Vincent-TR1316,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4530 author = {Vincent, Pascal and Larochelle, Hugo and Bengio, Yoshua and Manzagol, Pierre-Antoine},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4531 title = {Extracting and Composing Robust Features with Denoising Autoencoders},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4532 number = {1316},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4533 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4534 institution = {D{\'{e}}partement d'Informatique et Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4535 url = {http://www.iro.umontreal.ca/~vincentp/Publications/denoising_autoencoders_tr1316.pdf},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4536 abstract = {Previous work has shown that the difficulties in learning deep generative or discriminative models can be overcome by an initial unsupervised learning step that maps inputs to useful intermediate representations. We introduce and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. This approach can be used to train autoencoders, and these denoising autoencoders can be stacked to initialize deep architectures. The algorithm can be motivated from a manifold learning and information theoretic perspective or from a generative model perspective. Comparative experiments clearly show the surprising advantage of corrupting the input of autoencoders on a pattern classification benchmark suite.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4537 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4538
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4539 @PHDTHESIS{Vincent2003,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4540 author = {Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4541 title = {Mod{\`{e}}les {\`{a}} Noyaux {\`{a}} Structure Locale},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4542 year = {2003},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4543 school = {D{\'{e}}partement d'Informatique et Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4544 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4545
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4546 @ARTICLE{vincent:2001,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4547 author = {Vincent, Pascal and Bengio, Yoshua},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4548 title = {Kernel Matching Pursuit},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4549 journal = {Machine Learning},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4550 year = {2001},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4551 abstract = {We show how Matching Pursuit can be used to build kernel-based solutions to machine-learning problems while keeping control of the sparsity of the solution, and how it can be extended to use non-squared error loss functions. We also deriveMDL motivated generalization bounds for this type of algorithm. Finally, links to boosting algorithms and {RBF} training procedures, as well as extensive experimental comparison with {SVM}s are given, showing comparable results with typically sparser models.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4552 topics={HighDimensional,Kernel},cat={J},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4553 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4554
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4555 @INPROCEEDINGS{VincentPLarochelleH2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4556 author = {Vincent, Pascal and Larochelle, Hugo and Bengio, Yoshua and Manzagol, Pierre-Antoine},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4557 title = {Extracting and Composing Robust Features with Denoising Autoencoders},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4558 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4559 pages = {1096--1103},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4560 crossref = {ICML08-shorter},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4561 abstract = {Recently, many applications for Restricted {Boltzmann} Machines (RBMs) have been developed for a large variety of learning problems. However, RBMs are usually used as feature extractors for another learning algorithm or to provide a good initialization
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4562 for deep feed-forward neural network classifiers, and are not considered as a standalone solution to classification problems. In
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4563 this paper, we argue that RBMs provide a self-contained framework for deriving competitive non-linear classifiers. We present an evaluation of different learning algorithms for
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4564 RBMs which aim at introducing a discriminative component to RBM training and improve their performance as classifiers. This
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4565 approach is simple in that RBMs are used directly to build a classifier, rather than as a stepping stone. Finally, we demonstrate how discriminative RBMs can also be successfully employed in a semi-supervised setting.}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4566 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4567
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4568 @TECHREPORT{visualization_techreport,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4569 author = {Erhan, Dumitru and Bengio, Yoshua and Courville, Aaron and Vincent, Pascal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4570 title = {Visualizing Higher-Layer Features of a Deep Network},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4571 number = {1341},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4572 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4573 institution = {University of Montreal},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4574 abstract = {Deep architectures have demonstrated state-of-the-art results in a variety of
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4575 settings, especially with vision datasets. Beyond the model definitions and the quantitative analyses, there is a need for qualitative comparisons of the solutions learned by various deep architectures. The goal of this paper is to find good qualitative interpretations of high level features represented by such models. To this end, we contrast and compare several techniques applied on Stacked Denoising Autoencoders and Deep Belief Networks, trained on several vision datasets. We show that, perhaps counter-intuitively, such interpretation is possible at the unit level, that it is simple to accomplish and that the results are consistent across various techniques. We hope that such techniques will allow researchers in deep architectures to understand more of how and why deep architectures work}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4576 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4577
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4578 @INPROCEEDINGS{xAISTATS2009-short,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4579 title = {Proc. AISTATS'2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4580 booktitle = {Proc. AISTATS'2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4581 year = {2009}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4582 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4583
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4584
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4585 @MISC{Yoshua+al-snowbird-2008,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4586 author = {Bengio, Yoshua and Larochelle, Hugo and Turian, Joseph},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4587 title = {Deep Woods},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4588 year = {2008},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4589 howpublished = {Poster presented at the Learning@Snowbird Workshop, Snowbird, USA, 2008}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4590 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4591
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4592 @ARTICLE{Zaccaro-et-al-2005,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4593 author = {Zaccaro, Maria Clara and Boon, Hong and Pattarawarapan, Mookda and Xia, Zebin and Caron, Antoine and {L'Heureux}, Pierre-Jean and Bengio, Yoshua and Burgess, Kevin and Saragori, H. Uri},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4594 title = {Selective Small Molecule Peptidomimetic Ligands of TrkC and TrkA Receptors Afford Discrete or Complete Neurotrophic Activities},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4595 journal = {Chemistry \& Biology},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4596 volume = {12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4597 number = {9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4598 year = {2005},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4599 pages = {1015--1028}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4600 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4601
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4602
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4603
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4604 crossreferenced publications:
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4605 @INPROCEEDINGS{ICML09,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4606 editor = {Bottou, {L{\'{e}}on} and Littman, Michael},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4607 title = {Proceedings of the Twenty-sixth International Conference on Machine Learning (ICML'09)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4608 booktitle = {Proceedings of the Twenty-sixth International Conference on Machine Learning (ICML'09)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4609 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4610 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4611 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4612
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4613 @INPROCEEDINGS{NIPS7,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4614 editor = {Tesauro, G. and Touretzky, D. S. and Leen, T. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4615 title = {Advances in Neural Information Processing Systems 7 (NIPS'94)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4616 booktitle = {Advances in Neural Information Processing Systems 7 (NIPS'94)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4617 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4618 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4619 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4620
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4621 @INPROCEEDINGS{NIPS6,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4622 editor = {Cowan, J. D. and Tesauro, G. and Alspector, J.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4623 title = {Advances in Neural Information Processing Systems 6 (NIPS'93)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4624 booktitle = {Advances in Neural Information Processing Systems 6 (NIPS'93)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4625 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4626 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4627 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4628
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4629 @INPROCEEDINGS{NIPS8,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4630 editor = {Touretzky, D. S. and Mozer, M. and Hasselmo, M.E.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4631 title = {Advances in Neural Information Processing Systems 8 (NIPS'95)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4632 booktitle = {Advances in Neural Information Processing Systems 8 (NIPS'95)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4633 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4634 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4635 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4636
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4637 @ARTICLE{JMLR,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4638 journal = {Journal of Machine Learning Research},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4639 year = {-1}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4640 }
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4641
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4642 @INPROCEEDINGS{NIPS19,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4643 editor = {{Sch{\"{o}}lkopf}, Bernhard and Platt, John and Hoffman, Thomas},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4644 title = {Advances in Neural Information Processing Systems 19 (NIPS'06)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4645 booktitle = {Advances in Neural Information Processing Systems 19 (NIPS'06)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4646 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4647 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4648 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4649
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4650 @INPROCEEDINGS{NIPS10,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4651 editor = {Jordan, M.I. and Kearns, M.J. and Solla, S.A.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4652 title = {Advances in Neural Information Processing Systems 10 (NIPS'97)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4653 booktitle = {Advances in Neural Information Processing Systems 10 (NIPS'97)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4654 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4655 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4656 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4657
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4658 @INPROCEEDINGS{NIPS1,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4659 editor = {Touretzky, D. S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4660 title = {Advances in Neural Information Processing Systems 1 (NIPS'88)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4661 booktitle = {Advances in Neural Information Processing Systems 1 (NIPS'88)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4662 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4663 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4664 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4665
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4666 @INPROCEEDINGS{NIPS2,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4667 editor = {Touretzky, D. S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4668 title = {Advances in Neural Information Processing Systems 2 (NIPS'89)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4669 booktitle = {Advances in Neural Information Processing Systems 2 (NIPS'89)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4670 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4671 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4672 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4673
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4674 @INPROCEEDINGS{NIPS4,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4675 editor = {Moody, J. E. and Hanson, S. J. and Lipmann, R. P.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4676 title = {Advances in Neural Information Processing Systems 4 (NIPS'91)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4677 booktitle = {Advances in Neural Information Processing Systems 4 (NIPS'91)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4678 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4679 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4680 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4681
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4682 @INPROCEEDINGS{NIPS12,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4683 editor = {Solla, S.A. and Leen, T. K.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4684 title = {Advances in Neural Information Processing Systems 12 (NIPS'99)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4685 booktitle = {Advances in Neural Information Processing Systems 12 (NIPS'99)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4686 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4687 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4688 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4689
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4690 @INPROCEEDINGS{NIPS16,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4691 editor = {Becker, S. and Saul, L. and {Sch{\"{o}}lkopf}, Bernhard},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4692 title = {Advances in Neural Information Processing Systems 16 (NIPS'03)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4693 booktitle = {Advances in Neural Information Processing Systems 16 (NIPS'03)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4694 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4695 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4696
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4697 @INPROCEEDINGS{NIPS22,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4698 editor = {Bengio, Yoshua and Schuurmans, Dale and Williams, Christopher and Lafferty, John and Culotta, Aron},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4699 title = {Advances in Neural Information Processing Systems 22 (NIPS'09)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4700 booktitle = {Advances in Neural Information Processing Systems 22 (NIPS'09)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4701 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4702 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4703
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4704 @INPROCEEDINGS{NIPS20,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4705 editor = {Platt, John and Koller, D. and Singer, Yoram and Roweis, S.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4706 title = {Advances in Neural Information Processing Systems 20 (NIPS'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4707 booktitle = {Advances in Neural Information Processing Systems 20 (NIPS'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4708 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4709 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4710 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4711
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4712 @INPROCEEDINGS{xAISTATS2009,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4713 title = {Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS 2009)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4714 booktitle = {Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics (AISTATS 2009)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4715 year = {2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4716 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4717
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4718 @INPROCEEDINGS{NIPS9,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4719 editor = {Mozer, M. and Jordan, M.I. and Petsche, T.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4720 title = {Advances in Neural Information Processing Systems 9 (NIPS'96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4721 booktitle = {Advances in Neural Information Processing Systems 9 (NIPS'96)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4722 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4723 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4724 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4725
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4726 @INPROCEEDINGS{NIPS17,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4727 editor = {Saul, Lawrence K. and Weiss, Yair and Bottou, {L{\'{e}}on}},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4728 title = {Advances in Neural Information Processing Systems 17 (NIPS'04)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4729 booktitle = {Advances in Neural Information Processing Systems 17 (NIPS'04)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4730 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4731 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4732
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4733 @PROCEEDINGS{ICML08,
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4734 editor = {Cohen, William W. and McCallum, Andrew and Roweis, Sam T.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4735 title = {Proceedings of the Twenty-fifth International Conference on Machine Learning (ICML'08)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4736 booktitle = {Proceedings of the Twenty-fifth International Conference on Machine Learning (ICML'08)},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4737 year = {2008},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4738 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4739 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4740
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4741 @PROCEEDINGS{ICML07,
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4742 editor = {Ghahramani, Zoubin},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4743 title = {Proceedings of the 24th International Conference on Machine Learning (ICML'07)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4744 booktitle = {Proceedings of the 24th International Conference on Machine Learning (ICML'07)},
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4745 year = {2007},
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4746 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4747 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4748
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4749 @TECHREPORT{DIRO,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4750 title = {DIRO},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4751 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4752 institution = {D{\'{e}}partement d'Informatique et de Recherche Op{\'{e}}rationnelle, Universit{\'{e}} de Montr{\'{e}}al},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4753 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4754
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4755 @INPROCEEDINGS{NIPS18,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4756 editor = {Weiss, Yair and {Sch{\"{o}}lkopf}, Bernhard and Platt, John},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4757 title = {Advances in Neural Information Processing Systems 18 (NIPS'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4758 booktitle = {Advances in Neural Information Processing Systems 18 (NIPS'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4759 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4760 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4761 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4762
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4763 @INPROCEEDINGS{NIPS13,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4764 editor = {Leen, T. K. and Dietterich, T.G.},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4765 title = {Advances in Neural Information Processing Systems 13 (NIPS'00)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4766 booktitle = {Advances in Neural Information Processing Systems 13 (NIPS'00)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4767 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4768 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4769 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4770
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4771 @INPROCEEDINGS{ICML05,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4772 editor = {Raedt, Luc De and Wrobel, Stefan},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4773 title = {Proceedings of the Twenty-second International Conference on Machine Learning (ICML'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4774 booktitle = {Proceedings of the Twenty-second International Conference on Machine Learning (ICML'05)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4775 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4776 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4777 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4778
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4779 @INPROCEEDINGS{ICML06,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4780 editor = {Cohen, William W. and Moore, Andrew},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4781 title = {Proceedings of the Twenty-three International Conference on Machine Learning (ICML'06)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4782 booktitle = {Proceedings of the Twenty-three International Conference on Machine Learning (ICML'06)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4783 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4784 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4785 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4786
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4787 @INPROCEEDINGS{NIPS15,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4788 editor = {Becker, S. and Thrun, Sebastian},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4789 title = {Advances in Neural Information Processing Systems 15 (NIPS'02)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4790 booktitle = {Advances in Neural Information Processing Systems 15 (NIPS'02)},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4791 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4792 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4793 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4794
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4795 @INPROCEEDINGS{ICML01-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4796 title = {ICML'01},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4797 booktitle = {ICML'01},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4798 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4799 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4800 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4801 @INPROCEEDINGS{ICML02-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4802 title = {ICML'02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4803 booktitle = {ICML'02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4804 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4805 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4806 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4807 @INPROCEEDINGS{ICML03-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4808 title = {ICML'03},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4809 booktitle = {ICML'03},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4810 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4811 publisher = {AAAI Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4812 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4813 @INPROCEEDINGS{ICML04-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4814 title = {ICML'04},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4815 booktitle = {ICML'04},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4816 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4817 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4818 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4819 @INPROCEEDINGS{ICML05-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4820 title = {ICML'05},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4821 booktitle = {ICML'05},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4822 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4823 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4824 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4825 @INPROCEEDINGS{ICML06-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4826 title = {ICML'06},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4827 booktitle = {ICML'06},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4828 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4829 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4830 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4831 @INPROCEEDINGS{ICML07-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4832 title = {ICML'07},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4833 booktitle = {ICML'07},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4834 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4835 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4836 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4837 @INPROCEEDINGS{ICML08-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4838 title = {ICML'08},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4839 booktitle = {ICML'08},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4840 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4841 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4842 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4843 @INPROCEEDINGS{ICML09-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4844 title = {ICML'09},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4845 booktitle = {ICML'09},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4846 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4847 publisher = {ACM}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4848 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4849 @INPROCEEDINGS{ICML96-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4850 title = {ICML'96},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4851 booktitle = {ICML'96},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4852 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4853 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4854 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4855 @INPROCEEDINGS{ICML97-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4856 title = {ICML'97},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4857 booktitle = {ICML'97},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4858 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4859 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4860 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4861 @INPROCEEDINGS{ICML98-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4862 title = {ICML'98},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4863 booktitle = {ICML'98},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4864 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4865 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4866 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4867 @INPROCEEDINGS{ICML99-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4868 title = {ICML'99},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4869 booktitle = {ICML'99},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4870 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4871 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4872 }
604
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4873 @ARTICLE{JMLR-shorter,
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4874 journal = {JMLR},
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4875 year = {-1}
51213beaed8b draft of NIPS 2010 workshop camera-ready version
Yoshua Bengio <bengioy@iro.umontreal.ca>
parents: 583
diff changeset
4876 }
498
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4877 @INPROCEEDINGS{NIPS1-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4878 title = {NIPS'88},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4879 booktitle = {NIPS 1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4880 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4881 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4882 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4883 @INPROCEEDINGS{NIPS10-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4884 title = {NIPS'97},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4885 booktitle = {NIPS 10},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4886 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4887 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4888 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4889 @INPROCEEDINGS{NIPS11-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4890 title = {NIPS'98},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4891 booktitle = {NIPS 11},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4892 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4893 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4894 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4895 @INPROCEEDINGS{NIPS12-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4896 title = {NIPS'99},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4897 booktitle = {NIPS 12},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4898 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4899 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4900 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4901 @INPROCEEDINGS{NIPS13-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4902 title = {NIPS'00},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4903 booktitle = {NIPS 13},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4904 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4905 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4906 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4907 @INPROCEEDINGS{NIPS14-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4908 title = {NIPS'01},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4909 booktitle = {NIPS 14},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4910 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4911 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4912 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4913 @INPROCEEDINGS{NIPS15-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4914 title = {NIPS'02},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4915 booktitle = {NIPS 15},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4916 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4917 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4918 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4919 @INPROCEEDINGS{NIPS16-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4920 title = {NIPS'03},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4921 booktitle = {NIPS 16},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4922 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4923 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4924 @INPROCEEDINGS{NIPS17-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4925 title = {NIPS'04},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4926 booktitle = {NIPS 17},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4927 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4928 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4929 @INPROCEEDINGS{NIPS18-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4930 title = {NIPS'05},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4931 booktitle = {NIPS 18},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4932 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4933 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4934 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4935 @INPROCEEDINGS{NIPS19-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4936 title = {NIPS'06},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4937 booktitle = {NIPS 19},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4938 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4939 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4940 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4941 @INPROCEEDINGS{NIPS2-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4942 title = {NIPS'89},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4943 booktitle = {NIPS 2},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4944 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4945 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4946 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4947 @INPROCEEDINGS{NIPS20-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4948 title = {NIPS'07},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4949 booktitle = {NIPS 20},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4950 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4951 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4952 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4953 @INPROCEEDINGS{NIPS21-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4954 title = {NIPS'08},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4955 booktitle = {NIPS 21},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4956 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4957 publisher = {Nips Foundation (http://books.nips.cc)}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4958 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4959 @INPROCEEDINGS{NIPS22-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4960 title = {NIPS'09},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4961 booktitle = {NIPS 22},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4962 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4963 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4964 @INPROCEEDINGS{NIPS3-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4965 title = {NIPS'90},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4966 booktitle = {NIPS 3},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4967 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4968 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4969 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4970 @INPROCEEDINGS{NIPS4-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4971 title = {NIPS'91},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4972 booktitle = {NIPS 4},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4973 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4974 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4975 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4976 @INPROCEEDINGS{NIPS5-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4977 title = {NIPS'92},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4978 booktitle = {NIPS 5},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4979 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4980 publisher = {Morgan Kaufmann}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4981 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4982 @INPROCEEDINGS{NIPS6-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4983 title = {NIPS'93},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4984 booktitle = {NIPS 6},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4985 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4986 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4987 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4988 @INPROCEEDINGS{NIPS7-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4989 title = {NIPS'94},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4990 booktitle = {NIPS 7},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4991 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4992 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4993 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4994 @INPROCEEDINGS{NIPS8-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4995 title = {NIPS'95},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4996 booktitle = {NIPS 8},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4997 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4998 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
4999 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5000 @INPROCEEDINGS{NIPS9-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5001 title = {NIPS'96},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5002 booktitle = {NIPS 9},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5003 year = {-1},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5004 publisher = {MIT Press}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5005 }
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5006 @INPROCEEDINGS{xAISTATS2009-shorter,
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5007 title = {AISTATS'2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5008 booktitle = {AISTATS'2009},
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5009 year = {-1}
7ff00c27c976 add missing file for bibtex and make it smaller.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
5010 }