Mercurial > pylearn

annotate pylearn/formulas/noise.py @ 1486:cb2e07d99f5a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
switched inp==0 to T.eq(inp,0) in peppersalt noise
author Eric Thibodeau-Laufer <thiboeri@iro.umontreal.ca>
date Tue, 05 Jul 2011 14:31:10 -0400
parents 9a7dbcd0ebcf
children
rev   line source
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
1 """
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
2 Noise functions used to train Denoising Auto-Associators.
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
3
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
4 Functions in this module often include a `noise_lvl` argument that controls the amount of noise
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
5 that the function applies.
1122
be53f56b37b8 Added example cost formula with tags and created a formulas page in the doc
boulanni <nicolas_boulanger@hotmail.com>
parents: 1114
diff changeset
6 The noise contract is simple: noise_lvl is a symbolic variable going from 0 to 1.
1296
b9bc9d5a08cc Some language fixes (don't capitalize 'variable', changement->change.
David Warde-Farley <wardefar@iro.umontreal.ca>
parents: 1208
diff changeset
7 0: no change.
b9bc9d5a08cc Some language fixes (don't capitalize 'variable', changement->change.
David Warde-Farley <wardefar@iro.umontreal.ca>
parents: 1208
diff changeset
8 1: maximum noise.
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
9 """
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
10 import theano
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
11 import tags
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
12 s="""
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
13 * A latex mathematical description of the formulas(for picture representation in generated documentation)
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
14 * Tags(for searching):
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
15 * a list of lower lovel fct used
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
16 * category(name of the submodule itself)
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
17 * Tell if we did some work to make it more numerical stable. Do theano do the optimization needed?
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
18 * Tell if the grad is numericaly stable? Do theano do the optimization needed?
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
19 * Tell if work on gpu/not/unknow
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
20 * Tell alternate name
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
21 * Tell the domaine, range of the input/output(range should use the english notation of including or excluding)
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
22 """
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
23
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
24 @tags.tags('noise','binomial','salt')
1326
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
25 def binomial_noise(theano_rng, input, noise_lvl, noise_value=0):
1296
b9bc9d5a08cc Some language fixes (don't capitalize 'variable', changement->change.
David Warde-Farley <wardefar@iro.umontreal.ca>
parents: 1208
diff changeset
26 """
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
27 Return `inp` with randomly-chosen elements set to zero.
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
28
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
29 TODO: MATH DEFINITION
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
30
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
31 :type input: Theano tensor variable
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
32 :param input: input
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
33 :type noise_lvl: float
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
34 :param noise_lvl: The probability of setting each element to zero.
1326
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
35 :type noise_value: Theano scalar variable
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
36 :param noise_value: The value that we want when their is noise.
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
37 """
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
38 mask = theano_rng.binomial(
1326
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
39 size = input.shape,
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
40 n = 1,
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
41 p = 1 - noise_lvl,
1326
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
42 dtype=input.dtype)
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
43 value = theano.tensor.as_tensor_variable(noise_value)
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
44 if value.type.ndim!=0:
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
45 raise Exception('binomial_noise only support scalar noise_value')
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
46 if noise_value==0:
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
47 return mask * input
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
48 else:
1b97fae7ea0d added the parameter noise_value to binomial_noise formula.
Frederic Bastien <nouiz@nouiz.org>
parents: 1301
diff changeset
49 return mask * input + noise_value*(not mask)
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
50
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
51
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
52 @tags.tags('noise','binomial NLP','pepper','salt')
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
53 def pepper_and_salt_noise(theano_rng,inp,noise_lvl):
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
54 """ This add pepper and salt noise to inp
1296
b9bc9d5a08cc Some language fixes (don't capitalize 'variable', changement->change.
David Warde-Farley <wardefar@iro.umontreal.ca>
parents: 1208
diff changeset
55
b9bc9d5a08cc Some language fixes (don't capitalize 'variable', changement->change.
David Warde-Farley <wardefar@iro.umontreal.ca>
parents: 1208
diff changeset
56 :type inp: Theano variable
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
57 :param inp: The input that we want to add noise
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
58 :type noise_lvl: tuple(float,float)
1327
9a7dbcd0ebcf make function documentation more clear.
Frederic Bastien <nouiz@nouiz.org>
parents: 1326
diff changeset
59 :param noise_lvl: The probability of changing each element to zero or one.
9a7dbcd0ebcf make function documentation more clear.
Frederic Bastien <nouiz@nouiz.org>
parents: 1326
diff changeset
60 (prob of salt, prob of pepper)
9a7dbcd0ebcf make function documentation more clear.
Frederic Bastien <nouiz@nouiz.org>
parents: 1326
diff changeset
61
9a7dbcd0ebcf make function documentation more clear.
Frederic Bastien <nouiz@nouiz.org>
parents: 1326
diff changeset
62 :note: The sum of the prob of salt and prob of pepper should be less then 1.
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
63 """
1301
1a3090eca2ec make the noise type the same as the input.
Frederic Bastien <nouiz@nouiz.org>
parents: 1297
diff changeset
64 assert inp.dtype in ['float32','float64']
1a3090eca2ec make the noise type the same as the input.
Frederic Bastien <nouiz@nouiz.org>
parents: 1297
diff changeset
65 return theano_rng.binomial( size = inp.shape, n = 1, p = 1 - noise_lvl[0], dtype=inp.dtype) * inp \
1486
cb2e07d99f5a switched inp==0 to T.eq(inp,0) in peppersalt noise
Eric Thibodeau-Laufer <thiboeri@iro.umontreal.ca>
parents: 1327
diff changeset
66 + (theano.tensor.eq(inp,0)) * theano_rng.binomial( size = inp.shape, n = 1, p = noise_lvl[1], dtype=inp.dtype)
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
67
1297
24890ca1d96b small changes to formulas to bring them closer to style recommendation
James Bergstra <bergstrj@iro.umontreal.ca>
parents: 1296
diff changeset
68 @tags.tags('noise','gauss','gaussian')
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
69 def gaussian_noise(theano_rng,inp,noise_lvl):
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
70 """ This add gaussian NLP noise to inp
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
71
1296
b9bc9d5a08cc Some language fixes (don't capitalize 'variable', changement->change.
David Warde-Farley <wardefar@iro.umontreal.ca>
parents: 1208
diff changeset
72 :type inp: Theano variable
1114
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
73 :param inp: The input that we want to add noise
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
74 :type noise_lvl: float
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
75 :param noise_lvl: The standard deviation of the gaussian.
de153244c8e5 added example file for the formulas.
Frederic Bastien <nouiz@nouiz.org>
parents:
diff changeset
76 """
1301
1a3090eca2ec make the noise type the same as the input.
Frederic Bastien <nouiz@nouiz.org>
parents: 1297
diff changeset
77 assert inp.dtype in ['float32','float64']
1a3090eca2ec make the noise type the same as the input.
Frederic Bastien <nouiz@nouiz.org>
parents: 1297
diff changeset
78 return theano_rng.normal( size = inp.shape, std = noise_lvl, dtype=inp.dtype) + inp