pylearn: _test_nnet_ops.py annotate

annotate _test_nnet_ops.py @ 444:9cfc2fc0f4d1

New tests for softmax.

author	Pascal Lamblin <lamblinp@iro.umontreal.ca>
date	Fri, 22 Aug 2008 17:33:19 -0400
parents	43d9aa93934e
children

rev	line source
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	1
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	2 import unittest
419 43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	3 import theano
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	4 import theano._test_tensor as TT
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	5 import numpy
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	6
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	7 from nnet_ops import *
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	8
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	9 class T_sigmoid(unittest.TestCase):
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	10 def setUp(self):
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	11 numpy.random.seed(9999)
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	12 def test_elemwise(self):
117 3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	13 TT.verify_grad(self, sigmoid, [numpy.random.rand(3,4)])
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	14
69 8c2607f387e6 added softplus, elaborated sigmoid James Bergstra <bergstrj@iro.umontreal.ca> parents: 30 diff changeset	15 class T_softplus(unittest.TestCase):
8c2607f387e6 added softplus, elaborated sigmoid James Bergstra <bergstrj@iro.umontreal.ca> parents: 30 diff changeset	16 def setUp(self):
8c2607f387e6 added softplus, elaborated sigmoid James Bergstra <bergstrj@iro.umontreal.ca> parents: 30 diff changeset	17 numpy.random.seed(9999)
8c2607f387e6 added softplus, elaborated sigmoid James Bergstra <bergstrj@iro.umontreal.ca> parents: 30 diff changeset	18 def test_elemwise(self):
117 3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	19 TT.verify_grad(self, softplus, [numpy.random.rand(3,4)])
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	20
444 9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	21 class T_Softmax(unittest.TestCase):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	22 def setUp(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	23 numpy.random.seed(9999)
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	24 def test0(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	25 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	26 def make_node(self, a):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	27 return [softmax(a)[:,0]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	28 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	29 def test1(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	30 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	31 def make_node(self, a):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	32 return [softmax(a)[:,1]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	33 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	34 def test2(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	35 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	36 def make_node(self, a):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	37 return [softmax(a)[:,2]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	38 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	39 def test3(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	40 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	41 def make_node(self, a):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	42 return [softmax(a)[:,3]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	43 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	44
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	45
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	46 class T_SoftmaxWithBias(unittest.TestCase):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	47 def setUp(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	48 numpy.random.seed(9999)
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	49 def test0(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	50 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	51 def make_node(self, a, b):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	52 return [softmax_with_bias(a, b)[:,0]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	53 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4),
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	54 numpy.random.rand(4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	55 def test1(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	56 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	57 def make_node(self, a, b):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	58 return [softmax_with_bias(a, b)[:,1]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	59 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4),
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	60 numpy.random.rand(4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	61 def test2(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	62 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	63 def make_node(self, a, b):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	64 return [softmax_with_bias(a, b)[:,2]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	65 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4),
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	66 numpy.random.rand(4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	67 def test3(self):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	68 class Dummy(object):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	69 def make_node(self, a, b):
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	70 return [softmax_with_bias(a, b)[:,3]]
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	71 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4),
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	72 numpy.random.rand(4)])
9cfc2fc0f4d1 New tests for softmax. Pascal Lamblin <lamblinp@iro.umontreal.ca> parents: 419 diff changeset	73
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	74 class T_CrossentropySoftmax1Hot(unittest.TestCase):
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	75 def setUp(self):
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	76 numpy.random.seed(9999)
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	77 def test0(self):
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	78 y_idx = [0,1,3]
117 3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	79 class Dummy(object):
3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	80 def make_node(self, a,b):
3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	81 return crossentropy_softmax_1hot_with_bias(a, b, y_idx)[0:1]
3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	82 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4),
30 bf0145fa73e8 added c implementation for CrossentropySoftmax1Hot bergstrj@iro.umontreal.ca parents: 25 diff changeset	83 numpy.random.rand(4)])
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	84
70 76e5c0f37165 better docs & precondition testing for cross_entropy_softmax_1hot & friends James Bergstra <bergstrj@iro.umontreal.ca> parents: 69 diff changeset	85 def test1(self):
76e5c0f37165 better docs & precondition testing for cross_entropy_softmax_1hot & friends James Bergstra <bergstrj@iro.umontreal.ca> parents: 69 diff changeset	86 y_idx = [0,1,3]
117 3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	87 class Dummy(object):
3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	88 def make_node(self, a):
3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	89 return crossentropy_softmax_1hot(a, y_idx)[0:1]
3ef569b92fba ported nnet_ops to new theano James Bergstra <bergstrj@iro.umontreal.ca> parents: 70 diff changeset	90 TT.verify_grad(self, Dummy(), [numpy.random.rand(3,4)])
70 76e5c0f37165 better docs & precondition testing for cross_entropy_softmax_1hot & friends James Bergstra <bergstrj@iro.umontreal.ca> parents: 69 diff changeset	91
419 43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	92 class T_prepend(unittest.TestCase):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	93 def test0(self):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	94 """basic functionality"""
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	95 x=tensor.matrix('x')
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	96 y=Prepend_scalar_constant_to_each_row(4.)(x)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	97 f=theano.function([x],[y])
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	98 m=numpy.random.rand(3,5)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	99 my = f(m)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	100 self.failUnless(my.shape == (3, 6), my.shape)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	101 self.failUnless(numpy.all( my[:,0] == 4.0))
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	102
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	103
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	104 class T_prepend(unittest.TestCase):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	105 def test0(self):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	106 """basic functionality"""
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	107 x=tensor.matrix('x')
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	108 y=Prepend_scalar_to_each_row()(5.,x)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	109 f=theano.function([x],[y])
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	110 m=numpy.ones((3,5),dtype="float32")
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	111 my = f(m)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	112 self.failUnless(str(my.dtype) == 'float64')
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	113 self.failUnless(my.shape == (3, 6))
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	114 self.failUnless(numpy.all(my[:,0] == 5.0))
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	115
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	116 class T_solve(unittest.TestCase):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	117 def setUp(self):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	118 self.rng = numpy.random.RandomState(666)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	119
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	120 def test0(self):
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	121 A=self.rng.randn(5,5)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	122 b=numpy.array(range(5),dtype=float)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	123 x=numpy.linalg.solve(A,b)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	124 Ax = numpy.dot(A,x)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	125 are = theano.gradient.numeric_grad.abs_rel_err(Ax, b)
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	126 self.failUnless(numpy.all(are < 1.0e-5), (are, Ax, b))
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	127 #print A,b
43d9aa93934e added other_ops.py to nnet_ops; added basic tests, no docs. James Bergstra <bergstrj@iro.umontreal.ca> parents: 286 diff changeset	128 #print numpy.dot(A,x)
24 2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	129
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	130
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	131 if __name__ == '__main__':
2e8be9f5412b added nnet_ops bergstrj@iro.umontreal.ca parents: diff changeset	132 unittest.main()

Mercurial > pylearn

annotate _test_nnet_ops.py @ 444:9cfc2fc0f4d1