Mercurial > ift6266

annotate baseline/mlp/mlp_get_error_from_model.py @ 212:e390b0454515

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
added classic lr time decay and py code to calculate the error based on a saved model
author xaviermuller
date Wed, 10 Mar 2010 16:17:59 -0500
parents
children 9b6e0af062af
rev   line source
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
1 __docformat__ = 'restructedtext en'
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
2
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
3 import pdb
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
4 import numpy as np
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
5 import pylab
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
6 import time
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
7 import pylearn
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
8 from pylearn.io import filetensor as ft
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
9
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
10 data_path = '/data/lisa/data/nist/by_class/'
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
11 test_data = 'all/all_test_data.ft'
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
12 test_labels = 'all/all_test_labels.ft'
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
13
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
14 def read_test_data(mlp_model):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
15
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
16
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
17 #read the data
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
18 h = open(data_path+test_data)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
19 i= open(data_path+test_labels)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
20 raw_test_data = ft.read(h)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
21 raw_test_labels = ft.read(i)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
22 i.close()
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
23 h.close()
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
24
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
25 #read the model chosen
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
26 a=np.load(mlp_model)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
27 W1=a['W1']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
28 W2=a['W2']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
29 b1=a['b1']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
30 b2=a['b2']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
31
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
32 return (W1,b1,W2,b2,raw_test_data,raw_test_labels)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
33
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
34
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
35
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
36
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
37 def get_total_test_error(everything):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
38
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
39 W1=everything[0]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
40 b1=everything[1]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
41 W2=everything[2]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
42 b2=everything[3]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
43 test_data=everything[4]/255.0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
44 test_labels=everything[5]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
45 total_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
46 total_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
47
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
48 nb_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
49 nb_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
50
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
51 char_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
52 char_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
53
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
54 min_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
55 min_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
56
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
57 maj_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
58 maj_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
59
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
60 for i in range(test_labels.size):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
61 total_exemple_count = total_exemple_count +1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
62 #get activation for layer 1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
63 a0=np.dot(np.transpose(W1),np.transpose(test_data[i])) + b1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
64 #add non linear function to layer 1 activation
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
65 a0_out=np.tanh(a0)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
66
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
67 #get activation for output layer
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
68 a1= np.dot(np.transpose(W2),a0_out) + b2
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
69 #add non linear function for output activation (softmax)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
70 a1_exp = np.exp(a1)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
71 sum_a1=np.sum(a1_exp)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
72 a1_out=a1_exp/sum_a1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
73
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
74 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
75 wanted_class=test_labels[i]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
76
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
77 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
78 total_error_count = total_error_count +1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
79
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
80 #get grouped based error
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
81 if(wanted_class>9 and wanted_class<35):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
82 min_exemple_count=min_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
83 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
84 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
85 min_error_count=min_error_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
86 elif(wanted_class<10):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
87 nb_exemple_count=nb_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
88 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
89 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
90 nb_error_count=nb_error_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
91 elif(wanted_class>34):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
92 maj_exemple_count=maj_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
93 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
94 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
95 maj_error_count=maj_error_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
96
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
97 if(wanted_class>9):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
98 char_exemple_count=char_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
99 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
100 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
101 char_error_count=char_error_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
102
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
103
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
104 #convert to float
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
105 return ( total_exemple_count,nb_exemple_count,char_exemple_count,min_exemple_count,maj_exemple_count,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
106 total_error_count,nb_error_count,char_error_count,min_error_count,maj_error_count,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
107 total_error_count*100.0/total_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
108 nb_error_count*100.0/nb_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
109 char_error_count*100.0/char_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
110 min_error_count*100.0/min_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
111 maj_error_count*100.0/maj_exemple_count*1.0)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
112
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
113
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
114
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
115
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
116
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
117
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
118
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
119
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
120
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
121
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
122
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
123
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
124