Mercurial > ift6266

annotate baseline/mlp/mlp_get_error_from_model.py @ 239:42005ec87747

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Mergé (manuellement) les changements de Sylvain pour utiliser le code de dataset d'Arnaud, à cette différence près que je n'utilse pas les givens. J'ai probablement une approche différente pour limiter la taille du dataset dans mon débuggage, aussi.
author fsavard
date Mon, 15 Mar 2010 18:30:21 -0400
parents 9b6e0af062af
children
rev   line source
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
1 __docformat__ = 'restructedtext en'
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
2
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
3 import pdb
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
4 import numpy as np
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
5 import pylab
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
6 import time
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
7 import pylearn
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
8 from pylearn.io import filetensor as ft
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
9
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
10 data_path = '/data/lisa/data/nist/by_class/'
237
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
11 test_data = 'all/all_train_data.ft'
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
12 test_labels = 'all/all_train_labels.ft'
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
13
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
14 def read_test_data(mlp_model):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
15
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
16
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
17 #read the data
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
18 h = open(data_path+test_data)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
19 i= open(data_path+test_labels)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
20 raw_test_data = ft.read(h)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
21 raw_test_labels = ft.read(i)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
22 i.close()
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
23 h.close()
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
24
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
25 #read the model chosen
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
26 a=np.load(mlp_model)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
27 W1=a['W1']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
28 W2=a['W2']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
29 b1=a['b1']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
30 b2=a['b2']
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
31
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
32 return (W1,b1,W2,b2,raw_test_data,raw_test_labels)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
33
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
34
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
35
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
36
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
37 def get_total_test_error(everything):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
38
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
39 W1=everything[0]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
40 b1=everything[1]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
41 W2=everything[2]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
42 b2=everything[3]
237
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
43 test_data=everything[4]
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
44 test_labels=everything[5]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
45 total_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
46 total_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
47
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
48 nb_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
49 nb_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
50
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
51 char_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
52 char_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
53
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
54 min_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
55 min_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
56
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
57 maj_error_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
58 maj_exemple_count=0
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
59
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
60 for i in range(test_labels.size):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
61 total_exemple_count = total_exemple_count +1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
62 #get activation for layer 1
237
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
63 a0=np.dot(np.transpose(W1),np.transpose(test_data[i]/255.0)) + b1
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
64 #add non linear function to layer 1 activation
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
65 a0_out=np.tanh(a0)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
66
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
67 #get activation for output layer
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
68 a1= np.dot(np.transpose(W2),a0_out) + b2
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
69 #add non linear function for output activation (softmax)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
70 a1_exp = np.exp(a1)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
71 sum_a1=np.sum(a1_exp)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
72 a1_out=a1_exp/sum_a1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
73
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
74 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
75 wanted_class=test_labels[i]
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
76
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
77 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
78 total_error_count = total_error_count +1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
79
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
80 #get grouped based error
237
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
81 #with a priori
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
82 # if(wanted_class>9 and wanted_class<35):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
83 # min_exemple_count=min_exemple_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
84 # predicted_class=np.argmax(a1_out[10:35])+10
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
85 # if(predicted_class!=wanted_class):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
86 # min_error_count=min_error_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
87 # if(wanted_class<10):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
88 # nb_exemple_count=nb_exemple_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
89 # predicted_class=np.argmax(a1_out[0:10])
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
90 # if(predicted_class!=wanted_class):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
91 # nb_error_count=nb_error_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
92 # if(wanted_class>34):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
93 # maj_exemple_count=maj_exemple_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
94 # predicted_class=np.argmax(a1_out[35:])+35
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
95 # if(predicted_class!=wanted_class):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
96 # maj_error_count=maj_error_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
97 #
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
98 # if(wanted_class>9):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
99 # char_exemple_count=char_exemple_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
100 # predicted_class=np.argmax(a1_out[10:])+10
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
101 # if(predicted_class!=wanted_class):
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
102 # char_error_count=char_error_count+1
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
103
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
104
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
105
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
106 #get grouped based error
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
107 #with no a priori
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
108 if(wanted_class>9 and wanted_class<35):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
109 min_exemple_count=min_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
110 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
111 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
112 min_error_count=min_error_count+1
237
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
113 if(wanted_class<10):
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
114 nb_exemple_count=nb_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
115 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
116 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
117 nb_error_count=nb_error_count+1
237
9b6e0af062af corrected a bug in jobman interface
xaviermuller
parents: 212
diff changeset
118 if(wanted_class>34):
212
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
119 maj_exemple_count=maj_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
120 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
121 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
122 maj_error_count=maj_error_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
123
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
124 if(wanted_class>9):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
125 char_exemple_count=char_exemple_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
126 predicted_class=np.argmax(a1_out)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
127 if(predicted_class!=wanted_class):
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
128 char_error_count=char_error_count+1
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
129
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
130
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
131 #convert to float
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
132 return ( total_exemple_count,nb_exemple_count,char_exemple_count,min_exemple_count,maj_exemple_count,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
133 total_error_count,nb_error_count,char_error_count,min_error_count,maj_error_count,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
134 total_error_count*100.0/total_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
135 nb_error_count*100.0/nb_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
136 char_error_count*100.0/char_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
137 min_error_count*100.0/min_exemple_count*1.0,\
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
138 maj_error_count*100.0/maj_exemple_count*1.0)
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
139
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
140
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
141
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
142
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
143
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
144
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
145
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
146
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
147
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
148
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
149
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
150
e390b0454515 added classic lr time decay and py code to calculate the error based on a saved model
xaviermuller
parents:
diff changeset
151