Mercurial > ift6266

annotate utils/seriestables/test_series.py @ 224:0515a8901c6a

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
author fsavard
date Thu, 11 Mar 2010 11:52:43 -0500
parents e172ef73cdc5
children bfe20d63f88c
rev   line source
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
1 import tempfile
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
2
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
3 import numpy
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
4 import numpy.random
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
5
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
6 from jobman import DD
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
7
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
8 import tables
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
9
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
10 from series import *
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
11 import series
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
12
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
13 #################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
14 # Utils
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
15
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
16 def compare_floats(f1,f2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
17 if f1-f2 < 1e-3:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
18 return True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
19 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
20
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
21 def compare_lists(it1, it2, floats=False):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
22 if len(it1) != len(it2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
23 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
24
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
25 for el1, el2 in zip(it1, it2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
26 if floats:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
27 if not compare_floats(el1,el2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
28 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
29 elif el1 != el2:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
30 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
31
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
32 return True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
33
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
34 #################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
35 # Basic Series class tests
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
36
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
37 def test_Series_types():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
38 pass
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
39
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
40 #################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
41 # ErrorSeries tests
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
42
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
43 def test_ErrorSeries_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
44 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
45 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
46 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
47
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
48 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error",
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
49 hdf5_file=h5f, index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
50 title="Validation error indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
51
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
52 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
53 validation_error.append((1,1), 32.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
54 validation_error.append((1,2), 30.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
55 validation_error.append((2,1), 28.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
56 validation_error.append((2,2), 26.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
57
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
58 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
59
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
60 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
61
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
62 table = h5f.getNode('/', 'validation_error')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
63
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
64 assert compare_lists(table.cols.epoch[:], [1,1,2,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
65 assert compare_lists(table.cols.minibatch[:], [1,2,1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
66 assert compare_lists(table.cols.validation_error[:], [32.0, 30.0, 28.0, 26.0])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
67
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
68 def test_ErrorSeries_notimestamp(h5f=None):
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
69 if not h5f:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
70 h5f_path = tempfile.NamedTemporaryFile().name
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
71 h5f = tables.openFile(h5f_path, "w")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
72
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
73 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
74 hdf5_file=h5f, index_names=('epoch','minibatch'),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
75 title="Validation error indexed by epoch and minibatch",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
76 store_timestamp=False)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
77
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
78 # (1,1), (1,2) etc. are (epoch, minibatch) index
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
79 validation_error.append((1,1), 32.0)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
80
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
81 h5f.close()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
82
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
83 h5f = tables.openFile(h5f_path, "r")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
84
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
85 table = h5f.getNode('/', 'validation_error')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
86
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
87 assert compare_lists(table.cols.epoch[:], [1])
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
88 assert not ("timestamp" in dir(table.cols))
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
89 assert "cpuclock" in dir(table.cols)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
90
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
91 def test_ErrorSeries_nocpuclock(h5f=None):
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
92 if not h5f:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
93 h5f_path = tempfile.NamedTemporaryFile().name
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
94 h5f = tables.openFile(h5f_path, "w")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
95
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
96 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
97 hdf5_file=h5f, index_names=('epoch','minibatch'),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
98 title="Validation error indexed by epoch and minibatch",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
99 store_cpuclock=False)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
100
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
101 # (1,1), (1,2) etc. are (epoch, minibatch) index
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
102 validation_error.append((1,1), 32.0)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
103
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
104 h5f.close()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
105
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
106 h5f = tables.openFile(h5f_path, "r")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
107
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
108 table = h5f.getNode('/', 'validation_error')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
109
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
110 assert compare_lists(table.cols.epoch[:], [1])
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
111 assert not ("cpuclock" in dir(table.cols))
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
112 assert "timestamp" in dir(table.cols)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
113
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
114 def test_AccumulatorSeriesWrapper_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
115 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
116 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
117 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
118
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
119 validation_error = ErrorSeries(error_name="accumulated_validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
120 table_name="accumulated_validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
121 hdf5_file=h5f,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
122 index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
123 title="Validation error, summed every 3 minibatches, indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
124
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
125 accumulator = AccumulatorSeriesWrapper(base_series=validation_error,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
126 reduce_every=3, reduce_function=numpy.sum)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
127
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
128 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
129 accumulator.append((1,1), 32.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
130 accumulator.append((1,2), 30.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
131 accumulator.append((2,1), 28.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
132 accumulator.append((2,2), 26.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
133 accumulator.append((3,1), 24.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
134 accumulator.append((3,2), 22.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
135
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
136 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
137
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
138 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
139
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
140 table = h5f.getNode('/', 'accumulated_validation_error')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
141
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
142 assert compare_lists(table.cols.epoch[:], [2,3])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
143 assert compare_lists(table.cols.minibatch[:], [1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
144 assert compare_lists(table.cols.accumulated_validation_error[:], [90.0,72.0], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
145
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
146 def test_BasicStatisticsSeries_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
147 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
148 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
149 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
150
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
151 stats_series = BasicStatisticsSeries(table_name="b_vector_statistics",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
152 hdf5_file=h5f, index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
153 title="Basic statistics for b vector indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
154
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
155 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
156 stats_series.append((1,1), [0.15, 0.20, 0.30])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
157 stats_series.append((1,2), [-0.18, 0.30, 0.58])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
158 stats_series.append((2,1), [0.18, -0.38, -0.68])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
159 stats_series.append((2,2), [0.15, 0.02, 1.9])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
160
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
161 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
162
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
163 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
164
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
165 table = h5f.getNode('/', 'b_vector_statistics')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
166
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
167 assert compare_lists(table.cols.epoch[:], [1,1,2,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
168 assert compare_lists(table.cols.minibatch[:], [1,2,1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
169 assert compare_lists(table.cols.mean[:], [0.21666667, 0.23333333, -0.29333332, 0.69], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
170 assert compare_lists(table.cols.min[:], [0.15000001, -0.18000001, -0.68000001, 0.02], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
171 assert compare_lists(table.cols.max[:], [0.30, 0.58, 0.18, 1.9], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
172 assert compare_lists(table.cols.std[:], [0.06236095, 0.31382939, 0.35640177, 0.85724366], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
173
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
174 def test_SharedParamsStatisticsWrapper_commoncase(h5f=None):
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
175 import numpy.random
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
176
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
177 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
178 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
179 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
180
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
181 stats = SharedParamsStatisticsWrapper(new_group_name="params", base_group="/",
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
182 arrays_names=('b1','b2','b3'), hdf5_file=h5f,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
183 index_names=('epoch','minibatch'))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
184
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
185 b1 = DD({'value':numpy.random.rand(5)})
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
186 b2 = DD({'value':numpy.random.rand(5)})
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
187 b3 = DD({'value':numpy.random.rand(5)})
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
188 stats.append((1,1), [b1,b2,b3])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
189
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
190 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
191
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
192 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
193
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
194 b1_table = h5f.getNode('/params', 'b1')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
195 b3_table = h5f.getNode('/params', 'b3')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
196
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
197 assert b1_table.cols.mean[0] - numpy.mean(b1.value) < 1e-3
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
198 assert b3_table.cols.mean[0] - numpy.mean(b3.value) < 1e-3
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
199 assert b1_table.cols.min[0] - numpy.min(b1.value) < 1e-3
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
200 assert b3_table.cols.min[0] - numpy.min(b3.value) < 1e-3
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
201
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
202 def test_SharedParamsStatisticsWrapper_notimestamp(h5f=None):
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
203 import numpy.random
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
204
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
205 if not h5f:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
206 h5f_path = tempfile.NamedTemporaryFile().name
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
207 h5f = tables.openFile(h5f_path, "w")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
208
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
209 stats = SharedParamsStatisticsWrapper(new_group_name="params", base_group="/",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
210 arrays_names=('b1','b2','b3'), hdf5_file=h5f,
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
211 index_names=('epoch','minibatch'),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
212 store_timestamp=False)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
213
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
214 b1 = DD({'value':numpy.random.rand(5)})
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
215 b2 = DD({'value':numpy.random.rand(5)})
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
216 b3 = DD({'value':numpy.random.rand(5)})
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
217 stats.append((1,1), [b1,b2,b3])
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
218
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
219 h5f.close()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
220
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
221 h5f = tables.openFile(h5f_path, "r")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
222
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
223 b1_table = h5f.getNode('/params', 'b1')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
224 b3_table = h5f.getNode('/params', 'b3')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
225
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
226 assert b1_table.cols.mean[0] - numpy.mean(b1.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
227 assert b3_table.cols.mean[0] - numpy.mean(b3.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
228 assert b1_table.cols.min[0] - numpy.min(b1.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
229 assert b3_table.cols.min[0] - numpy.min(b3.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
230
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
231 assert not ('timestamp' in dir(b1_table.cols))
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
232
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
233 def test_get_desc():
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
234 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
235 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
236
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
237 desc = series._get_description_with_n_ints_n_floats(("col1","col2"), ("col3","col4"))
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
238
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
239 mytable = h5f.createTable('/', 'mytable', desc)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
240
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
241 # just make sure the columns are there... otherwise this will throw an exception
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
242 mytable.cols.col1
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
243 mytable.cols.col2
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
244 mytable.cols.col3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
245 mytable.cols.col4
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
246
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
247 try:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
248 # this should fail... LocalDescription must be local to get_desc_etc
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
249 test = LocalDescription
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
250 assert False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
251 except:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
252 assert True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
253
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
254 assert True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
255
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
256 def test_index_to_tuple_floaterror():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
257 try:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
258 series._index_to_tuple(5.1)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
259 assert False
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
260 except TypeError:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
261 assert True
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
262
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
263 def test_index_to_tuple_arrayok():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
264 tpl = series._index_to_tuple([1,2,3])
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
265 assert type(tpl) == tuple and tpl[1] == 2 and tpl[2] == 3
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
266
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
267 def test_index_to_tuple_intbecomestuple():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
268 tpl = series._index_to_tuple(32)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
269
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
270 assert type(tpl) == tuple and tpl == (32,)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
271
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
272 def test_index_to_tuple_longbecomestuple():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
273 tpl = series._index_to_tuple(928374928374928L)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
274
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
275 assert type(tpl) == tuple and tpl == (928374928374928L,)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
276
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
277 if __name__ == '__main__':
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
278 import tempfile
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
279 test_get_desc()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
280 test_ErrorSeries_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
281 test_BasicStatisticsSeries_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
282 test_AccumulatorSeriesWrapper_common_case()
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
283 test_SharedParamsStatisticsWrapper_commoncase()
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
284