Mercurial > ift6266

annotate utils/seriestables/test_series.py @ 227:acae439d6572

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Ajouté une modification sur stacked_dae qui utilise les nouvelles SeriesTables. Je le met dans le repository pour que mes expériences en cours continuent sans perturbation, et pour que Sylvain puisse récupérer la version actuelle; je fusionnerai à moment donné.
author fsavard
date Fri, 12 Mar 2010 10:31:10 -0500
parents bfe20d63f88c
children
rev   line source
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
1 import tempfile
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
2
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
3 import numpy
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
4 import numpy.random
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
5
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
6 from jobman import DD
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
7
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
8 import tables
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
9
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
10 from series import *
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
11 import series
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
12
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
13 #################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
14 # Utils
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
15
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
16 def compare_floats(f1,f2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
17 if f1-f2 < 1e-3:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
18 return True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
19 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
20
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
21 def compare_lists(it1, it2, floats=False):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
22 if len(it1) != len(it2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
23 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
24
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
25 for el1, el2 in zip(it1, it2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
26 if floats:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
27 if not compare_floats(el1,el2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
28 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
29 elif el1 != el2:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
30 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
31
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
32 return True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
33
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
34 #################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
35 # Basic Series class tests
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
36
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
37 def test_Series_types():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
38 pass
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
39
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
40 #################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
41 # ErrorSeries tests
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
42
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
43 def test_ErrorSeries_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
44 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
45 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
46 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
47
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
48 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error",
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
49 hdf5_file=h5f, index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
50 title="Validation error indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
51
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
52 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
53 validation_error.append((1,1), 32.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
54 validation_error.append((1,2), 30.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
55 validation_error.append((2,1), 28.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
56 validation_error.append((2,2), 26.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
57
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
58 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
59
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
60 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
61
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
62 table = h5f.getNode('/', 'validation_error')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
63
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
64 assert compare_lists(table.cols.epoch[:], [1,1,2,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
65 assert compare_lists(table.cols.minibatch[:], [1,2,1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
66 assert compare_lists(table.cols.validation_error[:], [32.0, 30.0, 28.0, 26.0])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
67
226
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
68 def test_ErrorSeries_no_index(h5f=None):
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
69 if not h5f:
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
70 h5f_path = tempfile.NamedTemporaryFile().name
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
71 h5f = tables.openFile(h5f_path, "w")
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
72
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
73 validation_error = series.ErrorSeries(error_name="validation_error",
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
74 table_name="validation_error",
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
75 hdf5_file=h5f,
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
76 # empty tuple
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
77 index_names=tuple(),
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
78 title="Validation error with no index")
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
79
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
80 # (1,1), (1,2) etc. are (epoch, minibatch) index
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
81 validation_error.append(tuple(), 32.0)
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
82 validation_error.append(tuple(), 30.0)
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
83 validation_error.append(tuple(), 28.0)
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
84 validation_error.append(tuple(), 26.0)
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
85
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
86 h5f.close()
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
87
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
88 h5f = tables.openFile(h5f_path, "r")
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
89
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
90 table = h5f.getNode('/', 'validation_error')
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
91
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
92 assert compare_lists(table.cols.validation_error[:], [32.0, 30.0, 28.0, 26.0])
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
93 assert not ("epoch" in dir(table.cols))
bfe20d63f88c Test pour cas sans index, après question de James
fsavard
parents: 224
diff changeset
94
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
95 def test_ErrorSeries_notimestamp(h5f=None):
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
96 if not h5f:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
97 h5f_path = tempfile.NamedTemporaryFile().name
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
98 h5f = tables.openFile(h5f_path, "w")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
99
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
100 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
101 hdf5_file=h5f, index_names=('epoch','minibatch'),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
102 title="Validation error indexed by epoch and minibatch",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
103 store_timestamp=False)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
104
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
105 # (1,1), (1,2) etc. are (epoch, minibatch) index
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
106 validation_error.append((1,1), 32.0)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
107
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
108 h5f.close()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
109
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
110 h5f = tables.openFile(h5f_path, "r")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
111
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
112 table = h5f.getNode('/', 'validation_error')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
113
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
114 assert compare_lists(table.cols.epoch[:], [1])
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
115 assert not ("timestamp" in dir(table.cols))
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
116 assert "cpuclock" in dir(table.cols)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
117
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
118 def test_ErrorSeries_nocpuclock(h5f=None):
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
119 if not h5f:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
120 h5f_path = tempfile.NamedTemporaryFile().name
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
121 h5f = tables.openFile(h5f_path, "w")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
122
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
123 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
124 hdf5_file=h5f, index_names=('epoch','minibatch'),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
125 title="Validation error indexed by epoch and minibatch",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
126 store_cpuclock=False)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
127
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
128 # (1,1), (1,2) etc. are (epoch, minibatch) index
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
129 validation_error.append((1,1), 32.0)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
130
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
131 h5f.close()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
132
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
133 h5f = tables.openFile(h5f_path, "r")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
134
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
135 table = h5f.getNode('/', 'validation_error')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
136
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
137 assert compare_lists(table.cols.epoch[:], [1])
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
138 assert not ("cpuclock" in dir(table.cols))
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
139 assert "timestamp" in dir(table.cols)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
140
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
141 def test_AccumulatorSeriesWrapper_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
142 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
143 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
144 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
145
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
146 validation_error = ErrorSeries(error_name="accumulated_validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
147 table_name="accumulated_validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
148 hdf5_file=h5f,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
149 index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
150 title="Validation error, summed every 3 minibatches, indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
151
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
152 accumulator = AccumulatorSeriesWrapper(base_series=validation_error,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
153 reduce_every=3, reduce_function=numpy.sum)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
154
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
155 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
156 accumulator.append((1,1), 32.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
157 accumulator.append((1,2), 30.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
158 accumulator.append((2,1), 28.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
159 accumulator.append((2,2), 26.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
160 accumulator.append((3,1), 24.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
161 accumulator.append((3,2), 22.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
162
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
163 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
164
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
165 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
166
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
167 table = h5f.getNode('/', 'accumulated_validation_error')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
168
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
169 assert compare_lists(table.cols.epoch[:], [2,3])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
170 assert compare_lists(table.cols.minibatch[:], [1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
171 assert compare_lists(table.cols.accumulated_validation_error[:], [90.0,72.0], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
172
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
173 def test_BasicStatisticsSeries_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
174 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
175 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
176 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
177
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
178 stats_series = BasicStatisticsSeries(table_name="b_vector_statistics",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
179 hdf5_file=h5f, index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
180 title="Basic statistics for b vector indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
181
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
182 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
183 stats_series.append((1,1), [0.15, 0.20, 0.30])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
184 stats_series.append((1,2), [-0.18, 0.30, 0.58])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
185 stats_series.append((2,1), [0.18, -0.38, -0.68])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
186 stats_series.append((2,2), [0.15, 0.02, 1.9])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
187
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
188 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
189
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
190 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
191
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
192 table = h5f.getNode('/', 'b_vector_statistics')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
193
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
194 assert compare_lists(table.cols.epoch[:], [1,1,2,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
195 assert compare_lists(table.cols.minibatch[:], [1,2,1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
196 assert compare_lists(table.cols.mean[:], [0.21666667, 0.23333333, -0.29333332, 0.69], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
197 assert compare_lists(table.cols.min[:], [0.15000001, -0.18000001, -0.68000001, 0.02], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
198 assert compare_lists(table.cols.max[:], [0.30, 0.58, 0.18, 1.9], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
199 assert compare_lists(table.cols.std[:], [0.06236095, 0.31382939, 0.35640177, 0.85724366], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
200
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
201 def test_SharedParamsStatisticsWrapper_commoncase(h5f=None):
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
202 import numpy.random
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
203
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
204 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
205 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
206 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
207
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
208 stats = SharedParamsStatisticsWrapper(new_group_name="params", base_group="/",
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
209 arrays_names=('b1','b2','b3'), hdf5_file=h5f,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
210 index_names=('epoch','minibatch'))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
211
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
212 b1 = DD({'value':numpy.random.rand(5)})
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
213 b2 = DD({'value':numpy.random.rand(5)})
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
214 b3 = DD({'value':numpy.random.rand(5)})
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
215 stats.append((1,1), [b1,b2,b3])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
216
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
217 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
218
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
219 h5f = tables.openFile(h5f_path, "r")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
220
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
221 b1_table = h5f.getNode('/params', 'b1')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
222 b3_table = h5f.getNode('/params', 'b3')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
223
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
224 assert b1_table.cols.mean[0] - numpy.mean(b1.value) < 1e-3
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
225 assert b3_table.cols.mean[0] - numpy.mean(b3.value) < 1e-3
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
226 assert b1_table.cols.min[0] - numpy.min(b1.value) < 1e-3
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
227 assert b3_table.cols.min[0] - numpy.min(b3.value) < 1e-3
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
228
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
229 def test_SharedParamsStatisticsWrapper_notimestamp(h5f=None):
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
230 import numpy.random
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
231
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
232 if not h5f:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
233 h5f_path = tempfile.NamedTemporaryFile().name
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
234 h5f = tables.openFile(h5f_path, "w")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
235
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
236 stats = SharedParamsStatisticsWrapper(new_group_name="params", base_group="/",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
237 arrays_names=('b1','b2','b3'), hdf5_file=h5f,
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
238 index_names=('epoch','minibatch'),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
239 store_timestamp=False)
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
240
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
241 b1 = DD({'value':numpy.random.rand(5)})
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
242 b2 = DD({'value':numpy.random.rand(5)})
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
243 b3 = DD({'value':numpy.random.rand(5)})
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
244 stats.append((1,1), [b1,b2,b3])
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
245
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
246 h5f.close()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
247
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
248 h5f = tables.openFile(h5f_path, "r")
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
249
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
250 b1_table = h5f.getNode('/params', 'b1')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
251 b3_table = h5f.getNode('/params', 'b3')
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
252
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
253 assert b1_table.cols.mean[0] - numpy.mean(b1.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
254 assert b3_table.cols.mean[0] - numpy.mean(b3.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
255 assert b1_table.cols.min[0] - numpy.min(b1.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
256 assert b3_table.cols.min[0] - numpy.min(b3.value) < 1e-3
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
257
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
258 assert not ('timestamp' in dir(b1_table.cols))
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 220
diff changeset
259
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
260 def test_get_desc():
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
261 h5f_path = tempfile.NamedTemporaryFile().name
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
262 h5f = tables.openFile(h5f_path, "w")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
263
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
264 desc = series._get_description_with_n_ints_n_floats(("col1","col2"), ("col3","col4"))
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
265
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
266 mytable = h5f.createTable('/', 'mytable', desc)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
267
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
268 # just make sure the columns are there... otherwise this will throw an exception
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
269 mytable.cols.col1
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
270 mytable.cols.col2
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
271 mytable.cols.col3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
272 mytable.cols.col4
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
273
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
274 try:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
275 # this should fail... LocalDescription must be local to get_desc_etc
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
276 test = LocalDescription
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
277 assert False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
278 except:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
279 assert True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
280
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
281 assert True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
282
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
283 def test_index_to_tuple_floaterror():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
284 try:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
285 series._index_to_tuple(5.1)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
286 assert False
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
287 except TypeError:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
288 assert True
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
289
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
290 def test_index_to_tuple_arrayok():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
291 tpl = series._index_to_tuple([1,2,3])
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
292 assert type(tpl) == tuple and tpl[1] == 2 and tpl[2] == 3
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
293
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
294 def test_index_to_tuple_intbecomestuple():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
295 tpl = series._index_to_tuple(32)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
296
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
297 assert type(tpl) == tuple and tpl == (32,)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
298
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
299 def test_index_to_tuple_longbecomestuple():
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
300 tpl = series._index_to_tuple(928374928374928L)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
301
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
302 assert type(tpl) == tuple and tpl == (928374928374928L,)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
303
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
304 if __name__ == '__main__':
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
305 import tempfile
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
306 test_get_desc()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
307 test_ErrorSeries_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
308 test_BasicStatisticsSeries_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
309 test_AccumulatorSeriesWrapper_common_case()
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
310 test_SharedParamsStatisticsWrapper_commoncase()
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
311