Mercurial > ift6266
annotate utils/seriestables/test_series.py @ 577:685756a11fd2
removed linebreaks
author | Yoshua Bengio <bengioy@iro.umontreal.ca> |
---|---|
date | Sat, 07 Aug 2010 22:56:46 -0400 |
parents | bfe20d63f88c |
children |
rev | line source |
---|---|
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
1 import tempfile |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
2 |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
3 import numpy |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
4 import numpy.random |
210
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
5 |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
6 from jobman import DD |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
7 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
8 import tables |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
9 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
10 from series import * |
218
4c137f16b013
Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents:
213
diff
changeset
|
11 import series |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
12 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
13 ################################################# |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
14 # Utils |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
15 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
16 def compare_floats(f1,f2): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
17 if f1-f2 < 1e-3: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
18 return True |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
19 return False |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
20 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
21 def compare_lists(it1, it2, floats=False): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
22 if len(it1) != len(it2): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
23 return False |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
24 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
25 for el1, el2 in zip(it1, it2): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
26 if floats: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
27 if not compare_floats(el1,el2): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
28 return False |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
29 elif el1 != el2: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
30 return False |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
31 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
32 return True |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
33 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
34 ################################################# |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
35 # Basic Series class tests |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
36 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
37 def test_Series_types(): |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
38 pass |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
39 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
40 ################################################# |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
41 # ErrorSeries tests |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
42 |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
43 def test_ErrorSeries_common_case(h5f=None): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
44 if not h5f: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
45 h5f_path = tempfile.NamedTemporaryFile().name |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
46 h5f = tables.openFile(h5f_path, "w") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
47 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
48 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error", |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
49 hdf5_file=h5f, index_names=('epoch','minibatch'), |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
50 title="Validation error indexed by epoch and minibatch") |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
51 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
52 # (1,1), (1,2) etc. are (epoch, minibatch) index |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
53 validation_error.append((1,1), 32.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
54 validation_error.append((1,2), 30.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
55 validation_error.append((2,1), 28.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
56 validation_error.append((2,2), 26.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
57 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
58 h5f.close() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
59 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
60 h5f = tables.openFile(h5f_path, "r") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
61 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
62 table = h5f.getNode('/', 'validation_error') |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
63 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
64 assert compare_lists(table.cols.epoch[:], [1,1,2,2]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
65 assert compare_lists(table.cols.minibatch[:], [1,2,1,2]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
66 assert compare_lists(table.cols.validation_error[:], [32.0, 30.0, 28.0, 26.0]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
67 |
226 | 68 def test_ErrorSeries_no_index(h5f=None): |
69 if not h5f: | |
70 h5f_path = tempfile.NamedTemporaryFile().name | |
71 h5f = tables.openFile(h5f_path, "w") | |
72 | |
73 validation_error = series.ErrorSeries(error_name="validation_error", | |
74 table_name="validation_error", | |
75 hdf5_file=h5f, | |
76 # empty tuple | |
77 index_names=tuple(), | |
78 title="Validation error with no index") | |
79 | |
80 # (1,1), (1,2) etc. are (epoch, minibatch) index | |
81 validation_error.append(tuple(), 32.0) | |
82 validation_error.append(tuple(), 30.0) | |
83 validation_error.append(tuple(), 28.0) | |
84 validation_error.append(tuple(), 26.0) | |
85 | |
86 h5f.close() | |
87 | |
88 h5f = tables.openFile(h5f_path, "r") | |
89 | |
90 table = h5f.getNode('/', 'validation_error') | |
91 | |
92 assert compare_lists(table.cols.validation_error[:], [32.0, 30.0, 28.0, 26.0]) | |
93 assert not ("epoch" in dir(table.cols)) | |
94 | |
224
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
95 def test_ErrorSeries_notimestamp(h5f=None): |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
96 if not h5f: |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
97 h5f_path = tempfile.NamedTemporaryFile().name |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
98 h5f = tables.openFile(h5f_path, "w") |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
99 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
100 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error", |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
101 hdf5_file=h5f, index_names=('epoch','minibatch'), |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
102 title="Validation error indexed by epoch and minibatch", |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
103 store_timestamp=False) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
104 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
105 # (1,1), (1,2) etc. are (epoch, minibatch) index |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
106 validation_error.append((1,1), 32.0) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
107 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
108 h5f.close() |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
109 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
110 h5f = tables.openFile(h5f_path, "r") |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
111 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
112 table = h5f.getNode('/', 'validation_error') |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
113 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
114 assert compare_lists(table.cols.epoch[:], [1]) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
115 assert not ("timestamp" in dir(table.cols)) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
116 assert "cpuclock" in dir(table.cols) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
117 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
118 def test_ErrorSeries_nocpuclock(h5f=None): |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
119 if not h5f: |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
120 h5f_path = tempfile.NamedTemporaryFile().name |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
121 h5f = tables.openFile(h5f_path, "w") |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
122 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
123 validation_error = series.ErrorSeries(error_name="validation_error", table_name="validation_error", |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
124 hdf5_file=h5f, index_names=('epoch','minibatch'), |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
125 title="Validation error indexed by epoch and minibatch", |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
126 store_cpuclock=False) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
127 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
128 # (1,1), (1,2) etc. are (epoch, minibatch) index |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
129 validation_error.append((1,1), 32.0) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
130 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
131 h5f.close() |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
132 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
133 h5f = tables.openFile(h5f_path, "r") |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
134 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
135 table = h5f.getNode('/', 'validation_error') |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
136 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
137 assert compare_lists(table.cols.epoch[:], [1]) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
138 assert not ("cpuclock" in dir(table.cols)) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
139 assert "timestamp" in dir(table.cols) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
140 |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
141 def test_AccumulatorSeriesWrapper_common_case(h5f=None): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
142 if not h5f: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
143 h5f_path = tempfile.NamedTemporaryFile().name |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
144 h5f = tables.openFile(h5f_path, "w") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
145 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
146 validation_error = ErrorSeries(error_name="accumulated_validation_error", |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
147 table_name="accumulated_validation_error", |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
148 hdf5_file=h5f, |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
149 index_names=('epoch','minibatch'), |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
150 title="Validation error, summed every 3 minibatches, indexed by epoch and minibatch") |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
151 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
152 accumulator = AccumulatorSeriesWrapper(base_series=validation_error, |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
153 reduce_every=3, reduce_function=numpy.sum) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
154 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
155 # (1,1), (1,2) etc. are (epoch, minibatch) index |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
156 accumulator.append((1,1), 32.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
157 accumulator.append((1,2), 30.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
158 accumulator.append((2,1), 28.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
159 accumulator.append((2,2), 26.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
160 accumulator.append((3,1), 24.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
161 accumulator.append((3,2), 22.0) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
162 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
163 h5f.close() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
164 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
165 h5f = tables.openFile(h5f_path, "r") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
166 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
167 table = h5f.getNode('/', 'accumulated_validation_error') |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
168 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
169 assert compare_lists(table.cols.epoch[:], [2,3]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
170 assert compare_lists(table.cols.minibatch[:], [1,2]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
171 assert compare_lists(table.cols.accumulated_validation_error[:], [90.0,72.0], floats=True) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
172 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
173 def test_BasicStatisticsSeries_common_case(h5f=None): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
174 if not h5f: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
175 h5f_path = tempfile.NamedTemporaryFile().name |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
176 h5f = tables.openFile(h5f_path, "w") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
177 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
178 stats_series = BasicStatisticsSeries(table_name="b_vector_statistics", |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
179 hdf5_file=h5f, index_names=('epoch','minibatch'), |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
180 title="Basic statistics for b vector indexed by epoch and minibatch") |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
181 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
182 # (1,1), (1,2) etc. are (epoch, minibatch) index |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
183 stats_series.append((1,1), [0.15, 0.20, 0.30]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
184 stats_series.append((1,2), [-0.18, 0.30, 0.58]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
185 stats_series.append((2,1), [0.18, -0.38, -0.68]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
186 stats_series.append((2,2), [0.15, 0.02, 1.9]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
187 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
188 h5f.close() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
189 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
190 h5f = tables.openFile(h5f_path, "r") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
191 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
192 table = h5f.getNode('/', 'b_vector_statistics') |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
193 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
194 assert compare_lists(table.cols.epoch[:], [1,1,2,2]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
195 assert compare_lists(table.cols.minibatch[:], [1,2,1,2]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
196 assert compare_lists(table.cols.mean[:], [0.21666667, 0.23333333, -0.29333332, 0.69], floats=True) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
197 assert compare_lists(table.cols.min[:], [0.15000001, -0.18000001, -0.68000001, 0.02], floats=True) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
198 assert compare_lists(table.cols.max[:], [0.30, 0.58, 0.18, 1.9], floats=True) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
199 assert compare_lists(table.cols.std[:], [0.06236095, 0.31382939, 0.35640177, 0.85724366], floats=True) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
200 |
210
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
201 def test_SharedParamsStatisticsWrapper_commoncase(h5f=None): |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
202 import numpy.random |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
203 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
204 if not h5f: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
205 h5f_path = tempfile.NamedTemporaryFile().name |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
206 h5f = tables.openFile(h5f_path, "w") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
207 |
210
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
208 stats = SharedParamsStatisticsWrapper(new_group_name="params", base_group="/", |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
209 arrays_names=('b1','b2','b3'), hdf5_file=h5f, |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
210 index_names=('epoch','minibatch')) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
211 |
210
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
212 b1 = DD({'value':numpy.random.rand(5)}) |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
213 b2 = DD({'value':numpy.random.rand(5)}) |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
214 b3 = DD({'value':numpy.random.rand(5)}) |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
215 stats.append((1,1), [b1,b2,b3]) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
216 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
217 h5f.close() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
218 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
219 h5f = tables.openFile(h5f_path, "r") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
220 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
221 b1_table = h5f.getNode('/params', 'b1') |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
222 b3_table = h5f.getNode('/params', 'b3') |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
223 |
210
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
224 assert b1_table.cols.mean[0] - numpy.mean(b1.value) < 1e-3 |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
225 assert b3_table.cols.mean[0] - numpy.mean(b3.value) < 1e-3 |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
226 assert b1_table.cols.min[0] - numpy.min(b1.value) < 1e-3 |
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
227 assert b3_table.cols.min[0] - numpy.min(b3.value) < 1e-3 |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
228 |
224
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
229 def test_SharedParamsStatisticsWrapper_notimestamp(h5f=None): |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
230 import numpy.random |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
231 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
232 if not h5f: |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
233 h5f_path = tempfile.NamedTemporaryFile().name |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
234 h5f = tables.openFile(h5f_path, "w") |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
235 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
236 stats = SharedParamsStatisticsWrapper(new_group_name="params", base_group="/", |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
237 arrays_names=('b1','b2','b3'), hdf5_file=h5f, |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
238 index_names=('epoch','minibatch'), |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
239 store_timestamp=False) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
240 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
241 b1 = DD({'value':numpy.random.rand(5)}) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
242 b2 = DD({'value':numpy.random.rand(5)}) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
243 b3 = DD({'value':numpy.random.rand(5)}) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
244 stats.append((1,1), [b1,b2,b3]) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
245 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
246 h5f.close() |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
247 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
248 h5f = tables.openFile(h5f_path, "r") |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
249 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
250 b1_table = h5f.getNode('/params', 'b1') |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
251 b3_table = h5f.getNode('/params', 'b3') |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
252 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
253 assert b1_table.cols.mean[0] - numpy.mean(b1.value) < 1e-3 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
254 assert b3_table.cols.mean[0] - numpy.mean(b3.value) < 1e-3 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
255 assert b1_table.cols.min[0] - numpy.min(b1.value) < 1e-3 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
256 assert b3_table.cols.min[0] - numpy.min(b3.value) < 1e-3 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
257 |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
258 assert not ('timestamp' in dir(b1_table.cols)) |
0515a8901c6a
Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents:
220
diff
changeset
|
259 |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
260 def test_get_desc(): |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
261 h5f_path = tempfile.NamedTemporaryFile().name |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
262 h5f = tables.openFile(h5f_path, "w") |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
263 |
218
4c137f16b013
Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents:
213
diff
changeset
|
264 desc = series._get_description_with_n_ints_n_floats(("col1","col2"), ("col3","col4")) |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
265 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
266 mytable = h5f.createTable('/', 'mytable', desc) |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
267 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
268 # just make sure the columns are there... otherwise this will throw an exception |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
269 mytable.cols.col1 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
270 mytable.cols.col2 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
271 mytable.cols.col3 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
272 mytable.cols.col4 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
273 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
274 try: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
275 # this should fail... LocalDescription must be local to get_desc_etc |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
276 test = LocalDescription |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
277 assert False |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
278 except: |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
279 assert True |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
280 |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
281 assert True |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
282 |
220
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
283 def test_index_to_tuple_floaterror(): |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
284 try: |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
285 series._index_to_tuple(5.1) |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
286 assert False |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
287 except TypeError: |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
288 assert True |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
289 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
290 def test_index_to_tuple_arrayok(): |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
291 tpl = series._index_to_tuple([1,2,3]) |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
292 assert type(tpl) == tuple and tpl[1] == 2 and tpl[2] == 3 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
293 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
294 def test_index_to_tuple_intbecomestuple(): |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
295 tpl = series._index_to_tuple(32) |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
296 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
297 assert type(tpl) == tuple and tpl == (32,) |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
298 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
299 def test_index_to_tuple_longbecomestuple(): |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
300 tpl = series._index_to_tuple(928374928374928L) |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
301 |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
302 assert type(tpl) == tuple and tpl == (928374928374928L,) |
e172ef73cdc5
Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents:
218
diff
changeset
|
303 |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
304 if __name__ == '__main__': |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
305 import tempfile |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
306 test_get_desc() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
307 test_ErrorSeries_common_case() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
308 test_BasicStatisticsSeries_common_case() |
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
309 test_AccumulatorSeriesWrapper_common_case() |
210
dc0d77c8a878
Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents:
208
diff
changeset
|
310 test_SharedParamsStatisticsWrapper_commoncase() |
208
acb942530923
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff
changeset
|
311 |