Mercurial > ift6266

annotate utils/tables_series/test_series.py @ 208:acb942530923

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
author fsavard
date Fri, 05 Mar 2010 18:07:20 -0500
parents
children dc0d77c8a878
rev   line source
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
1 import tempfile
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
2 import numpy
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
3 import numpy.random
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
4 from tables import *
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
5
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
6 from series import *
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
7
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
8
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
9 def compare_floats(f1,f2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
10 if f1-f2 < 1e-3:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
11 return True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
12 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
13
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
14 def compare_lists(it1, it2, floats=False):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
15 if len(it1) != len(it2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
16 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
17
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
18 for el1, el2 in zip(it1, it2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
19 if floats:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
20 if not compare_floats(el1,el2):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
21 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
22 elif el1 != el2:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
23 return False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
24
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
25 return True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
26
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
27 def test_ErrorSeries_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
28 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
29 h5f_path = tempfile.NamedTemporaryFile().name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
30 h5f = openFile(h5f_path, "w")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
31
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
32 validation_error = ErrorSeries(error_name="validation_error", table_name="validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
33 hdf5_file=h5f, index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
34 title="Validation error indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
35
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
36 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
37 validation_error.append((1,1), 32.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
38 validation_error.append((1,2), 30.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
39 validation_error.append((2,1), 28.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
40 validation_error.append((2,2), 26.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
41
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
42 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
43
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
44 h5f = openFile(h5f_path, "r")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
45
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
46 table = h5f.getNode('/', 'validation_error')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
47
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
48 assert compare_lists(table.cols.epoch[:], [1,1,2,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
49 assert compare_lists(table.cols.minibatch[:], [1,2,1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
50 assert compare_lists(table.cols.validation_error[:], [32.0, 30.0, 28.0, 26.0])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
51
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
52 def test_AccumulatorSeriesWrapper_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
53 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
54 h5f_path = tempfile.NamedTemporaryFile().name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
55 h5f = openFile(h5f_path, "w")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
56
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
57 validation_error = ErrorSeries(error_name="accumulated_validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
58 table_name="accumulated_validation_error",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
59 hdf5_file=h5f,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
60 index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
61 title="Validation error, summed every 3 minibatches, indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
62
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
63 accumulator = AccumulatorSeriesWrapper(base_series=validation_error,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
64 reduce_every=3, reduce_function=numpy.sum)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
65
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
66 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
67 accumulator.append((1,1), 32.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
68 accumulator.append((1,2), 30.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
69 accumulator.append((2,1), 28.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
70 accumulator.append((2,2), 26.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
71 accumulator.append((3,1), 24.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
72 accumulator.append((3,2), 22.0)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
73
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
74 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
75
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
76 h5f = openFile(h5f_path, "r")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
77
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
78 table = h5f.getNode('/', 'accumulated_validation_error')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
79
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
80 assert compare_lists(table.cols.epoch[:], [2,3])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
81 assert compare_lists(table.cols.minibatch[:], [1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
82 assert compare_lists(table.cols.accumulated_validation_error[:], [90.0,72.0], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
83
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
84 def test_BasicStatisticsSeries_common_case(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
85 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
86 h5f_path = tempfile.NamedTemporaryFile().name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
87 h5f = openFile(h5f_path, "w")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
88
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
89 stats_series = BasicStatisticsSeries(table_name="b_vector_statistics",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
90 hdf5_file=h5f, index_names=('epoch','minibatch'),
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
91 title="Basic statistics for b vector indexed by epoch and minibatch")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
92
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
93 # (1,1), (1,2) etc. are (epoch, minibatch) index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
94 stats_series.append((1,1), [0.15, 0.20, 0.30])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
95 stats_series.append((1,2), [-0.18, 0.30, 0.58])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
96 stats_series.append((2,1), [0.18, -0.38, -0.68])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
97 stats_series.append((2,2), [0.15, 0.02, 1.9])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
98
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
99 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
100
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
101 h5f = openFile(h5f_path, "r")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
102
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
103 table = h5f.getNode('/', 'b_vector_statistics')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
104
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
105 assert compare_lists(table.cols.epoch[:], [1,1,2,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
106 assert compare_lists(table.cols.minibatch[:], [1,2,1,2])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
107 assert compare_lists(table.cols.mean[:], [0.21666667, 0.23333333, -0.29333332, 0.69], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
108 assert compare_lists(table.cols.min[:], [0.15000001, -0.18000001, -0.68000001, 0.02], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
109 assert compare_lists(table.cols.max[:], [0.30, 0.58, 0.18, 1.9], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
110 assert compare_lists(table.cols.std[:], [0.06236095, 0.31382939, 0.35640177, 0.85724366], floats=True)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
111
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
112 def test_ParamsStatisticsWrapper_commoncase(h5f=None):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
113 import numpy.random
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
114
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
115 if not h5f:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
116 h5f_path = tempfile.NamedTemporaryFile().name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
117 h5f = openFile(h5f_path, "w")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
118
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
119 stats = ParamsStatisticsWrapper(new_group_name="params", base_group="/",
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
120 arrays_names=('b1','b2','b3'), hdf5_file=h5f,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
121 index_names=('epoch','minibatch'))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
122
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
123 b1 = numpy.random.rand(5)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
124 b2 = numpy.random.rand(5)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
125 b3 = numpy.random.rand(5)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
126 stats.append((1,1), [b1,b2,b3])
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
127
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
128 h5f.close()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
129
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
130 h5f = openFile(h5f_path, "r")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
131
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
132 b1_table = h5f.getNode('/params', 'b1')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
133 b3_table = h5f.getNode('/params', 'b3')
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
134
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
135 assert b1_table.cols.mean[0] - numpy.mean(b1) < 1e-3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
136 assert b3_table.cols.mean[0] - numpy.mean(b3) < 1e-3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
137 assert b1_table.cols.min[0] - numpy.min(b1) < 1e-3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
138 assert b3_table.cols.min[0] - numpy.min(b3) < 1e-3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
139
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
140 def test_get_desc():
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
141 h5f_path = tempfile.NamedTemporaryFile().name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
142 h5f = openFile(h5f_path, "w")
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
143
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
144 desc = get_description_with_n_ints_n_floats(("col1","col2"), ("col3","col4"))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
145
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
146 mytable = h5f.createTable('/', 'mytable', desc)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
147
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
148 # just make sure the columns are there... otherwise this will throw an exception
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
149 mytable.cols.col1
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
150 mytable.cols.col2
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
151 mytable.cols.col3
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
152 mytable.cols.col4
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
153
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
154 try:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
155 # this should fail... LocalDescription must be local to get_desc_etc
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
156 test = LocalDescription
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
157 assert False
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
158 except:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
159 assert True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
160
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
161 assert True
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
162
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
163 if __name__ == '__main__':
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
164 import tempfile
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
165 test_get_desc()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
166 test_ErrorSeries_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
167 test_BasicStatisticsSeries_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
168 test_AccumulatorSeriesWrapper_common_case()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
169 test_ParamsStatisticsWrapper_commoncase()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
170