Mercurial > ift6266

annotate utils/seriestables/series.py @ 572:7ee0e41dd3d5

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
gone through all
author Yoshua Bengio <bengioy@iro.umontreal.ca>
date Fri, 06 Aug 2010 15:12:01 -0400
parents 0515a8901c6a
children
rev   line source
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
1 import tables
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
2
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
3 import numpy
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
4 import time
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
5
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
6 ##############################################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
7 # Utility functions to create IsDescription objects (pytables data types)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
8
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
9 '''
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
10 The way these "IsDescription constructor" work is simple: write the
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
11 code as if it were in a file, then exec()ute it, leaving us with
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
12 a local-scoped LocalDescription which may be used to call createTable.
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
13
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
14 It's a small hack, but it's necessary as the names of the columns
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
15 are retrieved based on the variable name, which we can't programmatically set
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
16 otherwise.
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
17 '''
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
18
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
19 def _get_description_timestamp_cpuclock_columns(store_timestamp, store_cpuclock, pos=0):
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
20 toexec = ""
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
21
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
22 if store_timestamp:
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
23 toexec += "\ttimestamp = tables.Time32Col(pos="+str(pos)+")\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
24 pos += 1
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
25
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
26 if store_cpuclock:
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
27 toexec += "\tcpuclock = tables.Float64Col(pos="+str(pos)+")\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
28 pos += 1
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
29
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
30 return toexec, pos
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
31
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
32 def _get_description_n_ints(int_names, int_width=64, pos=0):
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
33 """
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
34 Begins construction of a class inheriting from IsDescription
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
35 to construct an HDF5 table with index columns named with int_names.
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
36
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
37 See Series().__init__ to see how those are used.
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
38 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
39 int_constructor = "tables.Int64Col"
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
40 if int_width == 32:
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
41 int_constructor = "tables.Int32Col"
221
02d9c1279dd8 Encore quelques typechecks à SeriesTables.
fsavard
parents: 220
diff changeset
42 elif not int_width in (32, 64):
02d9c1279dd8 Encore quelques typechecks à SeriesTables.
fsavard
parents: 220
diff changeset
43 raise "int_width must be left unspecified, or should equal 32 or 64"
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
44
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
45 toexec = ""
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
46
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
47 for n in int_names:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
48 toexec += "\t" + n + " = " + int_constructor + "(pos=" + str(pos) + ")\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
49 pos += 1
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
50
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
51 return toexec, pos
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
52
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
53 def _get_description_with_n_ints_n_floats(int_names, float_names,
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
54 int_width=64, float_width=32,
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
55 store_timestamp=True, store_cpuclock=True):
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
56 """
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
57 Constructs a class to be used when constructing a table with PyTables.
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
58
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
59 This is useful to construct a series with an index with multiple levels.
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
60 E.g. if you want to index your "validation error" with "epoch" first, then
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
61 "minibatch_index" second, you'd use two "int_names".
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
62
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
63 Parameters
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
64 ----------
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
65 int_names : tuple of str
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
66 Names of the int (e.g. index) columns
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
67 float_names : tuple of str
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
68 Names of the float (e.g. error) columns
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
69 int_width : {'32', '64'}
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
70 Type of ints.
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
71 float_width : {'32', '64'}
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
72 Type of floats.
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
73 store_timestamp : bool
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
74 See __init__ of Series
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
75 store_cpuclock : bool
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
76 See __init__ of Series
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
77
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
78 Returns
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
79 -------
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
80 A class object, to pass to createTable()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
81 """
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
82
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
83 toexec = "class LocalDescription(tables.IsDescription):\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
84
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
85 toexec_, pos = _get_description_timestamp_cpuclock_columns(store_timestamp, store_cpuclock)
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
86 toexec += toexec_
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
87
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
88 toexec_, pos = _get_description_n_ints(int_names, int_width=int_width, pos=pos)
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
89 toexec += toexec_
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
90
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
91 float_constructor = "tables.Float32Col"
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
92 if float_width == 64:
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
93 float_constructor = "tables.Float64Col"
221
02d9c1279dd8 Encore quelques typechecks à SeriesTables.
fsavard
parents: 220
diff changeset
94 elif not float_width in (32, 64):
02d9c1279dd8 Encore quelques typechecks à SeriesTables.
fsavard
parents: 220
diff changeset
95 raise "float_width must be left unspecified, or should equal 32 or 64"
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
96
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
97 for n in float_names:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
98 toexec += "\t" + n + " = " + float_constructor + "(pos=" + str(pos) + ")\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
99 pos += 1
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
100
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
101 exec(toexec)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
102
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
103 return LocalDescription
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
104
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
105 ##############################################################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
106 # Series classes
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
107
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
108 # Shortcut to allow passing a single int as index, instead of a tuple
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
109 def _index_to_tuple(index):
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
110 if type(index) == tuple:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
111 return index
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
112
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
113 if type(index) == list:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
114 index = tuple(index)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
115 return index
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
116
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
117 try:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
118 if index % 1 > 0.001 and index % 1 < 0.999:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
119 raise
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
120 idx = long(index)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
121 return (idx,)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
122 except:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
123 raise TypeError("index must be a tuple of integers, or at least a single integer")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
124
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
125 class Series():
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
126 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
127 Base Series class, with minimal arguments and type checks.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
128
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
129 Yet cannot be used by itself (it's append() method raises an error)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
130 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
131
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
132 def __init__(self, table_name, hdf5_file, index_names=('epoch',),
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
133 title="", hdf5_group='/',
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
134 store_timestamp=True, store_cpuclock=True):
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
135 """Basic arguments each Series must get.
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
136
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
137 Parameters
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
138 ----------
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
139 table_name : str
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
140 Name of the table to create under group "hd5_group" (other
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
141 parameter). No spaces, ie. follow variable naming restrictions.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
142 hdf5_file : open HDF5 file
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
143 File opened with openFile() in PyTables (ie. return value of
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
144 openFile).
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
145 index_names : tuple of str
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
146 Columns to use as index for elements in the series, other
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
147 example would be ('epoch', 'minibatch'). This would then allow
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
148 you to call append(index, element) with index made of two ints,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
149 one for epoch index, one for minibatch index in epoch.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
150 title : str
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
151 Title to attach to this table as metadata. Can contain spaces
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
152 and be longer then the table_name.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
153 hdf5_group : str
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
154 Path of the group (kind of a file) in the HDF5 file under which
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
155 to create the table.
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
156 store_timestamp : bool
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
157 Whether to create a column for timestamps and store them with
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
158 each record.
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
159 store_cpuclock : bool
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
160 Whether to create a column for cpu clock and store it with
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
161 each record.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
162 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
163
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
164 #########################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
165 # checks
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
166
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
167 if type(table_name) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
168 raise TypeError("table_name must be a string")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
169 if table_name == "":
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
170 raise ValueError("table_name must not be empty")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
171
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
172 if not isinstance(hdf5_file, tables.file.File):
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
173 raise TypeError("hdf5_file must be an open HDF5 file (use tables.openFile)")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
174 #if not ('w' in hdf5_file.mode or 'a' in hdf5_file.mode):
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
175 # raise ValueError("hdf5_file must be opened in write or append mode")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
176
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
177 if type(index_names) != tuple:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
178 raise TypeError("index_names must be a tuple of strings." + \
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
179 "If you have only one element in the tuple, don't forget " +\
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
180 "to add a comma, e.g. ('epoch',).")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
181 for name in index_names:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
182 if type(name) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
183 raise TypeError("index_names must only contain strings, but also"+\
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
184 "contains a "+str(type(name))+".")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
185
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
186 if type(title) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
187 raise TypeError("title must be a string, even if empty")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
188
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
189 if type(hdf5_group) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
190 raise TypeError("hdf5_group must be a string")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
191
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
192 if type(store_timestamp) != bool:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
193 raise TypeError("store_timestamp must be a bool")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
194
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
195 if type(store_cpuclock) != bool:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
196 raise TypeError("store_timestamp must be a bool")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
197
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
198 #########################################
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
199
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
200 self.table_name = table_name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
201 self.hdf5_file = hdf5_file
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
202 self.index_names = index_names
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
203 self.title = title
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
204 self.hdf5_group = hdf5_group
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
205
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
206 self.store_timestamp = store_timestamp
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
207 self.store_cpuclock = store_cpuclock
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
208
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
209 def append(self, index, element):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
210 raise NotImplementedError
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
211
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
212 def _timestamp_cpuclock(self, newrow):
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
213 if self.store_timestamp:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
214 newrow["timestamp"] = time.time()
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
215
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
216 if self.store_cpuclock:
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
217 newrow["cpuclock"] = time.clock()
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
218
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
219 class DummySeries():
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
220 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
221 To put in a series dictionary instead of a real series, to do nothing
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
222 when we don't want a given series to be saved.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
223
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
224 E.g. if we'd normally have a "training_error" series in a dictionary
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
225 of series, the training loop would have something like this somewhere:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
226
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
227 series["training_error"].append((15,), 20.0)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
228
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
229 but if we don't want to save the training errors this time, we simply
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
230 do
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
231
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
232 series["training_error"] = DummySeries()
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
233 """
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
234 def append(self, index, element):
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
235 pass
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
236
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
237 class ErrorSeries(Series):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
238 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
239 Most basic Series: saves a single float (called an Error as this is
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
240 the most common use case I foresee) along with an index (epoch, for
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
241 example) and timestamp/cpu.clock for each of these floats.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
242 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
243
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
244 def __init__(self, error_name, table_name,
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
245 hdf5_file, index_names=('epoch',),
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
246 title="", hdf5_group='/',
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
247 store_timestamp=True, store_cpuclock=True):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
248 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
249 For most parameters, see Series.__init__
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
250
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
251 Parameters
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
252 ----------
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
253 error_name : str
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
254 In the HDF5 table, column name for the error float itself.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
255 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
256
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
257 # most type/value checks are performed in Series.__init__
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
258 Series.__init__(self, table_name, hdf5_file, index_names, title,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
259 store_timestamp=store_timestamp,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
260 store_cpuclock=store_cpuclock)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
261
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
262 if type(error_name) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
263 raise TypeError("error_name must be a string")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
264 if error_name == "":
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
265 raise ValueError("error_name must not be empty")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
266
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
267 self.error_name = error_name
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
268
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
269 self._create_table()
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
270
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
271 def _create_table(self):
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
272 table_description = _get_description_with_n_ints_n_floats( \
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
273 self.index_names, (self.error_name,),
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
274 store_timestamp=self.store_timestamp,
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
275 store_cpuclock=self.store_cpuclock)
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
276
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
277 self._table = self.hdf5_file.createTable(self.hdf5_group,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
278 self.table_name,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
279 table_description,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
280 title=self.title)
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
281
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
282
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
283 def append(self, index, error):
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
284 """
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
285 Parameters
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
286 ----------
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
287 index : tuple of int
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
288 Following index_names passed to __init__, e.g. (12, 15) if
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
289 index_names were ('epoch', 'minibatch_size').
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
290 A single int (not tuple) is acceptable if index_names has a single
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
291 element.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
292 An array will be casted to a tuple, as a convenience.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
293
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
294 error : float
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
295 Next error in the series.
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
296 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
297 index = _index_to_tuple(index)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
298
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
299 if len(index) != len(self.index_names):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
300 raise ValueError("index provided does not have the right length (expected " \
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
301 + str(len(self.index_names)) + " got " + str(len(index)))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
302
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
303 # other checks are implicit when calling newrow[..] =,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
304 # which should throw an error if not of the right type
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
305
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
306 newrow = self._table.row
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
307
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
308 # Columns for index in table are based on index_names
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
309 for col_name, value in zip(self.index_names, index):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
310 newrow[col_name] = value
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
311 newrow[self.error_name] = error
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
312
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
313 # adds timestamp and cpuclock to newrow if necessary
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
314 self._timestamp_cpuclock(newrow)
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
315
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
316 newrow.append()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
317
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
318 self.hdf5_file.flush()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
319
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
320 # Does not inherit from Series because it does not itself need to
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
321 # access the hdf5_file and does not need a series_name (provided
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
322 # by the base_series.)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
323 class AccumulatorSeriesWrapper():
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
324 '''
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
325 Wraps a Series by accumulating objects passed its Accumulator.append()
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
326 method and "reducing" (e.g. calling numpy.mean(list)) once in a while,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
327 every "reduce_every" calls in fact.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
328 '''
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
329
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
330 def __init__(self, base_series, reduce_every, reduce_function=numpy.mean):
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
331 """
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
332 Parameters
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
333 ----------
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
334 base_series : Series
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
335 This object must have an append(index, value) function.
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
336
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
337 reduce_every : int
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
338 Apply the reduction function (e.g. mean()) every time we get this
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
339 number of elements. E.g. if this is 100, then every 100 numbers
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
340 passed to append(), we'll take the mean and call append(this_mean)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
341 on the BaseSeries.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
342
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
343 reduce_function : function
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
344 Must take as input an array of "elements", as passed to (this
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
345 accumulator's) append(). Basic case would be to take an array of
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
346 floats and sum them into one float, for example.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
347 """
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
348 self.base_series = base_series
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
349 self.reduce_function = reduce_function
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
350 self.reduce_every = reduce_every
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
351
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
352 self._buffer = []
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
353
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
354
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
355 def append(self, index, element):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
356 """
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
357 Parameters
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
358 ----------
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
359 index : tuple of int
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
360 The index used is the one of the last element reduced. E.g. if
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
361 you accumulate over the first 1000 minibatches, the index
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
362 passed to the base_series.append() function will be 1000.
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
363 A single int (not tuple) is acceptable if index_names has a single
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
364 element.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
365 An array will be casted to a tuple, as a convenience.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
366
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
367 element : float
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
368 Element that will be accumulated.
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
369 """
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
370 self._buffer.append(element)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
371
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
372 if len(self._buffer) == self.reduce_every:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
373 reduced = self.reduce_function(self._buffer)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
374 self.base_series.append(index, reduced)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
375 self._buffer = []
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
376
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
377 # The >= case should never happen, except if lists
221
02d9c1279dd8 Encore quelques typechecks à SeriesTables.
fsavard
parents: 220
diff changeset
378 # were appended by accessing _buffer externally (when it's
02d9c1279dd8 Encore quelques typechecks à SeriesTables.
fsavard
parents: 220
diff changeset
379 # intended to be private), which should be a red flag.
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
380 assert len(self._buffer) < self.reduce_every
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
381
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
382 # Outside of class to fix an issue with exec in Python 2.6.
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
383 # My sorries to the god of pretty code.
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
384 def _BasicStatisticsSeries_construct_table_toexec(index_names, store_timestamp, store_cpuclock):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
385 toexec = "class LocalDescription(tables.IsDescription):\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
386
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
387 toexec_, pos = _get_description_timestamp_cpuclock_columns(store_timestamp, store_cpuclock)
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
388 toexec += toexec_
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
389
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
390 toexec_, pos = _get_description_n_ints(index_names, pos=pos)
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
391 toexec += toexec_
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
392
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
393 toexec += "\tmean = tables.Float32Col(pos=" + str(pos) + ")\n"
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
394 toexec += "\tmin = tables.Float32Col(pos=" + str(pos+1) + ")\n"
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
395 toexec += "\tmax = tables.Float32Col(pos=" + str(pos+2) + ")\n"
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
396 toexec += "\tstd = tables.Float32Col(pos=" + str(pos+3) + ")\n"
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
397
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
398 # This creates "LocalDescription", which we may then use
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
399 exec(toexec)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
400
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
401 return LocalDescription
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
402
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
403 # Defaults functions for BasicStatsSeries. These can be replaced.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
404 _basic_stats_functions = {'mean': lambda(x): numpy.mean(x),
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
405 'min': lambda(x): numpy.min(x),
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
406 'max': lambda(x): numpy.max(x),
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
407 'std': lambda(x): numpy.std(x)}
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
408
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
409 class BasicStatisticsSeries(Series):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
410
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
411 def __init__(self, table_name, hdf5_file,
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
412 stats_functions=_basic_stats_functions,
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
413 index_names=('epoch',), title="", hdf5_group='/',
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
414 store_timestamp=True, store_cpuclock=True):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
415 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
416 For most parameters, see Series.__init__
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
417
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
418 Parameters
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
419 ----------
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
420 series_name : str
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
421 Not optional here. Will be prepended with "Basic statistics for "
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
422
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
423 stats_functions : dict, optional
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
424 Dictionary with a function for each key "mean", "min", "max",
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
425 "std". The function must take whatever is passed to append(...)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
426 and return a single number (float).
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
427 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
428
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
429 # Most type/value checks performed in Series.__init__
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
430 Series.__init__(self, table_name, hdf5_file, index_names, title,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
431 store_timestamp=store_timestamp,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
432 store_cpuclock=store_cpuclock)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
433
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
434 if type(hdf5_group) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
435 raise TypeError("hdf5_group must be a string")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
436
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
437 if type(stats_functions) != dict:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
438 # just a basic check. We'll suppose caller knows what he's doing.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
439 raise TypeError("stats_functions must be a dict")
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
440
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
441 self.hdf5_group = hdf5_group
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
442
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
443 self.stats_functions = stats_functions
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
444
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
445 self._create_table()
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
446
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
447 def _create_table(self):
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
448 table_description = \
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
449 _BasicStatisticsSeries_construct_table_toexec( \
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
450 self.index_names,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
451 self.store_timestamp, self.store_cpuclock)
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
452
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
453 self._table = self.hdf5_file.createTable(self.hdf5_group,
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
454 self.table_name, table_description)
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
455
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
456 def append(self, index, array):
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
457 """
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
458 Parameters
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
459 ----------
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
460 index : tuple of int
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
461 Following index_names passed to __init__, e.g. (12, 15)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
462 if index_names were ('epoch', 'minibatch_size')
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
463 A single int (not tuple) is acceptable if index_names has a single
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
464 element.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
465 An array will be casted to a tuple, as a convenience.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
466
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
467 array
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
468 Is of whatever type the stats_functions passed to
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
469 __init__ can take. Default is anything numpy.mean(),
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
470 min(), max(), std() can take.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
471 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
472 index = _index_to_tuple(index)
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
473
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
474 if len(index) != len(self.index_names):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
475 raise ValueError("index provided does not have the right length (expected " \
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
476 + str(len(self.index_names)) + " got " + str(len(index)))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
477
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
478 newrow = self._table.row
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
479
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
480 for col_name, value in zip(self.index_names, index):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
481 newrow[col_name] = value
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
482
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
483 newrow["mean"] = self.stats_functions['mean'](array)
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
484 newrow["min"] = self.stats_functions['min'](array)
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
485 newrow["max"] = self.stats_functions['max'](array)
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
486 newrow["std"] = self.stats_functions['std'](array)
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
487
218
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
488 self._timestamp_cpuclock(newrow)
4c137f16b013 Modifications pour stocker des timestamps/cpuclock avec chaque rangée créée, selon suggestion de Yoshua ce matin
fsavard
parents: 213
diff changeset
489
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
490 newrow.append()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
491
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
492 self.hdf5_file.flush()
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
493
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
494 class SeriesArrayWrapper():
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
495 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
496 Simply redistributes any number of elements to sub-series to respective
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
497 append()s.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
498
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
499 To use if you have many elements to append in similar series, e.g. if you
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
500 have an array containing [train_error, valid_error, test_error], and 3
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
501 corresponding series, this allows you to simply pass this array of 3
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
502 values to append() instead of passing each element to each individual
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
503 series in turn.
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
504 """
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
505
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
506 def __init__(self, base_series_list):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
507 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
508 Parameters
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
509 ----------
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
510 base_series_list : array or tuple of Series
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
511 You must have previously created and configured each of those
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
512 series, then put them in an array. This array must follow the
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
513 same order as the array passed as ``elements`` parameter of
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
514 append().
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
515 """
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
516 self.base_series_list = base_series_list
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
517
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
518 def append(self, index, elements):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
519 """
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
520 Parameters
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
521 ----------
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
522 index : tuple of int
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
523 See for example ErrorSeries.append()
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
524
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
525 elements : array or tuple
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
526 Array or tuple of elements that will be passed down to
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
527 the base_series passed to __init__, in the same order.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
528 """
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
529 if len(elements) != len(self.base_series_list):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
530 raise ValueError("not enough or too much elements provided (expected " \
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
531 + str(len(self.base_series_list)) + " got " + str(len(elements)))
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
532
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
533 for series, el in zip(self.base_series_list, elements):
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
534 series.append(index, el)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
535
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
536 class SharedParamsStatisticsWrapper(SeriesArrayWrapper):
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
537 '''
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
538 Save mean, min/max, std of shared parameters place in an array.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
539
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
540 Here "shared" means "theano.shared", which means elements of the
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
541 array will have a .value to use for numpy.mean(), etc.
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
542
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
543 This inherits from SeriesArrayWrapper, which provides the append()
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
544 method.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
545 '''
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
546
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
547 def __init__(self, arrays_names, new_group_name, hdf5_file,
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
548 base_group='/', index_names=('epoch',), title="",
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
549 store_timestamp=True, store_cpuclock=True):
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
550 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
551 For other parameters, see Series.__init__
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
552
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
553 Parameters
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
554 ----------
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
555 array_names : array or tuple of str
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
556 Name of each array, in order of the array passed to append(). E.g.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
557 ('layer1_b', 'layer1_W', 'layer2_b', 'layer2_W')
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
558
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
559 new_group_name : str
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
560 Name of a new HDF5 group which will be created under base_group to
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
561 store the new series.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
562
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
563 base_group : str
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
564 Path of the group under which to create the new group which will
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
565 store the series.
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
566
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
567 title : str
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
568 Here the title is attached to the new group, not a table.
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
569
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
570 store_timestamp : bool
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
571 Here timestamp and cpuclock are stored in *each* table
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
572
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
573 store_cpuclock : bool
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
574 Here timestamp and cpuclock are stored in *each* table
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
575 """
220
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
576
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
577 # most other checks done when calling BasicStatisticsSeries
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
578 if type(new_group_name) != str:
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
579 raise TypeError("new_group_name must be a string")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
580 if new_group_name == "":
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
581 raise ValueError("new_group_name must not be empty")
e172ef73cdc5 Ajouté un paquet de type/value checks à SeriesTables, et finalisé les docstrings. Ajouté 3-4 tests. Légers refactorings ici et là sans conséquences externes.
fsavard
parents: 218
diff changeset
582
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
583 base_series_list = []
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
584
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
585 new_group = hdf5_file.createGroup(base_group, new_group_name, title=title)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
586
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
587 stats_functions = {'mean': lambda(x): numpy.mean(x.value),
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
588 'min': lambda(x): numpy.min(x.value),
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
589 'max': lambda(x): numpy.max(x.value),
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
590 'std': lambda(x): numpy.std(x.value)}
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
591
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
592 for name in arrays_names:
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
593 base_series_list.append(
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
594 BasicStatisticsSeries(
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
595 table_name=name,
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
596 hdf5_file=hdf5_file,
210
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
597 index_names=index_names,
dc0d77c8a878 Commented table_series code, changed ParamsStatisticsArray to take shared params instead, create DummySeries to use when we don't want to save a named series
savardf
parents: 208
diff changeset
598 stats_functions=stats_functions,
224
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
599 hdf5_group=new_group._v_pathname,
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
600 store_timestamp=store_timestamp,
0515a8901c6a Corrigé un bug avec store_timestamp/cpuclock, et tests pour éviter ce cas
fsavard
parents: 221
diff changeset
601 store_cpuclock=store_cpuclock))
208
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
602
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
603 SeriesArrayWrapper.__init__(self, base_series_list)
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
604
acb942530923 Completely rewrote my series module, now based on HDF5 and PyTables (in a separate directory called 'tables_series' for retrocompatibility of running code). Minor (inconsequential) changes to stacked_dae.
fsavard
parents:
diff changeset
605