Mercurial > ift6266

annotate scripts/launch_generate100.py @ 647:47af8a002530 tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
changed Theano to ift6266 and remove numpy as we do not use code from numpy in this repository
author Razvan Pascanu <r.pascanu@gmail.com>
date Wed, 17 Oct 2012 09:26:14 -0400
parents 543ae35e387e
children
rev   line source
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
1 #!/usr/bin/env python
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
2
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
3 import os
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
4 dir1 = "/data/lisa8/data/ift6266h10/"
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
5 dir2="/data/lisa/data/ift6266h10/"
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
6
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
7 mach = "maggie16.iro.umontreal.ca,zappa8.iro.umontreal.ca,maggie15.iro.umontreal.ca,brams04.iro.umontreal.ca"
203
fd1b5237e49e Actual script used to launch images generation on 4 machines
boulanni <nicolas_boulanger@hotmail.com>
parents: 157
diff changeset
8
274
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
9 #test and valid sets
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
10 for i,s in enumerate(['valid','test']):
157
221799d79188 Ajouté seed 100 à 207 aux ensembles de données générés
boulanni <nicolas_boulanger@hotmail.com>
parents: 136
diff changeset
11 for j,c in enumerate([0.3,0.5,0.7,1]):
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
12 l = str(c).replace('.','')
274
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
13 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P%s_%s_data.ft -p %sdata/P%s_%s_params -x %sdata/P%s_%s_labels.ft -f %s%s_data.ft -l %s%s_labels.ft -c %socr_%s_data.ft -d %socr_%s_labels.ft -m %s -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s %d -y %d" % (mach, dir1, l, s, dir1, l, s, dir1, l, s, dir1, s, dir1, s, dir1, s, dir1, s, c ,[20000,80000][i], 200+i*4+j))
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
14
255
17c72763d574 added PNIST generation script
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 203
diff changeset
15 #P07
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
16 for i in range(100):
203
fd1b5237e49e Actual script used to launch images generation on 4 machines
boulanni <nicolas_boulanger@hotmail.com>
parents: 157
diff changeset
17 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P07_train%d_data.ft -p %sdata/P07_train%d_params -x %sdata/P07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir1, dir1, dir1, dir1, 100+i))
255
17c72763d574 added PNIST generation script
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 203
diff changeset
18
17c72763d574 added PNIST generation script
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 203
diff changeset
19 #PNIST07
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
20 for i in xrange(10,100):
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
21 os.system("dbidispatch --condor --mem=3900 --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/PNIST07_train%d_data.ft -p %sdata/PNIST07_train%d_params -x %sdata/PNIST07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d -t 1" % (mach, dir1, i, dir1, i, dir1, i, dir2, dir2, dir2, dir2, 100+i))
274
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
22
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
23
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
24
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
25 #PNIST_full_noise
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
26 for i in range(100):
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
27 os.system("dbidispatch --condor --mem=3900 --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/Pin07_train%d_data.ft -p %sdata/Pin07_train%d_params -x %sdata/Pin07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir2, dir2, dir2, dir2,100+i))
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
28
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
29
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
30
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
31 #P07_safe
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
32 for i in xrange(89,100,1):
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
33 os.system("dbidispatch --condor --mem=3900 --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P07safe_train%d_data.ft -p %sdata/P07safe_train%d_params -x %sdata/P07safe_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.0 -b 0.0 -g 0.0 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir2, dir2, dir2,dir2,100+i))
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
34