Mercurial > ift6266

annotate scripts/launch_generate100.py @ 595:da46a62ce402

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
submitted JMLR pdf
author Yoshua Bengio <bengioy@iro.umontreal.ca>
date Tue, 05 Oct 2010 15:07:33 -0400
parents 543ae35e387e
children
rev   line source
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
1 #!/usr/bin/env python
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
2
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
3 import os
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
4 dir1 = "/data/lisa8/data/ift6266h10/"
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
5 dir2="/data/lisa/data/ift6266h10/"
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
6
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
7 mach = "maggie16.iro.umontreal.ca,zappa8.iro.umontreal.ca,maggie15.iro.umontreal.ca,brams04.iro.umontreal.ca"
203
fd1b5237e49e Actual script used to launch images generation on 4 machines
boulanni <nicolas_boulanger@hotmail.com>
parents: 157
diff changeset
8
274
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
9 #test and valid sets
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
10 for i,s in enumerate(['valid','test']):
157
221799d79188 Ajouté seed 100 à 207 aux ensembles de données générés
boulanni <nicolas_boulanger@hotmail.com>
parents: 136
diff changeset
11 for j,c in enumerate([0.3,0.5,0.7,1]):
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
12 l = str(c).replace('.','')
274
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
13 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P%s_%s_data.ft -p %sdata/P%s_%s_params -x %sdata/P%s_%s_labels.ft -f %s%s_data.ft -l %s%s_labels.ft -c %socr_%s_data.ft -d %socr_%s_labels.ft -m %s -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s %d -y %d" % (mach, dir1, l, s, dir1, l, s, dir1, l, s, dir1, s, dir1, s, dir1, s, dir1, s, c ,[20000,80000][i], 200+i*4+j))
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
14
255
17c72763d574 added PNIST generation script
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 203
diff changeset
15 #P07
136
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch
boulanni <nicolas_boulanger@hotmail.com>
parents:
diff changeset
16 for i in range(100):
203
fd1b5237e49e Actual script used to launch images generation on 4 machines
boulanni <nicolas_boulanger@hotmail.com>
parents: 157
diff changeset
17 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P07_train%d_data.ft -p %sdata/P07_train%d_params -x %sdata/P07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir1, dir1, dir1, dir1, 100+i))
255
17c72763d574 added PNIST generation script
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 203
diff changeset
18
17c72763d574 added PNIST generation script
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 203
diff changeset
19 #PNIST07
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
20 for i in xrange(10,100):
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
21 os.system("dbidispatch --condor --mem=3900 --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/PNIST07_train%d_data.ft -p %sdata/PNIST07_train%d_params -x %sdata/PNIST07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d -t 1" % (mach, dir1, i, dir1, i, dir1, i, dir2, dir2, dir2, dir2, 100+i))
274
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
22
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
23
44409b6652aa small change in generating script (bug fix)
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 255
diff changeset
24
370
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
25 #PNIST_full_noise
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
26 for i in range(100):
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
27 os.system("dbidispatch --condor --mem=3900 --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/Pin07_train%d_data.ft -p %sdata/Pin07_train%d_params -x %sdata/Pin07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir2, dir2, dir2, dir2,100+i))
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
28
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
29
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
30
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
31 #P07_safe
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
32 for i in xrange(89,100,1):
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
33 os.system("dbidispatch --condor --mem=3900 --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P07safe_train%d_data.ft -p %sdata/P07safe_train%d_params -x %sdata/P07safe_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.0 -b 0.0 -g 0.0 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir2, dir2, dir2,dir2,100+i))
543ae35e387e changes in generation script for the new data
Xavier Glorot <glorotxa@iro.umontreal.ca>
parents: 274
diff changeset
34