ift6266: scripts/launch_generate100.py annotate

annotate scripts/launch_generate100.py @ 273:7800be7bce66

changes in ttf2jpg and pycaptcha to load a file list with cPickle (instead of doing a listdir()) in order to have always the same list order from different machine (to reproduce generation)

author	Xavier Glorot <glorotxa@iro.umontreal.ca>
date	Sun, 21 Mar 2010 16:45:48 -0400
parents	17c72763d574
children	44409b6652aa

rev	line source
136 4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	1 #!/usr/bin/env python
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	2
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	3 import os
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	4 dir1 = "/data/lisa/data/ift6266h10/"
4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	5
255 17c72763d574 added PNIST generation script Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 203 diff changeset	6 mach = "maggie16.iro.umontreal.ca,maggie15.iro.umontreal.ca"
203 fd1b5237e49e Actual script used to launch images generation on 4 machines boulanni <nicolas_boulanger@hotmail.com> parents: 157 diff changeset	7
136 4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	8 for i,s in enumerate(['valid','test']):
157 221799d79188 Ajouté seed 100 à 207 aux ensembles de données générés boulanni <nicolas_boulanger@hotmail.com> parents: 136 diff changeset	9 for j,c in enumerate([0.3,0.5,0.7,1]):
136 4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	10 l = str(c).replace('.','')
203 fd1b5237e49e Actual script used to launch images generation on 4 machines boulanni <nicolas_boulanger@hotmail.com> parents: 157 diff changeset	11 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P%s_%s_data.ft -p %sdata/P%s_%s_params -x %sdata/P%s_%s_labels.ft -f %s%s_data.ft -l %s%s_labels.ft -c %socr_%s_data.ft -d %socr_%s_labels.ft -m 0.3 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s %d -y %d" % (mach, dir1, l, s, dir1, l, s, dir1, l, s, dir1, s, dir1, s, dir1, s, dir1, s, [20000,80000][i], 200+i*4+j))
136 4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	12
255 17c72763d574 added PNIST generation script Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 203 diff changeset	13 #P07
136 4a78c9b83fee Added script to launch the ~108 data generation jobs with dbidispatch boulanni <nicolas_boulanger@hotmail.com> parents: diff changeset	14 for i in range(100):
203 fd1b5237e49e Actual script used to launch images generation on 4 machines boulanni <nicolas_boulanger@hotmail.com> parents: 157 diff changeset	15 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/P07_train%d_data.ft -p %sdata/P07_train%d_params -x %sdata/P07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d" % (mach, dir1, i, dir1, i, dir1, i, dir1, dir1, dir1, dir1, 100+i))
255 17c72763d574 added PNIST generation script Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 203 diff changeset	16
17c72763d574 added PNIST generation script Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 203 diff changeset	17 #PNIST07
17c72763d574 added PNIST generation script Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 203 diff changeset	18 for i in range(100):
17c72763d574 added PNIST generation script Xavier Glorot <glorotxa@iro.umontreal.ca> parents: 203 diff changeset	19 os.system("dbidispatch --condor --os=fc4,fc7,fc9 --machine=%s ./run_pipeline.sh -o %sdata/PNIST07_train%d_data.ft -p %sdata/PNIST07_train%d_params -x %sdata/PNIST07_train%d_labels.ft -f %strain_data.ft -l %strain_labels.ft -c %socr_train_data.ft -d %socr_train_labels.ft -m 0.7 -z 0.1 -a 0.1 -b 0.25 -g 0.25 -s 819200 -y %d -t %d" % (mach, dir1, i, dir1, i, dir1, i, dir1, dir1, dir1, dir1, 100+i,1))