[xcat-user] xCAT 1.3b2 and Torque 2.1.8 STDOUT andSTDERR problem

[Egan Ford]

You can use qsub -o {hostname}:path and -e {hostname}:path to pick any
directory for the output to be copied to after the job completes.

But for spooling to ~/.pbs_spool vs /var/spool/pbs/spool on the mom I think
it is a compile time option only.

> -----Original Message-----
> From: xcat-user-bounces at lists.xcat.org 
> [mailto:xcat-user-bounces at lists.xcat.org] On Behalf Of Joseph Hargitai
> Sent: Friday, December 14, 2007 12:42 PM
> To: xCAT Users Mailing list
> Subject: Re: RE: [xcat-user] xCAT 1.3b2 and Torque 2.1.8 
> STDOUT andSTDERR problem
> Importance: High
> 
> 
> 
> is there perhaps a runtime flag in to set/unset this? 
> 
> j
> 
> ----- Original Message -----
> From: Egan Ford <egan at sense.net>
> Date: Friday, December 14, 2007 2:06 pm
> Subject: RE: [xcat-user] xCAT 1.3b2 and Torque 2.1.8 STDOUT 
> and STDERR problem
> 
> > --disable-spool  if disabled, TORQUE will create output and 
> error files
> >                  directly in $HOME/.pbs_spool if it exists 
> or in $HOME
> >                  otherwise.  By default, TORQUE will spool files in
> >                  $TORQUEHOME/spool and copy them to the users home
> >                  directory when the job completes.
> > 
> > If you used the latest xCAT torquemaker, then 
> --disable-spool was used
> > when
> > building torque.  Without --disable-spool a user has to 
> wait until the 
> > job
> > completely finishes/fails to get any intermediate results 
> because the
> > temporary output files are on the mom node in 
> /var/spool/pbs/spool (I 
> > guess
> > they can ssh in to see them).  With --disable-spool a user 
> can tail -f
> > ~/.pbs_spool/file.[oe] while the job runs to get 
> intermediate results. 
> >  The
> > user may opt to kill the job sooner than later.
> > 
> > --disable-spool + your mom config change may be the best of both 
> > worlds. Can you create a .pbs_spool directory in your home 
> and verify 
> > that you can tail -f the output while it gets created and 
> then after 
> > job completion it
> > gets copied to $PBS_WORKDIR?
> > 
> > Thanks.
> > 
> > > -----Original Message-----
> > > From: xcat-user-bounces at lists.xcat.org
> > > [mailto:xcat-user-bounces at lists.xcat.org] On Behalf Of 
> Chris Beggio
> > > Sent: Thursday, December 13, 2007 3:55 PM
> > > To: xcat-user at lists.xcat.org
> > > Subject: [xcat-user] xCAT 1.3b2 and Torque 2.1.8 STDOUT and 
> > > STDERR problem
> > > Importance: High
> > > 
> > > 
> > > 
> > > xCAT users,
> > > 
> > > I have recently pounded a new flat spot on my head with a PBS
> > > problem on a recently installed cluster. Versions are as follows:
> > > 
> > > Red Hat Enterprise Linux WS release 4 (Nahant Update 5)
> > > 
> > > xCAT 1.3.0-beta2
> > > Fri Jul 13 00:18:24 MDT 2007
> > > 
> > > Torque 2.1.8
> > > 
> > > The problem is that out of the box, PBS was not copying the
> > > PBS prolog and epilog, along with STDERR (${PBS_JOBID}.ER) 
> > > and STDOUT (${PBS_JOBID}.OU) or the joined output and error 
> > > stream to the directory where the job was executed 
> > > (${PBS_O_WORKDIR}), and instead was copying them to the user 
> > > home directory (${PBS_O_HOME}). I compared two identical 
> > > machines and all the versions and configurations appear to be 
> > > the same, but while one was copying output to 
> > > ${PBS_O_WORKDIR}, the more recently installed was not. Then I 
> > > changed /var/spool/pbs/mom_priv/config on the compute nodes 
> > > to include:
> > > 
> > > $usecp *:/home  /home
> > > 
> > > This line does not exist on the other working cluster.
> > > Everything works now and the file 
> > > ${PBS_JOBNAME}.o${PBS_JOBID} is now deposited in 
> > > ${PBS_O_WORKDIR} as expected.  What am I missing?
> > > 
> > > Thanks and Happy Festivus.
> > > 
> > > Chris
> > > --
> > > __________________________________________________
> > > Chris Beggio
> > > 
> > > 1600 Computing Support Team
> > > 
> > > Commercial Data Systems
> > > Contracted by Sandia National Laboratories
> > > 
> > > Phone: 505-284-8001
> > > Fax: 505-284-6078
> > > Email: cabeggi at sandia.gov
> > > __________________________________________________
> > > 
> > > 
> > > _______________________________________________
> > > xcat-user mailing list
> > > xcat-user at lists.xcat.org
> > > http://www.xcat.org/mailman/listinfo/xcat-user
> > 
> > _______________________________________________
> > xcat-user mailing list
> > xcat-user at lists.xcat.org 
> > http://www.xcat.org/mailman/listinfo/xcat-user
> _______________________________________________
> xcat-user mailing list
> xcat-user at lists.xcat.org 
> http://www.xcat.org/mailman/listinfo/xcat-user