LSF system, stdout/stderr always goes to my email

[iiiir]

I am on the LSF system. I have a job file like this:

job_begin
name jobA
time 1h
memory 500m
queue normal
project CompBio
sge_options -A swang -e example2.jobA.e.txt -o example2.jobA.o.txt
cmd echo "hello from job jobA"
job_end

job_begin
name jobB
time 30m
memory 1g
queue normal
project CompBio
cmd echo "hello from job jobB";
job_end

order jobB after jobA
log_dir /home/iiiit/app/SJM/doc/log

regardless of how i do, the "-e example2.jobA.e.txt -o example2.jobA.o.txt" or "log_dir /home/iiiit/app/SJM/doc/log" is not effective. all notifications goes to my email (for jobA and jobB)
Can you help?
Thanks,

[nathankw]

Hi,

I haven't used SJM on LSF, but based on my knowledge of how things work on SGE, I should say that you shouldn't be getting any emails unless you specified the --mail option when running SJM on the command-line, or unless you specify the -M option in sge_options in your SJM file.

Also, I don't believe it will work as you expect if you are setting both log_dir and the -e and -o sge_options. When setting log_dir, all output goes in that directory, including the jobs stdout and stderr files, and those log files will have the default names of job_name.ejob_id and job_name.ojob_id.

Also note that if you don't specify an absolute directory path when using -e and -o (which I suggest doing), then (if log_dir isn't specified) the stdout and stderr job files will be written in your home directory.

[iiiir]

Thanks much.
I think getting emails is something enforced by our LSF system under the condition that -e -o is not specified. Looks like in my case these parameters were not passed to the job submit system -- i don't know i am just guessing.

Regard the log_dir, it will always be empty independent of how i run sjm or how my job looks like.

I even tried the sched_options for -e and -o, which did not work for me as well.

I am getting our HPCF support involved and hopefully work out something.

Shuoguo

[nathankw]

Hey Shuoguo,
I may know what the issue is. Please try sched_options instead of sge_options. sched_options is what's given in the MANUAL document, and even though sge_options does work for SGE, it likely shouldn't be used with LSF. I'm thinking that sched_options is universal with either LSF or SGE.

Please try that and let me know if it works or not.

[iiiir]

Thanks for the suggestion.
I actually tried sched_options before I started the thread. e.g.

job_begin
  name jobA
  time 1h
  memory 500m
  project CompBio
  sched_options -o example.jobA.o.txt -e example.jobA.e.txt
  cmd_begin
    echo "hello from job jobA"
  cmd_end
job_end

job_begin
  name jobB
  time 30m
  memory 1g
  project CompBio
  sched_options -u [email protected] -o example.jobB.o.txt -e example.jobB.e.txt
  cmd echo "hello from job jobB"
job_end
order jobB after jobA
log_dir /home/swang/app

for this test.
0. the jobs will finish successfully; but

1. i will not see the *.e.txt and *.o.txt
2. i will not receive emails in [email protected]
3. i will not see any log in /home/swang/app

I will continue to figure out a solution though. Thanks!

[HuanYuu]

hi iiiir
I met the same problem, did you get a solution to solve the problem and could you please share you solution?

[suqingdong]

we can redirect in cmd part, like this:

...
cmd_begin
    COMMAND  1> /path/to/log_dir/xxx.o.log 2> /path/to/log_dir/xxx.e.log
cmd_end
...

PS: need mkdir /path/to/log_dir firstly