We start by loading the backgroundjobs
library and defining a few trivial functions to illustrate things with.
from __future__ import print_function
from IPython.lib import backgroundjobs as bg
import sys
import time
def sleepfunc(interval=2, *a, **kw):
args = dict(interval=interval,
args=a,
kwargs=kw)
time.sleep(interval)
return args
def diefunc(interval=2, *a, **kw):
time.sleep(interval)
raise Exception("Dead job with interval %s" % interval)
def printfunc(interval=1, reps=5):
for n in range(reps):
time.sleep(interval)
print('In the background...', n)
sys.stdout.flush()
print('All done!')
sys.stdout.flush()
Now, we can create a job manager (called simply jobs
) and use it to submit new jobs.
jobs = bg.BackgroundJobManager()
# Start a few jobs, the first one will have ID # 0
jobs.new(sleepfunc, 4)
jobs.new(sleepfunc, kw={'reps':2})
jobs.new('printfunc(1,3)')
# This makes a couple of jobs which will die. Let's keep a reference to
# them for easier traceback reporting later
diejob1 = jobs.new(diefunc, 1)
diejob2 = jobs.new(diefunc, 2)
Starting job # 0 in a separate thread. Starting job # 2 in a separate thread. Starting job # 3 in a separate thread. Starting job # 4 in a separate thread. Starting job # 5 in a separate thread.
You can check the status of your jobs at any time:
jobs.status()
In the background... 0 Running jobs: 0 : <function sleepfunc at 0x102cc6848> 2 : <function sleepfunc at 0x102cc6848> 3 : printfunc(1,3) 5 : <function diefunc at 0x102cc68c0> Dead jobs: 4 : <function diefunc at 0x102cc68c0>
For any completed job, you can get its result easily:
jobs[0].result
j0 = jobs[0]
j0.join?
You can get the traceback of any dead job. Run the line below again interactively until it prints a traceback (check the status of the job):
print "Status of diejob1:", diejob1.status
diejob1.traceback() # jobs.traceback(4) would also work here, with the job number
In the background... 1 In the background... 2 All done!
File "<ipython-input-5-a90bd59af669>", line 1 print "Status of diejob1:", diejob1.status ^ SyntaxError: invalid syntax
This will print all tracebacks for all dead jobs:
jobs.traceback()
Traceback for: <BackgroundJob #4: <function diefunc at 0x102cc68c0>> --------------------------------------------------------------------------- Exception Traceback (most recent call last) /Users/bgranger/Documents/Computing/IPython/code/ipython/IPython/lib/backgroundjobs.pyc in call(self) 489 490 def call(self): --> 491 return self.func(*self.args, **self.kwargs) <ipython-input-1-7391f8ae281b> in diefunc(interval, *a, **kw) 14 def diefunc(interval=2, *a, **kw): 15 time.sleep(interval) ---> 16 raise Exception("Dead job with interval %s" % interval) 17 18 def printfunc(interval=1, reps=5): Exception: Dead job with interval 1 Traceback for: <BackgroundJob #5: <function diefunc at 0x102cc68c0>> --------------------------------------------------------------------------- Exception Traceback (most recent call last) /Users/bgranger/Documents/Computing/IPython/code/ipython/IPython/lib/backgroundjobs.pyc in call(self) 489 490 def call(self): --> 491 return self.func(*self.args, **self.kwargs) <ipython-input-1-7391f8ae281b> in diefunc(interval, *a, **kw) 14 def diefunc(interval=2, *a, **kw): 15 time.sleep(interval) ---> 16 raise Exception("Dead job with interval %s" % interval) 17 18 def printfunc(interval=1, reps=5): Exception: Dead job with interval 2
The job manager can be flushed of all completed jobs at any time:
jobs.flush()
Flushing 3 Completed jobs. Flushing 2 Dead jobs.
After that, the status is simply empty:
jobs.status()
It's easy to wait on a job:
j = jobs.new(sleepfunc, 2)
print("Will wait for j now...")
sys.stdout.flush()
j.join()
print("Result from j:")
j.result
Starting job # 0 in a separate thread. Will wait for j now... Result from j:
{'args': (), 'interval': 2, 'kwargs': {}}