Python tornado multi task

透過 IOLoop 的add_callback + self.finish() 可以讓 tornado 繼續在背景工作,但有一個問題是前一個工作太忙,會造成整個伺服器無法接受新的 request.

這要寫的有效率,似乎滿難的。

發現把某一個 function 加入 @gen.coroutine,主程式遇到這一個 function 自動變成 muti-thread .. @_@;

 

相關文章:

Tornado – getting return value from IOLoop.add_callback()
http://stackoverflow.com/questions/33742023/tornado-getting-return-value-from-ioloop-add-callback

I’m not sure what the difference is between the following two functions:

 IOLoop.add_callback(callback, *args, **kwargs) 

AND

IOLoop.spawn_callback(callback, *args, **kwargs)

You can’t get a return value, at least not in the way you’re thinking of. Your callback can’t even run until the function that called spawn_callback (i.e. on_message) returns. Just do whatever you want to do when that callback finishes in the callback itself (or pass it another callback to invoke with its result).

stack_context is an error-handling mechanism for callback-oriented code. It doesn’t let you handle exceptions in the calling function, but it does let you establish an error handler in the calling function that will “follow” the code even through a chain of multiple add_callback calls. This turned out to be confusing as often as it helped, so now that coroutines are available I recommend using coroutines as much as possible and when you do need to fall back to raw callbacks, use spawn_callback instead of add_callback and put your try/except handler in the callback itself.


https://huangxiaohen2738.gitbooks.io/tornado-tcp-program/content/chapter2-2.html

2.2 ioloop的回调函数

1.IOLoop.add_callback(callback, args, *kwargs)

这个函数是最简单的,在ioloop开启后执行的回调函数callback,args和*kwargs都是这个回调函数的参数。一般我们的server都是单进程单线程的,即使是多线程,那么这个函数也是安全的。

8.IOLoop.spawn_callback(callback, args, *kwargs)

这个函数也是去执行一个回调函数,但是和上面说过的其他callback不同,它和回调者的栈上下文没有关联,因此呢,他比较时候去做一些独立的功能回调。


Tornado Execution Order Spawn callback
http://stackoverflow.com/questions/38696131/tornado-execution-order-spawn-callback

No, in fact you’re guaranteed that all the functions are spawned before any of them starts running, because first does not yield between spawning func and spawning func2. You can verify this yourself by testing your code:

from tornado import gen, ioloop

@gen.coroutine
def func():
    print('func started')
    yield gen.moment
    print('func done')


@gen.coroutine
def func2():
    print('func2 started')
    yield gen.moment
    print('func2 done')


@gen.coroutine
def first():
    for i in range(2):
        ioloop.IOLoop.current().spawn_callback(func)

    ioloop.IOLoop.current().spawn_callback(func2)
    yield gen.sleep(1)

ioloop.IOLoop.current().run_sync(first)

It prints:

func started
func started
func2 started
func done
func done
func2 done

See, func2 begins before the coroutines running func complete.

To accomplish what you want:

@gen.coroutine
def first():
    yield [func() for i in range(2)]
    ioloop.IOLoop.current().spawn_callback(func2)

This prints:

func started
func started
func done
func done
func2 started
func2 done

If you want first to wait for func2 to finish before it exits, then:

@gen.coroutine
def first():
    yield [func() for i in range(2)]
    yield func2()

For more info on calling coroutines from coroutines, see my Refactoring Tornado Coroutines.


官方文件:

http://www.tornadoweb.org/en/stable/ioloop.html

 

Running in the background

PeriodicCallback is not normally used with coroutines. Instead, a coroutine can contain a while True: loop and use tornado.gen.sleep:

@gen.coroutine
def minute_loop():
    while True:
        yield do_something()
        yield gen.sleep(60)

# Coroutines that loop forever are generally started with
# spawn_callback().
IOLoop.current().spawn_callback(minute_loop)

Sometimes a more complicated loop may be desirable. For example, the previous loop runs every 60+N seconds, where N is the running time of do_something(). To run exactly every 60 seconds, use the interleaving pattern from above:

@gen.coroutine
def minute_loop2():
    while True:
        nxt = gen.sleep(60)   # Start the clock.
        yield do_something()  # Run while the clock is ticking.
        yield nxt             # Wait for the timer to run out.

 

發佈留言

發佈留言必須填寫的電子郵件地址不會公開。 必填欄位標示為 *