認識 tornado 的 coroutine 和 asynchronous

Coroutine 可以讓我們在程式中按照自己的意思去安排執行順序,允許短暫離開 function 並且保留 local variable 的狀態,等到某個時間點再跳回來,從上一次離開的地方繼續。

在 single thread 下,你執行到一個 blocking function,這時候如果讓 CPU 去做其他事情是不是很好,等到 I/O 有回應的,再跳回來原本的地方繼續執行。前提是執行在 single thread.

官方的說明文件:
http://www.tornadoweb.org/en/stable/guide/coroutines.html

http://www.tornadoweb.org/en/stable/guide/async.html

http://www.tornadoweb.org/en/stable/gen.html

 

用 tornado 做网站 (7)
http://wiki.jikexueyuan.com/project/start-learning-python/309.html

Blocking tasks in Tornado
https://lbolla.info/blog/2013/01/22/blocking-tornado

使用tornado让你的请求异步非阻塞
http://www.dongwm.com/archives/shi-yong-tornadorang-ni-de-qing-qiu-yi-bu-fei-zu-sai/


Here is a sample synchronous function:

from tornado.httpclient import HTTPClient

def synchronous_fetch(url):
    http_client = HTTPClient()
    response = http_client.fetch(url)
    return response.body

 

修改成非同步:

And here is the same function rewritten to be asynchronous with a callback argument:

from tornado.httpclient import AsyncHTTPClient

def asynchronous_fetch(url, callback):
    http_client = AsyncHTTPClient()
    def handle_response(response):
        callback(response.body)
    http_client.fetch(url, callback=handle_response)

 

再修改成 coroutine:

Here is the coroutine version of our sample function, which is very similar to the original synchronous version:

from tornado import gen

@gen.coroutine
def fetch_coroutine(url):
    http_client = AsyncHTTPClient()
    response = yield http_client.fetch(url)
    raise gen.Return(response.body)

The statement raise gen.Return(response.body) is an artifact of Python 2, in which generators aren’t allowed to return values. To overcome this, Tornado coroutines raise a special kind of exception called a Return. The coroutine catches this exception and treats it like a returned value. In Python 3.3 and later, a return response.body achieves the same result

 


asynchronous 範例,發出一個 HTTP request 去抓 Yahoo weather 的資訊,然後利用 XML parser 從回應得資料中取出溫度:

from tornado.ioloop import IOLoop
from tornado.web import Application, RequestHandler, asynchronous
from tornado.httpclient import AsyncHTTPClient
from xml.dom import minidom
 
class MainHandler(RequestHandler):
    url = "http://weather.yahooapis.com/forecastrss?w=2306179&u=c"
 
    @asynchronous
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch(self.url, callback=self._on_fetch)
 
    def _on_fetch(self, response):
        degree = self._parse_xml(response.body)
        self.finish("Taipei: %d" % degree)
 
    def _parse_xml(self, xml):
        xml_doc = minidom.parseString(xml)
        weather_list = xml_doc.getElementsByTagName('yweather:condition')
        degree = float(weather_list[0].attributes['temp'].value)
        return degree
 
if __name__ == "__main__":
    application = Application([
        (r"/", MainHandler),
    ])
    application.listen(8888)
    IOLoop.instance().start()

換成 coroutine:

from tornado.ioloop import IOLoop
from tornado.web import Application, RequestHandler, asynchronous
from tornado.httpclient import AsyncHTTPClient
import tornado.gen as gen
from xml.dom import minidom
 
class MainHandler(RequestHandler):
    url = "http://weather.yahooapis.com/forecastrss?w=2306179&u=c"
 
    @gen.coroutine
    def get(self):
        http_client = AsyncHTTPClient()
        response = yield http_client.fetch(self.url)
        degree = self._parse_xml(response.body)
        self.finish("Taipei: %d" % degree)
 
    def _parse_xml(self, xml):
        xml_doc = minidom.parseString(xml)
        weather_list = xml_doc.getElementsByTagName('yweather:condition')
        degree = float(weather_list[0].attributes['temp'].value)
        return degree
 
if __name__ == "__main__":
    application = Application([
        (r"/", MainHandler),
    ])
    application.listen(8888)
    IOLoop.instance().start()

asynchronous + callback 範例:

class MainHandler(RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        req1(argument1, callback=self._res1)
 
    @tornado.web.asynchronous
    def _res1(self, response1):
        ...do something with response
        req2(argument2, callback=self._res2)
 
    def _res2(self, response2):
        ...do something with response
        self.finish("result...")

換成 coroutine

 

class MainHandler(RequestHandler):    
    @tornado.gen.coroutine
    def get(self):
        response1 = yield req1(argument1)
        ...do something with response1
        response2 = yield req2(argument2)
        ...do something with response2
        self.finish("result...")

資料來源:
Coroutine in Tornado Web Framework
http://blog.yslin.tw/2014/04/coroutine-in-tornado-web-framework.html


sample:
http://stackoverflow.com/questions/8812715/using-a-simple-python-generator-as-a-co-routine-in-a-tornado-async-handler?rq=1

Here’s a basic version of what you are describing. To avoid blocking you can pass your generator to the IOLoop via a callback function. The trick here is since you are not using a process that does actual IO and so has no os level process/file handler to add to the IOLoop via add_handler, you can instead use a simple add_callback call and call it repeatedly from within the callback function to keep the function in the IOLoop callback queue until the generator has finished.

import tornado.httpserver
import tornado.ioloop
import tornado.web

class TextHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        self.generator = self.generate_text(1000)
        tornado.ioloop.IOLoop.instance().add_callback(self.loop)

    def loop(self):
        try:
            text = self.generator.next()
            self.write(text)
            tornado.ioloop.IOLoop.instance().add_callback(self.loop)
        except StopIteration:
            self.finish()

    def generate_text(self, n):
        for x in xrange(n):
            if not x % 15:
                yield "FizzBuzz\n"
            elif not x % 5:
                yield "Buzz\n"
            elif not x % 3:
                yield "Fizz\n"
            else:
                yield "%s\n" % x

application = tornado.web.Application([
    (r"/text/", TextHandler),
])

http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()

發佈留言

發佈留言必須填寫的電子郵件地址不會公開。 必填欄位標示為 *