Python3,5中async await特性的实现

2020-02-27 268浏览

1.探索Python 3.5中async/await特性的实现广州齐昌网络科技有限公司赖勇浩
2.2015-6-2
3.2015-6-2
4.自我介绍 4
5.赖勇浩http://laiyonghao.com5
6.创业者，程序员，社区控 2014 - 2005 - 2009 - 6
7.合著有《编写高质量代码：改善Python程序的91个建议》 7 7
8.2009年 8 8
9.2015年7月 9
10.背景知识 10 10
11.Python C语言虚拟机实现协程概念 11
12.Python协程的演化 12 12
13.• Python 2.2, 2001.12 – PEP 255 - Simple Generators 13
14.def generate_ints(N): for i in range(N): yield i 14 14
15.• Python 2.5 2006.8 – PEP 342 -- Coroutines via Enhanced Generators – In 2.5, yield is now an expression – Add send()/throw()/close() 15
16.• Python 3.3 2012.9 – PEP 0380 -- Syntax for Delegating to a Subgenerator 16 16
17.>>> def g(x): ... yield from range(x, 0, -1) ... yield from range(x) ... >>> list(g(5)) [5, 4, 3, 2, 1, 0, 1, 2, 3, 4] 17 17
18.• Python 3.5 2015.9 – PEP 0492 -- Coroutines with async and await syntax 18
19.async def read_data(db): data = await db.fetch('SELECT …') … 19 19
20.async/await的意义 • 定义了原生协程，与生成器彻底区分开来 • 解决with/for的异步需求。 20
21.async with async with EXPR asVAR:BLOCK 21
22.async with • 同步的 __enter__/__exit__ – 无法实现获取、释放资源时复杂耗时的操作异步 • 异步的 __aenter__/__aexit__ 22
23.async with classAsyncContextManager:async def __aenter__(self): await log('entering context') async def __aexit__(self, exc_type, exc, tb): await log('exiting context') 23 23
24.async for async for TARGET inITER:BLOCKelse:BLOCK2 24
25.async for • 同步的 __iter__/__next__ – 无法实现获取、释放资源时复杂耗时的操作异步 • 异步的 __aiter__/__anext__ 25
26.async for classAsyncIterable:async def __aiter__(self): return self async def __anext__(self): data = await self.fetch_data() ifdata:return dataelse:raise StopAsyncIteration async def fetch_data(self): ... 26
27.体验Python协程 27 27
28.pyenv install 3.5.0 28
29.神器！https://github.com/yyuu/pyenv29
30.mkdir py35lab cd py35lab pyenv local 3.5.0 30
31.python –version Python 3.5.0 31 31
32.探索async/await的实现 32 32
33.async/await的字节码 async def foo(): return 42 async def bar(): print(await foo()) import dis dis.dis(bar) 33
34.async/await的字节码 9 0 LOAD_GLOBAL 3 LOAD_GLOBAL 6 CALL_FUNCTION 9 GET_AWAITABLE 10 LOAD_CONST 13 YIELD_FROM 14 CALL_FUNCTION 17 POP_TOP 18 LOAD_CONST 21 RETURN_VALUE 0 (print) 1 (foo) 0 (0 positional, 0 keyword pair) 0 (None) 1 (1 positional, 0 keyword pair) 0 (None) 34
35.GET_AWAITABLE 35
36._PyCoro_GetAwaitableIter 36
37._PyCoro_GetAwaitableIter * This helper function returns an awaitable for `o`: * - `o` if `o` is a coroutine-object; * - `type(o)->tp_as_async->am_await(o)` 37
38.am_await static PyObject * coro_await(PyCoroObject *coro) { PyCoroWrapper *cw = ...New(PyCoroWrapper, &_PyCoroWrapper_Type); cw->cw_coroutine = coro; return (PyObject *)cw; } 38
39._PyCoroWrapper_Type PyTypeObject _PyCoroWrapper_Type = { ... PyObject_SelfIter, /* tp_iter */ (iternextfunc)coro_wrapper_iternext, /* tp_iternext */ coro_wrapper_methods, /* tp_methods */ ... }; 39
40.GET_AWAITABLE让coroutine的返回值（Awaitable）入栈 40
41.async/await的字节码 9 0 LOAD_GLOBAL 3 LOAD_GLOBAL 6 CALL_FUNCTION 9 GET_AWAITABLE 10 LOAD_CONST 13 YIELD_FROM 14 CALL_FUNCTION 17 POP_TOP 18 LOAD_CONST 21 RETURN_VALUE 0 (print) 1 (foo) 0 (0 positional, 0 keyword pair) 0 (None) 1 (1 positional, 0 keyword pair) 0 (None) 41 41
42.YIELD_FROM 42
43.YIELD_FROM • 获取栈顶元素, v • 调用_PyGen_Send(..., v)，并返回结果。 43
44.async/await真相 • Native Corotine 就是换了马甲的 generator 44
45.PythonVM中的协程 Corotine Generator typedef struct { _PyGenObject_HEAD(cr) } PyCoroObject; typedef struct { _PyGenObject_HEAD(gi) } PyGenObject; 45
46._PyGenObject_HEAD #define _PyGenObject_HEAD(prefix) \ PyObject_HEAD \ struct _frame *prefix##_frame; \ char prefix##_running; \ PyObject *prefix##_code; \ PyObject *prefix##_weakreflist; \ PyObject *prefix##_name; \ PyObject *prefix##_qualname; 46
47.协程从何处来？ • • • • 代码编译、执行，就是一个 PyCodeObject* co co->co_flag 标识了类型Py3.5:CO_COROUTINE, CO_ITERABLE_COROUTINE async def 使 co_flag 具有 CO_COROTINE if (is_coro) { gen = PyCoro_New(f, name, qualname); } 47
48.协程的运行与终止 • 解释器遇到 YIELD_FROM，调用 _PyGen_Send，主要逻辑在 gen_send_ex 48
49.gen_send_ex • • • • • 做好参数、状态的检查工作参数压栈保存要返回栈帧（PyFrameObject) 设置运行状态标志调用 PyEval_EvalFrameEx 从自己的栈帧执行代码 • 重置运行状态标志 • 恢复现场，异常处理，释放资源，返回结果 49
50.Q&A 50 50