带有Python请求的异步请求
我尝试了python 请求库文档中提供的示例。
使用async.map(rs)
,我得到了响应代码,但是我想获得所请求的每个页面的内容。例如,这不起作用:
out = async.map(rs)
print out[0].content
-
注意
下面的答案是不适用于请求v0.13.0 +。编写此问题后,异步功能已移至grequests
。但是,你可以将其替换requests
为grequests
下面的内容,它应该可以工作。我已经留下了这个答案,以反映原始问题,即有关使用请求
<v0.13.0
的问题。要
async.map
异步执行多个任务,你必须:- 为每个对象定义一个函数(你的任务)
- 将该功能添加为请求中的事件挂钩
- 调用async.map所有请求/操作的列表
例:
from requests import async # If using requests > v0.13.0, use # from grequests import async urls = [ 'http://python-requests.org', 'http://httpbin.org', 'http://python-guide.org', 'http://kennethreitz.com' ] # A simple task to do to each response object def do_something(response): print response.url # A list to hold our things to do via async async_list = [] for u in urls: # The "hooks = {..." part is where you define what you want to do # # Note the lack of parentheses following do_something, this is # because the response will be used as the first argument automatically action_item = async.get(u, hooks = {'response' : do_something}) # Add the task to our list of things to do via async async_list.append(action_item) # Do our list of things to do via async async.map(async_list)