spider.py 文件源码

python
阅读 30 收藏 0 点赞 0 评论 0

项目:wsp 作者: wangybnet 项目源码 文件源码
def start_requests(cls, spiders, start_urls, *, middleware=None):
        try:
            res = []
            for spider in spiders:
                for r in spider.start_requests(start_urls):
                    if inspect.iscoroutine(r):
                        r = await r
                    if r:
                        res.append(r)
            if middleware:
                res = await cls._handle_start_requests(res, middleware)
        except Exception:
            log.warning("Unexpected error occurred when generating start requests", exc_info=True)
            return ()
        else:
            return res
评论列表
文章目录


问题


面经


文章

微信
公众号

扫码关注公众号