collect_data.py 文件源码

python
阅读 22 收藏 0 点赞 0 评论 0

项目:ebay 作者: fgscivittaro 项目源码 文件源码
def clean_links_and_dynamically_scrape(filename, interval, num_retries = 10):
    """
    Repeatedly updates the link list and runs the scraper every time the
    specified interval has passed and continuously appends the data to a file.
    """

    def job():
        clean_links_and_scrape(filename, num_retries)

    job()
    schedule.every(interval).minutes.do(job)

    while True:
        schedule.run_pending()
        time.sleep(30)

    print "Dynamic scraping finished"
评论列表
文章目录


问题


面经


文章

微信
公众号

扫码关注公众号