Scrapy Twisted Connectionlost Error
I am learning scrapy and am having a hard time trying to figure out this issue. My spider will not crawl the macys website and keeps throwing the following error: [
Solution 1:
It seems that your IP from you are launching your scraper has been blacklisted.
You might want to read the following: https://doc.scrapy.org/en/latest/topics/practices.html#avoiding-getting-banned
Also, you might want to tune the settings concerning the number of requests outputted by scrapy: CONCURRENT_REQUESTS, DOWNLOAD_DELAY, etc.
Post a Comment for "Scrapy Twisted Connectionlost Error"