Change Number Of Running Spiders Scrapyd
Hey so I have about 50 spiders in my project and I'm currently running them via scrapyd server. I'm running into an issue where some of the resources I use get locked and make my s
Solution 1:
This can be controlled by scrapyd settings. Set max_proc
to 1
:
max_proc
The maximum number of concurrent Scrapy process that will be started.
Post a Comment for "Change Number Of Running Spiders Scrapyd"