Scrapy Unable To Make Request() Callback
I am trying to make recursive parsing script with Scrapy, but Request() function doesn't call callback function suppose_to_parse(), nor any function provided in callback value. I t
Solution 1:
Move the yield outside of the if
statement:
for link in hxs.select('//a/@href').extract():
url = link
ifnot link.startswith('http://') andnot link.startswith('#'):
url = (self.start_urls[0] + link).replace('//','/')
print url
yield Request(url, callback=self.suppose_to_parse)
Solution 2:
I'm not an expert but i tried your code and i think that the problem is not on the Request, the generated url's seems to be broken, if you add some url's to a list and iterate thorough them and yield the Request with the callback, it works fine.
Post a Comment for "Scrapy Unable To Make Request() Callback"