Skip to content Skip to sidebar Skip to footer

What Is The Correct Form Of Work With Cookies In Scrapy

I'm very newbie,I am working with scrapy in a web that use cookies, This is a problem for me , because I can obtain data the a web without cookies but obtain the data of a web wit

Solution 1:

You are very close! The contract for the parse() method is that it yields (or returns an iterable) of Items, Requests, or a mix of both. In your case, all you should have to do is

yield request_with_cookies

and your parse() method will be run again with a Response object produced from requesting that URL with those cookies.

http://doc.scrapy.org/en/latest/topics/spiders.html?highlight=parse#scrapy.spider.Spider.parsehttp://doc.scrapy.org/en/latest/topics/request-response.html

Post a Comment for "What Is The Correct Form Of Work With Cookies In Scrapy"