Skip to content Skip to sidebar Skip to footer
Showing posts with the label Screen Scraping

Requests.get(url) Not Returning For This Specific Url

I'm trying to use requests.get(url).text to get the HTML from this website. However, when reque… Read more Requests.get(url) Not Returning For This Specific Url

Get Instagram Followers

I want to parse a website's followers count with BeautifulSoup. This is what I have so far: use… Read more Get Instagram Followers

Does Httplib2 Support Http Proxy At All? Socks Proxy Works But Not Http

Here is my code. I cannot get any http proxy to work. Socks proxy (socks4/5) works fine though. Any… Read more Does Httplib2 Support Http Proxy At All? Socks Proxy Works But Not Http

Python Www Macro

i need something like iMacros for Python. It would be great to have something like that: browse_to(… Read more Python Www Macro

Scraping Way2sms With Mechanize

I am trying to send an sms with by scraping way2sms.com, but I am unable to login into way2sms.com … Read more Scraping Way2sms With Mechanize

How To Make Mechanize Not Fail With Forms On This Page?

import mechanize url = 'http://steamcommunity.com' br=mechanize.Browser(factory=mechanize… Read more How To Make Mechanize Not Fail With Forms On This Page?

Beautiful Soup And UTidy

I want to pass the results of utidy to Beautiful Soup, ala: page = urllib2.urlopen(url) options = d… Read more Beautiful Soup And UTidy

How To Scrape ID-less Website Elements With XPath-only Regex Patterns

There are several similar questions related to the usage of regex in XPath searches -- However, som… Read more How To Scrape ID-less Website Elements With XPath-only Regex Patterns