Python Google Wrapper For Number Of Results

I've seen some posts already to get the number of search results for a google search, but none is satisfying my needs so far. I want to search a string with blank spaces and get al

Solution 1:

It will just always differ on each send request. Google search results are different on different computers, they want and expect search results to be different from person to person. It just depends on many factors.

In terms of send requests from bots (scripts), it is likely that the results will be the same, but not all the time.

For example:

>>> soup.select_one('#result-stats nobr').previous_sibling
'About 4,100,000,000 results'# in fact, there're was 4,000,000 results in my browser

Code and example in the online IDE:

import requests, lxml
from bs4 import BeautifulSoup

headers = {
    "User-Agent":
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582"
}

params = {
  "q": "step by step guide how to prank google",
  "gl": "us",
  "hl": "en"
}

response = requests.get('https://www.google.com/search', headers=headers, params=params)
soup = BeautifulSoup(response.text, 'lxml')

number_of_results = soup.select_one('#result-stats nobr').previous_sibling
print(number_of_results)

-----
# About 7,800,000 results

---

Alternatively, you can use Google Organic Results API from SerpApi. It's a paid API with a free plan.

The main difference is that you only need to iterate over structured JSON, without figuring out how to extract certain elements and coding everything from scratch. No need to maintain the parser.

import os
from serpapi import GoogleSearch

params = {
    "engine": "google",
    "q": "step by step guide how to prank google",
    "api_key": os.getenv("API_KEY"),
}

search = GoogleSearch(params)
results = search.get_dict()

result = results["search_information"]['total_results']
print(result)

-----
# 7800000

Disclaimer, I work for SerpApi.