Selenium Python | 'find_elements_by_class_name' Returns Nothing

I am trying to scrape job titles from a dynamic job listing. When I use the function find_elements_by_class_name, the function doesn't return anything. I'm new to selenium so i'm n

Solution 1:

Root Cause:

col-md-8 and jobtitle are 2 different classes. When you use find_element_by_class_name it will internally convert the class name to a css selector and try to find the element.

Below is the evidence where find_element_by_class_name uses the css internally.

Answer :

As Selenium internally uses the css you have to make sure the classes are clubbed together meaning class1.class2.class3. In simple terms replace all white spaces with single dot in the class name from UI.

How to implement this to your situation:

You have to use the below syntax.

driver.find_element_by_class_name('col-md-8.jobtitle')

Solution 2:

Try:

jobs = driver.find_elements_by_xpath("//div[@class='col-md-8 jobtitle']/a")

I've switched the find element by class for xpath, this way you have more flexibility and it generally works better, I suggest you look into it!

Solution 3:

Seems like a bug? This works:

jobs = driver.execute_script("""
  return document.getElementsByClassName("col-md-8 jobtitle")
""")