Get Text Content From P Tag

I am trying to get description text content of each block on this page https://twitter.com/search?q=data%20mining&src=typd&vertical=default&f=users. html for p tag lo

Solution 1:

The issue might be that some div with class as ProfileCard-content may not have a child p element with class - ProfileCard-bio u-dir , when that happens , the following returns None -

div.find('p', attrs={'class' : ['ProfileCard-bio', 'u-dir']})

And that is the reason you are getting the AttributeError. You should get the return of above and save it in a variable , and check whether its None or not and take the text only if its not None.

Also, you should give class as a list of all the classes , not a single string, as -

attrs={'class' : ['ProfileCard-bio', 'u-dir']}

Example -

productDivs = soup.findAll('div', attrs={'class' : 'ProfileCard-content'})
for div in productDivs:
   elem = div.find('p', attrs={'class' : ['ProfileCard-bio', 'u-dir']})
   if elem:
       print elem.text