How To Construct Data Frame From Web Scraping In Python

I can fetch data from web page thru web scraping in Python. My data is fetched into a list. But don't know how to transform that list into a data frame. Is there any way I could we

Solution 1:

import requests
import pandas as pd

r = requests.get("https://www.worldometers.info/coronavirus/")
df = pd.read_html(r.content)[0]

print(type(df))

# <class 'pandas.core.frame.DataFrame'>

df.to_csv("data.csv", index=False)

Output: view

Solution 2:

Well read_html returns a list of DataFrames (as per documentation), so you have to get the "first" (and only) element of that list.

I would just add at the end (after you call read_html):

df = df[0]

Then you can inspect its info getting:

df.info()

# <class 'pandas.core.frame.DataFrame'># RangeIndex: 207 entries, 0 to 206# Data columns (total 10 columns):# Country,Other       207 non-null object# TotalCases          207 non-null int64# NewCases            59 non-null object# TotalDeaths         144 non-null float64# NewDeaths           31 non-null float64# TotalRecovered      154 non-null float64# ActiveCases         207 non-null int64# Serious,Critical    112 non-null float64# Tot Cases/1M pop    205 non-null float64# Deaths/1M pop       142 non-null float64# dtypes: float64(6), int64(2), object(2)# memory usage: 16.3+ KB