Youtube Webscraping Comments By New

I'm trying to web scrape a youtube comment section. But before that, I want to sort the comments by new. So I have to click on the SORT BY and then Newest First. But unfortunately,

Solution 1:

I would highly recommend to replace the autogui part with firstly element = driver.find_element_by_xpath or by css_selector and element.click And after you choose the xpath/css_selector from the "Newest" option and click again, i did that in another script and had no problems, i cant format it right since im on android tho

tl;dr: replace the autogui by clicking with selenium, firstly the button that makes the options chooseable and then again on the option you want to sort it for

Edit: okay i tried it out myself, the pyautogui did not work for me, not sure if it is because of me using another driver or something, but if it works for you thats fine. The problem is, that you need to scroll down a bit first, in order to load the comments. you can do that using

for i in range(5):
driver.execute_script("arguments[0].scrollBy(0, 500)", element)
time.sleep(2)

not sure if 5 is enough/too much, since i cant check myself, but thats basically how it would work. you can then locate the dropdown open button using

    element = driver.find_element_by_xpath("//*[@id="label"]")
    element.click
    #now we select the option from the dropdown options
    element = driver.find_element_by_xpath("/html/body/ytd-app/div/ytd-page-manager/ytd-watch-flexy/div[4]/div[1]/div/ytd-comments/ytd-item-section-renderer/div[1]/ytd-comments-header-renderer/div[1]/span/yt-sort-filter-sub-menu-renderer/yt-dropdown-menu/paper-menu-button/iron-dropdown/div/div/paper-listbox/a[2]/paper-item")  
    element.click

you could also use explicitWaits to ensure it is in view, but i dont feel confortable using that yet, you find them here
try it out and let me know if it worked :D