Hey guys i have visited web scraping topic taught by aakash sir few days back. But i have a doubt regarding the topic. I tried to scrape the link https://www.youtube.com/c/unboxtherapy/videos . But when i tried to download it gave me a certain html file. But i couldnt find the tags that are there in the developers tool in the html file that was downloaded. I have attached pics for reference!
I would make sure that what you’ve downloaded is really what you want to scrape.
Try visiting the webpage in private/incognito mode. You might encounter something like this:
This is not what you want to scrape sadly.
To avoid such thing you would have to use selenium that would visit the page for you, agree for spyware and then actually process the page.
hey hi @Sebgolos, what do you mean by visiting the webpage in incog mode. The downloaded web page runs only on on local host right. I am new to web scraping so this might be a silly question.
Private/incognito mode is a special mode of browser. When using it the browser won’t save any history and also doesn’t have anything from the history (so you won’t be logged into any site etc).
So when using it you have to agree again for any cookies/consents/whatever is required now.
Doesn’t matter that you run the page locally - the page is downloaded as if you would use this mode. So the page has no info that you actually read such thing and agreed - you have to do it again.