Learn practical skills, build real-world projects, and advance your career

WEB SCRAPING PROJECT

Pick a website and describe your objective

We shall be doing the following tasks:

  • We're going to scrape https://github.com/topics
  • We'll get a list of topics and for each topic, we'll collect topic title, topic page URL and topic description
  • For each topic, we'll get top 25 repositories from the topic page.
  • For each repo, we'll grab repo name, username, stars, repo_url.
  • For each topic, we'll create a csv file in the following format:
Repo_Name, Username, Stars, Repo_Url

Use the requests library to download web pages

!pip install requests --upgrade --quiet