Web Scraping Featured Jobs on jobberman.com using Python

Jobberman is a popular online jobs platform in Nigeria that connects qualified professionals to their dream jobs and employers to the best talent. Over 2 million people each year use jobberman to find jobs. The website homepage has a clean layout where job seekers can search for a job based on different categories. For each category, various jobs are featured.For example the Sales category, has different roles like Sales Manager, Investment Advisor and Sales Representative.

In this project, we will retrieve information from this job category Accounting, Auditing & Finance using a technique called web scraping.

What is Web Scraping?

Let’s assume you want to collect a line of information from a website, what do you do? The first line of action would be to copy and paste this information, but what if you want to collect a large amount of information running into hundreds of pages as quickly as possible, do you think you would be able to achieve this by manually copying and pasting? This is where web scraping comes in! By using simple libraries, the process of web scraping allows you to extract data from a website in an automated fashion using code. Most of this data is unstructured and in an HTML format which is then converted into structured data in a spreadsheet or a database so that it can be used in various applications.

So, when a web scraper needs to scrape a site, first the URL is provided, then it loads all the HTML code for the site. The scraper then obtains the required data from this HTML code and outputs this data in the format specified by the user. Mostly, this is in the form of an Excel spreadsheet or a CSV file, but the data can also be saved in other formats, such as a JSON file.

Here's an outline of the steps we'll follow.

1. Download the webpage using Requests
2. Parse the HTML source code using Beautiful Soup
3. Extract featured jobs from each job category
4. Compile extracted information into Python dictionaries
5. Extract and combine data from multiple pages
6. Save the extracted information to a CSV file

By the end of the project, we will create a CSV in the following format:

Job_Url,job_Title,Company
https://www.jobberman.com/listings/accountant-rvwnq9,Accountant,Jobberman (Third Party Recruitment)
https://www.jobberman.com/listings/investment-principal-5xn457,Investment Principal,Jobberman (Third Party Recruitment)
...

How to run the code

You can execute the code by selecting the "Run" button at the top of this page. You can make changes and save your own version to Jovian by executing the following cells.