Posts

Showing posts from April, 2024

Web scraping beginner guide

Image
 Every web scraping project should start with a goal in mind.  The goal of this beginner guide is to collect faculty details from a college website and save data to a spreadsheet using python. College website :  https://www.ifheindia.org/icfaitech/datascience.html Python code used : # imports for doing http request, reading text/image date and writing text/image data to spreadsheet import requests from bs4 import BeautifulSoup import xlsxwriter from io import BytesIO # HTTP request to staff website response = requests . get ( "https://www.ifheindia.org/icfaitech/datascience.html" ) # Parse HTML response got from above soup = BeautifulSoup ( response . content , "html.parser" ) # Get all div data with class 'profilePic' profile_pic_divs = soup . findAll ( "div" , { "class" : "profilePic" }) # Create spreadsheet 'AI_staff.xlsx' workbook = xlsxwriter .Workbook( 'AI_staff.xlsx' ) # Add w...