April 12, 2018 - All
Background For my PFCH project, I started out working with API to extract data. When I saw instructions from API provider, I realized there are a lot of limitations to what I could do with APIs. I learnt that many websites have APIs to help developers with data extraction. While APIs are created to be structured approach to data extraction, they are subject to limitations imposed by companies who created them and understandably so, because data is the core of most web services. For instance, with Zillow’s API, I was only able to request specific information for specific property. There is also a limit to number of API calls non-paying users can make in a day. As a result, I was unable use API to extract data I needed for manipulation and meaningful analysis. Because of these API limitations, I decided to try a different approach: web scraping, a less structured approach to getting data off of the Internet. When I learned about web scraping with BeautifulSoup in this class. I was excited at the prospect of being able to extract large amount of data from the internet without going through their providers.