A reference section for my presentation at the 4S Conference on September 4, 2019. Everything on this page should be considered a work in progress.

Instagram Scraper using Python and Beautiful Soup (Should be an API)

Thanks to Gentry Williams for help putting this together and doing the heavy lifting. This is good enough for quick scrapes, but for ideally getting quality data about Instagram posts should use their API. The method below is adequate for a quick snapshot and tends to throw errors at ~5,000 records.

The scrape waits six seconds between each post. This is to prevent undo traffic on Instagram and prevent the IP address from getting blocked. Of course, this makes capturing data incredibly slow.