Quant Investment — web scrape Congress Trades with Python
There are a million data vendors out there to provide you with fundamentals, alternatives, even satellite data, but more often than not, they are not cheap. But why pay when almost everything you see on a webpage can be scraped, cleaned and transformed, and used at your fingertips? This article will teach you instead of copy pasting into Excel, we can use very simple Python scripts to get everything you need into a DataFrame format for further joining into your data pipeline for various investment strategies.
I’ll show you how to use Python and its open library to web-scrape data, transform them, clean them and get them ready for your use. I’ll provide ready-to-use codes along the way as well, so as long as you are not a complete novice as having never read codes, you’d be able to easily implement this structure in your daily work.
Why bother web scraping?
The tremendous advantage of web scrapping in general is to bring various pieces of data, fundamental, technical or alternative, into the same place to tell a wholistic story, rather than being limited by each websites data supply format.
In another article I’ll talk about setting up a data pipeline through Python and flow the data into your own free data warehouse, so you can do all kinds of strategies back-testing on…
