ETL – EXTRACT, TRANSFORM, LOAD

Here we have a perfect example of an ETL in action:

  • One of the robot fetches daily data from Yahoo Finance about stock prices via Python (hosted at wayscript.com)

  • Store it into SQL Server (hosted at AWS)

  • Presenting it via Power BI (hosted at Office 365)

FETCHING DATA FROM YAHOO FINANCE

VERSION 1

FETCHING DATA WITH BEAUTIFULSOUP AND SAVING INTO SQL SERVER

VERSION 2

Less code

VERSION 3 MUCH SIMPLER

Fetching the last stored date from SQL Server and comparing with last available date from Yahoo

VERSION 4 FETCHING COUNTRY

With a new column “Country” we (me) now have to fetch for the existing country, if not for now it exits

Current Version