A Python-based ETL pipeline that extracts real-time weather data from the OpenWeather API for major Indian cities, transforms it, and loads it into a PostgreSQL database for analysis and Power BI visualization.
- Extracts weather metrics (temperature, humidity, wind, etc.)
- Transforms and stores clean data in CSV and PostgreSQL
- Supports visualization with Power BI
- Modular code structure (Extract, Transform, Load)
- Python, Pandas
- PostgreSQL
- OpenWeather API
- Power BI
-
Clone the repo:
https://github.com/coding-for-it/Weather-Analysis-ETL-Pipeline.git cd Weather-Analysis-ETL-Pipeline
-
Install dependencies:
pip install -r requirements.txt
-
Setup environment variables: #add your API key and DB credentials.
-
Run the pipeline:
python run_etl.py