This site is a work in progress
← Back to projects

Energy Market Data ETL

Python ETL SQL Server Google Sheets

An ETL framework built at LCG Consulting to pull, parse, and store data from all the major U.S. energy market operators. The framework was designed to streamline a process that previously required engineering involvement for every new data source.

Approach

Consultants in the energy consulting company frequently needed new data sources added, but each request required engineers to build ETL pipelines from scratch. To meet users where they are, I designed the framework so consultants could create new interfaces using Google Sheets, which automatically plugged into the pipeline. This let them add new data sources with minimal engineering intervention.

Other Work at LCG

  • Built a web map visualization tool with data points from energy market operators using TypeScript, Google Maps API, and MS SQL Server
  • Created Python scripts for internal task automation and system maintenance
  • Modernized the legacy UPLAN codebase from VB.NET to C#, applying refactoring techniques essential for incremental Agile development