Projects
Here are some of the key projects I've worked on. Each project demonstrates my expertise in different areas of data engineering and DevOps.
Text-to-SQL chatbot
Developed a chatbot that converts text data requests to SQL and return the SQL Query and data extracted from the query. It can also visualise the data
Technologies Used:
Databricks asset bundle: fast devlopment to production
Created a comprehensive Data and CI/CD pipeline using Databricks Asset Bundle & dbt. Automated build, test, and deployment processes for multiple data pipelines.
Technologies Used:
AWS infrastructure deployment
Terraform configuration fto provision essential AWS data pipeline resources.
Technologies Used:
Energy data pipeline in dagster
Build a fully automated streaming data pipeline that fetch energy consumption data from API and store data in database and visualise the data.
Technologies Used:
Open Source Contributions
I actively contribute to open-source projects and maintain several of my own. Check out my GitHub profile for more details.