About Me
I am a Senior Data Engineer with over 7 years of professional experience in designing and optimizing large-scale data pipelines, data warehouses and data lakehouse. Proven expertise in Apache Spark, Python, SQL and distributed computing, optimizing data workflows. Hands-on experience with DBT, Git, and CI/CD pipelines, ensuring high-quality, scalable solutions in fast-paced environments. Passionate for solving complex challenges with a strong analytical and mathematical mindset to consistently deliver high-quality projects.
Professional Experience
Senior Data Engineer
Department of Health and Social Care • 2022 - Present
- Utilizing Databricks to architect and optimize workflows, achieving significant improvements in both cost efficiency and performance across analytical pipelines and improved data delivery by 80%.
- Implemented CI/CD pipelines and automated deployment processes using Databricks Asset Bundles (DABs).
- Leading the implementation of DBT (Data building tool) to streamline SQL data processing, transformation and improving data quality. Significantly improved stakeholder engagement by 60%.
- Tech stack: Azure, Databricks, PySpark , DBT ,CI/CD, Git, Python, R, SQL, Power BI , SQL Server
Senior Data Engineer
Quantiphi Inc. • 2020 - 2021
- Designed and implemented automated data pipelines to seamlessly process 100 GB data daily streaming from multiple sources (IoT and API data) into a centralized Big Query data warehouse
- Collaborated closely with stakeholders in the renewable energy industry to gather requirements, ensuring a clear and unambiguous definition of Key Performance Indicators (KPIs) and logical frameworks.
- Tech stack: Google Cloud Platform, Looker, SQL, Python, Airflow, Pub Sub, Big Query, Git, CI CD
Education
Master of Business Administration
California State University - San Bernardino • 2016 - 2018
Bachelor of Engineering
Gujarat Technological University • 2010 - 2014
Certifications
- Microsoft Certified Azure Data Engineer
- Certificate of recognition From CEO of GCP
Interests
Outside of work, I enjoy trying new technology, building projects, contributing to open-source projects, writing technical blogs, and staying up-to-date with the latest developments in technology. I'm particularly interested in Data & AI, and emerging technologies.