Senior Data Engineer – Databricks - 1613
About the position We are seeking a Senior Data Engineer with expertise in Databricks and Snowflake to join our team. This role involves designing, building, and optimizing large-scale data pipelines and data models within a hybrid work environment. The ideal candidate will have a strong background in data engineering principles, advanced SQL, and experience with data ingestion, migration, and orchestration.
Responsibilities
- Design and build large-scale data pipelines for ingestion and transformation
- Develop ETL and ELT frameworks using Databricks and Spark
- Optimize SQL queries and improve data performance
- Build and maintain scalable data models across lakehouse platforms
- Lead data migration efforts across systems and environments
- Implement orchestration for reliable data workflows
- Monitor data pipelines and resolve production issues
- Ensure governance, data quality, and observability across platforms
Requirements
- Databricks and Snowflake for data platforms
- Spark or PySpark with Python for batch processing
- Advanced SQL with query tuning, partitioning, clustering
- Data modeling using star, snowflake, SCD, OBT, normalized models
- Experience with Medallion architecture
- Data ingestion pipelines and large-scale migrations
- Orchestration tools for data workflows
- Data platform debugging and observability
- 7+ years in data engineering or data platform roles
- Strong hands-on experience with Databricks or Snowflake
- Deep expertise in SQL and distributed data processing
- Experience building scalable data models and architectures
- Proven experience with large-scale data migrations
- Bachelor’s degree in Computer Science or related field
Nice-to-haves
- Experience with ML data pipelines and feature engineering
- Exposure to streaming frameworks like Kafka
- Knowledge of cloud platforms like AWS, Azure, or GCP
- Experience with data governance tools and frameworks