[Remote] Lead Azure Data Engineer-AI
Note: The job is a remote job and is open to candidates in USA. Source Select Group is seeking a Lead Azure Data Engineer-AI to lead a team of Data Scientists and Azure Data Engineers. The role involves designing, developing, and delivering machine learning models and enterprise data pipelines on Microsoft Azure and Fabric, while ensuring data quality and governance standards are met.
Responsibilities
- Lead Azure Data Engineer/Architect responsible for designing, building, and optimizing data solutions on Microsoft Azure
- Collaborate closely with stakeholders to gather requirements, translate them into technical designs, and ensure data quality, security, and governance standards are met
- Day-to-day responsibilities include hands-on development, performance tuning, code reviews, and mentoring other data engineers
- Architect and build scalable pipelines using Databricks, Apache Airflow, Fabric Data Factory, Azure Data Engineering, and Microsoft Fabric
- Implement medallion architecture and modern data warehousing
- Ensure scalability, performance, and resilience
- Own full lifecycle: feature engineering → model design → training → validation → deployment → monitoring
- Drive model selection, tuning, and evaluation strategies
- Deliver predictive analytics tied to project and financial data
Skills
- Applicants must be eligible to work direct without any sponsorship required
- U.S Citizen, Green Card
- Prior experience leading Data Science teams and translating business problems into analytical solutions
- Architect and build scalable pipelines using Databricks, Apache Airflow, Fabric Data Factory, Azure Data Engineering, Microsoft Fabric (Lakehouse, OneLake, Semantic Models)
- Implement medallion architecture and modern data warehousing
- Ensure scalability, performance, and resilience
- Own full lifecycle: feature engineering → model design → training → validation → deployment → monitoring
- Drive model selection, tuning, and evaluation strategies
- Deliver predictive analytics tied to project and financial data
- Expert in Python, advanced T-SQL, strong Spark/PySpark
- Deep experience in ML model architecture (required) – supervised, unsupervised, NLP, time-series
- ML deployment in production (required) – end-to-end lifecycle ownership
- Azure Data Factory (ADF) for ML pipelines (required)
- Data engineering – ETL/ELT, data warehousing, medallion architecture
- Advanced experience with Azure ecosystem, Databricks, Microsoft Fabric, Apache Airflow
- 7+ years in Data Engineering, Data Science, or Machine Learning Engineering
- 3+ years in technical leadership roles
- Bachelor's degree required
- Master's preferred
Company Overview