Middle/Senior Data Engineer
🧠 Tech Level: Middle 🗣️ Language Proficiency: Upper-Intermediate 🧾 Employment type: Full time 🌍 Candidate Location: Poland, Kazakhstan, EU, Georgia, Armenia, Serbia. 🕐 Working Time Zone: CET 🚀 Start: ASAP 🧭 Planned Work Duration: by the end of the year 👥 Customer Description: The customer is a global mobility platform providing on-demand transportation services. ⚙️ Project Phase:ongoing 🤝 Soft Skills:
- Strong analytical and problem-solving skills with the ability to make decisions based on technical evidence and logical reasoning.
- Excellent communication, active listening, collaboration, and consensus-building skills.
💡 Hard Skills / Must Have:
- 3+ years of experience in Data Engineering
- Strong proficiency in Python
- Experience with Spark/PySpark for streaming, batch, and asynchronous data processing
- Data modeling
- Distributed data processing
- SQL
- Data warehouse/lakehouse architecture
- Hands-on experience with Databricks, GCP, or AWS
- Terraform
- Designing data access and governance solutions
- Understanding of data products and governance controls
✨ Hard Skills / Nice to Have:
- Willingness to learn new technologies for integration purposes
📌 Responsibilities and Tasks:
- Develop, deploy, and maintain data pipelines and data governance systems on Databricks (Unity Catalog, Delta Lake, Workflows)
- Conduct in-depth data analysis, profiling, and processing to classify data by domain and detect personal data (PII)
- Contribute to the design of data access systems, focusing on data contracts, catalog access, and grant/rollback automation managed as Infrastructure-as-Code with Terraform
- Review access architectures and contracts to ensure flexibility and reusability
- Ensure that data products and access controls are deployed according to contracts and operate without unintended side effects
- Maintain documentation for data contracts, access models, and governance processes
- Monitor and support deployed pipelines and quality checks to ensure they meet data quality and performance expectations
- Actively seek out information and internal solutions to ensure reusability and adoption of common data platform technologies
🧪 Technology Stack:Python for streaming, batch, and asynchronous data, MLOps tools 📝 Additional notes: The work will be carried out either on client's equipment or via a virtual workstation. 📩 Ready to Join? We look forward to receiving your application and welcoming you to our team!