[Remote] Databricks Data Engineer

Note: The job is a remote job and is open to candidates in USA. Insight Global is seeking a skilled Entity Resolution Engineer to build a custom Master Data Management solution on top of Databricks using Splink. This senior-level, hands-on engineering role requires strong Python and Databricks engineering fundamentals, along with a solid understanding of MDM concepts, particularly in the healthcare provider domain.

Responsibilities

• Design, build, and maintain a custom Master Data Management solution on Databricks using the Splink open-source Python library
• Develop and configure entity resolution and record linkage pipelines to master provider domain data
• Write clean, production-quality Python code to support MDM pipeline development and ongoing enhancements
• Implement CI/CD workflows using Databricks Asset Bundles to manage code promotion and deployment across environments
• Collaborate with data engineers and governance stakeholders to ensure mastered data assets are properly integrated and exposed
• Perform QA and validation of MDM pipelines to ensure accuracy and reliability of provider data matching and deduplication
• Participate in design discussions and contribute to architectural decisions for the custom MDM platform
• Operate independently and with accountability — proactively communicating blockers, timelines, and delivery status to the team lead

Skills

• 5+ years of hands-on experience with Databricks as a core engineering platform
• 5+ years of Python development experience, with strong proficiency in writing production-grade code
• Demonstrated experience with Master Data Management (MDM) concepts, including entity resolution, record linkage, and deduplication — whether via Splink or comparable approaches
• Experience with Apache Spark for large-scale data processing within a Databricks environment
• Experience implementing CI/CD pipelines, preferably using Databricks Asset Bundles or equivalent code promotion tooling
• Strong accountability mindset — ability to self-manage, deliver independently, and proactively communicate status and blockers without waiting for direction
• Public/Private Sector provider services experience is a huge plus
• Healthcare/Medi-Cal/Medicaid/Medicare Experience

Benefits

• Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching.
• Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.

Company Overview

Company H1B Sponsorship