[Remote] Applied Machine Learning & Data Engineering Senior Consultant II
Note: The job is a remote job and is open to candidates in USA. National General is a part of The Allstate Corporation, which means we have the same innovative drive that keeps us a step ahead of our customers’ evolving needs. The role supports high-impact AI and data automation initiatives focused on improving data ingestion, transformation, and mapping processes across the organization, responsible for advancing AI-assisted data mapping capabilities to enhance accuracy and deliver scalable solutions.
Responsibilities
- Design and build AI-powered data pipelines to process and transform structured and unstructured data (PDF, Excel, CSV, etc.)
- Develop and optimize LLM-based solutions (prompt engineering, mapping, extraction, and transformation)
- Translate complex data and technical outputs into actionable insights for business stakeholders
- Partner with cross-functional teams (product, engineering, business SMEs) to define and deliver solutions
- Validate and improve solution accuracy through testing, iteration, and stakeholder feedback
- Support integration of ML/AI solutions into enterprise systems and workflows (APIs, databases, cloud platforms)
- Document solutions to ensure scalability, maintainability, and long-term adoption
- Analyze and process carrier and CMS datasets
- Build AI/ML mapping logic using LLMs and prompts
- Develop data pipelines (Python/C# data processing logic, SQL, APIs, Azure)
- Validate outputs against expected results with business stakeholders
- Troubleshoot accuracy, performance, and edge cases
- Document pipelines and ensure production-readiness
- Support integration into enterprise systems (C#, APIs, Azure)
Skills
- 3+ years of related experience
- Strong proficiency in Python and SQL for data analysis and engineering
- Experience working with Generative AI / LLMs (prompt engineering, APIs, or similar)
- Hands-on experience with data transformation, file parsing, and pipeline development
- Knowledge of DevOps practices (CI/CD pipelines, version control, cloud deployment)
- Familiarity with cloud platforms (preferably Azure)
- Ability to communicate effectively with both technical and non-technical stakeholders
- Proven ability to work in ambiguous, fast-evolving environments
- Experience with C# / .NET integration or enterprise application development
- Familiarity with LLM frameworks (LangChain, etc.) or document processing tools
- Knowledge of API design, JSON processing, and system integration patterns
- Background in insurance, healthcare, or other regulated industries
Benefits
- Allstate provides a comprehensive technology setup, including a laptop, monitors, headset, keyboard, and mouse.
- Employees eligible to work from home also receive a monthly connectivity reimbursement to help offset internet costs.
Company Overview