[Remote] AV Simulation Domain Expert (Sr. Principal) - US (Remote) or Chicago, IL

Note: The job is a remote job and is open to candidates in USA. HERE Technologies sits at a unique intersection of detailed mapping and generative AI capabilities. They are seeking an AV Simulation Domain Expert to bridge deep learning and AV simulation, focusing on developing map-grounded world foundation models and synthetic scenario generation.

Responsibilities

• Drive the technical direction for map-grounded world foundation models: how we condition generative video and world models using map data, drive data, and scenario semantics
• Train, fine-tune, and adapt generative models (diffusion, latent video, transformer-based world models) for driving scenario generation, including domain adaptation, controllability, and conditioning on structured inputs (maps, trajectories, agent behaviours, weather, lighting)
• Evaluate and extend state-of-the-art foundation models such as NVIDIA Cosmos / Cosmos-Transfer and comparable open-source world models, assessing fit for AV training data generation
• Own the full ML lifecycle end-to-end: data curation, model training, evaluation, iteration, and the path to production-grade pipelines
• Lead proof-of-concept initiatives demonstrating map-grounded synthetic scenario generation with key technology partners
• Define measurable success criteria that go beyond visual realism — focusing on ML training data utility, controllability, and sim-to-real transfer
• Deliver POC outcomes with clear GO / PIVOT / NO-GO recommendations backed by quantitative evidence
• Bridge generative world models with classical simulation stacks (CARLA, NVIDIA Drive Sim, AlpaSim) where structured, physics-grounded scenarios are needed
• Author and programmatically generate OpenSCENARIO / OpenDRIVE definitions that feed both classical simulators and generative pipelines
• Drive sim-to-real strategy: measure domain gap, identify failure modes, and define acceptable thresholds for downstream model training
• Define what 'good enough' synthetic data means for AV perception and planning: when is photorealism required, when is label consistency sufficient, when does controllability matter most?
• Establish validation frameworks combining objective metrics (distribution coverage, label accuracy, FID-style measures, downstream task performance) with expert evaluation protocols
• Specify sensor fidelity requirements: noise models, lens distortion, lidar return characteristics — and how generative models should or should not reproduce them
• Interface with ML research teams on generative model architecture, controllability, and conditioning strategies
• Collaborate with perception and planning teams to ensure synthetic data measurably improves real-world model performance
• Translate business requirements into technical feasibility assessments for product and executive stakeholders

Skills

• Proven experience training deep learning models end-to-end, with clear ownership across data, training, evaluation, and iteration
• Expertise in generative video, world models, or related generative AI research/engineering
• Deep working knowledge of diffusion models, latent video models, and/or transformer-based world models
• Experience with high-dimensional temporal or spatio-temporal data (video, multi-sensor fusion, driving data)
• Strong Python and PyTorch engineering fundamentals; comfortable building research-grade tooling that can scale toward production
• Demonstrated ability to take ML models from research into production, navigating real-world constraints, quality, and safety requirements
• 5+ years combined experience spanning AV simulation, perception/ML for AVs, or robotics simulation — with meaningful exposure to both simulation platforms and ML model development
• Hands-on experience with at least one major simulation platform: CARLA, NVIDIA Drive Sim, or equivalent
• Fluency with OpenDRIVE and OpenSCENARIO: can author and generate scenario definitions programmatically and understands map format specifications
• Understanding of AV testing workflows: scenario-based validation, ASAM OpenX standards, and awareness of frameworks such as ISO 34502
• Understanding of what scenarios stress-test AV perception and planning systems, and why
• Ability to evaluate synthetic data for ML training utility: distribution diversity, label consistency, edge-case coverage, downstream task performance
• Experience with synthetic-to-real transfer, domain adaptation, or closing the sim-to-real gap in a measurable way
• Clear point of view on trade-offs between photorealism, label accuracy, controllability, and computational efficiency
• Hands-on experience with NVIDIA Cosmos, Cosmos-Transfer, or comparable world foundation models
• Reinforcement learning experience, particularly where it measurably improved real-world performance
• Experience with end-to-end driving models
• Automotive, OEM, or other safety-critical ML deployment experience (ISO 26262, SOTIF awareness)
• Strong publication record in generative models, world models, or AV ML; or significant contributions to open-source ML tooling
• Game engine experience (Unreal, Unity) for rendering and sensor simulation pipelines
• Experience with PyTorch Lightning or similar large-scale training infrastructure

Company Overview

Company H1B Sponsorship