Senior ML Engineer - MLOps and AI Evaluation

• Implement model, prompt, retrieval-index, and evaluation-dataset versioning with lineage and reproducible release evidence.
• Build CI/CD evaluation gates for mapping, clinical extraction, recommendations, RAG, and agents. Validate automated judges against SME labels.
• Create synthetic healthcare fixtures and benchmarks for malformed data, corrections, missing evidence, rare cases, and permission boundaries.
• Deploy serving pipelines and monitor drift, model quality, latency, failure rates, and unit cost. Support shadow testing, canaries, rollback, and fallback.
• Partner with Security and Data Platform on tenant isolation tests, PHI-safe telemetry, approved training rights, and model-vendor data handling.
• Optimize inference resources and cost through profiling, batching, caching, and appropriate model sizing. Own reproducible model-training jobs and dependency tracking.
• Partner with Data Platform and product engineers, clinical/data SMEs, DME/QA, and security teams. 
• Build reusable services for ELLKAY products and preserve source evidence, approved data-use rights, and tenant isolation. 
• Model or agent proposals follow established human approval and platform release workflows.

Additional Information:

ELLKAY is committed to fostering a collaborative and high-performance work environment that supports innovation, teamwork, and professional growth. Most roles are designed to operate from our office locations to encourage effective collaboration and engagement across teams.
 Any alternative work arrangements may be considered at the company’s discretion based on role requirements and business needs.
For more information about our company, please visit www.ELLKAY.com.
 ELLKAY is a Smoke-Free Workplace.

ELLKAY, LLC provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.