AI Security Architect- ARC, Apple Information Security
Worldwide
Apple remote roles
Server-rendered summary of Apple's open remote positions.
AIML - Senior ML Engineer, Responsible AI and Safety
Worldwide
AIML - Senior ML Engineer, Responsible AI and Safety
Worldwide
Senior Machine Learning iOS Platform Engineer — Responsible AI and Safety
Worldwide
AIML - ML Researcher, Responsible AI
Worldwide
AIML - ML Engineer, Responsible AI
Worldwide
AI Security Engineer - Red Team
Worldwide
Staff Machine Learning Platform Engineer, AI Evaluation
Worldwide
Senior Applied Scientist - AI Evaluation & Quality Systems
Worldwide
Senior Manager - Machine Learning Applied Research and ML , Responsible AI and Safety
Worldwide
AIML - ML Researcher in Foundation Models, Responsible AI
Worldwide
AI Safety - Machine Learning and Applied Research
Worldwide
Staff Applied Research and ML , Responsible AI and Safety
Worldwide
AIML - Senior Machine Learning Engineer, Responsible AI and Safety
Worldwide
Research Scientist, AI Evaluation Science
Worldwide
Apple
🔍
Remote-Friendliness Score Coming Soon
We're analyzing Apple's remote work culture to provide detailed insights.
Open Positions at Apple
15
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
Job Description
We are seeking an AI Security Architect with deep expertise across security architecture, AI/ML systems, and threat modeling to join the Apple Information Security (AIS) Assurance ARC team. You will own security guidance for the most critical, high-risk AI initiatives across Apple, shaping architecture patterns, conducting design reviews, selecting or building security solutions for AI-driven environments, and translating expert decisions into AI-powered guidance that scales across the entire organization. Your work will directly determine how safely and confidently Apple adopts AI at scale. Our scope spans across Apple's products, services, and operations, where the adoption of AI is accelerating at an unprecedented pace. The Assurance ARC organization is entrusted with providing expert security architecture guidance for these AI systems, and your contributions will directly impact our ability to protect against evolving AI-specific threats while enabling the business to innovate securely at scale. DESCRIPTION In this high-impact role, you will bridge the gap between traditional security practices and emerging AI risks. Operating under Zero Trust principles and Secure by Default framework, you will partner with engineering, data science, product, and compliance teams to build "paved roads" — pre-approved, reusable, secure-by-default patterns that enable teams to build AI-driven systems without a bespoke review for every project. you will amplify the effectiveness of the entire security function, by reducing inbound request volume through proactive foundational work and providing developers, ML engineers, and traditional security architects with clear guidance and standardized controls, Leveraging deep AI/ML knowledge, you will evaluate risks such as data poisoning, model manipulation, and prompt injection, while simultaneously applying AI to security workflows to automate threat modeling, design reviews, and control validation. The third dimension of this role beyond prevention and response, is scale: applying AI-driven tooling to extend the team's reach far beyond what headcount alone can achieve. This is a highly strategic and technical role that bridges AI innovation with security, enabling Apple to adopt AI confidently and at pace. As part of the Advisory Arm, you will also contribute directly to the Self-Service Enablement Platform, codifying decisions, patterns, and trade-offs from your reviews into AI-powered guidance accessible to teams without direct advisory engagement. Additionally, by working on Apple's most cutting-edge AI projects, you will serve as an intelligence sensor for the broader ARC team: identifying emerging AI/ML technologies and architectural patterns that should become future foundational focus areas, ensuring the organization stays ahead of the threat landscape. MINIMUM QUALIFICATIONS BS in Computer Science, Computer Engineering, or Information Security, or 6+ years of equivalent, hands-on security experience in large enterprise environments a plus. Strong background in security architecture, application security, or cloud security Demonstrated understanding of AI/ML concepts, pipelines, and risks (e.g., data poisoning, prompt injection, model inversion, model theft) Experience with threat modeling methodologies (e.g., STRIDE) and secure design principles Familiarity with AI regulatory frameworks such as the EU AI Act, NIST AI RMF, ISO/IEC 42001, or comparable standards Ability to communicate security concepts clearly to both technical and non-technical stakeholders Proven ability to partner with engineering, data science, and product teams to drive adoption of security requirements Experience applying AI/LLMs for security automation (e.g., risk analysis, automated threat modeling, compliance validation) Proficiency in at least one scripting or programming language (e.g., Python, Swift, Java) PREFERRED QUALIFICATIONS Experience performing full-stack security architecture reviews encompassing cloud-native and emerging AI/ML technologies Experience designing security patterns for LLM-based systems, RAG pipelines, or agentic architectures Familiarity with Zero Trust architecture principles and Secure by Default design Experience with data protection, encryption architecture, and key management in AI/ML contexts Strong verbal and written communication skills, with the ability to build consensus across diverse teams A real passion for staying ahead of emerging AI security risks and translating that knowledge into actionable guidance Deeply accountable for your work; upbeat, adaptable, and results-oriented
Requirements
- BS in Computer Science or related field
- 6+ years of equivalent hands-on security experience
- Deep expertise in AI/ML systems and threat modeling
- Ability to partner with engineering and data science teams
- Strong communication and collaboration skills
- Experience with cloud security and compliance
- Familiarity with Zero Trust principles and Secure by Default framework
Benefits
- 401k Matching
- Certification Support
- Flexible Hours
- Health Insurance
- Home Office Budget
- Learning Budget
- Paid Time Off
- Remote Work
Skills
AI/ML Threat modeling Python Java Cloud Security Security ArchitectureFull Time
$120,000 - $180,000*
3 days ago
Worldwide
Job Description
Join Us in Shaping the Future of Generative AI at Apple! Are you passionate about making AI systems safer, more inclusive, and globally representative? Apple is seeking an expert Machine Learning Engineer to shape the future of responsible AI for the next generation of generative features. In this role, you will lead the responsible AI lifecycle end-to-end: assessing risks, defining policies, developing mitigation strategies, and driving continuous improvements. Your work will directly influence how we evaluate, align, and monitor the safety of large language and multimodal models. As part of Apple’s Responsible AI group within the Human-Centered Machine Intelligence (HCMI) organization, you’ll collaborate with cross-functional partners to minimize unintended consequences across people, systems, and society while elevating feature capabilities and the overall user experience. Together, we’ll anticipate challenges, measure real-world impact, and deliver trusted, high‑quality AI experiences to users around the globe. You’ll also contribute to forward‑looking research in fairness, robustness, uncertainty, and safety — pushing the boundaries of responsible AI at scale. DESCRIPTION Our team leads Responsible AI initiatives for global generative AI products, operating at the intersection of policy, product, and GenAI. We're seeking candidates who will shape safety policies in partnership with leadership, design, engineering, legal, and regulatory stakeholders—ensuring our safeguards advance both user protection and product innovation. These individuals will work on architecture mitigation and safety alignment strategies for generative models, drive integration in production. Additionally, they will work on developing models, tools, datasets, and evaluation methods to monitor, diagnose failures, and improve the safety of generative models throughout the deployment lifecycle. We do all these by incorporating human and automated feedback, post‑launch to continuously improve feature safety and user trust. MINIMUM QUALIFICATIONS 3+ years of proven ability in machine learning, including work with generative models (Transformers, LLMs, VLMs), NLP, or Computer Vision Proficiency in Python and data science libraries (e.g. Pandas) with strong skills in data analysis, visualization, and applied ML workflows Excellent interpersonal skills and proven ability to translate sophisticated technical insights for cross‑functional partners, senior leadership, and executives Strong analytical and independent problem-solving skills, with ability to navigate ambiguity Experience designing and supporting human and automated evaluations, particularly with complex, nuanced, or multi‑labeled data Hands‑on experience collecting and analyzing language, vision, or multimodal datasets Background in failure analysis, quality engineering, or robustness testing for ML‑driven systems Must be comfortable working with sensitive or potentially offensive content PREFERRED QUALIFICATIONS BS, MS, or PhD in Computer Science, Machine Learning, or related field, or equivalent experience Proven success contributing in a highly cross‑functional environment Experience shipping complex AI systems at global scale Background in model explainability, uncertainty estimation, or interpretability Curiosity and research interest in fairness, bias, and the societal impacts of generative AI Passion for building innovative, high‑impact products that draw upon interdisciplinary skills
Requirements
- 3+ years of proven ability in machine learning, including work with generative models
- Proficiency in Python and data science libraries (e.g. Pandas)
- Strong skills in data analysis, visualization, and applied ML workflows
- Excellent interpersonal skills and proven ability to translate sophisticated technical insights
- Strong analytical and independent problem-solving skills, with ability to navigate ambiguity
Benefits
- 401k Matching
- Certification Support
- Flexible Hours
- Gym Membership
- Health Insurance
- Home Office Budget
- Learning Budget
- Paid Time Off
Skills
Python Pandas Transformers LLMs VLMs NLP Computer VisionFull Time
$120,000 - $180,000*
3 days ago
Worldwide
Job Description
Join Us in Shaping the Future of Generative AI at Apple! Are you passionate about making AI systems safer, more inclusive, and globally representative? Apple is seeking an expert Machine Learning Engineer to shape the future of responsible AI for the next generation of generative features. In this role, you will lead the responsible AI lifecycle end-to-end: assessing risks, defining policies, developing mitigation strategies, and driving continuous improvements. Your work will directly influence how we evaluate, align, and monitor the safety of large language and multimodal models. As part of Apple’s Responsible AI group within the Human-Centered Machine Intelligence (HCMI) organization, you’ll collaborate with cross-functional partners to minimize unintended consequences across people, systems, and society while elevating feature capabilities and the overall user experience. Together, we’ll anticipate challenges, measure real-world impact, and deliver trusted, high‑quality AI experiences to users around the globe. You’ll also contribute to forward‑looking research in fairness, robustness, uncertainty, and safety — pushing the boundaries of responsible AI at scale. DESCRIPTION Our team leads Responsible AI initiatives for global generative AI products, operating at the intersection of policy, product, and GenAI. We're seeking candidates who will shape safety policies in partnership with leadership, design, engineering, legal, and regulatory stakeholders—ensuring our safeguards advance both user protection and product innovation. These individuals will work on architecture mitigation and safety alignment strategies for generative models, drive integration in production. Additionally, they will work on developing models, tools, datasets, and evaluation methods to monitor, diagnose failures, and improve the safety of generative models throughout the deployment lifecycle. We do all these by incorporating human and automated feedback, post‑launch to continuously improve feature safety and user trust. MINIMUM QUALIFICATIONS 3+ years of proven ability in machine learning, including work with generative models (Transformers, LLMs, VLMs), NLP, or Computer Vision Proficiency in Python and data science libraries (e.g. Pandas) with strong skills in data analysis, visualization, and applied ML workflows Excellent interpersonal skills and proven ability to translate sophisticated technical insights for cross‑functional partners, senior leadership, and executives Strong analytical and independent problem-solving skills, with ability to navigate ambiguity Experience designing and supporting human and automated evaluations, particularly with complex, nuanced, or multi‑labeled data Hands‑on experience collecting and analyzing language, vision, or multimodal datasets Background in failure analysis, quality engineering, or robustness testing for ML‑driven systems Must be comfortable working with sensitive or potentially offensive content PREFERRED QUALIFICATIONS BS, MS, or PhD in Computer Science, Machine Learning, or related field, or equivalent experience Proven success contributing in a highly cross‑functional environment Experience shipping complex AI systems at global scale Background in model explainability, uncertainty estimation, or interpretability Curiosity and research interest in fairness, bias, and the societal impacts of generative AI Passion for building innovative, high‑impact products that draw upon interdisciplinary skills
Requirements
- 3+ years of proven ability in machine learning, including work with generative models
- Proficiency in Python and data science libraries (e.g. Pandas)
- Strong skills in data analysis, visualization, and applied ML workflows
- Excellent interpersonal skills and proven ability to translate sophisticated technical insights
- Strong analytical and independent problem-solving skills, with ability to navigate ambiguity
Benefits
- 401k Matching
- Certification Support
- Flexible Hours
- Gym Membership
- Health Insurance
- Home Office Budget
- Learning Budget
- Paid Time Off
Skills
Python Pandas Transformers LLMs VLMs NLP Computer VisionFull Time
$120,000 - $200,000*
4 days ago
Worldwide
Job Description
Join Us in Shaping the Future of Generative AI at Apple! Are you passionate about making AI systems safer, more inclusive, and globally representative? Apple is seeking an expert Client Engineer to own the integration of our Responsible AI mitigation assets across the full deployment surface, from on-device foundation models running on Apple Silicon to server-side inference on Private Cloud Compute (PCC). In this role, you will be a pivotal technical leader, bridging Swift client engineering and ML deployment, driving the architectural vision, design, and implementation of how safety classifiers, guardrail models, and mitigation policies are shipped, invoked, and streamed alongside our generative features. You will take end-to-end ownership, from initial concept and rapid prototyping to delivering robust, high-performance, and maintainable solutions that minimize unintended consequences across people, systems, and society while elevating feature capabilities and the overall user experience. Together, we’ll anticipate challenges, measure real-world impact, and deliver trusted, high‑quality AI experiences to users around the globe. DESCRIPTION Our team leads Responsible AI initiatives for global generative AI products, operating at the intersection of policy, product, and GenAI. We build the safety classifiers, content filters, and policy enforcement layers that protect users from unintended model behavior. This role is about getting those assets into users' hands reliably, on the device or in the cloud, at the latency and quality bar Apple expects. We are seeking candidates who will work closely with multiple stakeholders, ranging from design, engineering, legal and regulatory to ensure our safeguards advance both user protection and product innovation. You will work on defining mitigation architectures, owning the implementation and overseeing the integration in production. Additionally, you will contribute to modeling, tooling and frameworks, as well as dataset, and evaluation methods to monitor, diagnose failures, and improve the safety of generative models throughout the deployment lifecycle. MINIMUM QUALIFICATIONS 12+ years of professional experience, with at least 5+ years in iOS / macOS application development in both Objective C and Swift Expertise in Apple's Core iOS and Foundation frameworks BS in Computer Science, Mathematics, Statistics, or a related field, or equivalent industry experience Experience in shipping impactful mobile frameworks used by others outside your direct team Experience leading the architecture and development of complex, high-performance production systems Demonstrated ability to technically lead projects, mentor engineers, and drive cross-functional initiatives from concept to delivery Excellent analytical, problem solving and communication skills PREFERRED QUALIFICATIONS Working knowledge of on-device ML runtimes (Core ML, MLX, or equivalent) and the model-export lifecycle: converting trained models into shippable assets, and loading them efficiently at runtime Working knowledge of frontier/LLM models including token-streaming inference, tokenization, and buffering strategies Experience building applications that utilize modern ML/AI technology
Requirements
- 12+ years of professional experience
- 5+ years in iOS / macOS application development
- Expertise in Apple's Core iOS and Foundation frameworks
- BS in Computer Science, Mathematics, Statistics, or a related field
- Experience in shipping impactful mobile frameworks used by others outside your direct team
- Experience leading the architecture and development of complex, high-performance production systems
- Demonstrated ability to technically lead projects, mentor engineers, and drive cross-functional initiatives from concept to delivery
- Excellent analytical, problem solving and communication skills
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Coworking Space
- Dental Insurance
- Employee Assistance Program
- Employee Discounts
Skills
Generative AI Swift Objective C Core ML MLX Apple Silicon Private Cloud Compute (PCC)Full Time
$120,000 - $180,000*
2 weeks ago
Worldwide
Job Description
Join us as we build world-class groundbreaking products for our customers! Apple's Data and ML Innovation team focuses on innovative technologies, methodologies, and research to enable fantastic user experiences and advance the frontier of machine learning. Our team is looking to hire a tech lead with a strong track record in Applied Research who is passionate about ML and Human-Computer Interaction - mainly as applied to the responsibility, fairness, and safety of Generative AI. In this role, you will lead research methods focused on enabling ML technologies that power breakthrough user experiences while upholding Apple's values, privacy, and quality standards. DESCRIPTION This role will be highly multifunctional. You will collaborate closely with top machine learning researchers and engineers, software engineers, and design teams to develop and deliver groundbreaking solutions for Apple products. We believe that the most exciting problems in machine learning research arise at the intersection of emerging technologies and real-world use cases. This is also where the most critical breakthroughs come from. As a researcher, you will: - Define and deliver responsible machine learning technologies - Research and advance red teaming methods for generative AI models - Research and develop mitigations and safeguards to ensure safe deployment of LLM's in Apple products - Develop methods and frameworks to evaluate our products - Advocate for scientific and engineering excellence: You will contribute to the architecture and high-level structure of Apple's AI-powered platform and features - Work multi-functionally on a diverse set of challenging problems and collaborate with extraordinary software engineers, machine learning engineers, and researchers to impact the future of Apple products - Work closely with product and partner teams to drive requirement definition, technical quality of deliverables, and execution. MINIMUM QUALIFICATIONS Strong research or product deployment record in areas related to responsible AI, with publications in top ML and HCI venues (e.g., ACL, CHI, CVPR, EMNLP, FAccT, ICML, Interspeech, NeurIPS, UIST, etc.) Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models Deep experience in human-centered research - you understand design fundamentals in Human-Centered AI, Responsible AI, Human-Computer Interaction, and related fields Work with highly-sensitive content with exposure to offensive and controversial content PREFERRED QUALIFICATIONS BS, MS or PhD in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues Strong organizational and operational skills working with large, multi-functional, and diverse teams
Requirements
- Strong research or product deployment record in areas related to responsible AI
- Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models
- Deep experience in human-centered research
- Work with highly-sensitive content with exposure to offensive and controversial content
- BS, MS or PhD in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
- Strong organizational and operational skills working with large, multi-functional, and diverse teams
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Coworking Space
- Dental Insurance
- Employee Assistance Program
- Employee Discounts
Skills
Machine Learning Generative AI Responsible AI Python Research Human-Computer InteractionFull Time
$120,000 - $180,000*
2 weeks ago
Worldwide
Job Description
Would you like to play a part in building the next generation of generative AI applications at Apple? We're looking for scientists and engineers to work on ambitious projects that will impact the future of Apple, our products, and the broader world. In this role, you'll have the opportunity to tackle innovative problems in machine learning, particularly focused on generative AI. As a member of the Apple HCMI group, you will be working on Apple's generative models that will power a wide array of new features. Our team is currently working on large generative models for vision and language, with particular interest on safety, robustness, and uncertainty in models. DESCRIPTION - Develop models, tools, metrics, and datasets for assessing and evaluating the safety of generative models over the model deployment lifecycle - Develop methods, models, and tools to interpret and explain failures in language and diffusion models - Build and maintain human annotation and red teaming pipelines to assess quality and risk of various Apple products - Prototype, implement, and evaluate new ML models and algorithms for red teaming LLMs MINIMUM QUALIFICATIONS Strong engineering skills and experience in writing production-quality code in Python, Swift or other programming languages Background in generative models, natural language processing, LLMs, or diffusion models Experience with failure analysis, quality engineering, or robustness analysis for AI/ML based features Experience working with crowd-based annotations and human evaluations Experience working on explainability and interpretation of AI/ML models Work with highly-sensitive content with exposure to offensive and controversial content PREFERRED QUALIFICATIONS BS, MS or PhD in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues Proven track record of contributing to diverse teams in a collaborative environment
Requirements
- Strong engineering skills and experience in writing production-quality code
- Background in generative models, natural language processing, LLMs, or diffusion models
- Experience with failure analysis, quality engineering, or robustness analysis for AI/ML based features
- Experience working with crowd-based annotations and human evaluations
- Work with highly-sensitive content with exposure to offensive and controversial content
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Coworking Space
- Dental Insurance
- Employee Assistance Program
- Employee Discounts
Skills
Python Generative Models Natural Language Processing LLMs Diffusion Models SwiftFull Time
$120,000 - $200,000*
2 weeks ago
Worldwide
Job Description
We are the Apple Services Engineering (ASE) Security Red Team. We focus on deep technical security review work of critical ASE services and infrastructure. These security reviews will be scoped and focused on review depth and quality. We are growing our team and looking an AI Security Engineer to lead deep reviews that identify meaningful security improvement opportunities. In this role, you will work closely with the security engineering, InfoSec, privacy, SRE, detection, and design review teams to keep Apple's services secure for our users. You will identify security weaknesses, validate and design detection mechanisms, and provide actionable recommendations to enhance our security posture. You will go beyond simple to find risks and identify obscure and complex risks within complex services. You will collaborate with various architecture and engineering teams to continuously validate and improve our security controls and detection capabilities, with a strong focus on developing repeatable testing frameworks and metrics-driven security improvements. If you love diving into complex and important systems, and driving the security of that system over time, we want to talk to you! DESCRIPTION In this role, you will scope and lead focused security reviews on critical internet scale applications and supporting infrastructure. You will learn the services architecture and risk profile to build a scope that enables a meaningful security review. You will use AI to expedite all stages of your work, while maintaining accountability for all outcomes. You will use your depth of knowledge to identify risks that go undetected by agentic tools and processes to reinforce and improve those processes. MINIMUM QUALIFICATIONS 6+ years in an information security field or software engineering; 2 or more of those years conducting security reviews 2+ years of manually reviewing source code to assist in finding vulnerabilities Experience building or contributing to AI security toolss Extensive infrastructure, cloud, and application security experience Experience communicating risk to engineering and leadership teams Experience going deep on complex systems for extended engagements PREFERRED QUALIFICATIONS Bachelors degree in Computer Science / Engineering or a related, with emphasis in security related fields (or equivalent experience) Experience constructing threat scenario narratives and building exploit chains Community contributions like public CVEs, bug bounty recognition, open source tools, blogs, talks etc.
Requirements
- 6+ years in an information security field or software engineering
- 2 or more years conducting security reviews
- 2+ years of manually reviewing source code
- Experience building or contributing to AI security tools
- Extensive infrastructure, cloud, and application security experience
- Experience communicating risk to engineering and leadership teams
- Experience going deep on complex systems for extended engagements
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Coworking Space
- Dental Insurance
- Disability Insurance
- Employee Assistance Program
Skills
AI Machine Learning Security Cloud Security Infrastructure Security Application Security Source Code Review Threat Scenario NarrativesFull Time
$120,000 - $200,000*
2 months ago
Worldwide
Job Description
Join Apple Services Engineering to build the next generation of AI evaluation systems. We are seeking a staff machine learning platform engineer to lead the architectural design and development of the high availability services and internal tools powering self-service evaluation at scale. You will partner with researchers to operationalize their innovations, transforming complex workflows into intuitive, developer-first platforms. We are looking for builders who thrive in the ambiguity of new initiatives and are passionate about creating scalable infrastructure. DESCRIPTION You will join the engineering team responsible for democratizing AI evaluation across the organization. Your focus will be on developing the developer experience—architecting and implementing the APIs, SDKs, and platform services that turn complex evaluation metrics into simple, self-service calls. You will work hand-in-hand with researchers to operationalize sophisticated measurement techniques, ensuring they scale reliably within our high-availability infrastructure. In this role, you will drive the engineering standards for a new organization, upholding the code quality, automation, and testing rigor required to support the rapid evolution of Generative AI and Agentic systems. MINIMUM QUALIFICATIONS 8+ years of hands-on software engineering experience, with a track record of owning the technical direction of a platform or infrastructure domain. Strong proficiency in the Python ecosystem (e.g., FastAPI, Pydantic, Pandas). You write production-grade code and lead architectural discussions on day one. Customer Obsession & Product Thinking: You have owned the technical roadmap for an internal platform, presented it to senior stakeholders, and shipped against it. You independently translate vague requirements from other teams into concrete engineering specifications and platform roadmaps. Demonstrated experience leading technical partnerships with Data Scientists or Researchers: You have taken research code and shipped it as a production service and built the abstractions, testing frameworks, and deployment pipelines that made the next handoff faster than the last.. Strong expertise in API Design & Platform Infrastructure: You have designed and owned APIs and SDKs that other developers rely on, with a focus on versioning, backward compatibility, and developer experience at scale. Operational excellence background: You have architected and owned CI/CD pipelines, containerization (Docker/Kubernetes), and monitoring (Datadog/Prometheus) for production services, and have been accountable for their reliability. Bachelors in Computer Science or related field, Masters preferred. PREFERRED QUALIFICATIONS Deep familiarity with AI Evaluation Frameworks: You have built, extended, or contributed to modern evaluation tools like DeepEval, Ragas, TruLens, or LangSmith. You understand how to implement and scale model-based evaluation workflows across a large organization. Evaluation Service Deployment: Own the deployment, scaling, and operational health of evaluation services in production - including high-throughput evaluation job orchestration (queueing, prioritization, concurrency, auto-scaling), and defining SLAs for evaluation pipeline latency and availability. Observability & Reliability: Experience instrumenting production ML evaluation pipelines including tracking evaluation job throughput, queue depth, judge model latency SLAs, scoring drift over time, and failure modes specific to non-deterministic LLM-based evaluation workflows. Deep understanding of Generative AI & Agents: You understand the engineering challenges of relying on LLMs and Agents as software components—specifically managing token economics, handling rate limits, and evaluating non-deterministic, multi-step reasoning capabilities. You have built production systems that depend on these components and have solved these problems at scale. Builder Experience: You have thrived in startup-like environments, navigating high ambiguity to deliver complex technical roadmaps from scratch.
Requirements
- 8+ years of hands-on software engineering experience
- Strong proficiency in the Python ecosystem
- Customer Obsession & Product Thinking
- Demonstrated experience leading technical partnerships with Data Scientists or Researchers
- Strong expertise in API Design & Platform Infrastructure
- Operational excellence background
- Bachelors in Computer Science or related field, Masters preferred
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Coworking Space
- Dental Insurance
- Disability Insurance
- Employee Assistance Program
Skills
Python Pandas API design Docker Kubernetes FastAPI Pydantic Platform InfrastructureFull Time
$120,000 - $180,000*
2 months ago
Worldwide
Job Description
Apple Services Engineering (ASE) powers the AI and LLM features behind experiences that hundreds of millions of users love every day. As these systems increasingly rely on human-in-the-loop evaluation, the quality of our products is directly constrained by the quality of our evaluation systems. We believe that to build exceptional AI, you need exceptional mechanisms to validate the signals used to train and evaluate them. DESCRIPTION The Human-centered AI, Data Quality Operations team is looking for a Senior Applied Scientist to join our growing team. We are building the systems and methodologies that make AI evaluation trustworthy, and scalable — directly shaping how Apple develops and validates AI across products and services. In this role, you will develop novel, scalable quality control solutions, working closely with cross-functional teams to ensure the data powering our AI/ML systems meets the highest standards of accuracy, consistency, and relevance. Your work will span two connected problem spaces. The first is the methodology and tooling that generates reliable ground truth and detects quality failures across human annotation and automated evaluation pipelines. The second is the autonomous QA agents that make those methodologies generalizable across teams and use cases. This role demands fluency across research thinking and engineering execution — you will prototype, validate, and ship. A strong point of view on when not to use a model or agent is as valued here as the ability to build one. MINIMUM QUALIFICATIONS 5+ years of industry experience in applied science or machine learning with demonstrated impact on shipped systems Strong hands-on experience with Large Language Models including prompt engineering and applied use cases such as grading, validation, or classification Strong working knowledge of evaluation methodology for generative AI, including LLM-as-a-judge design, meta-evaluation, and failure mode analysis Familiarity with human-in-the-loop evaluation systems and the operational dynamics that affect data quality at scale Hands-on experience designing ground truth generation pipelines across varied task types and annotation modalities Proficiency in Python and relevant ML frameworks, with production experience building, deploying, and monitoring LLM-based pipelines and agents MS or PhD in Computer Science, Machine Learning, Statistics, or a related quantitative field, or equivalent practical experience PREFERRED QUALIFICATIONS PhD in Computer Science, Machine Learning, Statistics, or a related field Experience designing agent architectures that are configurable and extensible by practitioners who did not build them Hands-on experience building anomaly detection systems for evaluation quality, including drift detection, distribution analysis, and systematic bias identification Strong communication skills with the ability to influence technical direction across cross-functional teams Demonstrated passion for leveraging AI to improve work efficiency and scale
Requirements
- 5+ years of industry experience in applied science or machine learning
- Strong hands-on experience with Large Language Models
- Strong working knowledge of evaluation methodology for generative AI
- Familiarity with human-in-the-loop evaluation systems
- Proficiency in Python and relevant ML frameworks
- MS or PhD in Computer Science, Machine Learning, Statistics, or a related quantitative field
Benefits
- 401k Matching
- Certification Support
- Flexible Hours
- Gym Membership
- Health Insurance
- Home Office Budget
- Learning Budget
- Paid Time Off
Skills
Python Machine Learning Large Language Models prompt engineering Generative AI Evaluation Methodology Human-in-the-loop EvaluationFull Time
$120,000 - $200,000*
3 months ago
Worldwide
Job Description
Join us as we build world-class groundbreaking products for our customers! Apple's Responsible AI and Safety team focuses on innovative technologies, methodologies, and research to enable fantastic user experiences and to push the frontier of machine learning. Our team is looking to hire a leader with a strong track record in Applied Research, who is passionate about ML and foundation models with a focus on responsibility, fairness, and safety. In this role, you will lead the research and application of ML methods for technologies that power breakthrough user experiences while upholding Apple's values, privacy, and quality standards. DESCRIPTION As a manager, you will lead a growing team to innovate and deliver high-impact features with state-of-the-art machine learning technologies. You will be working on the design and development of core technologies in the areas of GenAI and Responsible AI and Safety. An ideal candidate should have experience leading and supporting teams to develop ML-based solutions from initial concept through product release. MINIMUM QUALIFICATIONS Proven technical leadership with 5+ years of team management or leadership experience PhD or equivalent experience, in Computer Science, or related technical field is helpful Demonstrated expertise in ML research and/or ML production deployment lifecycle Deep understanding of LLM Hands-on experience working with deep learning toolkits such as Tensorflow or PyTorch Ability to formulate a research problem, design, experiment, implement and communicate solutions Ability to work in a diverse collaborative environment PREFERRED QUALIFICATIONS Publication record in relevant conferences (e.g., NeurIPS, ICML, ICLR, EMNLP, etc.) Experience working on Responsible AI and AI Safety
Requirements
- Proven technical leadership with 5+ years of team management or leadership experience
- PhD or equivalent experience, in Computer Science, or related technical field is helpful
- Demonstrated expertise in ML research and/or ML production deployment lifecycle
- Deep understanding of LLM
- Hands-on experience working with deep learning toolkits such as Tensorflow or PyTorch
- Ability to formulate a research problem, design, experiment, implement and communicate solutions
- Ability to work in a diverse collaborative environment
- Publication record in relevant conferences (e.g., NeurIPS, ICML, ICLR, EMNLP, etc.)
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Dental Insurance
- Employee Assistance Program
- Employee Discounts
- Flexible Hours
Skills
TensorFlow PyTorch Deep Learning Machine Learning Artificial Intelligence Python Computer ScienceFull Time
$120,000 - $180,000*
3 months ago
Worldwide
Job Description
Join us as we build world-class groundbreaking products for our customers! Apple's Data and ML Innovation team focuses on innovative technologies, methodologies, and research to enable fantastic user experiences and to push the frontier of machine learning.
Our team is looking to hire a tech lead with a strong track record in Applied Research who is passionate about ML and foundation models with a focus on responsibility, fairness, and safety. In this role, you will lead the research and application of ML methods for technologies that power breakthrough user experiences while upholding Apple's values, privacy, and quality standards.
This role will be highly multifunctional. You will collaborate closely with top machine learning researchers and engineers, software engineers, and design teams to develop and deliver groundbreaking solutions for Apple products. We believe that the most exciting problems in machine learning research arise at the intersection of emerging technologies and real-world use cases. This is also where the most critical breakthroughs come from. As a researcher, you will: - Define and deliver responsible machine learning technologies - Develop methods and frameworks to train and evaluate foundation models with responsibility and safety in mind - Research and advance safety alignment and model robustness methods for foundation models - Research and develop mitigations and safeguards to ensure safe deployment of LLM’s in Apple products - Advocate for scientific and engineering excellence: You will contribute to the architecture and high-level structure of Apple's AI-powered platform and features - Work multi-functionally on a diverse set of challenging problems and collaborate with extraordinary software engineers, machine learning engineers, and researchers to impact the future of Apple products - Work closely with product and partner teams to drive requirement definition, technical quality of deliverables, and execution
Strong research or product deployment record in areas related to responsible AI, with publications in top ML venues (e.g., ACL, CHI, CVPR, EMNLP, FAccT, ICML, Interspeech, NeurIPS, UIST, etc.) Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models Proficient programming skills in Python and deep learning toolkits (e.g. JAX, PyTorch, Tensorflow) Work with highly-sensitive content with exposure to offensive and controversial content BS, MS or PhD in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
Experience with LLM training and safety alignment Strong organizational and operational skills working with large, multi-functional, and diverse teams
Description
This role will be highly multifunctional. You will collaborate closely with top machine learning researchers and engineers, software engineers, and design teams to develop and deliver groundbreaking solutions for Apple products. We believe that the most exciting problems in machine learning research arise at the intersection of emerging technologies and real-world use cases. This is also where the most critical breakthroughs come from. As a researcher, you will: - Define and deliver responsible machine learning technologies - Develop methods and frameworks to train and evaluate foundation models with responsibility and safety in mind - Research and advance safety alignment and model robustness methods for foundation models - Research and develop mitigations and safeguards to ensure safe deployment of LLM’s in Apple products - Advocate for scientific and engineering excellence: You will contribute to the architecture and high-level structure of Apple's AI-powered platform and features - Work multi-functionally on a diverse set of challenging problems and collaborate with extraordinary software engineers, machine learning engineers, and researchers to impact the future of Apple products - Work closely with product and partner teams to drive requirement definition, technical quality of deliverables, and execution
Minimum Qualifications
Strong research or product deployment record in areas related to responsible AI, with publications in top ML venues (e.g., ACL, CHI, CVPR, EMNLP, FAccT, ICML, Interspeech, NeurIPS, UIST, etc.) Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models Proficient programming skills in Python and deep learning toolkits (e.g. JAX, PyTorch, Tensorflow) Work with highly-sensitive content with exposure to offensive and controversial content BS, MS or PhD in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
Preferred Qualifications
Experience with LLM training and safety alignment Strong organizational and operational skills working with large, multi-functional, and diverse teams
Requirements
- Strong research or product deployment record in areas related to responsible AI
- Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models
- Proficient programming skills in Python and deep learning toolkits
- Work with highly-sensitive content with exposure to offensive and controversial content
- BS, MS or PhD in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
Benefits
- 401k Matching
- Health Insurance
- Paid Time Off
- Remote Work
- Stock Options
Skills
Python PyTorch TensorFlow Machine Learning Deep Learning JAXFull Time
$120,000 - $180,000*
4 months ago
Worldwide
Job Description
Apple Intelligence is designed as a deeply integrated, privacy-preserving capability that seamlessly enhances what people can do across iPhone, iPad, Mac, and other devices. Our goal is to elevate the user experience without requiring our customers to learn new products or fundamentally change how they interact with their technology.
Our team leads Responsible AI & Safety initiatives for global generative AI products, operating at the intersection of policy, product, and GenAI. We're seeking candidates who will shape safety policies in partnership with leadership, design, engineering, legal, and regulatory stakeholders—ensuring our safeguards advance both user protection and product innovation. You will collaborate closely with top machine learning researchers and engineers, software engineers, and design teams to develop and deliver groundbreaking solutions for Apple products. We believe that the most exciting problems in machine learning research arise at the intersection of emerging technologies and real-world use cases. This is also where the most critical breakthroughs come from. You will also work on producing safety evaluations that uphold Apple’s Responsible AI values requires thoughtful data sampling, creation, and curation for evaluation datasets; high quality, detailed annotations and careful auto-grading to assess feature performance; and mindful analysis to understand what the evaluation means for the user experience.
3+ years of proven ability in machine learning, including work with generative models (Transformers, LLMs, VLMs), NLP, or Computer Vision 4+ Years research or product deployment record in areas related to responsible AI, with publications in top ML venues (e.g., ACL, CHI, CVPR, EMNLP, FAccT, ICML, Interspeech, NeurIPS, UIST, etc.) Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures. PhD, MS or BS in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
Experience working in the Responsible AI space. Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable. Proven success contributing in a highly cross‑functional environment Experience shipping complex AI systems at global scale
Description
Our team leads Responsible AI & Safety initiatives for global generative AI products, operating at the intersection of policy, product, and GenAI. We're seeking candidates who will shape safety policies in partnership with leadership, design, engineering, legal, and regulatory stakeholders—ensuring our safeguards advance both user protection and product innovation. You will collaborate closely with top machine learning researchers and engineers, software engineers, and design teams to develop and deliver groundbreaking solutions for Apple products. We believe that the most exciting problems in machine learning research arise at the intersection of emerging technologies and real-world use cases. This is also where the most critical breakthroughs come from. You will also work on producing safety evaluations that uphold Apple’s Responsible AI values requires thoughtful data sampling, creation, and curation for evaluation datasets; high quality, detailed annotations and careful auto-grading to assess feature performance; and mindful analysis to understand what the evaluation means for the user experience.
Minimum Qualifications
3+ years of proven ability in machine learning, including work with generative models (Transformers, LLMs, VLMs), NLP, or Computer Vision 4+ Years research or product deployment record in areas related to responsible AI, with publications in top ML venues (e.g., ACL, CHI, CVPR, EMNLP, FAccT, ICML, Interspeech, NeurIPS, UIST, etc.) Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures. PhD, MS or BS in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
Preferred Qualifications
Experience working in the Responsible AI space. Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable. Proven success contributing in a highly cross‑functional environment Experience shipping complex AI systems at global scale
Requirements
- 3+ years of proven ability in machine learning
- 4+ Years research or product deployment record in areas related to responsible AI
- Strong research fundamentals, machine learning principles, and development methodologies around LLMs, foundation models, and diffusion models
- Experience working with generative models for evaluation and/or product development
- Up-to-date knowledge of common challenges and failures
- PhD, MS or BS in Computer Science, Machine Learning, or related fields or an equivalent qualification acquired through other avenues
Benefits
- 401k Matching
- Certification Support
- Flexible Hours
- Gym Membership
- Health Insurance
- Home Office Budget
- Learning Budget
- Paid Time Off
Skills
Machine Learning Generative Models LLMs NLP Computer Vision Transformers VLMsFull Time
$120,000 - $180,000*
4 months ago
Worldwide
Job Description
Play a part in building the next generation of generative AI applications at Apple. We’re looking for AI Leaders, Scientists, Engineers and Researchers to tackle ambitious projects that will impact the future of Apple and our products.
Research and advance red teaming methods for LLM's and diffusion models Research and develop mitigations and safeguards to ensure safe deployment of LLM's in Apple products Develop tools, metrics, and datasets for assessing and evaluating the safety of LLM's over the model deployment lifecycle, as well as methods and tools to help interpret and explain failures in language models
PhD or equivalent experience, in Computer Science, or related technical field is helpful Demonstrated expertise in ML research and/or ML production deployment lifecycle Deep understanding of LLM Hands-on experience working with deep learning toolkits such as Tensorflow or PyTorch Ability to formulate a research problem, design, experiment, implement and communicate solutions Ability to work in a diverse collaborative environment
Publication record in relevant conferences (e.g., NeurIPS, ICML, ICLR, EMNLP, etc.) Experience working on Responsible AI and AI Safety
Description
Research and advance red teaming methods for LLM's and diffusion models Research and develop mitigations and safeguards to ensure safe deployment of LLM's in Apple products Develop tools, metrics, and datasets for assessing and evaluating the safety of LLM's over the model deployment lifecycle, as well as methods and tools to help interpret and explain failures in language models
Minimum Qualifications
PhD or equivalent experience, in Computer Science, or related technical field is helpful Demonstrated expertise in ML research and/or ML production deployment lifecycle Deep understanding of LLM Hands-on experience working with deep learning toolkits such as Tensorflow or PyTorch Ability to formulate a research problem, design, experiment, implement and communicate solutions Ability to work in a diverse collaborative environment
Preferred Qualifications
Publication record in relevant conferences (e.g., NeurIPS, ICML, ICLR, EMNLP, etc.) Experience working on Responsible AI and AI Safety
Requirements
- PhD or equivalent experience in Computer Science or related technical field
- Demonstrated expertise in ML research and/or ML production deployment lifecycle
- Deep understanding of LLM
- Hands-on experience working with deep learning toolkits such as Tensorflow or PyTorch
- Ability to formulate a research problem, design, experiment, implement and communicate solutions
Benefits
- 401k Matching
- Health Insurance
- Paid Time Off
- Remote Work
- Stock Options
Skills
TensorFlow PyTorch Deep Learning LLM AI Safety Responsible AIFull Time
$120,000 - $180,000*
4 months ago
Worldwide
Job Description
Would you like to play a part in building the next generation of generative AI applications at Apple? We’re looking for Machine Learning Engineers to work on ambitious projects that will impact the future of Apple, our products, and the broader world. This role is directed at assessing, quantifying, and improving the safety and inclusivity of Apple’s Generative-AI powered features and products. In this role you’ll have the opportunity to tackle innovative problems in machine learning, particularly focused on large language models for text generation, diffusion models for image generation, and mixed model systems for multimodal applications. As a member of Apple’s Responsible AI group you will be working on a wide array of new features and research in the generative AI space. Our team is currently interested in large generative models for vision and language, with particular interest on Responsible AI, safety, fairness, robustness, explainability, and uncertainty in models.
This role focuses on developing, carrying-out, interpreting, and communicating pre- and post-ship evaluations of the safety of Apple Intelligence features. Both human grading and model-based auto-grading are thoughtfully leveraged to power these evaluations. Additionally, this role researches and develops auto-grading methodology & infrastructure to benefit ongoing and future Apple Intelligence safety evaluations. Producing safety evaluations that uphold Apple’s Responsible AI values requires thoughtful data sampling, creation, and curation for evaluation datasets; high quality, detailed annotations and careful auto-grading to assess feature performance; and mindful analysis to understand what the evaluation means for the user experience. This role heavily draws on applied data science, scientific investigation and interpretation, cross-functional communication and collaboration, and metrics reporting and presentation.
MS, or PhD in Computer Science, Machine Learning, Statistics, or related fields; or an equivalent qualification acquired through other avenues. Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures. Strong engineering skills and experience in writing production-quality code in Python. Deep experience in foundation model-based AI programming (i.e.: using DSPy for optimizing foundation model prompts, for example) and a drive to innovate in this space. Experience working with noisy, crowd-based data labels and human evaluations.
Experience working in the Responsible AI space. Prior scientific research and publication experience. Strong organizational and operational skills working with large, multi-functional, and diverse teams. Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable.
Description
This role focuses on developing, carrying-out, interpreting, and communicating pre- and post-ship evaluations of the safety of Apple Intelligence features. Both human grading and model-based auto-grading are thoughtfully leveraged to power these evaluations. Additionally, this role researches and develops auto-grading methodology & infrastructure to benefit ongoing and future Apple Intelligence safety evaluations. Producing safety evaluations that uphold Apple’s Responsible AI values requires thoughtful data sampling, creation, and curation for evaluation datasets; high quality, detailed annotations and careful auto-grading to assess feature performance; and mindful analysis to understand what the evaluation means for the user experience. This role heavily draws on applied data science, scientific investigation and interpretation, cross-functional communication and collaboration, and metrics reporting and presentation.
Minimum Qualifications
MS, or PhD in Computer Science, Machine Learning, Statistics, or related fields; or an equivalent qualification acquired through other avenues. Experience working with generative models for evaluation and/or product development, and up-to-date knowledge of common challenges and failures. Strong engineering skills and experience in writing production-quality code in Python. Deep experience in foundation model-based AI programming (i.e.: using DSPy for optimizing foundation model prompts, for example) and a drive to innovate in this space. Experience working with noisy, crowd-based data labels and human evaluations.
Preferred Qualifications
Experience working in the Responsible AI space. Prior scientific research and publication experience. Strong organizational and operational skills working with large, multi-functional, and diverse teams. Curiosity about fairness and bias in generative AI systems, and a strong desire to help make the technology more equitable.
Requirements
- MS or PhD in Computer Science, Machine Learning, Statistics, or related fields
- Experience working with generative models for evaluation and/or product development
- Up-to-date knowledge of common challenges and failures
- Strong engineering skills and experience in writing production-quality code in Python
- Deep experience in foundation model-based AI programming
- Experience working with noisy, crowd-based data labels and human evaluations
- Experience working in the Responsible AI space
- Prior scientific research and publication experience
Benefits
- 401k Matching
- Certification Support
- Company Retreats
- Conference Budget
- Coworking Space
- Dental Insurance
- Employee Assistance Program
- Employee Discounts
Skills
Python Machine Learning Large Language Models Responsible AI Generative Models Diffusion Models Mixed Model Systems FairnessFull Time
$120,000 - $180,000*
4 months ago
Worldwide
Job Description
AI systems are only as trustworthy as the methods used to evaluate them. At
Apple, where AI powers experiences for billions of people, getting evaluation
right is not a support function—it is a foundational science. Our team, part of
Apple Services Engineering, is building that scientific foundation: rigorous,
scalable evaluation methodology for LLMs, agentic systems, and human-AI
interaction. What makes this team unusual is its interdisciplinary core. You
will work alongside measurement scientists (psychometrics, validity theory), ML
researchers, and platform engineers—bringing together ML research, statistical
rigor, and production engineering. We are looking for a Research Scientist who
treats evaluation methodology itself as a first-class research problem—someone
with deep technical fluency in preference learning, reward modeling, or
calibration theory, and the drive to advance the field while solving real
problems at scale. We're hiring at multiple levels (early-career to senior
researchers). What unites all candidates is depth of thinking about evaluation
as a research problem.
DESCRIPTION
This is primarily a research role. You will formulate open problems in
evaluation science, design experiments, publish findings, and drive projects
from conception through completion. While you will also partner with platform
engineers to ensure your methods are productionized into SDKs and APIs, the
focus of the role is original research. Our research team brings together ML
scientists and measurement scientists to tackle evaluation as both a machine
learning and a measurement problem, building methods that are technically
innovative and scientifically valid. You will also work closely with a platform
engineering team that translates research into production-ready SDKs and APIs
used across Apple. The successful candidate will have a strong publication
record in evaluation-adjacent ML areas and a demonstrated ability to implement
complex methods from recent papers, run large-scale experiments, and communicate
results to both technical and non-technical audiences.
MINIMUM QUALIFICATIONS
Ph.D. in Computer Science, Machine Learning, or a closely related field, with a
research focus in evaluation-adjacent areas (preference learning, RLHF, human
feedback, calibration, automated assessment) Strong publication record at
top-tier conferences (NeurIPS, ICML, ICLR, ACL, EMNLP), including first-author
publications demonstrating independent research contributions Deep technical
expertise in at least one evaluation-adjacent ML area, with strong mathematical
foundations: preference learning and reward modeling (RLHF, DPO, reward hacking,
specification gaming); OR calibration theory, proper scoring rules, and
statistical reliability; OR human-AI interaction methodology (active learning,
annotation quality, preference elicitation) Demonstrated ability to implement
complex methods from recent papers and run large-scale experiments Track record
of translating research into practical systems—prototypes, tools, or methods
adopted by others Excellent written and verbal communication skills, including
the ability to write clear research papers and explain complex concepts to
diverse audiences
PREFERRED QUALIFICATIONS
Publications specifically on evaluation methodology—papers about how to
evaluate, not just papers that use evaluation to demonstrate model improvements
Strong hands-on experience with modern ML frameworks (PyTorch, JAX, or
TensorFlow) and training or fine-tuning large language models Experience with
theoretical foundations of evaluation: measurement theory and validity
frameworks, statistical learning theory (calibration, reliability, decision
theory), or preference elicitation and aggregation Specific research experience
in one or more of: reward modeling and RLHF for alignment; LLM-as-judge
approaches (calibration, rubric design, bias mitigation); benchmark design and
validation (IRT, contamination detection); human evaluation methodology
(protocol design, quality control); or agentic and multi-agent system evaluation
Demonstrated passion for evaluation as a research area: conference
presentations, workshops, or tutorials on evaluation topics; open-source
contributions to evaluation tools or benchmarks; active engagement with the
evaluation research community Experience with cross-disciplinary research, such
as collaboration with social scientists, psychometricians, or domain experts
Requirements
- Ph.D. in Computer Science, Machine Learning, or a closely related field
- Strong publication record at top-tier conferences
- Deep technical expertise in at least one evaluation-adjacent ML area
- Demonstrated ability to implement complex methods from recent papers
- Track record of translating research into production-ready SDKs and APIs
Benefits
- 401k Matching
- Health Insurance
- Paid Time Off
- Flexible Hours
- Remote Work
- Stock Options
- Training Budget
- Wellness Programs
