Remote Opportunity

Associate Director, Generative AI Evaluation & Standards

Join Johnson & Johnson Innovative Medicine as a senior professional working remotely from Spain. Explore the role, benefits, and apply in one place.

Full Time
$120,000 - $180,000*
2 months ago
Spain
Worldwide
AI Governance & Programs
Senior
Generative AI
Machine Learning
Data Science
+4 more

Job Description

At Johnson & Johnson, we believe health is everything. Our strength in healthcare innovation empowers us to build a world where complex diseases are prevented, treated, and cured, where treatments are smarter and less invasive, and solutions are personal. Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity. Learn more at jnj.com.

As guided by Our Credo, Johnson & Johnson is responsible to our employees who work with us throughout the world. We provide an inclusive work environment where each person is considered as an individual. At Johnson & Johnson, we respect the diversity and dignity of our employees and recognize their merit.

Job Function:

Data Analytics & Computational Sciences

Job Sub Function:

Data Science

Job Category:

People Leader

All Job Posting Locations:

Cornellà de Llobregat, Barcelona, Spain, Madrid, Spain

Job Description:

Associate Director, Generative AI Evaluation & Standards

Location: This position will be located in either Barcelona or Madrid.

About J&J

At Johnson & Johnson, we believe health is everything. Our strength in healthcare innovation empowers us to build a world where complex diseases are prevented, treated, and cured, where treatments are smarter and less invasive, and solutions are personal. Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow and profoundly impact health for humanity.

Our expertise in Innovative Medicine is informed and inspired by patients, whose insights fuel our science-based advancements. Visionaries like you work on teams that save lives by developing the medicines of tomorrow. Join us in developing treatments, finding cures, and pioneering the path from lab to life while championing patients every step of the way.

About the Role

Johnson & Johnson Innovative Medicine is recruiting an Associate Director, GenAI Evaluation & Standards to join our Data Science and Digital Health team (DSDH). This is a newly created leadership role within the Generative AI organization, reporting directly to the Head of Generative AI.

The GenAI team is deploying AI across discovery, development, regulatory, and operations to create AI assistants and agentic systems across the therapeutic areas and functions. As we scale from proof-of-concept to enterprise deployment, we need a dedicated evaluation function that defines what quality means for generative AI in a regulated pharmaceutical R&D environment, communicates the metrics effectively, and enforces that standard across every platform, vendor, and capability we build or adopt. This role owns evaluation and governance for all generative AI work across R&D.

Key Responsibilities

Evaluation Framework Design

  • Design and maintain evaluation frameworks for GenAI solutions spanning LLM quality, RAG performance, agent reliability, safety, and scientific accuracy.

  • Develop therapeutic-area-specific evaluation criteria with business teams, reflecting domain-specific quality requirements per therapeutic area and application type.

  • Define benchmarks for scientific validity, regulatory compliance, data quality, and operational reliability.

  • Establish gold-standard validation datasets and automated evaluation pipelines.

Governance and Standards

  • Serve as technical lead for the Evaluation & Standards Board, a cross-functional governance body for GenAI quality.

  • Present evaluation findings and recommendations to the GenAI Portfolio Steering Committee.

  • Define what “production-ready” means for GenAI in a regulated R&D environment and enforce that standard.

  • Set evaluation gates for product development stages: PoC, Limited Release, Scaled Product, and Product Ops.

Cross-Functional Collaboration

  • Partner with QMS/MLOps on the handoff from pre-adoption evaluation to production quality monitoring.

  • Work with therapeutic area teams on domain-specific evaluation standards.

  • Collaborate with Data Strategy & Products on data quality as an input to the evaluation rubric.

  • Engage JJIT architecture teams on platform-layer evaluation and security assessments.

Opinion Leadership and Capability Building

  • Contribute to the weekly GenAI Outlook newsletter, specifically the evaluation implications of frontier AI developments.

  • Build internal evaluation capability through training, documentation, and tooling for teams conducting assessments.

  • Publish evaluation standards and rubrics as reference material for the broader organization.

Team Leadership

  • Build and lead the GenAI Evaluation & Standards team

  • Attract, develop, and retain top talent in AI evaluation, quality science, and governance.

  • Establish a culture of scientific rigor and independent judgment within the team.

What This Role Is Not

  • Not MLOps or production monitoring: the QMS function handles post-deployment operations.

  • Not platform engineering.

  • Not regulatory affairs or medical writing.

  • Not vendor management or procurement: evaluation is independent of purchasing.

Key Qualifications

  • Advanced degree (PhD strongly preferred) in computational biology, bioinformatics, data science, computer science, AI/ML, biomedical engineering, applied mathematics, or related discipline.

  • Minimum 8 years of post-academic industry experience, with significant time in pharmaceutical or biotechnology R&D.

  • Hands-on expertise with generative AI systems: large language models, retrieval-augmented generation, agentic frameworks, and prompt engineering.

  • Demonstrated track record designing evaluation frameworks, scientific benchmarks, or quality standards for AI/ML systems.

  • Strong people leadership experience, including building and managing technical teams in a matrixed organization.

  • Excellent communication skills: ability to present complex technical findings clearly to non-technical senior stakeholders and defend unpopular conclusions.

  • Strategic thinking capability: ability to connect technical evaluation to business impact and organizational decision-making.

Preferred Qualifications

  • Experience with AI/ML evaluation in regulated environments (FDA, EMA, or equivalent).

  • Domain expertise in at least one therapeutic area: oncology, immunology, or neuroscience.

  • Experience influencing build-versus-buy decisions at an organizational level.

  • Prior collaboration with enterprise IT delivery organizations.

  • Experience with evaluation of multi-modal AI systems (text, image, structured data).

  • Familiarity with enterprise data standards and frameworks (CDISC, FHIR, ontologies).

  • Track record of publishing evaluation methodologies or contributing to AI governance frameworks.

Required Skills

Advanced Analytics, AI/ML Evaluation, Benchmarking, Critical Thinking, Data Science, Evaluation Framework Design, Generative AI, Inclusive Leadership, Leadership, Quality Standards, RAG Systems, Scientific Rigor, Strategic Thinking

Preferred Skills

Agentic AI, Bioinformatics, CDISC, Clinical Data, Computational Biology, Data Governance, FHIR, Knowledge Graphs, Multi-modal AI, Ontology, Pharmaceutical R&D, Regulatory Compliance, Therapeutic Area Expertise

#LI-GR

#LI-Hybrid

#JRDDS
#JNJDataScience
#JRD

 

 

Required Skills:

 

 

Preferred Skills:

Advanced Analytics, Budget Management, Compliance Management, Critical Thinking, Cross-Functional Collaboration, Data Analysis, Data Privacy Standards, Data Quality, Data Reporting, Data Savvy, Data Science, Data Visualization, Developing Others, Digital Fluency, Inclusive Leadership, Leadership, Strategic Thinking, Team Management

Requirements

  • Develop and implement evaluation frameworks for generative AI
  • Collaborate with cross-functional teams to ensure AI governance and compliance
  • Communicate effectively with stakeholders on AI metrics and standards
  • Enforce AI standards across all platforms, vendors, and capabilities
  • Stay up-to-date with industry trends and advancements in AI

Benefits

  • 401k Matching
  • Certification Support
  • Company Retreats
  • Conference Budget
  • Dental Insurance
  • Employee Assistance Program
  • Employee Discounts
  • Flexible Hours

Skills

Generative AI
Machine Learning
Data Science
Python
TensorFlow
Keras
Deep Learning

About AI-Estimated Salary

The salary range shown was not provided by the employer. Our AI has estimated it based on the job title, required experience, location, and industry standards (confidence: 80%). This estimate should be used as a general guide only and may not reflect the actual compensation. Always confirm salary details directly with the employer during the application process.

Ready to Apply?

Join Johnson & Johnson Innovative Medicine today

Salary Range (AI-Estimated)*
$120,000 - $180,000
80% confidence
Posted 2 months ago

More AI Governance & Programs roles you might like

Discover similar opportunities from companies that are also hiring remotely.

Full Time
$7048.161k - $1061.802k
21 hours ago
United States
Worldwide
AI Governance & Programs
Senior
Python
SQL
AI/ML
+4 more
Full Time
$135k - $150k
23 hours ago
Worldwide
AI Governance & Programs
Mid
Python
Machine Learning
LLM
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Governance & Programs
Senior
Data governance
AI Policy
Risk Management
+5 more

Explore more remote openings

Browse fresh listings from our global community of remote-friendly teams.

Full Time
$120,000 - $180,000*
23 hours ago
Worldwide
Americas
AI Security & Privacy
Senior
AI
Machine Learning
Cyber Security
+3 more
Full Time
$150,000 - $250,000*
1 day ago
Worldwide
AI Security & Privacy
Senior
OWASP ZAP
Nmap
Postman
+5 more
Full Time
$150k - $200k
1 day ago
Worldwide
AI Governance & Programs
Mid
AI
Python
Clinical AI
+5 more
Full Time
$120,000 - $180,000*
1 day ago
Australia
Worldwide
AI Governance & Programs
Senior
Data governance
AI Ethics
Regulatory Compliance
+3 more
Full Time
$120,000 - $180,000*
1 day ago
Australia
Worldwide
AI Governance & Programs
Senior
Data governance
AI Ethics
Regulatory Compliance
+3 more
Full Time
$85k - $95k
1 day ago
United States
Model Risk Management & Validation
Senior
Model Risk Management
Quantitative Risk Management
Financial Modeling
+4 more
Full Time
$85k - $95k
1 day ago
United States
Model Risk Management & Validation
Senior
Model Risk Management
Quantitative Risk Management
Financial Modeling
+5 more
Full Time
$80,000 - $140,000*
1 day ago
United States
AI Risk & Controls
Mid
Excel
SQL
Python
+1 more
Full Time
$80,000 - $120,000*
1 day ago
United States
Model Risk Management & Validation
Mid
Excel
SQL
Python
+1 more
Full Time
$129k - $175k
2 days ago
Worldwide
AI Audit / Assurance / Controls Testing
Senior
API
Automation
Python
+3 more
Full Time
$129k - $175k
2 days ago
Worldwide
AI Audit / Assurance / Controls Testing
Senior
API
Automation
Python
+3 more
Full Time
$119.7k - $191.1k
2 days ago
Worldwide
AI Governance & Programs
Senior
Risk Management
Model Risk
Governance
+5 more
Full Time
$120,000 - $180,000*
2 days ago
Ireland
Worldwide
AI Compliance & Legal
Senior
Data Protection
AI Compliance
Regulatory Requirements
+3 more
Full Time
$100,000 - $150,000*
2 days ago
Worldwide
AI Governance & Programs
Mid
AI/ML Concepts
Tableau
JIRA
+1 more
Full Time
$204k - $255k
2 days ago
Worldwide
AI Policy, Enablement & Training
Senior
AI
Machine Learning
Policy Development
+4 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Security & Privacy
Staff
Python
ISO 27001
ISO 27701
+4 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Security & Privacy
Staff
Python
ISO 27001
ISO 27701
+4 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Security & Privacy
Staff
Python
Adversarial Machine Learning
AI Deployment Architectures
+4 more
Full Time
Up to PHP 150k
2 days ago
Worldwide
AI Security & Privacy
Senior
PyTorch
TensorFlow
Containerized Environments
+4 more
Full Time
Up to PHP 150k
2 days ago
Worldwide
AI Security & Privacy
Senior
PyTorch
TensorFlow
Gradient-based attacks
+4 more
Full Time
$209k - $309k
4 days ago
Worldwide
AI Security & Privacy
Senior
API
AI
Security
+1 more
Full Time
$239.5k - $351.5k
4 days ago
Worldwide
AI Security & Privacy
Senior
API
AI
Security
+1 more
Full Time
$230k - $280k
5 days ago
United States
Worldwide
AI Governance & Programs
Senior
OWASP
NIST AI RMF
AI/ML systems
+5 more
Full Time
$230k - $280k
5 days ago
Worldwide
AI Governance & Programs
Senior
Agentic Trust Framework
OWASP
NIST AI RMF
+5 more
Full Time
$120,000 - $180,000*
5 days ago
Worldwide
AI Security & Privacy
Senior
Python
Go
Git
+5 more
Full Time
$159.3k - $273.2k
5 days ago
Worldwide
AI Governance & Programs
Senior
Python
Machine Learning
Data Science
+5 more
Full Time
$120,000 - $180,000*
5 days ago
Worldwide
AI Security & Privacy
Staff
Python
Go
Threat modeling
+3 more
Full Time
$80,000 - $140,000*
5 days ago
Worldwide
AI Governance & Programs
Mid
Responsible AI
ISO/IEC 42001
ISO/IEC 27001
+2 more
Full Time
$120,000 - $180,000*
6 days ago
United States
Worldwide
AI Governance & Programs
Senior
AI Ethics
Risk Management
AI governance frameworks
+5 more
Full Time
$120,000 - $180,000*
6 days ago
Worldwide
AI Security & Privacy
Senior
Security Operations
Cybersecurity
NG-SIEM
+5 more
Full Time
$163k - $237k
6 days ago
Worldwide
AI Governance & Programs
Senior
API
Product Management
AI
+4 more
Full Time
$80,000 - $140,000*
6 days ago
United States
Worldwide
AI Governance & Programs
Mid
Python
Data Analysis
Financial Data
+3 more
Full Time
$80,000 - $140,000*
6 days ago
United States
Worldwide
AI Governance & Programs
Mid
Python
Data Analysis
Machine Learning
+2 more
Full Time
$80,000 - $140,000*
6 days ago
Worldwide
AI Governance & Programs
Mid
Python
Excel
Google Sheets
+4 more
Full Time
$120,000 - $180,000*
6 days ago
Australia
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Data Science
+4 more
Full Time
$120,000 - $180,000*
6 days ago
Worldwide
AI Governance & Programs
Senior
AI Governance
Model Risk Management
Regulatory Compliance
+5 more
Full Time
$120,000 - $180,000*
1 weeks ago
Worldwide
AI Governance & Programs
Senior
Python
ML frameworks
LLM/GenAI tooling
+2 more