AI Governance and Evaluation Executive

WHY UT SOUTHWESTERN?
With over 75 years of excellence in Dallas-Fort Worth, Texas, UT Southwestern is committed to excellence, innovation, teamwork, and compassion. As a world-renowned medical and research center, we strive to provide the best possible care, resources, and benefits for our valued employees. Ranked as the No. 1 hospital in Dallas-Fort Worth according to U.S. News & World Report, we invest in you with opportunities for career growth and development to align with your future goals. Our highly competitive benefits package offers healthcare, PTO and paid holidays, on-site childcare, wage, merit increases and so much more. We invite you to be a part of the UT Southwestern team where you’ll discover a culture of teamwork, professionalism, and a rewarding career! 

JOB SUMMARY
The AI Governance and Evaluation Executive is the program and operations leader responsible for implementing, scaling, and continuously improving UTSW's enterprise approach to responsible AI governance and evaluation. Reporting to the CAIO, this role executes the governance and evaluation roadmap and adoption plan across the health system, research and academic mission, biomedical education, and administrative and business functions. *** The leader in this role converts enterprise direction into annual priorities and repeatable workflows for intake, classification, evaluation, review, vendor assessment, deployment conditions, monitoring, reassessment, and stakeholder support. The role maintains enterprise portfolio visibility, drives reuse and consistency, and coordinates cross-functional execution and works closely with Compliance, Information Security, and Information Technology Services on AI risk management, while those functions retain formal ownership of their respective risk-management domains. 

 

BENEFITS
UT Southwestern is proud to offer a competitive and comprehensive benefits package to eligible employees. Our benefits are designed to support your overall wellbeing, and include:

• PPO medical plan, available day one at no cost for full-time employee-only coverage
• 100%25 coverage for preventive healthcare-no copay
• Paid Time Off, available day one
• Retirement Programs through the Teacher Retirement System of Texas (TRS)
• Paid Parental Leave Benefit
• Wellness programs
• Tuition Reimbursement
• Public Service Loan Forgiveness (PSLF) Qualified Employer
• Learn more about these and other UTSW employee benefits!

EXPERIENCE AND EDUCATION

Required

• Education
  • Master's Degree in Public Health, Biomedical Informatics, Data Science, Computer Science, Engineering, Research Administration, or a related field. 
     

• Experience
  • 8 years of progressively responsible experience in AI, digital health, healthcare operations, technology governance, clinical informatics, data analytics, research administration, compliance, or a related field. 
  • 4 years leading people, programs, or complex cross-functional initiatives. 
  • Demonstrated experience operating cross-functional governance, evaluation, assurance, or enterprise programs. 
  • Demonstrated experience implementing enterprise roadmaps, operating models, or new capabilities, and translating priorities into measurable outcomes. 
     

Preferred

• Education
  • PhD or additional advanced degree in a related field. 
     

• Experience
  • Experience in an academic medical center, integrated health system, life sciences organization, or similarly complex regulated environment. 
  • Experience with AI/ML evaluation, deployment, governance, responsible AI, or assurance. 
  • Experience evaluating AI vendors and coordinating due diligence. 
  • Technical fluency across LLM ecosystems, MLOps, retrieval and vector technologies, AI APIs and integrations, lifecycle monitoring, and incident-response interfaces sufficient to engage technical teams and assess governance evidence. 
  • Experience building or scaling a new team or function and managing tools, vendors, contracts, or external resources. 
  • Familiarity with generative AI, LLMs, multimodal models, foundation models, and agentic AI. 
  • Working knowledge of healthcare AI evaluation and lifecycle monitoring, including quantitative methods and model cards. 
  • Familiarity with NIST AI RMF, ISO/IEC 42001, and related responsible AI frameworks or standards. 
     

JOB DUTIES

• Lead Governance and Evaluation Operations and Roadmap Execution
  • Independently run the day-to-day enterprise Responsible AI Governance and Evaluation program on behalf of the CAIO.
  • Translate program strategy into an annual operating roadmap, including priorities, dependencies, resource needs, adoption activities, and measurable program outcomes.
  • Operationalize AI policies, standards, workflows, decision rights, review pathways, and documentation requirements.
  • Coordinate governance across institutional and domain-specific structures including work with UTSW executives and senior leadership.
  • Prioritize work based on institutional value, risk, dependencies, and capacity, and surface tradeoffs for decision.
  • Ensure governance is risk-proportionate, timely, consistent, and designed to enable responsible innovation.
  • Identify complex, novel, or high-consequence cases requiring escalation to the CAIO or appropriate governance body.
• Manage End-to-End Governance Workflow
  • Operate overall intake, triage, AI risk classification, evaluation, multidisciplinary review, decision, deployment conditions, monitoring, and reassessment workflows.
  • Coordinate enterprise governance committee agendas, case materials, decisions, action items, and follow-up.
  • Maintain authoritative records of approvals, conditions, exceptions, reassessments, and governance precedents.
  • Develop expedited and standardized pathways for low-risk or well-understood AI uses.
  • Create and maintain reusable review patterns, templates, evidence packages, evaluation assets, and precedents that improve consistency and reduce duplicate work.
  • Provide clear guidance, consultation, education, and stakeholder support to promote early and ongoing engagement and adoption of governance requirements.
  • Track service levels, bottlenecks, and outstanding governance actions and drive closure.
• Operationalize AI Evaluation and Assurance
  • Implement evaluation requirements proportionate to AI risk, intended use, and deployment context.
  • Coordinate and ensure assessment of performance, reliability, safety, bias, security considerations, and unintended consequences as applicable.
  • Coordinate independent evaluation of consequential AI systems as appropriate.
  • Lead assessment of vendor evidence and claims and identify when supplemental local testing or independent evaluation is required.
  • Facilitate prospective validation, simulation, and other assurance activities as warranted.
  • Translate evaluation findings into governance conditions, remediation actions, monitoring requirements, and escalation.
  • Connect evaluation plans and findings to measurable outcomes where appropriate.
• Support Governance of Generative and Agentic AI
  • Ensure application of governance and evaluation practices to foundation models, LLMs, multimodal models, and agentic AI.
  • Coordinate review of tool and data access, identity and authorization, human oversight, autonomous actions, memory, agent-to-agent interactions, observability, guardrails, testing, and fail-safe mechanisms.
  • Ensure UTSW maintains practical governance approaches across commercial frontier models, institutionally hosted models, open-weight models, and internally developed AI.
  • Convert emerging model and agent capabilities, research, and standards into practical review criteria, test protocols, and playbooks.
• Coordinate AI Portfolio, Vendor and Adoption Operations
  • Maintain enterprise visibility of AI systems, vendors, accountable owners, intended uses, risk classifications, evaluation status, and monitoring obligations.
  • Coordinate vendor due diligence and evidence collection with Procurement, Legal, Compliance, Information Security, IT, Data, and domain teams, supporting build-buy-partner decisions without assuming ownership of procurement or technical architecture.
  • Track portfolio priorities, review readiness, dependencies, decisions, and open conditions and prepare concise materials for decision forums.
  • Responsible for continuous adoption through training, guidance, communications, and integration of governance requirements into business, research, clinical, and technology workflows.
  • Promote common methods, reusable evidence, and standard controls to reduce duplication and improve enterprise consistency.
• Partner on AI Risk Management
  • Work directly with Compliance and Information Security to identify and assess AI-related risks and to incorporate AI considerations into their established enterprise risk-management processes.
  • Lead coordination of AI governance requirements, evidence, mitigations, and escalation with the risk-owning functions and other control partners.
  • Partner with other senior leaders to help translate risk findings into governance decisions, deployment conditions, monitoring requirements, remediation plans, and reassessment triggers.
  • Stay current on emerging AI regulatory requirements, standards, and professional guidance relevant to UTSW.
• Enable Domain Governance
  • Serve as a central operating leadership partner to domain leaders and governance groups.
  • Engage stakeholders early so governance and evaluation requirements are built into planning, procurement, research, development, implementation, and operations.
  • Health System: clinical and operational AI, patient-facing tools, predictive models, generative AI, ambient AI, and autonomous workflows.
  • Research & Academic: basic, translational, and clinical research AI, research data, scientific AI, external models, AI-enabled experimentation, and biomedical education.
  • Administrative/Business: workforce productivity, finance, HR, revenue cycle, procurement, communications, operations, and automation.
• Build and Lead the Governance Team
  • Lead and oversee UTSW AI Governance Stewards and other governance staff supporting enterprise and domain workflows; define roles, structure, decision rights, and ways of working.
  • Develop practical expertise in AI governance, evaluation operations, assurance, and responsible AI.
  • Build strong working relationships with institutional partners.
  • Create reusable playbooks, templates, review criteria, and operating procedures.
  • Responsible for overall management of assigned budgets, governance and evaluation tooling, contracts, and external resources within approved plans.
• Measure Program Performance
  • Maintain a balanced program scorecard including governance cycle time; enterprise portfolio coverage; adoption of standard pathways; reuse of evaluation assets; use-case volume and risk distribution; evaluation and vendor-review completion; backlog and dependency status; closure of conditions and mitigations; monitoring findings; incidents and near misses; measurable outcomes; and stakeholder experience.
  • Use relevant data to improve throughput, consistency, adoption, and institutional confidence.
• Leadership and Decision Support
  • Advise the CAIO on governance operations, evaluation findings, emerging issues, and institutional AI governance maturity.
  • Represent the Office of the CAIO in designated governance forums and working groups.
  • Facilitate multidisciplinary and/or complete organizational decisions and resolve operational barriers to responsible deployment.
  • Escalate significant or unresolved governance concerns and risk findings to the CAIO and applicable risk-owning functions.
  • Recommend additional evaluation, mitigation, monitoring, conditions, or reassessment when warranted.
  • Prepare executive-level reporting on priorities, performance, dependencies, unresolved issues, resource needs, and decisions required.
  • Explain complex AI, governance, and evaluation concepts clearly to technical and nontechnical audiences.

Knowledge, Skills and Abilities

• Ability to translate enterprise direction into priorities, tradeoffs, repeatable workflows, and measurable program performance.
• Ability to move work across matrixed organizations without relying on command-and-control authority.
• Ability to work comfortably with AI scientists, engineers, clinicians, researchers, and vendors.
• Understands when deeper evaluation or stronger controls are warranted.
• Ability to treat governance stakeholders as partners and designs for timely, usable experiences.
• Ability to evolve governance as models, agents, regulations, and institutional use cases change.
• Strong project and program management and organizational leadership skills.
• Ability to translate complex technical, clinical, regulatory, and ethical issues into practical workflows and decisions.
• Excellent communication, facilitation, negotiation, and stakeholder-management skills.

PHYSICAL REQUIREMENTS/WORKING CONDITIONS

Physical Requirements

• Talking

Working Conditions

• Office Setting

Salary Information
Salary is commensurate with education, skills, and relevant experience.

SECURITY AND EEO STATEMENT

Security
This position is security-sensitive and subject to Texas Education Code 51.215, which authorizes UT Southwestern to obtain criminal history record information. To the extent this position requires the holder to research, work on, or have access to critical infrastructure as defined in Section 117.001(2) of the Texas Business and Commerce Code, the ability to maintain the security or integrity of the critical infrastructure is a minimum qualification to be hired and to continue to be employed in the position.

EEO
UT Southwestern Medical Center is committed to an educational and working environment that provides equal opportunity to all members of the University community. As an equal opportunity employer, UT Southwestern prohibits unlawful discrimination, including discrimination on the basis of race, color, religion, national origin, sex, sexual orientation, gender identity, gender expression, age, disability, genetic information, citizenship status, or veteran status.