Remote Opportunity

Human AI Evaluation Specialist

Join Virtusa as a mid professional working remotely from India. Explore the role, benefits, and apply in one place.

Full Time
$80,000 - $120,000*
4 months ago
India
AI Governance & Programs
Mid
AI evaluation frameworks
Data labeling
Quality rubrics
+4 more

Job Description

Human AI Evaluation Specialist - (CREQ245208)

Description

 

Human AI Evaluation Specialist

Role Focus: Human-in-the-loop Validation · Model Output Quality

Role Summary

Human AI Evaluation Specialists will perform structured human evaluation of Gemini AI outputs by labeling, rating, and providing qualitative feedback on model relevance, correctness, safety, and enterprise usability.

Key Responsibilities

Perform manual evaluation of AI-generated responses

Label datasets for training and validation

Apply quality rubrics and evaluation guidelines

Identify hallucinations, bias, and factual errors

Provide structured feedback for model improvements

Support benchmark creation and golden dataset validation

Required Skills & Experience

2+ years in content review, data labeling, QA, or AI evaluation

Strong analytical and critical thinking skills

Excellent written communication skills

Familiarity with AI evaluation frameworks is preferred

High attention to detail

   

Primary Location

: IN-AP-Hyderabad

Schedule

: Full Time

Employee Status

: Individual Contributor

Job Type

: Experienced

Travel

: No

Job Posting

: 11/02/2026, 9:09:08 AM

Requirements

  • 2+ years in content review, data labeling, QA, or AI evaluation
  • Strong analytical and critical thinking skills
  • Excellent written communication skills
  • Familiarity with AI evaluation frameworks
  • High attention to detail
  • Perform manual evaluation of AI-generated responses
  • Label datasets for training and validation

Benefits

  • Health Insurance
  • Dental Insurance
  • Disability Insurance
  • Employee Assistance Program
  • Employee Discounts
  • Flexible Hours
  • Gym Membership

Skills

AI evaluation frameworks
Data labeling
Quality rubrics
Model relevance
Model correctness
Model safety
Model usability

About AI-Estimated Salary

The salary range shown was not provided by the employer. Our AI has estimated it based on the job title, required experience, location, and industry standards (confidence: 80%). This estimate should be used as a general guide only and may not reflect the actual compensation. Always confirm salary details directly with the employer during the application process.

Ready to Apply?

Join Virtusa today

Salary Range (AI-Estimated)*
$80,000 - $120,000
80% confidence
Posted 4 months ago

More AI Governance & Programs roles you might like

Discover similar opportunities from companies that are also hiring remotely.

Full Time
$7048.161k - $1061.802k
21 hours ago
United States
Worldwide
AI Governance & Programs
Senior
Python
SQL
AI/ML
+4 more
Full Time
$135k - $150k
23 hours ago
Worldwide
AI Governance & Programs
Mid
Python
Machine Learning
LLM
+4 more
Full Time
$120,000 - $180,000*
1 day ago
Worldwide
AI Governance & Programs
Senior
Data governance
AI Policy
Risk Management
+5 more

Explore more remote openings

Browse fresh listings from our global community of remote-friendly teams.

Full Time
$120,000 - $180,000*
23 hours ago
Worldwide
Americas
AI Security & Privacy
Senior
AI
Machine Learning
Cyber Security
+3 more
Full Time
$150,000 - $250,000*
1 day ago
Worldwide
AI Security & Privacy
Senior
OWASP ZAP
Nmap
Postman
+5 more
Full Time
$150k - $200k
1 day ago
Worldwide
AI Governance & Programs
Mid
AI
Python
Clinical AI
+5 more
Full Time
$120,000 - $180,000*
1 day ago
Australia
Worldwide
AI Governance & Programs
Senior
Data governance
AI Ethics
Regulatory Compliance
+3 more
Full Time
$120,000 - $180,000*
1 day ago
Australia
Worldwide
AI Governance & Programs
Senior
Data governance
AI Ethics
Regulatory Compliance
+3 more
Full Time
$85k - $95k
1 day ago
United States
Model Risk Management & Validation
Senior
Model Risk Management
Quantitative Risk Management
Financial Modeling
+4 more
Full Time
$85k - $95k
1 day ago
United States
Model Risk Management & Validation
Senior
Model Risk Management
Quantitative Risk Management
Financial Modeling
+5 more
Full Time
$80,000 - $140,000*
1 day ago
United States
AI Risk & Controls
Mid
Excel
SQL
Python
+1 more
Full Time
$80,000 - $120,000*
1 day ago
United States
Model Risk Management & Validation
Mid
Excel
SQL
Python
+1 more
Full Time
$129k - $175k
2 days ago
Worldwide
AI Audit / Assurance / Controls Testing
Senior
API
Automation
Python
+3 more
Full Time
$129k - $175k
2 days ago
Worldwide
AI Audit / Assurance / Controls Testing
Senior
API
Automation
Python
+3 more
Full Time
$119.7k - $191.1k
2 days ago
Worldwide
AI Governance & Programs
Senior
Risk Management
Model Risk
Governance
+5 more
Full Time
$120,000 - $180,000*
2 days ago
Ireland
Worldwide
AI Compliance & Legal
Senior
Data Protection
AI Compliance
Regulatory Requirements
+3 more
Full Time
$100,000 - $150,000*
2 days ago
Worldwide
AI Governance & Programs
Mid
AI/ML Concepts
Tableau
JIRA
+1 more
Full Time
$204k - $255k
2 days ago
Worldwide
AI Policy, Enablement & Training
Senior
AI
Machine Learning
Policy Development
+4 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Security & Privacy
Staff
Python
ISO 27001
ISO 27701
+4 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Security & Privacy
Staff
Python
ISO 27001
ISO 27701
+4 more
Full Time
$120,000 - $180,000*
2 days ago
Worldwide
AI Security & Privacy
Staff
Python
Adversarial Machine Learning
AI Deployment Architectures
+4 more
Full Time
Up to PHP 150k
2 days ago
Worldwide
AI Security & Privacy
Senior
PyTorch
TensorFlow
Containerized Environments
+4 more
Full Time
Up to PHP 150k
2 days ago
Worldwide
AI Security & Privacy
Senior
PyTorch
TensorFlow
Gradient-based attacks
+4 more
Full Time
$209k - $309k
4 days ago
Worldwide
AI Security & Privacy
Senior
API
AI
Security
+1 more
Full Time
$239.5k - $351.5k
4 days ago
Worldwide
AI Security & Privacy
Senior
API
AI
Security
+1 more
Full Time
$230k - $280k
4 days ago
United States
Worldwide
AI Governance & Programs
Senior
OWASP
NIST AI RMF
AI/ML systems
+5 more
Full Time
$230k - $280k
4 days ago
Worldwide
AI Governance & Programs
Senior
Agentic Trust Framework
OWASP
NIST AI RMF
+5 more
Full Time
$120,000 - $180,000*
5 days ago
Worldwide
AI Security & Privacy
Senior
Python
Go
Git
+5 more
Full Time
$159.3k - $273.2k
5 days ago
Worldwide
AI Governance & Programs
Senior
Python
Machine Learning
Data Science
+5 more
Full Time
$120,000 - $180,000*
5 days ago
Worldwide
AI Security & Privacy
Staff
Python
Go
Threat modeling
+3 more
Full Time
$80,000 - $140,000*
5 days ago
Worldwide
AI Governance & Programs
Mid
Responsible AI
ISO/IEC 42001
ISO/IEC 27001
+2 more
Full Time
$120,000 - $180,000*
6 days ago
United States
Worldwide
AI Governance & Programs
Senior
AI Ethics
Risk Management
AI governance frameworks
+5 more
Full Time
$120,000 - $180,000*
6 days ago
Worldwide
AI Security & Privacy
Senior
Security Operations
Cybersecurity
NG-SIEM
+5 more
Full Time
$163k - $237k
6 days ago
Worldwide
AI Governance & Programs
Senior
API
Product Management
AI
+4 more
Full Time
$80,000 - $140,000*
6 days ago
United States
Worldwide
AI Governance & Programs
Mid
Python
Data Analysis
Financial Data
+3 more
Full Time
$80,000 - $140,000*
6 days ago
United States
Worldwide
AI Governance & Programs
Mid
Python
Data Analysis
Machine Learning
+2 more
Full Time
$80,000 - $140,000*
6 days ago
Worldwide
AI Governance & Programs
Mid
Python
Excel
Google Sheets
+4 more
Full Time
$120,000 - $180,000*
6 days ago
Australia
Worldwide
AI Governance & Programs
Senior
AI
Machine Learning
Data Science
+4 more
Full Time
$120,000 - $180,000*
6 days ago
Worldwide
AI Governance & Programs
Senior
AI Governance
Model Risk Management
Regulatory Compliance
+5 more
Full Time
$120,000 - $180,000*
1 weeks ago
Worldwide
AI Governance & Programs
Senior
Python
ML frameworks
LLM/GenAI tooling
+2 more