DevOps Engineer - AI Model Evaluator

**About The Job** **Mercor** connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include **Benchmark** , **General Catalyst** , **Peter Thiel** , **Adam D'Angelo** , **Larry Summers** , and **Jack Dorsey** . **Position:** DevOps / SRE / Cloud Engineer (Coding Agent Experience) **Type:** **Contract** **Compensation:** **$85/hour** **Location:** **Remote** **Role Responsibilities** - Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. - Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, and infrastructure automation. - Identify bugs, edge cases, reliability issues, and failure modes in model outputs. - Compare outputs from multiple frontier models to assess strengths and weaknesses. - Apply professional engineering judgment to realistic infrastructure engineering scenarios. **Qualifications** **Must-Have** - 2+ years of professional DevOps, SRE, or Cloud Engineering experience. - Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling. - Regular use of AI coding agents like Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools. - Ability to evaluate model-generated infrastructure and reliability engineering solutions. **Preferred** - Experience supporting production-scale systems. **Compensation & Legal** - $400 per accepted task. - Compensation is tied to accepted work. **Application Process (Takes 20–30 mins to complete)** - Upload resume - AI interview based on your resume - Submit form **Resources & Support** - For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome - For any help or support, reach out to: support@mercor.com *PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.* , Puna remote: Po Burimi: https://mk.linkedin.com/jobs/view/devops-engineer-ai-model-evaluator-at-mercor-4431365263

Niveli: mid level