LLM Evaluator – Reasoning Analysis, Accuracy & Content Quality | Remote
Position: Generalist Expert Type: Hourly contract Compensation: $60 $200/hour Location: Remote Commitment: Part-time or full-time, flexible based on project needs Role Responsibilities
- Evaluate AI-generated responses for accuracy, reasoning, and clarity
- Provide structured, detailed written feedback on model outputs
- Identify gaps in logic, implicit assumptions, and reasoning flaws
- Apply evaluation guidelines consistently across tasks
- Write clear, evidence-based rationales beyond surface-level observations
- Ensure high-quality, unbiased, and precise assessments
Requirements
- Bachelor s degree preferred from a top-ranked university
- Strong analytical thinking and critical reading skills
- Strong written communication with clear and precise reasoning
- Ability to identify nuanced issues in complex responses
- Strong attention to detail and consistency in evaluations
- Ability to work independently and follow structured guidelines
- Native-level English proficiency
- Ability to complete tasks without using AI writing tools
Application Process (Takes 20 Mins)
- Upload your resume.
- Complete an interview.
- Submit a short form.