Freelance AI Evaluation Engineer (Remote, Tech)

Job summary

This role involves creating and evaluating coding test cases for AI systems, focusing on testing and improving their performance through realistic challenges and comprehensive functional tests.

Qualifications

• Degree in Computer Science, Software Engineering, or a related field
• Over 5 years of experience in software development, primarily using Python (pytest, async/await, subprocess, file operations)
• Background in Full-Stack development, with experience in both React-based interfaces and robust Back-end systems
• Proficient in writing tests (functional, integration)
• Familiarity with Docker containers for local evaluations
• Understanding of CI/CD processes, particularly with GitHub Actions
• English proficiency at B2 level

Responsibilities

Compensation

Contributors can earn up to $50 per hour, depending on their expertise and contribution pace. Compensation may vary across different projects based on their scope and complexity.

• Review and refine coding tasks based on production codebases
• Write functional tests that validate end-to-end behavior and edge cases
• Create challenging coding scenarios that require complex reasoning
• Analyze AI failures to identify strengths and weaknesses
• Iterate on tasks based on feedback from quality assurance reviewers