Freelance AI Evaluation Engineer (Remote, Technology)

Job summary

This role involves creating and refining coding test cases for AI systems, focusing on evaluating and improving their performance through realistic challenges and comprehensive testing. It is a project-based position suitable for experienced developers and software engineers.

Qualifications

• Degree in Computer Science, Software Engineering, or a related field
• Over 5 years of experience in software development, primarily using Python (including pytest, async/await, subprocess, file operations)
• Background in Full-Stack development, with expertise in both React-based interfaces and robust back-end systems
• Experience in writing functional and integration tests, not just executing them
• Familiarity with Docker containers for local evaluations
• Understanding of CI/CD processes, particularly with GitHub Actions
• English proficiency at B2 level

Responsibilities

Compensation

Contributors can earn up to $50 per hour, depending on their expertise and contribution pace. Compensation varies based on project scope and complexity.

• Review and refine coding tasks based on production codebases, ensuring realistic scope and requirements
• Write functional tests that validate end-to-end behavior and edge cases
• Create challenging coding tasks that require complex reasoning and context
• Analyze AI failures to identify strengths and weaknesses of the model
• Iterate on tasks based on feedback from expert QA reviewers