Freelance AI Evaluation Engineer (Remote, Tech Industry)

Job summary

This role involves creating and evaluating coding test cases for AI systems, focusing on enhancing their performance through rigorous testing and analysis. The position is project-based and allows for flexible working hours.

Qualifications

• Degree in Computer Science, Software Engineering, or a related field
• Over 5 years of experience in software development, primarily using Python (pytest, async/await, subprocess, file operations)
• Strong background in Full-Stack development, with experience in both React-based interfaces and robust back-end systems
• Proven experience in writing functional and integration tests
• Familiarity with Docker containers for local evaluations
• Understanding of CI/CD processes, particularly with GitHub Actions
• English proficiency at B2 level

Responsibilities

Compensation

Contributors can earn up to $50 per hour, depending on their expertise and contribution pace. Compensation may vary across different projects based on their specific requirements.

• Review and refine coding tasks based on production codebases, ensuring realistic scope and requirements
• Write comprehensive functional tests that validate end-to-end behavior and edge cases
• Create challenging coding tasks that require complex reasoning and information gathering
• Analyze AI failures to identify strengths and weaknesses of the model
• Iterate on test cases based on feedback from quality assurance reviewers