Job summary
This role involves creating and refining coding test cases for AI systems, focusing on testing and evaluating their performance. The position is project-based and allows for flexible working hours.
Qualifications
- Degree in Computer Science, Software Engineering, or a related field
- Over 5 years of experience in software development, with a strong emphasis on Python (pytest, async/await, subprocess, file operations)
- Background in Full-Stack development, particularly with React for front-end and robust back-end systems
- Experience in writing various types of tests, including functional and integration tests
- Familiarity with Docker for local evaluations
- Understanding of CI/CD processes, specifically with GitHub Actions
- Proficient in English (B2 level)
Responsibilities
Compensation
Contributors can earn up to $50 per hour, with compensation varying based on project scope and complexity. Tasks are estimated to take around 20 hours to complete, with flexibility in scheduling.
- Review and enhance realistic coding tasks based on production codebases
- Develop comprehensive functional tests that validate end-to-end behavior and edge cases
- Create challenging coding tasks that require complex reasoning and information synthesis
- Analyze AI failures to identify strengths and weaknesses of the model
- Iterate on tasks based on feedback from quality assurance reviewers