职位描述
About the Role
Join an expert network supporting AI human-data projects focused on agents and reinforcement learning. You will design training environments and evaluation systems, and assess agent behavior as projects become available.
What You'll Do
- Design reinforcement learning environments, including Python and Docker setups, state spaces, tool integrations, and scoring mechanisms.
- Build robust verifiers and reward functions that resist agent gaming.
- Translate real-world business workflows in areas such as ERP, CRM, finance, and operations into measurable scenarios.
- Integrate tools and REST APIs, and build test harnesses using pytest or similar tools.
- Lead agent demonstration and computer-use programs, setting standards for trace collection quality.
- Grade and repair agent trajectories, and mentor contributors on evaluation practices.
What We're Looking For
- At least 5 years of experience in software engineering, QA automation, machine learning, reinforcement learning, or business systems and operations paired with hands-on agent or RL data work.
- Strong Python skills and experience building tools or services used by others.
- Experience designing environments, verifiers, reward functions, or test harnesses that evaluate software or agent behavior, using tools such as pytest, Playwright, Selenium, or Gymnasium-style environments.
- Hands-on experience with agent or RL data, such as demonstrations, trajectory grading, trace collection, or computer-use demos.
-
Experience with REST APIs and Docker.
- Experience leading or reviewing data, evaluation, or demonstration work is valuable, as is expertise modeling workflows in business systems such as Salesforce or ERP platforms.
- A degree in computer science, engineering, or a related field is relevant.
Compensation & Benefits
Compensation is $80 to $200 per hour, depending on project, field, and level. Pay is based on hours worked.
Location
Remote. Projects typically run from one week to several months.
Compensation Range: $166.4K - $416K
Originally posted on Himalayas