Spanish (Spain) AI Product Evaluator

🎯 The Project

Productive Playhouse is building a talent pool of Spanish (Spain) speakers for an upcoming project testing and evaluating leading AI chatbots. The objective is to enhance response quality through direct user interaction with the various AI models.

Open to freelancers based outside the U.S.

This is more than one job. Apply once, and we’ll keep you in mind for this project — and future ones that need your language skills.

The Details

We’re looking for independent contractor engagement (task-based - project-based).

• Pay Rate: $15.00 USD per hour
• Remote position
• You choose which tasks to take, set your own hours, and are free to work with other clients any time. You can stay flexible as the project evolves.
• This is an iterative project, work arrives in batches (there may be pauses between them).

What You’ll Do

As an AI Evaluator, you’ll play a critical role in shaping and improving the next generation of Generative AI. You’ll participate in structured, hands-on evaluations by interacting directly with various AI models to assess their capabilities, safety, and helpfulness. Your insights and data will directly inform model development and optimization.

Exact tasks vary by project and will be spelled out in that project’s Statement of Work (SOW) before you start. Depending on the role, your work may include:

• Testing and evaluating AI chatbots or language models through structured conversations, using assigned goals and prompts.
• Voice-based interactions with AI models — for projects that involve this, sessions are recorded, and audio may be shared with the client as part of the evaluation deliverable.
• Submitting deliverables (write-ups, ratings, screenshots, or recordings) in the format each task specifies.

✅ What We’re Looking For

• Native or expert-level fluency in Spanish, as spoken in Spain
• Strong English reading, writing, and communication skills - assessment maybe required
• 18 years of age or older
• Hands-on experience using AI models or large language models
• Comfortable with both written and voice-based conversation — and good at asking sharp follow-ups
• Your own smartphone, computer, reliable internet, and comfort with web-based tools
• Able to pass a language proficiency and technical literacy assessment
• Responsive communicator — quick replies move you through onboarding faster
• Able to start within 48 hours

Why PPH? 🌍

🌍 Your language skills are the whole point — and this project pays for them

⚡ Apply once, get contacted directly when a spot opens — no chasing

🏠 Work from anywhere, on your own schedule, alongside any other work you do

🤝 Clear task specs, no ongoing oversight, no micromanagement

💼 Real project experience with a global data and language services company

✨ Work that actually improves how AI understands your language

About Productive Playhouse

We started by teaching kids through award-winning programming. We grew into a global data and language services company trusted by clients worldwide. But the mission never changed: keep language alive.

Transcription, translation, localization, linguistic analysis, AI evaluation — it all comes back to preserving languages and the cultures they carry.

All engagements are contingent upon successful completion of identity verification. Productive Playhouse is an equal opportunity organization headquartered in California and committed to diversity and inclusion across our global workforce. We welcome applicants of all backgrounds and abilities, regardless of location or engagement type. For accommodations or inquiries, please contact [email protected].

Please submit your CV in English and indicate your level of English proficiency.

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

What this opportunity involves We’re building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks.

You’ll create challenging tasks and evaluation criteria within realistic simulated environments:

• Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history
• Design tasks from intermediate states of these environments - craft the prompt, define what “solved” means, and ensure the task is solvable by an AI agent
• Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient
• Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust

What this is NOT

• Not data labeling
• Not prompt engineering
• Not writing code from scratch - the agent writes most of the code; you guide and evaluate

What we look for

• 5+ years in software development
• Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis
• Experience writing tests (functional, integration)
• English proficiency - B2+

Why this is hard

Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds.

How it works

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid

Compensation

Up to $50/hr equivalent, depending on level and pace. Tasks are estimated at ~20 hours each; you set your own schedule.

Role Overview & Mission: Securly On-Call

As a member of the Securly On-Call team (formerly Securly 24), you are the critical human intelligence that powers our safety mission. Reporting to the Director of Student Safety, you play a vital role in reviewing online activity recognized and “flagged” by Securly’s AI as potentially harmful to students. You will work in tandem with our award-winning technology to perform thorough risk assessments, distinguishing between curiosity and crisis, and executing critical communication protocols to alert school safety teams. This is a high-impact, remote-first role requiring a mission-driven mindset to support student well-being in real-time.

• Location: 100% fully Remote (candidates must live in US or UK based)
• Shift Hours: Monday - Friday 7-3 Central US Time (some flexibility possible)

Impact of the Securly On-Call Team:

• Life-Saving Response: Our analysts notify school personnel of extreme risk situations—including potential self-harm, suicide, or violence—within 5 min or less.
• Beyond the AI: You will conduct deep-dive analyses of student activity history to provide schools with the critical context needed for effective, life-saving interventions.
• 24⁄7 Protection: Safety never sleeps. Our team provides a watchful eye whenever schools need us most, whether during the school day, after hours, or 24/7/365.
• See Your Impact: Watch this Playlist of Analyst Testimonials to hear directly from our team about the profound impact this role has on student lives.

About Securly

Securly is the market leader in AI-powered student wellness and safety solutions, protecting over 20M students across 20,000+ schools worldwide. Our technology has analyzed over 10B activities to help keep students safe, secure, and ready to learn. We operate with the scale of a tech giant and the agility of a startup, combining innovation, data, and compassion to solve real problems.

Our Impact & Recognition

• 2,000+ student lives saved.
• Named to the 2026 GSV 150, honoring the world’s top 150 companies driving growth and innovation in digital learning.
• Repeatedly recognized as a Top Place to Work and for EdTech Product of the Year.
• Named to TIME’s World’s Top EdTech Companies of 2026

Our Culture of Engagement

At Securly, culture is our foundation. We are a fully remote, people-first organization built on trust and accountability. Our engagement data far exceeds global benchmarks:

• 82% employee engagement (vs. 73% global).
• 94% of employees are proud to work for Securly (vs. 81% global).
• 91% rate manager effectiveness as high (vs. 77% global).

Key Expectations in First 12 Months

1. High-Impact AI Support: Support Securly’s student safety AI suite by accurately reviewing and categorizing potentially harmful online activity.
2. Safety Protocol Mastery: Execute complex communication processes with the Student Safety Group to proactively prevent harm and promote student well-being.
3. Queue & Metric Excellence: Maintain consistent ownership of the online alert queue, meeting rigorous goals for response time, documentation accuracy, and resolution quality.

Key Development Milestones

• 30 Days: Complete comprehensive training on Securly’s AI safety suite, emergency communication protocols, and internal support software.
• 60 Days: Independently manage the live alert queue while maintaining high accuracy in identifying harmful vs. non-harmful content.
• 90 Days: Master all emergency escalation workflows; independently manage communications with school districts via email and phone.
• 6 Months: Identify recurring student behavior patterns or emerging digital trends and contribute improvements to internal safety documentation.
• 12 Months: Serve as a trusted subject matter expert; consistently meet or exceed expectations and provide insights to evolve safety technology.

Key Responsibilities

• Incident Analysis: Review potentially harmful online activity flagged by AI and decide on appropriate intervention levels.
• Crisis Response & Communication: Manage emergency protocols and communicate with school districts via email or phone to prevent harm.
• Operational Queue Management: Actively monitor the real-time online queue to ensure no critical incident is missed.
• Data & Documentation: Maintain meticulous records in support software, detailing investigative steps and outcomes for all incidents.

Qualifications & Skills

• Applying Domain Expertise: Leverage a background in education, psychology, or law enforcement to accurately distinguish between typical student behavior and high-stakes crisis indicators.
• Executing Under Pressure: Draw upon past crisis intervention or counseling experience to execute immediate, high-pressure communication protocols during extreme-risk incidents.
• Analytical Endurance: Maintain high-level concentration to manage a continuous real-time queue, ensuring all critical alerts are documented and escalated within the required 5 min window.
• Translating Culture: Regularly interpret social media trends and pop culture slang to provide schools with the critical context needed for effective safety interventions.
• Bilingual Intervention: Utilize bilingual proficiency to conduct real-time safety

Why Join Securly

• Direct Mission Impact: Join the GSV 150–recognized team that is the direct “human intelligence” behind student life-saving interventions.
• Industry-Leading Technology: Work with the most innovative, AI-powered safety suite trusted by 20,000+ districts worldwide.
• Remote-First, People-First: Thrive in a trust-based culture with high engagement and 91% manager effectiveness ratings.
• Professional Mastery: Build deep expertise in digital student safety and crisis management within a supportive, mission-driven team.

Wellness & Benefits Overview

• Work-Life Balance: Remote-first work culture with Unlimited Vacation (Flex Time), 13 paid holidays, and a full week of paid holiday break around Christmas and New Year.
• Parental & Family Support: 12 weeks of fully paid parental leave for birth, adoption, or fostering.
• Health & Well-being: Company-sponsored medical, dental, and vision benefits, comprehensive EAP, and unlimited free access to well-being and mental health resources.
• Financial Security: 401(k) plan with employer match and tax-advantaged spending accounts (HSA/FSA).
• Professional Growth: $1,000 annual stipend for professional development to support continuous learning.
• Perks: Exclusive access to the PerkSpend discount platform for savings on travel, electronics, gym memberships, and more. Enjoy a culture recognized as a Top Place to Work for 3+ years running.

Equal Opportunity Employer

Securly is committed to building a diverse and inclusive workplace. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, disability, or any other legally protected characteristic. Accommodations are available throughout the hiring process. Please contact recruitment.us@securly.com.

#LI-REMOTE #LI-DO1

Welo Data, part of Welocalize, is a global AI data company with 500,000+ contributors delivering high-quality, ethical data to train the world’s most advanced AI systems.

Headquarters: 1 Sansome Street, STE 1400 San Francisco, CA 94104 USA
URL: http://www.argosmultilingual.com

We are looking for experienced audio data annotator with US English for our new project! 

• Profile we are looking for: 

  • Detail-obsessed, highly analytical, and possess excellent English comprehension. 
  • Comfortable reading audio waveforms and navigating visual software interfaces. 
  • Equipped with a reliable computer, high-quality headphones, and a stable internet connection. 
  • Ready to learn—no advanced technical background required! 
  • Flexible remote work

   

  Qualifications

  Education, skills, and experience
  • Native-level or near-native English proficiency with exceptional listening comprehension and demonstrated ability to accurately distinguish spoken English at the word and phonetic level. 
  • Experience with audio, transcription, speech data, annotation, or QA. 
  • Strong attention to detail and ability to follow detailed guidelines. 
  • Comfortable working with audio waveforms and annotation tools. 
  • Reliable computer, quality headphones, and stable internet. 

To apply: https://weworkremotely.com/remote-jobs/argos-multilingual-inc-ai-data-annotator-remote

Welo Data is an award-winning localization and data transformation company. We run one of the world’s largest Ads Rating Programs and we want you to join! As an Ads Quality...

Role Title: Avid Media Composer EditorRole Type: ContractorLocation: Remotemicro1 is engaging Avid Media Composer Editors to support a customer's project in high-impact video data annotation and AI model training.

Headquarters: Las Vegas, Nevada
URL: https://jobs.telusdigital.com/search/jobs?cfm5=Artificial+Intelligence&ns_category=artificial-intelligence

We are looking for an independent, flexible, remote opportunity where you can help improve AI-powered search technology from the comfort of your home? If you're curious, internet-savvy, and enjoy evaluating online content, this freelance project could be a great fit.

 

A Day in the Life of a Content Reviewer - US

In this role, you will analyze and provide feedback on text, webpages, images, and other types of online content for leading search engines using a specialized online platform.

 

By reviewing and rating search results for relevance and quality, you will help improve the overall search experience for millions of users around the world, including yourself.

Join our global community and put your skills to work supporting one of the world's leading search technologies.

 

Service Rates 

The rate of pay is $0.2333 per completed task, with an estimated earning potential of $14 per hour. Compensation is based on tasks completed and project availability. Estimated earnings may vary depending on task volume and program requirements including, quality, and productivity, in accordance with the program's quality standards and guidelines.

 

Please note that only one member per household may participate in this program. If it is identified that more than one person from the same household is participating in the TELUS Digital Rating Program, all associated accounts may be removed from the program.

 

Independent Contractor Relationship

This opportunity is offered on an independent contractor basis. Contributors have the flexibility to choose when and how much they work, subject to project availability and quality requirements. This opportunity is not and should not be construed as creating an employment relationship with TELUS Digital.

 

Qualification Process

No previous professional experience is required to apply for this opportunity. However, participation in this project requires meeting the basic requirements and successfully completing the qualification process.

 

Basic Requirements

• Excellent written and verbal communication skills in English

• Must have resided in the United States for the past three consecutive years

• Familiarity with current and historical business, media, sports, news, social media, and cultural affairs in the United States

• Active use of Gmail and social media platforms

• Experience using web browsers to navigate and interact with a variety of online content

• Daily access to a reliable broadband internet connection

• Access to a smartphone (Android 5.0 or higher, or iOS 14 or higher)

• Access to a personal computer

 

Assessment

You must complete an English language assessment, pass an open book qualification exam, and complete an ID verification in order to be successful for this role. These assessments will determine your suitability for the position.

To apply: https://weworkremotely.com/remote-jobs/telus-digital-content-reviewer-us-5

Headquarters: Atlanta, AZ.
URL: https://trainbrain.space/

What if the work you already do every day — shipping code, closing books, reading charts — became the training signal that teaches frontier AI how to think in your field?

TrainBrain builds expert-authored training data and evaluations for AI labs. We're hiring practising professionals to write, solve, and grade the tasks advanced models are measured against. When a model returns a broken function, a wrong reconciliation, or an unsafe clinical suggestion, it's because no expert caught it during training. You'd be that expert.

This isn't your day job in a new wrapper. There are no clients, no month-end close, no tickets, no on-call. You work in writing, on your own schedule, on problems chosen to sit right at the edge of what current models can handle.

No AI background required — we train you on the tooling.

Track 1 — Software Engineering

Write, debug, and optimise code across languages and problem domains. Review AI-generated solutions for correctness, security, and performance, and design edge cases that expose model weaknesses. Provide structured feedback explaining why code works — or doesn't — and compare competing solutions on engineering quality.

You'll need:

• Advanced proficiency in Python, JavaScript/TypeScript, Java, C/C++, Go, or Rust
• A solid grasp of data structures, algorithms, and software design principles
• A CS degree or equivalent professional experience

Nice to have:

• Open-source contributions or competitive programming
• Distributed systems, cloud, or DevOps background
• Code review, technical writing, or mentoring experience

Track 2 — Finance & Accounting

Author reconciliation scenarios: payment-to-invoice matching, billing versus recognised revenue, margin and variance analysis. Build the underlying data and source documents so each scenario is realistic, write rubrics specifying exact figures and required reasoning steps, then solve every task yourself to confirm the numbers hold.

You'll need:

• Hands-on experience as an accountant, financial analyst, controller, or auditor
• Working knowledge of revenue recognition, matching principles, and standard close workflows
• Comfort with financial documents, spreadsheets, and accounting system outputs

Nice to have:

• CPA, CA, ACCA, CIMA, or CFA
• IFRS or US GAAP depth
• ERP familiarity (SAP, Oracle, NetSuite, QuickBooks)
• Audit, financial controls, or revenue assurance background

Track 3 — Medical & Clinical

Challenge models on differential diagnosis, drug interactions, treatment protocols, pathophysiology, and evidence appraisal. Write clinical vignettes with defensible, sourced answers, verify model outputs against current evidence, and document precisely where reasoning breaks down.

You'll need:

• An MD, DO, MBBS, PharmD, RN/BSN, or advanced health sciences degree
• Real clinical or research experience
• Command of medical terminology and clinical reasoning

Nice to have:

• Board certification or active licensure
• A specialty focus (internal medicine, oncology, psychiatry, emergency medicine, radiology, nursing practice)
• Peer-reviewed publications
• Clinical trial, epidemiology, or medical education experience

What the work demands

Whatever your track, four things matter more than anything else.

Show your work. Making your reasoning explicit matters as much as the answer itself.

Be verifiable. Every task you write needs a correct answer someone else can confirm.

Iterate. You'll refine tasks until difficulty and gradability are both right.

Flag uncertainty. Saying "I'm not sure, and here's why" is a feature, not a failure.

You'll also need excellent written English, sharp attention to detail, and a secure computer with reliable internet.

Terms

This is a 1099 independent contractor position, not W-2 employment. You're responsible for your own taxes, and company-sponsored benefits such as health insurance, PTO, and retirement contributions don't apply. Contractors outside the US are engaged under the equivalent local arrangement.

Pay: $45–$100/hr, set by domain, seniority, and assessment performance. Specialised and hard-to-source expertise sits at the top of the band, and you're paid on a regular cadence.

Schedule: Choose your own projects, hours, and volume. Scale up in quiet weeks, scale down when your day job gets busy.

Growth: Strong contributors are invited into higher-rate specialist projects and review roles, with ongoing work as new engagements launch.

To apply: https://weworkremotely.com/remote-jobs/trainbrainspace-expert-ai-trainers-in-coding-finance-accounting-and-medicine-nursing

Are you a Portuguese (Brazil) language expert eager to shape the future of AI?

Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.

Mindrift, powered by Toloka — a leading enterprise AI and machine learning data partner since 2014 — connects top domain experts with cutting-edge AI initiatives. Backed by Toloka’s deep expertise in scalable data generation, crowd technology, and applied ML systems, Mindrift enables experts to shape how next-generation generative models learn, reason, and perform.

We are launching a Management Consulting domain focused on translating real-world consulting engagements into structured learning environments for advanced AI systems. To do this credibly, we are assembling a team of strategy consultants from top-tier firms who can convert authentic project experience into end-to-end examples — from problem structuring and work planning to analysis, synthesis, and client-ready recommendations.

You will join a growing team of consultants from leading strategy firms shaping how AI learns high-level business reasoning.

Important: This role is exclusively for consultants with direct experience at a top-tier strategy consulting firm. If you do not have hands-on project experience at one of the firms listed below, please do not apply. This requirement ensures the domain is built by practitioners trained to the highest standards of structured problem-solving and client delivery.

Eligible firms: McKinsey & Company, Boston Consulting Group (BCG), Bain & Company, Oliver Wyman, Roland Berger, Monitor Deloitte (Deloitte S&C), EY-Parthenon, Kearney, and Strategy& (PwC).

Who We’re Looking For

Consultants with 3+ years of experience at one of the firms listed above, with hands-on project experience in:

• Structuring ambiguous client problems into workable analytical plans
• Building financial models, market analyses, or synthesized findings from messy inputs
• Producing client-ready deliverables under time pressure
• Forming and defending recommendations under uncertainty

No deep technical background is required — we will onboard you on the lightweight tools involved.

What You’ll Do

• Build realistic consulting project environments — create detailed project scenarios grounded in real engagement dynamics: industry context, financials, constraints, conflicting inputs, and incomplete information.
• Design structured consulting tasks for AI agents— break projects into discrete tasks that mirror real consulting work: market sizing, commercial due diligence, cost optimization, growth strategy, operational diagnosis, benchmarking, and more.
• Define evaluation criteria and quality standards — develop grading frameworks, evaluation rubrics, and golden-answer solutions for each task, used to train and calibrate an LLM-based grading system that evaluates AI outputs at scale.

This is a remote, project-based, individual-contributor role focused on analytical design and evaluation.

Skills & Requirements

• 3+ years at McKinsey, BCG, Bain, Oliver Wyman, Roland Berger, Monitor Deloitte, EY-Parthenon, Kearney, or Strategy&
• Strong structured problem-solving and hypothesis-driven thinking
• Ability to translate vague problems into clear analytical steps and deliverables
• High attention to logical consistency and output quality
• Independent, self-directed working style
• Clear written English (B2+)

Compensation

On this project, contributors can earn up to $60 per hour equivalent, depending on their level and pace of contribution.

Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements.

For this project, tasks are estimated to require around 25-30 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.

Mindrift, powered by Toloka — a leading enterprise AI and machine learning data partner since 2014 — connects top domain experts with cutting-edge AI initiatives. Backed by Toloka’s deep expertise in scalable data generation, crowd technology, and applied ML systems, Mindrift enables experts to shape how next-generation generative models learn, reason, and perform.

We are launching a Management Consulting domain focused on translating real-world consulting engagements into structured learning environments for advanced AI systems. To do this credibly, we are assembling a team of strategy consultants from top-tier firms who can convert authentic project experience into end-to-end examples — from problem structuring and work planning to analysis, synthesis, and client-ready recommendations.

You will join a growing team of consultants from leading strategy firms shaping how AI learns high-level business reasoning.

Important: This role is exclusively for consultants with direct experience at a top-tier strategy consulting firm. If you do not have hands-on project experience at one of the firms listed below, please do not apply. This requirement ensures the domain is built by practitioners trained to the highest standards of structured problem-solving and client delivery.

Eligible firms: McKinsey & Company, Boston Consulting Group (BCG), Bain & Company, Oliver Wyman, Roland Berger, Monitor Deloitte (Deloitte S&C), EY-Parthenon, Kearney, and Strategy& (PwC).

Who We’re Looking For

Consultants with 3+ years of experience at one of the firms listed above, with hands-on project experience in:

• Structuring ambiguous client problems into workable analytical plans
• Building financial models, market analyses, or synthesized findings from messy inputs
• Producing client-ready deliverables under time pressure
• Forming and defending recommendations under uncertainty

No deep technical background is required — we will onboard you on the lightweight tools involved.

What You’ll Do

• Build realistic consulting project environments — create detailed project scenarios grounded in real engagement dynamics: industry context, financials, constraints, conflicting inputs, and incomplete information.
• Design structured consulting tasks for AI agents— break projects into discrete tasks that mirror real consulting work: market sizing, commercial due diligence, cost optimization, growth strategy, operational diagnosis, benchmarking, and more.
• Define evaluation criteria and quality standards — develop grading frameworks, evaluation rubrics, and golden-answer solutions for each task, used to train and calibrate an LLM-based grading system that evaluates AI outputs at scale.

This is a remote, project-based, individual-contributor role focused on analytical design and evaluation.

Skills & Requirements

• 3+ years at McKinsey, BCG, Bain, Oliver Wyman, Roland Berger, Monitor Deloitte, EY-Parthenon, Kearney, or Strategy&
• Strong structured problem-solving and hypothesis-driven thinking
• Ability to translate vague problems into clear analytical steps and deliverables
• High attention to logical consistency and output quality
• Independent, self-directed working style
• Clear written English (B2+)

Compensation

On this project, contributors can earn up to $60 per hour equivalent, depending on their level and pace of contribution.

Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements.

For this project, tasks are estimated to require around 25-30 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.

Mindrift, powered by Toloka — a leading enterprise AI and machine learning data partner since 2014 — connects top domain experts with cutting-edge AI initiatives. Backed by Toloka’s deep expertise in scalable data generation, crowd technology, and applied ML systems, Mindrift enables experts to shape how next-generation generative models learn, reason, and perform.

We are launching a Management Consulting domain focused on translating real-world consulting engagements into structured learning environments for advanced AI systems. To do this credibly, we are assembling a team of strategy consultants from top-tier firms who can convert authentic project experience into end-to-end examples — from problem structuring and work planning to analysis, synthesis, and client-ready recommendations.

You will join a growing team of consultants from leading strategy firms shaping how AI learns high-level business reasoning.

Important: This role is exclusively for consultants with direct experience at a top-tier strategy consulting firm. If you do not have hands-on project experience at one of the firms listed below, please do not apply. This requirement ensures the domain is built by practitioners trained to the highest standards of structured problem-solving and client delivery.

Eligible firms: McKinsey & Company, Boston Consulting Group (BCG), Bain & Company, Oliver Wyman, Roland Berger, Monitor Deloitte (Deloitte S&C), EY-Parthenon, Kearney, and Strategy& (PwC).

Who We’re Looking For

Consultants with 3+ years of experience at one of the firms listed above, with hands-on project experience in:

• Structuring ambiguous client problems into workable analytical plans
• Building financial models, market analyses, or synthesized findings from messy inputs
• Producing client-ready deliverables under time pressure
• Forming and defending recommendations under uncertainty

No deep technical background is required — we will onboard you on the lightweight tools involved.

What You’ll Do

• Build realistic consulting project environments — create detailed project scenarios grounded in real engagement dynamics: industry context, financials, constraints, conflicting inputs, and incomplete information.
• Design structured consulting tasks for AI agents— break projects into discrete tasks that mirror real consulting work: market sizing, commercial due diligence, cost optimization, growth strategy, operational diagnosis, benchmarking, and more.
• Define evaluation criteria and quality standards — develop grading frameworks, evaluation rubrics, and golden-answer solutions for each task, used to train and calibrate an LLM-based grading system that evaluates AI outputs at scale.

This is a remote, project-based, individual-contributor role focused on analytical design and evaluation.

Skills & Requirements

• 3+ years at McKinsey, BCG, Bain, Oliver Wyman, Roland Berger, Monitor Deloitte, EY-Parthenon, Kearney, or Strategy&
• Strong structured problem-solving and hypothesis-driven thinking
• Ability to translate vague problems into clear analytical steps and deliverables
• High attention to logical consistency and output quality
• Independent, self-directed working style
• Clear written English (B2+)

Compensation

On this project, contributors can earn up to $60 per hour equivalent, depending on their level and pace of contribution.

Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements.

For this project, tasks are estimated to require around 25-30 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

ABOUT PERLE Perle is an AI infrastructure company building expert-driven training data, evaluation systems, and applied AI products for the world's leading labs and enterprises.

ABOUT PERLE Perle is an AI infrastructure company building expert-driven training data, evaluation systems, and applied AI products for the world's leading labs and enterprises.

ABOUT PERLE Perle is an AI infrastructure company building expert-driven training data, evaluation systems, and applied AI products for the world's leading labs and enterprises.

Hindi AI Product Tester

Active Project Status: This project is now live, and we are urgently onboarding participants to begin immediately. Because we are moving quickly, we kindly ask that you keep a close eye on your communication (email and messages) after applying. It is essential that you respond promptly so we can get you cleared and ready to start without delay.

Location: 100% Remote, global

Device: Applicants must own a qualifying Google Pixel device for the duration of the project

Compensation:USD $140.00 upon successful completion of the 14 qualified submissions (each submission should take 10–15 minutes of activity per day)

Project Requirements

• Native or expert-level fluency in Hindi
• Comfortable reading and writing in English
• Must complete a language proficiency assessment as part of the application process
• Must be comfortable using a Google Pixel 9 or newer device
• Must be comfortable downloading and using common mobile applications that generate notifications (messaging apps, email apps, social media apps, etc.)
• Must be comfortable using the Google Pixel device as their primary phone for the duration of the project
• Must be willing to participate daily for 14-21 consecutive days
• Must be 18 or older to meet project eligibility requirements
• Must have access to reliable Wi-Fi/internet
• Participants must be able to read and understand instructions and company communications in English

The Project

We are seeking detail-oriented and motivated speakers to participate in an upcoming notification sorting project.

Participants will use a Google Pixel 9 or newer device throughout the project period while interacting naturally with apps that generate notifications. Participants will complete daily project tasks involving notification organization/sorting workflows and provide related feedback/documentation as instructed.

Participants will also complete a short daily feedback/check-in form throughout the duration of the pilot to assist with participation tracking and troubleshooting support.

This project is part of an ongoing effort to improve emerging mobile AI and notification-management technologies for international users.

Workload

• Participants will engage with the project daily until 14 qualified submissions have been submitted.
• Expected daily participation time may vary slightly depending on notification volume and project tasks, but participants should expect approximately 10–15 minutes of activity per day.
• This is project-based work, and there is no guarantee of additional work beyond the pilot period. However, participants who perform well may be considered for future related projects.

Duties

• Use a Google Pixel 9 or newer device throughout the project duration
• Interact naturally with apps that generate notifications
• Complete assigned notification sorting / organization tasks
• Submit required daily feedback and documentation
• Follow all provided project instructions carefully
• Communicate promptly regarding technical issues or project blockers
• Perform all work with attention to quality, completeness, and detail

Engagement Requirements

All participants must have, or be willing to create, an Upwork account. The project will be managed via the Upwork platform.

While participating in this project, adherence to the confidentiality terms outlined in the Upwork User Agreement is required. Any information accessed or received related to this project is confidential and may not be shared or disclosed to third parties.

Participants must:

Use their own qualifying Google Pixel device.

Qualifying Device Examples

Google Pixel 9

Google Pixel 9 Pro

Google Pixel 10

Google Pixel Fold (newer generation)

Please note:Google Pixel 9a and Pixel 10a devices are not compatible with project requirements.

About Us:

As a global data company, Productive Playhouse “PPH”, is pioneering our approach to language and data services, while incorporating their roots as a production company. Originally creating content to support children’s language acquisition, our commitment to excellence, forward-thinking strategies, and world-wide cultural experience has proven key for delivering exceptional service.

Originally founded as an educational production company, Productive Playhouse made a mark with our award winning children’s series, which taught fundamental subjects through engaging and effective programming. This early success paved the way for our evolution into a comprehensive data services provider.

Since 2011, Productive Playhouse has expanded rapidly to offer an extensive suite of data services. Our current offerings include transcription, translation, linguistic analysis, rating, systems testing, localization, field and studio recording, language skill verification, and specialized data handling with a focus on sensitivity and diversity.  Our commitment to innovation means we continually enhance our service portfolio to meet the evolving needs of our clients.

At Productive Playhouse, we are proud of our reputation for addressing complex challenges with agility and delivering premium, secure data solutions across diverse environments. Our dynamic team is dedicated to maintaining the highest standards and ensuring exceptional service every time.

Disclaimer:

All engagements are contingent upon successful completion of a background screening. Productive Playhouse is committed to diversity and inclusion across our global contractor network. We welcome applicants of all backgrounds and abilities.

About UsUdacity is now an Accenture company, and exciting things are happening!

Are you experienced in garment manufacturing quality control and interested in helping train the next generation of AI systems? As AI models are increasingly applied to global manufacturing, supply chain...

About the jobMercor connects elite creative and technical talent with leading AI research labs.