About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback in London vacancy
$70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses to... ...Provide structured written feedback to improve training data quality... ...university preferred. Strong analytical and written communication...SuggestedRemote jobSummer workMercor
London14 days ago- £36k - £46k per annumEstimated...Data Analytics – Senior Associate The Chief Data Office (CDO) is... ...in a fast-moving analytics and AI environment. The Enterprise... ...analytics, and help evaluate what is production-ready. Rapid... ...learning through iteration and feedback. Data Pipeline & Integration...SuggestedInternshipOn-siteImmediate start
$60 - $70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses for... ...violations. Provide structured feedback to improve model alignment and... ..., critical thinking, and analytical reasoning skills. ~ Ability...SuggestedRemote jobSummer workMercor
London12 days ago$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against... ...and defensible. Incorporate structured feedback from senior reviewers and iterate on...SuggestedRemote jobSummer workMercor
London8 days ago$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and technical reports. Evaluate AI-generated or human-created work against... ...assessments. Incorporate structured feedback from senior reviewers and iterate on submitted...SuggestedRemote jobSummer workMercor
London9 days ago- £84k - £112k per annumEstimated...through your team, but also raise the overall analytical and technical bar of the organisation,... ...data to shape strategy, prioritise work, and evaluate impact. What You’ll Do Lead, develop... ..., experimentation standards, and AI-enabled tooling to improve efficiency, consistency...Long-term contractFull-timeHybrid working
- £72k - £96k per annumEstimated...Responsibilities Run large-scale evaluations across human UAT batches and automated evaluation sets. Measure factual grounding and... ...outputs, test automation frameworks, and scripting. Strong analytical, problem-solving, communication, collaboration, and prioritization...Full-time
$35 per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...the task 20–60% of the time. Iterate based on feedback from expert QA reviewers. Later: review...Hourly payPermanentTemporaryPart-timeInternship- £54k - £71k per annumEstimated...major airlines) and many more. Our Analytics team is at the heart of supply optimisation... ...communicating trade-offs. Build, evaluate, and iterate on statistical models (e.g.... ...findings. Provide coaching and constructive feedback to junior team members on statistical...
- £60k - £80k per annumEstimated...everywhere. Senior Data Scientist, Analytics The Product Team creates high-... ...learning. Safely integrate and operate AI/ML‑enabled solutions that improve outcomes... ...data and LLMs — including prompt design, evaluation (e.g., golden sets, LLM-as-judge), and RAG...On-siteFlexible hours
$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and defensible. Incorporate structured feedback from senior reviewers and iterate quickly... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobSummer workMercor
London14 days ago$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review... ...involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems...Remote jobSummer workMercor
London15 days ago- £94k - £121k per annumEstimated...volumes of disparate data, advancement in AI technologies and increasing global regulatory... ...The opportunity EY’s Forensic Data Analytics (FDA) team is a growing global, AI and data... ...entirely confidential and will not be used to evaluate your candidacy. We collect this data to...Immediate start
- £61k - £81k per annumEstimated...transformational growth. WPP Media is WPP's AI-driven media operating unit, bringing... ...information, visit wppmedia.com . 2548 - Analytics Director ABOUT WPP & OPENDOOR At... ...reach, attention & other metrics. ~ Evaluate measurement capabilities, ensuring a solid...Full-timeHybrid workingOn-site
$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Software / AI / IT / data Evaluator Type: Contract Compensation... ...outputs. Provide clear, structured written feedback to improve quality. Work independently...Remote jobSummer workMercor
London17 hours ago$50 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar... ...Provide clear, well-reasoned written feedback and ratings to improve AI model outputs...Remote jobSummer workMercor
London9 days ago- £82k - £105k per annumEstimated...Senior Engineering Manager (Data Platform & Analytics Engineering) Location: London, Bristol... ...global team and our sponsorship policy is evaluated on a role-by-role basis. We encourage you... ..., electrified future. Leveraging Data, AI, and real-time decision-making we turn energy...Hybrid workingOn-siteFlexible hoursShift work
- £78k - £104k per annumEstimated...usher in this new era, we seek AI-native thinkers across every... ...to best support them in their evaluation and ultimately help our customers... ...relevant industry experience (analytics, data science, data... ...open to giving and receiving feedback and are passionate about making...Full-time
- £49k - £65k per annumEstimated...Search Analyst to work on data-led analytical elements of our paid search... ...you will help the team adopt AI. This means finding real-world... ...tests and experiments to evaluate the impact of different bidding... ...assumptions, give actionable feedback, and set each other up for success...Full-timeRemoteFlexible hours
$44 per hour
...about linguistics, language quality, and evaluation? We are launching an exciting language evaluation... ...In this role, you will assess and refine AI-generated language outputs, focusing on... ...ability to deliver clear, constructive feedback. This is a unique opportunity to apply...FreelanceImmediate startFlexible hours- £58k - £77k per annumEstimated...change that. At Zego, we're leading the AI evolution in insurance. We move fast,... ...will need to be successful Senior Analytics engineering experience - 5+ years as an Analytics... ...complete ownership and embracing honest feedback that helps us continuously learn and...Full-timeHybrid workingOn-siteShift work
- £64k - £83k per annumEstimated...trends Ideally, you’ll also have SAP analytics experience (SAP BW, HANA, Datasphere or... ...Plus, we offer: Support, coaching and feedback from some of the most engaging colleagues... ...trust in capital markets. Enabled by data, AI and advanced technology, EY teams help...Immediate startFlexible hoursShift work
- ...data science, data analysis and/or data initiatives. This is a genuinely unique role which is centred around leveraging data analytics and AI initiatives to identify issues in early stage. You will use your excellent communication skills to influence stakeholders and cross...PermanentWork from home
- £64k - £85k per annumEstimated...Data Analytics – Principal Consultant Location : London (Hybrid) | Practice Area: Data & Analytics | Type: Permanent Champion innovation... ...talent, guiding clients and driving innovation across Cloud, AI and Machine Learning. You'll champion AI-enabled ways of working,...PermanentHybrid working
£70k - £90k per annum
...Second hire in BCB's data function, reporting to the Head of Data and Analytics, with day-to-day ownership of our data warehouse. This is... ...clearly to people who do not work with data. How we think about AI We are AI-first and will give you the tooling for it....Hybrid workingOn-site£99k - £148.51k per annum
...Senior Quants Analytics Manager | S4 | Risk Division | London | Country: United Kingdom... ...performance assessment, challenger model evaluation and ongoing model improvements. Ensuring... ...and auditors (Required) Experience using AI enabled tools and large language models,...Full-timePart-timeHybrid workingOn-siteFlexible hoursShift work- £57k - £76k per annumEstimated...business decision-making through our leading AI-infused scenario planning and analysis... ...intelligence, or a similar strategic/analytical role in a fast-paced environment. • Strong... ...intelligence or automated tools to assist our team in evaluating qualifications....
$60 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Marketing Expert — Marketing Analytics (AI Agent Environments) Type: Contract... ...them against established standards. Evaluate AI-generated content in Marketing Analytics...Remote jobSummer workMercor
London17 hours ago- £150k - £196k per annumEstimated...of business strategy, digital transformation and state-of-the-art AI? Do you want to take ownership of critical strategic decisions... ...AI Practice? EPAM is looking for a Principal (Director) - Data Analytics & AI Consulting to join our team in London, UK in a hybrid working...Hybrid working
- £36k - £47k per annumEstimated...opportunity, and deliver transformational growth. WPP Media is WPP's AI-driven media operating unit, bringing together media, data, and... ...more information, visit wppmedia.com . 835 - Manager, Client Analytics - DTA About WPP Media WPP is the creative transformation...Hybrid workingOn-site
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!
