Evaluate and refine AI-generated code for efficiency and reliability.
About Turing
Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing supports customers in two ways: first, by accelerating frontier research with high-quality data, advanced training pipelines, plus top AI researchers who specialize in coding, reasoning, STEM, multilinguality, multimodality, and agents; and second, by applying that expertise to help enterprises transform AI from proof of concept into proprietary intelligence with systems that perform reliably, deliver measurable impact, and drive lasting results on the P&L
Role Overview
As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. This includes curating code examples, providing precise solutions, and making corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; evaluating and refining AI-generated code for efficiency, scalability, and reliability; and working with cross-functional teams to enhance enterprise-level AI-driven coding solutions.
What does day-to-day life look like?
- Working on AI model training initiatives by curating code examples, building solutions, and correcting code in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go.
- Evaluate and refine AI-generated code to ensure that it is efficient, scalable, and reliable.
- Collaborate with cross-functional teams to enhance AI-driven coding solutions against industry performance benchmarks.
- Build agents that can verify the quality of the code and identify error patterns.
- Hypothetize on steps in the software engineering cycle (prototyping, architecture design, API design, production implementation, launch, experiments, monitoring, operation maintenance) and evaluate model capabilities on them
- Design verification mechanisms that can automatically verify a solution to a software engineering task.
Requirements
- Several years of software engineering experience, including 2+ years of continuous full-time experience at a top-tier product company (e.g., Google, Stripe, Amazon, Apple, Meta, Netflix, Microsoft, Datadog, Dropbox, Shopify, PayPal, IBM Research).
- Strong expertise in building full-stack applications and deploying scalable, production-grade software using modern languages and tools.
- Deep understanding of software architecture, design, development, debugging, and code quality/review assessment.
- Excellent oral and written communication skills for clear, structured evaluation rationales.
Offer Details
- Commitment : flexible engagement, minimum 10 hrs/week, up to 40 hrs/week (partial PST overlap required)
- Type : Contractor (no medical/paid leave)
- Duration :1 month (starting next week; potential extensions based on performance and fit)
Skills: JavaScript, JavaScript, Python
Vacancy posted 12 days ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer - LLM Evaluation in Remote vacancy
- ...Analyze GitHub issues and evaluate LLM performance on software tasks. Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering...SeniorPart-timeFreelanceRemote
Turing
Remote12 days ago $50 - $100 per hour
...About the Role We are looking for software engineers to validate real-world repositories used to evaluate large language models. You will triage issues, set up reproducible... ...setting. Preferred Qualifications Prior LLM research or evaluation projects. Experience...SuggestedHourly payRemoteRemoExperts
Remote12 days ago- ...with frontier AI labs to accelerate model development through high-quality training data, evaluations, and engineering talent. About the Role We are looking for senior software engineers with 7+ years of industry experience to support the training and evaluation of large...SeniorPart-time
Turing
Remote12 days ago - ...open-source codebases for AI evaluation projects. About Turing... ...training data, evaluations, and engineering talent. About the Role... ...are looking for experienced software engineers with strong technical... ...overall test quality. Assess LLM performance on bug-fixing and...Senior40 hours/weekPart-time
Turing
Remote12 days ago $100 - $200 per hour
...for more details: We’re looking for highly experienced software engineer (SR+) to help evaluate the quality of interactions with modern coding agents... ...judgments, and explaining them clearly. WHO YOU ARE • Senior, Staff or Principal-level engineer (or equivalent...SeniorFull-timeRemoteFlexible hours$113.08k - $175.73k per annum
...Senior Software Engineer, MediaWiki Summary The Wikimedia Foundation is looking for a Senior Software Engineer to join the MediaWiki Platform... ...measurable outcomes that guide our decisions and help us to evaluate success; Leading by example in code review, decision-...SeniorPermanentFull-timeRemote- ...data lab for training frontier models and evaluating AI agents. Experts contribute their... ...domains such as finance, healthcare, STEM engineering, and more. micro1 transforms that real-world... ...success in identifying and resolving software bugs and system bottlenecks. Experience...SeniorPart-timeRemoteFlexible hours
micro1
Remote13 days ago £85k - £105k per annum
...business is an operational puzzle most software has never touched. Shift-by-shift... .... About the Role We're hiring Senior Software Engineers to join our product teams and help build... ...your primary workflow - directing and evaluating agent-produced code, not writing everything...SeniorRemote jobFull-timeFlexible hoursShift workRotating shifts- ...talented developers proficient Java Engineer. The engineer will be working... ...AI models. Conduct evaluations (Evals) to benchmark model performance... ...Required Skills: Train LLM models with high-quality,... ...Nice to have some prior software Quality Assurance and Test Planning...Part-timeFreelanceRemote
Turing
Remote12 days ago - £65k - £84k per annumEstimated...transformation into an AI-native company. As a Senior Backend Engineer for internal AI, you will own the... ...reliability. You have built AI- or LLM-powered applications for production use... ..., including model integration, evaluation, observability, and responding when systems...SeniorFull-timeOn-siteRemote
- ...quality Python code for AI model training and evaluation. About Turing: Turing is one of... ...a project with one of the foundational LLM companies. The goal is to assist these... ...quality, formatting, and best practices of software development ~ Experience with Python’...SeniorPart-timeFreelanceRemote
Turing
Remote12 days ago - ...Senior Software Engineer - (Agent Training and Evals) Engineering Prolific Prolific is not just another player in the AI space... ...Training & Evals team. You'll build infrastructure to evaluate AI agents at scale against real human expertise, in safe and...SeniorPart-timeRemote
Prolific
Remote12 days ago - £69k - £90k per annumEstimated...applications and next steps. Our partner is looking for a Senior Software Engineer (Golang/TypeScript/React) - Independent Contractor based in... ...improvement in production. - Contribute to technical strategy by evaluating architectural options, balancing existing systems with new...SeniorFull-timeRemote
- £45k - £59k per annumEstimated...ABOUT THE ROLE We’re looking for a Lead AI Engineer to own the backend and Agent systems behind our LLM-powered products. The role is end to end: you’ll build... ...RAG), state management, and human handoff. Drive evaluation and continuous improvement – Define success...Full-timeRemoteFlexible hours
- ...accelerate model development through high-quality training data, evaluations, and engineering talent. Engagement Details Compensation: Market,... ...detection, time-series baselines, and forecasting. LLM/agent engineering: experience with tool calling, multi-step...SeniorHourly pay40 hours/weekPart-time
Turing
Remote12 days ago £60k - £80k per annum
...old see the new. Java is a language that goes back as far as software engineering itself. One of the originators and still remains one of the... ...commonly used today. I am offering an opportunity for the role of Senior Java developer to come and take charge of a team of 8 mostly...SeniorRemote jobPermanent£70k - £80k per annum
...while enjoying substantial financial backing from the larger enterprise. This organisation is currently in search of a skilled Senior Software Engineer in Test to spearhead streamlined testing and automation strategies. In this role, you'll be a linchpin in the team, taking...SeniorRemote jobOdd jobPermanentOn-site- £65k - £86k per annumEstimated...hardened, secure, and production-ready builds of all the open source software engineers and AI agents rely on, Chainguard helps organizations build... ...Spark Capital. The role, in a nutshell We’re looking for a Senior Software Engineer to join our Platform team, which owns the...SeniorFull-timeRemote
£70k - £85k per annum
...’re looking for that role that will allow you to make a real impact. This is your opportunity to join a global leader as a Senior Software Engineer in Test and not just keep up with market growth, but lead the way while fostering a lasting cultural shift. The tech stack...SeniorRemote jobPermanentHybrid workingShift work- £69k - £90k per annumEstimated...and platform quality, enabling teams across the company to ship quickly and safely on top of Canvas Core. YOUR MISSION As a Senior Software Engineer, you’ll help design, build, and improve the Canvas platform so that the Mural editor remains reliable, high-performing, and intuitive...SeniorLong-term contractFull-timeRemote
- ...We are hiring skilled software engineers with strong full-stack experience. The ideal candidate is comfortable architecting and working across multi-file codebases, structuring realistic development environments, and building reproducible setups supported by clear documentation...
AfterQuery
Remote12 days ago - £54k - £72k per annumEstimated...world, Mozillians design, build and distribute open-source software that enables people to enjoy the internet on their terms. About... ...of Reviewer tools and automated moderation systems. As a Senior Software Engineer, you’ll bring a passion for crafting resilient systems that...SeniorFull-time
£70k - £80k per annum
...within a tight-knit team and the robust financial support of a larger corporation. The team is on the lookout for a seasoned Senior Software Engineer to spearhead the development of scalable, reliable, and efficient software solutions. Collaborating closely with cross-...SeniorRemote jobPermanentOn-site- £69k - £90k per annumEstimated...listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Founding Senior Software Engineer based in United Kingdom. Join a fast-growing technology startup building an AI-powered platform that transforms how organizations...SeniorLong-term contractFull-timeImmediate start
- £55k - £71k per annumEstimated...threats. About The Role We are looking for a Software Engineer to help build and evolve our platform as... ...agentic components, working closely with senior engineers to improve system reliability,... ...& Agents - Help design and build LLM-powered features and agentic workflows (e...Full-time
- £82k - £106k per annumEstimated...vendor data goes in, and clean, trustworthy records come out. We’re looking for the engineer who will own that transformation. At DistantJob, we’re searching for a Senior Full-Stack Software Engineer for one of our U.S.-based clients. They’re building a platform that...SeniorLong-term contractFull-timeRemote5 days/week
$156.88k - $220.52k per annum
...day. About MetaMask MetaMask aims to create a thriving engineering organization that supports the well-being of our engineers... ...Prediction Markets directly into the MetaMask platform. As a Senior Software Engineer on this team, you will be instrumental in building the...SeniorFull-timeRemoteShift work- £67k - £87k per annumEstimated...How you will make an impact: As a Senior Solutions Engineer at Hygraph, you’ll be the trusted technical... ...and content leaders at organizations evaluating Hygraph. This is a US-based role... ...is a customer-facing role built on a software engineering foundation, not a software...SeniorFull-timeHybrid workingRemote
£91.76k - £110.11k per annum
...opportunity. This is a remote opportunity and we would be interested in applicants living in UK time zones only at this time. Senior Software Engineer - Grafana Databases, Managed Services The Opportunity: The Managed Services team is a newly formed squad within the...SeniorLong-term contractFull-timeRemoteFlexible hours£66k - £87k per annum
...design, build and distribute open-source software that enables people to enjoy the internet... ...: The Firefox team is a community of engineers who care deeply about delivering the... ...desktop platforms. About the role: A Senior Software Engineer contributes to architectural...SeniorFull-timeImmediate startRemote
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Software Engineer - LLM Evaluation. Be the first to apply!
Related searches
- senior operations manager food manufacturing Remote
- senior quality manager food Remote
- senior information technology manager Remote
- senior sql server dba Remote
- senior financial manager Remote
- senior bi analyst Remote
- senior technician Remote
- audit senior Remote
- senior operations manager Remote
- senior legal counsel Remote