Benchmark jobs in Remote

 ...cases in terminal workflows. We are seeking experienced native-speaking software engineers to design, build, and validate these benchmarks. You will create high-signal, high-quality tasks that genuinely test a model’s ability to handle multilingual environments without relying... 

 ...BUILDING THE FUTURE OF OPEN FINANCE Payward – the parent company behind Kraken, NinjaTrader, Breakout, xStocks, Payward Services and CF Benchmarks – has spent the last 15 years building one of the most modern and globally accessible financial infrastructure platforms in the... 

 ...verify rigorous multiple-choice questions across core physics domains, evaluate solution quality, and help establish gold-standard benchmarks used to advance AI capabilities. You will be assigned one of two task types: Question Authoring — Create original, challenging... 

 ...About the Role Join our global AI Community as a Bioinformatics AI Expert specializing in advanced benchmark dataset creation and AI model evaluation. In this role, you will design complex, reasoning-intensive, "information-reversal" evaluation tasks to measure the... 

 ...Research and build benchmark tasks to expose AI model weaknesses. About Turing: Based in San Francisco, California, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing... 

 ...Paid, remote work authoring and validating research-grade ML benchmark tasks. For Master's or PhD-level ML researchers with at least one first-author paper. Domain: Machine Learning / AI Research Requirements Experience: 1+ years of hands-on ML research; no upper limit Education... 

Contract & Payment Terms You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will...

 ...understanding of customer requirements for the proposal as well as key drivers for CRO selection decision. ~ Risk-informed recruitment benchmark data analysis: Create the basic foundations for the enrolment scenario building, namely range of countries and sites as well as... 

 ...actions, the interface state at every step, and the files before and after — and becomes both a demonstration the model learns from and a benchmark it is measured against. Because a model can only learn from work that can be scored, an equally important part of the job is... 

 ...lead post-sale, ensuring customer workloads successfully transition from the PoC/design phase into production deployment. Performance Benchmarking (Post-Sales): Guide the benchmarking of cluster performance during deployment using industry-standard tools (NCCL tests, GPUDirect... 

 ...a platform for building, evaluating, deploying, and operating agents at scale. With $125M raised at Series B from IVP, Sequoia, Benchmark, CapitalG, and Sapphire Ventures, we’re at a stage where we’re continuing to develop new products, growth is accelerating, and all team... 

 ..., including meshing. Solid verification and validation practice: mesh-independence studies, and validation against experiment or benchmarks. Comfortable with Python and GitHub. English at B2 level or above. Nice to have: aerospace, automotive, offshore or turbomachinery... 

 ...is essential. Key responsibilities Design, configure, and optimize Kubernetes clusters to support large-scale experiments and benchmarks of P2P technologies. Continuously improve cluster scalability and resource utilization, enabling increasingly complex deployments... 

 ...$110/hour Commitment: Hourly, ~10 h/week Field: Life, Physical, and Social Science About the work CritPt is a public benchmark of research-level physics challenges, built to test whether frontier AI models can carry out genuine physics research reasoning rather... 

 ...forecasting: Build models that analyze in-flight pipeline progress, execution velocity, and resource consumption against historical benchmarks to forecast SLA breaches 1-2 hours ahead, triggering priority re-queueing or upstream escalation. Bottleneck profiling and self-... 

 ...biological context behind computational analyses. Bonus Points • Experience with AI agents or coding agents. • Experience designing benchmark datasets or automated graders. • Publications involving computational genomics or bioinformatics. • Experience with Docker or... 

 ...or professional expertise within your specialization Strong analytical reasoning and ability to evaluate complex mathematical or statistical work Prior AI training, evaluation, benchmarking, or LLM experience is a plus Flexible, remote contract work starting ASAP.... 

 ...pharmaceutical knowledge work tasks, write the instruction specs and golden solutions that define what "correct" looks like, and build the benchmarks that show whether the model is genuinely improving. We are looking for a practicing specialist rather than a generalist. This is... 

 ...AI Trainer Project World Wide - Remote Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly handle complex workflows in this domain—specifically... 

 ...independent Business Owners across the food, lodging, service, and shopping sectors to provide their practical expertise for an AI benchmark evaluation and product feedback project. As AI models and digital commerce platforms increasingly integrate local business data, search... 

 ..., or reproducible research workflows. Experience developing or evaluating AI coding agents, terminal-based agents, scientific benchmarks, or automated grading systems . Publications, research contributions, or open-source development experience in Earth or Environmental... 

 ...and tell you what interferes at full travel, or predict what a design change breaks downstream. Your engineering judgment becomes the benchmark those models are measured against. What you’ll actually do Write challenging, closed-ended problems from your area of... 

 ...Information / Condensed Matter) to contribute to a research-driven customer project at the intersection of theoretical physics and numerical benchmarking. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn,... 

 ...respiratory, digestive, urinary, reproductive, musculoskeletal, and integumentary systems; Score the clinical reasoning quality of benchmark items: vignette accuracy and completeness, whether the vignette gives the answer away, answer correctness and gradeability, and trap... 

 ...SWE, SRE or security, or 3+ years of CTF. Reviews tasks in every category and tries to pass the tests without solving the problem. Benchmark Infra Engineer: DevOps, SRE or platform, 5+ years; Docker/BuildKit, CI pipelines, cloud GPUs. Builds the environments tasks run on;... 

 ...and climate, physics, biology Able to explain why a problem is hard for a frontier model Nice-to-haves Experience writing benchmark or evaluation questions (a big plus) PhD in ML, CS or physics, or an MSc plus strong research output Publications Prior Mercor... 

 ...seeking Life Sciences Subject Matter Experts to develop advanced scientific reasoning tasks for LifeSciBench. Unlike coding-focused benchmarks, this role emphasizes deep biological knowledge, scientific interpretation, and rigorous reasoning. You will create challenging questions... 

 ...slide decks across core workflows: client proposal decks, transformation roadmaps, operating model presentations, executive workshops, benchmarking exhibits, and final report production. Compensation $80/hour Commitment: Flexible, 5-20 hours per week (or more if... 

 ...structured feedback on messaging, composition, branding, hierarchy, visual storytelling, and overall creative effectiveness. Help define benchmarks for what makes advertising creative engaging, memorable, on-brand, and likely to perform. Compare multiple creative approaches,... 

 ...whether outputs meet professional brand design standards across consistency, versatility, and visual impact Help define creative benchmarks and taste standards for what great brand design looks like in the next generation of AI-powered tools Contribute to research on how...