AI Evaluation Engineer Job at Aurora Jobs ApS, San Francisco, CA

U1BvT2pycHlvT0xjTkljUGhqRGUzUHFkQ0E9PQ==
  • Aurora Jobs ApS
  • San Francisco, CA

Job Description

This position is at Aurora Jobs ApS

The selection process will be fully managed by Aurora Jobs ApS.

--

**Sede:** San Francisco, CA - Full-time, On-site **Salario:** $130K - $220K **Equity:** $60K - $120K options/year **Sponsorizzazione Visto:** Disponibile caso per caso **About Artificial Analysis** Artificial Analysis is the leading independent AI benchmarking and insights company. They help engineers, enterprises, investors, media, and policymakers understand AI capabilities and make critical decisions about AI strategy. Their benchmarks are trusted by hundreds of thousands of users and are the go-to reference for leading AI labs including OpenAI, Google, Meta, NVIDIA, and Anthropic. Their work is also cited by the Wall Street Journal, Bloomberg, Financial Times, and The Economist. Their benchmarks do not just measure frontier AI — they actively shape it. The company was founded in 2024, has raised $3.5M, and has a team of 30+ employees across San Francisco, Sydney, and Melbourne, with plans to double by mid-year. They are backed by Nat Friedman, Daniel Gross, Andrew Ng, Adam D’Angelo, Clem Delangue, and other leading figures in AI. **About Aurora** Aurora helps top engineers discover opportunities at some of the most ambitious startups worldwide. We work closely with companies to identify exceptional talent and match them with roles where they can have real impact. We are currently helping Artificial Analysis expand their team. **Position: Member of Technical Staff** **What This Role Actually Is** This role is best described as an AI Evaluation Engineer / Technical Generalist. It is not a traditional software engineering role and not a pure ML research position. It combines AI benchmarking, technical product work, strategic analysis, customer-facing collaboration, and startup execution. **About the Role** Artificial Analysis is hiring Members of Technical Staff to design the evaluations that define how AI is measured, produce analysis that shapes how companies and the broader market understand AI, and work directly with the leading AI labs and enterprise customers who rely on their insights. You will develop new benchmarking methodologies, manage relationships with some of the most important AI labs and enterprise customers in the world, and help drive the product direction of the platform. The success bar for this role is becoming a world expert in modern AI technologies. This is a unique combination of product, research, technical, and client-facing work suited to highly driven technical generalists who thrive on breadth and ownership. You will work directly with the founders and across the full team, with outsized impact on both the company and product. **What You’ll Do** * Design, structure, and execute projects evaluating AI systems and technologies. Develop new benchmarking methodologies and datasets that improve evaluation capabilities. * Build reports, strategic analysis, and data visualizations that help enterprises shape their AI strategy. * Collaborate with leading AI companies to benchmark agentic AI applications, foundation models, and hardware systems. * Identify opportunities to improve the benchmarking platform and work with developers to ship those improvements. * Operate in an AI-native workflow using cutting-edge AI tools to create leverage and maintain speed. * Contribute to company strategy and lead major initiatives that support the company’s goal of becoming the leading AI benchmarking platform. **Requirements** * 2–10 years of experience in MBB consulting or technical roles such as TPM, ML Engineer, or similar. * Strong analytical ability, technical fluency in modern AI systems, excellent communication skills, ownership mindset, and comfort operating in a fast-moving startup environment. **Tech Stack** * Python **Why Join** This is a rare chance to get a front-row seat to frontier AI and evaluate cutting-edge systems before they are publicly released. You’ll work directly with the world’s top AI labs while helping define what best-in-class AI performance actually means. The company is financially strong, fast-growing, highly connected, and offers strong ownership and visibility from day one.

--

[#J-ONE] [#J-MIN]

[#J-INTERNAL]

Job Tags

Full time, Worldwide

Similar Jobs

Nebo

Email Marketing Specialist Job at Nebo

 ...Email Marketing Specialist The email team at Nebo connects people to the brands they love. We don't believe in spamming inboxes, but creating thoughtful, strategic email campaigns that surprise and delight customers all while helping businesses reach their goals. That... 

Ourisman Kia

Internet Sales Representative Job at Ourisman Kia

 ...Kia is looking for a full time Internet Sales Consultant to come join our award winning...  ...internet. We use state of the art CRM software with our extensive inventory to make sure...  ...growth. Most of our management team began in entry level roles and got promoted through their... 

PGA National Resort (Salamander Collection)

Front Desk Agent Job at PGA National Resort (Salamander Collection)

 ...as they arrive and leave the hotel. Help guest with issues that may...  .... When servicing guests, the Front Desk Agent is responsible for being...  ...all monies taken in during your shift according to the procedures....  ...flexible schedule including day/night shifts, weekends and holidays... 

Aulani, A Disney Resort & Spa

Poolside Server Assistant - Full Time, $29.15/Hour Job at Aulani, A Disney Resort & Spa

 ...admission and much more! Do you have a passion for serving others? If so, consider supporting the Guest experience as a Poolside Server Assistant. Poolside Service will have you working on all types of terrain - sand included so our Guests can continue to enjoy the... 

Livetripexperience

Remote Travel Support Specialist Job at Livetripexperience

 ...We are looking for a dedicated Remote Travel Support Specialist to ensure our clients have a seamless journey from start to finish. In this role, you will be the primary point of contact for travelers, resolving inquiries and managing itinerary changes with efficiency...