As large language models (LLMs) become increasingly intertwined with clinical care, detailed and rigorous validation is essential. Healthcare in particular demands exceptional standards of accuracy, reliability, and transparency, particularly when patient outcomes can depend on the insights produced by these systems. No industry requires this more or has higher stakes. By leveraging only the most trusted resources combined with System’s patented graph architecture, System's Synthesize API delivers rapid, accurate, and verifiable solutions to a diverse range of clinical scenarios.

Today, we are excited to announce that System’s Synthesize API has achieved a major milestone in its validation journey. Using sample exam data provided by the United States Medical Licensing Examination (Kung et al, 2023), the Synthesize model achieved a score of 100% on Step 1, 2 CK, and 3. Synthesize delivers guideline-based recommendations with detailed clinical explanations for each response, highlighting its value as both a decision-support partner and an educational resource. Together with Synthesize's superior accuracy in blind testing, we are continuing to build on Synthesize's record of accuracy, reliability, and transparency.

‍

^{*OpenEvidence Report: https://www.openevidence.com/announcements/openevidence-creates-the-first-ai-in-history-to-score-a-perfect-100percent-on-the-united-states-medical-licensing-examination-usmle
**Reference results from Siam et al. (Scientific Reports, Dec 3 2025; doi: 10.1038/s41598-025-31010-4), who evaluated a distinct but similar USMLE sample-question set. Scores computed across Step 1, 2 and 3 on 324 text-based and text-with-mathematical-reasoning (TM) items.}

Methodology

We evaluate System’s Synthesize API using a subset of 325 multiple-choice questions drawn from the publicly accessible official USMLE sample exams (Step 1, Step 2 CK, and Step 3). Following the method of others, questions containing images are removed, leaving only text-based questions, including those requiring mathematical reasoning. Each question is evaluated by submitting only the raw question text to the Synthesize API, with no additional prompting or instructions.

Each question–answer pair is included below, along with System’s Synthesize API reasoning and reference citations.

‍USMLE Step 1 Answers

USMLE Step 2 Answers

USMLE Step 3 Answers

We adopt a similar correction for Question 125 – Step 3 as previously described by OpenEvidence. Their conclusion, validated by seven independent psychiatrists, is treated as the gold-standard answer for this question.

System’s Synthesize API provides a more up to date answer for Question 14.1 – Step 2 by selecting (D) non–HIV–related malignancy, incorporating the latest Infectious Disease Society of America guideline Primary Care Guidance for Persons with Human Immunodeficiency Virus published in October 2024. The official USMLE key indicates a response of (A) Cardiovascular Disease, but given that the sample exam was released in June 2022, the response does not incorporate the updated guidance.

Because the Synthesize API can filter evidence recommendation by publication date, it can also reproduce recommendations within the exam’s historical context. When restricted to pre-2022 sources, Synthesize selects (A) Cardiovascular Disease, matching the USMLE answer key.

The System Synthesize API is powered by a novel AI agent-based retrieval and rerank algorithm and a set of processes to enforce strict citation validation. It does not rely on knowledge encoded in LLMs to generate answers to clinical queries; we have run extensive testing to ensure that answer accuracy is not LLM-dependent.

About System

System builds knowledge infrastructure to transform decision-making from silos to systems — starting in healthcare. System’s APIs are used today by leading healthcare providers in the US and Europe to power groundbreaking clinical decision support systems (CDSS). At the core of System is the System Graph, a patented, large-scale, statistical graph of the world modeled as one interconnected system, based on trusted sources of evidence that are updated daily. System Inc. is a Public Benefit Corporation committed to advancing systems thinking in the world.

‍