- AI Spotlight
- Posts
- AI Model Achieves Higher Scores Than Most Physicians on U.S. Medical Licensing Test
AI Model Achieves Higher Scores Than Most Physicians on U.S. Medical Licensing Test
Greetings AI enthusiasts. An AI tool built by University of Buffalo researchers has performed exceptionally well on the U.S. Medical Licensing Examination (USMLE). The tool could also support physicians by providing evidence based insights during clinical decision-making. Read more below.
In today’s email:
AI Model Achieves Higher Scores Than Most Physicians on U.S. Medical Licensing Test
How Could AI Influence the Field of Architecture? Insights from a Yale Expert
Workers could save 122 hours a year by using AI
LATEST NEWS
HEALTHCARE AI
image credit: Suriphon Singha / iStock / Getty Images Plus
AI Spotlight: Researchers at the University of Buffalo have developed an AI tool called Semantic Clinical Artificial Intelligence (SCAI), which has demonstrated exceptional performance on the U.S. Medical Licensing Examination (USMLE). This tool could play a role in supporting doctors by delivering reasoned, evidence-based insights during clinical decision-making.
Key details:
SCAI outperforms prior models and most human test takers: Scoring as high as 95.1% on Step 3 of the USMLE, SCAI surpassed GPT-4 Omni and significantly outperformed previous AI models across all three USMLE steps, highlighting its advanced reasoning capabilities.
Built on formal semantic reasoning: The system leverages 13 million clinically validated facts encoded as semantic triples and integrates knowledge from trusted medical sources while avoiding biased data inputs, like clinical notes.
Enhanced performance across model sizes: While larger LLMs (70B and 405B parameters) passed all USMLE steps even without augmentation, their scores improved further when paired with SCAI’s retrieval-augmented generation (RAG) framework, showcasing the power of semantic enrichment.
Future potential and limitations: Though the results are promising, the researchers emphasize that SCAI is not a replacement for physicians. The tool still needs validation in real-world settings, and its current evaluation was limited to specific models and semantic frameworks.
The SCAI tool represents an advancement in clinical AI, offering structured medical reasoning that supports—not replaces—physicians. The next phase will involve testing its real-world applicability while ensuring safe, ethically governed use in healthcare environments.
ARCHITECTURAL AI
AI Spotlight: Artificial intelligence is making waves across industries, including architecture, where its role is evolving but far from transformative. Phillip Bernstein, Yale architecture professor and former Autodesk executive, offers a grounded perspective on AI's current and future influence on architectural practice in his recent interview and book.
Key details:
Technology has long shaped architectural methods: From hand-drawing to computer-aided drafting and now to Building Information Modeling (BIM), architects have historically adopted tools to represent and manage complex design data. Despite technological advances, architects still make all core design decisions.
AI’s role remains task-specific and supportive: Current AI applications in architecture assist with documentation, basic code compliance, and translating design data into construction logistics. These tools improve efficiency but fall short of understanding or generating architectural design on their own.
AI lacks spatial and temporal reasoning: Bernstein emphasizes that true disruption would require an AI capable of reasoning about 3D spaces over time, something existing models like ChatGPT or image generators cannot do. Experiments with visual prompts show creative variety but not architectural understanding.
The profession faces a data bottleneck: Bernstein introduces the concept of a “data interstice” — a period where the building industry produces vast digital outputs but lacks a unified system to structure this data. Without that foundation, training an AI capable of meaningful design reasoning is out of reach.
AI is enhancing architectural workflows in limited ways but isn’t poised to replace architects. As the technology matures, its greatest impact may come not from designing buildings, but from supporting the complex processes behind bringing those buildings to life.
image credit: fizkes/iStockphoto/Getty Images
AI Spotlight: A recent Google report reveals the UK could unlock £400 billion in economic growth through the effective adoption of AI, particularly by equipping workers with basic training and permission to use the technology. Pilot programs across various sectors demonstrated productivity gains when workers used AI for administrative tasks.
Key details:
Productivity potential: AI could save UK workers over 120 hours a year on administrative tasks, with economic modeling suggesting this could translate into a £400 billion boost to the UK economy.
Barriers to adoption: A major hurdle is psychological, not technical—many workers, especially older women from lower-income backgrounds, were unsure if using AI was allowed or appropriate in their roles.
Training makes a difference: When employees received just a few hours of training and explicit permission to use AI, adoption rates doubled. Confidence and continued use were notably higher even months after the intervention.
Dramatic usage increases: In one cohort, weekly AI use among women over 55 jumped from 17% to 56% after three months, and daily use rose from 9% to 29%, narrowing a clear digital adoption gap.
This report underscores that simple, inclusive steps can lead to widespread AI adoption and major productivity gains. With proper support, AI can become a routine and beneficial tool across the workforce.
MORE STORIES IN AI