Author(s): Marie Humbert-Droz, PhD

Originally published on Towards AI.

I ran OpenAI, Anthropic, and local models head-to-head on clinical note extraction. Here’s what I found.

After setting up my local clinical abstraction agent and testing its ability to call tools, I wanted to see how it stacks up against cloud APIs from OpenAI and Anthropic. It’s slower (it runs on my MacBook, after all), but is the tradeoff worth it and how close can local AI get on accuracy?

This article explores the effectiveness of local AI models compared to cloud-based models for clinical data extraction, focusing on speed, accuracy, and cost. The author sets up a local clinical abstraction agent and comprehensively tests various models, including OpenAI’s and Anthropic’s APIs. Findings indicate local models are slower yet maintain a reasonable accuracy, while cloud models generally outperform in complex extraction tasks, leading to discussions on trade-offs between privacy, cost, and model performance.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI