Do We Still Need Medical Segmentation Models?

TL;DR. We think a generalist model with a simple harness has the potential, in the long run, to replace medical segmentation models. The task has long been formulated as dense prediction, but when the object is well defined, as the structures in medical images are, it can be recast as reasoning. Generalist models now have robust visual and language understanding and, most importantly, strong multimodal reasoning: given a scan and Python, they look at slices and write the code that draws the mask. These abilities keep evolving, so for deployment the whole system evolves with the model, while a specialist model has to be retrained.

3D ultrasound

0.835

GPT-6 Astra's Dice on 3D ultrasound, against at most 0.342 for the models that take a text prompt

Subregions of an organ

Head, body, tail

GPT-6 Astra divides the pancreas. BiomedParse and SegVol, given the subregion's name, returned the whole gland every time

Similar score, far simpler harness

Just Python

on par on a published benchmark with agents that use 18 custom tools, a checker agent and long step-by-step instructions

Figure 1. Three results of GPT-6 Astra with Python as its only tool. Image sources: CETUS, PanTS.

PlaySlice

Image source: TotalSegmentator MRI v3. All outlines come from one real run on the liver.

Image source:

What the model was asked

the model has not looked at an image yet. It starts by reading the file.

PreviousPlayNext

The code the model wrote at this step

What it printed

Figure 2. Three real runs. Tab ① sums up one of them in five steps, from the 3D scan to 2D slices and back to a 3D mask. Tabs ② to ④ replay the three runs step by step, with the pictures the model looked at and the code it wrote.

Code as contouring

A clinician who outlines an organ by hand, as in planning radiotherapy, scrolls through the scan, finds the structure, traces its border on every few slices and lets the software interpolate the slices in between. A general model that can look at images and write code can follow the same steps, and then nobody has to train a segmentation model for each new structure, because a sentence that describes the structure is enough. If that works, segmentation turns from something you train into something you describe. We call this Code as Contouring (CaC), and we tested how far it gets when the general model is GPT-6 Astra and its only tool is Python.

Every other route to a 3D segmentation depends on a model built for the job and inherits that model's limits (Figure 3). TotalSegmentator (Wasserthal et al. 2023) segments only the classes it was trained on. MedSAM (Ma et al. 2024) accepts any structure, but someone has to mark it with a box, points or a scribble, and a poor prompt gives a poor mask. VoxTell (Rokuss et al. 2026) replaces the marking with a free-text prompt, and in exchange it is sensitive to wording and fails outside its training data. The nnInteractive Agent (Isensee et al. 2026) puts a generalist agent in front, which writes the prompts for the nnInteractive model (Isensee et al. 2025), so its masks are still only as good as that model, and the agent needs 3,800 words of step-by-step instructions and purpose-built tools. Code as Contouring keeps the agent and drops everything built for the job.

Tap the figure to enlarge

How it works. Each run starts from one 3D scan, a case, and one prompt. The prompt names every target, one structure to segment in that scan, in a sentence of its own, and CaC works through all the targets of a case in one conversation, with a Python interpreter and a way to view the images its code saves. It never writes out a mask voxel by voxel. Instead it writes code that draws a slice with a coordinate grid, looks at the picture to read off where the structure is, and types those coordinates into code that builds the 3D mask, the array marking which voxels belong to the structure.

One run. A 3D scan is a stack of 2D slices, so CaC looks at it one picture at a time and chooses its own key slices. On the liver of a TotalSegmentator MRI, it first viewed every second slice, then slices 3 to 26 with a grid drawn on them, and typed outlines on 14 of them. Its code fills each outline with scikit-image's polygon, fills the slices in between from signed distance maps of the two nearest outlines, and smooths the result with a Gaussian filter. The resulting mask scores 0.88 in Dice, the overlap between a mask and the reference on a scale from 0 to 1. Tab ① of Figure 2 shows this run in five steps, and the code below is step 9 of the run, with 12 of its 14 outlines left out.

from scipy.ndimage import distance_transform_edt as edt, gaussian_filter
from skimage.draw import polygon
def polys(d):
 m=np.zeros(a.shape,bool); keys=sorted(d)
 for y,pts in d.items():
  p=np.array(pts);r,c=polygon(p[:,0],p[:,1],(a.shape[0],a.shape[2]));m[r,y,c]=1
 for lo,hi in zip(keys[:-1],keys[1:]):
  if hi-lo>1:
   u=edt(m[:,lo,:])-edt(~m[:,lo,:]);v=edt(m[:,hi,:])-edt(~m[:,hi,:])
   for y in range(lo+1,hi):m[:,y,:]=((hi-y)*u+(y-lo)*v)>0
 return m
d={
3:[(230,432),(269,443),(296,432),(311,405),(310,359),(292,324),(280,322),(265,347),(246,365),(225,378),(217,407)],
4:[(224,437),(268,450),(295,441),(314,417),(322,385),(314,341),(294,305),(278,304),(267,335),(250,363),(225,384),(217,410)],
# 12 more slices, typed the same way
}
m=polys(d);m=gaussian_filter(m.astype(float),(.9,.45,.9))>.48;save(m,'t01_liver')

How the requests are written. The same name can mean different masks, so a request sentence has to say what the reference mask holds. We therefore wrote each sentence from the dataset's official label definition and added nothing to it. Where the name alone is ambiguous, the sentence says what is included and where the structure ends: "all vertebral bodies (without the vertebral arches), as one object", or "the left ventricular cavity (inside the endocardial border), including the papillary muscles and the trabeculations, closed at the mitral valve plane". In hearts with congenital defects a ventricle can lie on the other side, so there the sentence asks for "the morphologic left ventricle (identified by anatomy, not by side)". Only where a dataset publishes no definition is the sentence the plain name, such as "the left breast".

Main results

We compare CaC with three kinds of system built on a segmentation model. They differ in how they are told what to segment:

1. Fixed-class models. TotalSegmentator (Wasserthal et al. 2023) and nnU-Net (Isensee et al. 2021) take no text or visual prompt. They output the classes in their training labels and nothing else.
2. Text-prompted models. VoxTell (Rokuss et al. 2026), BiomedParse (Zhao et al. 2025), SegVol (Du et al. 2024) and SAT-Pro (Zhao et al. 2023) are also trained on labeled volumes, but they take the target as a sentence or an organ name. They learn what the words mean, so in principle one model can segment structures beyond a fixed list.
3. Agents that drive a segmentation model. In the nnInteractive Agent (Isensee et al. 2026), a general agent writes the clicks, boxes and scribbles for the nnInteractive model (Isensee et al. 2025) and reviews the masks it returns.

Visual-prompt models such as MedSAM (Ma et al. 2024) are left out, because in their evaluations the box or click comes from the reference mask, which tells the model where the target is. The one exception is in § Does it match a complex agentic system?. The paper behind nnIA-Bench, the benchmark of the nnInteractive Agent, reports nnInteractive given one simulated click per target with no agent in front of it, and that score is what the segmentation model reaches on its own, so it shows how much the agents add.

The first four experiments ask for whole organs and structures, and the fifth for subregions, the named parts of one organ such as the head, body and tail of the pancreas, most of which no model has a class for.

Experiment 1

How accurate is Code as Contouring?

It varies widely with the organ and the kind of image. Mean Dice is 0.93 on lungs and airways in CT, 0.81 to 0.85 on 3D ultrasound of the heart and the prostate, and 0.40 to 0.49 on the three TotalSegmentator sets.

The ranking does not follow the modality: the two 3D ultrasound sets are among the four highest of the ten, and the three lowest sets all come from one source, TotalSegmentator (Table 1). The main cause of their low scores is that on these scans CaC often swaps left and right, and the swap follows how the file stores its axes (see § When does it swap left and right?).

Elsewhere CaC finds the structure, and what it loses is at the edge. Of its masks, 84% overlap the reference with a Dice of at least 0.3. The most common error is an edge drawn a few millimeters off (see § Where do the errors come from?), and it shows in the surface score: on the heart ultrasound set Dice is 0.85 and NSD, the share of the mask's surface within 2 mm of the reference's, is 0.49.

Table 1. CaC's scores on the ten datasets. HD95 and ASSD are in millimeters.

A typical mask from each dataset

Tap the figure to enlarge

Experiment 2

Does it match TotalSegmentator?

TotalSegmentator has no class for nearly half of our targets, so over all of them CaC scores 0.628 Dice against 0.396. On the CT classes it does have, TotalSegmentator leads, 0.833 against 0.637.

TotalSegmentator has one general model for CT and one for MRI, and no model for ultrasound. Each outputs a fixed list of classes, and for a target outside the list it returns nothing. That is the case for just under half of our targets (Figure 5, folded below): every ultrasound target, the chambers of the congenital hearts, the lung airways and the liver veins. CaC's lead over all targets comes entirely from these (the "Dice, all" column of Table 2).

How many targets TotalSegmentator covers, by dataset (Figure 5)

Tap the figure to enlarge

Table 2. "Dice, all" counts every target, with 0 where TotalSegmentator has no class. The other columns count only the targets it has a class for. The parentheses give all targets and covered targets. On the nnIA-Bench row, "Dice, all" is averaged within each image first.

Where it has a class on CT, TotalSegmentator is clearly better. Part of the gap is CaC swapping left and right (see § When does it swap left and right?), which is most frequent on TotalSegmentator's own two CT datasets. The rest is accuracy: on the pediatric CT, where CaC swapped the sides in no case, TotalSegmentator still leads, 0.78 against 0.66, and only on the lungs do the two tie, at 0.97.

On MRI CaC comes out ahead, 0.578 against 0.517, because TotalSegmentator can have a class and still return nothing. The aorta and inferior vena cava of the congenital-heart scans are on its class list, yet it returned an empty mask for 32 of the 120. On its own MRI scans TotalSegmentator leads, 0.76 against 0.46, so CaC's lead comes from the congenital hearts.

TotalSegmentator is far faster and cheaper to run. A case took it a median of 34 seconds on one NVIDIA RTX 6000 Ada GPU, and renting that GPU for 34 seconds costs about a cent at a public cloud rate (RunPod 2026), while a CaC case took about eight minutes and cost $3.23 on average.

What if labeled data exist?

A model trained on them scores higher: nnU-Net reaches 0.934 Dice on prostate ultrasound, against CaC's 0.812.

TotalSegmentator has no ultrasound model, but where labels exist one can be trained, so we trained nnU-Net on labeled volumes from the same prostate ultrasound dataset and scored it on the cases CaC had segmented. Training took 2.2 GPU-hours, about $2 at the same cloud rate.

Experiment 3

Does it beat text-prompted models?

On the two 3D ultrasound datasets, CaC scores 0.835 Dice and the best of the four models 0.342. On CT and MRI, VoxTell is ahead of CaC.

Tap the figure to enlarge

The four models work where they were trained and fail outside it. On ultrasound none of them comes near CaC: BiomedParse had seen ultrasound of the heart in training and still scores less than half of CaC's Dice on our heart set. nnIA-Bench shows both sides at once: VoxTell is ahead of CaC on 8 of the 12 images and far behind on the other four, among them a micro-CT of a mouse and of an ant.

On CT and MRI the trained model wins, and it does not need to have seen the dataset: of the five sets where VoxTell leads, three come from sources it was never trained on. What it has seen is the modality, more than 62,000 CT, PET and MRI volumes.

Does the wording of the request matter? A single sentence can take a model from a good mask to no mask. On a dataset it was trained on, SAT-Pro scores 0.90 given only the organ's name and 0.10 given the sentence CaC got, and most of its masks come back empty. VoxTell is built for free text and barely moves on CT and MRI, yet on the adrenal CT of nnIA-Bench the sentence takes it from 0.79 to 0.00. The image is as fragile as the words: without its CT intensity window, BiomedParse loses more than half of its Dice. Given the name (the orange ticks in Figure 6), SAT-Pro moves ahead of CaC on CT, and nothing else in the comparison changes.

Experiment 4

Does it match a complex agentic system?

On nnIA-Bench, CaC scores 0.824 Dice, and the two agents 0.838 and 0.825. CaC had a prompt of 200 to 400 words and Python. The agents add a segmentation model, 18 purpose-built tools and about 3,800 words of instructions.

The nnInteractive Agent is a system stacked from many parts. A coding agent sits in front of the nnInteractive segmentation model. It reads the request, writes the clicks, boxes or scribbles that prompt the model, and reviews the masks that come back. It works from a long set of instructions, with tools written for each of these steps and an optional second agent that checks the work. CaC got the same request sentences and none of these parts.

Table 4. CaC, with no segmentation model, scores about the same as the two agents that drive nnInteractive.

The means are close, but single images differ widely: nnInteractive draws the 17 bones of the foot CT better than CaC, 0.98 against 0.84, and CaC is ahead on the two micro-CT images and on the bladder MRI.

What do the parts buy? They make nnInteractive segment better. Given one click and no agent, nnInteractive scores 0.695, and everything stacked around it raises that by 0.13 to 0.14. On the liver cancer CT one click scores 0.37 and both agents 0.96. CaC does not need nnInteractive made better, because it does the segmentation itself: on that image it scores 0.94 with no model.

Experiment 5

Can it segment subregions of an organ?

CaC divides the organ from the written definitions alone. Asked for the head, body or tail of the pancreas, the trained models mostly return the whole gland.

Apart from the prostate's peripheral and transition zones, no model's label list has these subregions: TotalSegmentator has "pancreas" and "prostate" and nothing below them. A lab that wants them from a trained model therefore needs new labels and new training for each one, whereas for CaC a new structure is a new sentence.

On the pancreas, BiomedParse and SegVol returned nearly the same mask three times, a mask of the whole gland. VoxTell's three masks are mostly the gland as well, and SAT-Pro left 12 of its masks empty. CaC's three masks are separate in every case (Figure 7).

Tap the figure to enlarge

Dice misses all of this. On PanTS the methods look tied, and a perfect mask of the whole pancreas, handed in for every subregion, scores higher than all of them (Table 5). The reason is arithmetic: a subregion that is a third of the gland gives that mask a recall of 1 and a precision of one third, which makes a Dice of 0.5. Precision and HD95, the distance between the two surfaces, punish the extra volume, and they rank CaC first.

Table 5. "Whole pancreas" and "whole prostate" are a perfect mask of the organ handed in for every subregion. Overlap is the Dice between a method's own masks of different subregions: 0 means separate and 1 means the same mask. HD95 is in millimeters.

Does the full definition help the models? No. The sentence says where each subregion lies, for example "to the right of the superior mesenteric vessels." On neither dataset did any model score significantly higher with the sentence than with the bare name, and in five of the model and dataset pairs the name was significantly better (uncorrected p < 0.05).

On the prostate the difference shows in Dice as well. CaC scores above every model in all 10 cases, at 0.454, close to the 0.505 at which the two human readers agree with each other. VoxTell was trained on the peripheral and transition zones and beats CaC on the transition zone, 0.85 against 0.76, but asked for the central zone, the stroma or the urethra, it draws the transition zone again (Figure 8), and SAT-Pro has the peripheral and transition zones the wrong way round.

One ProstateZones case (Figure 8)

Tap the figure to enlarge

CaC's accuracy on a single subregion is moderate. Its lowest score is the pancreas body, 0.25, and part of that is a disagreement about where the body ends: the written definition ends it at the left edge of the aorta, and CaC's boundary lies a median 1.6 mm from that edge, while the boundary in the released labels lies 18 mm from it.

Analysis

This section uses only CaC's own runs on the nine datasets other than nnIA-Bench.

Experiment 6

Where do the errors come from?

Mostly from boundaries: CaC finds the structure and draws its edge a few millimeters off.

• 61%: imprecise boundaries of usable masks. Small structures suffer more: in 6 of 9 datasets, Dice rises with the size of the structure. On MRI, where CaC outlines most targets by hand, the masks run large, a median 1.49 times the reference's volume on TotalSegmentator MRI.
• 17%: left and right swapped. Most of this is on the three TotalSegmentator sets, the subject of § When does it swap left and right?. The rest is swapped heart chambers, which go with unusual anatomy: a ventricle or atrium pair is swapped in 11 of the congenital-heart cases, 10 of them with a severe defect (see § Can its knowledge of anatomy mislead it?).
• 14%: wrong location or wrong object. The typical case is the right structure in a plausible but wrong place, for example the right adrenal gland drawn on the back edge of the liver in 19 pediatric cases.
• 8%: everything else. Poor boundaries of failed masks, empty masks, and targets whose official definition and released mask disagree. For the TotalSegmentator iliac vessels, the definition says "common iliac" and the reference also includes the external iliac.

An edge a few millimeters off and a structure in the wrong place are different problems, and they need different checks.

Experiment 7

When does it swap left and right?

On files stored as RAS, CaC swapped the sides in 26 of 47 cases. On files stored as LPS, it swapped them in none of 20.

A NIfTI file's header says which way each voxel axis points in the patient. TotalSegmentator stores its volumes as RAS (right, anterior, superior), in which the first index increases toward the patient's right. DICOM, the format of clinical scanners, uses LPS (left, posterior, superior), in which the first axis increases toward the left. The difference matters because of how scans are viewed: a radiology screen shows the patient as seen from the feet, so the patient's left is on the right of the screen.

The swapped runs treat every file as LPS. CaC's drawing code transposes each slice before plotting, so the first index runs left to right across the picture. On an LPS file the picture then looks like a radiology screen, with the patient's left on the right. On an RAS file the same code puts the patient's right there, and CaC still names the sides as on a radiology screen, so every name is reversed. What separates the correct runs is that they worked the mapping out: none of the 26 swapped runs wrote down a correct mapping from voxel index to patient side, and 10 of the 21 correct runs did. Reading the header helps as well: runs that called nibabel's orientation function swapped the sides in 5 of 19 cases, and the others in 21 of 28.

The swapped masks are good masks with the wrong name (Figure 9). Scored against the reference of the other side, their Dice rises from about 0 to 0.73 and 0.71 on the two CT sets and to 0.52 on the MRI set. With the names corrected, the means of the three sets rise to between 0.59 and 0.70. A swap can also leave a mask empty. In one CT, CaC searched for the left humerus on the patient's right, found no bone there, and saved an empty mask with the message "the left humerus is outside the scan's field of view". That is true of the right humerus, and the left one is in the scan.

Tap the figure to enlarge

Some runs saw that the anatomy looked reversed and explained it away: six swapped runs mention situs inversus, a rare mirror-image arrangement of the organs. One reads: "I'm analyzing an image that suggests possible situs inversus due to the liver's position on the right side and the spleen being high" (CT v1, case s1248). None of the six printed the file's orientation codes. Checking such a system means checking what it assumed about the data, and the reasoning summaries let us read those assumptions.

Does one note fix it?

Yes. None of the 12 cases with the note was swapped.

The note tells CaC to take left and right from the file header and says how to read it. Both parts matter. In a pilot on 5 cases that had been swapped, the first two sentences alone, which point to the header, fixed 2, and the third sentence, which says what the axis codes mean, fixed the other 3. Five of the swapped runs had printed "RAS" in their first step and never wrote down what it meant.

Experiment 8

Can its knowledge of anatomy mislead it?

Yes. CaC knows the textbook rule, and it fails on the patient who breaks the rule.

In a normal heart the aorta leaves the left ventricle. One patient in the congenital-heart set has a transposition: the ventricles are inverted, and the aorta leaves the morphologic right ventricle. CaC saw this early. Its summary at step 7 reads "the right ventricle is anterior, and it seems to feed into the aorta, while the left ventricle feeds the pulmonary artery", and the two ventricle masks it then drew had the correct names: the one it called left ventricle scores a Dice of 0.74 against the left ventricle's reference. The request asks for the morphologic left ventricle, identified by anatomy and not by side, and thirteen steps later CaC decided that the chamber with the aortic outflow was the morphologic left ventricle and overwrote its own answer:

# Morphologic LV: rounded rightward chamber, with paired papillary muscles and aortic outflow.
m=np.load('rv.npy');save(m,'t01_left_ventricle');np.save('lv_correct.npy',m)

The right-ventricle mask, renamed and stored as lv_correct.npy, scores a Dice of 0.00 against the left ventricle's reference and 0.81 against the right ventricle's (Figure 10), and the mask that had been correct for the left ventricle was saved as the right one. A second run on a heart with inverted ventricles also exchanged its two correct masks late in the run, under the comment "correcting morphologic identities".

Tap the figure to enlarge

Tap the figure to enlarge

Counting fails in the same way. Asked for the right 7th rib on a CT that is cut off at the top, CaC split the ribs into connected pieces of bone, colored them on a projection and counted upward from the lowest one. This patient's 12th rib is rudimentary, 1.4 ml of bone, and it was not among the pieces, so the count started one rib too high. The saved mask is a clean rib that scores 0.92 against the 6th rib and 0 against the 7th (Figure 11).

In both runs the drawing is right and the rule that names the mask is wrong. A rule that holds in almost every patient is what a general model learns best, so its naming errors gather in the patients whose anatomy is unusual.

Experiment 9

Does it work like an annotator?

In the order of its steps, yes: a run renders the scan, looks, reasons from anatomy and then draws. What it lacks is a check of the finished masks.

Our prompt names the structures and Python as the only tool and says nothing about what to render or how to use the pictures, so the order of steps in a run is CaC's own. Nearly every run renders the image before it draws any mask. One run on an AeroPath CT shows the order: it began with overview montages, coronal views and per-slice statistics, chose a method and its thresholds, computed an airway mask and rendered a projection of it. From that image its summary placed a possible leak, so it changed the region growing to block the leak and saved, for an airway Dice of 0.90.

The rule it draws with changes with the physics of the scanner. CT values are calibrated Hounsfield units, and on CT it types fixed cutoffs after looking at the image, with medians of -850 for lungs and airways and 170 for bone. MRI values have no fixed meaning, so there it picks a new number for each image: on the heart MRIs its cutoffs range from 125 to 1,025, yet divided by each image's blood brightness most land between 0.44 and 0.66. On ultrasound it thresholds in 1 of 95 runs, and in some heart scans it searches for the dark-to-bright wall along rays. When a job repeats, it also builds its own tools: in every heart MRI run (8 targets) and every pediatric CT run (23 targets) it wrote a helper module, with functions like show(), montage() or ellipses(), and called it from most later steps.

The step an annotator ends with is the one the runs skip. CaC viewed its saved mask over the image for 27 of 1,369 targets, and 255 of 271 runs close with the same message, that all masks are saved and the geometry matches the input. The doubts in its reasoning summaries do not mark the failures either, since as a review flag they would catch 22% of them. Yet the misses were in view: for 1,366 of 1,369 targets the reference structure lay on at least one slice the model looked at, and the adrenal and thyroid glands, its weakest structures, were mostly drawn in the wrong spot.

Each failure suggests its own check. Writing the patient's left and right on every rendered image exposes a swapped side. A zoomed view before saving catches a misplaced gland, and a test for overlap with the neighbors catches masks that run into each other.

Discussion

Keep the system simple. On nnIA-Bench, the model's own agentic ability was enough to work out the workflow, so step-by-step instructions and purpose-built tools had little left to add. CaC's runs arrive on their own at much of the workflow that the nnInteractive Agent's long instructions spell out (Isensee et al. 2026). Sutton's bitter lesson (Sutton 2019) describes the same pattern in AI research, where general methods overtake hand-built knowledge as they scale, and here the hand-built knowledge is a set of step-by-step instructions written into a prompt. Model makers give the same advice for prompts, and prior exploration of generalist agents in medicine points the same way (Schluntz et al. 2024, Anthropic 2025, OpenAI 2026, Zhang et al. 2026). We would state the task, give general tools, measure, and add back only what the measurement shows is missing, as we did for swapped sides with three sentences on reading the file header.

New structure, new sentence. The same short prompt carried across structures and scanners, and a new structure cost one new sentence. Trained networks stay tied to their training sites: in one study, conventionally trained models lost 39% of their Dice on an unseen site or scanner (Zhang et al. 2020). CaC adapts because it changes its rule with the physics of the scanner and writes that rule down in code. A rule written in code can be read, corrected and reused.

Write the definition down. A generalist follows the convention it is given, so segmentation by description needs its conventions in writing. Where nothing is written even experts differ: a radiation oncologist and a radiologist outlining the same rectal tumors agreed at a median Dice of 0.75 (Stratulat et al. 2026). In the pediatric CT references an adolescent's breast is the whole mound including its fat, a convention that lives only in the labels, so CaC drew a much smaller patch and scored 0.01-0.28. Scores need the same care, because on the pancreas subregions Dice ranked a mask of the whole gland above every method we ran. An evaluation has to report several metrics, add reference lines that show what a trivial answer would score, and include a visual check of the masks; on the subregions, one figure of the masks (Figure 7) showed at a glance what the means had hidden.

The missing self-check. In these runs CaC did not notice its own errors, and language models in general need outside feedback to correct themselves (Huang et al. 2024). On the targets that clearly failed, the wrong side or the wrong place accounts for four fifths of the Dice lost, and the final messages stay silent on those failures. The run that replaced its correct ventricle masks with wrong ones, described in § Can its knowledge of anatomy mislead it?, closed by reporting that all eight masks were saved under the requested names. Over-trust in auto-contours is a recognized clinical risk (De Biase et al. 2024), and a reviewer needs the system to say where it may be wrong. A general model can check a mask that it is handed: in the nnInteractive Agent, where the agent reviews the masks it gets back, Dice rose from 0.74-0.76 after the first prompt to 0.81-0.84 at the end (Isensee et al. 2026).

Generalist and specialist together. A trained model remains the cheaper and more accurate route to most structures it already covers, at about a cent per case. Its coverage grows only by training again, and the dependable way to do that is still to train on old and new data together: continual-learning methods are still measured against this joint training (Qazi et al. 2024), and TotalSegmentator retrained all of its models when its second version added classes (Wasserthal 2023). For VoxTell one such run took roughly 9,200 GPU-hours on more than 62,000 scans (Rokuss et al. 2026). CaC, by contrast, gains a structure with one sentence and improves whenever the general model under it does. The two also fail in different places, so the system we would build today uses both. Scoring ultrasound with CaC's masks and CT and MRI with VoxTell's gives a mean Dice of 0.784, against 0.730 for VoxTell alone and 0.622 for CaC alone, and all of the gain comes from ultrasound. They can also work in sequence: label-building loops start from a few hand-segmented cases (Wasserthal et al. 2023), and a generalist could draft those for a structure that no model knows yet. Each new general model then moves the line between the two toward the generalist.

References

Close