When the Algorithm Misses You: Medicine’s Due Process Problem
Most systems that aim to allocate benefits or make classifications at scale face a familiar structural problem: general rules are necessarily imperfect. Every general rule is either over-inclusive or under-inclusive, and usually some of both. This is not a flaw of execution. It is a structural fact about rules.
In law, this principle is taught early and often. First-year law students learn to argue not only about what a rule says, but about how its application can deviate from its intended spirit. Edge cases are where the health of a legal system is tested: whether the system can recognize when a literal application of a general principle produces an unjust outcome in a particular instance.
The legal solution to this problem is well understood. The system builds in discretion, or mechanisms that allow decision-makers to recognize and correct for the gap between the general rule and the particular case. Judicial discretion, equitable remedies, individualized hearings, and administrative appeals all exist to prevent individuals from being treated as statistical noise when, for them, the decision may mean everything.
This is what due process is: a structural method for reconciling generalization with particularity. It is not merely an empty ritual, a performance meant to capture and subsume the emotional and social elements of the case.
Over- and Under-Inclusiveness: A Brief Primer
A rule is over-inclusive when it captures more cases than intended and applies to individuals who don’t truly fit the purpose it was meant to serve. For example, suppose a municipality implements a fine for failing to respond to a jury summons. The fine is meant to punish those who willfully ignore the same. However, if applied to a person who was in a coma throughout the period covering the mailing of the summons and the response deadline, it would feel unfairly punitive to most of us. Of course, there is a case to be made that anyone rendered comatose without a backup system to manage their administrative life during an unforeseen medical crisis has “failed” some test of executive function and organization. But we are stipulating that, in this instance, there is no evidence that the law was meant to root out people like that. Nothing in the legislative history suggests that it was, and most of the lawmakers also lack teams of administrative assistants that would keep their lives humming along in a similar crisis. Therefore, it would be fair to say that the rule is over-inclusive in this application.
A rule is under-inclusive when it fails to capture cases that it should, leaving out those who by spirit, if not by letter, deserve its protection or classification. For example, suppose a school in Japan announces a program to distribute a packet of free books to each member of the student body. To take advantage of the deal, students must register their full legal names via a particular app. Unfortunately, due to an oversight by the app’s designer, the name field only accepts kanji characters. Ten of the school’s students have names that are rendered in a different character set; the students are unable to complete the registration process. Without an alternative or corrective process to get the students into the system some other way, the rule as applied is under-inclusive. (Welcome to Japan.)
In both cases, the mismatch between the rule and the particulars matters. Without further tinkering, the outcome is undesirable and the affected individuals have suffered a miscarriage of justice, however trivial. A just system would recognize the limits of its generalizations and create mechanisms for correction: allowing the coma patient to submit proof of their hospitalization, or allowing students who cannot complete the app registration process to email their details to a support email address. These corrections are operationalized in the legal system under the umbrella of “due process,” allowing the system to implement rules that work well in 99% of cases without bogging the outliers down with Hellerian absurdities.
How Medicine Mirrors—And Diverges
Medicine also encounters this structure. In diagnostic reasoning, the statistical counterparts to over- and under-inclusiveness are known as sensitivity and specificity.
A highly sensitive test captures most true cases but risks more false positives (over-inclusiveness).
A highly specific test minimizes false positives but risks missing true cases (under-inclusiveness).
Clinicians are taught to think about these tradeoffs statistically in designing diagnostic criteria and interpreting screening results. But what medicine largely lacks is a formalized, patient-level counterpart to legal discretion. There is no structural mechanism or built-in safeguard for recognizing when a patient’s lived experience, clinical pattern, or individualized presentation reveals the limits of the algorithm itself. Law recognizes the inevitable failures of general rules and builds formal safeguards for the individual. Medicine, facing the same structural problem, has built almost none.
Instead, patients who do not fit are typically treated in one of two ways:
As noise: dismissed as atypical, irrelevant, or outside the intended scope of the diagnostic framework.
As noncompliant: implicitly blamed for not conforming to the expectations embedded in the rule.
In legal terms, it is as if someone brought an unjust application of the law to court and was told the law could not be questioned because it was "validated" statistically.
Without a method for recognizing and bridging the gap between the general and particular, the medical system fails both doctors and patients. Doctors are not empowered to exercise discretion in applying or discarding rules for maximum fidelity to individual patients, and patients are ignored in favor of the profiles generated about them after their parameters have been run against the rules.
Lived Experience of Diagnostic Under-Inclusiveness
My own experience of chronic illness has made these structures visible to me in ways that might otherwise have remained abstract. When I first began experiencing destabilizing physiological symptoms—marked drowsiness, post-exertional crashes, blood sugar instability—the diagnostic frameworks available to my doctors were ill-equipped to capture what was happening. My symptoms were real, disabling, and progressive. But they did not fit the thresholds or definitions built into standard algorithms.
I was not alone. Many patients living with complex, multi-system diseases find themselves similarly excluded—not because no biological pathology exists, but because the metrics, categories, and filters simply fail to map onto what they are experiencing.
In law, a just system facing this kind of under-inclusiveness would build an individualized appeal pathway. In medicine, the patient seeking relief is pathologized, a Kafkaesque punishment for simply pointing out what should be obvious to the doctor, too: something is wrong here, and the tools (the rules) aren’t finding it.
The Consequences of Missing the Individual
This absence matters. In disability systems, an over-inclusive application rule might be balanced by individualized audits or reviews. But the U.S. disability system attempts to cure both types of error at once, and the interaction between the epistemological failures of medical diagnostics is under-theorized.
In under-inclusive systems in which decisions do not rely on the judgment of outside experts, legal review can expand protections case by case. But in the disability context, the normal features of due process are stymied by the calcification of the diagnostic process: the patient with a real disability but with incompetent, ignorant, disinterested, under-resourced, or merely fallible doctors hits a solid brick wall. The failure to build discretion into the diagnostic process systematically locks out entire classes of patients and conditions and prevents due process from curing the gap between the general and the particular. The flaws in diagnostics are imported into disability adjudication, hollowing out due process and leaving many true patients without the support that the system is meant to provide.
Patients whose diseases are heterogeneous, emerging, systemically under-researched, or slow to develop diagnostic markers are especially vulnerable. Their suffering becomes illegible not because it is undetectable in principle, but because no structural method exists for acknowledging that the diagnostic algorithm itself is imperfect.
Medicine has inherited the statistical language of sensitivity and specificity, and this is reflected in medical teaching. But it has not inherited the legal intuition that edge cases are where a system’s justice is tested, nor has it built a bridge between that theoretical knowledge and the application thereof in practice, when it counts.
In principle, medicine could correct under-inclusiveness the way law does: by empowering individualized discretion. In practice, however, the dominance of insurance company protocols systematically removes this possibility. Doctors are financially constrained to follow rigid "standards of care" designed for statistical populations, not for the patient in front of them. Deviations from these standards—even when clinically warranted—are often refused coverage, effectively punishing both doctor and patient. Insurance appeals nominally exist, but they shift the entire burden onto the patient and offer no true analogue to due process. Thus, not only does the medical system lack mechanisms for recognizing edge cases, it actively punishes attempts to correct for them.
Toward a Better Model
What would it look like for medicine to internalize what law already knows?
To recognize in practice that no diagnostic algorithm is perfect, and that this imperfection is predictable, not exceptional.
To create formal mechanisms for individualized discretion when diagnostic pathways fail, such as by formally documenting and validating symptoms even when they cannot yet be explained with a single, sweeping diagnosis.
To acknowledge that rejecting a patient’s experience based solely on its misalignment with statistical norms is itself a form of structural error, not a reflection of scientific rigor.
To recognize that external financial constraints, especially insurance protocols, actively obstruct individualized physician discretion, and that medical records are mere artifacts rather than founts of ironclad truth.
In short, medicine should treat patients the way a just court treats petitioners: not as aberrations, but as signals. Every system of generalization faces edge cases, and how a system treats its edge cases is the clearest measure of its integrity.
Moreover, any serious reform must address the external structures that freeze discretion and exacerbate under-inclusiveness. Doctors must be empowered to depart from standardized pathways without financial penalty when the particulars of a patient's case demand it. A just system must restore the possibility of clinical judgment, rather than punishing it.