Why AI Facial Recognition is Completely Useless for Art History

Why AI Facial Recognition is Completely Useless for Art History

Art historians are falling over themselves to celebrate a computer program that claims a famous Holbein sketch isn't Anne Boleyn, but her mother, Elizabeth Howard. They think tech has solved a 500-year-old mystery.

They are wrong. They are being conned by algorithmic confidence. Read more on a related subject: this related article.

The recent facial recognition analysis of the Royal Collection’s 16th-century portrait drawing—traditionally inscribed as "The Anna Bollein Queen"—is a masterclass in technological hubris. Researchers fed Holbein’s drawings into facial recognition software, compared the facial geometry to other period portraits, and declared that the bone structure matches Elizabeth Howard instead of Anne.

It sounds objective. It sounds scientific. It is fundamentally scientifically illiterate. Additional reporting by The Verge explores related perspectives on the subject.

The art world is suffering from a severe case of math envy, eagerly accepting flawed machine learning outputs to bypass the messy, difficult work of traditional connoisseurship. Relying on facial recognition to identify Tudor portrait sitters fundamentally misunderstands both how AI works and how Renaissance art was made.


The Fatal Flaw of Algorithmic Connoisseurship

Facial recognition software operates on a basic premise: it measures the spatial relationships between fixed facial landmarks—the distance between the pupils, the angle of the jaw, the height of the cheekbones. This works reasonably well when you are analyzing a high-resolution, unedited digital photograph of a human being at an airport security gate.

It fails completely when applied to a 16th-century drawing.

When you run an algorithm on a sketch by Hans Holbein the Younger, you are not analyzing a human face. You are analyzing an artist’s interpretation of a human face. You are running a biometric scan on a series of chalk strokes, charcoal lines, and ink washes.

To believe this AI is accurate, you have to accept a ridiculous premise: that Hans Holbein possessed the exact pixel-perfect accuracy of a Sony alpha mirrorless camera.

He didn't. No painter did. Even the greatest masters of the Northern Renaissance compressed proportions, idealized features to please their patrons, and altered angles for better composition. If Holbein drew the same person on Tuesday and then again on Thursday, the facial geometry of those two drawings would not match closely enough to satisfy a modern smartphone lock screen.

Using biometrics on a Renaissance sketch is like using a digital caliper to measure a cloud. You are applying a hyper-precise tool to an inherently fluid medium.


The "Perfect Data" Delusion

I have spent years watching tech companies sell over-engineered solutions to industries that do not need them, convincing executives that data can replace human intuition. The art market is the latest victim. Tech evangelists sweep in, throw around phrases like "neural networks" and "geometric deep learning," and academics fold because they don't want to look left behind.

But any machine learning model is only as good as its training data. Look at the training data available for Tudor-era women.

  • The Pool is Tiny: We have a handful of surviving portraits, many of which are copies of lost originals painted decades later by lesser artists.
  • The Subjects are Misidentified: Half of the portraits in country houses across Britain are mislabeled. We routinely discover that a portrait labeled "Jane Seymour" is actually "Catherine Parr," or vice versa.
  • The Technique varies Wildly: Comparing a quick chalk study by Holbein to a finished oil painting on oak panel by an anonymous studio assistant is an apples-to-rotten-oranges comparison.

When the algorithm compares the "Anne Boleyn" sketch to a portrait of Elizabeth Howard, it isn't comparing two women. It is comparing the stylistic quirks of the artists, the degradation of the paint over five centuries, and the distortion introduced by past restorations.

If you train a model on corrupted, scarce, and heavily stylized data, the output will be pure fiction. The computer will always give you an answer, because that is what it is programmed to do. It cannot throw up its hands and say, "There is not enough evidence to make a claim." It gives you a probability score, and human researchers turn that score into a definitive headline.


Propaganda is Not a Biometric Sample

The biggest blind spot in this entire tech-driven narrative is the erasure of historical context. Tudor portraiture was never meant to be photographic. It was political propaganda, genealogical branding, and marital advertising.

Consider the reality of how these portraits were created. A nobleman or noblewoman did not sit still for eighty hours while an artist meticulously captured every asymmetry of their nose. The artist made a quick preparatory sketch—like the Holbein drawing in question—and then retreated to the workshop to paint the finished portrait, often using standardized templates for bodies, hands, and clothing.

Furthermore, physical features were intentionally altered to conform to 16th-century beauty standards. High foreheads, pale skin, elongated necks, and small mouths were the cultural currency of the era. Every high-born woman wanted to look like the ideal courtly lady.

"Tudor portraiture is a system of symbols, not a collection of passport photos."

If two faces look identical under an AI scan, it is highly unlikely to be because they share a genetic link as mother and daughter. It is far more likely because they were painted using the same workshop template of beauty, or because the artist was adhering to the exact same stylistic conventions. The algorithm sees a match in bone structure; the historian should see a match in cultural expectation.


How to Actually Spot a Tudor: Bring Back Connoisseurship

The obsession with tech solutions has caused a dangerous atrophy in genuine art historical skills. We do not need better code to solve the Anne Boleyn puzzle; we need better looking.

True attribution and identification do not come from a computer calculating the distance between two eyes. It comes from holistic connoisseurship, material analysis, and archival tracking.

Method What It Measures Why It Beats AI
Dendrochronology The growth rings of the wooden panels. Establishes an absolute baseline date for when the object was physically made, preventing modern fakes from entering the data pool.
Pigment Analysis The chemical composition of the paint layers using XRF or Raman spectroscopy. Identifies workshop-specific recipes and chronological anomalies that algorithms completely miss.
Provenance Tracking The physical paper trail of ownership through wills, inventories, and letters. Connects the object to reality rather than stylistic guesswork.
Stylistic Connoisseurship The idiosyncratic, unconscious habits of the artist (e.g., how they draw an earlobe or a fingernail). Recognizes the human hand, which is always variable, rather than assuming a mechanical consistency.

When we rely on facial recognition, we are choosing the easy path. We are letting a black box do the thinking for us so we can print a press release with a definitive headline. It is lazy scholarship wrapped in the shiny veneer of innovation.


The Cost of Algorithmic Certainty

There is a real danger to this trend. Once an algorithm declares a portrait to be "94% likely to be Elizabeth Howard," that percentage becomes canonized. It gets cited in museum labels, written into textbooks, and uploaded to digital archives. Future research is corrupted because the foundational data point has been warped by a machine's hallucination.

I acknowledge the appeal. I understand why a museum or a research team would want to leverage AI. It drives engagement, it wins grants, and it positions an old institution as forward-thinking. But we must be honest about what is happening: we are sacrificing historical accuracy on the altar of tech-bro PR.

Stop looking for a digital silver bullet to solve problems that require deep, human, historical context. The Holbein sketch may be Anne Boleyn, it may be Elizabeth Howard, or it may be a court lady whose name has been entirely lost to time. To pretend a piece of facial recognition software knows the difference isn't progress. It’s an insult to the complexity of history.

Shut down the software. Pick up the magnifying glass. Go back to the archives.

JH

James Henderson

James Henderson combines academic expertise with journalistic flair, crafting stories that resonate with both experts and general readers alike.