Rembrandt van Rijn died 350 years ago this October. As museums all over the world (particularly in the Netherlands) are taking his works out of storage for display, they’re also paying special attention to a handful of historically disputed paintings, once thought to have been made by the master, but later deemed to be by his students—and vice versa. While experts in Holland continue to debate the authenticity of some portraits, on the other side of the Atlantic, Steven and Andrea Frank—a technologist and art historian, respectively, living near Boston—have created something they think can help: an artificial intelligence (AI) system trained to differentiate between works made by Rembrandt’s hand and those of his followers.
The Franks think of their project not as a means of replacing art historians and their forensic knowledge of artists’ working methods, but rather as another tool they can use to help determine the authenticity of portraits by Rembrandt and others. The couple’s project, cheekily dubbed A-Eye, was inspired by an article they read last year about using neural networks to detect art forgery.
“The approach seemed kind of limited to me,” Steven said. “In August, I finished a graduate certificate program in AI. That’s when the idea hit me to use small pieces of big, high-resolution images to identify an artist or authenticate a painting. We’ve been researching, programming, and tinkering since then.” Andrea said they chose to focus specifically on Rembrandt portraits for their pilot project because of the “psychological intensity of his paintings,” the history of controversy over their attribution, and the ready availability of extremely detailed images of the works from institutions like the Metropolitan Museum of Art and the Rijksmuseum.
“Activation layers” of A-Eye’s neural network when it analyzed the 400×400-pixel face fragment at the top of Rembrandt’sPortrait of a Young Gentleman. Image courtesy of Steven and Andrea Frank.
The Franks said their AI system is unique in that they trained their computer to recognize a Rembrandt by inputting images of small pieces or “tiles” of a painting, rather than entire paintings. Using high-resolution digital images found on museum websites, they fed information into their AI system of both undisputed Rembrandts and undisputed non-Rembrandts—Dutch portraits, similar in style and from the same period. They trained their AI system with images from 80 paintings: half by Rembrandt and half not by Rembrandt.
“There aren’t thousands of Rembrandts, but if you break one high-res image of a Rembrandt up into hundreds of visually meaningful pieces, you can get thousands of training images pretty easily,” Steven said. And now that the computer “knows” what to look for, when they ask it if a certain painting is a Rembrandt or not, it picks out particular parts to analyze in order to make its determinations.
After months of programming and tweaking, the Franks put A-Eye to the ultimate test, feeding the system a handful of historically disputed Rembrandts. Per the program’s prognostication, anything with a result below 50-percent certainty is deemed not to be a Rembrandt, and the higher the percentage of certainty, the higher the probability that it’s a genuine Rembrandt. The Rijkmuseum’s Portrait of an Old Lady (ca. 1640–45), now attributed to
Ferdinand Bol
, came in at 41 percent; both The Polish Rider (ca. 1655) from the Frick Collection and The Man with the Golden Helmet (ca. 1650) from Berlin’s Gemäldegalerie were deemed 74-percent Rembrandt; and one of the most recently disputed discoveries, Portrait of a Young Gentleman (ca. 1634), owned by the Dutch art dealer Jan Six, came in at 55 percent, barely over A-Eye’s threshold for a genuine Rembrandt.
Rembrandt van Rijn
Self Portrait, 1659, 1659
National Gallery of Art, Washington D.C.
Ferdinand Bol (attributed to), Portrait of an Old Lady, Possibly Elisabeth Bas, c. 1640–45. Courtesy of the Rijksmuseum.
The Franks maintain that these results—which are roughly in keeping with expert opinion on these particular paintings—prove that A-Eye works. While evaluating Portrait of a Young Gentleman, their system zoomed in on the sitter’s intricate lace collar, which Six himself had pointed to as proof that the painting was, in fact, a Rembrandt. Still, some experts are skeptical of trusting an AI to authenticate works.
Milko den Leeuw, founding conservator and researcher at Atelier for Restoration & Research of Paintings (ARRS) and organizer at the nonprofit Authentication in Art (AiA), said that while the Franks’ project is very interesting, “many of these disputes have already been settled by the thorough research done by the Rembrandt Research Project [RRP].” Founded in 1968 and headed by renowned Rembrandt expert Ernst van de Wetering, the RRP served the sole purpose of authenticating Rembrandt works, until it shut down in 2011. Of course, the most recently disputed Rembrandts—like Portrait of a Young Gentleman and another Six discovery, Let the Children Come to Me (1627–28)—were found after 2011.
“I believe, for [Portrait of a Young Gentleman] and other attributions of Rembrandt, AI and similar techniques will not settle disputes on their own,” Den Leeuw added. “What these techniques will do is provide powerful tools to researchers, helping them reach a conclusion on attribution.” (Den Leeuw himself co-wrote a 2017 paper on using AI to distinguish between drawings by
Picasso
,
Matisse
, and
Schiele
and specially made copies.)
A-Eye picks out the most pictorially interesting/meaningful fragments from an original. Image courtesy of Steven and Andrea Frank.
Emilie Gordenker, the director of the Mauritshuis museum in The Hague, believes that new technologies won’t be able to solve problems of attribution so easily by themselves, as their data still need to be interpreted. These kinds of advances, she added, have a tendency to uncover a whole new set of questions, potentially raising doubts about heretofore attributed paintings.
“In a sense, these things have complicated the process,” Gordenker said. “The more tech you throw at it, the more complicated—and also the more interesting—it becomes.”
“We haven’t solved the problem of attribution,” Steven Frank said, adding that A-Eye is still very much a work in progress. “We definitely aren’t looking to monetize the technology—we hope lots of people use it and do great things with it. What we’d like to do next is to see if our system can identify areas of a single painting that may have been painted by different artists.” (Coincidentally, Den Leeuw cited the task of identifying various different hands in a single work and tracing them through other paintings as an exciting potential use for AI in art authentication.)
This isn’t the first time AI has been used to analyze Rembrandt’s oeuvre, either. In 2016, an advertising firm in Amsterdam unveiled “The Next Rembrandt,” a 3D-printed painting of a middle-aged man in a lace collar and black hat, the result of an AI project that involved feeding a computer data from hundreds of paintings. The computer processed the data in order to create the new painting, which, at first glance, resembled a Rembrandt portrait. Although the project’s creators maintained that the new technology was not a means of creating a forgery—which, as they were the first to admit, would be immediately detected—they hoped it might be helpful in aiding restoration work. (Rembrandt experts were unamused.)
Andrea Frank noted that after Rembrandt, she and Steven hope to tackle the works of
Leonardo da Vinci
. But authenticators need not worry about their jobs just yet. Like infrared reflectography and other forms of non-invasive imaging, AI is just another tool to add to their kit.
[“source=artsy”]
