Humane Ingenuity 15: Close but Not Quite
by Dan Cohen
The Picture Description Bot, by Elad Alfassa, runs random Wikimedia Commons images through Microsoft’s Computer Vision API, and then posts the best-guess caption that API produces along with the image to the bot’s Tumblr and Twitter feeds. This process was featured in HI3 for archival photographs, although I also included the API’s confidence scores for the caption and associated tags, which is helpful in any overall assessment.
The Picture Description Bot’s close misses are the most revealing and humorous:
Via Jen Serventi, Sheila Brennan, and Brett Bobley of the National Endowment for the Humanities, who posted from the Digging Into Data conference, two interesting search tools:
Dig That Lick lets you play some notes on a virtual keyboard, and then it finds similar melodic patterns in a large database of jazz performances.
I played the first six notes of “Hey Jude,” and it found dozens of instances of that lick embedded in the middle of jazz solos.
(Yes, in the Upside Down of contemporary copyright litigation, technology such as this is being used, in the wake of the “Blurred Lines” case, to analyze every new hit song for potential litigation. No, this is not good for pop music.)
ISEBEL, the Intelligent Search Engine for Belief Legends, is a search engine for folktales from northern Europe:
The focus of ISEBEL is mainly on orally transmitted legends: traditional stories about ghosts, hauntings, devils, witches, wizards, spells, werewolves, nightmares, giants, trolls, goblins and the like, as well as stories about hidden treasures, famous robbers, underground passages and sunken castles. Queries can be made in English, or in the languages the stories are in.
The stories are geolocated and visualized on a map. This helped me see that Danes are really into trolls (the hairy mythical kind, not the current online annoyances), and have many local stories about trolls throwing giant rocks around for sport (which of course explains the boulders in the center of some villages).
Fernando Domínguez Rubio and Glenn Wharton have published a very good article on the future of preserving digital art, “The Work of Art in the Age of Digital Fragility.” The article lays out better than I’ve seen elsewhere the entire preservation process for digital art and the serious problems that museums face. One case study is the Museum of Modern Art’s acquisition of an interactive video artwork called I Want You to Want Me, by Jonathan Harris and Sep Kamvar, which drew on live profile information from dating sites:
Initially, the acquisition of [I Want You to Want Me] followed the standard route museums use to acquire any other artwork. Once the legal paperwork was completed, the museum sent some “preparators”—the museum personnel specialized in moving artworks—to the artists’ studio to collect the custom-made monitor. After the monitor arrived at the museum, it underwent a routine condition assessment to determine whether there was any physical damage in it prior to being sent to its final destination in the museum’s storage facility in Queens.
This time, however, the routine inspection could not be completed. Despite their best efforts, the museum staff could not get the artwork to run on the monitor. No matter how hard they tried, the hard drive attached to the monitor did not produce any data from dating websites. It was only after several attempts that they finally realized that the problem was in fact not technical—since both the monitor and the hard drive were working perfectly—but, alas, ontological. In other words, the problem was not that the museum had acquired a malfunctioning object but that it had acquired a different kind of object. More specifically, the museum had acquired a “distributed object”…
Although Kamvar and Harris wrote the source code for the artwork, the music running in the background was produced by a Canadian band; the data filling the balloons was produced by anonymous users in online dating sites; the touch screen was made by a private company; while the operating system and software on which the artwork runs were produced by different companies.
In addition to these many issues, such preservation raises even larger philosophical questions about today’s art, because most digital art will have to be migrated to new platforms over time, which in a way isn’t preservation, but rather close-but-not-quite replication.
The Faustian bargain that digital offers to the museum: either we let these artworks die, or we keep them alive, but at the cost of embedding artworks in…an environment in which artworks can exist as both copies and originals, regenerated and authentic, past and present.
Speaking of authenticity, although I am not a regular (or even occasional) reader of Harvard Business School case studies, I suspect that HI subscribers might be interested in Ryan L. Raffaelli’s “Reinventing Retail: The Novel Resurgence of Independent Bookstores,” in which he outlines factors in the phoenix-like rise from near-death of local booksellers. I don’t think there’s much surprising in Raffaelli’s analysis, but it’s a good summary of the conventional wisdom, backed by some data and many interviews with successful bookstore owners.
I found this section on human vs. AI recommendations particularly cogent for this newsletter:
Online shopping platforms present consumers with seemingly unlimited inventory. However, research suggests that consumers can become overwhelmed when presented with too many options and seek guidance on how to narrow their choices.
Rather than stocking larger inventories, indie booksellers have mastered the art of “handselling” books that are uniquely tailored to specific tastes of the readers who most frequent their stores. The practice of handselling involves an expert bookseller asking the consumer a series of questions about their recent reading habits, then handing them the “perfect” book (often an unexpected hidden gem not found on popular bestseller lists). To accomplish this task, independent bookstores employ talent who are themselves voracious readers and possess deep knowledge and passion for books. Consequently, booksellers serve the role of matchmaker between a customer and each book on the shelves in the store.
They try to expose readers to up-and-coming authors before anyone else, or steer the reader into genres he or she might not venture into without expert guidance. Booksellers keep an ear to the ground for soon-to-be- bestselling books by monitoring the reading habits of visiting authors, publishers, and their most loyal customers. While artificial intelligence and algorithms are becoming the norm to help retailers anticipate consumer buying behaviors, indie bookstores have been able to counter this trend by offering a unique personal buying experience where the consumer enters into a relationship with a bookseller, often over a series of ongoing conversations about their evolving reading preferences. Artificial intelligence-based algorithms have yet to fully replicate the human experience associated with the art of handselling that successful independent booksellers have mastered.
