Archive

[Place mat by a student of the textile artist Mariska Karasz, c. 1950-60, via Cooper Hewitt/Smithsonian Design Museum.]

An update: Earlier this year I wrote a piece for The Atlantic on an important court case between publishers and the Internet Archive over the lending of digitized books. The publishers won that case, and based on some further developments on Friday, it sounds likely that IA will have to stop loaning books published by the large publishers who dominate the market, as well as others who object. Yesterday, the New York Times covered this ongoing battle. (In related news, the big record labels filed a similar lawsuit against IA on Friday opposing IA's program to digitize and stream old 78rpm albums.)

I have nothing to add to my Atlantic article, which I think explains the issues in depth, except to note that in almost all of the coverage of the conflict, including that piece in the NYT, there so many words about publishers, writers, and libraries, and so few words about readers. To me, readers are the most important constituency, and yet they remain undercovered and mostly voiceless. We should be asking: What is the best ecosystem of books (and ebooks) for readers?

[A still from the video artwork "Signs Facing the Sky" by Allora & Calzadilla, from the DVD-R held in UC San Diego's Special Collections & Archives.]

In his most recent and perhaps strangest change to Twitter, Elon Musk curtailed access to the service itself, reducing the number of tweets users can view in a day, and requiring everyone to log in to see tweets that were once publicly available. He justified these new limits, which seem at odds with advertising-based media that seeks the largest possible audience, by pointing to the tech obsession of our time, artificial intelligence — specifically, its insatiable need for large amounts of texts and images on which to train its powerful automated systems. Bots are out there copying collections of writing, art, and photographs, Musk fumed, and Twitter — like Reddit before it, with its forums providing millions of posts for these new machines to chew on — shouldn’t be a free meal for the hungry maw of AI.

There likely were more mundane reasons for Musk’s sudden restrictions, including a degradation of Twitter’s infrastructure after relentless layoffs and a failure to pay its bills, but Musk is not wrong about the growing need for human-created outputs to input into AI models. The more data these models acquire, the stronger they become.

However, in the quest for these training sources, many AI purveyors now want to look in a very different direction than low-quality tweets: the high-quality, peer-reviewed, deeply researched works of academia. Forget the dross of social media and web forums; AI is coming for the gold standard of scholarship next — the articles of academic journals, the books on the shelves of research libraries, the data recorded in university labs.

As we await the next generation of engineered writing, of tools like ChatGPT that are based on large language models (LLMs), it is worth pondering whether they will ever create truly great and unique prose, rather than the plausible-sounding mimicry they are currently known for.

By preprocessing countless words and the statistical relationships between them from million of texts, an LLM creates a multidimensional topology, a complex array of hills and valleys. Into this landscape a human prompt sets in motion a narrative snowball, which rolls according to the model's internal physics, gathering words along the way. The aggregated mass of words is what appears sequentially on the screen.

This is an impressive feat. But it has several major problems if you are concerned about writing well. First, a simple LLM has the same issue a pool table has: the ball will always follow the same path across the surface, in a predictable route, given its initial direction, thrust, and spin. Without additional interventions, an LLM will select the most common word that follows the prior word, based on its predetermined internal calculus. This is, of course, a recipe for unvaried familiarity, as the angle of the human prompt, like the pool cue, can overdetermine the flow that ensues.

("The Library of the Distant Future," as envisioned by Midjourney, when I was let into the beta in March 2022.)

Before one can become a Cassandra or Pollyanna about the uses or abuses of impressive text-to-image AI tools like DALL•E and Midjourney, it is worth stepping back and reflecting about the fundamental nature of this new technology. What is it actually designed to do?

Just as text generators like GPT-3 are engineered to provide highly plausible sequential arrangements of words, these AI image generators are designed to meet our expectations, visually. This agreeableness is right there in the math, in the way these tools distill millions of images into a multidimensional array of the proximities of various styles and shapes. They angle to be familiar, and from what we have seen so far, they are succeeding.

(Sara Gothard, Library of Babel, 2017, in the Jamaica Plain branch of the Boston Public Library, part of their recently digitized art collection.)

We have become familiar with how technology, media, commerce, and forms of human expression are deeply intertwined. Streaming music services and apps like TikTok, and the models behind them, encourage the production of shorter songs that begin with the catchiest riff in the track, so as to maximize quick streams and thus revenue; similarly, when radio airplay was the primary way to push the sales of singles, it helped if the lyrics of a potential hit began with the title of the song, sung before the dial could be turned (just ask the incomparable Nile Rodgers). Despite the protestations of the New York Times art critic, NFTs clearly encourage a specific kind of art, namely one with slight variations on a theme rather than artistic diversity — even on serious subject matters — since the centrality of crypto exchange prods artists to think about the community of owners rather than other audiences for their art, like art critics or the general public.

Whenever I check out a library book that has been underlined or annotated, I think about the two anonymous students who aggressively marked up Widener Library's copy of Rollo May's Man's Search for Himself:

I hope these two students did in fact meet at some point, although they may have been separated by decades. It would make for a good short story or film (or U2 song).

I also happen to love this passage from Rollo May's book, which is incredibly relevant to the Humane Ingenuity newsletter.

(Noah Kalina, Lumberland / 20180716)

Noah Kalina is a gifted photographer who has a commercial practice and also works as an artist. He is probably best known for his Everyday project, in which he has been taking a photograph of himself each day for the last two decades. I am more interested in his nature photography, which is uniformly gorgeous. Noah lives in Lumberland, in upstate New York, and his photos across the seasons — of a single tree or river bend — are evocative and engrossing.

I want to buy a print of one of these photographs, but I can't, for reasons you can probably imagine, since it is 2021: these remarkable images are only available as NFTs. Thus far, as I write this newsletter, Noah has sold 16 Lumberland NFTs, for a total of 13 ETH (Ether cryptocurrency), which is about $55,000.

Here are sixteen "sketches of a 3D printer by Leonardo da Vinci," as envisioned by AI using those words as a prompt:

By Rivers Have Wings and John David Pressman, using a CLIP-guided diffusion.

I'm back from a summer hiatus — perhaps not into the carefree fall I (and you) had hoped for. But with students streaming once again into my library, the beginning of this academic year still has that rejuvenating anticipation of new experiences and encounters — a prompt for all of us to shake out of our complacency, to open ourselves once again to new ways of seeing.

Seeing as an underexplored, strange experience animates the art of James Turrell. Our family made one of our pilgrimages to Mass MoCA to see the new Turrell exhibit "Into the Light," which I recommend if you can make the journey to the far northwestern corner of Massachusetts. The exhibit restages some of his classic approaches to abstract lightwork, including a room where a floating pink cube is actually, somehow, an inset into a curved wall, and darkened spaces with just enough reflected light to confuse and, ultimately, enthrall.

(Alice Baber, Noble Numbers, 1964-1965, acrylic on canvas, Smithsonian American Art Museum.)

The Leventhal Map & Education Center has a new tool called Moviemaps that allows you to pair an explanatory video with a map (or maps) in a separate window, and that can contain hidden triggers within the video that zoom and pan the adjacent map to highlight certain elements. At the same time, viewers can diverge at any time from the guided script to explore the map on their own. It's a good model and a clever use of IIIF (the International Image Interoperability Framework).

Here's a fun, brief introduction by Garrett Dash Nelson, with zooming and snapping hand motions reminiscent of Minority Report:

Sixty years ago, illustrator Arthur Radebaugh drew scenes from the future — that is, our present — including, quite presciently, remote education and work, self-driving cars, and an "electronic home library." His Sunday strip that ran in newspapers, including the Chicago Tribune, was called "Closer Than We Think."

If there’s one thing we’ve learned about the many datasets we’ve wrestled with this year, it’s that all the data — every single point — is the result of human decision-making.

These essential words are the lede of a great reflection by Erin Kissane, a co-founder of the COVID Tracking Project and CTP’s managing editor. The project is a terrific case study in humane ingenuity, because what seemed like a straightforward data and technology project — tracking COVID cases across the United States — was in fact primarily animated by skills from the humanities and deeply imbued with a humane spirit.

As majors have sharply declined over the last decade, a thousand verbose defenses of the humanities have been published. But as all writers should know, it’s better to show than to tell, and CTP did a damn good job embodying key methods and ethical choices from the humanities. The project also made the implicit point that data work doesn’t belong, by default or fiat, to STEM fields.

CTP put those values into practice through the foregrounding of uncertainty, context, and care. Although the project compiled reams of numbers, they refused to let those numbers drift off into pure quantitative metrics, and they always noted the potential fallibility of each digit. Human error, at the state or local level or within the project itself, the peculiarities of health reports or highly variable definitions, were all measured and analyzed by CTP staff and volunteers. The data was not just accumulated into a spreadsheet; it was tightly coupled with careful interpretation, glosses, and a close reading of primary sources.

In a wonderful new article, film and television scholar Jason Mittell provides an extremely creative, occasionally bizarre, frequently hilarious, and ultimately rather helpful “inventory of deformative practices” to uncover hidden layers of meaning in media. These practices use the malleability of digital formats to convert traditional media, like films, into new forms that provide insight into their art.

Or put less academically: What can we learn about staid video culture from TikTok and GIFs, or the stranger, more elastic memes enabled by contemporary video editing software?

Mittell chose a perfect film to run transformative digital experiments on: the canonical musical Singin’ in the Rain. In one experiment, for instance, he used software to isolate Gene Kelly’s hands and feet, which, in masking the rest of his dancing body and the set in black, shows Kelly’s talent and energy literally in a new light:

The Fleet Library at the Rhode Island School of Design has digitized their collection of books created by artists. It is an exhibit of the infinite malleability of the creative technology we call the book:

(Jeannie Meejin Yoon, Hybrid Cartographies: Seoul's Consuming Spaces, 1998)

(Julie Chen, Radio Silence, 1995.)

(David Stairs, Boundless, 2013)

Although we regularly use and rely on numbers, human beings are simply not very good at understanding them. Most of us can effortlessly feel the shading of a spoken word. But present us with big numbers, or probabilities, and we have no idea how to put them into context, or properly understand their scale or likelihood.

The pandemic has only underlined this mental deficiency. How can we comprehend the death toll, or the risk factors, in purely mathematical terms?

In the field of data visualization, there has been a movement to make numbers more vivid, and their scale more comprehensible, through physical representations. The sea of empty chairs, for instance, shows us, rather than tells us, about all of those who have died from Covid.

Recently there was a conversation at Northeastern University on “,” moderated by Dietmar Offenhuber, whose own work translates the physical into data, as in . The conversation had some great examples of the effective — and affective — representation of data in the real world, and how to take advantage of more visceral sensory responses, rather than abstract mental models, to understand the import of numbers.

A visualization of the colors of the objects in our lives over the last two centuries:

Cath Sleeman took the digitized images of household and commercial artifacts in the Science Museum (UK) and distilled them down to the colors of their pixels. This process ended up documenting the loss of colorful items around home and work spaces, in favor of objects covered in more drab shades of black and white. Spunky colors like red and yellow, once very prevalent among objects of the Victorian era, have been nearly pinched out of existence. On the positive side for those who enjoy more expressive colors, purple and blue, once rare, have made some progress in material culture since 1960s, and especially since around 1980. (Hypothesis: Prince.)

As Sleeman notes, the shift in the palette we see in our artificial environments is due in large part to the rise of plastic, and the related decline of wood, leather, and other natural materials. But there are also aesthetic choices at work. For instance, phones used to come in colors other than black and white:

If for some reason you could use some relaxation right now, I recommend heading over to Faint Signals, an interactive work of art that was one of the clever entries in the annual competition run by the British Library Labs for creative reuses of their collection.

Faint Signals generates an imagined Yorkshire forest, which you can then explore through the seasons. As you meander through the digital woods, peaceful natural sounds from the British Library's extensive audio collection—birds, rain, wind—are encountered. Faint Signals doesn't exactly rival the real signals of the real thing, but HIers, it's winter, and there's a pandemic still going on. So put your headphones on, turn your phone off, and take a leisurely stroll through the virtual forest.

Pattern recognition, as it was practiced before computers:

(Via William J. Paisley, "The Museum Computer and the Analysis of Artistic Content," in Computers and Their Potential Applications in Museums, proceedings of a 1968 conference at the Metropolitan Museum of Art, sponsored by IBM. Paisley calls this "empirical connoisseurship," which I rather like. Also, this study made me realize how strange and slightly creepy Botticelli's hands are, which is very odd when you think about it.)

The Vienna Museum has just put online 47,000 objects and 75,000 images, with the vast majority of them available to freely download and reuse. Kudos to Evi Scheller, Head of the Online Collection at the museum, and her team, for this release.

(Wilhelm Bernatzik, Die Flamme, 1902, Foto: Birgit und Peter Kainz, Wien Museum, CC BY 3.0 AT.)

The collection is filled with great art and artifacts, but as a Victorianist and a historian of science, I was especially attracted to over 1,500 rare photographs of the 1873 World's Fair. So many new technologies in the process of rapidly changing the world, for better and for worse.

(Weltausstellung 1873: Maschinenhalle, Deutsches Reich (Nr. 819), Foto: Joseph Lowy, Wien Museum, CCO.)

Like you, I've been spending a lot of time near home this year. Without the stimuli and novelty of travel, I've tried to be more aware of my well-trodden surroundings, like the small plaques that Boston's sidewalk masons used to proudly embed in their work.

Good craftsmanship, and worthy recognition, all these decades later.

In 1986, on the 900th anniversary of the completion of the Domesday Book, the comprehensive survey of England after William was done conquering it, the Domesday Project attempted to recreate this record. Instead of trusty vellum, the project used the not-so-future-proof LaserDisc, attached to an even-less-future-proof BBC Microcomputer.

It’s stressful out there; maybe some of you could use a little levity right now. My old colleagues at the Digital Public Library of America, along with our international friends at other expansive digital libraries, including Europeana, Trove, and Digital NZ, are running the fun GIF IT UP competition again this year. Contestants take open access digitized materials from libraries, archives, and museums, and turn them into whimsical GIFs.

Sorry that it’s been over a month since I last wrote. I’ve been working overtime with my colleagues to reopen a large library that adheres to Covid safety rules while also providing the community with the resources and services they need. Not easy.

And “reopening” is not the right word. We’ve been open all along, but simply shape- and medium-shifting as needed throughout this dreadful year. But I’m writing to you from within Snell Library at Northeastern University, and it feels pretty good. Onward.

Next month in Barcelona at the PH21 Gallery there will be an exhibit of photography documenting the aching feeling of being alone in normally crowded urban spaces during this pandemic.

Zoltán Dragon,

If you need a break, have been at home for a very long time, or are sick of the view out of your window, you can try Window Swap, which shows a video clip of the view out of someone else’s window.

Over the past month, our library has been discussing ways to address—and more concretely take action to oppose—racism in the aftermath of the murder of George Floyd. We have some ideas that we will be accelerating work on this summer and that I’ll talk about in future HIs, but we also have many existing projects that we can build upon. A cluster of these projects work to shed light on the long history of systemic racism in the United States, to show how George Floyd’s death is, tragically and outrageously, yet another case in what seems like an endless line of countless cases.

But counting and making sure we have fully documented each case—each one a human being with family and friends, who had their life brutally taken away—is necessary. We recently rebroadcast our podcast episode on Professor Margaret Burham and the Civil Rights and Restorative Justice project, which is doing this hard work with the assistance of our library and archives (and generous funding from the Mellon and Ford Foundations), not only to bear witness to thousands of racially motivated killings, but to bring what they discover back to the cities and towns where the killings happened for communal discussion and memorialization. I encourage you to spend 30 minutes to , or visit .

Carrie Ferrin, the first female bicyclist in Nobles County, Minnesota. Photograph by E. F. Buchan, c. 1880. (From the Nobles County Historical Society, via Minnesota Reflections/Digital Public Library of America.)

My life has become a series of meetings on "reopening," a word I don't much like. We need to be more precise with our words during this crisis, and open/closed is a poor and false dichotomy, just as "social distancing" is horrible branding for temporary physical separation.

Yes, people are re-enacting and re-creating artworks in their homes during the quarantine. No, this is not a new pastime—people have been doing this for years—and while it’s fine for a while, there are more creative ways to reuse art.

The Rijksmuseum—which may have started the art re-enactment craze some years ago—has been especially inventive on this front. Forget duplicating paintings for Instagram—they encourage people to rethink and remix their artworks across multiple media. (And as I noted in , the Rijksmuseum also digitized their artworks relatively early, thus allowing for this kind of wide experimentation.)

, Egyptian, Old Kingdom, Dynasty 6, 2323–2150 BCE. Carved limestone. Harvard Art Museums.

Drawing of the set for John Taverner's opera Thérèse, designed by Alan Barlow, 1979, via the Victoria and Albert Museum's opera exhibit.

What would you save from 2020 to inform future generations about what we are going through? A number of formal and informal collecting projects have launched, and are worth tracking.

Viola Canady, , Anacostia Community Museum. CC0 photograph from .

Metamorphic library table-steps, by Thomas Sheraton, c. 1795. (CC0-licensed image from the Smithsonian Institution’s new open access collection, which I’ll be writing about in the next issue.) A piece of furniture that is both useful and a metaphor.

A rather nice letterpress QR code from Northeastern University’s traditional print technology lab, Huskiana Press. (Via Ryan Cordell, who is the founder and proprietor of Huskiana. It’s great to have this on our campus.)

David Fletcher is a video game artist in London who on the side creates hyper-realistic 3D photogrammetry models of cultural heritage sites and works of art, architecture, and archeology. I particularly like how he captures soon-to-be-obsolete aspects of the city he lives in, and our modern life, like the beautiful cab shelters for hackney carriage drivers:

()

The Picture Description Bot, by Elad Alfassa, runs random Wikimedia Commons images through Microsoft’s Computer Vision API, and then posts the best-guess caption that API produces along with the image to the bot’s Tumblr and Twitter feeds. This process was featured in HI3 for archival photographs, although I also included the API’s confidence scores for the caption and associated tags, which is helpful in any overall assessment.

The Library of Necessary Books. An where visitors can leave their favorite books. ()

Happy New Year, and welcome to 2020! My constant reminder of the passage of time is a small lake near where we live, which transforms itself delightfully month by month, season by season.

In this issue of HI: dispatches from the frontiers I traversed at the fall meeting of the Coalition for Networked information.

The National Gallery of Denmark has a nicely designed new website that makes all of their digitized artworks openly available, and about two-thirds downloadable under a public domain declaration. The rest is under copyright but can still be downloaded at a generously high resolution and can be used for non-commercial purposes, like this newsletter. Hence: Henning Damgård-Sørensen's "Maleri VI, 2004," above. They also have an API and multiple ways to search the collection, including by color. So go on and add a rotating series of paintings to your website that match its palette exactly.

Middle-Aged Software

It's getting to be that time of the semester when extracurricular activities, like writing this newsletter, become rather difficult. My day job as a university adminstrator has many to-dos that crescendo in November; I will not trouble HIers with most of these, although I've also been on a special detail this fall co-chairing an initiative to highlight and expand our efforts to combine technical/data skills with human skills, about which I will write in this space in due time. It's very much in the spirit of Humane Ingenuity.

"Desakyha," Artist unknown, Cornell Ragamala Paintings Collection.

Carlotta Corpron (1901-1987) was an artist and photographer who used light to create abstract works from which figures would sometimes emerge. (“Strange Creature of Light,” 1948, via the International Center for Photography.)

René Descartes designed a deck of playing cards that also functioned as flash cards to learn geometry and mechanics. (King of Clubs from Via the Beinecke Library, from which .)

One of the best ways that we can react to new technology, to sense its contours and capabilities, and also, perhaps slyly, to assert our superiority over it, is to get weird with it. There is a lot of heavy thinking right now about the distinctions between artificial intelligence and human intelligence, but sometimes we just need to lighten up, to remember that human beings are oddballs and so we can’t help but use technology in ways that cannot be anticipated. And this weirdness, this propensity to play with technology, and to twist it to human, and humane, ends, should be taken seriously.

In 1993, the artist Spencer Finch, fresh out of RISD, started playing around with a Mac, a VCR, and a Radio Shack’s haul of other technology, including a directional radio wave transmitter, and he came up with “Blue (One Second Brainwave Transmitted to the Start Rigel).” When I first saw it in 2007 at Mass MoCA, it made me smile.

I am very fortunate to live a short drive from Walden Pond, of Henry David Thoreau fame. With the hordes of summer tourists finally thinning out, and with the leaves changing with the arrival of fall, it's a good time to stroll around the pond, which we did last weekend.

Those who haven't taken the walking path around Walden Pond before are generally surprised by several things: 1) it's rather small; 2) train tracks run right next to the walking path on one side of the pond; and especially 3) Thoreau's cabin is not that far off the road, and within trivial walking distance of the center of Concord. If Thoreau were alive today, he could, on a whim, go grab some nice warm coffee and a book at a really good book store, and be back in the woods in time to light a fire for dinner.

Welcome back to Humane Ingenuity, a continuing exploration of technology that helps rather than hurts human understanding, and human understanding that helps us create better technology.

[3D-printed zoetrope from The National Science and Media Museum’s Wonderlab, via Sheryl Jenkins]

Welcome back to Humane Ingenuity. It’s been gratifying to see this newsletter quickly pick up an audience, and to get some initial feedback from readers like you that can help shape forthcoming issues. Just hit reply to this message if you want to respond or throw a link or article or book my way. (Today’s issue, in fact, includes a nod to a forthcoming essay from one of HI’s early subscribers.)

OK, onto today’s theme: what can artificial intelligence do in a helpful way in archives and special collections? And what does this case study tell us more generally about an ethical and culturally useful interaction between AI and human beings?

Today Apple will release new iPhones and other gizmos and services, and as they do every year, the tech pundits will ask: "Does this live up to the expectations and vision of Steve Jobs?" I, on the other hand, will ask: "Does this live up to the expectations and vision of Jef Raskin?" Apple likes to imagine itself as the humane tech company, with its emphasis on privacy and a superior user experience, but the origins of that humaneness—if it still exists beyond marketing—can be traced less to the ruthless Jobs than to the gentler Raskin. Jobs may have famously compared a computer to a "bicycle for the mind," but Raskin articulated more genuinely a desire that computers be humane and helpful instruments.

To be clear, Raskin, like Jobs, wanted to sell millions of personal computers, but only Raskin worried aloud about what would happen if that seemingly ridiculous goal was achieved: "Will the average person's circle of acquaintances grow? Will we be better informed? Will a use of these computers as an entertainment medium become their primary value? Will they foster self-education? Is the designer of a personal computer system doing good or evil?" It is remarkable to read these words in an internal computer design document from 1980, but such reflections were common in Raskin's writing, and clearly more heartfelt than Google's public, thin, and short-lived "Don't be evil" motto.

Jobs may have dabbled in calligraphy and obsessed over design, but Raskin was the polymath who truly lived at the intersection of the liberal arts and technology. In addition to physics, math, and computer science, Raskin studied philosophy, music (which he also composed and performed at a professional level), and visual arts (he was also an accomplished artist). He clearly read a lot, which was reflected in his clear and often mirthful writing style, flecked with nerdy guffaws. (The end of one of his long Apple memos: "Summery: That means fair, warm weather, just after spring.") He wrote a book on user interface design called The Humane Interface and sought to build a new computing system called The Humane Environment. For the purposes of this newsletter, and for some ongoing conversations I would like to have with you about the ethical dimension of technological creation, he is one important touchstone.

(Jef Raskin with a model of the Canon Cat, photo by Aza Raskin)

An increasing array of cutting-edge, often computationally intensive methods can now reveal formerly hidden texts, images, and material culture from centuries ago, and make those documents available for search, discovery, and analysis. Note how in the following four case studies, the emphasis is on the human; the futuristic technology is remarkable, but it is squarely focused on helping us understand human culture better.

Gothic Lasers

If you look very closely, you can see that the stone ribs in these two vaults in Wells Cathedral are slightly different, even though they were supposed to be identical. Alexandrina Buchanan and Nicholas Webb noticed this too and wanted to know what it said about the creativity and input of the craftsmen into the design: how much latitude did they have to vary elements from the architectural plans, when were those decisions made, and by whom? Before construction or during it, or even on the spur of the moment, as the ribs were carved and converged on the ceiling? How can we recapture a decent sense of how people worked and thought from inert physical objects? What was the balance between the pursuit of idealized forms, and practical, seat-of-the-pants tinkering?