What do machines see when they look at art?

Cabot Master Class

A major task for museum staff is to describe objects in their collections. These descriptions tend to be factual and typically answer the who, what, when, where, how, and why questions about an object. Very rarely do these descriptions tell you what is actually depicted. There are two main reasons why this is the case. It can be quite time consuming to describe everything you see in an object and it is highly subjective. So what if we defer to a computer and ask a few computer vision services to look at and describe the museum’s art objects? For the past two years the Harvard Art Museums staff have been doing just this. They’ve been exploring alternative methods for describing the museum’s collections using machine processing and computer vision. During this talk we’ll dive in to the project, explain what we’ve implemented, share what’s been learned, and where we’re headed with it.