How can users experience the Rijksmuseum’s art remotely in new ways that are not available through a museum visit?
The ways we think about and experience art are changing as many recent initiatives spearhead the digitization of art. Large-scale, innovative projects, like Google’s Arts & Culture, originally inspired us to look into this domain. A few high profile museums are making all, or a portion, of their collections available to the public online. (We were drawn to Amsterdam's Rijksmuseum because of its newly released and comprehensive API.) Our drive to empower users to experience this art grew as we learned more about the scale of art kept in museums' storage and, therefore, unavailable to the public.
We were also impressed by creative art dimensions visualizations completed with the MoMA's API and Tate Modern’s API, and color visualizations by Gabriel Gianordoli (left). Furthermore, projects like The Next Rembrandt and Local Projects' museum installation inspired us to incorporate facial recognition.
As the Rijksmuseum's collection is so vast, it took us a while to get acquainted with all that we had access to and narrow in on interesting portions of the data. We created an interface to quickly filter and explore the art. Eventually, we decided to focus on the museum's collection of 4,000 paintings.
We were interested in presenting each painting as a strip of its colors because the artists of The Hague School allegedly had palettes that transitioned from gloomy and gray to lighter and brighter, under the influence of French Impressionism. (However, our visualization did not support this claim.) To accomplish this, we went through every image of every painting in The Hague School and pulled 600 pixels from it. We ensured that the pixels would be a sampling of colors that accurately summarized the painting by selecting evenly-spaced pixels: each x and y position of a pixel had the same ratio as the width and height of the painting, and they stretched across the entire image.
Next, we wanted to sort the pixel colors to make them more easily comparable. As color can be sorted primarily across 3 dimensions (hue, saturation, and value), it proved to be difficult to get a smooth sort in a linear, 2 dimensional representation. We experimented with many strategies, for example sorting by saturation and then subsorting by hue, but this created unattractive bands. In the end, sorting by hue was the most effective option. It was fun to sort colors of filtered portions of the Rijksmuseum's collection. For example, the vertical strips (left) each represent a Rembrandt painting.
We quickly tested a slew of facial recognition APIs on a few Rijksmuseum paintings, but we struggled to find one that was very effective at identifying the faces. However, eventually we settled on Kairos because it accurately located the majority of the faces. It also estimated details such as demographics and emotions, which added some character and amusement to our work.
Once we were able to access and organize the data, our team brainstormed together how to best represent it. We considered timelines, filters, sorting, small multiples, and more (left). We gave form to our rough ideas by drawing on paper and whiteboards. We iterated and revised based on suggestions from one another. After generating many concepts, we were collectively attached to 4. We decided to try and implement all 4 concepts, each as their own "exhibit", but prioritized them, actually finishing the last concept after we had already handed-in our final project!
We added character to our work by incorporating imagery that complemented the paintings from the Rijksmuseum's API. More specifically, we created a vectorized map of Amsterdam (based off of Google Maps) and highlighted various cultural attractions throughout the city.
Our final website showcases 4 “exhibits,” each an exploration of a novel way to empower users to remotely experience the art of the Rijksmuseum through technology. Each is described in detail below. This project was accepted to the IEEE VIS 2017 Conference as a public installation for the Arts Program (VISAP) and as a poster presentation for IEEE Information Visualization (InfoVis). Browse a video summary of our submission and see other accepted projects here. Additionally, a Data Visualization panel at Carnegie Mellon University voted our site the winner of the “Best Project Award” for the Visualization in HCI course, taught by Adam Perer. Visit the site!
This “exhibit” features all of the Rijksmuseum’s 4,000 paintings as thumbnails stacked in histogram form, ordered from the earliest to latest time period. The top of the chart showcases paintings currently on display at the museum, while the bottom displays paintings in storage. It becomes immediately clear that visiting the exhibits only scratches the surface of the impressive collection. Additionally, the number of paintings varies by era in which the art piece was created, revealing many acquisitions of Dutch Golden Age paintings from the 17th century. Looking forward, we are interested in expanding this section to empower users to browse archived work further. Revealing stored art provides an exciting opportunity, as it is one of the few clear advantages technology has over an in-person museum experience. Visit this exhibit!
This “exhibit” features the prominent and exhibited members of the Hague School, a group of artists who lived and worked in the city between 1860 and 1890. Users can compare and explore the artists' work through the paintings' colors and canvas sizes. Each painting is presented as a strip of its colors, ordered by year, to explore the claim that the artists’ palettes transitioned from gloomy and gray to lighter and brighter, under the influence of French Impressionism. (Our visualization did not support this.) The dimensions are all aligned at their bottom left corners. These two visualizations are connected with interactivity, so users can identify which dimensions pair with which colors and observe the paintings in a new light. Visit this exhibit!
Using facial recognition technology, we identified all of the faces within the Rijksmuseum’s paintings. This “exhibit” playfully brings the museum’s characters to life by giving each of the faces a personalized social media page. By clicking through "friends" (other faces identified in the same painting or randomly selected from other paintings), the user can browse new pages and traverse the museum's collection. Visit this exhibit!
This “exhibit” showcases the Rijksmuseum’s most well-known paintings: a set of 60 masterpieces curated by the museum itself. Each painting is broken down into 600 pixels and sorted by hue. Drawing inspiration from Amsterdam's cityscape, the pixels animate across a vectorized map of the city, as if flowing rapidly through its famous canals. The pixels aggregate on a blank canvas, where the full image is progressively revealed. Visit this exhibit!