{"id":7035,"date":"2016-06-16T11:48:59","date_gmt":"2016-06-16T09:45:20","guid":{"rendered":"http:\/\/blog.museunacional.cat\/?p=7035"},"modified":"2019-02-26T07:56:17","modified_gmt":"2019-02-26T07:56:17","slug":"computer-vision-as-applied-to-works-of-art-a-new-perspective","status":"publish","type":"post","link":"https:\/\/blog.museunacional.cat\/en\/computer-vision-as-applied-to-works-of-art-a-new-perspective\/","title":{"rendered":"Computer vision as applied to works of art: a new perspective"},"content":{"rendered":"
Pilar Rosado<\/h6>\n
\"<\/a>

Image with SIFT descriptors (Scale Invariant Feature Transform, Lowe, 2001) de Pilar Rosado.<\/p><\/div>\n

We have invited Dr Pilar Rosado to talk about the most outstanding aspects of her research on computer vision as applied to art, the subject of her PhD thesis. This is a post of a specialization that is maybe unusual in blogs, but clearly explained with very explanatory images, which we think is very interesting for the possibilities of new visual connections between works of art.<\/em><\/p>\n

Computer vision techniques make available new possibilities of approaching the collections of art institutions. Here I present this alternative point of view that may help to stimulate the interest of the public and scholars.<\/p>\n

Using computer vision models that emulate the initial stages of human visual perception, we have managed to categorize large groups of images from abstract works of art by the artist Antoni T\u00e0pies. The images have been analysed based on their compositional, colour and organizational characteristics<\/strong>, without the need to make any textual notes about them, so that the analogies found may take us closer to, and help us to understand, the creator\u2019s original values.<\/p>\n

The whole is greater than the sum of the parts<\/h3>\n

Artists are specialist image generators; in their creative process they produce them constantly. In the collections of images by artists there are some links, formal relationships that make them a family of common meaning<\/strong>.<\/p>\n

\"Storage<\/a>

Storage room of the MUSAC (Museum of Contemporary Art Castilla-Le\u00f3n). Artworks by Daniel Verbis. Photo: Jes\u00fas F. Salvadores, Diario de Le\u00f3n<\/a><\/em><\/p><\/div>\n

The arrangement and categorization of a large collection of images gives us an inestimable overall view of the whole. In the field of artistic creation and its teaching, it is a very common practice to place groups of photographic images, sculptures or paintings next to one another. It is surprising to observe how the visual objects establish a dialogue with one another, sometimes in harmony and others in contradiction<\/strong>. Thus the artist, the professor or the art student has information <\/strong>available that, treating the same objects separately, would be inaccessible<\/strong>. This aspect is especially valuable when studying abstract images, given that in them the subject, the meaning or the sense is not the product of a social pact, but of visual resonances and synchronies that the artist-creator relates and connects<\/strong>.<\/p>\n

New technologies: computer vision algorithms<\/h3>\n

The new technologies make it possible to digitize this content and make quick access to their visualization easier. But to seek genuine relationships between<\/strong> images, with visual content of their own, more sophisticated search tools are necessary<\/strong>, which require no textual notes, seeing as this very writing would condition the relationships that the image might establish a posteriori. A sub-field of artificial intelligence, computer vision, performs this task<\/strong>.<\/p>\n

From pixels to visual resonances<\/h3>\n

How can we extrapolate information about the overall characteristics of an image, starting only with certain pixels?<\/p>\n

In 1997, the neurobiologist Tanaka<\/a> published his conclusions about the neuronal mechanisms of the recognition of objects in primates, and in 2001, computer vision expert Lowe<\/a>, based on this information, described a system of computer vision based on descriptors like scaling, translation, rotation and illumination, and he called them SIFT (Scale-invariant feature transform<\/em>) descriptors. He found the best compromise solution between performance and speed using a sample grid of 16 x 16 pixel gradients. Each of these small regions would summarize the information of the region surrounding the image.<\/p>\n

We could thus say that the computer algorithm, through this model, \u201ccuts up\u201d all the images in the collection into small fragments of 16 x 16 pixels<\/strong> and groups them together according to the mathematical distances between their SIFT descriptors. We call each one of the groups established a \u201cvisual word\u201d and the whole makes up the \u201cvisual vocabulary\u201d of the collection.<\/p>\n\n\n\n
\"Foto2\"<\/a><\/td>\n\"Foto3\"<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

With this information, the system is able to determine concurrences of these \u201cvisual words\u201d in certain images<\/strong>. For example, in portraits, two eyes, a nose, a mouth and a forehead concur. The system will group all the images of faces together in the category \u201cportrait\u201d, and this category would be the one we call the \u201clatent aspect\u201d.<\/p>\n

\"Foto4\"<\/a><\/p>\n

Computer vision applied to the work of Antoni T\u00e0pies<\/h3>\n

The Antoni T\u00e0pies Foundation<\/a> in Barcelona has kindly allowed me access to the collection in order to carry out this study.<\/p>\n

I have applied computer vision models to a group of 434 images<\/strong> of works by the artist from different periods, done with multiple techniques on varying mediums \u2013 engravings, watercolours, drawings, paintings and collages<\/em>.<\/p>\n

I shall now briefly show and describe some of the latent aspects detected by the algorithm, with no manual intervention whatsoever.<\/p>\n

Latent aspect 1<\/strong>: Has a large amount of empty space, measured, with different, very studied motifs, located in a more or less central or balanced position.<\/p>\n

\"Aspecte<\/a>

1. Llull-T\u00e0pies, 1985. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photo: \u00a9 Llu\u00eds Bover, 2016 2. Cartes per a la Teresa, 1974. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photo: \u00a9 Llu\u00eds Bover, 2016 3. Terra d’ombra I, 1998. \u00a9 Fundaci\u00f3 Antoni T\u00e0pies, Barcelona \/ VEGAP. Photo: \u00a9 Gasull Fotografia, 2016<\/p><\/div>\n

Latent aspect 2<\/strong>:\u00a0Contains works done with a vigorous dark-coloured line on a lighter background.<\/p>\n

\"aspecte<\/a>

1. Dans la double d\u00e9pendance du dit, 1984. 2. A. T\u00e0pies. Cat\u00e0leg. Sala Gaspar, Barcelona, 1969. 3. Air, 1971. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photos: \u00a9 Llu\u00eds Bover, 2016<\/p><\/div>\n

Latent aspect 3<\/strong>:\u00a0Groups together works that have a scattered distribution of clear areas of flatter ink, although not completely smooth, with greyer areas in which the brushstroke is not wholly dense (textured brushstroke).<\/p>\n

\"Aspecte<\/a>

1. Int\u00e9rieur avec figures, 1987. 2. L’estaci\u00f3, 1984. 3. Ce que je vous dis trois fois est vrai, 1982. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photos: \u00a9 Llu\u00eds Bover, 2016<\/p><\/div>\n

Latent aspect 4<\/strong>:\u00a0Agglutinates works that have a dotted surface or on which the paint seems to have been vaporized, in general, granulated texture.<\/p>\n

\"Aspecte<\/a>

1, 3 i 4. Tres aiguaforts, 1943. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photo: \u00a9 Llu\u00eds Bover, 2016. 2. Origen. S\u00e8rie Hist\u00f2ria Natural, 1950-1951. \u00a9 Fundaci\u00f3 Antoni T\u00e0pies, Barcelona \/ VEGAP. Photo: \u00a9 Gasull Fotografia, 2016<\/p><\/div>\n

Latent aspect 5<\/strong>:\u00a0Is made up of works with few broad gestural lines. They are dark brushstrokes on a lighter background. The tendency is for the principal motif to be in the centre, covering the entire image.<\/p>\n

\"Aspecte<\/a>

1. S\u00e8rie negre n\u00famero XIV, 1967. \u00a9 Fundaci\u00f3 Antoni T\u00e0pies, Barcelona \/ VEGAP. Photo: \u00a9 Gasull Fotografia, 2016. 2. S\u00e8rie negre n\u00famero III, 1967. \u00a9 Fundaci\u00f3 Antoni T\u00e0pies, Barcelona \/ VEGAP. Photo: \u00a9 Gasull Fotografia, 2016. 3. \u00c9quation, 1987. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photo: \u00a9 Llu\u00eds Bover, 2016<\/p><\/div>\n

Latent aspect 6<\/strong>:\u00a0Presents images with a diffuse atmosphere and a hazy, blurred treatment.<\/p>\n

\"Aspecte<\/a>

1. Esb\u00f3s, 1958. \u00a9 Fundaci\u00f3 Antoni T\u00e0pies, Barcelona \/ VEGAP. Photo: \u00a9 Gasull Fotografia, 2016. 2. Llull-T\u00e0pies, 1985. \u00a9 Comissi\u00f3 T\u00e0pies, VEGAP. Photo: \u00a9 Llu\u00eds Bover, 2016. 3. Dibuix, 1948. \u00a9 Fundaci\u00f3 Antoni T\u00e0pies, Barcelona \/ VEGAP. Photo: \u00a9 Gasull Fotografia, 2016<\/p><\/div>\n

The system learns<\/h3>\n

An added benefit of the arrangement of the collection is that once the latent aspects have been established as categories, the system \u201clearns\u201d<\/strong>; this model would automatically make it possible to classify images from other collections, according to the aesthetic criteria of the categories found.<\/p>\n

It is impossible to do this kind of study on large collections in another way, because of the obvious limitations of access to the works and of human analytical capacity. And, even though any categorization is by definition subjective by being based on a certain criterion, digital technology offers us this way of approaching information that would otherwise be impossible due to the huge amount of visual content stored.<\/p>\n

The system that I have described would make it possible to repeat the studies on different periods of the same artist, or on collections by different artists or periods, with the same criteria<\/strong>. In this way, the results obtained could be compared without the risk of making subjective interpretations conditioned by prior preferences or knowledge. A consultation tool of this kind made available to museum visitors would enable them to establish their own aesthetic selection criteria.<\/p>\n

Acknowledgements<\/h5>\n
My thanks to the archive of the Antoni T\u00e0pies Foundation in Barcelona and the T\u00e0pies Commission for kindly giving me access to the collection of graphic work, paintings and artist\u2019s books by Antoni T\u00e0pies, and especially to their directors Laurence Rassel and Carles Guerra for their willingness to facilitate my research.<\/h5>\n

Recommended links<\/h3>\n

Semantic-Based Image Analysis with the Goal of Assisting Artistic Creation<\/a>.<\/p>\n

Artistic ideation based on computer vision methods, Journal of Theoretical and Applied Computer Science<\/em><\/p>\n

Formas latentes: protocolos de visi\u00f3n artificial para la detecci\u00f3n de analog\u00edas aplicados a la catalogaci\u00f3n y creaci\u00f3n art\u00edstica.<\/a> Tesi doctoral, Pilar Rosado<\/p>\n

Pilar Rosado
\nBRAC group researcher, University of Barcelona<\/h6>\n

Any form of total or partial reproduction, distribution, public communication or transformation of the images without the written permission of the rights holders is totally prohibited. <\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Pilar Rosado We have invited Dr Pilar Rosado to talk about the most outstanding aspects of her research on computer vision as applied to art, the subject of her PhD thesis. This is a post of a specialization that is maybe unusual in blogs, but clearly explained with very explanatory images, which we think is…<\/p>\n","protected":false},"author":47,"featured_media":7047,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[16,1,42],"tags":[996,999,993,335],"class_list":["post-7035","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-digital-en","category-general","category-research","tag-antoni-tapies-en","tag-computer-vision","tag-images","tag-technology","author-guest"],"jetpack_featured_media_url":"https:\/\/blog.museunacional.cat\/wp-content\/uploads\/cap\u00e7alera-5.jpg","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p4tWCI-1Pt","_links":{"self":[{"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/posts\/7035","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/users\/47"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/comments?post=7035"}],"version-history":[{"count":12,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/posts\/7035\/revisions"}],"predecessor-version":[{"id":15566,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/posts\/7035\/revisions\/15566"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/media\/7047"}],"wp:attachment":[{"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/media?parent=7035"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/categories?post=7035"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.museunacional.cat\/en\/wp-json\/wp\/v2\/tags?post=7035"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}