Kress, G. (2004). Reading images: Multimodality, representation and new media. Information Design Journal, 12(2), 110-119.
In this article, Gunther Kress examines the shift that has taken place from the “mode of writing and medium of a book/page” to the “mode of image and medium of screen” (p. 110). He argues that the mode of how meanings are expressed has rhetorical consequences. He first argues that the material of the material of the mode has consequences for meaning. Nonverbal communication has shifted from words to images. He argues that words can be limiting: “only that for which there is a word can be brought into communication” (p. 112). The spatial relations between words written on a page and images shown on screen have important consequences. Readers only have one point of entry to a text; in order for it to be understood, it must be read chronologically and in order, as it is written out on the page. On a website, however, there are multiple points of entry, which allows the reader to design the text for themselves. As Kress states, “the reader’s interest determines where he or she wishes to enter the page” (p. 114). Also related to the mode of display, he argues that on screen, text is “subordinated to the logic of the image” (p. 116) and vice versa. He connects the rhetorical difference between the mode of writing versus the mode of the image to design. He states that because of the “multimodal landscape of communication” (p. 116), design is a central issue to communicating meaning. Design requires a person to consider environmental, audience, resources, constraints, and interests in shaping the argument. Because the page, with its central point of entry, is no longer the main mode of communication, communication has become fragmented and therefore, he argues, “rhetoric has become a major issue for design” (p. 119).
Kress’ discussion of multimodality and design choices is central to my research on GIFs in online discourse on news sites. He notes in this article that there is a shift where images are used with more frequency, “even in situations where previously writing would have been used” (112). This the question that I am interested in exploring—in what scenarios do commenters make the rhetorical decision to use a GIF in lieu of text? And why do news commenting sections make the decision to limit images in online conversation? The mode of the screen allows for many means of communication, yet major media sites such as New York Times and Washington Post make the rhetorical decision to only allow for text in their discussions. A small handful, like Gawker, allow a variety of means, such as links, images, and moving images. I think that GIFs are an interesting case because users have control over their “point of entry,” to borrow Kress’ terminology. The moving image is a short, precisely selected clip, often from a video, stripped of sound. The movement is not implied, but explicit. Take the below example of a popular Beyoncé reaction GIF versus a screenshot. Both have a humorous tone. Both carry the same cultural connotations that Beyoncé carries as a pop culture icon. The moving GIF is clear—she is saying “no,” and there is little wiggle room to interpret. The static image, on the other hand, is not so clear. Is she wagging her finger? Is she trying to catch someone’s attention? Is she saying “hold on a sec?” This difference is why GIFs are important to consider and can be very useful to contributing to discourse.