On Pictures of Picturebooks: The Role of Illustration in Printed and Interactive Picturebooks

 

Translation is the process of mediation, of holding and including two languages and ​​transferring meaning from one language to the other. It is based on the ability to synthesize, to connect the concepts, perceptions, perspectives, patterns, and structures reflected in one language, and convert them into another. The limits of language are the limits of the mind. As we move from one language to another, there shall always be parts that are incomprehensible. As we learn to read, write, speak, hear and understand different languages, we get a wider, more tolerant, more inclusive and more holistic perspective, even a multiplicity of perspectives.

This paper is an invitation to take a look at the format of the picture book, read its languages ​​and examine what happens when this form of art immigrates from the traditional territory, print, to a new territory - the digital medium.

The picturebook is a unique medium, in which text and illustrations are being integrated to create a meaningful narrative. Dialogs between words and images in picture books are divided into two main models: parallel storytelling, in which text and illustration form the same narrative simultaneously, and interdependent storytelling, in which both channels have to be read at the same time in order to fully understand the book.

The books I shall present here are from the second category, interdependent storytelling. They offer a complex and dynamic relationship between the two channels. The linguistic channel functions linearly, while the visual channel is being spread out fully, without verbal mediation. These channels influence each other and it's their synergy that creates the meaning of the book.

For preschool children, written words are an inaccessible territory. Through the illustrations, they can read and understand the story. As children are able to "read" the book by themselves, they accomplish a sense of control, success and independence. The more dominant the role of the illustrations in the book, the better can children decode the book and understand it on their own.

As soon as we learn to read fluently, our reading experience involves mainly the literal channel. Therefore, for a reading adult, the meaning of a picturebook is being created by integrating words and images, as the text is essential and frequently more dominant than the illustrations.  

This process is being held by both reader and creator (or creators) - the writer and the illustrator of the book, who read and understand written words. But in order to examine the children's experience, we should first look at the book's illustrations alone. Then we should read the text independently, and finally we can consider the two channels together, examining the dialog between them.

Firstly, I shall suggest a new look at the classic picturebook Rosie's Walk[1]. In order to facilitate reading the book visually, without words, like preschool children do, I deleted the text completely. The illustrator of the book seems to share a slapstick comedy with the reader, as she draws both Rosie and the fox from the same angle and from a similar distance along the whole walk. Each spread features a scene in which the wicked fox is sneaking up behind Rosie as she walks, almost catching her, but never quite so (picture1).

 

Picture 1

Pat Hutchins, Rosie's walk, 1968

The illustrations create a sense of concern and protection: Rosie will be fine, she can manage, and would not be left alone as long as the fox is around. The illustrator leaves Rosie, and ends the book, only when the fox finally runs away. As we join her, we are being convinced that she keeps her promise to take good care of Rosie. The turning of the pages creates a "Cut" and an expectation for the next scene, which is constructed in the same format: unexpectedly, the fox gets into trouble and the hen continues her walk, but the danger is not over yet: the fox will not give up, and we look forward to seeing what happens next. Interestingly, the image of Rosie, her movements and expressions, remain almost the same throughout all 14 pictures of the book, emphasizing her innocence and her unawareness of the fox (picture2).

Picture 2

Pat Hutchins, Rosie's walk (details), 1968

 

Now, let us consider the text:

Rosie the hen went for a walk

across the yard,

around the pound,

over the haycock,

past the mill,

through the fence,

under the beehives,

and got back in time for dinner.

This manuscript, I shall presume, without the illustrations, would have never been published. From a linguistic perspective, the text suggests a narrative, but it is a shallow one, without any drama or inner development. Literally speaking, it holds 32 words: one sentence, built in a repetitive and rhythmic structure, emphasizing prepositions of movement in space: across, around, over, past, through, under.

Although the story is being told in the third-person, it reveals the plot form Rosie’s viewpoint. It presents a partial layer of the story, and reflects Rosie’s mind: naive and unaware. The name “Rosie” (a classical and popular English and American name) creates personification and empathy for the hen, focusing on a specific character, while in real life hens usually live in large flocks and chicken coops.

The illustrations add a significant narrative dimension to the book. The written story has only one character – Rosie the hen; the other character, the fox, appears in the illustrations. The narrator describes a route of a daily walk, while the illustrator observes Rosie and protects her, describing an existential dramatic experience.

The visual channel, the pictures a child can "read" independently, is dominant in Rosie’s Walk: 27 colorful pages presenting characters and events, experiences and consciousness that are not mentioned in the text. The humor in the book lies, partly, in the split between what we see and what is being told, creating a partnership between the illustrator and the young reader. Together they follow the wicked fox as it slips into the water, falls into the pile of flour, lands on the cart and runs away from the bees - while the adult, who reads the text, holds the hen’s viewpoint and awareness.

This split creates a role reversal: the child, who “reads” the illustrations, fully understands the book, while the adult, who reads the text aloud, articulates his own limited, literal perception. If we were to state it the way children do, we could say that the adult, who reads this book aloud, is as dumb as the hen. The young and vulnerable, the child and Rosie, laugh at the expense of the fox and the adult.

I would like to mention, briefly, two other books that offer interesting relationship between text and illustration. The first part of Good Night Gorilla[2] features a structure similar to Rosie's Walk. The illustrations present the gorilla as it follows the zookeeper on his nightly rounds at the zoo, quietly pick-pocketing the zookeeper and stealing his keys, stealthily trailing him, and unlocking the cages of every animal the oblivious fellow bids goodnight to. As we consider the text, we find one sentence, “Good night”, being uttered to the gorilla, the lion, the giraffe, and so on. The complete text of the book has 18 repetitions of this greeting that do not yield a meaningful narrative. As in Rosie's Walk, through the pictures the young reader can detect the narrative, while the text reflects the zookeeper’s unawareness (picture3).

 

Picture 3

Peggy Rathmann, Good Night, Gorilla, 1994

The turning point of the book happens after the zookeeper returns home, followed by the animals. As his wife bids him "good night" and turns off the light, she hears seven different voices answering her. The roles have changed: the animals are now the speakers. They are making use of the verbal, textual channel, and unintentionally disclose themselves by speaking (picture4).

Picture 4

Peggy Rathmann, Good Night, Gorilla, 1994

 

The zookeeper's wife is now using the wider visual channel. Quietly, she turns on the light, to see (picture5).

Picture 5

Peggy Rathmann, Good Night, Gorilla, 1994

Without saying a word, she takes the animals back to the zoo. In the third and last part of the book, the little gorilla sneaks back behind the zookeeper’s wife, in order to sleep in her bed. The zookeeper’s wife says "Good Night, zoo", (again, expressing through the textual channel her unawareness of the gorilla following her ) while the deceitful gorilla acknowledges the reader, signaling her directly to be quiet and keep its secret (picture6).

Picture 6

Peggy Rathmann, Good Night, Gorilla, 1994

The gorilla breaks the "fourth wall" - a term used in the world of theater, which describes an imaginary boundary between any fictional work and its audience. The gorilla's gesture reflects, again, higher consciousness: the zookeeper’s wife, her eyes closed, does not notice the gorilla and the reader, sharing a secret behind her back. The use of the verbal channel in Good Night Gorilla alters the plot: once the animals talk, they are disclosed. The importance of the verbal channel lies in its very use. The textual content is secondary to the role of the speakers. One might say that the meaning of the book is based on the transition between the visual territory and the textual territory, so the dialog between these channels is symbiotic and inherent.

 The first pages of the famous book The Giving Tree[3] feature a tall, expressive tree and a little playful boy. The book ends with illustrations presenting an old man sitting on an old stump. The illustrations tell us about changes through time. The old man was once a little boy, the stump was once a strong apple tree. Literally speaking, the term "Boy" is used throughout the whole book. In the first part of the book, the word and the image are linked, but as the story goes on, the gap between them increases. As in the previous books I discussed, the text expresses a subjective, partial perception, and the illustrations present a broad and holistic view. In this case, the name "Boy" reflects the maternal point of view of the tree. For the tree, the child will always be a child, so she will always forgive him, and will forever give him anything she can (picture7).

Picture 7

Shel Silverstein, The Giving Tree, 1964

The dialog between illustration and text is essential to every picturebook. In recent years, the prevalence of e-books is growing dramatically. The digital picturebook is new territory. Its channels - words and illustrations, are supposedly the same building blocks as those of the printed book. Does the digital picturebook require a new literacy? Does it create a new reading experience? Is it a new mode of storytelling, or is it just an instrumental, functional and technical change? Digital books are cheaper than printed books; they save room and are available widely for immediate use. As we wait for a medical checkup or go on a family backpacking trip, reading a digital book is an available, inexpensive and convenient option.

According to the famous expression by Marshall McLuhan, the medium is the message. "It is only too typical that the “content” of any medium blinds us to the character of the medium."[4] (McLuhan, 1964.) The boundaries of the medium are not just a technical issue. As the book passes from one medium to another, the nature of the reader’s experience changes.

Each medium has specific characteristics that create a unique reading experience. The printed picturebooks are stepping stones towards reading “real” books. Children read them with a parent or another adult, usually as part of a calming bedtime routine. Families reading together enjoy warmth and fun; studies show that children who are read to, read more on their own than children who are not. Reading a digital picturebook is different. Children often use tablets on their own. The child could be a single user, or the tablet would be shared by two children, but almost always children will not share it with an adult. The text in a digital picturebook is usually an integration of the spoken and the written word, by the voice that tells the story. We hear the story, and simultaneously see each word as it is delineated. Children are familiar with electronic devices and enjoy exploring the various modes of operation, and therefore a non-interactive digital book would be considered an inferior product.

A Digital pictureboook is called an App, Application, meaning a practical implementation of theoretical content. The printed picturebook invites the reader to calm down: as the child lies down in her bed, she quietly and passively enters a fictional world. The application, as its name implies, invites the child to act: to push, pull, shake, rotate, enlarge, reduce, play, paint, and sometimes even blow on the screen. Inherently, the digital picturebook breaks the fourth wall and engages the child directly. Without interaction, the plot will not advance, the book will not perform. The book responds to the reader instantly, through sounds, images, music and animation, and thus invites the child to search for more. Unlike the printed book, which draws the reader to an inner world, the digital book creates an active atmosphere that requires alert and active reading.

In order to create an interactive book, one must think interactively. Digital picturebooks can take up to about six lines of text, sixty words per page. The illustrations should be large and split into several layers in Photoshop, in order to create movements, interactions and animations. In a course I instructed at Holon Institute of Technology in 2011, the students created picturebooks simultaneously in two formats: a printed book and a digital book, produced as iPad applications.

The first stage, and I think the most important one, in creating a picturebook, is creating a Storyboard. The storyboard is a tool for planning the book - an overview of the entire book which facilitates the planning of the main visual elements. This is a storyboard we made for the app The Selfish Giant, illustrations by Jenny Milichov. The drawings are quite rough, but clear. They are the means for visualizing the story (picture8).

Picture 8

Jenny Milichov, The Selfish Giant, storyboard, pencil and digital

The storyboard specifies every interaction and animation planned, and requires an accurate reading and approval of the technical staff, to make sure the plan can be executed. The interactions between the reader and the book are meaningful and an integral part of the book. For example, I shall present one of the illustrations, split into six frames, thus enabling the reader to “sprout” the leaves and flowers (picture9).

 

Picture 9

Jenny Milichov, The Selfish Giant, mixed media and digital

 

These leaves and flowers stand for the friendship between the giant and the children. Through the interactions, the reader takes an active part in the story: when the giant builds a wall that prevents the children’s play in his garden, touching the screen will tell us what the children think and what each one of them recalls. Through the interaction we can make the naked tree bloom, and this interaction consequently allows us to cover the dead giant with flowers at the end of the story (picture10).

Picture 10

Jenny Milichov, The Selfish Giant, mixed media and digital

 

As I mentioned, for preschool children, illustrations are a means of reading. Similarly, interactive book applications invite children to explore picturebooks on their own. They suggest a dynamic, independent reading and playing experience. Children are knowledgeable in illustrations and interactions. They figure out both of these channels easily, faster and better than adults. The more productive the teamwork of the writers, illustrators, editors, programmers and producers - the more we can enjoy innovative picturebooks, challenging the book's linear structure and examining the boundaries between a book, a film and a game. Getting back to where I started, this process is based on the ability to synthesize; in this case, to integrate the visual channel, the literal channel, the limits of technology, the user experience, the marketing, and total cost of this production.