Visual sonority in Brazilian Sign Language literature¹

Sonoridade visual na literatura em Língua Brasileira de Sinais

ABSTRACT

The idea of sonority in sign languages was treated by Perlmutter (1992) as perceptibility, a property of a segment that uses movement rather than one in which the hands stay in the same position. Sandler (1993) states that the visual salience of movement in sign languages plays a role similar to sonority in spoken languages. For Brentari (1998), perceptually, a sign is visible from considerable distances, and measurement of its visual sonority is based on the joints involved in its production. This work focuses on visual sonority in literature in Brazilian Sign Language and considers the relevance of manual and non-manual elements, rhythm, symmetry, the scale of signs, and the effect of video on this concept. Two signed stories “The King’s Parrot” and “Little Ping Pong Ball” were analysed, highlighting specific signs in which the use of joints, non-manual features, and other resources are influenced by the size of the performance space and the distance of the audience from the signing. Three types of ‘sonority’ were observed: in the movement of the whole body on the stage, in the size of arms and trunk movement, and in the hands. In addition to the joints, non-manual features, rhythm and symmetry play an important role in visual sonority and influence the viewer’s experience.

KEYWORDS:
visual sonority; Literature; Brazilian Sign Language

RESUMO

A noção de sonoridade nas línguas de sinais foi tratada por Perlmutter (1992) como perceptibilidade, a propriedade que tem o movimento em comparação a um segmento em que as mãos permaneçam em uma única posição. Sandler (1993) diz que a saliência visual no movimento joga um papel nas línguas de sinais semelhante à sonoridade nas línguas orais. Para Brentari (1998), perceptualmente, um sinal pode ser visto a grandes distâncias e sua sonoridade visual é mensurada com base nas juntas envolvidas. Este trabalho foca a sonoridade visual na literatura em Língua Brasileira de Sinais e discute como elementos manuais e elementos não manuais (ENM), ritmo, simetria, escala dos sinais e efeitos do vídeo têm relevância para este conceito. Assim, observou-se, nos contos sinalizados “O Papagaio do Rei” e “Bolinha de Ping-Pong”, como são ressaltados determinados sinais a partir das juntas, dos ENM e outros recursos e a influência do tamanho do espaço e a distância da plateia na sinalização. Como resultado, foram observados três tipos de “sonoridades”: foco em todo o corpo; da cintura/ tronco para cima; e nas mãos. Foi possível perceber que, além das juntas, os ENM, o ritmo e a simetria têm um papel importante para a sonoridade visual e influenciam a experiência do espectador.

PALAVRAS-CHAVE:
sonoridade visual; literatura; Língua Brasileira de Sinais

Introduction

The aim of the research reported in this article is to understand the way in which Visual Sonority works in Brazilian Sign Language literature, specifically focusing on texts that are performed in different contexts, such as on video screen or stage. This article shows how visual sonority in sign language, discussed in Perlmutter (1992), Sandler (1993), and Brentari (1998), may also be present in non-manual elements, such as trunk, head, and gaze movements, as well as in the rhythm and symmetry in signing, the influence of space and audience, and, hence, in visual sonority.

Visual sonority, as Perlmutter (1992), Sandler (1993), and Brentari (1998) have defined, is a property of signs, which makes them more visually noticeable or salient. The first two authors discuss it in terms of syllable, in which movement is the point of maximum sonority. For Sandler (1993), repetitive internal movement (RIM) adds sonority. Brentari (1998) states that sonority is as much perceptual as it is articulatory. Articulatorily, sonority is measured by the joint used to perform a movement. Perceptually, it causes a sign to be seen at greater distances. However, there is little evidence with respect to the perceptual level. Crasborn (2012) explains that phonetic research has almost exclusively focused on the articulation of sign languages. This may be because visual perception is very complex, such as abstract visual categories that the brain can recognize, as well as because there is no specialized field of perception of body movements from which linguists could borrow the terminology, as they do with anatomical and physiological areas at the articulatory level. Similarly, this research focuses on production at the articulatory level.

Sign language can also employ different cinematic shots, a feature discussed in Bauman (2006) and Pimenta (2012), where signers use different scales, distant or close-up shots, according to the choice on how to represent the referent. Thus, we will discuss the relationship between scales and visual sonority.

This study analyzed two stories performed in different settings: one designed for a large audience in a live performance, and another, in video form, posted on the internet.

This research aimed to study visual sonority in the context of signed artistic productions and answer the following questions:

1. How does the artist highlight certain signs?

2. How can a gaze, other non-manual elements, and the use of space contribute to making the movement more sonorous?

3. In artistic signing, do the signs made for a live audience differ from those made in front of a camera?

From the data generated by analyzing the two stories, specific research questions arose, regarding the influence of the signing context (space and reception), the “visual sonority” involved in the manual and non-manual elements, and the effect that sonority can have on the audience’s experience.

1 Literature review

1.1 Visual Sonority

Sign language sonority was discussed in Perlmutter (1992) as “perceptibility”, and the movements in this sense are more noticeable than other segments in which the hands do not move, even though this statement has not been proven, since little research has been devoted to the perceptual level. Perlmutter discussed sonority in terms of syllables, with the sonority peak in the syllabic structure representing the movement. Sandler (1993) also studied the sonority in syllables and claims that the movement is more perceptually salient and that visual salience plays a role in sign language that is similar to the sonority of an oral language. In Sandler’s proposal of a sonority hierarchy (1993), the signs with path movements and repetitive internal movements are the most sonorous. Internal repetitive movements are associated with the sign’s most sonorous segment, and when there is only one non-sonorous segment, these movements are associate with the non-sonorous segment; that is, repetitive internal movements add sonority. Brentari (1993) proposed that the maximum local sonority found in the syllable peak is represented by docking sites where the secondary movement is attached. Later, Brentari (1998) proposed a prosodic model for sign language phonology, which works with the concept of visual sonority, both perceptual and articulatory. Perceptually, it is the property of a sign that causes it to be seen at great distances, and articulatorily, sonority is measured according to the joints used to articulate a movement (BRENTARI, 1998). Phonetically, in the oral languages, the oral cavity degree of opening - which determines the amount of resonance and sonorous range - makes the sound more noticeable over long distances. Thus, [a] is more sonorous than [i]. In signs, a phonetic-articulatory correspondent would be the relative proximity of the joint that links the sign to the body’s midline, resulting in greater visibility.

Brentari (1998) presents a hierarchical sonority table for signs, in which different features (location, path, direction, opening) are related to an articulator and its value (shoulder 6, elbow 5, wrist 4, metacarpal 3, interphalangeal 2). The sonority value 1 would be for internal repetitive movement, which is not listed, because it is not a simple movement, and can occur at any node class in the prosodic branch.

However, many factors could cause a sign to have its value changed in the form of output, as in the processes known as distalization (phonetic reduction) and proximalization (phonetic enhancement) (BRENTARI, 1998). In these cases, the input joint specification is changed, since in distalization the movement is transferred to a more distant articulator (for example, the knuckle) and in proximalization to a closer one (e.g. the elbow). These operations occur through physiological and social factors, as well as during the interaction itself (BRENTARI, 1998). Napoli et al. (2014) state that a whisper in sign language so that others cannot see or a sign in limited space - chat on a small device like an iPhone - is often the reason for a sign to be diminished or distalized, while shouting for bigger signing is often due to the situational context. Furthermore, while spoken languages have a working pressure towards facilitating articulation, sign languages also have this property, as the more distal joints have less mass and require less articulatory effort, thus the more proximal joint, the shoulder, requires greater effort when compared to other joints (NAPOLI et al., 2014). Brentari (2012) claims that the movements of the proximal joints, such as shoulder and elbow, are more visible than those articulated by the distal joints, like the opening and closing of one’s hands, and are a manifestation of visual sonority.

From this theoretical framework, one of the questions of this research is: In artistic signing, do the signs made to a live audience differ from those made in front of a camera?

1.2 Cinematic shots and spatial scale

The signer’s body is capable of performing various effects similar to what we see in the movies, leading to claims that sign languages ​​are essentially cinematic. In this sense, Bauman’s proposal (2003) is to bring the terms used in the movies as part of a standardized lexicon to describe poetry in American Sign Language. The poet, through language, is simultaneously a screenwriter, a cameraman, an editor, an actor, and a director. Body movements use space in three-dimensions; non-manual markers can also transmit a close-up of a character, and classifiers can easily create distant or extremely close shots, detailing the shape and dimensions of an object (BAUMAN, 2006).

Pimenta (2012) - a Brazilian deaf sign language artist - researched signed narratives and their features in common with the cinematic shots that have more imagetic similarities to sign language. He stated that shots are part of the syntax of the movies’ language, along with framing, movement and editing. One of the categories Pimenta created to analyze narratives was called “camera movement”: this movement is shown via language, and comes in for a close up view (zoom in) and pulls away from (zoom out) the person or object, in order to emphasize their status and their emotions. The representation of the referent is made by facial expressions and body movements related to signs, classifiers, or gestures. However, it is not the sign size that changes when signs are used in narratives, but rather the approximation of the signer’s body to the referent. In the case of classifiers, they can vary in size or change their shape.

The close-up and distance shots are related to the use of space in small and large scale. Large-scale space is associated with manual constructions representing the manipulation of an object, and signers seem to interact with people or objects in a real-world scale. In small-scale space, associated with entity constructions, signers use their hands to represent a referent or part of it. Scales may also be mixed when, for example, the signer represents a referent in large scale, while the hands represent the same referent, or another, on a small scale (SMITH; CORMIER, 2014, p. 278). Constructed action also uses the large-scale space, given that “the signer’s head, face, torso, arms and/or hands [are used] to directly represent the same articulator(s) of a referent” (SMITH; CORMIER, 2014, p. 279). Large-scale space is the character scale, because the signer can be considered within the story space, while in a small-scale space, the signer is outside of the story and is considered an observer.

For this research, we can relate sonority to close-up and distancing movements/space of small and large scales. The referent size in the real world is not necessarily the same as the signer chooses to represent it, since he has the option of using his whole body or just his hands. Thus, if the referent is bigger in the real world, it does not mean that it is more sonorous, as this will depends on how it will be signed. By moving away from the body, the signer uses more distal joints, the hands. However, when he approximates or is incorporated, he uses more proximal joints, closer to the midline of the body, such as the shoulders and elbows, and thus a wider space.

1.3 Live performance and video productions

Art in sign language, in general, and literature, in particular, is often considered an art of performance. Rose (1992, p. 93) says that there is no single definition for performance, but that theorists recognize it as the esthetic representation of a text in which the performer is also the author of the message, in which the performance artists speak in their own voice and have a close, corporal relationship with the public. To Rose (2006), literature in signs is more than a literature of the body, it is a literature of performance “that moves through time and space, embodied in the author’s physical presence” (ROSE, 2006, p. 131). Rose (2006) further asserts that the gift of deaf poet with language is also the gift of stage presence and body expression.

Art in signs is done through performance. Examples of art signs are storytelling, percussion signs, and poetry, which have been present in the deaf community for a very long time (VALLI, 1993, p. 10). “Art-sign” is a term coined by Klima and Bellugi (1976) to refer to an intensified use of sign language. Creative sign language, in general, is the use of sign language for artistic purposes. In creative sign language, the way something is said is as significant as what is said, that is, the form is as important as the content (KANEKO; MESCH, 2013).

The circumstances in which art sign is produced or presented influence and determine its totality and reception. Its time, space, and relationship between artist and audience are inferred by circumstances. Informal meetings, social events, and academic events are some of the places where a poem or story can be transmitted to an audience. When it is live, the audience can participate. On the other hand, watching a signed video is like reading the lines of a written text, since the reader/viewer engages with it more passively. Machado (2013) describes that the deaf poet knows what elements are required to present a live poem or a poem on video. In the live presentation, there is a more direct relationship with the audience, and the signing artist is aware of the technical and specific adaptations needed for each space. According to Bahan (2006), a storyteller needs to know how to select an appropriate story for the audience to manage its interest, which encourages the storyteller to repeat his/her performances and adjust the details to suit the profile of those who are watching. Peters (2000) claims that getting the audience engaged in the process of storytelling is inherent to this art form, so that the event is more important than the text. According to Sutton-Spence et al. (2005, p. 134), the audience’s involvement is essential, and she mentions that a good performance includes the “visual qualities of the signs”, such as attention to the size of the signs - are they large, small, or do they vary? The signing energy, intensity, and speed can also be controlled in performance, creating a strong effect on the audience. Dorothy Miles (1991 apudSUTTON-SPENCE, 2005) defends that poets and performers should involve the audience with images, and gaze at the shapes they create as long as necessary for the audience to look in that direction. At the same time, they must not focus on the images for too long so that they do not ignore the audience. In sign language poetry, the gaze has various functions. It may represent the eyes of a character, guide the audience - or communicate to an imaginary audience in front of a camera (KRENTZ, 2006) - as well as highlight, reflect, or complement information carried in a manual sign, develop the story without a manual sign, and function as an orientational metaphor (KANEKO; MESCH, 2013).

In video, the viewer’s participation and interpretation are different from that in a real-time performance, where there is an influence from both the audience and the performer. In video, ASL literature is more static and shifts from a public space to a private space. The viewer can watch but not participate in real time.² This increases the distance between artist and audience (KRENTZ, 2006). In addition, signing space is more restricted than in live performance. In a video recording in sign language, the signer must be alert to the camera position, or guide the camera, so that it captures the whole signing space. In live performance, the signing space, which is defined by sign language grammar rules, can be extended, and the entire stage can be exploited, although the poets (or storytellers) are “confined to movements of those parts of the body visible in the small stage of signing space” (SUTTON-SPENCE; BOYES BRAEM, 2013, p. 269). The video space is even more restricted. Since it usually shows only the top half of the body, signs need to be smaller. However, this limitation of the video space can lead the performer to explore different shots and angles. In designing the text and in working with the videographer, closed or open shots can be selected, choosing to fit different body parts, such as open plans (body in signing space) or closed (face or hands). Lucas et al. (2013) summarizes research on how technology can influence signs’ size and shape, while another author (MIRUS, 2008 apudLUCAS, 2013) shows an example of two signers talking on the phone through video with the screen so close to their face that they occupy a large portion of the screen. In this example, the signing shifts to near the mouth, eliminating the need to show the whole trunk. That is, deaf people consciously change their way of signing according to the available video screen size.

Finally, in a live performance, the movements on stage, as well as the gaze to focus on certain signs and communicate with the audience, can be explored, and the performer can receive immediate feedback from the audience on the text reception. In video, although the author and viewer are separated by different time and space, the performer can explore different planes and angles with the camera and make use of different effects. Sign breadth and visual perceptiveness can also be explored. However, in a live performance, the distance between the artist and the audience seems to have an effect on the signs’ visibility, so the artist must use this to achieve different esthetic effects. Similarly, in video, the size and angle of shots help the artist to draw the viewer’s attention to the visual quality of the manual or non-manual signs.

2 Non-manual elements, rhythm, and their relationship to sonority

Regarding sonority in non-manual elements, body movements complement the manual sign, increasing its sonority with the degree of leaning and twisting of the trunk and head movements. The gaze can also create the perception of a bigger or smaller space, thus contributing to the sonority. Kaneko and Mesch (2013) studied the gaze in creative sign language and concluded that it has three main functions: 1) the poet’s gaze recognizes the presence of the audience (outside the story world); 2) the poet’s gaze represents the character’s gaze (inside the story world); 3) the gaze focuses on the hands or points in signing space, and the direction of the gaze can be dependent or independent of the manual sign.

From these functions, Kaneko and Mesch (2013, p. 384) present a classification of gaze patterns. They claim that, in poetry, gaze behavior is more complex than in narratives and is broken down into six categories: 1) gaze at the audience, 2) characters’ gaze, 3) spotlight gaze, 4) reactive gaze, 5) panoptic gaze, and 6) prescient gaze. Some of these gaze categories can relate to sonority. As mentioned above, in some cases, no manual elements are responsible for phonetic enhancement, i.e., in the output form, the sign presents an articulator, like the trunk, the head, or the gaze, which adds sonority to the sign’s basic movement in the citation form. In addition, this can create more visibility in the sign at the perceptual level. The spotlight gaze emphasizes the configuration or movement of a given sign, causing the audience to look at it, guided by the signer’s gaze. It is as if the gaze were a camera making a close-up picture of the articulators. The function of the reactive gaze is to react or reflect on a sign, or to generate an associated emotion (surprise, distrust etc.). In these types of gaze, at the articulatory level, increased visual sonority may be the result of enhancement through the use of non-manual elements. At the perceptual level, the sign also becomes more salient because the signer draws the audience’s gaze toward a given sign. Panoptic gaze and prescient gaze have immediate connection with space. While the signer makes the manual sign, the panoptic gaze suggests other elements in the poetic scene. The prescient gaze shows that something is about to happen, predicting the location of the next sign in that area of space. In this sense, it also has the power of extending the scene. Both the panoptic and the prescient gazes can be indicative of sonority, as they expand the space around the manual sign.

Rhythm is also related to visual sonority, but before describing how this occurs, it is necessary to observe what some researchers have said about rhythm in poetry in sign language. Valli (1993, p. 90) states that rhythm in poetry in American Sign Language (ASL) uses the body and the space with movement contours, assimilation, alternating movements, laterality, movement duration, and size. He also states that in ASL, lexical signs consist of hold and movement segments, sequentially, that can produce metric syllables (prosody) by using stress. The stress can be created from the emphasis on holds (long, subtle, sharp pauses), emphasis on movement (long, short, alternating, repeated), movement size (enlarged or reduced path, shortened and accelerated motion), movement duration (regular, slow or fast) (VALLI, 1993, p. 68).

In my doctoral dissertation (KLAMT, 2014), I studied poetic rhythm in Brazilian Sign Language (Libras), using the poems “Flying over Rio” by Fernanda Machado and “Brazilian Flag” by Nelson Pimenta. The study aimed to determine whether sign language poetry has rhythm and to identify elements that could show how it is created. It was observed that the repetition of signs and rhyme (repetition of linguistic parameters) create rhythm patterns. The poetic device of ‘morphism’ blends signs, giving the idea of continuity; pauses and holds emphasize certain passages and highlights of the poem, the size, the emphasis, the movement duration allow signs to be shortened, extended, accelerated, decelerated, modified as to its size and movement type. Visual sonority can make signs more salient to the audience; and symmetry can, for example, create contrast between the beginning and the end of the poem. Finally, many elements found in the poems corroborated with the statement that the two analyzed poems have a strong pace, and I concluded that the rhythm in sign language poetry is defined by a repetition pattern that occurs at regular time intervals during signing, taking into account the movement (duration, size, and emphasis) and non-movement (holds or pauses) periods, the rhyme, the repetition of signs, and the metric movement (in which syllables form accented and non-accented feet). In this study I also found that>Valli’s third category (1993, p. 68)movement size, is a characteristic of poetic manipulation of language, but it is directly related to sonority. Visual sonority in each type of articulator enhances rhythm perception, since it is known that one way to create the rhythm of signed poetry is to manipulate the size of the signs’ movement by shortening or stretching them, or even using signs with the same movement pattern. In other words, it can be said that sonority creates rhythm when a sign movement has proportions that are repeated at regular intervals or when movement size is changed so that the sign is more or less perceptually visual (for example, a poem in which selected signs have their movement amplified or signs that repeat short movements). In poetry, as well as in general artistic signing, sonority can be manipulated, creating rhythm in the production. When creating a movement size pattern - with more or less sonority - the poet is dealing with rhythm. Rhythm is present in everyday and artistic signing, but poetry exploits this resource in a particularly fruitful way.

3 Research Method

This research comes from a linguistic perspective and aims to investigate and describe how performers and artists can work with language resources to create more sonorous signs and communicate more effectively with the audience.

The questions that guide the research and its method are:

1. Regarding sonority, do signs made for a live audience differ from those performed in front of a camera?

2. How does the artist highlight certain signs?

3. How can the gaze, the other non-manual elements, and the use of space contribute to make the movement more sonorous?

The principal criterion for the selection of texts to be analyzed was that performances should be presented in different contexts, such as those filmed for video and on the theater stage, to investigate if variables, such as space and distance from the audience, are relevant in the prosody and selection of signs. To this end, recorded performances were chosen, as examples of art in signs presented live on large stages in front of a large audience, which is at a considerable distance from the signer. The chosen story is “The King’s Parrot” presented by Bruno Ramos at the 1^st Signed Folklore Festival/Libras Hotshots (Craques da Libras - 2014). To reflect on how a storyteller/performer creates art within a limited space and still exploits the video features, the story “Little Ping Pong Ball” by Rimar Romano was chosen (available on Youtube). For data analysis, I used ELAN³ to watch the videos, create glosses to facilitate the signs’ segmentation, select signs that stand out for their sonority, and create analysis tiers.

To answer the research questions, the data generated from the notes on the tiers were analyzed: glosses, manual articulators and repetitive internal movements, movement, non-manual elements, gaze, rhythm, repetition, symmetry, zoom in and out, video, and sonority levels.

The articulators that were used, the sonority degree involved, possible influencing factors, the importance of sonority for artistic signing, and the differences between live and video signing are discussed based on these data.

It was necessary to make some methodological decisions when it came to taking notes about the articulators. First, the joints (metacarpophalangeal and interphalangeal) were analyzed together (as “fingers”), due to the difficulty of segmenting and distinguishing their movements. The notes concerning the fingers were made only when there were changes in the hand configuration within the sign, resulting in finger movement. Hand configuration at the beginning of the sign was not noted, because it is not part of the sign movement as a whole. The joint used in the transition between two signs was always noted on the second sign. The joint that performs the arm movement or the non-dominant hand movement was also noted. The signed unit was, thus, globally analyzed.

Returning to the main objective of this article, that is, of understanding Visual Sonority in the field of literary texts presented in different contexts, such as on the video screen and the stage, we will now analyze Bruno Ramos’s “King’s Parrot” and Rimar Romano’s “Little Ping Pong Ball”.

4 “The King’s Parrot”

The story “The King’s Parrot”, which lasts 06’47’’, deals with the loss of deaf identity and culture in hearing society. In the story, a hunter imprisons a bird in a cage and a bat sucks out the bird’s identity and culture. In this tale, 159 glosses of marked units⁴ were noted. Most of them combine two articulators - shoulder, elbow - and internal and repetitive movements (60 signs = 38%), that is, proximal joints, which have more sonority, as well as the additional sonority of the internal repetitive movements. Others (39 signs = 25%) use the more proximal joints together - shoulder and elbow - but without the additional sonority, 13 (8%) only use the elbow as the articulation joint and 13 (8%) combine shoulder, elbow, and fingers. Figure 1 shows that the combination of the proximal joint is present in most signed units and makes this production notably sonorous at the articulatory level.

However, saying that the story used mainly proximal joints is not enough to show the nature of the signed units that make this production more sonorous, nor to show how non-manual elements play an important role in this performance. All these additional elements show a sonority level that goes beyond the use of joints and are essential for the performance as a whole, since they guide the audience’s gaze to different parts of the body, creating an esthetic effect from the prosody and the performer’s intention, and, as a final result, causes emotion in the spectator.

Three types of “loudness” or “sonorities” in the signed units are identified as the first research result. They direct the spectator’s attention to:

1. the signer’s entire body, because he or she uses the legs to move around or flexes the knees; the arm movement can be large;

2. the trunk, down to the waist, as the signer leans the trunk from side to side or twists, with or without highlighted movement and size of the arms in the signing space;

3. the hands, showing their configuration or movement. There are also signs that have no movement (only a hold) that do not fit in these categories; however, they are relevant when they are “frozen” for the audience to see.

These three categories denote three different sonority levels, using joints, and signer’s entire body, including the additional elements which make the signed unit more noticeable at large distances. However, they were not mentioned in Brentari’s concept of sound (1998), since its focus was to study sonority from the phonological aspect.

In this story, there are signed units that use the trunk leaning forward, bending the knees and making movements across the stage - which seems to be a characteristic of live performances. Figure 2 shows that most units have sonority level 2 - with movement focused on the trunk or arms (76 units = 48%), followed by signs that draw attention to the hands (46 units = 29%), 31 signs (20%) focused on the whole body (level 1), and 6 holds (3%) (absence of movement, although the handshape and location are meaningful).

Within the first category - level 1 - we can highlight the transfer of person in the bird-cage (Picture 1), in which the performer uses the shoulder and elbow with flexion, extension, hyperextension, and rotation, even though there is no manual sign. Additionally, there is a shake (repetitive internal movement) when the bird flaps its wings. What draws attention in this transfer, however, is the movement along the stage, the trunk, the knees (bent), and the strong facial expression. This movement expands the restricted space of signing and provides more sonority. Moreover, the movement is in slow motion, which has a rhythmic effect and makes it stand out from other signs by slowing down. Symbolically, it shows the suffering of the bird that cannot escape the cage, and the slow motion effect contributes to this perception.

Picture 1
Person transfer in “The King’s Parrot”

The second group - level 2 - I illustrate with a double transfer: of person and of size and shape (picture 2), incorporating the bird character and providing a description of the size and shape of its body. The signer uses the movements of abduction and adduction, expanding the space and focusing the viewer’s gaze onto the movement from the waist up. A description of the bird’s wings and feathers uses shoulder, elbow, and repetitive internal movement. However, the shoulder makes the movement of abduction (to move away from the body, first image) and adduction (the second picture shows one arm in abduction - distant, and one in adduction, close to the body midline). The extent of abduction movement causes the signed unit to be more noticeable to the viewer than one focusing only on the hands, for example. Additionally, there is a flexion and extension movement of the elbow. All movements are rotated: what is done to one side is repeated to the other. There are also two repetitive internal movements: move and shake. Moreover, the head tilts, there is a mouth opening and a reactive gaze at the hands. All these non-manual elements, in addition to having a prosodic feature, highlight the signed unit by adding sonority.

Picture 2
Double transfer in “The King’s Parrot”

In the third category - level 3 - I present an example of size and shape transfer, which shows the brightness of the sea (picture 3), with special attention to the symmetrical hands, which make a repetitive internal movement throughout the whole section. Additionally, elbow and shoulder are involved in the articulation of this signed unit. The rising and falling movement results in alternation and repetition, and the gaze is one of emphasis during the execution of this unit. There is a pause at the end, with eyes turned to the hands, drawing the viewer’s attention to them.

Picture 3
Transfer size and form in “The King’s Parrot”

Picture 4 shows an example of sequences, performed in this story, of signs that can be understood in terms of camera movements, typical of sign language storytelling - close-up and distancing - in which the sign shows the referent in large or small scale. It shows the bird’s character via person transfer, as it comes out of the sea and flaps its wings (approximation). Then a distancing takes place when the bird is signed by a classifier, and it sees a flock of birds in the distance (shown by a manual sign), returning again to close-up, with person transfer (bird) and distancing when it sees the flock of birds. As the section continues, the signer comes out and in of the scene several times, in small or large scale, showing what the bird sees or incorporating the bird’s actions.

Picture 4
Approximation and distancing in “The King’s Parrot”

Analyzing this story, we reflected on the question “How does the artist (author and/or performer) highlight certain signs?” It was observed that the performer uses the movements along the stage, the size of movement of the arms, the gaze at the hands and other non-manual elements (such as pause) as a way of highlighting the signed units or direct the viewer’s look in different ways: whole body, from the waist up, hands. The viewer’s gaze is directed, sometimes, to the entire stage, because the performer moves and the viewer follows this movement with the eyes and sometimes to the strict signing space that focuses on a broad movement from the waist up (and the gaze of the performer may be directed to the audience, to the arms, or to certain parts of the signing space). At other times, the viewer is invited to focus on the signer’s hands, because their shape or movement is highlighted by the signer’s gaze.

Thus, it can be said that, besides the basic concept of visual sonority, the one that assigns values to the different arms’ joints, other sonority levels may also exist in the performance, different ways to make a sign, to make it more or less wide. It seems, then, that non-manual and rhythmic elements aid in emphasizing a sign. However, this is a quite subjective matter, because we cannot say for sure if the performer intended to highlight the sign through these elements, beyond the choice of the proximal joints. Still, this research has clearly shown that the different gaze pattern, the trunk, the displacement, the speed, the pauses, and other non-manual and rhythmic elements collaborate to make signs more sonorous, aiming to achieve a more effective communication with the audience.

5 “Little Ping Pong Ball”

The 3’39’’ story “Little Ping Pong Ball” was published on Youtube by Rimar Romano, of the Company Art and Silence. The video was converted to mp4 format and analyzed using ELAN. The story describes a ping-pong championship, but it is a metaphor for the life of deaf people in society. The ball is understood to be a deaf character that is thrown from side to side, and the signer, through incorporation by role shift, shows the suffering of the ball, who asks for help. One hundred and one glosses were noted in this story. The main articulators involved in the production of signed units were shoulder and elbow (60 signs = 60%) or only elbow (21 signs = 21%), that is, the more sonorous joints (see Graph 1). There were only two person transfers that created additional sonority via repetitive internal movement.

Graph 1
Use of joints compared between the stories

Graph 2
Sonority levels compared between the stories

Regarding the sonority types described as a result of this research, since it is a video, there was no displacement through signing space (level 1), but there is a recurrent use of the trunk leaning to the sides, due to the nature of the content, being a ping-pong tournament that features two opponents.

The third sonority level proposed in this paper is the most common in “Little Ping Pong Ball”, with 66 occurrences (65%) of signs focusing on the hands and 35 events (35%) at the second level, focusing on signs using the trunk or extension of the arms. Due to the limited space of the video frame and the proximity of the camera to the signer’s body, there were no large movements of the arms, as in “The King’s Parrot”; however, some signs stood out, by aligning shoulder and elbow and by having more “height” than the other (picture 5).

Picture 5
Extent of movement in signs in “Little Ping Pong Ball”

Among these signs, the person transfers of the man and the woman (pictures 6 and 7), use more sonorous joints and large movements, and have rhythmic elements, such as repetitive internal movement, alternate movements, and symmetry. In the description of the man, the hands were always symmetrical, and when describing his beard, the signer used vibration within the repetitive internal movement. In the woman, there is a repetitive internal movement in crooking the finger to spray perfume and an alternating movement when the perfume is sprayed on each side of the face and showing the gloves on each hand. The fact that the man’s movements are broader and symmetrical, and the woman’s movements shorter and asymmetrical, create the perception that the man’s movements are more sonorous.

Picture 6
Person transfer - man in “Little Ping Pong Ball”

Picture 7
Person transfer - woman in “Little Ping Pong Ball”

With respect to the third level of sonority, the signs that focus on the hands, the sign referring to the little ball is an example that highlights the shape of one of the hands (picture 8). The signer’s gaze also highlights this. In this story, the signer constantly shifts role between narrator and characters, mostly the woman and the man who are competing, and the referee, who, besides being a character, seems to act as an observer, as well as the audience. The gaze of the narrator directed to the audience does not highlight any sign, but the eyes of the observer or narrator may direct the viewer to the hands (especially the spotlight or reactive gaze, the only ones occurring in this story) and point out the sign (picture 9).

Picture 8
Sign that highlights the handshape in “Little Ping Pong Ball”

Picture 9
Spotlight gaze and reactive gaze in “Little Ping Pong Ball”

However, the rhythmic sequence that draws the most attention in this video occurs when the ball is hit from side to side. The signer incorporates the man and the woman hitting the ball, the referee following the movement of the ball, the ball, and signs the audience’s gaze. In this constant exchange of roles, he alternates the direction of the trunk, of the manual sign, and of the head; repeats several signs; and changes the sign speed to make it faster or slower (picture 10). This passage presents a rhythm - movement size and speed - and sonority relationship, by showing that the most proximal joints are not the only ones to produce more or less sonorous signed units. The emphasis on the trunk, shoulder, or head altered the size of the movement and influenced the signing speed. Thus, the more sonorous signs are the use of the trunk as non-manual elements and decreased signing speed. On the other hand, the less sonorous signs are those that used the head as a non-manual element and were performed fast (beginning and ending of the signing unit). Examples of enhanced movement and reduced speed are the torsion of the trunk when the man or woman holds the bat, and trunk inclination to the side when the narrator incorporates the ball. Examples of reduced movement and increased speed are seen in the referee’s head movement, looking from one side to the other, and the ball going from one side to the other. This shows a pattern since the movements are broader at the beginning and end of the signing unit than they are in the rest of the sequence.

Picture 10
“Little Ping Pong Ball” Rhythmic sequence

The close-up and distancing movements are exemplified here with a sequence (picture 11), which shows a close-up of the referent (the little ball) and then a distance shot. After the referee grabs the ball with one hand, and the signer then shows a hand covering his face, the ball is incorporated (close-up) and is taken again by the referee (distance shot).

Picture 11
Little ball’s close-up and distance shots in “Little Ping Pong Ball”

In this story, it was possible to think about how the artist highlights certain signs. Most signs use proximal joints, while some signs stand out because of the size of the arms’ movement, even within the video frame’s limited space; the emphatic use of the trunk; the alternation and repetition of motion; and signs with rhythmic and visual effect. Though the effect on the audience is not the focus of this research, as a researcher and observer, I see that the artist emphasizes certain signs through sonority to highlight the tension in the scene. The audience and the referee (and ultimately we, the spectators) observe everything and wait for the outcome of the championship. In fact, because it is signed directly for a video, we do not expect the signs to be large or sonorous to draw the audience’s attention so they can be seen from great distances.

6 Discussion

Comparing “The King’s Parrot” and “Little Ping Pong Ball” it can be seen that, when signing to a large audience, as in a theater, or to a video recording, not only the signs, but also the signer’s body and gaze, behave in a different way. Firstly, the space can be broader or more restricted, thus influencing the signing. For example, the movements along the stage or knees bending, that cannot be made in the video (unless the signer goes further from the camera). The size of the arms’ movement also tends to be reduced in video. Figure 2 shows that “The King’s Parrot” has signed units that focus on the whole body (31 signs = 20%), while “Little Ping Pong Ball” has none. The percentage of level 2 signed units - arms’ amplitude and trunk inclinations - is bigger in the “The King’s Parrot”, presented live (76 signs = 48%), than in the “Little Ping Pong Ball”, filmed for video (36 signs = 36%). The video makes it possible to focus on a specific part of the body (whole body, hands, face), if desired, but the “Little Ping Pong Ball” kept a general shot. However, although this story has not used the camera for this purpose, the signed units direct the viewer’s gaze predominantly to the hands - Level 3 - (65 signs = 65%). Thus, in the live performance of “The King’s Parrot”, most of the units focus on trunk movements or arms’ extension, while in the “Little Ping Pong Ball” video, influenced by space restriction, most signs bring the viewer’s attention to the signer’s hands. The holds are meaningful, through motionless signs (6 signs = 3% in “The King’s Parrot”). These can also be considered sonorous, since they are “frozen” and are viewed by the spectator for a longer time, increasing their impact.

These space and audience differences seem to result in more or less sonorous signs. However, stating that the manual and non-manual signing actually change when in front of an audience or a video camera would only be possible if the same story were analyzed in these two different circumstances.

7 Concluding Remarks

In spoken language, phonetically, a sound is most audible and perceptible at great distances according to the degree of opening of the oral cavity, which determines the amount of resonance and sound amplitude. In addition, prosody also influences the sonority, as in the sound loudness. In sign language, a movement is more sonorous if performed with a more proximal joint. However, there are other ways to express the visual sonority, especially in artistic signing, with movements across the stage, size of the signs’ movement, non-manual elements, rhythm, and signs that transfer space using different scales.

The present study summarized in three levels the possibilities of making the movement more sonorous through signs and use of space: focusing on the use of the legs and the entire body, the size of the movement of the arms, trunk movements or in the hands. In addition, this study demonstrated that the physical space in which the signer acts and the audience size seem to greatly influence the way the signs are performed, as can be seen when comparing a live performance, with large-scale stage and large audience, to a video with restricted space and an audience separated from the artist in space and time.

Regarding non-manual elements, the trunk has proven to be important for sonority, because the signs that used the trunk were notable for their size. There is clearly a relationship between the trunk and arms’ movement and the speed, and, therefore, with rhythm. Signed units that had their speed decreased or used slow motion effect were those with large arm and trunk movements, and those that had increased speed used short movements. The gaze also plays an important role in visual sonority, especially those categorized as spotlight and reactive, because they complement the existing sonority in the manual sign. Additionally, rhythm is allied to visual sonority when a movement is repeated, alternated, has its speed altered, or is symmetrical, or when a signed unit employs pauses in sonorous signed units articulatorily.

Besides bringing new perspectives in research on artistic signing, visual sonority can show the richness of these signed productions and reveal how the audience and space are considered by the artist when preparing their work, in order to obtain a more effective communication.

In short, visual sonority in the literature of Brazilian Sign Language - or more specifically, in the analyzed stories - can express itself in different ways, depending on the space, the audience, the performer’s choices, and style. In its purely linguistic concept, visual sonority assigns different values to the articulators, but in literature it emerges as a new research area, showing new aspects when relating to non-manual elements, rhythm, and other elements that will be revealed in further research.

References

• BAHAN, B. Face-to-Face Tradition in the American Deaf Community: Dynamics of the Teller, the Tale, and the Audience. In: BAUMAN, D. L.; NELSON, J. L.; ROSE, H. M. Signing the Body Poetic: Essays on American Sign Language Literature. 1. ed. Los Angeles: UC Press, 2006. p. 21-50.
• BAUMAN, H.-Dirksen L. Getting-out of Line: Toward a Visual and Cinematic Poetics of ASL. In: BAUMAN, H. et al. Signing the Body Poetic: Essays on American Sign Language Literature. Los Angeles: UC Press, 2006. p. 95-117.
• BAUMAN, H.-Dirksen L. Redesigning Literature: The Cinematic Poetics of American Sign Language Poetry. Sign Language Studies, Washington, v. 4, n. 1, p. 34-47, 2003.
• BRENTARI, D. Establishing a Sonority Hierarchy in American Sign Language: The Use of Simultaneous Structure in Phonology. Phonology, Cambridge, v. 10, n. 2, p. 281-306, 1993.
• BRENTARI, D. A Prosodic Model of Sign Language Phonology. Cambridge/London: Massachusetts Institute of Technology, 1998.
• BRENTARI, D. Phonology. In: PFAU, R.; STEINBACH, M; WOLL, B. Sign Language: An International Handbook. Berlin/Boston: Gruyter Mouton, 2012. p. 21-54.
• CRASBORN, O. Phonetics. In: PFAU, R.; STEINBACH, M; WOLL, B. Sign Language: An International Handbook. Berlin/Boston: Gruyter Mouton, 2012. p. 4-20.
• CUXAC, C.; SALLANDRE, M. Iconicity and arbitrariness in French Sign Language. 2004. Disponível em: <Disponível em: http://archive.sfl.cnrs.fr/sites/sfl/IMG/pdf/Verbal-Signed-Languages_02_Cuxac-Sallandre_last_draft_2007.pdf >. Accessed on March 11, 2016.
  » http://archive.sfl.cnrs.fr/sites/sfl/IMG/pdf/Verbal-Signed-Languages_02_Cuxac-Sallandre_last_draft_2007.pdf
• KANEKO, M.; MESCH, J. Eye Gaze in Creative Sign Language. Sign Language Studies, Washington, v. 13, n. 3, p. 372-400, 2013.
• KLAMT, M. M. O Ritmo na Poesia em Língua de Sinais. 2014. 147 p. Masters Thesis. (M.A. in Linguistics) - Universidade Federal de Santa Catarina, Post-Graduate Program in Linguistics Florianópolis, 2014.
• KLIMA, E.; BELLUGI, U. Poetry and song in a language without sound. Cognition, Lausanne, v. 4, p. 45-97, 1976.
• KRENTZ, C. B. The Camera as Printing Press: How Film Has Influenced ASL Literature. In: BAUMAN, H. et al. Signing the Body Poetic: Essays on American Sign Language Literature. 1. ed. Los Angeles: UC Press, 2006. p. 51-70.
• LUCAS, C. et al. The Effect of New Tecnologies on Sign Language Research. Sign Language Studies, Washington, v. 13, n. 4, p. 541-564, 2013.
• MACHADO, F. de A. Simetria na Poética Visual na Língua de Sinais Brasileira. 2013. 149 p. Masters Thesis (M.A. in Translation). - Universidade Federal de Santa Catarina, Post-Graduate Program in Translation Studies, Florianópolis., 2013.
• NAPOLI, D. J.; SANDERS, N; WRIGHT, R. On the Linguistic Effects of Articulatory Ease, with a focus on Sign Languages. Language, Washington, v. 90, n. 2, p. 424-456, June 2014.
• PERLMUTTER, D. M. Sonority and Syllable Structure in American Sign Language. Linguistic Inquiry, Cambridge, v. 23, n. 3, p. 407-442, Summer 1992.
• PETERS, C. The Oral Tradition: Deaf American Storytellers as Tricksters. In: PETERS, C. Deaf American Literature: from carnival to the canon. Washington: Gallaudet University Press, 2000. p. 52-77.
• PIMENTA, N. A tradução de fábulas seguindo aspectos imagéticos da linguagem cinematográfica e da língua de sinais. 2012. 165 p. Masters Thesis (M.A. in Translation) - Universidade Federal de Santa Catarina, Post-Graduate Program in Translation Studies, Florianópolis., 2012.
• RAMOS, B. O Papagaio do Rei. Festival de Folclore Sinalizado. Os Craques da Libras. 2014. Unpublished.
• ROMANO, R. S. Bolinha de Ping-Pong. 2009. Available at: <Available at: https://www.youtube.com/watch?v=VhGCEznqljo >. Accessed on March 20, 2016.
  » https://www.youtube.com/watch?v=VhGCEznqljo
• ROSE, H. M. A critical methodology for analyzing American Sign Language literature. 1992. 192 p. Doctoral Dissertation. (Ph.D. in Philosophy), Arizona State University, Department of Communication, Tempe, 1992.
• ROSE, H. M. The Poet in the Poem in the Performance: The Relation of Body, Self, and Text in ASL Literature. In: BAUMAN, H. et al. Signing the Body Poetic: Essays on American Sign Language Literature. Los Angeles: UC Press, 2006. p. 130-146.
• SANDLER, W. A Sonority Cycle in American Sign Language. Phonology, Cambridge, v. 10, n. 2, p. 243-279, 1993.
• SMITH, S.; CORMIER, K. In our out? Spatial Scale and Enactment in Narratives of Native and Nonnative Signing Deaf Children Acquiring British Sign Language. Sign Language Studies, Washington, v. 14, n. 3, p. 275-301, 2014.
• SUTTON-SPENCE, R.; BOYES BRAEM, P. Comparing the Products and the Processes of Creating Sign Language Poetry and Pantomimic Improvisactions. Journal of Nonverbal Behavior, New York, v. 37. n. 4, p. 245-280, Dec. 2013.
• SUTTON-SPENCE, R.; LADD, P.; RUDD, G. Analysing Sign Language Poetry. New York: Palgrave Macmillan, 2005.
• VALLI, C. Poetics of American Sign Language Poetry. 1993. 156 f. Masters Thesis (M.A. in Linguistics and ASL Poetics). - Union Institute Graduate School, Cincinnati, 1993.

• 1
  I am thankful to Bruno Ramos and Rimar Romano for giving me permission to use their stories for this work, part of my doctoral research, funded by the National Council for Scientific and Technological Development (CNPq) and linked to the Post Graduate Program in Linguistics at the Federal University of Santa Catarina. Special thanks to Ronice Quadros and Rachel Sutton-Spence for their efforts in guiding this research.

• 2
  Lucas et al. (2013) point out that in the Youtube website users interact and make comments about the video using written text, and that in the Deafvideo tv, it is possible to answer to the video content with another video.

• 3
  ELAN is a software for language analysis, developed by the Max Planck Institute, which assists the researcher in transcription, organization, and data analysis, and allows one to pause the video and count the number of frames.

• 4
  Cuxac and Sallandre (2004) use the term “unit” broadly, and I use here “signed unit” or “signs”, to refer to all units in signs, whether they are conventional signs, gestures, or imagetic descriptions (transfers).

Publication Dates

• Publication in this collection
  23 Mar 2017
• Date of issue
  Apr-Jun 2017

History

• Received
  24 May 2016
• Accepted
  23 Sept 2016