Same-Language-Subtitling (SLS):

Using Subtitled Music Video for Reading Growth

Greg McCall

Carmen Craig

Association for the Advancement of Computers in Education

World Conference on Educational Multimedia, Hypermedia and Telecommunications (EDMEDIA) 2009

Honolulu, Hawaii

wgregmccall@gmail.com

Abstract: This paper introduces the use of Same-Language-Subtitling (SLS), or Subtitled-Music-Video as a repeated reading exercise. The purpose of this study is to demonstrate the effectiveness of SLS related activities on Reading Engagement and Reading Proficiency. For the basic SLS exercise students repeatedly viewed brief excerpts of musical video with dynamic subtitles while completing matching cloze-style worksheets. In the advanced SLS activity students used Karafun, a karaoke program, to create and manipulate SLS projects for class presentation.  Student attitude, engagement and reading proficiency levels improved during the course of this study. There is a need to develop further applications and lesson plans to take advantage of this learning resource.

Discussion

Historically there have been many attempts to demonstrate that video captioning can impact learning for a wide range of readers. When we reviewed research on Clozed Captioning/Subtitling and Literacy before 2000 it was difficult to site a study that did not demonstrate limited results. However, the technological growth in video and computer programs over the last ten years has increased the availability and quality of captioning software programs, while greatly decreasing the costs associated with subtitling media. Further the decrease in presentation costs and increased interest in subtitling and Karaoke has created more opportunities to study the use captioning as a reading resource. There is an ongoing study in India television using music video and various Hindu and Indian dialects. This application of captioning is called “Same-Language-Subtitling” (SLS) and it  has demonstrated positive results.

Illustration 1: Same-Language-Subtitling styled music video activity.

                SLS is the practice of using high quality text subtitling and video editing technologies (comparable to Karaoke) with music video and multi-media presentations as a repetitive reading resource. The activity combines captioned music videos with a response worksheet to create group rehearsed reading exercises. The visual format of SLS is very similar to Karaoke, where the captioning changes color in exact synchronization to the audio. However, with SLS as compared to Karaoke, the audio model is usually a very strong language model. Karaoke normally uses secondary vocal models or fades the vocal model into the background. With SLS the strong dynamic vocal model is used, and the subtitling is designed such that even emergent level readers are able to visually track the lyrics as they are performed. In addition, SLS generally uses a response activity, to increase viewer engagement in the activity.

SLS generally uses highly repetitive, dynamic audio/video resources such as music, poetry, audio-books, or famous speeches as the base resource. While music strongly reinforces SLS, any strong audio model can be used as a starting base. The ‘Musical Theatre’ genre provides some of the best resources for SLS activities. Musicals generally have a literary base or have story elements in common with novels and provide rich text resources. A teacher or student can find lyrics or texts to target any reading level and response activities can then be added to target specific reading skills. The activities can be used to teach phonics, to explicitly model language, to increase sight word recognition, to increase fluency, and to build on reading comprehension. From the basic SLS format there are many possible accommodations and projects for students to explore.

Further, students can compose their own SLS multi-media presentations and interact with subtitled media as a process. For this activity the student locates an audio source, creates and edits a script and using a word processing program plus video editing software then creates a unique SLS presentation. The hard work comes in as the student matches his visual script in content and in synchronization to the audio model. The SLS approach is innovative because of multiple reading opportunities and benefits packed into a deceptively simple approach.

 Research by PlanetRead and Dr. Brij Kothari

Dr. Brij Kothari of Cornell University coined the term Same-Language-Subtitling in 1996 to describe his application of Karaoke-styled captioning.  Kothari developed his SLS using Bollywood film songs on TV to promote literacy in India. His massive SLS study has received awards from the Tech Museum of Innovation (San Jose), Development Marketplace (World Bank), and the Institute for Social Inventions (London). His project has major financial support from the Google Foundation and the OLPC group has plans for incorporating his application into their educational package.

Kothari is currently studying the impact of SLS on illiterate and emergent level readers. In his article “Same language subtitling: a butterfly for literacy, Kothari proposed using the SLS format to raise the literacy in India on a massive scale. In that paper he documented his initial trials at a primary school, which was a two-phase study designed with a controlled pre- and post-test. The experiment was conducted over a three-month period at a rural Indian Public Primary School. Three groups of 46 were formed with an equal number of children from the fifth and sixth grades, and with the same male-female ratio. Student populations were primarily low-income. All students were pre-tested for mono-syllabic word recognition. Each group received 18 hours of class time with presenters following three different programs targeting vocabulary. The first group watched five subtitled Hindi film songs each session, three sessions a week. The second group viewed the same program without subtitles. The third group did directed reading activities. After three months the same monosyllabic word recognition test was repeated. Group means were then compared statistically. The results demonstrated a significant improvement in phoneme acquisition for the SLS group. (Kothari et al, 2002).

                Kothari’s current application of SLS is an ongoing experimental presentation on Indian National TV. Basically, they take popular movie music and add strong subtitling. Two nationally telecast Hindi film song programs (Chitrahaar and Rangoli) are used with the SLS format to provide reading practice for at least an hour per week to hopefully more than 100 million struggling and emergent readers. His current usage of SLS uses newspaper response puzzles and mail-in contests to engage the viewing audience with the musical texts. This study has revisited more than 13,000 non-readers and early-readers both children and adults, randomly drawn from five Indian states, over a six year period first in 2001 as a baseline and then again in 2003, and recently in 2007. All three surveys were done independently by Nielsen’s ORG-Centre for Social Research, and applied the same reading and writing assessment tools to the same individuals. This study attempts to definitively answer the question: Can SLS improve reading skill for in-school children and out-of-school adults? This is a massive, ongoing, long-term study that proves that SLS can significantly impact non-readers through emergent level readers for phoneme, syllabic, and vocabulary acquisition. Further, Kothari’s study demonstrates that SLS can significantly impact reading engagement and maintenance amongst functional level readers. According to Kothari, SLS strengthens decoding capacity, promotes functional reading levels, stanches reading skill loss and increases overall reading behaviors. (Kothari & Bandyopadhyay, 2007)

SLS in the Classroom

Given the strength of Kothari's research, one would expected to see a dramatic rise in research and use of this format in both the classroom settings and in public broadcasting. Just a few years ago SLS as a process had few resources and could be cost prohibitive for small scale classroom use. Now, there are many computer programs available that give teachers and students the ability to compose their own Karaoke/SLS presentations. Ideally, there should be karaoke style subtitling for all music based programs on TV, video and the web. Currently, most television and movie content is required by law to include captioning, while music video is one of the few exceptions and most music videos do not come with any form of subtitling. In contrast to MTV and music video, DVD-Movie-Musicals generally include good quality subtitling (not quite karaoke level) and can provide dynamic material for the basic SLS activities.

                There are a number of concerns with using SLS in the classroom. First, Kothari’s early SLS research looked at school children in early reading stages, however he did not explore usage as a classroom activity or usage with secondary students and his study was targeted at emergent level readers not struggling to functional level readers. Further, this application has had very little study in application to English speakers learning to read English. Kothari's primary focus has been to expand the SLS application to India’s national television level to address widespread illiteracy in India. His study was not actually directed towards developing classroom applications of the technology or to addressing a wider range of disengaged readers. Second, while Kothari’s study is showing good skill retention for both functional and literate level readers, it is also showing limited results with functional level readers for proficiency growth. This low impact on a wider level of struggling readers may simply be a function of Krashen’s hypothesis of language acquisition: that the input model needs be slightly above the student’s language level (Krashen, 1985:72-73). Kothari's SLS ‘input’ targets lower level readers, ergo his SLS impacts lower level readers. Third, Kothari's study does not explore the possibilities of SLS and computer technology in the classroom where students can be an active part in creating SLS materials.

The Study

            This study attempts to determine whether the use of Same-Language-Subtitling (SLS) can be effective in a Secondary School Special Education English and Reading classroom. Can the simple addition of twenty minutes a day of subtitled Broadway Musicals and cloze-style worksheets really impact learning? Is it possible to incorporate this format without taking away from current reading programs? Can SLS be used equally well with a wide range of American High School Students? Prior to this study the team predicted that the SLS format would increase student engagement, and that there would be a corresponding impact on reading growth.

The study group believed that to demonstrate the wide scale efficacy of SLS that it was important to experiment with music that was outside of the students general experience range and with lyrics that were significantly higher than the groups’ instructional reading levels.  The team agreed in advance to use the “Les Miserables in Concert” DVD and “Cats the Musical” DVD and “Big River –the Broadway Recording” CD and corresponding readings as the base materials for the intervention SLS activities. This provided text material that ranged between an 800 lexile reading levels (5th to 6th Grade Equivalency) to the 1200 lexile level (12th G.E).  Teachers in the control groups agreed to use or assign readings from corresponding resources. The team also wanted to include the students themselves in creating media for classroom use.  Karafun (a free karaoke creation program) and several other editing programs were made available to teachers and students for creating and editing media.

Methodology

            This study followed quantitative and qualitative research methodology. This consisted of documenting pre- and post treatment reading assessments, observing classroom behaviors, monitoring classroom activities, and conducting student engagement surveys. A time monitoring sheet and an engagement rubric was developed and special education staff made random observations before the intervention and throughout the intervention period.

                For baseline and comparative purposes, approximately 1200 students were tracked on multiple standardized reading assessments. Pre-and post implementation scores were tracked on Accelerated Reader’s STAR computerized reading assessment and compared to both annual Stanford Diagnostic Reading Test, Scholastic Reading Inventory (SRI), and to the Gates-McGinnity Reading Test. 203 Special Education Learning Disabled (LD) students were screened further and subjected to triangulation and validity checks on a comparison between two or more reading tests within a sixty day period prior to the SLS intervention. Assessment data was examined to determine if the variance of scores predicted by the Pearson Product-Moment Correlation was greater than the actual variance. This screening and testing process identified 158 LD students to be used as the study group, 149 of whom make up the data group. These students had reading levels ranging between 2.2 G.E. and 9.4 G.E., and had an average reading level of 5.37-Grade Equivalency (GE). Further, the data group averaged a .24 GE gain on reading scores during the previous school year.  Students from this group were randomly placed into five possible settings over twenty classes. This resulted in 51 students being placed in the SLS intervention group and 98 LD students in other special education or regular education classes. This also resulted in an additional 14 students in SLS group and 26 students in control groups for whom pre/post test data did not pass validity checks or was not available. The data for this group was examined; and as predicted, the likely false low data would have inflated gains for all groups. Further, it should be noted that reliability testing was not done for non-special education populations (General Education Students); however, for comparison purposes, full school data was gathered on all students.

                As much as possible, for the initial study the control group classes had similar ranges of students. Reading levels in all classes ranged between second grade equivalencies to above ninth grade equivalencies. Student ages ranged between fourteen and nineteen. For the most part, at the beginning of this study the majority of these students would be considered novices with limited exposure to technology. While the mastery of computer technology was not the main focus of this study, it should be noted that the secondary SLS activity can be technologically intensive. In balance to this, within the control-groups one of the teachers also made extensive use of technology for research and writing assignments.

                The intervention portion of this research project documented the addition of two types of SLS music exercises: first, engaging in SLS viewing/response activities; and second, engaging in creating SLS/Karafun projects. First; for the last 12 weeks of the 2004-2005 school year, for 15-20 minutes at the beginning of classes the SLS group:

1.       watched three brief repetitions of SLS musical video; (SLS played before classes, three repetitions after bell to start each class, and played during passing times.)

2.       while watching: completed cloze-style worksheets (Lyrics with blanks based on a pattern, such as type of word, related vocabulary, alliterations, etc.), this response sheet also included short responses to comprehension questions.

The entire activity took less than 20 minutes per day. The source materials for this activity came from three popular story-musicals containing lyrics well above students’ instructional reading levels and were paired with class reading activities: Les Miserables with Hugo's novel, ‘Cats’ with ‘Old Possum’s Book of Practical Cats’, 'Big River' with Huckleberry Finn. The activity included simple cloze worksheets and echo reading sing-a-longs.  The idea was to promote the active tracking of the subtitling. Exercise was targeted to support reading strategies, story elements and vocabulary from related reading assignments. The materials for the first eight weeks of intervention were primarily straight from the DVD or teacher generated Karafun media, and provided a model for later student composed SLS presentations.

During the last six weeks of the intervention period, for the secondary SLS activity, students produced subtitled multi-media files from the ‘Big River’ CD during Lab (study hall time periods). This activity took a minimum of 90 minutes per week for initial projects. One of the indictors for engagement was the additional time students voluntarily devoted to their multimedia projects. This activity primarily used ‘Karafun’ (a free karaoke production program), and Microsoft Word and PowerPoint programs. These activities centered on creating and manipulating subtitling with audio and visual files. This activity started with students producing Karaoke versions of their favorite songs and quickly progressed to work products for group SLS.

For the secondary SLS activity students did:

(1) create exact matching transcripts for their assigned audio content, including marking syllabication

            (2) combine text, audio and visual files to create multimedia file

            (3) synchronize the text timing with audio;

                (4) add background video and convert composition into presentation format;

                (5) make engagement worksheet or vocabulary activities. 

Steps one and three were the most time and labor intensive, since these steps required the students to listen to audio multiple times and to create, match and synchronize texts to the audio file. These projects were then incorporated into the classroom SLS presentation activity.

                This format was then expanded upon and used in the writer’s classroom for an additional three years. During that time the class actively used SLS viewing and production activities along with supporting reading activities (Read-a-long, Audio-books, Sustained Silent Reading, Reading response/exploration activities). Students used Karafun and additional editing programs within English class time. In addition to some very well produced Karaoke products, students recorded SLS audio-text books, created poetry slams, explored American rhetoric, and created vocabulary mash-ups with popular music. For comparative purposes, the reading assessments were revisited for an additional three years to document reading growth in the SLS classroom versus SLS students who went on to other English placements versus students who were not exposed to SLS activities

Results

The primary goal of this study was to determine if the addition of Same-Language-Subtitled (SLS) music lyrics as a supplementary reading activity could positively impact the problem of low reading growth in a high school special education English classroom. A time monitoring sheet and an engagement rubric was developed and random observations were made before the intervention and throughout the intervention period. For control and comparison purposes use of reading activities and use of video were carefully documented, and the detailed time monitoring records can be viewed at www.sls4reading.com The following table (Table 1) provides a summarized analysis of averaged number of minutes per week for various classroom activities.

Table 1: Averaged minutes per week on various classroom activities (summarized), previous 12 weeks to    intervention compared to weeks 9 through 12 of SLS intervention activity.

                For the SLS Groups during this time period the average weekly reading times went from 95 to 105 minutes per week. Based on this observation for the intervention model, the SLS activity did not take time away from classroom reading activities, and indeed the SLS group teacher was able to slowly increase time students spent in reading activities.  Further, if the SLS activity is added to reading activities (105 plus 90), the total reading activities increased  to 195 minutes per week or from 29% to 60% of class time available. A review of the detailed data sheet shows that the SLS activity came primarily from classroom management and lecture time. On the subject of 'time-spent-reading' there are some differences between groups in types of reading activities and amount of time spent reading; however, there is not a major difference in total reading times over the course of the study within the control groups. This sets the basics of the experiment, while there is very little change in reading times there should also have been little change in rate of reading growth.

                The following chart (Table 2) illustrates the change in observed focus during group reading activities. Detailed data and observation tool is available online at www.sls4reading.com

Table 2: Observations on SLS Group - Engaged Reading Rubric and Use of Video/SLS Activity.

                This study anticipated that the addition of classroom SLS activities would correspond with an increased engagement in other classroom reading tasks. This second goal (increased engagement in reading tasks) was met as evaluated by qualitative observation rubrics conducted by SPED staff. The observations were taken weekly on a random basis during reading tasks (Sustained Silent Reading, Directed Reading, etc,) and during the SLS activities. 

                Another interesting result of study methodology, a careful review of comparison groups shows there was some shift to using Captioning and/or response activities with video. However when testing data was reviewed there is no indication that the increase usage of subtitled video within the control groups had any statistical impact on reading proficiency growth within control groups. This would indicate that subtitling on its own, has significantly less impact compared to SLS (the addition of music, repetition, and response worksheets).

The previous  chart (Table 3) summarizes reading assessment data. The SLS intervention group showed marked improvement in reading; a gain of an average of SS 66 points, or GE 0.7, or more than 25 weeks growth in a 12 week period. In (Table 3), for comparisons purposes, data is included from students [Non-validated (14)] that participated in SLS group. These are students whose pre/post-test data failed the reliability checks. As predicted, the [Non-validated (14)] group tested showing higher growth patterns and including their data in either the SLS or control groups would have inflated growth scores.

When data from full school population (1000+ students) and LD control groups was reviewed there were a number of patterns that can be identified for comparison purposes. First, if reading levels are compared across multiple years, typical grade equivalency gain can be readily identified for both the general school population and for subgroups: previous to this study average growth for LD students was .24GE, and average growth for general education students was .41G.E. Second, there is some real loss that happens over the summer breaks. After screening assessments to eliminate false low tests at the beginning of the school; previous to this study the average loss for LD student was GE 0.12, and for general education students the average loss was GE 0.062.