DATA ANALYSIS
EXAMINING MY DATA
Considerations For This School Year
Students' constant need to adapt to change had a tremendous impact on their performance. Students had to transition often from remote to in-person and back to remote, in a cycle that interrupted their learning more than once. For this reason, the study met multiple internal validity challenges:
​
-
History: Students struggled during the pandemic, as they tried to adapt to new circumstances. Teaching and learning had to be modified many times in an attempt to support in-person and remote students simultaneously.
-
Experimental Mortality: Many students who were involved in the study ended up moving to another district or enrolling during the second half of the study. When it comes to comparing pre-study and post-study data, the results become invalid or skewed. There were many absences during the study period, which means students missed instruction, lost a day of writing, or participated in one affective survey and not the other (pre-study survey/post-study survey). Due to the fact that many student’s parents did not return the study participation consent form, more than half of my students were excluded from the data analysis.
-
Experimenter Bias: Although students were assessed with the same rubric, I as the teacher and the experimenter found myself holding my higher students to a higher standard, and therefore grading them harsher than my lower students.
RESEARCH QUESTIONS:
-
Does the use of a writer’s checklist improve students’ organization when writing a personal narrative?
-
Does a writer’s checklist increase students’ self-efficacy and motivation in personal narrative writing?
-
Does a writer’s checklist affect overall writing quality when writing a personal narrative by prompting students to use the following elements:
-
beginning, middle, and end
-
story sequencing
-
dialogue
-
vivid description
-
etc.
-
-
Does a writer’s checklist have a greater impact on students of a particular ability level?
Affective Surveys -
Changes Between Pre and Post Study Survey
Of the 13 students whose data was analyzed, 8 (61.54%) students' confidence in their writing ability improved, 4 (30.77%) students' confidence stayed the same, and 1 (7.69%) showed a decline in confidence.
Of the 13 students whose data was analyzed, 8 (61.54%) students' confidence in their ability to write a personal narrative improved, 1 (7.69%) student's confidence stayed the same, and 4 (30.77%) students showed a decline in confidence.
Of the 13 students whose data was analyzed, 12 students (92.31%) expressed they knew what goes into writing a good personal essay. The data shows an increase from the pre-study survey.
Of the 13 students whose data was analyzed, 11 students (84.62%) claimed they thought they could write another personal essay without their teacher's help. This statement was used to measure students' self-efficacy regarding narrative writing. The data shows an increase from the pre-study survey.
Of the 13 students whose data was analyzed, 12 students (92.31%) claimed they thought that the writer's checklist helped them write their personal essay. The data shows an increase from the pre-study survey.
Student Essay Scores -
Assessing Organization & Overall Writing Quality
Context for Results:
-
Student writing was assessed by the following criteria: Organization, Ideas & Evidence, and Language for an overall score.
-
For the results below, only Organization and Overall Quality were analyzed for change and improvement (Pre-Study Assessment and Post-Study Assessment).
-
Students were assessed on a 4-point grading scale, but scores were measured out of 100 points to aid in data analysis.
-
Letters and colors were assigned to students depending on their ability level:
-
Student A & Blue = High-Achieving Student (Honors Level)​
-
Student B & Green = Middle/Average Level Student (Note: this student was remote during the study period)
-
Student C & Yellow = Low-Achieving Student (Enrolled in a co-taught ELA class)
-
Summary of Results: Organization in Writing
A paired-samples t-test was conducted to determine the effect of the use of a writer’s checklist on students’ organization when writing a personal narrative. There was no significant difference in the scores prior to implementing the writer’s checklist (M=63.46, SD=25.22) and after implementing (M=76.92, SD=18.24) the writer’s checklist; t(13)= 1.62, p = 0.06558. The observed effect size d is medium, 0.45. This indicates that the magnitude of the difference between the average and μ0 is medium. These results suggest that the use of a writer’s checklist had no effect on students’ organization when writing a personal narrative. Specifically, the results suggest that the use of a writer’s checklist did not improve organization in students’ personal narratives.​
​
-
Student A improved writing organization by a whole letter grade (or 25 points). This means the student moved from proficient to advanced in organization when comparing pre-study and post-study writing assessments.
-
Student B did not make any improvements in writing organization, dropping a letter grade (or 25 points). It is important to consider that this student was full remote during the study period and I had very limited interactions with this student due to that fact. The results indicated that student moved from proficient to basic in organization when comparing pre-study and post-study writing assessments.
-
Student C improved writing organization by a whole letter grade (or 25 points). This means the student moved from proficient to advanced in organization when comparing pre-study and post-study writing assessments.
Summary of Results: Overall Quality in Writing
-
Student A improved writing quality by a whole letter grade (or 25 points). This means the student moved from proficient to advanced overall when comparing pre-study and post-study writing assessments.
-
Student B did not make any significant improvements in writing quality, scoring the same on both the pre-study assessment and the post-study assessment. It is important to consider that this student was full remote during the study period.
-
Student C improved writing quality by half a letter grade (or 13 points). This means the student moved from proficient to approaching advanced overall when comparing pre-study and post-study writing assessments.
Use of the Writer's Checklist
​
HOW WAS THIS MEASURED?
-
Answers to the question on post-study survey: How much do you think you used your writer's checklist during your writing of the personal essay?
-
Anecdotal notes taken during writing conferences
-
Analyzing student samples for evidence of use
I only peeked at the writer's checklist. For editing, I read my essay and looked over the editor's checklist.
"I think I have the order of my story down, but it's boring. I think I need to add sensory words. Like, really describe what I see and feel."
Student a
I only peeked at the writer's checklist. For editing, I read my essay and looked over the editor's checklist.
"I feel like I already know how to write and I think writing with the checklist will make me sound robotic."
Student b
(Remote)
I only peeked at the writer's checklist. For editing, I read my essay and marked on the editor's checklist where I had done those things in my essay.
"My dad and I looked over my essay and we looked at the checklist too and he told me he thought it was good to go. My dad is a good writer and he told me to use the checklist you gave us so I did."
Student C
This student focused on using emotion and feeling in the writing to create suspense and excitement for the reader.
SAMPLE FROM
Student a's personal essay
As I open the doors, my hands begin to heat up and sweat, my heart pounding in my chest and pumping my veins full of adrenaline. Now the cutscene begins, my first look at what the boss looks like. The screen shows two knights fighting each other, more knights, the same kind, are shown lying on the floor, dead. One of them slashes the other with a dagger and the other goes for the killing blow, and stabs the other, ending him and then he turns to me and readies himself for the fight, even though he’s clearly exhausted. Now as the screen fades, my heart pounds harder, my adrenaline rushes even crazier than before, and my hands sweating more as my nervousness and excitement grows with every second as I wait for the battle to start. As the screen starts to get brighter I get to see the name of the boss and his fellow fallen knights, The Abyss Watchers. He dashes towards me with a great sword slashing furiously, and in lots of small fast bursts, and it’s too much for me, so I die. This process happened too many times to count, each time going in giving me another and slightly painful adrenaline boost. But little did I know how different this run was going to be.
SAMPLE FROM
Student B's personal essay
The reason I chose to read write about this is because I mostly talk about books and that always motivates me. There are other things that motivate me but this is the biggest one for me. It’s probably a boring topic but nothing has motivated me as much. Everything I read in books always makes me want to change and sometimes it doesn’t have a good outcome. My parents called me manipulative and thought I should be reading different kind of books. Usually when I see a show or movie I always try to see if there’s a book because the books are always better.
This student establishes the chosen topic (a time he/she was really motivated to do something), but does not focus on a particular event.
This student explains why this topic is important to them, but does not focus on only one experience. This requirement can be found under CONTENT on the Writer's Checklist.
After writing the essay, Student C used the Editor's Checklist to make sure he/she had all the elements of a good personal essay.
Student C's observations are marked in green.
Triangulation of Data: ​
The three data collection methods that told the story of student learning throughout the research period were: the affective surveys, students' assessment performance, and anecdotal records.
​
The affective surveys told the stories of individual students and their unique relationships with writing. These surveys allowed me to adapt instruction accordingly by giving more support to those students who expressed anxiety about writing. Seeing a change in their attitude toward writing after the study period told me that they gained confidence thanks to the guidance of the writer's checklist.
Students' assessment performances showed me their improvement in their writing organization and overall writing quality. The biggest change occurred in students' self-efficacy and confidence. They went from using huge paragraph frames, templates where they simply had to input their own information, to taking control of their own writing. Although some students needed more guidance than others, every students' writing was his/her own.
The anecdotal records explained students' choices, both with writing and use of the writer's checklist. My observations and individual conferences with students showed me where students were and where they needed to go. It helped me collaborate with students to develop unique goals for each of them, goals that were attainable but challenging. Students were able to monitor their own growth when they saw themselves reaching goal after goal, and in the end changing their relationship with writing.
SAMPLE FROM
Student C's personal essay
This student focused on using intense feelings and exciting punctuation in their writing to keep the reader interested. The student noticed he/she did that as well, as observed by the Editor's Checklist below.
Then we heard something go CRACK. I was scared to death. The sound was a car hitting the railing by the pool. The car was hanging over the edge! Hanging over my dad, uncle and grandparents! I freaked out I was speechless, shook, scared a car was hanging over my family! My mom went over and helped get our things. Aunt Leslie on the other hand just pulled out her phone and started taking pictures. I could not move and my breathing was heavy. I could not look at my family. All I could look at was the car just hanging there. The only thing that was supporting the vehicle was the concrete block!
Intense feelings
Intense feelings
Punctuation
Exciting sounds!