Exploring student thinking, problem solving and collaboration in iPad-supported learning environments

Introduction

Despite successive waves of technological innovation being rapidly adopted by schools, teaching methods and curriculum designs have been slow to evolve in response to the affordances offered by an increasing array of digital devices (Cuban, 2017). This phenomenon is not new, with articles dating back to the early 1980s when digital devices were beginning to appear in our classrooms, commenting that educational computing is following the well-trodden path of other “educational innovations (which) seem doomed to follow a cycle of unrealistic expectations, followed by disappointment, disillusionment, and abandonment” (Maddux, 1986, p. 27). Maddux (1986) terms this the “pendulum syndrome” (p. 27), that centres on education’s susceptibility to “fad and fashion … fashionable movements that suddenly come into vogue, generate brief but intense enthusiasm and optimism, and fall quickly into disrepute and abandonment” (Maddux & Cumming, 2004, p. 511). The earlier writing of Benjamin Bloom alluded to education’s tendency towards uncritical seduction by “fads and nostrums (that) promised to solve our educational problems”, but most of which have since been relegated as “forgotten relics, to the libraries and basements of our schools” (Bloom, 1981, p. 15).

The iPad’s release in 2010 was surrounded by considerable hype and rhetoric, with some education commentators claiming it was a potential “game changer” (Geist, 2011, p. 1)—an affordable and portable device that could radically enhance student learning, by enabling them to collaborate and access information from anywhere at any time. Media at the time hyped the iPad’s release by documenting what bordered on ‘consumer hysteria’, with queues of early adopters waiting anxiously outside Apple stores in an effort to be first to secure their device. Schools also greeted the iPad’s release with enthusiasm. In New Zealand, enthusiasts in early adopter schools such as Albany Senior High School and Orewa College drove a movement towards BYOD (Bring Your Own Device) programmes, where mobile devices (frequently iPad’s as the ‘device of choice’) were phased in across all class levels. These pioneering schools were quickly followed by others across the country enthusiastic to join the BYOD movement. However, schools’ motives and understandings underpinning the mass movement to BYOD are yet to be explored or substantiated, with virtually no research detailing solid evidence of learning or other benefits from movement in this direction. It could reasonably be speculated that one of these drivers might be a concern for a perception of currency or modernness in a ‘competition for student’ environment. Indeed, schools may be concerned about being perceived as ‘lagging behind their neighbours’ in terms of offering students the most up-to-date methods and resources possible. This speculation is consistent with the early work of Cuban (2001), who commented that “a ‘good’ school has become, by definition, a technologically-equipped one … with the term ‘high tech’ conveying an aura of superiority relative to other ‘low tech’ methods and materials” (p. 159). This argument suggests it is important for schools to be seen to be embracing contemporary methods and resources to maintain or extend any competitive advantage they may have.

Given the problematic history of digital innovation adoption in schools and the rapid uptake of mobile devices through BYOD programmes, this study was concerned to bridge the gap between the hype and the reality of if, how, and where these emerging technologies might ‘add value’ in New Zealand schools. It specifically focused on exploring the relationship between curriculum and learning task design, student grouping and organisation, device use and teacher pedagogy, to explore how these might be combined to support key competency development, with a focus on collaboration, thinking, and problem-solving skills.

Background and context

The earlier studies

At the beginning of 2012, a small research partnership was formed between the staff and students of Leamington Primary School and the researcher. A small amount of initial funding was secured from the University of Waikato to purchase eight iPads and a range of apps that were used in a literacy study focusing on phonics skill development. From this early work, other studies were developed and undertaken from 2013–14.

These included analysing the influence of app content and design features of students’ learning pathways (Falloon, 2013a), evaluating student–student and student–device interactions with iPads (Falloon, 2013b), and exploring the nature of student verbal interaction and discourse when learning collaboratively using iPads (Falloon & Khoo, 2014). Analysis of display recorder data from these early studies provided unique insights into the problem-solving and thinking strategies used by the young students as they collaborated on their learning tasks using the iPads. It was reviewing these data that acted as a catalyst for this TLRI study. This project delved more deeply into the type of thinking skills and strategies applied by the students as they used their devices individually and collaboratively to solve learning problems integrated into curriculum tasks. It explored the nexus of learning task design, student organisation, device use, and teacher pedagogy. It set out to learn more about how each element individually and collectively contributed to the establishment of digitally-enhanced learning environments focused on building higher order thinking and learner independence and competency, aligned with the future-focused goals of the New Zealand curriculum. It also investigated how teachers planned for and managed use of the devices in their classrooms, with an emphasis on learning task design, student organisation, key competency integration, and teaching pedagogy.

Research questions

The overall research questions guiding the study were:

1. What thinking skills and collaborative strategies do junior and middle/senior primary students apply when using iPads and open/closed design apps for learning tasks?
2. What teaching, planning, and management strategies support the effective^[1] use of iPads for learning purposes in BYOD, inquiry, values, or thinking skill-based programmes at primary school level?

The research focuses

Over the course of this TLRI project, three main focuses were identified and planned into the classrooms’ learning programmes. Several rounds of data collection occurred within each of these focuses, as different aspects of the learning activities were introduced and progressed.

The focuses were

• investigating digitally-supported collaboration for solving learning problems in the senior Innovative Learning Environment (ILE)
• investigating the nature of thinking skills used by students when engaged in computational learning tasks (coding in the junior ILE)
• using apps as digital scaffolds for learning basic science concepts (‘Forms of Energy’—senior ILE; ‘Building Basic Circuits’—junior ILE).

The research context

The TLRI partnership was undertaken between the researcher from the University of Waikato and the staff and students of Leamington Primary School in Cambridge. It built on and extended an existing research relationship established in 2012, by introducing new focuses, teachers, and classes. Data were collected in two ILEs comprising between 34–44 Year 1 and 2 students and 50–66 Year 5 and 6 students. It total, five teachers were involved in the partnership, with their experience varying between 10 and 38 years. The senior ILE was an iPad-equipped BYOD environment, while the junior ILE had a class set of iPads, sufficient for a 1:2 ratio. The junior ILE set was supplied by the university for the purposes of this project, with each machine having installed on it a unique recording app that was used for data collection (see below). This set was moved to the senior ILE at times where data were to be collected in that environment, due to difficulties installing the recording app on personally owned devices.

The learning programmes in both classes were designed around a learner virtues model based on The New Zealand Curriculum’s Key Competency Framework. These virtues underpinned the programmes operating in all classes, and were known as ‘The Leamington Learner COGS’ (Figure 1). The COGs were Effective Communicators, Technologically-capable Learners, Active Learners, Effective Thinkers, and Making a Difference. Sitting alongside conventional knowledge and skill outcomes, the teachers purposefully integrated the virtues with all curriculum tasks. Students were also reminded of them regularly during the school day, enabling them to focus on the specific competency or competencies being targeted during particular learning activities. The portrayal of these competencies as a network of cogs is significant, as each relies on and in turn contributes to the others. The meshing of the cogs is meant to signify the interdependence of the competencies—as one turns, so do all the others.

The COGs also provided a suitable framework within which to plan and implement this TLRI project, as they reflected the skills and dispositions identified by the school as desirable across all curriculum areas, and in students’ interactions in and beyond the school. This was important, as the research design required a broad perspective to be taken of any role the mobile devices played, defined by a holistic view of student development and outcomes. These included not only cognitive aspects, but also others such as social, collaborative, and affective.

FIGURE 1: The Leamington Learner COGs

Methodology, data methods, and ethical considerations

While the study was located within an interpretive, qualitative methodological framework, it utilised mixed methods to collect and analyse data. Traditional qualitative data methods including interviews, focus groups, document analysis, and observer field notes were used to gather data relating to teachers’ and students’ views and understandings, while in-class field data of students using the devices and apps in their learning programmes were collected using a custom built recording app installed on a set of university-supplied iPads. The recording app captured the display activity and user audio on each device as a single video file that could be exported later to a laptop and taken back to the university for analysis. As data that captured the students’ natural interactions with and use of the devices and apps integral to their learning was needed, it was important that a mobile data collection method was developed that could move with them to different places within their classroom and beyond. This was also important as the earlier studies had identified a strong Hawthorne Effect when observing and externally videoing the very young students, and because all classrooms involved in the research were set up as large ILEs, where students were free to move with their iPads to work in different spaces (Figure 2). These conditions proved a challenging environment in which to collect data that accurately reflected the students’ work practices.

The display capture app operated ‘in the background’, behind other apps the students were using in their learning tasks. Student finger placement on the display was recorded by a white dot, while verbal interactions between the students and all system sounds were recorded via the built-in microphone (Figure 3). Formal ethics approval for using the recorder was granted by the Waikato University Education Research Ethics Committee in 2012, with updated applications being made each year subsequent to reflect new groups of students involved in the research (including in this TLRI project).

FIGURE 2: Using mobile devices in the ILEs posed data collection challenges

Before parents signed consent agreements for their child’s participation in each study, a number of information afternoons and evenings were held. During these, parents were invited to learn more about the studies, why they were being carried out, how data were being collected and used, and ethical considerations relating to their child’s participation (including assent processes). At these meetings the recording system was demonstrated, and parents were invited to ask questions about its use and the data it yields. Taking these extra steps ensured parents fully understood what their child was going to be involved in, and allayed any concerns they may have had about use of the recording system. Interestingly, since the studies commenced, parental consent has only been declined on four occasions, and on two of these it was related to cultural or religious considerations about technology use. In this TLRI project, all students returned consent forms indicating willingness to be involved in the study.

Gaining student assent to record data was an ongoing process. Initially, before each data collection point, all students were verbally reminded that they were going to be recorded, and their oral permission for this was secured. However, as the project progressed and the students became more familiar with the system, both groups were able to activate and deactivate the recorders by themselves, after being asked to do so. Doing this as a deliberate act meant they were aware they were being recorded, although the verbal responses in some data indicated they forgot that this was the case.

FIGURE 3: Sample screen shot of recorder data. Note display finger placement (white dots)

Analysing the display capture data

A major advantage of using the display recording app is that it enables the capture of a large volume of data with all students engaged in the same task, at the same time. This mitigates possible data reliability issues brought about by maturation effects etc. A major disadvantage is the sheer volume of data the recording method produces, and the time-consuming nature of analysing and coding video. Data coding utilised different analysis frameworks according to the context of the investigation. For example, for the junior ILE computational and coding study, an analysis framework was generated from a combination of Brennan and Resnick’s (2012) computational thinking skills matrix and Anderson and Krathwohl’s (2001) revision of Bloom’s taxonomy (Cognitive Domain); while the senior ILE study on digitally-supported learner collaboration adopted a grounded theory thematic analysis. Specific details of the analysis methods used in each of the studies can be found in project publications by the author (Falloon, 2015, 2016a, 2016b, 2016c; Falloon, in review).

To expedite analysis of video display data, Studiocode was used. Studiocode was originally developed to support analysis of video data of athletic performance, but more recently has been used in areas such as marketing, public relations, and executive training. Some education researchers have used Studiocode to analyse video of classroom interactions and teacher performance to help improve teaching practice and classroom management. Studiocode enables coded events in video data to be logged on to a timeline, and supports the export of a range of quantitative information relating to the frequency, total time, average time per event, and so on, that can then be imported into either a spreadsheet or other quantitative analysis application. Studiocode also ‘bundles’ all video excerpts aligned with a specified code, allowing them to be exported as a group and shared for multiple-coder analysis, or to help with transcription and/or content or textual analysis. Figure 4 displays a typical Studiocode screen—in this case, displaying the coding framework and timeline from the senior ILE study on computational and general thinking. In Studiocode, the top left section displays the recorded video, the top right contains the coding template with the coloured arrows indicating activation/ deactivation relationships between the codes, and along the bottom runs the event timeline. For those studies adopting grounded theory analysis, the coding template was generated from a double blind analysis of a representative sample of data from each dataset.

Typically, coding involved the researcher conducting an initial review of a representative sample of data from each dataset (usually at least 3–5 hours of video), then using this to generate a series of draft primary codes (code labels) aligned to the specific focus of the investigation (e.g. thinking skills, device-supported collaboration etc.). Selection of sample data was undertaken in conjunction with the classroom teachers, to ensure it represented a blend of student abilities, and included data from different pair combinations (e.g. boy/boy, girl/ boy, girl/girl; across year levels; from different data collection points). These data were then used to generate a coding template, like the one displayed in Figure 4. Both the draft template and the sample data were distributed to at least one other coder (usually a postgraduate student). He/she used the template to re-code the sample, after which both coders met and discussed both the template and the coding decisions that had been made using it. Frequently this resulted in changes being made to the template to improve code definitions and thereby decision-making consistency. Once a final template and clarity relating to data selection against it had been agreed to, the second coder applied it to a larger representative data sample (usually an additional 10–15 hours of video, depending on the overall volume of video). When this was completed, the researcher randomly selected between 2 and 3 hours of uncoded data from the same dataset coded by the assistant. This was then coded, with results being used to determine levels of rater agreement using Cohen’s Kappa. While this process was very time consuming, it was deemed necessary to add robustness and trustworthiness to coding decisions. Individual rater agreement results are included in the published articles listed previously, but generally they fell between .62 and .75—indicating ‘good’ levels of coding agreement (Landis & Koch, 1977).

FIGURE 4: A typical Studiocode window showing the video being coded, the code template, and the event timeline

Exporting and working with the quantitative data

For some studies (e.g. general thinking skills and computational learning; apps as digital scaffolds for learning science), quantitative data were exported for further analysis and charting. Doing this enabled patterns in data to be identified, such as the time (average/total) students spent applying particular skills or engaged in using the device/apps for addressing different task requirements, occurrence frequency for different coded events, and so on. Studiocode data exports directly in .csv format for spreadsheet analysis (Figure 5), which can then be charted for display and publication. One identified issue relating to calculating the average percentage of time the students spent within a defined period engaged in a specific coded event (i.e. x hours/minutes, for each pair or as a whole class) was that raw time exported data needed to be factored as a percentage of a whole day (i.e. 24 hours) and then ‘back calculated’ to determine the percentage of actual run time (i.e. total coded task time) students spent on that particular coded activity. Figure 5 shows a sample data export for a student pair from the junior ILE computational and general thinking skill study. The additional columns (D and G) show the converted raw data as fractions of a day. This was used to calculate the percentage of run time students spent on a coded activity, and supported comparisons across all pairs. Figure 6 shows charted data from the pair in Figure 5.

FIGURE 5: Sample data export for a student pair in the computational/general thinking study

FIGURE 6: Charted data from the pair in Figure 5, as a percentage of total task run time

Working with qualitative data

While the Studiocode timeline provided a means of organising coded data into bundles useful for analysis, this needed to be converted into a format suited to publication. As few academic journals are presently able to accommodate significant volumes of streamed video as data, examples had to be provided in printable format. To facilitate this, verbatim transcripts were made of the students’ recorded verbal interactions while they were working in pairs and groups on their learning tasks. The audio from selected video excerpts chosen to be representative of data aligned with each code was transcribed, and still images were captured from the video to provide more contextual detail. This was supported by a textual description of events displayed in the video and descriptive information and definitions of the coding classifications. This information was organised into table format suitable for publication (Figure 7).

FIGURE 7: Audio from display data were transcribed and supported by contextual information for publication

Other qualitative data from interviews and a student focus group discussion were transcribed and thematically analysed. During some interviews and the focus group discussion, students viewed selected clips from the display recordings and were invited to comment on their content and explain the problem-solving and decisionmaking processes illustrated in them. These data were integrated into publications and presentations from the studies generally to provide more detail about how certain decisions were arrived at and to gain a deeper understanding of strategies students used to solve problems.

Key findings from the studies

Study 1: Learner collaboration and mobile devices

The first study in this project investigated how students in the senior ILE used different apps and device features to collaborate when solving learning problems.

The research questions were:

1. What design and technical features of iPads and apps appear to support work collaboration in three primary classrooms?
2. How do students perceive these features as supporting, or not, work collaboration?

The motivation for this study came after examination of earlier display recorder data from the 2012–14 work that suggested the students benefited from device capabilities and app features that enabled them to capture and share information while they were working, and at other times. The earlier studies contained display capture examples of students collaborating via cloud services such as Google Docs and Edmodo, and exchanging and sharing images, textual information, and website data using Dropbox and between devices through Bluetooth.

Data were collected using a 20-item slider scale/Likert student survey and three focus groups (one at Year 3&4 level and two at Year 5&6 level). Items for the survey were generated from observations made in the earlier studies about how students used their devices and apps to collaborate, asking them to rate each of the observed features according to how important they considered each was for supporting their collaboration. Several short response questions were also included in the survey. These investigated the perspectives of the students about how they saw the various features assisting (or not) their work, and any effects they had on their work quality or efficiency. Responses to both question sets were used to generate starter questions for the focus groups. Focus group data supplemented survey data by providing more detailed information about how and why students used the features and the benefits they saw in doing so.

Key outcomes from Study 1

• Students considered cloud services such as Google Docs to be highly beneficial for facilitating timely feedback that enabled them to improve their work, without the need to wait for dedicated ‘teacher time’. Often this feedback came from their peers in class, or out of class from extended family or friends living at a distance, who would either work with the students synchronously or leave feedback asynchronously for them to consider. The timeliness of feedback was important, with several students commenting on the benefit of not having to wait until they were back at school next day to progress their work. Forty-eight percent of students rated use of cloud services at home for school work as sometimes or more, suggesting a level of ‘seamlessness’ between school and home-based learning. Cloud services expanded the audience for students’ work, enabling them to share outcomes and experiences with others important to them.
• Students viewed using cloud services in and out of class as beneficial to work quality. Synchronous feedback supported editing and mistake correction while work was being drafted, and there was a general sense that two heads are better than one when it came to ideas generation and solving problems. Students often combined other apps such as Skype or Facetime with Google Docs or Edmodo to provide video and audio they considered enhanced their experience.
• A few students valued Google Docs and Dropbox for organisational reasons such as helping catch up with homework or other unfinished tasks, and also for storing their work so they didn’t forget or lose anything.
• Specific technical and design features of the iPad were viewed as supporting collaboration. These include (av. rating/10) an ability to rotate the display (6.7); the device’s small, compact form factor enabling them to be shared or passed around a group (6.2); the wide viewing angle allowing good display visibility from the side (6.7); being able to move safely with the device so they can share their work with others or work in different spaces (8.6); and varying the angle of the device to either play and share media, or input data if using other apps (8.6). Other performance attributes were also identified, such as sound quality (7.9) and an ability for more than one person at a time to access the display to input data (5.7).
• A culture and curriculum encouraging and expecting collaboration had been built in each classroom. Teachers had invested much time developing the skill sets in their students to support collaborative learning. Deliberate teaching included face-to-face and technology-mediated tasks requiring debate and negotiation, team research and outcome presentation, framing up good questions, and strategies to identify and utilise the strengths of different team members.
• Student collaboration was deliberately planned into learning tasks across the curriculum. The design and implementation of tasks frequently required group work, with outcomes and assessment criteria being linked to the quality of student collaboration and teamwork. Most often students self-organised in their groups, allocating specific roles according to task requirements. Task planning and design for collaboration was critical to effective use of the devices and apps in these classes, and was consistent with the goals of the schools’ learning virtues framework. Students were also required to reflect on and self-assess against the class’s ‘6Cs’ framework (Figure 8), which was built on the Leamington Learner virtues model.

FIGURE 8: The senior ILE ‘6Cs’ Reflection chart (based on the ‘Leamington Learner’ virtues)

• Well-designed and robust school technical infrastructure was needed to support synchronous collaboration via wifi. This also applied to sharing progress and outcomes using Apple TV (Figure 9). This system was frequently used by students and teachers for providing formative and summative feedback in class.

FIGURE 9: Robust wifi infrastructure is needed to support synchronous collaboration and feedback

Study 2: Investigating students’ thinking during computational learning tasks

The second study investigated the nature of student thinking when they were engaged in basic computational learning tasks using Scratch Jnr (junior ILE) and Pyonkee (based on Scratch, senior ILE) on their iPads. At both class levels activities were integrated into the classroom mathematics programmes. For the older students, Pyonkee was used to build geometry knowledge about angles (degrees, acute, obtuse) and ordinality (arranging and sequencing), while the younger students used Scratch Jnr. to learn about the attributes and characteristics of basic shapes (squares, rectangles, triangles). In both classes introductory work was completed using other apps, including Daisy the Dinosaur in the junior ILE and Cargobot and Lightbot in the senior room. These apps were used to familiarise the students with the basic ‘drag and drop’ process of building code, and to teach core computational concepts such as repeats and working with variables, that were to be used in the later work. The same data methods were employed in this study as described previously, with analysis focusing on the type and sophistication of thinking and problem solving the students used when working collaboratively using the apps, to solve the computational and mathematical challenges set for them by their teachers.

The research questions were:

1. What types of general thinking skills were evident, and how were they applied, in these students’ computational work?
2. What relationship exists between the exercise of general thinking skills and these students’ computational work?

Interest in this study developed from the early work of pioneers such as Seymour Papert who theorised about the relationship between computational thinking processes and the development of general and transferable thinking skills, while young children were using the LOGO programming language. However, critique of Papert’s ideas at the time, particularly by Roy Pea (1983), cast doubts on the idea of any general skill transfer from computational work. He claimed that evaluating thinking skill transferability from children’s work with LOGO was problematic, and that research instead needed to focus on investigating the type and quality of thinking the students used while completing computationally-based tasks. Despite the substantial period of time since this early work, little appears to have been done to address Pea’s concerns.

Data from the junior classroom study were analysed using a framework developed from a combination of Brennan and Resnick’s (2012) computational thinking evaluation matrix and Anderson and Krathwohl’s (2001) revision of Bloom’s Taxonomy (Cognitive Domain). Over a period of several sessions the students’ work was recorded using the display capture tool, and the problem-solving and thinking strategies they applied while solving the computational problems were evaluated against the analysis framework. Full details of the framework and results from this study have been published elsewhere (see Falloon, 2016b; Falloon, Hale & Fenemor, 2016).

Key outcomes from Study 2 (senior ILE)

• Senior students were able to successfully transfer geometry concepts learnt using Pyonkee into more formal mathematics learning. Although not a planned part of this study, the senior teachers conducted a small quasi-experimental investigation where the performance of two matched ability groups (one group with prior exposure to the geometry concepts via work on Pyonkee, the other without) was compared, using a pre-/post-test methodology. Results suggested the experimental group benefited from prior exposure to the geometry concepts via Pyonkee, producing higher average results post-test than the control group, and a greater capacity to solve more complex geometry problems more quickly.
• There was substantial evidence that computational concepts learnt in one app transferred directly to others. Introducing apps so that basic computational concepts were learnt in a systematic and structured way early on proved useful when more complex problem-based tasks were set later that required less scaffolded, independent application of the concepts. Display data captured students at both class levels applying concepts such as loops, repeats and variables from one app to another, without the need for repeated learning.
• Setting open-ended, ill-structured problems for students to respond to by developing their own code proved an excellent means of building collaboration and creative problem solving. In the senior ILE, the teachers set a number of design problems that embedded the geometry concepts into tasks requiring students to code movement, distance, and direction. Typically, these involved using Pyonkee to code environments where sprites moved and interacted in different ways (i.e. coding sprite movements so they included different distances and angles set by the teachers in the form of problem-solving challenges). Several examples of students sharing, building on, or redeveloping code from other groups were recorded. In the senior room, some of this collaboration was facilitated electronically (e.g. via wifi or Bluetooth), while in other examples, this occurred physically. The iPad’s portability assisted students to collaborate, through ease of movement with the device between work spaces.

Key outcomes from Study 2 (junior ILE)

Data from the junior study were analysed to determine any relationship between the students’ computational work and the exercise of general and higher order thinking skills. The aim of this work was not to measure any progression or qualitative improvement in these skills, but rather to investigate whether or not computationallybased activities such as coding might provide environments for students to exercise and potentially develop a range of general thinking capabilities. Data analysis mapped the elements in Brennan and Resnick’s (2012) computational thinking framework against the different thinking types identified by Anderson and Krathwohl (2001) to identify the nature of thinking the 5- and 6-year-olds engaged in, while they were completing the basic shape-building coding tasks set by their teachers.

Key outcomes were:

• Students utilised a range of general and higher order thinking skills in various phases of their computational work. These were operationalised on a ‘fit for purpose’ basis and mapped loosely to the computational practices, perspectives, and concepts identified in Brennan and Resnick’s (2012) evaluation framework. For example, skills such as remembering, recalling, or understanding were generally associated with students’ conceptualisation of the learning task and building understanding of the various tools and code blocks they could use to develop their procedures. Likewise, ‘higher order’ skills such as analysis and evaluation were aligned with code development and modification, debugging, and assessment of procedures against success criteria.
• Embedding computational work in the classroom mathematics programme supported achievement of mathematics objectives (attributes of basic shapes), competency objectives (thinking, relating to others, symbols and texts), and outcomes aligned with the new digital technologies area of the Technology curriculum. Post-teaching assessment data indicated students held a solid understanding of the targeted mathematics concepts, including knowledge of shape side length, number of sides, right angles, and parallels. While no comparison was made with traditional approaches to teaching these concepts, these assessment results were encouraging.
• Some data illustrated the importance of classroom environments supportive of student risk taking, when undertaking problem-based coding work. Some recordings indicated some students were overly concerned with getting it right first time. This discouraged them from experimenting with different code combinations, and stifled engagement in the subsequent processes of debugging and modification—processes that were frequently aligned with the exercise of higher order evaluation and analysis skills.
• Evidence of predictive thinking was captured in some data. Predictive thinking was described as “thinking whereby they (students) predicted or hypothesized a result from running code, before actually testing it” (Falloon, 2016b, p. 14). While in some recordings this represented quite sophisticated behaviour, in others predictive thinking could stifle progress as students spent prolonged periods discussing and planning what they were going to do without actually doing it! When asked about this, many indicated their concern for making mistakes that may be difficult to rectify later on. Some students used small whiteboards to plan their code sequences before inputting them to Scratch Jnr. for testing. Others planned at length using their whiteboards, but spent little or no time testing or refining their code (Figure 10).

FIGURE 10: Students often spent considerable time planning their procedures on their whiteboards

• Problem and tool conceptualisation was a very important component of the students’ computational activity. Significant periods of time were recorded where students experimented with different code blocks to determine what each did, and/or held lengthy discussions about the tasks, and how they might approach them. A noticeable characteristic of some students’ conceptualisation was a process described as ‘chunking’, where problems or tasks were broken down into manageable-sized pieces to facilitate easier completion. During coding, these students selected, arranged, and tested sequences of code blocks aligned with completing a single ‘chunk’, eventually assembling all ‘chunks’ to complete their procedure. Doing this was a deliberate strategy employed to make the process of solving complex problems manageable.
• Interestingly, no evidence was found in the junior study that aligned with Brennan and Resnick’s (2012) Connecting perspective. They identified connecting as a process by which students reused and/or repurposed code, or shared and used each other’s code. Instead, these young students appeared to treat each of their shape-building coding tasks as separate entities. While opportunities existed for them to reuse or modify existing code for different purposes (e.g. changing variables in a square to create a rectangle, or adjusting code for a lower case ‘t’ to create an upper case ‘T’), no evidence of this was recorded, despite prompting from their teachers. This suggested limitations to the extent of conceptual transfer between tasks.
• While students experienced few difficulties using Scratch Jnr., its design held limitations for the use to which it was applied by these teachers. The main issue related to the lack of a pencil option, whereby students could set the sprite to leave a visible trail as it moved. This made it challenging for these young students to judge the accuracy of their shapes, and thereby evaluate their outcomes against the success criteria. In response, some students tracked the sprite’s movement on their whiteboards (see Figure 11), while others calibrated the sprite’s movement (with varying degrees of accuracy) using the app’s grid. Acknowledging that Scratch Jnr.’s primary purpose is not to help teachers teach their students geometry concepts, the addition of a pencil option would support teachers’ efforts to integrate computational work with other curriculum areas such as Mathematics. Such a scenario is likely as teachers grapple with managing the requirements of the digital technologies curriculum strand.

FIGURE 11: Some students mirrored sprite movements on their whiteboards

Study 3: Investigating apps as scaffolds for learning basic science concepts

The final study in this series commenced mid-2016 and is ongoing during 2017. There are two components to this study—one was completed in the senior ILE in 2016 (Study 3a) and the other is being undertaken in the junior classroom during 2017 (Study 3b). The focus of both investigations is to explore whether iPads and apps can independently scaffold students’ learning of basic science concepts. The specific emphasis is on device features and app content that students use—and how they are used—to learn science conceptual and procedural knowledge, and to understand strategic processes. Both studies drew on the earlier work of Sharma and Hannafin (2007) and Kim and Hannafin (2011) who speculated on how technology-based tools might act in a similar way to human input to scaffold science learning. It also built on Zydney and Warner’s (2016) synthesis of studies on the use of mobile devices in school science that identified limitations to how the performance of apps for supporting claimed knowledge and procedural outcomes was supported by data, highlighting a “lack of alignment between the underlying theoretical framework or learning issue and the studies’ measured outcomes” (p. 14).

The research questions were:

1. How did students integrate app scaffolds and device features with their practical science work?
2. How did teachers plan and facilitate use of the apps, during the students’ self-directed science activities?

Study 3a: The senior ILE

The senior study was based around a ‘Forms of Energy’ topic, integrating apps from the French-developed Okiwibook science series to help students understand basic energy concepts, mainly in physics and chemistry. It combined the apps with ‘hands-on’ experiments set out in a range of practical workshops undertaken over a period of about 3 weeks. The Okiwibook apps contained basic details of simple physics and chemistry experiments designed to demonstrate key concepts such as sound, chemical, magnetic, and electrical energy. The students were organised into groups generally of three or four, and were responsible for bringing to school the equipment needed to complete the experiments. This was shared across the groups and supported by equipment provided by the university.

The apps contained an array of information scaffolds of different forms including text-based details of equipment needed, a basic list of instructions, and a short explanation of the ‘science behind the experiment’. They also contained and a short video (no audio) showing the method and expected results (Figure 12). The apps could be described as ‘minimalist’, providing only basic information for the students. They were deliberately chosen for this reason so that the students needed to interpret and analyse the experiment’s requirements and procedures themselves before linking them to their practical work. Figure 13 shows some of the workshops set up in the ILE.

FIGURE 12: The typical layout of an Okiwibook science app (silent video on left)

It should be noted that the design of this phase of the unit was not inquiry-based, but rather set up and structured to build the students’ knowledge of key science concepts that were to be used later in the topic.

FIGURE 13: The practical workshops set up in the ILE

Key outcomes from Study 3a (senior ILE)
Data from the senior study were captured using the same methods detailed earlier. In this study in particular, the display capture proved invaluable for recording students as they moved between the workshops, and for logging their interactions with each other and the equipment as they completed their experiments. In addition to capturing information about how students used the apps, it also recorded how they used different tools integrated into the iPad’s operating system to log and share methods and results with others sometimes as they worked, using wifi and Bluetooth. Also of note was the value of the device’s camera that was used by several groups to record their progress and outcomes. These recordings were shared via Apple TV during whole-class plenaries held at the conclusion of each workshop session.

Data were coded under three main themes. These were students’ use of app scaffolds and device functions for planning and preparation (before experiment); understanding and accessing the experimental method (before, during, and after experiment); and sharing, recording, and communicating outcomes (during, for formative purposes, and after experiment). Quantitative data for each group relating to which scaffolds were accessed, how often, and the total and average times spent interacting with each scaffold (as a percentage of total task time) were entered into Excel and analysed across the whole class. These were then charted as shown in Figure 14.

FIGURE 14: Illustrative chart for all groups showing data coded as accessing scaffolds for understanding experimental method

Key outcomes were:

• Students seamlessly integrated the different scaffolds contained in the apps to support their practical work at various stages during their experiments. Most often scaffolds were used to build procedural understanding of methods either before they commenced or during the experiments (44% of runtime coded as ‘Understanding and Accessing Method’), or evaluating their methods post-experiment, usually in response to variations in their results compared with those shown in the app (27%). Of the app scaffolds, the video was used most often and in a variety of ways. Some students ‘mirrored’ the procedures shown in the video, adopting a step-by-step approach whereby they played and paused the recording as they copied each stage with their equipment in real time. Others visually compared variables existing between their procedures and outcomes and those shown on the video, usually in response to differences in results. Both of these actions provided formative feedback that enabled the students to progress, independent of teacher input.
• Device recording functions such as the iPad’s video recorder and camera were used frequently by many groups, mostly to record the methods and results as they completed each experiment (57% of runtime coded as ‘Sharing, Recording, or Communicating’). Other uses included sharing methods or results with others during the experiments (22%, formative) and during plenaries held at the end of each workshop series (21%, summative). Recordings proved to be a valuable formative resource across the groups, being shared either electronically or physically while the students worked. They supported knowledge sharing, enabling groups to benefit from the experiences of others through comparison of methods and outcomes. They were also essential for knowledge building when shared using Apple TV during the plenaries (Figure 15). Due to the apps’ limited capacity to support conceptual knowledge development (see later discussion) the plenary sessions were a critical component to ensure students generated accurate science understandings from the workshops.

FIGURE 15: Recordings were shared during plenaries to support accurate conceptual knowledge building

• Students accessed and used text-based scaffolds infrequently after they had commenced their experiments, and once they had an understanding of its methods and procedures. Very few students (only two groups) accessed the science explanatory scaffold at any stage, preferring to play the video and discuss amongst themselves what appeared to be happening, and comparing this with their own results. While these discussions were worthwhile in terms of promoting analysis and debate, frequently they did not result in scientifically accurate understandings. When asked why the explanatory scaffold was not used, students commented on the complexity of explanations and a lack of supporting visuals as barriers. Knowing this highlighted the importance of teachers possessing a solid conceptual knowledge base that could be used to guide construction of accurate science understandings during the plenary sessions.
• The video was the most frequently accessed scaffold. The absence of audio, whether a deliberate design decision or not, forced students to observe closely and discuss and debate what they observed— interpreting the information in relation to their own activities. Display recordings suggested this had a significant positive effect on the level of the students’ thinking as they engaged in analysis and evaluation of the visual data in deciding their own methods, and/or speculating on the effect of variables on results. However, as discussed earlier, the teachers played a vital role in ensuring these interpretations resulted in accurate science understandings.

Study 3b: The junior ILE

The new entrant ILE science study commenced in late March 2017. Its focus is on investigating if and how knowledge relating to different types of simple electrical circuits and the components of these circuits might be transferred between students building virtual circuits using apps, and actual circuit construction using physical equipment.

Phase 1

In phase 1 three apps were introduced to the students, one per week over the course of 3 weeks. Each app explored concepts relating to circuit design (simple parallel and series), what a battery does, how switches work, what resistance is, and so on. Over the course of three sessions the students were introduced to the apps, and progressively challenged to use the ‘drag and drop’ components to build an array of different circuits. The tasks were set up as a range of ‘Can You’ challenges, involving minimal teacher input or scaffolding. The challenges included (examples only):

• Can you build a circuit where one bulb is glowing?
• Can you build a circuit where two bulbs are glowing?
• Can you build a circuit where you can turn one bulb off and on using a switch?
• Can you build a circuit where you can turn two bulbs off and on using a switch?
• Can you build a circuit with two bulbs glowing, but where you can take one of them out, the other keeps glowing?

Phase 2

Approximately 2 weeks after the conclusion of phase 1, phase 2 involved the students using physical equipment to build the same circuits introduced using the apps. Sufficient bulbs, bulb holders, batteries, switches, and alligator-clipped connecting wires were made available to the students to choose from to complete the practical challenges. As before, this phase involved minimal teacher introduction and scaffolding, with the same ‘Can you …’ challenge format being used. The purpose of phase 2 is to investigate if, and the extent to which, knowledge generated in phase 1 can be transferred and applied to the practical work.

Data collection and analysis
Initial phase 1 data were gathered using the same procedure as in the previous studies. The display capture app recorded strategies the students applied when solving the challenges, and any knowledge transfer between the apps. Recordings will be used to gauge the extent of conceptual knowledge development relating to different circuit designs (serial, parallel) and the function of circuit components (e.g. switches, wires, bulbs, batteries, etc.). The practical work in phase 2 was recorded by a group member using the iPad’s video recorder. This data will be analysed to determine any retention of conceptual knowledge, and whether this can transfer from work with the apps to the practical tasks.

It is expected that results from this study will be available in late 2017.

Conclusions and major implications for practice

Learning design and pedagogy

This project was diverse in nature, investigating related but different aspects of mobile device use in a primary school. However, a common linking theme emerged across all studies relating to the interconnected nature of curriculum design, teacher pedagogy, class organisation, assessment, and technology use. Fundamental to this interconnectedness was the teachers’ deep understanding of socio-culturally-based learning and how this reflects in the design and implementation of learning tasks, the relationships developed with students, expected levels of student collaboration, responsibility, and autonomy; and what and how outcomes from a curriculum of this design would be assessed. The unifying structure for this was the school’s COGs or Learner Virtues framework (see discussion surrounding Figure 1) that identified the competencies expected of students at this school. This framework was loosely built around the curriculum key competencies, but was customised to suit local priorities after a period of community, staff, and student consultation. The Learner Virtues framework permeated everything that occurred at all levels of this school, and its goals, and how it was expected these manifest in the behaviours and work of both teachers and students, was very well understood. Put simply, it was just how we do things here.

At a practical level, this deep understanding reflected in curriculum and learning task designs of a constructivistreferred, thinking skill-oriented, collaborative problem-solving and project-based nature. In all research classes there was a strong expectation from teachers that students develop learning independence and collaborative skills at an early stage, and that they combine these with an attitude of thinking for themselves and giving it a go, to solve learning problems embedded in open-ended curriculum tasks, specifically designed to build such capabilities. However, there was a balance between this goal and a recognition that all students needed a solid foundation of base knowledge upon which to exercise and build these capabilities. This reflected in a pedagogical approach to curriculum that combined both conventional whole-class and group teaching often focused on building foundational knowledge and skills, with the more open-ended tasks that applied and extended these through problem- and project-based work using the technology. The importance teachers placed on the accuracy of student knowledge construction is illustrated in the senior science study through the extensive use of plenaries, where direct teaching was used to correct any student misconceptions that may have existed. All teachers recognised the importance of pedagogical content knowledge to the success of student learning. They saw it as their professional responsibility to ensure they had strong conceptual understandings that, where needed, could be directly communicated to their students. Critical to the successful use of iPads and apps in these classrooms was the teachers’ well-developed understanding of socio-cultural learning and how to best balance the capabilities of new technologies with more traditional approaches to achieve both knowledge and competency goals.

Device organisation and access

Notable across all studies was how teachers managed student access to and use of the devices. Organisational systems were flexible, and varied between class levels. As the senior environments were BYOD, students had ‘on tap’ access to their devices, although they were encouraged to select the best tool for the work they had to do— whether this involved their iPads or not. It was apparent that the senior students had used devices for some time (in various ways for over 4 years), as their decision making around device use (or not), was completely seamless. There was no novelty effect. The students were very capable of deciding for themselves whether or not their iPad was the best tool for the job, and if not, selecting resources they considered more appropriate. If they did choose to use their devices they were completely competent at selecting appropriate apps to support their work, independent of teacher input. At the senior level it was apparent that the iPads were simply one learning resource amongst others that the students could use, if it suited their purpose.

In the junior ILE, the teachers provided far more structure around student device access and use. The junior classrooms were not BYOD, but had access to a class set of around 20 iPads installed with relatively few apps. The early studies preceding this project identified major challenges with younger students having access to too many apps, and issues with the iPad’s design meaning access to apps could not be restricted (see Falloon, 2013a). For this reason, teachers in the junior classrooms installed only a small number of key apps on the devices—the majority of these being of a content-authoring nature (e.g. Pic Collage, Popplet). Apps that teachers wanted to use specific to curriculum areas such as mathematics or literacy were organised into folders according to the learning area, days of the week, or by groups. Students were closely monitored to ensure they accessed only those apps that were intended to be used. Tight structures were essential at the junior level to ensure the devices were used productively.

Outcomes from device use

Problems gathering data that illustrate or provide evidence of student learning linked to technology use is an ongoing issue. This particularly applies to so-called ‘soft skills’ such as collaboration, or thinking and problemsolving capabilities. It is very difficult for teachers to monitor and track students’ use of devices in a busy classroom, to determine the ‘learning process’ occurring while they are engaged in a task, or to discover how students might be sharing their knowledge or collaborating with others to achieve a goal. Most often teachers only have access to the outcome or results of a passage of learning, having little idea of what knowledge or strategies their students used in reaching that point. This particularly applies to using technology, as typically no ‘trail’ is left for teachers to review.

The display recorder used in these studies provided some unique insights into such processes. Analysis of data was shared with the teachers, who used this to make changes to how they managed student access to the devices, their teaching strategies, and the design of learning tasks. Some data were also shared at parent evenings to help explain the rationale behind BYOD, the nature of learning supported by device use, and how this links to future-focused competencies. Being able to play the recordings and discuss with parents what was happening and why proved invaluable for building understanding in the school’s community of changes to curriculum and student competencies, and why the school was moving down the BYOD path. Unfortunately, most teachers do not have the time to gather data of this nature and use it in this way. More researchers working in the field should consider using digital data tools of this type, both for their own purposes and for feeding back useful information to teachers to support improvement in their practice.

Limitations of the study

Like any single case study of this type there are limitations to the generalisability of its findings. These are identified as:

• All studies were undertaken in the same school, the staff and community of which had developed a unified vision and understanding of the contribution digital technologies were to make to curriculum and learner competency goals. This may not be the case in other contexts.
• The data methods used in these studies may not be available to all researchers. A bespoke data capture solution was developed, due to restrictions imposed by manufacturer protections to the iPad’s operating system.
• High levels of device access, robust technical infrastructure, and high speed Internet access are needed to support learning programmes of the design detailed in these studies. This may not be the case in other contexts.
• The researcher was in the privileged position of being well known to the staff and students of this school, having been researching there since 2012. The existence of this relationship provided many advantages, allowing the research to commence and progress rapidly across a range of areas. Other studies may take time to set up and for such relationships to be established.

 

Footnote

1. In this study, the term ‘effective’ refers to the use of the devices and apps in ways and for purposes that promote thinking and problem-solving skill development as defined in the Key Competencies of the New Zealand Curriculum (Ministry of Education, 2007). ↑

 

References

Anderson, L., & Krathwohl, D. A. (2001). Taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.

Bloom, B. S. (1981). All our children learning. New York: McGraw-Hill.

Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. Paper presented at AERA, Vancouver, BC. Retrieved from http://web.media.mit.edu/~kbrennan/files/Brennan_Resnick_AERA2012_CT.pdf

Cuban, L. (2001). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press.

Cuban, L. (2017). Larry Cuban on school reform and classroom practice: How and why I research exemplars of technology integration. Retrieved from https://larrycuban.wordpress.com/2017/02/07/how-and-why-i-research-exemplars-of-technology-integration/

Falloon, G., & Khoo, E. (2014). Exploring young students’ talk in iPad-supported collaborative learning environments. Computers & Education, 77, 13–28. doi:10.1016/j.compedu.2014.04.008

Falloon, G. W. (2013a). Young students using iPads: App design and content influences on their learning. Computers & Education, 68, 505–521.

Falloon, G. W. (2013b). What’s going on behind the screens? Researching young students’ learning pathways using iPads. Journal of Computer Assisted Learning, 1–19. doi:10.1111/jcal.12044

Falloon, G. W. (2015). What’s the difference? Learning collaboratively using iPads in conventional classrooms. Computers & Education, 84, 62–77.

Falloon, G. W. (2016a). Researching students across spaces and places: Capturing digital data ‘on the go’. International Journal of Research and Method in Education, Special issue: eResearch, 1–16. Retrieved from http://www.tandfonline.com/doi/full/10.1080/1743727X.2016.1219983

Falloon, G. W. (2016b). An analysis of young students’ thinking when completing basic coding tasks using Scratch Jnr. on the iPad.Journal of Computer Assisted Learning, 1–18. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/jcal.12155/epdf

Falloon, G. W. (2016c). iPads, apps and student thinking skill development. In N. Kurcikova & G.W. Falloon (Eds.), Apps, technology and younger learners: International evidence for teaching (pp. 224–239). Milton Park, England: Routledge.

Falloon, G. W. (in review). Using apps to scaffold science learning in primary classrooms: Design, pedagogical and curriculum considerations. Journal of Science Education and Technology.

Falloon, G. W., Hale, P. & Fenemor, T. (2016). Learning mathematical concepts and building thinking skills through coding: Experiences from the junior primary school. SET Research Information for Teachers, 1, 8–16 . Retrieved from http://www.nzcer.org.nz/system/files/2016_1_008_0.pdf

Geist, E. (2011). The game changer: Using iPads in college teacher education classes. College Student Journal, 45(4), 1–8.

Kim, M., & Hannafin, M. (2011). Scaffolding 6th graders’ problem solving in technology-enriched science classrooms: A qualitative case study. Instructional Science, 39(3), 255–282.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.

Maddux, C. (1986). The educational computing backlash: Can the swing of the pendulum be halted? Computers in the Schools, 3(2), 27–30.

Maddux, C., & Cumming, R. (2004). Fad, fashion and the weak role of theory and research in information technology in education. Journal of Technology and Teacher Education, 12(4), 511–533.

Ministry of Education. (2007). The New Zealand Curriculum. Wellington: Learning Media Ltd.

Pea, R. (1983). Logo programming and problem solving. ERIC Technical Report No. 12. Retrieved from http://eric.ed.gov/?id=ED319371

Sharma, P., & Hannafin, M. (2007). Scaffolding in technology-enhanced learning environments. Interactive Learning Environments, 15(1), 27–46.

Zydney, J., & Warner, Z. (2016). Mobile apps for science learning: Review of research. Computers & Education, 94, 1–17.

Author information

Professor Garry Falloon is presently involved in postgraduate teaching, supervision, and research in digital learning at Te Hononga School of Curriculum and Pedagogy, the University of Waikato.

Past research activities include exploring the effect of app design on student learning pathways, technology-facilitated school-scientist partnerships, the use of synchronous virtual classrooms in supporting distance students, online learning environment design, and the design and use of digital learning objects in schools.

From mid-2017 he will begin a new position as Fairfax Foundation Chair of Teacher Education at Macquarie University in Sydney, Australia.

Contact details: garry.falloon@mq.edu.au

Research team members

Mike Malcolm, Tonia Fenemor, Lissa Mangino and Sjann McDivitt
Leesa Mangino, Margaret Lelieveld, Paula Hale and Tonia Fenemor, Leamington Primary School, Cambridge