Promoting Science, Technology, and Engineering Self-Efficacy and Knowledge for All With an Autism Inclusion Maker Program

Introduction

Making, tinkering, and crafting have existed as long as people have needed to create tools and materials to address the challenges and opportunities of daily existence. However, the Maker Movement is a more recent phenomenon, particularly in the education community, with the growing recognition that engaging in iterative design and production, which can involve both physical and digital experiences and artifacts, can be valuable for developing a range of academic, social-emotional, and interpersonal abilities (Halverson and Peppler, 2018). Our understanding of the value of making is influenced by foundational learning theories such as constructionism (Papert, 1980) which argues that learners construct new knowledge by building on what they already know through participation in active, open-ended challenges, and hands-on projects rather than didactic lectures or structured, closed-ended lessons. In addition to being open-ended and hands-on, because of its focus on the production of artifacts through the use of tools, making takes participants through the engineering design process (EDP see Figure 1)—in which they learn to identify a problem, brainstorm ideas, plan, make, test, improve, and finalize—skills that are valued in formal and informal education as well as workforce settings (Martin and Dixon, 2016; Bevan et al., 2018) Another central characteristic of making is that it is interest-driven, allowing participants to connect their interests to deeper STEM learning by providing the time and space for them to do projects they initiate (Blikstein and Krannich, 2013; Honey and Kanter, 2013; Peppler et al., 2016). Such opportunities to pursue interests can benefit a wide range of young people (Quinn and Bell, 2013).

For neurotypical youth, programs that harness established interests centered around social connections and peer culture may inspire them to delve into more academically-oriented experiences, explore their identity and see how disciplines such as science technology, engineering, and math (STEM) may be relevant to their lives (Ito et al., 2013). However, autistic youth¹ often already have deep interests related to academic topics, including STEM fields (South et al., 2005). In fact, autistic people who enter higher education are drawn to majors in STEM fields at higher rates than the general population (Wei et al., 2013). To prepare autistic youth for careers, they can benefit from experiences that help them translate their interests into real-world applications through engaging in practices such as the EDP (Klin et al., 2007) and experiences that help them interact productively with peers (Wolfberg et al., 2015). Inclusive interest-driven maker programming, therefore, would seem a promising method for enabling a wide range of neurodiverse youth to pursue their personal interests while also gaining STEM skills that are valued in higher education and the workforce. With this premise in mind, a multidisciplinary team of maker educators, experts in autism inclusion, engineers, educators from autism-inclusion middle schools, and researchers worked to create an inclusive maker program for middle school students (ages 10–14) called IDEAS: Inventing, designing and engineering for All Students.

One core principle that shaped the IDEAS project, informed by Universal Design for Learning, was the recognition that well-designed inclusion programming can be beneficial for everyone involved (Rao and Meo, 2016). Not only is this because the strategies that support autistic youth can be effective for many students with other kinds of learning differences, but also because an inclusive environment allows for neurodiverse youth and adults to learn from the unique perspectives that everyone brings to the table (Gallagher and Gallagher, 2002; Happé and Frith, 2006; Kapp et al., 2012). Autistic people have historically made meaningful contributions to the communities in which they live and to society in general, well before we even had a name for the constellation of characteristics that is referred to as autism (Silberman, 2015). In fact, sometimes the very qualities that lead to a diagnosis—such as intense, focused interests, attention to detail, and hypersensitivity to sensory input—lead autistic people to perceive the world in a unique way, which can make them exceptionally inventive problem-solvers and creative thinkers (Baron-Cohen et al., 2009; Grandin and Panek, 2013). With the prevalence of autism at 1 in 54 people in the United States [Centers for Disease Control Prevention (CDC), 2020], there is a growing need for programs and opportunities that enable neurodiverse youth to work and learn together, so that everyone can benefit from what they all have to offer.

Self-determination theory (SDT) provides the conceptual framework for IDEAS. SDT provides a way for understanding the role that intrinsic and extrinsic motivation play in human behavior, learning, and development (Deci and Ryan, 1985; Ryan and Deci, 2000). Ryan and Deci (2000) argue that although pursuing areas of personal interest is intrinsically motivating, learning how to successfully accomplish extrinsically motivating tasks is essential for achieving long-term goals. Intrinsically motivating experiences are characterized by feelings of competence and autonomy. Extrinsically motivating activities fall along a continuum from externally enforced to self-endorsed to integrated with personal goals. Activities designed to promote feelings of competence and autonomy are more likely to be integrated and internalized, leading to better performance, while tasks accomplished only through external pressure or to achieve approval from others are not likely to be engaging or performed well (Miserandino, 1996; Ryan et al., 1997). Fostering a sense of connection and relatedness to teachers and peers is also an important component of programs that help students see the value of engaging in extrinsically motivating activities (Ryan et al., 1994). Some students may be intrinsically motivated to learn about specific STEM topics, while others may be motivated to make videogames. SDT provides a conceptual framework for designing activities that can build on those interests to support students' internalization of practices, such as the components of the EDP that are challenging for them. When provided with structured opportunities to relate with peers, and to see themselves as competent and autonomous actors, students will be more likely to develop abilities that they may not be intrinsically motivated to learn but which are essential to prepare them to pursue future coursework and careers. The open-ended, interest-driven nature of maker programming can provide these kinds of opportunities and potentially can support autistic and neurotypical youth to gain confidence in their abilities to do challenging work and to achieve their full potential.

While SDT is the conceptual framework for how interests can be leveraged to affect change, this project is also informed by the research on how to design productive inclusion environments with appropriate supports and opportunities to ensure that autistic students can engage fully in the academic and social life of schools or out-of-school-time programs (Baglieri and Knopf, 2004; Obiakor et al., 2012; U. S. Department of Education Institute of Education Sciences, 2012; de Bruin et al., 2013). For example, effective inclusion instruction includes hands-on, visual, and tactile experiences, in addition to auditory instruction (Lequia et al., 2012). Depth and detail of understanding needs to be valued as well as broad, standardized content knowledge (Happé and Frith, 2006; Dawson et al., 2007; Beardon, 2008; Mottron, 2011). Non-academic periods, such as lunchtime, recess, and after school, must offer supported interaction around interests, rather than unstructured social time that many students, but autistic students particularly, can find stressful (Barros et al., 2009; Koegel et al., 2012). Most important, it is beneficial to take a strengths-based approach to student interests by integrating them into instruction, rather than pathologizing those interests (Klin et al., 2007; Dunst et al., 2012; Gunn and Delafield-Butt, 2015; Kryzak and Jones, 2015). Connecting to student interests and strengths (what they already know and know well), is an effective way to build new knowledge and abilities (Papert, 1980).

Materials and Methods

The IDEAS Maker Program

The IDEAS project was conceived as a way to bring interest-driven maker programming into autism-inclusion public schools in New York City, providing opportunities for diverse, low-income adolescents to engage in the EDP. The project used a collaborative co-design model of research and development (Penuel et al., 2007; Roschelle et al., 2008), drawing on project team members with different forms of expertise to adapt a museum-based maker program so that it could be integrated into public autism-inclusion middle schools and facilitated by teachers in those schools. For each team member, we describe the positionality within the research and development process (Cresswell and Poth, 2018). The project team partners are described below.

Maker Programming Expertise

One key partner on this project was New York Hall of Science (NYSCI), one of the nation's leaders in maker programming. In addition to a Maker Faire that it held annually for 8 years, NYSCI offers a wide range of summer camps, afterschool programs, weekend workshops, and teacher professional development focused on creating experiences that allow young people to follow their interests through playful tinkering while also learning the EDP as they go through different stages of creation (ideation, planning, building, testing, improving, finalizing, sharing). The IDEAS program started with one of NYSCI existing maker programs, 3D Design and Fabrication. In the original 10-week program, which was run at NYSCI, participants explored the form and function of materials to design their own inventions. They engaged in the EDP multiple times, first doing mini projects on proper tool use and materials literacy—such as working with circuits, LEDs, and motors—leading to the creation of a 3D structure made with wooden blocks. Participants transferred the structures to Tinkercad, a free, online 3D design software. This tool provides a set of constraints different from the blocks, leading to iteration on the original design. Participants scaled down their 3D models and printed them out on a 3D printer. This process provides participants the opportunity to purposefully design, transfer, and iterate during the skill-building process. Participants create and keep written design journals to record their ideas, plans, and revisions.

These team members brought to the project the maker mindset, valuing youth creativity, agency and voice in the making process. They helped to revise the original curriculum for use by teachers in school-based programs. They also brought experience training educators to facilitate maker activities in formal and informal environments. All of the project team members as well as the teachers in the program attended this training. In the first year of the program one team member from NYSCI conducted the maker activities in the schools with the help of teachers and engineering Master's students. These team members did not conduct research nor are they authors of this manuscript.

Autism Inclusion Expertise

Another key partner was the ASD Nest Support Project at New York University (Nest). Since 2003, the ASD Nest Support Project has provided the professional development (PD) and the model for autism inclusion for primary through secondary autistic students who use speech to communicate and are academically on grade level in New York City (Koenig et al., 2009; Cohen and Hough, 2013). In its 54 schools, the ASD Nest Support Project supports a diverse population of approximately 1,500 autistic students learning alongside 5,000 typically developing peers. In the ASD Nest Support Project model, four to five autistic students learn the same educational curriculum as their peers in reduced-size classes with two teachers in each class, one of whom is general education and the other special education. There also are cluster special education teachers to provide support during special subjects and lunch, and speech and occupational therapists to provide those services. Many Nest schools offer afterschool and lunch clubs that are open to all students in the school, so autistic students have opportunities to interact with general education peers outside of class. These clubs are generally offered one or two times per week for 20 weeks of the school year, and are run by teachers. Experts from the ASD Nest Support Project and educators in the autism inclusion schools were part of the project team.

These team members brought experience in designing programs, materials and strategies to support autistic students, and providing PD to teachers in their use. All of the project team members attended this training, and all of the educators on this project had already received this PD because they teach in autism inclusion schools in New York City. These team members suggested instructional supports to include in the IDEAS curriculum. They also collected data about student social interactions and conducted that analysis and helped collect and analyze the qualitative data. They did not lead any of the activities in the schools. They are authors on this manuscript.

Engineering Education Expertise

A third key partner was Tandon School of Engineering at New York University, which specializes not only in training engineers but also supports educators in providing innovative STEM education experiences to primary and secondary school children. The project funded two graduate fellows pursuing Masters of Science degrees in Integrated Digital Media. This program specializes in computing, engineering, and creative practice around STEM. These team members worked with NYSCI team members to revise the original maker curriculum so it could be used in school-based informal programs by teachers. They participated in both the maker PD and the autism support PD. They helped to run the maker program in schools during Years 1 and 2. They did not conduct any of the research nor are they authors of this manuscript.

Research Expertise

The project partners also included qualitative researchers with expertise in collaborative co-design of educational technology-enhanced programs, and quantitative researchers with expertise in statistical modeling, experimental design, and psychometrics as they apply to large-scale, longitudinal datasets and complex data systems for measuring outcomes of programs and services for autistic children and adolescents. The qualitative research team members designed the co-design process for bringing the other team members together to redesign the maker program curriculum though they did not create the curriculum themselves. They participated in both the maker PD and the autism support PD. They also collected the qualitative and quantitative data about the program but did not run the program activities. The quantitative researchers designed the quantitative study and conducted the data analysis. They neither ran the program in schools nor designed the curriculum. Both groups of researchers are authors of this manuscript.

Program Development

At the beginning of the project, the partners participated in PD provided by both the autism inclusion experts and by the maker programming experts, to ensure that the team had a foundational understanding of the nature of the maker movement and techniques for supporting autistic youth. The researchers also conducted interviews with the principals of the three pilot schools to learn what kind of program design would work within the existing practices of their schools. For example, one school's normal practice was to have lunchtime clubs available for students, so the maker program could become one of those clubs. Other schools had afterschool programs, so the maker program could be offered then.

The team then spent 5 months adapting the 3D Design and Fabrication program so that it could be delivered in a school setting rather than a formal makerspace, modularizing long sessions into shorter segments to accommodate either 45-minute lunch clubs or 1.5- or 2-hour afterschool clubs, replacing activities that required specialized tools (like laser-cutters) with ones that used more common tools (like cardboard box cutters), and making the objectives of the activities more explicit so the non-maker professionals could understand why they were doing them. Since the autism experts on the team created the inclusion model for the city, they were able to develop materials and strategies that were consistent with the strength-based philosophy and instructional supports that the teachers were used to using in the autism inclusion schools (such as checklists to help with organization, visual templates for the products they would make, and prompts in curriculum when facilitators should check in for understanding). One of the most important aspects of the program was in the implementation requirements rather than the curriculum design, which was the stipulation that the teacher teams that ran maker programs would always include one special education teacher and one subject teacher (science, art, math). We understood that even with well-designed support materials there is no substitution for a skilled teacher who knows the needs of the different students and has experience using the support techniques that were provided. Another important aspect of the design of the program is that we had an external advisory board that reviewed and guided the development which was led by an autistic self-advocate and professor of special education.

Once the team had a draft program, a NYSCI educator piloted the revised program activities in the three partner schools, with teachers and Tandon engineering graduate students assisting. Researchers observed the sessions and interviewed teachers about their experience but did not lead the maker activities. Based on this trial run, the development team made revisions and created a set of 12 teacher-friendly, modular lesson plans that outlined the learning objectives, materials, complexity, and preparation time for each activity, along with more in-depth activity descriptions and sample products (https://www.edc.org/ideas-inventing-designing-and-engineering-all-students-maker-program).

The goal of this project was to have a program that teachers ran themselves. Therefore, in Year 2 of the project, the adapted IDEAS Maker Program was implemented in the three partner schools by teams of two to four science and special education teachers, with assistance from the engineering graduate students. In Year 2 there were 50 students across the three schools. Half of the participants were autistic. In that year researchers piloted a STEM self-efficacy and career interest survey (Chen and Usher, 2013; Bathgate et al., 2014; Kier et al., 2014) and an EDP assessment (Hsu et al., 2012) and collected qualitative data through observations and interviews.

In Year 3, teams of two to four teachers in each of three schools ran the program completely on their own, with no additional support from project team members. Researchers collected various forms of data during Year 3. One component of the research involved the collection of video data of neurodiverse students engaged in maker activities. These data were analyzed using a social engagement protocol based on existing autism research instruments (Bauminger, 2002; Usher et al., 2015). The findings we are reporting on here are from the STEM self-efficacy and career interest survey, EDP assessment, classroom observations, and participant interviews, described below, though we will refer back to the social interaction findings in the discussion.

Research Questions

The researchers conducted a mixed-methods study to evaluate the effectiveness of the IDEAS Maker Program on improving outcomes of neurodiverse middle-school students on the following: (1) STEM self-efficacy, (2) STEM career interest, and (3) understanding of/engagement in the EDP. In addition, this study investigated whether the impact of the maker program differed between autistic students and their general education peers, and whether and how students in general were able to create and complete projects that reflected their personal interests.

Specifically, the following research questions were addressed in this study:

1. Do students who participate in the IDEAS Maker Program have higher STEM self-efficacy, career interest, and an understanding of the engineering design process than do their peers who do not participate?

2. Did the impact of the IDEAS Maker Program differ for autistic students vs. their general education peers?

3. In what way, if at all, does the IDEAS Maker Program enable students to engage in the EDP as they create and complete projects that reflect their interests?

Participants

Study participants included 54 makers and 55 non-makers; 30 students who had an autism diagnosis that qualified them to be in the district autism inclusion program (18 makers and 12 non-makers) and 79 general education students (36 makers and 43 non-makers) from the same three middle schools. Students were enrolled in grades 6 through 8 (ages 10–14) in the fall of 2018. At the beginning of the year, all students in these schools were informed about the maker program through recruitment flyers and recruitment events advertising the various clubs that the schools run. There were approximately 15–20 students in each club. Parent/guardian consent forms describing the study were distributed to all maker program students. Any maker who was interested in participating and received parent consent/child assent was included in pre- and post- testing as the maker group. We created the comparison group from among students whose parents/guardians consented to have their children in the study. The sample matched the treatment group as closely as possible by grade and autism status. Ultimately, four students left the study because they moved away or were absent when post-surveys and assessments were administered. Table 1 shows the demographics of the sample.

Quantitative Data Collection Measures

In addition to the demographic variables we gathered for each participant, the following assessment scores were collected.

STEM Self-Efficacy

Prior to the study, we developed and piloted a STEM Self-Efficacy and Career Interest survey based on existing validated surveys (Chen and Usher, 2013; Bathgate et al., 2014; Kier et al., 2014). In our analysis of the pilot survey, we determined whether survey items that had previously been tested only on general education students were valid measures of self-efficacy and career interest among autistic students. Pilot findings of the survey's psychometric properties found reasonably strong reliability coefficients (α > 0.70) for constructs pertaining to science appreciation, vicarious experience of science, technology, and engineering self-efficacy, and technology and engineering career interest. In addition, there were no statistically significant differences in the way autistic students responded to these constructs compared to students in general education. These findings imply that the psychometric properties of the validated instruments from which these survey items are derived remain intact for autistic students.

EDP Assessment

Adapted from a study of the EDP by Hsu et al. (2012), we conducted brief, audio-recorded interviews with students who answered questions based on a picture of a student using a process to design a container for an egg drop contest. Students were asked what they thought about the process shown and whether they would do it differently. Responses were then coded according to a three-point scale (0 = no evidence noted, 1 = evidence noted, and 2 = evidence noted and elaborated). Researchers completed interrater reliability on 21 EDP assessments and achieved 93% reliability across all students and all codes. Once this level of reliability was achieved, they then coded the rest of the interviews (N = 87).

Qualitative Data Collection

In addition to collecting survey and EDP assessment data, the research team also conducted observations of all maker program sessions over the school year, and interviewed students and teachers at the end of the year about their maker program experiences. Data collection protocols were designed to focus the attention of the observers and interviewers on the participants' engagement in the EDP, pursuit of personal interests, and social interactions. The qualitatitve data we collected helped to inform the redesign of the IDEAS Maker Program curriculum and implementation strategies, and provided descriptive context for the quantitative data.

Statistical Analysis

We used two-level hierarchical linear modeling (HLM) (Raudenbush and Bryk, 2002) to test the difference in outcomes between maker and non-maker/comparison students, adjusting for the nesting structure of the data (students nested in schools) and controlling for student baseline scores, autism status, and other demographic characteristics. An interaction term between maker and autism indicator was added to the HLM to estimate whether the effect of the maker program differs between autism and general education groups. We conducted our analyses using SAS 9.0 PROC MIXED commands. The self-efficacy, career interest, outcome expectation, vicarious experience, science appreciation, or EDP scores are dependent variables.

Outcome Y for student i in School j is given as:

where Maker_ij = 1 for students in the maker program and 0 for students in comparison group.

A_ij = an indicator variable for students with autism.

$A_{ij}^{*}Make{r}_j=$ an interaction term between maker and autism.

COV_ij = student baseline survey or EDP assessment scores and demographic characteristics, including gender race/ethnicity, and grade level.

μ_0j = school random effect.

e_ij = student random effect.

The coefficient γ₀₁ associated with Maker_ij in the above HLM indicates the overall average treatment effect in promoting improved student outcomes. The coefficient γ₀₃ associated with the interaction term can be interpreted as the measure of differential maker impact for the autism group vs. general education group. If γ₀₃ is significant, it indicates that the relationship between the maker and student outcome was different for the autism group than for the general education group. All the covariates were grand-mean centered; therefore, the intercept estimates the mean posttest score for the non-maker general education group; intercept +γ₀₁ estimates the mean posttest score for the maker general education group; intercept +γ₀₂ estimates the mean posttest score for the non-maker autism group; intercept +γ₀₁ +γ₀₂ +γ₀₃ estimates the mean posttest score for the maker autism group. Figures 2–7 display these model-adjusted posttest means in gray bars as compared with pretest means in black bars for each group. To indicate the magnitude of the difference between maker and comparison groups, we report Hedges' g effect sizes for continuous outcomes. We calculated this by dividing the coefficient associated with the intervention effect from the HLM by the pooled within-group standard deviation of the outcome at the student level (What Works Clearinghouse, 2017).

Results

Survey and EDP Assessment

Table 2 provides descriptive statistics on the pre-test and post-test scores for the maker program group and non-maker comparison group. Table 3 presents HLM results comparing the maker and comparison groups on all outcomes after controlling for baseline score and student demographic characteristics. We found that the maker group had higher scores than the comparison group on four outcomes: understanding of the EDP, engineering and technology self-efficacy, vicarious experience of science, and science appreciation score, with Hedges'g¹ effect size 0.56, 0.82, 0.60, and 0.20, respectively. For example, an effect size of 0.56 means the maker program improved student performance on the EDP assessment by a 0.56 standard deviation. There were no statically significant impacts on other post-test outcomes.

The interaction term of maker by autism indicates whether the maker program has a differential impact for the autism and general education groups. Table 3 shows a differential impact of the maker program on technology and engineering interest (γ₀₃ = −0.78, p < 0.05) and science appreciation (γ₀₃= −0.70, p < 0.05). The maker program improved career interest and science appreciation score for neurotypical students more than it did for students with autism, indicating that the program is likely to have stronger impact among neurotypical students on these two outcomes. Figures 2–7 provide the estimated mean score for the posttest score across the four categories of students included in the interaction term. Both autistic students and neurotypical students show an increase in their understanding of the EDP.

Makers in Action

Qualitative observations of the program and interviews with teachers who facilitated the program provide descriptive information that demonstrates what student engagement in the EDP looks like in practice as well as how the program provides all students the opportunity to create projects that represent their personal interests.

EDP

Students in the maker program engaged repeatedly in various steps of the EDP as they followed the sequence of maker program activities, from creating their own design journals to learning how to create circuits to make LED greeting cards to creating robots made from craft materials powered by small motors, to brainstorming final project ideas (Figure 8), designing on Tinkercad software (Figure 9) and prototyping and finalizing their final 3D printed objects (Figure 10).

The Paper Circuit activity, e.g., has students light an LED by connecting each leg of the LED to copper tape that they attach to an index card and then connect to a lithium coin cell battery. Because the materials can be finicky and the circuit design has very specific requirements, students' initial prototypes often fail the first time. This provides the facilitators and the makers with the opportunity to discuss how prototype failures are a normal and expected part of the EDP—as it says on the EDP model that the students paste into their journals and that the teachers are given as a poster to display in the classroom (Figure 1). As one teacher noted, “For us, we go over the engineering design process, we go over the steps and we name them … It's more comforting for them to say, ‘Oh, OK, so we go through this when a prototype fails. We go through iteration. We change it and then we improve it.”’ The makers need to iterate on their designs to see what has to change for the LED to light. Many students find this activity frustrating at first, but overcoming the initial obstacle and showing that obstacles are an expected part of the process helps make the EDP real for them in a way that always being successful in their designs would not. “Some kids struggle through getting their circuit to work,” said one teacher. “It's fine, because the kid who is struggling to make it work is going through a valuable learning process by figuring it out on their own and using their peers as resources. That's something that, as teachers, we want to see in our classes every day.”

One particular instance provided an interesting window into how the whole group of makers could apply the EDP process to solve a real problem the club experienced. On this occasion, teachers were running the Paper Circuits activity. They discovered that they did not have enough lithium coin cell batteries for everyone in the group, though they did have plenty of AA batteries. They handed these out, but none of the students could get the LED to light with a AA battery. Neither the students nor the teachers knew why. They tested the paper circuits that did not work with the AA battery with the lithium coin cell batteries and those worked, so they could rule out problems with the way the copper tape was connected to the LED on the paper. Then they connected copper tape directly to the AA battery and LED light and still it would not work, but the lithium coin cell battery did, so they could rule out problems with the copper tape and LEDs. Then one general education student noticed the voltage information on the two kinds of batteries. The lithium coin cell was 3 volts while the AA battery was 1.5 volts, which was counterintuitive, given that the AA battery was much bigger. The students then made numerous attempts to connect two AA batteries in a way that would get enough power to the LED. An autistic student finally came up with a design that placed the batteries end to end, held together with copper tape (Figure 11).

Throughout this session, all of the students interacted with each other and learned from what others were doing. This incident illustrates how the maker program participants were able to engage together in the EDP to solve a real problem and construct a solution that allowed them to complete the activity. In the maker program where this incident occurred, the majority of autistic students tended to socialize with each other, sitting apart from the general education students, although two autistic students regularly sat with general education students. The socialization among the students increased when there were problems, such as the situation described above, or when someone needed help with Tinkercad, in which two of the autistic students were highly skilled, or when students wanted to show others what they had made.

Some of the teachers who ran the club also teach science, and they reported that they could see how the experiences the students had in the club engaging in the EDP translated into their formal academic classes.

“I had H, who is a maker, and we were doing an experiment where we tried not giving the students steps and letting them design their own process to test a question, and she kind of acted as the leader for her group. She was like, ‘Oh my gosh, this is just like what we do in maker program. I'm going to first brainstorm ideas and if this doesn't work, we're going to go through iteration and we're going to improve our design.’ So I feel like the makers, they'll be an asset in science classes because they're able to make the connection between this design process and what usually happens in scientific design processes and experiments.” This teacher felt that her own deepening knowledge of how to support students' understanding of the EDP would be essential in the future, “because that's the way science instruction is going, giving more research-based questions and then having kids experiment and investigate and develop their own research.” This teacher appreciated “seeing [the EDP] play out here in a more flexible environment” because it was good preparation for integrating that process into instruction. “I was able to go through the engineering design process, which is in the standards. I would read articles about kids grappling with the learning process and the design process and seeing the way kids will work through something like this was really helpful. Next year, we're getting [new science curriculum] and there's going to be tons of experiments, it's very hands-on. So I'm excited about that and I feel like I can use what I learned in maker program about the design process in the classroom.”

Pursuing Interests

The maker program afforded participating students the opportunity to find a way to design and build or 3D print something that reflected an existing interest. Whether it be 3D-printed Lego pieces glued onto a Rubik's cube, a figurine of a favorite animal or video game character, or a miniature golf course made out of cardboard and straws, this open-ended and non-prescriptive aspect of the program appealed to neurodivergent and neurotypical students.

In one school, two autistic boys found participation in the maker program allowed them to express their love for politics, particularly United States presidents, and, for one of these boys, related topics like foreign espionage and colonialism. They came to the program without knowing one another, but sharing this strong interest, and the club gave them the opportunity to socialize with one another around this topic. One of these boys went on to create 3D busts of presidents while the other boy created game pieces for a board game on the colonization of Africa (Figure 12).

When speaking to the teachers in this school about these particular students, a teacher appreciated that in maker program, students “found other people that have the same targeted interests. For example, Ronald and Jasper², they both like presidents. That's something very targeted, you know?” She added, “When he's in the maker program, it's super cool to talk about these presidents. It's super cool for him to go on and on and on, because they're all going on and on. So it's an identity thing. And he works very well with that. … I just think, for Jasper, it's become almost like a badge of honor, like ‘I'm a maker’ and it just fills his bucket, in terms of himself and who he is and his identity, and to find that identity in this place where he's very good at it and he can socialize.”

Similarly, in one of the other schools, a group of neurotypical girls joined together in their love for Korean Pop, or K-Pop. This resulted in a collaborative, intricate K-Pop party room, featuring a dance floor, a refreshments table, a DJ booth, and ornate decorations, including a rose made from glue (see Figure 13). When these students were asked how and why they were able to do something like this in maker program, one girl said “Opportunity. You can create whatever you want.” Another neurotypical student in the same school, who is fascinated by skyscrapers, created the city skylines of Chicago, New York, and Toronto in Tinkercad and printed them (see Figure 14).

In a third school, Daniel harnessed his interest in both creating and understanding how electronics work over the course of the project's 3 years, creating a speaker out of an Altoid canister in the second year of the project and, over Years 2 and 3, creating and revising a mini-laptop using a Raspberry Pi single board computer. His teachers reported that Daniel struggled in his academic classes because they did not allow him to pursue what he cares about, which he found frustrating. “I think the fact that the maker program allows him to really focus on things that he enjoys has really helped him balance the day and the week,” said one teacher. Knowing he had this outlet in school helped him get through the rest of the day. In addition, the club helped his confidence and his socialization skills. The club facilitator described the relationships that Daniel had formed with neurotypical students in the club over 3 years: They're friendships, and they go two ways. He really struggled forming relationships with kids last year. And I think just finding other people that share common interests, it's really hard for kids to do that. … When we're adults, we sort of find ways to get to things that we're interested in and we know we're going to meet other adults that are interested in those things. When you're a student, that doesn't happen naturally. But I think the maker program is a place that offers them that opportunity to really connect with other people that are really interested.

Discussion

The original reason for creating the IDEAS Maker Program was to see whether this kind interest-driven, open-ended experience would provide all participants with a way to pursue their interests and engage with peers in a way that they generally cannot do during typical classroom instruction. We hypothesized that participation in the maker program would increase all of the students' interest in engineering and technology careers, and STEM self-efficacy, and understanding of the EDP. Given the opportunity to pursue their interests in a deep way by engaging over time in the EDP to produce final projects, we hypothesized that all students would demonstrate creativity, solve problems, and persist through challenges to achieve their design goals, and that this might be particularly notable among autistic participants who generally might struggle to complete projects or assignments. We also predicted that autistic students would socialize more with peers if they could have interest-driven maker projects as a source of conversation.

Results from our survey and EDP assessment demonstrated that the maker program was effective for students overall, with increases in science and engineering self-efficacy, career interest, vicarious experience, science appreciation, and knowledge of the EDP. However, on some measures, such as career interest and science appreciation, the program's impact was different for neurotypical students than autistic students. Specifically, a statistically positive effect was found only on typically-developing students but not for autistic students. A possible explanation for this finding is that students' participation in the maker program was voluntary. While neurotypical students may have elected to participate in the maker program for a variety of reasons (for instance, interest in trying a new subject matter, a chance to be involved in any kind of recreational activity, or peer influence), autistic students are more likely to have self-selected into the maker program because they already had a strong career interest and appreciation for science, making it more difficult to improve their scores on these measures when compared to the neurotypical participants. This would align with findings from other studies, which show that autistic students' participation in social activities are more likely to occur if it is incorporated into activities that are already strongly aligned with the specific interests of the individual (Baker et al., 1998; Koegel et al., 2013).

Findings from the social engagement analysis, which we have reported on elsewhere, showed that autistic students interacted 50–100% of the time during their maker program, a higher rate than the neurotypical students. This interaction was characterized by high frequency of spontaneous social interaction, attentive responses to others, and reciprocal conversations with peers (Koenig et al., 2018; Martin et al., 2019; Chen et al., 2020). All students shared a similar ratio among the types of initiations. Sharing and seeking initiation types were the highest among all students, followed by attending, joking, and offering. Both groups made more initiations based on social/relational purposes than functional ones. Topic-relevant responses were most common, followed by topic-extending. However, while autistic and neurotypical students had comparable rates of social initiation and response, they presented a clear preference for same-group social partners that related to their social initiation type and outcome. Participants of each group tended to choose social partners within the same group, consistent with current research on autistic socialization characteristics (Heasman and Gillespie, 2018, 2019).

Qualitative observations and interviews largely supported the idea that both autistic and neurotypical students were able to engage in the EDP and pursue their interests through making, though some of the most striking findings from the qualitative research were related to the autistic participants. Nearly all of the autistic students who participated in the maker programs that took place in three schools over 3 years (N = 18 per year) successfully engaged in program activities, completed final projects, and presented their projects at maker showcases in their schools; some presented their projects at NYSCI as well, not only to the makers and their parents from other schools, but also to the general public. In our first year we developed a number of autism support materials for the activities that were similar to the kinds of materials used to assist students during normal instruction, such as checklists and visual supports, but the teachers who facilitated the program observed that the students needed far fewer instructional supports in maker program than they needed during the school day. The maker program gave the facilitators, who were all either science or special education teachers, a chance to see what their autistic students were capable of, which they found gratifying. They observed that some students who normally would complete the bare minimum to get through their classes would create careful and detailed projects when they were allowed to pursue what they cared about (for example, memes, food, video game characters, anime).

Teachers also reported that students who spoke very little in class would engage in in-depth conversations or joke with peers in the maker program, showing a different side of their personalities. This is not to say that there were not challenges, and certainly some students at times got frustrated to the point of tears when projects did not turn out as expected, or found it difficult to come up with a realistic final project idea, or could not resist playing video games instead of designing in Tinkercad when they had access to laptops. In those cases, the most effective instructional support was having facilitators who knew the students well and knew what usually works for them when they face difficulties, since there was no one particular strategy that worked for everyone. For the most part, however, the students persisted through difficulties to complete their work. The maker program was a place for the autistic students who were already predisposed toward making and STEM to have the time and space to be themselves, socialize in their own way, and show what they could do when given the opportunity to do what they wanted.

Autistic students are, of course, not alone in having deep interests, or in engaging more enthusiastically in activities that enable them to pursue those interests (Ito et al., 2013). These findings are consistent with the principles that underpin much of the work of the universal design for learning (UDL) community (Meyer et al., 2014). By recognizing that there is no “average” or “typical” learner, and instead developing interventions that focus on those on the “margins,” we intentionally create an environment with embedded supports and options that reduce barriers and ultimately ensure that learning is accessible and engaging for a diversity of learners (Rose, 2016). In doing so, educational materials and experiences that may originally have been created to support people with disabilities frequently also benefit those without disabilities (Meyer et al., 2014). The maker movement appeals to a wide range of people, young and old, neurotypical and neurodivergent, but it is particularly compelling for those who have particular interests and passions that they want to delve into deeply. Those intrinsically motivating interests carry makers through the challenges that are a necessary component of the EDP, allowing them to experience creation, failure, iteration, and revision in a way that helps them understand it on a visceral level (Petrich et al., 2013) and internalize those practices so they can achieve long-term goals (Ryan and Deci, 2000).

These findings have implications for the design of other inclusion programs for autistic and neurotypical youth in both formal and informal settings (Obiakor et al., 2012; Wolfberg et al., 2015). Making as a pedagogical strategy may be beneficial for inclusion because its openness and flexibility allows for a diversity of entry points into learning and doing, and in particular can support in-depth pursuit of areas of interest (Halverson and Peppler, 2018). Drawing on the interests of autistic children has already proven effective in many interventions (Klin et al., 2007; Dunst et al., 2012; Gunn and Delafield-Butt, 2015; Kryzak and Jones, 2015), but making presents a way to expand upon interests in a productive way. Using student interests as topics of conversation for social groups or as examples in content learning (i.e., having a videogame club or using characters from a videogame in math word problems) can increase engagement. However, creating products based on interests (i.e., learning how to program a videogame of their own) can help students to see how their interests connect to the larger world, and help them gain skills that may lead them into higher education and career pathways based on those interests. Finally, these findings can also help inform the design of maker programs. Supports such as checklists of steps to follow, visual guides of the EDP and activity templates, and prompts for facilitators to check in for understanding can help to provide structure for an experience that can be overwhelming to any students, but particularly those who may have sensory processing challenges (Grandin and Panek, 2013). In addition, having special education experts and teachers on the development team and as program facilitators will bring those perspectives to bear on how the activities are designed and implemented. In our own current iteration of the IDEAS project, we are involving autistic youth in the design and facilitation of a new high school version of the program (Fletcher-Watson et al., 2019; Nicolaidis et al., 2019).

Limitations

The interpretation of the findings from these studies must be tempered by several limitations. First, the autistic students who participated in this study use speech to communicate and are academically on grade level, so we cannot generalize study findings to other populations of autistic students with differing support needs. In addition, this was not a randomized controlled trial, so we cannot make causal claims about the findings. All students who were in the treatment group self-selected into the club. Because it was a voluntary club, they could not be randomly assigned into it. Therefore, students who already had some interest in making and engineering would be those who chose to be involved. Having pre-surveys and assessments should control for this to some extent, but it is necessary to view the groups as potentially different due to this self-selection. Lastly, the current design of the study provides limited information about whether the maker program worked to improve student outcomes by simply exposing students to more science learning. Future studies should compare the maker program to a comparison condition that uses a different type of science and engineering program.

Data Availability Statement

The datasets generated for this study will not be made publicly available due to data protection regulation.

Ethics Statement

The Education Development Center Institutional Review Board approved this study. Researchers obtained signed consent forms from all adult study participants and signed parent/guardian consent forms and student assent forms for all children in the study. No teacher or student names were recorded in the interview, observation, survey, or assessment data to protect participant data and privacy.

Author Contributions

WM led the research and, with RV and AR, engaged in qualitative data collection and analysis and wrote about those as well as the Introduction, Methods, and Discussion. XW and JY contributed to the design, analysis, and interpretation of the quantitative measures used in this study. They also contributed to drafts of the Methods and Results sections of this manuscript. KP contributed to the theoretical framework. All authors approved the submitted version.

Funding

This project was funded by the National Science Foundation award #1614436, a federally funded agency in the United States of America.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

References