DRK-12 Collaborative Projects

This Spotlight features DRK-12 collaborative projects, provides insight into the affordances and challenges of partnering with multiple organizations, and offers advice for those considering a collaborative proposal.

In this Spotlight...

Featured Projects

logo

A Partnership to Adapt, Implement and Study a Professional Learning Model and Build District Capacity to Improve Science Instruction and Student Understanding

PIs: Hilda Borko and Emily Weiss; Co-PIs: Jonathan Osborne and Craig Strang
Grade Level(s): Elementary School
STEM Discipline(s): Science, STEM Practices
Focus Audience(s): Teachers, District and School Administrators, and Teachers on Special Assignment (TOSAs)
Proposal Strand: Teaching
Project Type: Implementation & Improvement

About the Collaborative Project: Improving Practice Together (DRL # 1720894, #1720930) is a six-year collaborative project between the Lawrence Hall of Science, Santa Clara Unified School District, and Stanford University. The professional learning goals of this project are to develop teachers’ capacity to teach argumentation from evidence in elementary school science, and to develop a cadre of teacher leaders who can support related district professional learning goals after the project has finished. The research goals are to study the professional learning process, development of teachers’ leadership capacity, resulting changes in classroom practice, and the evolution of the partnership over time. The impact of the pandemic posed a unique set of challenges, which led to shifted leadership opportunities for teacher leaders associated with teacher and student needs in the virtual learning environment.

The project is creating a website with materials to support professional learning experiences and teacher leadership capacity development, research and evaluation tools, and presentations and publications to share the findings emerging from the project.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: This work built on a previous NSF-funded collaborative project—PRACTISE (DRL #1223021, #1220666)—between Lawrence Hall of Science and Stanford University, which demonstrated the effectiveness of a professional learning model to develop teachers’ capabilities to teach argument from evidence in elementary schools. Santa Clara Unified School District had undertaken previous work with the Lawrence Hall of Science through a project focused on K-12 district-wide science implementation and was enthusiastic to develop teacher leadership capacity in elementary science education that would build on this effort.

Affordances of Pursuing and Implementing Your Collaborative Project: The partnership enabled us to draw upon the expertise of professional learning leaders, researchers, evaluators, district administrators, and classroom teachers to create a robust set of research-based learning experiences tailored to the district’s priorities. The collaboration afforded a sharing of perspectives, priorities, and constraints, which enriched both the work with teachers and leaders and the research. The project has afforded the opportunity to see the challenges of working with a school district to build its capacity to provide ongoing, sustainable professional learning experiences and to develop the flexibility to address those challenges.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: The major challenge has been working with a school district which had a change in leadership one year after the project began and an associated change in district priorities. The pandemic also forced a re-evaluation of teacher and leader needs, what could be done in terms of data collection and professional learning and which aspects of the work might be most valued. These challenges resulted in redefining overall project goals and refocusing activities.

Advice for Those Considering a Collaborative Grant Proposal: The context in which the project is established will change, and projects will need to be ready to learn how to adapt in a flexible manner which sustains the value of the work in a changed or changing context. This requires good communication between all partners, an infrastructure for ongoing partner meetings, a liaison(s) whose role is to communicate within and across all teams (boundary crosser), and a commitment to the value and importance of the work.

Project Product(s):

  • Website (Coming soon!)
    • Publications and conference presentations
    • Research and evaluation instruments
    • Professional learning resources to support teaching science argumentation in upper elementary school
    • Teacher leadership capacity building resources

Building a Teacher Knowledge Base for the Implementation of High-Quality Instructional Resources through the Collaborative Investigation of Video Cases

logoPIs: Joseph DiNapoli, David Jabon, and David Wilson
Grade Level(s): Middle School, High School
STEM Discipline(s): Mathematics
Focus Audience(s): Teachers
Proposal Strand: Teaching
Project Type: Early Stage Design & Development

About the Collaborative Project: This project is focused on developing, testing, and refining a professional development (PD) model. The Analyzing Instruction in Mathematics using the Teaching for Robust Understanding framework (AIM-TRU) project team has developed a PD model in which secondary teachers collaboratively investigate high-quality instructional materials to deepen instructional knowledge and support shifts in practice aligned to the Teaching for Robust Understanding (TRU) framework (Schoenfeld, 2015). In order to leverage mathematically rich student conversations for teacher learning, our PD model focuses on a lesson’s mathematical content, studies of video clips featuring students engaged in rich mathematical activity, and sets of reflective discussion questions based on the TRU framework. We call this the AIM-TRU learning cycle. This project allows us to test its use more broadly across three different geographical contexts. 

We take a communities of practice perspective in our research design, and are guided by the following research questions: 1) What is the impact of a community of practice centered on the collective investigation of video cases grounded in high-quality instructional materials? and 2) How does participation in communities of practice centered on the collective investigation of video cases grounded in high-quality instructional materials impact teachers’ use of these materials and practice? Our findings influence in-service teacher PD, especially where administrators seek to create coherent mathematical systems that organize PD around standards-aligned instructional resources. Our findings also influence preservice teacher education, where high-quality video cases are coveted for use in methods courses. Moreover, our findings suggest a need for further support for PD facilitators, which we intend to pursue in a future project.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: The project had the benefit of building on existing relationships among three of the partners that had developed in the years prior to the proposed collaboration. At its core, the project brings teachers together through PLCs that focus on Formative Assessment Lessons and the Teaching for Robust Understanding framework (map.mathshell.org), so the identification of additional partner candidates included familiarity with those materials as a basis for initial consideration. In addition, as our research is grounded in communities of practice with secondary teachers comprising the community, we sought to identify a partner that could bring their experience in working with secondary teachers to the research problem around teacher learning.

Affordances of Pursuing and Implementing Your Collaborative Project: The collaboration has provided for ongoing refinement of the AIM-TRU professional learning cycle (https://tle.soe.umich.edu/AIM-TRU) through feedback from teachers and teacher leaders in two major metropolitan districts and an assortment of rural and suburban school districts across three distinct regions of the country. Senior project personnel work closely with the PLCs and facilitators and attend PLC meetings, then bring their observations to project-wide meetings to share their region’s experiences and insights on project goals. The strength of the research is in part due to this geographic diversity and the subsequent diversity within classrooms and among teachers, all of which inform the AIM-TRU learning cycle and offer a broad cross-section of sites from which to analyze data.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: As with many educational, school-based projects, the past four years have provided an abundance of challenges. Those challenges both strained and enhanced the research opportunities. The study of learning communities when people are not able to meet face-to-face certainly was a hurdle to overcome, yet the opportunities presented with Zoom and breakout rooms, the creative tools used to capture thinking that became additional sources of data, and the need for communities that allowed for social interaction in a time of separation and distancing, proved to be a valuable opportunity for our project. The adaptation of the AIM-TRU learning cycle for remote PLC meetings was highly effective and is still being used at present by teachers that are separated by long distances. The enthusiasm that mathematics teachers had for the rare opportunity to come together once a month with their colleagues allowed for them to engage in congenial and collegial conversations around big mathematical ideas and reflect on practice in productive ways. 

Perhaps the biggest challenge was for project personnel to come together as a team given that the pandemic began in the first year of the project. Again, the need to address the many challenges provided an opportunity and need for greater frequency of meetings among senior personnel. Monthly site leader meetings were scheduled, project-wide team meetings began to happen at regular intervals, and these set the stage for greater collaboration when the third year allowed for a return to in-person gatherings.

Advice for Those Considering a Collaborative Grant Proposal: As indicated above, the collaboration brought together partners that had direct experience working with the core components of the project. That, in turn, led to the development of project goals and research methodology that provided opportunities for each partner to build on those early experiences. This is one reason for the success of the implementation of the AIM-TRU model, the significant contribution to the study of PLCs and teacher learning, and the growth of teacher leaders.

Project Product(s):

Design and Development of a K-12 STEM Observation Protocol

PIs: Emily Dare, Elizabeth Ring-Whalen, and Gillian Roehrig; Co-PI: Joshua Ellis
Grade Level(s): Elementary School, Middle School, High School
STEM Discipline(s): STEM Practices
Focus Audience(s): Teachers
Proposal Strand: Teaching
Project Type: Late Stage Design & Development

About the Collaborative Project: The main goal of our project - Collaborative Research: Design and Development of a K-12 STEM Observation Protocol - was to create a valid and reliable observation instrument for use in K-12 science and engineering classrooms using integrated STEM approaches. Previously, no instrument to characterize integrated STEM education existed. Our work has resulted in a detailed conceptual framework for integrated STEM education and the STEM Observation Protocol (STEM-OP), which includes 10 observable items with four descriptive levels for each item (scored 0-3): 1) Relating content to students’ lives, 2) contextualizing student learning, 3) developing multiple solutions, 4) cognitive engagement in STEM, 5) integrating STEM content, 6) student agency, 7) student collaboration, 8) evidence-based reasoning, 9) technology practices in STEM, and 10) STEM career awareness. The STEM-OP can be used in a variety of educational contexts and research. To assist others in learning about the STEM-OP and how to use it, we created an online training course via Canvas that guides users through informative videos about each item and their corresponding scoring levels, including examples of classroom video. The online platform includes opportunities for users to test their knowledge and receive feedback through course quizzes. The STEM-OP and its associated training materials are appropriate for a variety of stakeholders, including K-12 teachers, district administrators, teacher educators, and educational researchers.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: Four of the five members of our PI team knew each other previously, with three members being former doctoral students (PIs Dare, Ellis, and Ring-Whalen) at the University of Minnesota, with PI Roehrig as their advisor. After several years of working together on previous projects—and much conversation and complaining about the need for a classroom observation instrument focused on K-12 integrated STEM education—we realized that we were the team to propose this idea to NSF. Our team expanded to include PI Rouleau, who PIs Dare and Ellis connected with their time as faculty at Michigan Technological University. PI Rouleau’s expertise in quantitative methods and survey design was the missing piece for this work as the other PIs are more well-versed in qualitative research methods. For us, the team made sense and we knew we would work well together.

Affordances of Pursuing and Implementing Your Collaborative Project: One of the biggest benefits from our collaboration has been in working with graduate students from multiple institutions who get to know one another through project Zoom meetings, conference presentations, and student-driven research projects. They also have opportunities to work with different faculty, which helps to expand their networks. As the PI team, we each bring different perspectives to our work, even if we don’t always agree on ideas. This is certainly a strength as we challenge each other and navigate those differences to come up with better solutions than any one person could have conceptualized on their own. This has led us to publish multiple papers and present at a variety of conferences over the course of the grant. Because our team includes individuals at different points of their careers, we’re able to lean on each other when it comes to deciding who takes the lead on a publication or conference presentation. When it is clear that someone is taking a lead or even when the work is truly collaborative, we discuss author order early on. This has allowed us to be productive in getting our work out there. Our collaboration has also enabled us to access different types of schools (NGSS-aligned and non-NGSS) in different parts of the country (upper Midwest and south Florida).

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: The most challenging aspect has probably been scheduling, but that’s a struggle for most collaborative efforts! We have been limited in our opportunities to physically gather as a full team. While part of this was due to COVID travel restrictions, most of our work had taken place remotely. We’re certainly proficient with Zoom meetings, but getting together physically for a project meeting has only happened once. While we’ve certainly been productive, being together could have made certain conversations easier, for example, those related to different methodologies or mapping out our plans, which may require physical tools to explain.

Speaking of Zoom, we’ve also experienced other technological problems. The first challenge was in identifying a collaborative tool that everyone had access to. Since different universities use different tools and even have different permissions within those tools, we had to play around with a few different methods. Another technological challenge specific to our work related to the STEM-OP online training platform. Once we decided on using Canvas to host the platform, we had to decide which institution would host the platform, meaning that one person would inevitably be in charge of it. Fortunately, Canvas offers a “Free for Teachers” version that allows us to all take ownership in an institution-agnostic Canvas course.

Advice for Those Considering a Collaborative Grant Proposal: Most of the advice we have to share is probably advice for any kind of proposal, but perhaps more important for collaborative efforts:

  1. Schedule regular meetings and keep detailed notes of those meetings. This record keeping is especially important when people can’t make meetings or you have to remind yourself of discussions from months ago.
  2. Set deadlines of when tasks should be completed as it helps with accountability, scheduling, and planning.
  3. Get to know the people you are planning to collaborate with. Don’t have the proposal be your “first date” with others. It might work out, but knowing your teammates and how you work together is important to keep things moving along.
  4. Plan the collaborative proposal early because creating or accessing all documents from each individual can be time consuming!
  5. Lay out clear roles and responsibilities for each team member. This might change over time, but having some clarity here will help prevent conversations like, “I thought you were working on that!”

Project Product(s):

  • The main product from our work is the STEM Observation Protocol (STEM-OP), which is made available through an ASEE Conference proceeding (Dare, Hiwatig, et al., 2021).
  • We have also created an online training platform, which is available through a Canvas course. This course allows new users to learn about the STEM-OP through a series of videos that explain each item and provide examples from video-recorded classroom observations. Users have opportunities to test their knowledge and receive immediate feedback on “score on your own” video examples.
  • Complete List of Published Works

Design Talks: Building Community with Elementary Engineering

PIs: Chelsea Andrews and Jessica Watkins; Former PI: Kristen Wendell
Grade Level(s): Elementary School, Middle School
STEM Discipline(s): Engineering, STEM Practices
Focus Audience(s): Students, Teachers
Proposal Strand: Learning
Project Type: Early Stage Design & Development

About the Collaborative Project: The goal of the Design Talks project is to enact and characterize multiple types of whole-class engineering design conversations in first-grade through sixth-grade classrooms. We are developing case studies of specific types of teacher-supported conversation in which students are asked to consider design decision-making not just as a technical task, but as a complex socio-technical activity with ethical, economic, and political dimensions. The project addresses important questions of classroom practice for engineering education, e.g., What prompts can teachers use to elicit a diverse set of student ideas about how a design problem should be defined? How can educators attune students to their peers’ differing ideas about why a design prototype failed or succeeded? How can classroom norms for large-group design discussions enable students to prioritize care and justice as they make design decisions?

Partnering with elementary and middle school teachers, we are developing resources for five kinds of whole-class Design Talks (Problem Scoping, Idea Generation, Design-in-Progress, Design Synthesis, or Impact) in which classroom communities can deepen their reasoning about design problems, prototypes, and impacts. Each kind of Design Talk centers on a different framing question and is intentionally facilitated by a teacher with prompts to help students voice their ideas and make connections to others’ ideas.

By using Design Talks in their classrooms and tailoring each Design Talk to a specific purpose, teachers can help students collectively connect the NGSS practices to an engineering design experience in a way that highlights different student strengths and makes space for socioethical reasoning about how society and technology interact. Our research on these talks foregrounds a perspective of care in students’ engineering discourse and builds on emerging frameworks exploring compassionate design, macroethics and ideology, and critical socio-technical literacy. 

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: We conceived of the project in response to our shared interest in pre-college engineering classroom discourse and our shared vision of expanding engineering to include care and empathy. Our collaborative proposal grew out of many previous conversations about the need for more explicit tools for whole-class conversation in engineering education. Our approach to giving shape to the project partnership involved building heavily on our prior relationships as early career peers. Because of our existing relationships, we were able to talk explicitly about how the perspectives and resources we would each bring to the project would be different yet complementary to each other. For example, Tufts University had strong connections to local schools, while Vanderbilt University offered connections to a larger research network. We also used the strength of our existing relationships as a platform for early transparent conversations about the relationship between the potential project and our own individual goals. Focusing on our relationships helped us feel confident that we would be able to handle tricky situations that require trust, vulnerability, and negotiation.

Affordances of Pursuing and Implementing Your Collaborative Project:
Kristen: Collaborating with researchers at another university enables me to hear about and learn from the conversations going on at a different institution with a different combination of education research areas, projects, and perspectives than I am exposed to at Tufts.

Jess: I had recently moved to a new institution and had not had time to form deep partnerships with teachers yet. This project afforded me opportunities to maintain research productivity and connections in that initial start-up period.

Chelsea: On a smaller grain size, the limitation of half the team being unable to go into classrooms was an unexpected affordance: It continually pushed us to understand and better articulate what was happening and what we were noticing in each classroom visit.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: When the pandemic hit just before our project launched and (nearly) every meeting moved online, we predicted that the physical distance between collaborators would no longer be a defining feature of this project, but that turned out not to be the case. We underestimated how important the informal check-ins that happen outside of regular meeting times are to group intimacy; without these small, frequent chances to catch up, we had a less full picture of each other’s lives. We realized that we needed to make more time for check-ins on everyone’s professional and personal lives and explicitly lengthened our meetings to accommodate this time. This check-in time goes deeper than a “temperature check”; we use these stories and details to help us determine who has the most time and energy available for project tasks and shift responsibilities accordingly. Beginning every meeting by centering the people, instead of the tasks, aligns with and reinforces our goal to create a culture of humility, trust, and openness.

Advice for Those Considering a Collaborative Grant Proposal:

  1. Make explicit your core shared values & goals for the project. We found this especially important early in the project but also revisiting these 1-2 times each year as the project unfolds. These values can span how to organize the project or partner with teachers, as well as how we understand issues of care and equity in our outcomes. In our discussion of goals, we discussed not only what we hope to get out of the project but also how the project can help us realize our personal and professional milestones. 
  2. Anticipate where there might be tensions around values and goals. We found it crucial to set up norms for transparency, trust, and vulnerability early on so that when we did encounter tensions, we could create space to voice concerns and negotiate ways forward. 
  3. Create multiple paths for managing tensions. We also recognized that there may be times when our tensions could be resolved collaboratively and times when the project could best be served by making space to explore different threads or goals within and across teams. 

Project Product(s):

  • Wendell, K., J. Watkins, C. Andrews, N. De Lucca, & T. J. Pangan, (2021, June) Design Talks: Building Community with Elementary Engineering. Poster presented at the 2021 Annual DRK-12 PI Meeting [Virtual].
  • De Lucca, N., Pangan, T. J., Watkins, J., Wendell, K., & Andrews, C. (2022, January 28-31). SEEDS 2022 Scholarship Incubator: Power, Positioning & Middle Schoolers’ Socioscientific Sensemaking [Conference presentation]. SEEDS Virtual Meeting 2022, [Virtual]. 
  • Wendell, K., & Watkins, J., & De Lucca, N., & Pangan, T. J., & Woodcock, R., & Andrews, C. (2022, August), “Should we build this?”: Student reasoning in intentionally facilitated socio-technical design talks. Paper presented at 2022 ASEE Annual Conference & Exposition, Minneapolis, MN. https://peer.asee.org/42015 
  • Wendell, K., Watkins, J., Andrews, C., DeLucca, N., Pangan, T., Woodcock, R., Gor, V., Malinowski, M., Sood, N. (in press). Design Talks: Whole-Class Conversations during Engineering Design Units. Science and Children.

Designing for Science Learning in Schools by Leveraging Participation and the Power of Place through Community and Citizen Science

logoPIs: Heidi Lyn Ballard and Sol Henson
Grade Level(s): 3-5
STEM Discipline(s): Science, Environmental Science
Focus Audience(s): Elementary School, Teachers, Environmental Educators, Conservation Practitioners, Rural
Proposal Strand: Learning
Project Type: Early Stage Design and Development

About the Collaborative Project: We are developing and testing a place-based environmental science research and monitoring program for elementary school students and their teachers in the foothills of the Sierra Nevada Mountains of California, called Our Forests. In the face of catastrophic wildfires sweeping the West, students investigate locally relevant phenomena related to forest health, such as fire management and invasive species. The students collect and analyze forest data related to resource management issues and share their findings with community partner forest managers. We are developing and studying a reproducible and adaptable place-based instructional model for schools having underserved rural populations that link their own real science to management of local forests. The study includes 15 comprehensive public schools and charter schools across Nevada County in a rural region having limited access to the formal and informal science learning opportunities typically available in urban centers. Based on our previous research, all students engage in 3 key design features: 1) Collecting place-relevant environmental data, 2) Meaning-making with collected data embedded within larger data sets, and 3) Community-engaged, place-based projects and interactions with community partners. To study student science learning through this program, our research questions ask: 1) To what extent and in what ways do students participating in the Our Forests program develop a sense of identity and agency to take action with the forest science they are conducting (environmental science agency)? and 2) In what ways do the three key design features foster these science learning outcomes for students? We use a design-based research approach to determine how the design variations impact learning through two cycles of the program. We use pre- and post-program surveys, field observations, semi-structured interviews with students and teachers, and examination of student work and artifacts to track changes in students’ environmental science agency.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: The University of California, Davis Center for Community and Citizen Science has been conducting education research on how and what people learn through participatory approaches to science for many years. Sierra Streams Institute is a watershed science and education organization that has been committed to citizen science since its inception over 20 years ago and has been delivering watershed education, often in partnership with the Nevada County, CA schools. When PIs Ballard and Henson met at a conference and discovered our perfectly complementary interests and expertise, we began over a year in advance to write our ideal proposal for a school-based, locally driven, scientifically rigorous citizen science program for elementary students. We were able to bring the Nevada County Superintendent of Schools into the process early to make sure the needs of the county schools and teachers were being met. The process of meeting, brainstorming, narrowing, defining, and putting into writing our shared goals and responsibilities helped to build the partnership we needed to enact the project.

Affordances of Pursuing and Implementing Your Collaborative Project: Our project has benefited through a combination of strengths around theory and practice. The Center has created a framework for Youth-focused Community and Citizen Science (YCCS) based on previous research in formal and informal settings. Sierra Streams has been able to benefit from this framework so that implementation of the work on the ground is informed by the latest educational research around YCCS. Sierra Streams is a long-time community-based organization with a large network of local environmental organizations and land managers that has allowed us to connect schools to the forest and fire ecology work taking place in the area and the community members doing that work. Together, we are able to develop and implement a program, and study its impacts on student learning, in a way we couldn’t have done without each other. Also, a note about administrative support: This is the first large NSF grant that Sierra Streams has worked on. It has been helpful to have a large institution like UC Davis as a collaborator to help navigate the administrative ins and outs of these grants.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: It takes time to build a strong working relationship with new collaborators. There are a lot of moving pieces to these projects and fitting them all together across different organizations with different norms (academic institutions, school districts, non-profits) takes a lot of time, commitment, and patience. Working across many people with multiple goals and interests and getting everyone on the same page all the time is challenging, so we have had to revisit our communications and meeting structures frequently over the life of the project. Luckily our shared vision is strong and that has helped us navigate many of the challenges that have come up.

Advice for Those Considering a Collaborative Grant Proposal: If possible, partner on small collaborative projects or just attend events together early on in the process. Learn how the different organizations operate and the strengths of the different players that will be your team on a multi-year project. But also, team members come and go from the different organizations, so establishing the foundations of goals, norms, and structures for the project that can be passed on for onboarding new personnel can be really helpful. Ongoing communications are really key, so using a platform like Slack has allowed us to stay in relatively good touch even though we are not in the same city. With all of this said, meeting in-person is invaluable and should be prioritized whenever possible. Also, make time to have fun with your team; it doesn’t have to all be work!

Project Product(s):

  • Research instruments for studying youth agency and identity with environmental science for elementary students, including observation and fieldnotes guides, pre-post surveys, and semi-structured photo elicitation interview protocols.
  • Handbook for Community and Citizen Science with elementary-aged students that highlights best practices learned from the project for facilitating community and citizen science amongst schools and community-based watershed organizations.
  • NGSS standards-aligned supplementary curriculum for 3-5 grade teachers and students, guiding them through forest science data collection and analysis and sharing their results with community partners.
  • Research publications in peer-reviewed journals and short briefs for practitioners.

Developing an Online Game to Teach Middle School Students Science Research Practices in the Life Sciences 

logoPIs: Ryan Baker, David Gagnon, and Shari Metcalf
Grade Level(s): Middle School, High School
STEM Discipline(s): Science
Focus Audience(s): Students, Teachers
Proposal Strand: Learning
Project Type: Early Stage Design & Development

About the Collaborative Project: This is a four-year project to explore the high-level research goal of how learning progressions can inspire educational game design and be empirically derived from large audience game play data. Wake: Tales from the Aqualab, is an immersive web-based middle school science game designed to teach science practices of experimentation, modeling, and argumentation in aquatic ecosystems. Students travel to life sciences research stations across different ecosystems (e.g., kelp forests, coral reefs, arctic seas) and take on jobs that consist of specific forms of observation, experimentation, modeling, and arguing from evidence. We are developing educational assessment and data mining approaches for game data, in order to assess different learning pathways within the game and to improve learning outcomes.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: Our approach to identifying and forming partnerships was based on identifying researchers with the necessary expertise within this very specific intersection of games, data, and classrooms, specifically in science. David Gagnon is the director of Field Day at the Wisconsin Center for Education Research, which has an award-winning game design studio and experience in large-scale distribution. Shari Metcalf at the Harvard Graduate School of Education is highly experienced in classroom research with inquiry-based simulations, working closely with teachers and students. And Ryan Baker at University of Pennsylvania brings expertise in educational data mining and learning analytics to improve design and to learn about learners. So, it really was just a matter of segmenting off the kinds of expertise we needed and finding partners who can do that.

Affordances of Pursuing and Implementing Your Collaborative Project: We're doing things that none of us could have done alone. We're all overlapping in our interests and our experiences but also know how to stay in our own lane, and because of that, we can do a project that that doesn't look like a traditional research project, in that it's a large public project that has actual impact on the way teaching and learning will take place. But it also is really pushing on the kinds of data analysis that we can do.

We're working with very large data sets that allow us to do different kinds of analyses than often come up in a learning science related project. Then, because of the classroom component, we're really getting that qualitative research on how individual kids and teachers are responding to and working with the software.

So, we're drawing on the resources and expertise of the people and teams at the different institutions, as well as each organization’s infrastructure: prior teacher-researcher relationships at Harvard, existing modes of engaging teachers in Madison, and the educational data mining tools and expertise at Penn.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: One of the challenges that comes to mind is that we’re all very busy people who have a lot of different moving parts in our lives, so a challenge for a multi-institutional grant is the lack of day-to-day interactions where ideas can emerge over coffee and casual conversations. Rather, everything needs to be planned. Working across institutions means that for every single intellectual move, we have to create a space for that to happen. But it was helpful that we brought together a team of experienced people, who all realized that it's not a matter of being physically in the same spot, it's just the need for focused, collaborative planning.

Through the pandemic, people and teams, even within the same organization, have had to adjust to a lot of online work, which turns out to really help support the needs of collaborative multi-institution grants. We schedule regular meetings over Zoom, and we use online tools for shared documents and shared task management and design, so we find all these ways to collaborate online. And we made sure to include in the grants the budget to be able to meet in person occasionally. It's really important to build moments of social presence, and when that has been possible, it has always been a fun and productive time, a time to connect and celebrate our work together.

Advice for Those Considering a Collaborative Grant Proposal: Our primary advice on working collaboratively is to be very clear about everybody’s role, and think about how to plan around that, because it really seems to work better when there's clarity and understanding about who’s responsible for each part of the project. That can be hard to do when you have a team of people who are all interested and curious and believe in the project. Figuring out what the roles are, and where responsibility is sliced up is challenging to do, but it's also important for having everybody know how they can contribute and what they're going to add to the project, and to make sure we’re all working together to move things forward. Communication is key. Finding regular times to meet virtually, keeping on top of goals and schedules, and getting to meet in person occasionally to bring it all together.

Project Product(s): The game has been recently released publicly and is being broadly disseminated through BrainPOP and PBS Learning Media. Visit the Field Day website for the Wake game and for publications related to this project. We have upcoming presentations at NSTA, ISTE, iLRN in 2023, as well as proposals for Play Make Learn and Meaningful Play.

Developing and Validating Assessments to Measure and Build Elementary Teachers' Content Knowledge for Teaching about Matter and Its Interactions within Teacher Education Settings

logoPIs: Deborah Hanuscin and Jamie Mikeska
Grade Level(s): Post-Secondary
STEM Discipline(s): Science
Focus Audience(s): Preservice Teachers, Post-Secondary Faculty
Proposal Strand: Assessment
Project Type: Exploratory

About the Collaborative Project: In this project we are developing assessment tools to measure and support the development of preservice elementary teachers’ content knowledge for teaching (CKT) about matter and its interactions–the professional knowledge that teachers use to recognize, understand, and respond to the teaching challenges they encounter in the work of teaching elementary science. The tools include: 1) a summative assessment instrument that can be used in an elementary science methods or content course to measure preservice elementary teachers’ mastery of CKT about matter and 2) a set of six CKT instructional modules that can be used by elementary science teacher educators to provide substantive learning experiences within their courses and support their own learning about how to build preservice elementary teachers’ CKT about matter. On the project, we have worked with a professional learning community of elementary science teacher educators across the United States as they integrate these assessment tools and instructional materials into one or more of their elementary science courses. We are studying how teacher educators use these materials to support preservice teacher learning and examining potential growth in the preservice teachers’ CKT about matter from the beginning to end of the course by using the summative assessment as a pre/post measure.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: Our collaborative project actually grew out of a conversation we had about a previous NSF-funded project–we learned we were focusing on similar challenges related to elementary science teachers’ knowledge for teaching, but different aspects of the challenge. We both saw a need for tools that we–and others–could use to move this work forward. By collaborating, we felt we could strengthen each other’s efforts and address the challenge in a more robust way. We have collaborated with an advisory board with relevant expertise and with a diverse group of teacher educators from different backgrounds, career stages, and institutions to ensure the quality and utility of the tools and resources we are developing.

Affordances of Pursuing and Implementing Your Collaborative Project: We have learned from each other throughout our collaboration–from ways to manage a project, how to collaborate with advisory boards, and disseminate project outcomes. We all bring diverse perspectives and backgrounds, which helps us expand each other’s ideas, push our collective thinking forward, and keep our efforts focused on our project goals.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: We have found that developing a shared language is critical to strengthening and enacting our shared vision. This has developed through co-authoring and co-presenting our materials, research, and professional learning sessions.

Timing sequential aspects of the project that are dependent on one another has necessitated coordination and frequent communication. Being in two different time zones can sometimes make that challenging, but we have all been flexible with time and have held regular meetings. We have also identified key leads and roles for all involved to share the responsibilities associated with managing multiple aspects and tasks.

Advice for Those Considering a Collaborative Grant Proposal: Coming together around shared goals is important and essential to putting together a clear and coherent proposal in which the research design and development activities of all partners work together and inform one another. Teams must specify the roles and responsibilities of each partner, so that they are distinct, but also show how they complement one another to achieve the goals of the project. Additionally, we have found that having ongoing open and transparent communication about any questions or concerns as they arise along the way is key to a successful partnership.

Project Product(s): Products from our project, including tools and resources as well as publications and presentations, can be found at our project website: http://cktscience.org.

High School Students' Climate Literacy Through Epistemology of Scientific Modeling

PIs: Mark Chandler and Cory Forbes
Grade Level(s): High School
STEM Discipline(s): Science
Focus Audience(s): Students, Teachers
Proposal Strand: Learning
Project Type: Implementation and Improvement

About the Collaborative Project: The High School Students' Climate Literacy Through Epistemology of Scientific Modeling (CliMES) project is a five-year project funded by the National Science Foundation (DRL 1720838 and 1719872), which involves a collaborative partnership between scientists, school district officials, teachers, and education researchers. Through the project, a three-week curricular module was developed, piloted and improved to allow high school students to use a web-based climate-modeling tool called EzGCM, which was designed for non-scientists using authentic NASA global climate data. EzGCM affords students opportunities to engage in scientific practices of climate science, with a particular emphasis on scientific modeling, which is a critical science and engineering practice foregrounded in the Next Generation Science Standards (NGSS). Through design-based research, evidence of teaching and learning was collected using different research instruments. This data was analyzed to understand both teachers’ and students’ experiences using EzGCM and modify the curriculum accordingly based on these findings. More recently, project resources were disseminated among an entirely different and much larger group of high school science teachers from across the United States to support their use in other classrooms.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: The focus of the project evolved naturally out of a synergistic alignment between district needs and research and development interests. We partnered with a school district engaged in a process to revise a compulsory geoscience course in anticipation of new state science standards that emphasize global climate change. During that time, the project PIs had been collaborating on a climate education project proposal that embraced the idea of making realistic climate modeling tools available to teachers and students. This context provided a natural partnership underlying the CliMES project. There were two critical ingredients in this collaborative context. The NGSS, with their new emphasis on climate education, has opened the door for directly addressing anthropogenic change in Earth’s climate system through disciplinary core concepts. Equally as important, the NGSS’ emphasis on the science and engineering practice of modeling at the secondary level has facilitated our focus on the use of global climate models and epistemic dimensions of climate science using a suite of web-based tools designed for this purpose (EzGCM). Influence from NGSS was also felt strongly in state science standards, which support teaching and learning of these concepts and practices. Additionally, the semester-long geoscience course was a relative rarity in the United States, where Earth science is not typically required or may not even be offered at the high school level.

Affordances of Pursuing and Implementing Your Collaborative Project: One of the greatest affordances of pursuing and implementing a collaborative grant is having the opportunity to work with other practitioners, researchers, and scholars on an interdisciplinary team. By gaining perspectives from multiple disciplines, we are able to learn from each other to produce more robust work that can address cutting-edge complex topics. This can hopefully lead to additional large-scale projects and can positively impact the field and target population.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: Some of the challenges of pursuing and implementing a collaborative grant include: 1) Ensuring that all members have a clear understanding of the project’s goals and objectives which can sometimes be difficult when there are different perspectives and priorities, 2) managing resources and time, 3) anticipating change and evolving contextual factors, and 4) establishing good communication and coordination to ensure goals are being met and the project tasks are being completed in a timely manner. Some strategies to mitigate these challenges include: 1) Establishing clear roles and responsibilities, 2) developing a shared project plan, 3) scheduling regular check-ins and meetings, 4) encouraging frequent and open communication, 5) planning for adjustments and change during the project period, and 6) celebrating team successes and accomplishments.

Advice for Those Considering a Collaborative Grant Proposal: For those who are considering a collaborative grant proposal, we recommend: 1) Developing a clear and comprehensive project plan with goals and timelines, 2) ensuring all stakeholders have a shared understanding of the budget and responsibilities, 3) checking that the proposal aligns with the goals and priorities of the funding agency, and 4) thoroughly reviewing and proofreading the proposal before submission.

Project Product(s):

  • Bhattacharya, D., Carroll-Steward, K., & Forbes, C. T. (2021). Climate education in secondary science: Comparison of model-based and non-model-based investigations of Earth’s climate. International Journal of Science Education, 43(13), 2226-2249. https://doi.org/10.1080/09500693.2021.1958022
  • Bhattacharya, D., Carroll-Steward, K., & Forbes, C. T. (2021). Empirical research on K-16 climate education: A systematic review of the literature. Journal of Geoscience Education, 69(3), 223-247. https://doi.org/10.1080/10899995.2020.1838848
  • Bhattacharya, D., Chandler, M., Carroll-Steward, K., & Forbes, C. T. (2020). Using climate models to learn about global climate change: Investigating the phenomenon of increasing surface air temperatures using a global climate modeling approach. The Science Teacher, 88(1), 58-66. https://www.jstor.org/stable/27048205
  • Carroll-Steward, K., Bhattacharya, D., Chandler, M. C., & Forbes, C. T. (2022). Secondary science teachers’ implementation of a curricular intervention when teaching with global climate models. Journal of Geoscience Education, 70(4), 474-489. https://doi.org/10.1080/10899995.2021.1980706
  • Forbes, C. T., Chandler, M., Blake, J., Bhattacharya, D., Carroll-Steward, K., Johnson, V., DeGrand, T., Mason, W., & Murrow, B. (2020). Fostering climate literacy with global climate models in secondary science classrooms: Insights from a collaborative partnership. In J. Henderson & A. Drewes (Eds.), Teaching Climate Change in the United States (pp. 29-43). Routledge.
  • Olsen, A. A., Mostacedo-Marasovic, S. J. & Forbes, C. T. (in press). Applying Rasch Modeling to a Global Climate Change Concept Knowledge Assessment for Secondary Students. In X. Liu & W. J. Boone (Eds.), Advances in Applications of Rasch Measurement in Science Education.

Improving Multi-dimensional Assessment and Instruction: Building and Sustaining Elementary Science Teachers' Capacity Through Learning Communities

LogoPIs: James Pellegrino and Carla Strickland
Grade Level(s): 3–5
STEM Discipline(s): Science
Focus Audience(s): Elementary Science Teachers
Proposal Strand: Assessment
Project Type: Design and Development

About the Collaborative Project: The central and ultimate goal of the Next Generation Science Assessment (NGSA) project is to better understand how to build and sustain the capacity of elementary science teachers to instruct and formatively assess in ways that are aligned with the vision of the Framework for K-12 Science Education (i.e., multi-dimensional learning). We address the central goal through two activities:

  • Developing, testing, refining, and disseminating classroom assessments for formative use that will promote high-quality science instruction and student learning in grades 3–5.
  • Using classroom-based science assessment as a focus around which to promote teacher collaboration and capacity in multi-dimensional science instruction and learning, as promoted by the NGSS.

Elementary science teachers in the greater Chicagoland area have been valuable co-developers, implementers, and thought partners in this work. We capitalize on the digital environment to disseminate our tasks and create virtual community amongst teachers—supporting them to solve problems of practice that they may struggle with in isolation.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: A chance conversation amongst colleagues revealed that an elementary science assessment idea might garner interest at University of Illinois Chicago and University of Chicago. We started having meetings to define our focus and the project proposal. Very quickly we realized that the pairing of institutions and range of related expertise would make a fruitful collaborative proposal for DRK-12.

Affordances of Pursuing and Implementing Your Collaborative Project: Diversity of perspectives and expertise! And while individuals from each collaborating institution had their own expertise, we did not divide project activities so that only one group worked on them. We have been working to create and maintain cross-institution activities/tasks so that we can capitalize on this diversity.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: There were few challenges based around the collaborative aspect of the project. There are minor things like working with multiple institutions’ grant offices, IRBs, etc. but those haven’t been too bad. Though this particular team had not worked together previously on the same project, we were all familiar with the structure of the collaborative grant from prior work.

Advice for Those Considering a Collaborative Grant Proposal: Ensure that in addition to shared research interests, there is shared respect for different perspectives and expertise, shared vision/values for the work, and that personality and work styles are compatible. Consider how an evaluator would be valuable to maintain the health of the collaboration.

Project Product(s):

Investigating Productive Use of High-Leverage Student Mathematical Thinking

LogoPIs: Blake Peterson, Shari Stockero, and Laura Van Zoest; Co-PI: Keith Leatham
Grade Level(s): Middle School, High School
STEM Discipline(s): Mathematics
Focus Audience(s): Teachers
Proposal Strand: Teaching
Project Type: Early Stage Design & Development

About the Collaborative Project: In our previous project, we defined teachable moments in a mathematics classroom as student contributions that have three critical characteristics: student mathematical thinking, significant mathematics, and pedagogical opportunity. We call these moments Mathematical Opportunities in Student Thinking or MOSTs. In essence, MOSTs are student mathematical contributions that provide an in-the-moment opportunity to engage students in joint sense making. Building on MOSTs is a teaching practice that takes advantage of the opportunity that a MOST provides. This project focuses on understanding this teaching practice. We formally define building on MOSTs (building) as engaging the class in making sense of a MOST to better understand the mathematics of the MOST and theorize it as being comprised of four elements:

  1. Establish the student mathematics of the MOST as the object to be discussed;
  2. Grapple Toss that object in a way that positions the class to make sense of it;
  3. Conduct a whole-class discussion that supports the students in making sense of the student mathematics of the MOST; and
  4. Make Explicit the important mathematical idea from the discussion.

Our experience studying teachers’ use of MOSTs suggested that building is seldom enacted in classrooms. Thus, we have been engaged in studying building on MOSTs by working with teacher-researchers (6 middle school teachers and 6 high school teachers) to generate and then analyze instantiations of the practice. The teachers engaged their students in mini tasks that gave rise to predictable MOSTs and then enacted building in response to those MOSTs. We studied these enactments, shared with the teacher-researchers what we had learned along with our refined conceptualization of building and had them enact building with the same mini-tasks again. We are further refining our conceptualization of building based on our analysis of these second collection of enactments.

Approach to Identifying and Forming the Partnership(s) for Your Collaborative Grant Proposal: Our origin story begins in 2009 at the annual conference of the Association of Mathematics Teacher Educators. We found ourselves attending each other's sessions and realized that we were all interested in better understanding the same basic phenomenon—teachable moments in mathematics classrooms and how teachers take advantage of them. We began collaborating at that conference and haven’t stopped since.

Affordances of Pursuing and Implementing Your Collaborative Project: Pursuing a collaborative grant provided a focal point for our joint discussions and allowed us to leverage our individual expertise to develop a stronger project than we would have been able to do on our own. The collaboration has allowed us to continue to draw on each other’s strengths, and our discussions generate synergy that pushes us to clarify and develop our ideas. The structure of a collaborative grant allows us to maintain local control of our individual budgets while benefiting from working together.

Challenges of Pursuing and Implementing Your Collaborative Project and Strategies to Address Those Challenges: With multiple PIs on the project, one challenge that we faced early on was who was “in charge”. For instance, we would sometimes come to our bi-weekly project meetings and find ourselves asking, “What’s on the agenda today?” On the advice of our external evaluators, we now designate one of the PIs to be the project lead each month on a rotating schedule. This person plans our meeting agendas, handles communications (for instance, with our external evaluators), and generally makes sure that the project is moving forward. Another challenge was that although we enjoyed working together as a whole group, we weren’t progressing as fast as we wanted to in our work. This led us to develop a system where we assign one or two PI leads to some specific sub-projects or dissemination activities; these leads work with our research assistants to carry out the activities, bringing in the other PIs as needed to provide input and feedback. The PIs intentionally reevaluate our project and dissemination plans at least twice a year to check in on our progress and assign or reassign activities as needed.

Advice for Those Considering a Collaborative Grant Proposal: One of the keys to our successful collaboration is that it started with a common interest in understanding how to help preservice teachers identify and effectively use teachable moments. It is that common interest that continues to drive our work.

Another key is that we feel accountable to each other and do what we have committed to do, whether it be writing a piece of an article or coding a certain amount of data by a particular date. Basically, it becomes really important to develop a team you can trust and with whom you can work well.

Project Product(s): See the Publications and Presentations tabs of our website: BuildingonMOSTs.org.

Related Resources

See our Related Spotlights section for other Spotlights on project design and implementation topics.

RELATED SPOTLIGHTS

The following Spotlights highlight project design & implementation topics:

  • Adapting Research Amid COVID-19 (2023) 
    In this Spotlight, four projects share the changes they made to their questions and/or collection, measurement, and analysis of data in response to the COVID-19 pandemic.
  • Conferences and Workshops Organized by DRK-12 Projects (2021)
    This Spotlight highlights the work of DRK-12 project focused on organizing conferences or workshops to advance their work and the STEM education field.
  • DRK-12 CAREER Awards (2022)
    Hear from 27 DRK-12 CAREER Awardees about their work, plus advice on developing a proposal and managing an NSF grant.
  • DRK-12 Impact Studies (2023)
    This Spotlight features DRK-12 impact studies that are researching the efficacy or effectiveness of previous work. We hope this Spotlight provides insight into this research type.
  • Exploratory Research (2022)
    This Spotlight features descriptions of DRK-12 funded Exploratory projects and projects that have built on a previously funded Exploratory project, and a blog with more information about this project type, including how it is different from Early-Concept Grants for Exploratory Research (EAGER).
  • Replication & Building Evidence (2020)
    This Spotlight features perspectives, research, and resources related to building evidence in STEM education, particularly by means of replication.
Year