Stories 

of Impact

Stories of Impact involves stories amongst Crossroads Education employees as well as:

newNextroadsLogo_071521-04

xrTech_stackedColor

crefLogo_040121-88

From Team Members

sharing impactful moments of their work in/with schools and students

Three Partners Working Together to Improve Reading Ability and Confidence during a Uniquely Hectic School Year:

An Overview of their Success

May 2021

In the midst of COVID—when educators’ workload increased exponentially—Crossroads chose to pilot BookNook, a reading support program, to best serve the needs of one of our partner schools, Ignite Achievement Academy.

igniteStudents1 IgniteStudent2

Crossroads XR Coach Elizabeth started the BookNook program midyear with 38 students—both in-class and virtual. We had teachers identify students in their classrooms who would benefit the most from the program. Once identified, students took a BookNook placement test to put them into the appropriate reading group.


Following placement, Elizabeth met with each student twice a week for 30-45 minutes per session. Not long after, Ignite's IREAD scores came back. We worked with teachers to reevaluate who exhibited the most need and adjusted the roster.

The students have been working hard to improve their reading skills and expand their vocabulary and comprehension. In just three months, 50 reading levels have been gained! [BookNook uses Fountas & Pinnell reading leveling system] Alongside this, the program now has 62 participating students, and over 1,000 sessions have been hosted.

We love working with students to help them see their potential and we are grateful to Ignite's leadership for giving our team this opportunity. We are also thankful for our partnership with BookNook because they make reading remediation straightforward and fun!

Image from iOS (53)-1

 

igniteBooknook_sm

 


From the Desk of Crossroads

sharing non-news projects and reports

Whitepaper on Bright Spots grant from the Bill & Melinda Gates Foundation

Finalized May 2021

whitepaperCoverPhotologosStories-88

This paper is based on research funded by the Bill & Melinda Gates Foundation. The findings and conclusions contained within are those of the authors and do not necessarily reflect positions or policies of the Bill & Melinda Gates Foundation.


 

INTRODUCTION

Crossroads Technology (CRT)—DBA Crossroads Education—balances the power of technology and human relationships to improve educators’ experiences and provide all students with personalized learning.

CRT began this work by creating the Learning Commons (LC), a peer-to-peer tutoring program.    The LC focused on building the capacity of students to be a part of their learning and alleviate some of the burdens on the vital classroom teacher. Early success with the LC led to $750,000 in funding from the Bill & Melinda Gates Foundation to rapidly prototype the evolution and scalability of the LC model. This funding specifically stretched the capabilities of peer-to-peer tutoring into the K8 grade band where autonomy of learning was posited as a potential impediment to the model being successful.

The purpose of this white paper is to discuss the outcomes from this funded project, what the CRT team learned about our models for education and how we pivoted in response to the COVID-19 pandemic. 

PRESS COVERAGE ON GRANT • ARTICLE 1 ARTICLE 2ARTICLE 3

 

The Problem

In the Spring of 2018, the Gates Foundation launched a “Bright Spots” Request for Information (RFI) aimed at identifying existing breakthrough programs, models, platforms, or tools, supporting students who identified as Black, Latino, and/or from communities experiencing poverty in mathematics. Based on CRT’s previous experience and success in near-peer tutoring at the high school and post-secondary levels, the Gates Foundation was interested to learn if the model could be impactful with younger students.

The Crossroads Bright Spots Solution: The Learning Commons

The Learning Commons (LC) model designed by Crossroads Technology has four core components: 

Blended Learning Classroom

Crossroads works with participating schools to co-design and build an open-access collaborative peer-tutoring space. The space is designed to be a welcoming and exciting environment, a place free from the cultural restraints of the historical classroom design. The LC is designed for multiple learning events, where peer-to-peer engagement through tutoring and mentorship take precedence over large group instruction or lecture.

IMG_E31EDD0A01BC-1 CRE Classroom_3DView
Blended Learning Space 3D Mockup
 

 

Technology

The LC operations utilize web-based applications and hardware to support the blended learning events that are scheduled for the peer-to-peer space. These tools schedule event experiences like virtual private and small group tutoring, track student and adult usage of the physical and virtual space, and provide digital access to a multitude of resources available to learn and grow mathematics literacy.

nathalie_nexus element
Nexus (Virtual Collaboration + Tutoring Platform) Element (Data Tracking + Scheduling Platform)
 

 

Learning Commons Director

Crossroads provides a Learning Commons Director (LCD), a professional position in education not requiring a college diploma or teaching license. This professional is employed by Crossroads Technology but works onsite at the school location managing the entirety of the LC operations.

The LCD works closely with school staff to ensure strong program implementation and alignment for what is happening in the core mathematics classrooms. The LCD is also responsible for tracking student usage and reporting those outcomes in partnership with their back-office data team.

Near-Peer Tutors

Crossroads works with the partner school to identify students to serve as near-peer tutors and train them in best practices for collaborative peer-tutoring. As a part of blended learning, the peer tutors learn to engage with their fellow students in small group instruction as well as virtual tutoring in the LC environments.

Peer-led learning heightens the engagement around content for both the students and the peer tutors. Teaching is a very strong way to engage with learning mathematical content. 

Screen Shot 2021-05-24 at 4.12.33 PM Screen Shot 2021-05-24 at 4.12.48 PM
Middle School Learning Commons in Urban School District

 

Researching the Outcomes of the Learning Commons

Researching the Learning Commons’ (LC) impact on student performance in their mathematics courses started with quasi-experimental studies at the university level. This work was prior to the currently supported project. This research framed the LC impact by creating an ethical way of creating groupings of students inside of raw data and testing whether the group that chose to attend the LC outperforms the group of students that did not attend. In social sciences research, this is often done through a technique called ordinary least square regression (OLS) testing the relationship between demographic information about students and its impact on course grades in courses that offered access to an LC.

The first outcomes models showed that in a matched sample of two groups of students, those who visited the LC had a higher course grade in a standard undergraduate mathematics course than those who did not visit the LC. Similar findings were found for remediation level college courses, meaning that students that would not have passed that course for zero credit, were more likely to pass and more likely to persist into their next credit-bearing courses. This early research was promising and led to the funding of this project and subsequent research into the impacts as the LC scaled into the K-12 schools.

Methods for Measuring K–12 Learning Commons Outcomes

whitepaperFig1-01

The research methods for measuring LC outcomes benefited from an early funded pilot project in a small middle school that served the target student demographics of this funded research project. The early research methodology focused on how to optimize the utilization and intensity of use by students in the school. With this research approach, we were able to see extremely high levels of impact on student mathematics proficiency as measured by national standardized mathematics assessments.

New Research Methodologies for the LC

After the first pilots, researching the outcomes at the K-12 level required a necessary shift in the way that data could be collected and quasi-experimental studies could be conceptualized. The changes to the model that were most problematic for research were:

  • Open-access. Due to the logistical constraints of the typical K-12 school day, it is not possible to provide students true choice for using the LC as part of their mathematics learning. This required a significant shift in understanding the impact of the LC on students while existing in parallel to core mathematics teaching that might or might not match the cultural approach of the LC to mathematics and learning.
  • Sharing students. Classroom teachers’ resistance to sharing students during designated mathematics teaching and learning time with the LC services was anticipated. But, the level of resistance and the difficulty for scheduling students was very problematic without full buy-in from our professional teaching partners.
  • School Schedules. The chaotic nature of the logistics of a typical school day for a school serving a student population of more than 500 students often of which a majority experience trauma, poverty, and food insecurity should not be underestimated. Researching the correlates for LC success is difficult when operations do not permit students to access the services.

Research also needed to focus on the purpose of this newly funded project. Due to our earlier outcomes-based research, a key metric was to understand utilization and intensity of use by students in the funded LCs at four different K-12 school locations. We were then asked to implement the LC solution and look for early signs of success. We worked closely with the funder and independent evaluation team to build the specific metrics for success. If at least one out of the four schools met both the utilization and intensity of use targets at the “Good” level, then the project could move forward. The next stage would be to establish good methods for learning outcomes research where the LC was not students’ only access to mathematics content. For schools that did not meet the utilization and intensity of use targets, the partner research team would support the identification of promising practices in the school(s) that meet targets and develop a plan for implementing those practices in other poorly performing LCs. 
 
Results: What We Learned

The results of measuring the utilization and intensity of use for the four Learning Commons failed to meet the performance metrics to move on to the learning outcomes research. The team has several hypotheses as to the cause of these outcomes that could benefit other groups that are seeking to work on peer learning models in partnership with schools to improve student performance in mathematics. 

Data Collection and Cleaning 

The fidelity of data in all research projects is imperative in order to understand what is truly happening in the researched educational environments. Our team struggled with data collection operations that could ensure our ability to measure utilization and intensity of use for student participation with out-of-classroom learning experiences. We hypothesize that the following are some causes of this lack of capacity to execute this vital task: 

  • Open-access centers are not viable in the K-8 model because of student maturity or adult perception of student maturity. 
  • Teachers and administration must put effort into the viability of the operations which impedes their other key performance metrics for their work in the school.
  • The extremely high rate of transience of students in the target student population longitudinally makes it difficult to measure learning outcomes.
  • Student schedules and availability to structure visits to the LC were dependent upon the collaboration of the classroom teacher that shared the students. The willingness of this sharing of student time was difficult due to the complexity of the teacher’s own understanding of how their own job efficacy was being measured.
  • The LC space should be selected to be single-purpose, rather than allowing for multiple activities. We wanted the schools to see the investment in the activated space to be warranted by including many different uses, but ultimately that made it difficult to get consistent operations established.

After early failures during this funded project in meeting the agreed metrics for success, our team was perplexed. We knew we were doing great work at the schools but wondered greatly why our outcome metrics were not representative of this type of operation. Our team pivoted to create a new data collection strategy to evaluate the validity of the direct pull data from the schools’ student information system (SIS). Below in Table 1, we can show the difference between the new data strategy and the direct pull numbers from the SIS. These differences were significant and directly impacted our ability to meet the criteria for the success of the funded project.

  School 1 School 2 School 3 School 4

SIS Data for Number of Students in Focus Cohort

111

101

127

131

New Data Collection Number of Students in Focus Cohort

105

104

128

125

School Reported numbers to 3rd Party Evaluator

114

131

129

157

Percent Change

8% decrease

21% decrease

Negligible Change

20% decrease

Table 1. Inconsistency of data across different collection mechanisms is problematic for research outcomes

 

Services out of the Learning Commons 

Towards the end of the funded project, our team noticed a consistent pattern that our partner schools were shifting demands of LC programming. The partner teachers, as the pressure for student performance continued to increase, required the LC model to shift to exporting its services into the classrooms as a push-in intervention. In an effort to meet utilization and intensity of use metrics the Crossroads team tried to fulfill this requirement. In hindsight, this was a fatal flaw to the model and prevented the proper creation of truly collaborative and out-of-classroom peer-to-peer experiences for our target population of students.

School Calendars are Complex 

The assumptions that the typical school month of operation has a minimum of 4 active weeks, and therefore a minimum of 20 active days underestimated the complexity of the school calendar. Within this framework, the CRT team posited that with proper planning the LCs could achieve the utilization and intensity of use metrics of 2 visits per full active week for an average of 30 minutes per visit for each targeted student. Our assumptions vastly underestimated the complexity of the school schedule and resulted in a failure in how we calculated the utilization and intensity of use statistics. It is important to point out for other researchers that the cohort size statistic and visits data statistic will play a significant role in the n size to calculate the average time of each visit. Please see Table 2 below on how these can quickly change utilization and intensity of use statistics if utilizing them for measuring project outcomes:

 

School 1

School 2

School 3

School 4

Assumption Average Visit time per visit in minutes

30

30

30

30

Average Amount of Time Per Visit in minutes

27.52

32.11

23.40

38.84

Differential

-8.3%

+7%

-22%

+29%

Assumption active days

20

20

20

20

Active Days Actual

15

16

16

17

Differential

-25%

-20%

-20%

-15%

Table 2. Utilization and intensity of use differences from different data collection techniques

The Future of the Learning Commons

We understand the limitations of having auxiliary services like the Learning Commons (LC) for schools that are looked upon as only remediation or ancillary to the core instruction happening in the classroom. Through funding like this research project, we were able to limit the risk for schools adopting the LC and learn much more about how schools budget for professional and para-professional labor costs. What we learned about labor in education is profound, we began to question if how we have historically designed the LC to lessen the burden on the classroom teacher could impact the talent pipeline someday for recruiting and retaining classroom teachers.

During the second month of refining our understanding of this potential and our researching our funded project’s outcomes, the US education system was paralyzed by the COVID-19. All of our partner schools and most of the world saw students at home for the first time in a complete system onsite schooling shutdown. All momentum on the LC ended abruptly, including a pipeline for new LCs in Indiana including more than 25 new school partners. Our teams had to pivot like much of the world. We began to focus our efforts on crisis response, including virtual tutoring, food security, internet device security, and an alternative voice to the narrative of a frozen education system. While stifling to our current work, it led to opportunities where the LC could transition away from auxiliary programming and to where our methodologies would gain access to running core classroom instruction for the first time.

 

Learning Commons to XR Mathematics 

We readily acknowledge that we failed to meet our own metrics for success for the funded Bright Spots project. But, failure breeds innovation, and the funded LC project provided significant opportunities for rethinking where and how the benefits of mentorship and tutoring could be applied in the typical school day. Our team questioned if the LC could be positioned as one part of blended learning that took place in the classroom. Could classroom teachers be trained to implement a blended learning classroom that had multiple types of learning events? Could we train our own people to implement this type of model and get schools to outsource licensed teaching positions to our team as a service?

As the pandemic continued, the crisis in teacher recruiting and retention was heightened. Our team seized this as an opportunity to answer the above questions. Meaning, what if the future of the LC became that Crossroads deployed a high-tech, high-relationship-based blended learning mathematics program in partnership with schools for core instruction? While many saw this as a completely radical approach, we did secure a partner school that allowed us to build this model for the first time. We called the new model XR Mathematics, standing for Crossroads Mathematics, but also a nod to the extended reality (XR) of what we were trying to build. With our partnership with a charter school network, we built the first-of-its-kind mathematics department as a service in Gary, Indiana.

 

The Freedom to Operate Core Mathematics 

The opportunity to question everything about the core deployment of mathematics instruction was a profound leap forward for our company and team. With this approach, by contracting directly to already funded core mathematics positions, we no longer needed to seek auxiliary funding to provide to schools. In theory, XR Mathematics scales much more quickly than a Learning Commons model could ever scale due to this type of approach. Our design for XR Mathematics was heavily based on the use of technology and data to lessen the day-to-day burden on the classroom teacher. This approach, focusing on the activities of the teacher with technology, gave us new language. Rather than Educational Technology, or Ed Tech, we were designing Educator Technology or a new type of Ed tech. To deploy, we started with the types of learning events we knew to be beneficial to a diverse set of learners within the context of mathematics. These include:

learningEvents-01

Figure 2. Learning events of an XR Mathematics classroom.

The scheduling of these learning events is currently entirely manual, where our back office and onsite teams collaborate to build models of students’ mathematics progression and then schedule them into the different learning events throughout the academic week. The onsite teachers, or XR Coaches, are Crossroads employees that manage a cohort of up to 125 students by building relationships with the students and designing the weekly schedules. By the nature of this design, our XR Coaches are not required to be mathematics experts, but rather experts at building relationships and connecting emotionally with students. The online tutors, hired specifically for high mathematical expertise, connect into the classroom through our own virtual collaboration platform. When you hire XR Mathematics, you hire a team instead of a collection of individual teachers. By removing significant barriers of teacher job efficacy, we believe we have created the right circumstances to attract high-quality, diverse talent and retain them in the teaching profession. With further scale beyond the first 5 schools, we can begin to research and build a tiered talent pipeline in education that includes tutors (degree and/or non-degree), Learning Commons Directors (HS diploma non-degree), and XR Coaches (degree/licensed). Our vision is to expand XR Mathematics into other content domains and further work with partners to build the capacity of a new type of teacher recruitment, training, and retention effort.