Overview

Tangerine is an open source software application initially designed to collect and analyze reading and mathematics assessment data in primary schools (typically grades one through four), most commonly in low-income countries. Data collectors use the Tangerine platform on mobile devices like tablets and smartphones to administer the Early Grade Reading Assessment (EGRA) and Early Grade Mathematics Assessment (EGMA) to measure literacy and numeracy among primary school students. Developed by RTI International, Tangerine is a trio of products deployed based on audience type, length of assessment and how results are used:

• Tangerine: Class, used by teachers to track individual learning with short, formative assessments throughout the school term.
• Tangerine: Tutor, used by coaches and tutors for rapid assessments and lesson observations conducted during school support visits, the data from which are used by the application to inform the feedback that coaches provide to teachers.
• Tangerine, used by Ministries of Education, researchers and program evaluators for large-scale assessments, evaluations and national surveys.

Objectives

• Provide teachers, coaches and tutors and education professionals real-time assessment results to understand and track literacy and numeracy rates at the classroom, school district and national levels.
• Improve data quality by reducing data entry errors and eliminating manual entry from paper forms.
• Use prebuilt tools and resources embedded in the Tangerine software to reduce time spent in the field preparing for and conducting assessments.
• Evaluate and measure literacy and numeracy understanding to gauge and address knowledge gaps prior to national and international assessments, which typically begin at grade four.

Background

Using RTI International’s internal research funds, Tangerine launched in 2011 as open source software under a GNU General Public License. Since its inception, Tangerine has been used by more than 50 organizations to conduct more than 1.5 million assessments and surveys in over 100 languages and 65 countries (predominately in Central America, Africa and Southeast Asia). Tangerine was initially designed to facilitate the EGRA and EGMA methodologies that RTI developed in 2006 under the U.S. Agency for International Development’s Education Data for Decision Making (EdData) II project to assess numeracy and literacy in primary school (grades one through four). RTI International’s commitment to improving early-grade reading in low-income countries continues to be on the forefront of its agenda, and through the Tangerine products RTI has been able to extend the use of EGRA and EGMA worldwide.

With the trio of Tangerine products – Tangerine, Tangerine: Class and Tangerine: Tutor – users observe classrooms, conduct surveys, and administer EGRA and EGMA assessments orally (with a paper prompt), measuring student ability through the offline mobile application. To ensure precision of measurement, Tangerine’s built-in timer automatically records start and end times for reading and math tests, and when internet becomes available, the user uploads the results into comma-separated values (CSV) files available for analysis. Tangerine can be installed on any device running Android version 2.3 or higher; recommended specifications include a minimum 6-inch screen display, 1-GHz processor, 1 GB of RAM and a battery life of at least six hours. Depending on their requirements, users can collect and store data on RTI’s server; however, as Tangerine is open source software, some organizations and Ministries of Education install the open source software code on their own website hosts.

Project Lifecycle Application

• Analyze & Plan

Guided by the Agile Software Development methodology, we curate, create and provide Tangerine’s open source code base, which is available on GitHub. We work with education professionals and stakeholders, project staff, data collectors and Ministry of Education officials at more than 50 organizations, adapting the platform for many contexts varying by language, server specification and data collection needs. When customizing Tangerine applications, we provide feedback to the user regarding features and storage needs based on Tangerine’s pricing structure.

• Design & Develop

We follow a software design process that solicits user feedback early and often to ensure long-term cost-effectiveness and deliver value for our users. For example, during a four-day formal design sprint held in Nairobi for a Kenya-specific implementation of Tangerine: Tutor, our developers engaged with target users (early-grade instructional coaches) to discuss user experience, desired software features, preferred data collection methods and attainable goals given funding constraints. In doing so, our software developers and RTI project staff gained buy-in from stakeholders while creating an effective platform for users. We were also able to test new features and hardware, identify and fix bugs, and tailor the user experience prior to deployment. We regularly communicate with target users during the development phase to ensure that they understand the benefits and limitations of the software, which helps to reduce misunderstandings and related implementation delays.

• Deploy & Implement

Before deployment, it is important for the Tangerine team to understand how users will engage with the software. For example, in 2014 our staff conducted a randomized control trial in Kenya to test how to measure the software’s cost-effectiveness for use by teachers and coaches. While this research showed improvements in learning outcomes for a variety of ICT-based methods, the trial demonstrated that it would be more cost-effective to have coaches use Tangerine because this model results in one tablet serving between 12 and 15 schools, whereas other models required one tablet per teacher or per student. Following this research, the Kenyan Ministry of Education supported the national scale up of Tangerine: Tutor; now more than 1,300 coaches are using Tangerine: Tutor to improve instruction and increase literacy and numeracy rates in primary schools in Kenya.

• Cross-cutting: Monitor & Evaluate

Tangerine staff members regularly monitor the software and its functionality to identify flaws, improve usability, address user errors and suggest improvements to mitigate data collection and quality issues. Additionally, we are collecting and analyzing data from RTI’s projects to understand the data collection challenges that assessment administers face. Our platform provides real-time (and offline) data to teachers, tutors and directors to inform decision making and action. Teachers can track students’ progress and guide them to focus on specific skills to build knowledge. During their monthly instructional support visits with teachers, tutors can draw on Tangerine data from assessments and observations for their post-lesson feedback sessions. Ministry of Education officials can use customized reports and online dashboards to gain insights on national learning trends that inform and drive policy.

Lessons Learned and Recommendations

•  Gather users and stakeholders to discuss shared objectives, experiences, areas of improvement and findings and to achieve stakeholder and user buy-in. Prioritizing this discussion early in the development process through a formal design sprint ensures that you are building a system that is customized to the needs of the user.
• Allow sufficient time to validate user processes; though it may seem costly upfront, the result is a more effective product. One- or two-week development sprints, following the Agile Software Development methodology, allow sufficient time to complete software code reviews, test results from a sprint and complete end-user testing.
• Communicate early with users about the benefits and limitations of the software to prevent delays due to misunderstandings around functionality, content or support. Before making modifications, it is essential to understand fully which data users want to collect, where users want the data to be stored and how users will use the data for decision making.
• Regularly assess software functionality to detect flaws, locate and address user errors, and identify proactive ways to mitigate data collection mistakes. Conducting consistent software development sprints ensures that the code base is up-to-date and robust.
• Routinely use real-time and offline data to highlight impact through the platform. Teachers, coaches, tutors and educational professionals rely on this information to inform the way they approach addressing areas of concerns with assessment results.

Resources

Tangerine, RTI International. http://www.tangerinecentral.org/

Tangerine: Take a Tour, RTI International. http://www.tangerinecentral.org/demo/

Tangerine – Mobile Learning Assessments Made Easy, RTI International. https://www.rti.org/impact/tangerine-mobile-learning-assessments-made-easy

Images and Quotes

“Allows educators to focus more on students’ performance and results data rather than on the data collection process.” – Center for Educational Innovations (CEI)

Claudine Lim

Manager at The Principles for Digital Development

Claudine first joined the Digital Impact Alliance in October 2017, shortly after receiving a dual masters in international relations and public relations from the Maxwell School and S.I. Newhouse School at Syracuse University. After working as a Program Coordinator and Researcher for DIAL’s Business Operations, she is currently working with the Principles of Digital Development.