A scalable evaluation solution securely managed millions of constructed responses annually with role-based access, monitoring, and reporting for distributed and in-person scoring.
Client
A major educational testing and assessment organization
Goal
To build a sustainable and scalable evaluation solution for varied scoring systems across millions of constructed responses
Tools and Technologies
AWS CloudFront, AWS CloudWatch, Lamda, Amazon EndPoint, AWS, and Twilio
Business Challenge
The client needed a sustainable and scalable system to manage the evaluation of millions of constructed responses (CRs) annually through distributed or in-person scoring.
The goal was to develop streamlined processes, standardized reporting features, and secure, role-based access, as well as admin modules for monitoring.
Solution
Developed an online network for evaluation with enhanced capabilities for sample selection, training, monitoring, scoring and reporting, like:
- Streamlined reports to facilitate online communications and rater mentoring
- Secure, role-based access with appropriate restrictions for system functions
- Admin modules for broadcast, central documentation and communications
- Facilities for monitoring, broadcast, search and other tasks
- Multi-media modes for scoring
- Monitoring for ratings, reviews and other activities as required
Outcomes
- Ensured robust scheduling and integrated forecasting to project rater demand
- Enabled configurable role-based permissions to specific scoring and administrative functions
- Established program hierarchy setup with specific scoring/scheduling needs
- Created user-friendly interface with role specific navigation
- Developed direct tie to IBIS item bank for multiple functions
- Provided real-time, tier 2 support for active test evaluators
- Minimized downtime to avoid extending billable hours for evaluators
