AI-Generated Rubric Interfaces: K-12 Teachers' Perceptions and Practices

This study investigates K–12 teachers’ perceptions and experiences with AI-supported rubric generation during a summer professional development workshop ($n = 25$). Teachers used MagicSchool.ai to generate rubrics and practiced prompting to tailor criteria and performance levels. They then applied these rubrics to provide feedback on a sample block-based programming activity, followed by using a chatbot to deliver rubric-based feedback for the same work. Data were collected through pre- and post-workshop surveys, open discussions, and exit tickets. We used thematic analysis to analyze the qualitative data. Teachers reported that they rarely create rubrics from scratch because the process is time-consuming and defining clear distinctions between performance levels is challenging. After hands-on use, teachers described AI-generated rubrics as strong starting drafts that improved structure and clarified vague criteria. However, they emphasized the need for teacher oversight due to generic or grade-misaligned language, occasional misalignment with instructional priorities, and the need for substantial editing. Survey results indicated high perceived clarity and ethical acceptability, moderate alignment with assignments, and usability as the primary weakness – particularly the ability to add, remove, or revise criteria. Open-ended responses highlighted a ``strictness-versus-detail’’ trade-off: AI feedback was often perceived as harsher but more detailed and scalable. As a result, teachers expressed conditional willingness to adopt AI rubric tools when workflows support easy customization and preserve teacher control.

💡 Research Summary

This paper reports on a mixed‑methods study of 25 K‑12 teachers who participated in a summer professional‑development workshop where they used the AI‑driven rubric‑generation platform MagicSchool.ai. The authors aimed to (1) explore the potential of large‑language‑model (LLM) tools for creating assessment rubrics and (2) understand teachers’ perceptions, experiences, and future intentions regarding such tools in the context of block‑based programming assignments.

Prior to the workshop, teachers completed a survey capturing demographics, teaching experience, and current use of rubrics and AI. Most taught grades 6‑8, and only about half had any experience with block‑based programming. Teachers uniformly reported that rubrics are essential for clarity and consistency but described rubric authoring as time‑intensive, especially the articulation of middle‑performance descriptors and the translation of creative, open‑ended tasks into concrete criteria.

During the workshop, participants first created a manual rubric for a sample coding task, then entered contextual information (grade level, scoring scale, standards, assignment description, and optional prompts) into MagicSchool.ai. The system generated a draft rubric, which teachers used to assess the same student work twice: once directly and once via the platform’s chatbot “Reina,” which delivered rubric‑based feedback.

Qualitative data were analyzed through thematic analysis by two independent coders, yielding eight major themes: (1) current rubric practices, (2) challenges in rubric creation, (3) prior AI use, (4) perceived strengths of AI‑generated rubrics, (5) mixed perceptions (positive speed/clarity versus generic language and misalignment), (6) concerns (fairness, equity, accuracy, privacy, workflow constraints), (7) recommendations for tool improvement, and (8) future adoption intentions.

Key findings include:

• Time‑saving and structural benefits – teachers described AI drafts as strong starting points that clarified vague criteria and provided a coherent scaffold for new or creative assignments.
• Need for extensive teacher editing – AI often produced generic phrasing, grade‑inappropriate vocabulary, or misaligned content relative to standards and instructional goals, requiring substantial refinement.
• Survey results – perceived clarity (high) and ethical acceptability (high) scored above 4.5/5, while alignment with assignments (moderate, ~3.7/5) and usability (low, ~3.2/5) lagged. The most cited usability weakness was limited ability to add, remove, or revise criteria without regenerating the whole rubric.
• “Strictness‑versus‑detail” trade‑off – AI feedback was seen as more detailed but sometimes harsher than human feedback, reflecting the model’s strict application of criteria without the nuanced judgment teachers typically exercise.
• Conditional willingness to adopt – teachers expressed interest in using AI rubric tools provided that (a) the interface allows easy, granular editing; (b) language can be tailored to specific grade levels; (c) the tool integrates smoothly with existing Learning Management Systems; and (d) ultimate control remains with the teacher.

The authors acknowledge limitations: a small, self‑selected sample and a single‑session context. Nevertheless, the study illuminates a realistic picture of teacher attitudes: enthusiasm for efficiency and structure tempered by concerns over fidelity to curriculum, equity, and loss of professional agency.

Implications for designers of educational AI include prioritizing transparent, editable outputs, offering scaffolded prompting that separates learning objectives from rubric descriptors, and ensuring seamless LMS integration. Future research should expand to diverse grade bands, longitudinal usage, and experimental comparison of different editing‑centric prototypes to quantify the impact of AI‑assisted rubric creation on teacher workload and assessment quality.