AI Failure Loops in Devalued Work: The Confluence of Overconfidence in AI and Underconfidence in Worker Expertise

A growing body of literature has focused on understanding and addressing workplace AI design failures. However, past work has largely overlooked the role of the devaluation of worker expertise in shaping the dynamics of AI development and deployment. In this paper, we examine the case of feminized labor: a class of devalued occupations historically misnomered as ``women’s work,’’ such as social work, K-12 teaching, and home healthcare. Drawing on literature on AI deployments in feminized labor contexts, we conceptualize AI Failure Loops: a set of interwoven, socio-technical failure modes that help explain how the systemic devaluation of workers’ expertise negatively impacts, and is impacted by, AI design, evaluation, and governance practices. These failures demonstrate how misjudgments on the automatability of workers’ skills can lead to AI deployments that fail to bring value to workers and, instead, further diminish the visibility of workers’ expertise. We discuss research and design implications for workplace AI, especially for devalued occupations.

💡 Research Summary

This paper presents a critical analysis of the recurrent failures of workplace AI systems, particularly within “feminized labor” sectors such as social work, K-12 teaching, and home healthcare. The authors argue that a key, overlooked driver of these failures is the systemic devaluation of worker expertise inherent in these historically undervalued occupations. They introduce the concept of “AI Failure Loops” to explain the vicious cycle where underestimation of worker capabilities and overestimation of AI capabilities reinforce each other, leading to flawed deployments that further erode the visibility and recognition of worker skills.

The analysis is grounded in a comparative case study of AI deployments across the three feminized occupations, drawing on both literature review and the research team’s extensive field experience. The paper identifies six interconnected failure modes that constitute the AI Failure Loop: 1) Expertise Misunderstanding, where AI design is based on reductive views of workers’ complex, tacit, and social expertise; 2) Managerial Over Worker Needs, where AI tools prioritize organizational efficiency metrics over frontline workers’ practical needs; 3) Design Exclusion, where workers are marginalized in the design process; 4) Inappropriate Evaluation, where AI performance is measured against flawed benchmarks that don’t capture real-world work; 5) Forced Use, where workers lack the autonomy to refuse ineffective tools; and 6) Unwarranted Blame, where workers are held responsible for failures stemming from poor system design.

Through detailed examples, the paper illustrates how these loops manifest. In social work, risk assessment algorithms ignore caseworkers’ contextual judgment, increasing error risk and complicating workflows. In home healthcare, remote monitoring systems disrupt the relational care essential to the job by focusing solely on biometric data. In education, AI tutoring systems undermine teachers’ pedagogical autonomy, forcing them to perform extra “patchwork” labor to integrate the technology. In each case, the AI deployment fails because it is built on a devalued understanding of the work, and its subsequent failure further entrenches the perception that the work is less skilled.

The conclusion calls for a fundamental shift in AI research and practice to break these failure loops. This goes beyond participatory design to advocate for a deep re-evaluation and centering of worker expertise within AI development culture. The authors urge for new evaluation frameworks that capture tacit knowledge, system designs that enhance worker autonomy, and broader organizational and policy changes that improve the conditions and power of workers in devalued occupations. Ultimately, the paper advocates for a pro-worker future where AI actively contributes to the dignity and visibility of labor, rather than perpetuating its devaluation.