CrowdTone: Crowd-powered tone feedback and improvement system for emails

In this paper, we present CrowdTone, a system designed to help people set the appropriate tone in their email communication. CrowdTone utilizes the context and content of an email message to identify and set the appropriate tone through a consensus-building process executed by crowd workers. We evaluated CrowdTone with 22 participants, who provided a total of 29 emails that they had received in the past, and ran them through CrowdTone. Participants and professional writers assessed the quality of improvements finding a substantial increase in the percentage of emails deemed “appropriate” or “very appropriate” - from 25% to more than 90% by recipients, and from 45% to 90% by professional writers. Additionally, the recipients’ feedback indicated that more than 90% of the CrowdTone processed emails showed improvement.

💡 Research Summary

The paper introduces CrowdTone, a crowd‑powered system that helps email senders achieve an appropriate tone without requiring them to provide explicit tone instructions. The system accepts the email’s subject, body, and a set of mandatory (sender/receiver relationship, brief context) and optional (gender, native language, hierarchy, relationship type) metadata. Using Amazon Mechanical Turk workers, CrowdTone runs a two‑phase workflow: (1) a “tone scaffolding” phase where each worker reads the email from the recipient’s perspective, selects a primary tone (formal or informal) and one of ten secondary tones (e.g., appreciative, confident, courteous, emotional, enthusiastic, humorous, apologetic, serious, cold, enraged), and then either directly edits the email if the tone is judged appropriate, or first defines the ideal tone, lists concrete improvement points, and iteratively edits the message; three workers independently produce three revised versions. (2) a “consensus” phase where two of the three versions are selected based on agreement (≥66 % yes/no) or similarity of tone attributes, and a new set of three workers choose the better of the two, perform an additional refinement, and finally output the version that receives a majority vote. The entire pipeline produces a JSON or GUI output containing the improved email and, when relevant, the identified target tone.

To evaluate the approach, the authors first surveyed 92 internal participants, finding that 94 % consider tone important and that the greatest need for assistance occurs when writing “cold” emails to senior or unfamiliar recipients. They then recruited 22 employees who supplied 29 real emails they had received and found problematic in tone. Each email was processed through CrowdTone, and both the original recipients and a panel of professional writers rated the emails before and after processing on a four‑point scale (inappropriate, somewhat appropriate, appropriate, very appropriate). Prior to processing, only 25 % of recipient ratings and 45 % of writer ratings fell into the “appropriate/very appropriate” categories. After processing, these figures rose to over 90 % for both groups. Moreover, more than 90 % of participants reported that the tone had improved, and 75 % agreed or strongly agreed that the final emails met their quality expectations. Workers also reported that the step‑by‑step scaffolding was easy to follow and effective.

The study demonstrates that a carefully designed micro‑task workflow can enable non‑expert crowd workers to perform nuanced tone adjustments that rival professional editing. However, the authors acknowledge limitations: the approach incurs monetary cost and latency due to multiple rounds and multiple workers, and the current implementation is limited to English and to the cultural norms of the MTurk workforce. Future work is proposed to integrate automated tone prediction models to reduce the number of required human iterations, to develop real‑time plug‑ins for email clients, and to expand the system to other languages and cultural contexts.

Overall, CrowdTone contributes a novel combination of tone identification, structured improvement, and consensus mechanisms that together achieve high‑quality email tone refinement, opening avenues for broader applications in written communication assistance.