LLM-Assisted Authentication and Fraud Detection

User authentication and fraud detection face growing challenges as digital systems expand and adversaries adopt increasingly sophisticated tactics. Traditional knowledge-based authentication remains rigid, requiring exact word-for-word string matches that fail to accommodate natural human memory and linguistic variation. Meanwhile, fraud-detection pipelines struggle to keep pace with rapidly evolving scam behaviors, leading to high false-positive rates and frequent retraining cycles required. This work introduces two complementary LLM-enabled solutions, namely, an LLM-assisted authentication mechanism that evaluates semantic correctness rather than exact wording, supported by document segmentation and a hybrid scoring method combining LLM judgement with cosine-similarity metrics and a RAG-based fraud-detection pipeline that grounds LLM reasoning in curated evidence to reduce hallucinations and adapt to emerging scam patterns without model retraining. Experiments show that the authentication system accepts 99.5% of legitimate non-exact answers while maintaining a 0.1% false-acceptance rate, and that the RAG-enhanced fraud detection reduces false positives from 17.2% to 3.5%. Together, these findings demonstrate that LLMs can significantly improve both usability and robustness in security workflows, offering a more adaptive , explainable, and human-aligned approach to authentication and fraud detection.

💡 Research Summary

The paper addresses two pressing challenges in modern digital security: the rigidity of knowledge‑based user authentication and the brittleness of conventional fraud‑detection pipelines. Traditional authentication systems require exact string matches, which penalize users for minor linguistic variations, memory lapses, or language differences. Meanwhile, static machine‑learning classifiers for fraud detection struggle to keep pace with rapidly evolving scam tactics, leading to high false‑positive rates and frequent retraining cycles. To overcome these limitations, the authors propose two complementary solutions that leverage large language models (LLMs).

The first solution is an LLM‑assisted authentication mechanism that evaluates the semantic correctness of a user’s natural‑language response rather than insisting on word‑for‑word matches. The system first retrieves a user‑specific profile document and segments it to ensure that generated challenge questions draw evenly from the entire document, mitigating the known positional bias of LLMs. For each login attempt, the LLM generates personalized questions and reference answers, presents them to the user, and then assesses the user’s response for semantic alignment. Crucially, the LLM’s judgment is combined with a statistical similarity metric—cosine similarity between the embedding of the user’s answer and the reference answer. This hybrid scoring approach provides a safety net: if the LLM flags a correct but atypical response, the cosine similarity can still grant acceptance, while the statistical component prevents the model from being overly permissive. Experiments using Llama‑3.3 demonstrate that the system accepts 99.5 % of legitimate non‑exact answers while maintaining a false‑acceptance rate of only 0.1 %. The authors argue that this approach transforms authentication into a conversational, human‑friendly interaction, reducing cognitive load for elderly users or those with impairments, without sacrificing security.

The second contribution is a Retrieval‑Augmented Generation (RAG)‑based fraud‑detection pipeline. When a message arrives, the LLM first extracts key features—intent, urgency, tone, requested actions, entities, and contextual cues. These features are used as queries to retrieve relevant evidence from multiple external repositories, including confirmed scam databases, corporate policy documents, and domain‑specific reference materials. The retrieved evidence is incorporated into a structured prompt that grounds the LLM’s final assessment. The LLM then produces a scam‑likelihood score together with an explanatory rationale, enhancing transparency and trust. By anchoring the model’s reasoning in up‑to‑date, curated evidence, the system dramatically reduces hallucinations and over‑sensitivity that plague standalone LLMs. Moreover, because updates are applied at the retrieval‑corpus level rather than by retraining the model, the pipeline can adapt to emerging scam patterns with minimal engineering effort. Empirical evaluation shows a reduction in false‑positive rate from 17.2 % (baseline classifiers) to 3.5 % with the RAG‑enhanced system.

Overall, the paper makes three key contributions: (1) a novel semantic knowledge factor for authentication that replaces rigid string matching with LLM‑driven natural‑language evaluation, reinforced by document segmentation and hybrid scoring; (2) a RAG‑grounded fraud‑detection framework that ties LLM reasoning to curated evidence, mitigating hallucinations and enabling rapid adaptation without model retraining; and (3) empirical evidence that both approaches substantially improve usability, robustness, and explainability compared to existing methods. The work demonstrates that LLMs, when combined with carefully designed auxiliary mechanisms, can simultaneously enhance user experience and security in critical workflows, offering a practical roadmap for integrating generative AI into authentication and fraud‑prevention systems.