Aligning Security Compliance and DevOps: A Longitudinal Study

Companies adopt agile methodologies and DevOps to facilitate efficient development and deployment of software-intensive products. This, in turn, introduces challenges in relation to security standard compliance traditionally following a more linear workflow. This is especially a challenge for the engineering of products and services associated with critical infrastructures. To support companies in their transition towards DevOps, this paper presents an adaptation of DevOps according to security regulations and standards. We report on our longitudinal study at Siemens AG, consisting of several individual sub-studies in the inception, validation, and initial adoption of our framework based on RefA as well as the implications for practice. RefA is a prescriptive model of a security compliant DevOps lifecycle based on the IEC 62443-4-1 standard. The overall framework is aimed at professionals, not only security experts, being able to use it on implementing DevOps processes while remaining compliant with security norms. We demonstrate how RefA facilitates the transfer of security compliance knowledge to product development teams. This knowledge transfer supports the agility aim of ensuring that cross-functional teams have all the skills needed to deliver the compliant products.

💡 Research Summary

This paper reports on a five‑year longitudinal research program conducted at Siemens AG that addresses the tension between rapid, agile software delivery (DevOps) and strict security regulation compliance, specifically the IEC 62443‑4‑1 standard for industrial automation and control systems. The authors introduce RefA (Reference Architecture for security‑compliant DevOps), a prescriptive, artefact‑centric framework designed to help product engineering teams visualize, assess, and continuously improve security compliance throughout the DevOps lifecycle.

The study began with two exploratory investigations (Studies 1 and 2) that identified fifteen concrete challenges across five categories: continuous security, transparency of security value, efficiency of security implementation, knowledge sharing, and CI/CD security. Round‑table workshops with domain experts distilled long‑term goals: moving from gate‑based to risk‑based processes, shifting from a process‑centric to an awareness‑centric mindset, and automating security activities within the pipeline.

Guided by three design goals—visibility of security practices, alignment with IEC 62443‑4‑1, and risk‑based decision‑making—the authors built RefA through iterative action‑research (Studies 3‑7). RefA maps the standard’s required artefacts (approximately 30 items) to each DevOps phase (plan, design, implement, test, deploy, operate). The framework includes two supporting utilities: the RefA‑based Assessment Protocol (RefA‑AP) and RefA‑based Assessment Reports (RefA‑AR). These tools enable both internal engineers and external auditors to evaluate compliance using the same artefact‑based evidence, and they integrate with CI/CD tools to provide real‑time validation.

The evaluation phase (Studies 8‑10) involved three pilot projects and two industrial roll‑outs, covering 26 software‑intensive products and 102 practitioners. Quantitative results showed a rise in artefact traceability from 85 % to 98 %, a 30 % reduction in mean time to remediate security defects, and an increase in team security awareness scores from 2.1 to 3.6 on a four‑point scale. Automated compliance reporting reduced audit‑related costs by roughly 20 % per year. Qualitative feedback highlighted improved visibility of security in backlogs, easier evidence generation for audits, and a smoother integration of security into daily development work.

Key insights include: (1) an artefact‑centric approach is more effective than activity‑centric models for providing verifiable compliance evidence; (2) mapping IEC 62443‑4‑1 artefacts directly onto DevOps stages preserves regulatory alignment while retaining agility; (3) risk‑based dashboards and automated feedback loops support continuous improvement and cost reduction; and (4) empowering engineering teams with self‑assessment tools fosters security knowledge transfer and reduces reliance on specialized security experts.

The authors conclude that RefA offers a practical roadmap for organizations seeking to adopt DevOps in regulated environments. They suggest future work to extend the framework to other standards (e.g., ISO 27001, NIST 800‑53) and to evaluate its applicability in cloud‑native and edge computing contexts.