{"id":44411,"date":"2020-11-02T14:13:35","date_gmt":"2020-11-02T18:13:35","guid":{"rendered":"https:\/\/caireadydev1.wpenginepowered.com\/life-sciences\/cai-qrm-and-the-system-classification-big-reshuffle\/"},"modified":"2025-06-21T17:53:44","modified_gmt":"2025-06-21T21:53:44","slug":"qrm-and-the-system-classification-big-reshuffle","status":"publish","type":"post","link":"https:\/\/caiready.com\/life-sciences\/blog\/qrm-and-the-system-classification-big-reshuffle\/","title":{"rendered":"QRM and the System Classification \u201cBig Reshuffle\u201d: Don\u2019t Miss the Forest for the Trees"},"content":{"rendered":"

Recently, under its new Insights offering, KenX published an article, The Big Reshuffle \u2013 Impact Assessments and System Criticality<\/a>, discussing the implications regarding the change in approach from System Level Impact Assessment (SLIA) and Component Criticality Assessment (CCA) to System Classification (SC) and System Risk Assessment (SRA) in ISPE Baseline Guide Volume 5, Commissioning & Qualification<\/em>, 2nd<\/sup> Edition (2019). It is, on the whole, an excellent article and well-worth the time to read. In describing the impact of the \u201cbig reshuffle\u201d, the article addresses some common practices and paradigms that I believe merit further discussion, particularly with respect to understanding the role of System Classification and System Risk Assessment in the application of Quality Risk Management (QRM) principles to the integrated Commissioning & Qualification (C&Q) process.<\/p>\n

<\/p>\n

Based on input from owner company focus groups and C&Q and QRM SMEs, the article indicates several perceived concerns with SC\/SRA, including that SC\/SRA are not as robust as SLIA\/CCA, that use of SC\/SRA could thus lead to \u201cgaps\u201d due to details from CCA being missed, that \u201cdirect\u201d and \u201cnot direct\u201d classification results in ambiguity, that SRA is a duplication of effort, and that lack of documented CQAs and CPPs precluded performance of SRA. The article then discusses several responses: a paradigm shift (\u201creshuffle\u201d) in the understanding of system classification and impact assessments, applying SC\/SRA as a primer for the verification testing strategy, viewing SRA as a scalable QRM tool, and using SC\/SRA to \u201cconnect the dots\u201d from product and process to control strategy.<\/p>\n

While I agree at a high level with these responses, I think the article misses an opportunity to explain how the QRM-based approach using SC\/SRA is more robust, more rigorous, and more streamlined, in that the article allows to stand some assumptions, misunderstandings, and vestiges of the legacy C&Q approach; namely that SLIA (and, therefore, SC) is intended to be a \u201cfilter\u201d to determine which systems to apply QRM, that SRA is analogous to and intended to be a replacement for CCA, and that understanding of CQAs and CPPs is a new requirement in the C&Q process.<\/p>\n

Alternate Perspectives<\/span><\/h2>\n

CQAs and CPPs as C&Q Inputs<\/h3>\n

Starting with the latter: the article indicates that, while noteworthy, the lack of availability of CQAs and CPPs in an accessible format is an industry issue that is outside the scope of C&Q. I fundamentally disagree with this conclusion. The QRM-based integrated C&Q process described by the revised Baseline Guide assumes and requires<\/em> product and process knowledge (PPK) \u2013 i.e. CQAs and CPPs \u2013 as an input to the process. Without CQAs and CPPs, QRM cannot be applied<\/em> to the C&Q process. I would argue further that the legacy C&Q approach similarly required this knowledge. Both the SLIA and CCA criteria provide the following caveat (emphasis<\/em> added): \u201cThese criteria should be used to inform a judgment based on a comprehensive understanding of the product, process and the nature of the system<\/em>. They should not<\/strong> be used to replace the exercise of informed judgment by appropriately qualified personnel.\u201d<\/p>\n

Thus, even in the legacy C&Q approach described by the original Baseline Guide, understanding of the product and process \u2013 i.e. CQAs and CPPs \u2013 was required.<\/p>\n

I discussed this article with a lead author for both the original and revised Baseline Guide, who stated:<\/p>\n

The article correctly noted that many companies lacked initial documentation of CQA\/CPP at the start of a project and incorrectly viewed the SC process as a question-based approach to classify systems as direct for continuing on the Guide Process.  The article missed the point that once the decision is made to design a facility around a defined process (as a CMO does) that one can associate the CPPs associated with the process steps, with most likely 99% accuracy, and apply a RA to the designated system.  As such, SC is not a required QRM C&Q prior process step. Therefore, it is not and was not designed to be a key determination factor for connecting the dots in control strategy. (Note, the Guide includes a list of systems and normally associated CQAs\/CPPs in an appendix).<\/p>\n

One might describe the legacy C&Q approach as a partial<\/em> application of QRM to the C&Q process. CQAs and CPPs are identified as part of the process but are not used as the basis for risk assessment. SLIA and CCA are used as a form of \u201cbottom-up\u201d risk assessment based on understanding of the system\/component rather than a \u201ctop-down\u201d risk assessment based on understanding of product quality and process-related risks to product quality. Thus, system and component impact to product quality are determined, but specific process-related quality risks are neither identified nor assessed. As a result, risk evaluation becomes a binary: a system has impact, or it does not; a component is critical, or it is not. Similarly, risk control is merely implicit: components assessed as \u201ccritical\u201d comprise the risk control strategy. Accordingly, verification of the risk control strategy \u2013 ensuring that testing is commensurate with risk \u2013 is also merely implicit (and binary): critical components are commissioned and qualified; non-critical components are commissioned-only.<\/p>\n

So, this process incorporates risk-based rationale and decision-making, but that rationale and those decisions are binary. As made clear in regulatory guidance, we must understand risk not as a binary<\/em> but as a continuum<\/em>. Further, the risk basis for the legacy C&Q approach is based on implicit understanding of processes and systems that does not explicitly tie the risk control strategy to specific, identified process risks to product quality.<\/p>\n

By contrast, System Risk Assessment identifies specific process risks to identified product quality \u2013 i.e. specific CQA failures related to process failure to maintain specific CPPs. Those risks are assessed qualitatively (high\/medium\/low) rather than as a binary, and specific risk controls \u2013 design controls, procedural controls, alarms, etc. \u2013 are identified to mitigate those risks to an acceptable level. Thus, the verification strategy can be tied directly to an explicit risk control strategy, and the rigor (effort, formality, and documentation) of verification testing can be demonstrated to be commensurate with the degree of risk (and with the degree of applied risk mitigation).<\/p>\n

System Impact\/Component Criticality Assessment as a QRM \u201cFilter\u201d<\/h3>\n

In the legacy C&Q approach, SLIA was often implemented as a \u201cfilter\u201d or \u201cgatekeeper\u201d for further application of QRM. Systems would first be classified, and then systems classified as direct impact would undergo CCA. The assessed component criticality then dictated the verification strategy.<\/p>\n

The original Baseline Guide suggested that CCA should be performed on indirect, and sometimes<\/em> on no-impact, systems, to ensure that such systems \u201chave not subsequently acquired a critical function as the detailed design has progressed to conclusion.\u201d Informed SME judgment was intended to be used as the sanity check to ensure that system classification remained correct; systems that contain critical components should, by definition, be classified as direct impact.<\/p>\n

This is the inherent danger both in assessing risk from a system\/component perspective rather than from a product\/process perspective and in using system impact (and to a lesser extent, component criticality) assessment as a QRM \u201cfilter\u201d: a change in design definition or system boundaries could cause process-related risks to product quality to be missed and\/or could lead to an inappropriate verification testing strategy. This inherent danger is even more readily apparent when applying the same approach to the SC\/SRA process in the revised Baseline Guide. Unfortunately, the Baseline Guide itself states that SRA is performed for systems identified as direct impact through System Classification. More on that, later.<\/p>\n

SRA Concerns<\/h3>\n

The article discusses perceived concerns with SRA as a replacement for CCA. The article states, \u201cIt must be noted, nevertheless, that the SRA is not a direct replacement for the CCA<\/em>. It is more like a lens, through which aspects that impact on CQA and CPP are viewed.\u201d However, this directly contradicts the Baseline Guide, which states, \u201cIn the previous edition of this Baseline\u00ae Guide, component classification could be carried out as a subset of system classification. Component classification is no longer necessary since the System Risk Assessment\u2026identifies CDEs for the system in a more efficient manner<\/em>.\u201d<\/p>\n

I think part of this disparity comes from the view \u2013 as reflected by industry representatives referenced in the article \u2013 that SRA is merely a repackaged FMEA. Much of the concerns regarding SRA may stem from conflating the purpose and outputs of these two risk tools. SRA is intended<\/em> to be a direct replacement for CCA; SRA is not intended<\/em> to be a direct replacement for FMEA.<\/p>\n

As stated by the previously referenced Baseline Guide lead author:<\/p>\n

There also appears to be a good bit of confusion concerning the \u201cSRA\u201d.  The primary motivation in developing the rational for the SIA was to take advantage of the reality that most equipment\/systems used by pharma is of relative standard\/generic design and most likely has most (80%) of the controls\/CAs and CDEs to implement the CPPs designed in. This has a positive benefit in changing the approach of RA from \u201cgetting down into the weeds\u201d of identifying as many failure modes as possible to focusing of the CPPs being controlled by the system and having adequate SMEs assess the adequacy of the existing system \u201ccontrol strategy\u201d for a defined product. If unacceptable risk is identified additional controls can be added. The SRA suggested format does this by inclusion of identification of engineering and automation design elements (CDEs), procedural controls, and also identifies critical instruments associated with critical CPP alarms. As noted in the article FMEA is usually the tool of choice and often not the desirable tool for the scope of the RA.<\/p>\n

The purpose of FMEA is to assess the impact of various failure modes quantitatively based on severity, likelihood of occurrence, and detectability. The output of FMEA is a relative ranking of process risks according to Risk Priority Number (RPN), the purpose of which is to inform decision-making regarding the focus of risk mitigation. The purpose of SRA is to identify specific process-related risks to product quality, to identify the applied risk controls strategy, and to evaluate residual risk. The outputs of SRA are 1) a list of identified process risks to product quality, 2) the risk profile (qualitative risk assessment), and 3) the identified risk control strategy (CAs\/CDEs, procedural controls, and alarms).<\/p>\n

Understanding this purpose and these outputs addresses the misconception that SC\/SRA can lead to \u201cgaps\u201d and eliminates the confusion regarding the manner in which critical design elements and critical alarms are identified. Criticality is directly linked to product quality through identified process-related risks to product quality and the risk control strategy identified to mitigate those risks. If a component or alarm either contributes a risk to product quality or acts as a mitigation for an identified risk, then it is, by definition, critical. Further, rather than being viewed as redundant exercises, SRA can be used to inform\/as an input to process FMEA, or vice versa, depending on when each activity is performed.<\/p>\n

Indirect Impact Classification Ambiguity<\/h3>\n

The article discusses the inconsistency in C&Q strategy within the industry with respect to \u201cindirect impact\u201d systems as assessed via SLIA in the legacy C&Q process, indicating that some firms commission these systems while others commission and quality these systems \u2013 speculating that the difference may be a matter of degree of (quality) risk aversion on the part of a given firm. However, the original Baseline Guide was prescriptive on this point, stating that \u201c\u2019Indirect Impact\u2019 \u2026systems and their components are designed, installed, and commissioned according to GEP only.\u201d<\/p>\n

This industry confusion and inconsistency further demonstrates the limitation of applying QRM based on a binary understanding of risk and based on a component-focused rather than a product-focused perspective.<\/p>\n

The \u201cCAI Way\u201d<\/span><\/h2>\n

To address several of these questions, concerns, and points of confusion, CAI recommends applying QRM to the integrated C&Q process as described in the revised Baseline Guide, but with some subtle, yet key, changes in perspective:<\/p>\n

            1. Risk assessment should be performed by process, rather than by system
            2. System classification is redundant with robust risk assessment
            3. System classification should not be used as a QRM \u201cfilter\u201d or \u201cgatekeeper\u201d <\/p>\n

The problem with assessing risk by system rather than by process is three-fold: one, the process development risk assessment that initially identifies CQAs, CPPs, and the baseline risk control strategy (i.e. initial CAs) is developed prior to and independently from the eventual process Basis of Design; two, differentiating between systems is highly sensitive to definition of system boundaries; and three, assessing risk by process results in a more holistic understanding of risks that might exist outside of defined system boundaries. Further, in terms of the overall process, system distinctions are largely arbitrary. They serve primarily as a means to facilitate design and delivery of the facilities, utility systems, and equipment that comprise the process. Thus, it is advantageous to assess risk, if not for the process as a whole, at least by functional area rather than by system.<\/p>\n

Where a project has already identified specific systems as part of the Basis of Design, the risk profile and associated risk controls for those systems can be incorporated into the risk assessment to facilitate evaluation. For most scenarios, the SRA template in the Baseline Guide can serve as a useful tool.<\/p>\n

Obviously, this ideal approach must account for the current industry reality that CQAs and CPPs (and the process development risk assessment that should identify them) are often not well-documented. Mitigation strategies exist, but they are outside the scope of this blog post.<\/p>\n

When we understand the relationship between product quality (CQAs), process-related risks to product quality (e.g. CPPs), the risk control strategy (Critical Aspects), and the design definition (CDEs), it should make sense why system classification is a redundant activity in terms of the integrated C&Q process. SRA outputs include identified process risks and the design controls, procedural controls, and alarms that comprise the risk control strategy. Each of those risks and each of those risk controls can be associated with a system. Thus, any system that includes an identified risk or an identified risk control is, by definition, direct impact \u2013 and any system that does not is, by definition, not direct impact. Further, this understanding is both more robust and more explicit that the output of system classification.<\/p>\n

Therefore, since system classification is a \u201cbottom-up\u201d form of risk assessment not directly tied to product and process knowledge and is less robust than SRA, it should not be seen as a \u201cfilter\u201d or \u201clens\u201d and should not be used as a \u201cgatekeeper\u201d to determine further QRM application. That is not to say that system classification can\u2019t serve a useful purpose. In fact, SC can be incredibly useful as a project delivery tool, especially in early project phases. As stated in the Baseline Guide, \u201cProjects are commonly divided up into systems to facilitate construction management, document collation, turnover, and C&Q.\u201d Identifying direct impact systems using SC can be useful to prioritize long-lead design focus, schedule, and effort (such as hygienic piping systems vs plant piping systems) and to define contractual obligations for system testing, documentation, and handover requirements for vendors and fabricators.<\/p>\n

Arriving at the Same Conclusion<\/span><\/h2>\n

Having said all that, I want to give credit to the article and to its authors for a thoughtful, well-written discussion of the issues at hand. Ultimately, I believe that we arrive at the same conclusion on some of the more-critical points. As the article states:<\/p>\n

Modern validation is a series of activities planned, designed and executed to demonstrate that the controls you have implemented as a result of risk-based decision-making are valid and protect the patient. The SCA and SRA concentrates qualification rationale on quality, via CQA\/ CPPs with an efficient route to test planning. This supports a state of control within the pharmaceutical manufacturing space which is critical to achieving quality while still allowing scope to innovate and continually improve.<\/p>\n

As the article recognizes, product and process knowledge must ultimately drive our approach to C&Q and therefore documented CQAs and CPPs must serve as inputs to the C&Q process. The article notes that the underlying intent of SC\/SRA is to drive C&Q scope, strategy, and decision-making based on understanding the definition of product quality, process-related risks to product quality, the risk control strategy. Thus, proper application of SC\/SRA leads to understanding of Qualification as verification that installation and operation of the risk control strategy is fit for intended use to ensure that the systems perform as intended to deliver CPPs. The article correctly places C&Q in context of both ICH Q8\/Q9\/Q10 and the Process Validation lifecycle, and notes that applying this understanding results in a more robust, more focused, and streamlined C&Q process. And that is exactly the sort of \u201creshuffle\u201d that could benefit our industry, and ultimately, our patients.<\/p>\n

\n

About the Author:<\/strong><\/span><\/p>\n

\"Chip-Bennett\"Chip Bennett, PMP<\/strong><\/span><\/p>\n

Chip is a consultant and a PMI\u00ae Certified Project Management Professional (PMP) with over 20 years of experience as a validation engineer and project manager in the pharmaceutical and regulated non-pharmaceutical industries, Chip is the lead subject matter expert for commissioning, qualification, and validation program development, and is a subject matter expert in Quality Risk Management, aseptic manufacturing, cleaning validation, quality systems, and owner project management. Chip is responsible for developing and implementing QRM-based Commissioning and Qualification programs and projects, with a focus on assessing and training clients regarding transition to risk-based approaches.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Recently, under its new Insights offering, KenX published an article, The Big Reshuffle \u2013 Impact Assessments and System Criticality, discussing the implications regarding the change in approach from System Level Impact Assessment (SLIA) and Component Criticality Assessment (CCA) to System Classification (SC) and System Risk Assessment (SRA) in ISPE Baseline Guide Volume 5, Commissioning & […]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[97],"tags":[],"resource-featured-status":[],"resource-type":[819],"class_list":["post-44411","post","type-post","status-publish","format-standard","hentry","category-qualification-and-validation","resource-type-blog"],"acf":[],"featured_image_src":null,"featured_image_src_square":null,"author_info":{"display_name":"mikeviele","author_link":"https:\/\/caiready.com\/life-sciences\/blog\/author\/mikeviele\/"},"_links":{"self":[{"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/posts\/44411","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/comments?post=44411"}],"version-history":[{"count":0,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/posts\/44411\/revisions"}],"wp:attachment":[{"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/media?parent=44411"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/categories?post=44411"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/tags?post=44411"},{"taxonomy":"resource-featured-status","embeddable":true,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/resource-featured-status?post=44411"},{"taxonomy":"resource-type","embeddable":true,"href":"https:\/\/caiready.com\/life-sciences\/wp-json\/wp\/v2\/resource-type?post=44411"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}