Computerized systems are widely used in the pharmaceutical industry. An important step in implementing them in a company is to make sure that the solutions:

• Were designed and adapted to their intended use.
• Were in compliance with current legal requirements (GMP – Annex 11).

Before selecting a computerized system supplier, the user should evaluate the parameters of the system proposed by the provider and its ability to ensure quality. Particularly important is the evaluation of how the supplier develops the software, the process of testing the system and an effectively functioning quality scheme during software production.

Typically, this type of vendor assessment is carried out through an audit, which, depending on the risk, complexity and innovation of the system, is carried out by the user.

The selection of a product or service provider should be based on diagnostics of its competence and reliability, and the need for an audit should be the result of a risk analysis, according to the GMP guidelines – Annex 11.

During the inspection, the user should evaluate the supplier and obtain properly formalized evidence of:

• The supplier’s acceptable Quality System (e.g., Quality Management System (QMS) (ISO 9001)).
• The supplier’s technical capabilities.
• The supplier’s use of good practices so that the information obtained from the supplier is accurate and useful for achieving the verification objective.

When evaluating a supplier and conducting an audit, there are different types of evaluation:

• Basic assessment on the basis of available information (e.g.: analysis of past experience and cooperation with the supplier, the supplier’s quality system and its status, the offering of commonly available products, market reputation).
• Postal audit (using a questionnaire).
• Physical audit on site.

The decision to choose the above form of assessment should be based on a risk assessment determining the most appropriate form of supplier assessment.

For postal audit

A postal audit (using a questionnaire) can be used mainly in:

• When determining whether a supplier deserves to still be considered for an offer during the bidding process.
• As a “pre-audit” that provides information to the team and focuses on critical areas, significantly reducing on-site audit time.
• As a review of company information (product history, product versioning, quality book and key processes, as well as system life cycle and support activities confirmed by relevant documents).
• As a follow-up audit after a physical (on-site) audit to verify corrective actions.
• For regular evaluation of the supplier.
• As a check of other system manufacturing sites where the same Quality Management System is implemented.
• As a way to identify weaknesses in business processes that may indicate the need for a physical audit at the system manufacturing site.

A supplier postal audit is conducted based on a system-approved questionnaire. This is a standard list of questions that, depending on the needs, can cover both general and specific issues for a particular process or product. This type of form is sent by mail to the supplier with a request to answer the questions it contains and complete the required information.

The questionnaire may include information such as

• Company information, including all product locations.
• Organization, roles and responsibilities, staff training and experience.
• Key products and/or service history and development plans.
• Confirmation of QMS implementation at the company level and for product-related processes.
• Confirmation of product/project management process.
• Confirmation of procedures for software lifecycle development processes.
• Description of service delivery processes.
• Description of conducting effective user training.
• Procedures and agreements for product support/maintenance.
• Rules on security.
• Information on dealing with subcontractors.

The questionnaire should be completed and returned. If discrepancies are found, corrective and preventive actions should be developed with feedback to the supplier who completed the form.

Physical on-site audit

An on-site audit of a supplier’s computerized system is not very different from methodologies conducted in-house for other suppliers and for GxP-regulated companies.

1. Defining the scope and purpose. For audits of computerized systems, the most common scope is to address: product improvement and procurement, software development, equipment manufacturing, support services or software integration, supplier’s ability to produce a quality product or service.
   
   
2. Development of a schedule
   This involves defining several factors: the tasks and responsibilities of the auditee, describing the methods and tools for conducting the audit (document analysis, interviews with employees, observation of processes), identifying the areas to be evaluated, reviewing documentation, and preparing resources.

3. Appointment of the audit team
   This is the selection of an appropriate team and the appointment of a lead auditor who will be responsible for planning, conducting the audit and preparing the report. The team may also include a technical expert, such as an IT specialist, who has detailed knowledge of the area in which it operates. Such support can be crucial in situations where the auditor lacks expertise in a particular area, such as IT.

An important point is to notify the audited side in advance of the place and time of the activities. These topics should be agreed upon to avoid misunderstandings and time conflicts on both sides. The lead auditor presents the action plan to the audited side.

This plan should include:

• Audit number.
• Purpose and Scope.
• Criteria/reference documents.
• The name of the audited entity.
• Date and location of the activities.
• Time schedule.
• Composition of the audit team.
• Documents required to be submitted during the meeting.

This plan must be agreed with people who are responsible for the audited area.
This avoids unnecessary repetition while maintaining clarity in the text.

Supplier audit during system implementation

• Presentation of participants.
• Overview of the objectives and scope of the audit.
• Identification of key people required during the audit from the auditee.
• Detailed plan and schedule of activities (including breaks and follow-up meetings)
• Organizational arrangements (including confirmation of confidentiality during collection of audit evidence).
• Discussion of the reporting method (including classification of nonconformities).

2. Conducting an audit

This is the most important part of the process of evaluating a vendor of computerized systems. In this stage, the team verifies the correctness of the supplier’s activities according to the agreed terms and plan.

The auditors use a checklist, prepared before the physical visit. However, it is important to remember that the questions are only meant to support the collection of evidence, not to be rigidly followed, as this can lead to missing the main objectives of the assessment and exceeding the planned time.

During the site visit, it is important for the auditor to be flexible, adapting the approach to changing circumstances. A competent specialist is able to change work techniques according to needs and new information. Effective time management and the ability to identify relevant aspects during evidence collection are also key. They are gathered through observation, interviews with staff and verification of documentation.
The auditor should take clear notes, including references to documents and locations.

When auditing a supplier of computerized systems, it is often verified:

• Whether the supplier has established an adequate Quality Management System to ensure control over the development and maintenance of the system/application.
• Whether the supplier’s Quality Management System (QMS) includes a documented set of procedures and instructions.
• whether the activities carried out under the QMS are performed by competent and properly trained personnel.
• Whether the processes for delivering, supporting the product / application or services comply with documented procedures and standards.
• Whether the supplier manages the life cycle of systems properly and systematically.
• Whether the supplier takes responsibility (with clear separation of authority between quality assurance and, for example, product development, support, finance and marketing).
• Whether the procedures in place describe policies for dealing with :
  • software management, control and release,
  • change control during development,
  • configuration management of individual IT resources,
  • audit trail,
  • training of vendor personnel,
  • records management,
  • backup and restore,
  • risk management.
• Whether the QMS is based on the life cycle concept of computerized system development.
• Whether internal audits and cyclic (regular) reviews of the QMS are conducted as part of the QMS.
• Whether the supplier uses a continuous improvement approach within the QMS,
• Whether the supplier has implemented and properly applies data processing and storage activities in its enterprise (whether standards of data integrity, availability and confidentiality are maintained).
• Whether it has put in place procedures related to research and development (R&D) of the system/software and its module/software integration/computer network (technical/engineering documentation on system design and construction).
• Whether it has described policies for guidelines for: software code production, source code review, and security analysis against unauthorized persons in accordance with the latest good engineering practices.
• Whether there are documented application / system testing activities prior to release for marketing / sale (whether performance, load and fault tolerance tests are conducted in addition to functionality tests).
• Whether it has put in place procedures describing the release / release of software for use / marketing.
• Whether there are documented system elements for hardware requirements, specifications for the software itself and systems functionality.
• Whether there are documented maintenance plans for systems / applications.
• Whether there are procedures defining the company’s problem management and policies for receiving and handling complaints / requests.
• Whether there is maintenance and post-implementation system maintenance services are carried out (whether the company ensures its activities in accordance with SLA – Service Level Agreements).
• Whether the agreements (SLA) contain basic information: response time to failures, availability of systems, downtime, technical support and a detailed method of contact and customer service.
• Whether the software is legal and properly licensed.
• Whether there are documented procedures and processes related to information security (for example, access management, incident management, business continuity procedures, disaster recovery procedures and procedures for dealing with such disasters).

3. Closing meeting

During this meeting, the lead auditor (after consulting with the team) presents to the supplier (auditee) the observations that were noted during the assessment. It is important not to focus solely on problematic issues, but also to include positive aspects of the process.

This meeting is also intended to remind the purpose and scope of the assessment, present the findings with their prioritization, and inform about the deadline for sending the final report.

The auditor should thank them for the opportunity to conduct the assessment, for their cooperation and for providing the necessary information.
Identified nonconformities and observations should be classified according to the criteria contained in the audit procedure or other relevant document, in accordance with company policy. It is important that all noted nonconformities are discussed and recorded before the final meeting. This will ensure that only those issues that were clearly identified and agreed upon with the audited team will be included in the report.

Audit Report

The audit report, a key document after the assessment, should be based on facts and provide a detailed summary of the results, addressing the evidence gathered. Its main purpose is to support the decision-making process for signing a contract with a supplier.

This document is prepared by the lead auditor, who may delegate its preparation to members of the audit team, retaining oversight of the report development process.

The report contains accurate information summarizing the audit of the supplier of computerized systems carried out.

The following is a list of items that should be included in the document:

• Number and year of the report.
• Name and address of the audited site.
• The purpose and scope of the audit, including the date of the audit.
• Reference documents that provide a reference for the audited team.
• A summary of the documents reviewed during the audit.
• A description of the audit.
• Observations, observations and any identified non-conformities.
• Suggestions for corrective actions to be taken to improve compliance and efficiency (if applicable).
• Confidentiality clause, in case the report contains confidential information.
• Names and positions of audit team members.
• Names and positions of persons representing the audited party.
• The date of the report and the signatures of the auditors.
• It should be noted that the audit report should be subject to agreement by both parties. This allows all involved to jointly understand the conclusions of the audit and take appropriate post-audit actions.

Corrective and Preventive Actions

Corrective and Preventive Actions (CAPA) resulting from an audit report are intended to clarify, correct and prevent deviations observed supplier of computerized systems.

This process includes identifying the problem, determining corrective actions, analysing the root cause, implementing preventive actions and documenting them. It is usually carried out by a CAPA team that includes the system owner, the quality department and, if necessary, a subject matter expert. These actions must be documented and supervised according to established responsibilities, as well as company procedures, instructions or standards. An audit is ineffective if it does not lead to corrective or preventive actions, which are central to the auditing process.

Note that the supplier’s audit procedure and audit reports are part of the system’s validation documentation – so their importance and documentation is often emphasized.

The selection of the right supplier has a tremendous impact on further work related to the proper implementation of the computerized system. Suppliers play an important supporting role: as part of the contract, suppliers take responsibility for providing key documentation, provide direct technical support as external SME’s, and perform initial system/application testing.

Vendor support is very important not only at the design, implementation stage but also during servicing, achieving and maintaining the system in a validated state throughout the system/application life cycle.

Stable systems designed, developed and implemented in accordance with good engineering practices and regulatory requirements (such as GMP – Annex 11) by suppliers are the backbone in the process of manufacturing and controlling products in GxP companies. In view of the above, it is extremely important to take a serious approach to meticulously preparing and effectively conducting audits of suppliers of computerized systems.

Kamil Melson
Professional Validation Specialist