System Development Audit of the GCDOCS Project AU1308

Table of Contents

• EXECUTIVE SUMMARY
  • Strengths
  • Areas for Improvement
  • Audit Conclusion and Opinion
  • Statement of Conformance
• INTRODUCTION
  • Audit Purpose and Objectives
  • Departmental Risk
  • Scope
  • Approach and Methodology
  • Criteria
• FINDINGS AND RECOMMENDATIONS
  • Governance
  • Project Management
  • Application Service Provider
  • Organizational Readiness
  • Classification Structure and Business Requirements
  • Legacy Systems Review
  • Technical and Functional Requirements
  • Implementation and Rollout
• APPENDIX A – AUDIT OBJECTIVES AND CRITERIA
• APPENDIX B – REFERENCES
• APPENDIX C – ACRONYMS USED

EXECUTIVE SUMMARY

The objectives of the audit were to provide reasonable assurance that the GCDOCS project was adequately managed based on project management risks related to project scope, milestones and costs, and accepted best practices for systems development; and to provide independent and timely feedback to the department on any issues that could affect the success of project delivery.

The audit was approved by the Deputy Minister, as recommended by the Departmental Audit Committee (DAC), as part of the 2013-16 Risk-Based Audit Plan.

A System Development Audit allows the audit team to maintain an on-going presence throughout the project in order to ensure that senior management is advised of findings and recommendations by way of ‘real-time’ advisory briefings. This approach maximizes the value of the audit to the ongoing project as issues are identified during system development. The audit was conducted during the period from March 2013 to September 2013.

During the course of this audit two advisory briefings were provided to management, one in May 2013 and one in September 2013, for which action plans were developed to address findings. This synopsis report provides a summary of the two advisory briefings, including key issues identified and communicated to senior management. It also presents outstanding recommendations for which management is in the process of addressing. There were a total of seven recommendations presented. Of those seven management action plans, three have already been fully implemented.

STRENGTHS

The GCDOCS Project has an established governance structure in place and sound processes for risk and issue monitoring and the synchronization of the GCDOCS team activities with CIOB. The project created a Human Resources Plan which was updated on a continuous basis and, overall, project management is consistent with the departmental requirements. In addition, a review of NRCan legacy information systems and repositories, an implementation requirement of the system, was comprehensive and took into consideration user representation across the department.

AREAS FOR IMPROVEMENT

Senior management focus will be required throughout the implementation and post-implementation periods in order to effectively manage key project activities with dependencies on outside bodies. These dependencies carry accompanying risk of ongoing revisions to project schedule and budget.

In the short term, those dependencies relate to the finalization of a Service Level Agreement with CIC, the finalization of an MOU to formalize training commitments by the Canada School of Public Service to NRCan, and to ensure full technical testing of the system is conducted prior to the implementation of the system.

Within NRCan, the creation of an overarching plan to define measures of project success and report on the expected outcomes and benefits for the project, will allow management to validate the success of the project.

AUDIT CONCLUSION AND OPINION

Overall, the audit can provide reasonable assurance that the GCDOCS project is being adequately managed by the Department.

In my opinion, opportunities exist to further mitigate risks related to stress and load testing, system change and incident management.

STATEMENT OF CONFORMANCE

In my professional judgement as Chief Audit Executive, the audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the internal Quality Assurance and Improvement Program.

Christian Asselin, CPA, CA, CMA, CFE
Chief Audit Executive

INTRODUCTION

On June 1, 2009, the Treasury Board Secretariat (TBS) Directive on Recordkeeping came into effect, requiring that all federal departments identify and protect information resources of business value. As such, all federal departments were required to implement the necessary methodologies, mechanisms and tools to safeguard and manage the life cycle of their information resources in line with the Directive.

In order to address the requirements of the TBS Directive and to meet departmental business needs, the Deputy Minister, at the recommendation of the Business Transformation Committee, approved the GCDOCS Business Case on August 22, 2012. This decision would provide the Department with its first enterprise-wide Electronic Document Records Management Solution EDRMS. The approved Business Case outlined a system launch with deployment originally scheduled from August 1, 2013 to February 28, 2014.

In order to implement GCDOCS at NRCan, the Department must work with external stakeholders. The GCDOCS project is dependent upon the services provided by the Enterprise Project Management Office, a division of PWGSC. Enterprise Program Management Office (EPMO) is responsible to establish the baseline for the design and configuration of GCDOCS. In addition, Citizenship and Immigration Canada, designated as the GCDOCS Application Service Provider (ASP) is responsible to provide application management.

As presented in the approved Project Charter, the total project costs over two years were estimated at $6,224,999, which includes pre-project costs of $204,469 and a project contingency. In addition, there are projected ongoing annual costs of $914,550.

AUDIT PURPOSE AND OBJECTIVES

The objectives of the audit were to provide reasonable assurance that the GCDOCS project was adequately managed based on project management risks related to project scope, milestones and costs and accepted best practices for systems development; and to provide independent and timely feedback to the department on any issues that could affect the success of project delivery.

DEPARTMENTAL RISK

Should the implementation of the GCDOCS project fall short of expectations, it exposes the Department to unnecessary risks, given the importance of effective Information Management towards meeting departmental objectives. Furthermore, this solution is necessary to ensure the Department complies with the requirements identified within the TB Directive.

SCOPE

The audit covered key aspects of the project between the period of February 27 to September 24, 2013.

The audit excluded coverage in the following areas:

• Supporting rationale to initiate a project
• Business case development and procurement
• Post implementation and management of the in-service solution
• Roles and Authorizations^{Footnote 1}

APPROACH AND METHODOLOGY

The approach and methodology followed in the Internal Auditing Standards for the Government of Canada, which incorporates the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. These standards require that the audit be planned and performed in such a way as to obtain reasonable assurance that audit objectives are achieved. The audit included various tests, as considered necessary, to provide such assurance.

Internal auditors performed the audit with independence and objectivity, as defined by the Internal Auditing Standards for the Government of Canada.

The audit approach and methodology utilized for this type of system initiative is described as a System Development Audit. The audit utilizes a “real time” approach to brief and report prior to the system implementation.

During System Development Audits, the audit teams maintain an on-going presence in the system development project and conduct the audit in phases that correspond to key project development phases. The audit team provided advisory briefings as the project progressed. In addition, the audit team attended the weekly meeting of the development team leaders and GCDOCS Steering Committee meetings, as observers.

During this audit the audit team reviewed information and documents as they were being produced by the project team, and used interviews to understand and confirm the project team strategy, plans and tasks before they were implemented. As part of the audit, the audit team interviewed project team members, and other key stakeholders.

The audit review included (but was not limited to):

• Key deliverables and planning documents;
• Project risk registers;
• Project documentation;
• System and planning documentation;
• Cost expense and forecasting reports;
• Detailed master schedules; and
• Executive Dashboard reporting.

For a list of audit references, see Appendix B.

Throughout this report various organizational acronyms are used. For a complete description of these acronyms see Appendix C.

CRITERIA

Audit criteria used in the audit were developed based on various best practices identified by the Information Systems Audit and Control Association (ISACA), as well as other relevant Treasury Board Secretariat (TBS) guidance. The criteria were approved by management prior to the commencement of the audit. These are included in Appendix A.

FINDINGS AND RECOMMENDATIONS

GOVERNANCE

Adequate governance provides effective oversight enabling the successful delivery of a project. Within this context, the audit found that roles, responsibilities and accountabilities for the GCDOCS project were clearly defined, communicated and implemented. Specifically, the audit found that the Project Charter and Business Plan formally defined the roles of key project authorities and oversight committees.

The Charter describes key roles such as the Associate Deputy Minister as the Executive Sponsor and the ADM-Corporate Management and Services Sector (CMSS), as the Project Sponsor. The Charter also describes the role of the two key oversight committees, the Business Transformation Committee and the NRCan GCDOCS Steering Committee.

The Business Transformation Committee (BTC) is an NRCan governance committee chaired by the Associate Deputy Minister with the mandate to transform NRCan business practices to be more effective and efficient in helping Ministers serve the public interest and in achieving the strategic outcomes stated in the NRCan Program Activity Architecture^{Footnote 2}.

The NRCan GCDOCS Steering Committee is the governance committee that oversees the GCDOCS project. This ADM level committee is chaired by the ADM-CMSS. Its mandate is to provide oversight and direction to the project to ensure effective and efficient delivery in support of the department’s mandate and in achieving a successful project outcome, as stated in the GCDOCS business case^{Footnote 3}.

Furthermore, the Charter and Business Plan also describe the roles of the Project Manager, the project management office, and other key stakeholders.

Although the audit found an effective governance structure in place, the following two issues were identified during the advisory briefings conducted during this audit:

1. A lack of a finalized Service Level Agreement (SLA) with CIC; and
2. Limited evidence of a plan to monitor and report on expected outcomes and benefits for the project.

The project plan identified the completion of an SLA between NRCan and CIC as a key deliverable. As of September 24, 2013, this had not yet been finalized. However, a written commitment from the Chief Information Officer to complete this was provided in the Management Action Plan for the audit.

A Service Level Agreement is a formal agreement between two or more departments that articulates the expectations of both parties to the agreement. It describes the service being delivered, documents service level targets and specifies the responsibilities of the service provider and recipient.

The absence of such an agreement increases the risk that issues are not resolved in a timely manner and limits the ability to measure performance of the service provider.

The GCDOCS Project Charter provides measurement criteria that can be used to assess the achievement of identified project objectives. The audit, however, found limited evidence to confirm that a plan is in place to define, measure, monitor, and report on the expected outcomes and benefits for the project.

At the time of the audit there was no cohesive and documented plan to conduct the measurement activity, or describe how metrics would be gathered to assess success. More significantly, there was no high level process identified to measure whether GCDOCS meets business requirements and to measure the usage of GCDOCS across the department. The establishment of an overarching plan is a good practice that allows management to validate the success of the project.

RECOMMENDATIONS

1. The Chief Information Officer (CIO) should confirm with Enterprise Project Management Office (EPMO) that the development of the Service Level Agreement (SLA) for the application is on track and obtain information on the proposed Terms and Conditions.
    
2. The Project Sponsor, in consultation with Chief Information Officer Branch (CIOB) should develop a plan to measure, monitor and report on the expected outcomes and benefits for the project.

MANAGEMENT ACTION PLAN

1. Discussions have been initiated with EPMO and the Application Service Provider (ASP) regarding the development of the SLA. Partners have agreed in principle that the SLA must be finalized prior to go-live (scheduled for November 4, 2013). The CIO will confirm the development schedule with EPMO and monitor progress through the Government of Canada Electronic Document Record Management Solution (GCDOCS) Program Director General (DG) Oversight Committee.
    
2. The Assistant Deputy Minister Corporate Management & Services Sector (ADM CMSS) agrees with the recommendation. The GCDOCS Project Team will work in collaboration with CIOB to develop a plan to measure, monitor and report on the project outcomes and benefits of the project for review and approval by the Project Sponsor. The plan will be completed prior to deployment (scheduled for January 20, 2014).

PROJECT MANAGEMENT

Effective project management ensures adequate monitoring and oversight over critical aspects of the project. These include risks, milestones, costs, resourcing, stakeholder involvement and issue escalation. The Treasury Board Secretariat provides guidance to departments to support the effective management of their projects. Similarly, the department has established its own project management framework, the NRCan Project Management Framework (NPMF), building upon TBS guidance, for its own projects.

The audit found that the GCDOCS project has an adequate project management methodology and approach in place. Specifically, cost management processes are established to report and monitor project costs and are compliant with the requirements of the NPMF. For example, an established process is in place for reporting project expenses and financial forecasts on a monthly basis via the Executive Dashboard. Furthermore, the GCDOCS team uses a forecasting tool to generate and forecast expenditures.

The audit also confirmed that other elements of project management were also adequate, including an established risk management process, an HR Plan and change management processes.

The audit found that processes were in place to monitor project deliverables and milestones within the GCDOCS project team, however, there were opportunities to enhance certain aspects of project tracking and reporting. For example, of the original fourteen project milestones, eleven milestones had been extended and delayed. Furthermore, the Project Management Plan (PMP), which included the approved schedule of project activities, remained in draft form seven months after project approval.

At the time of the audit, management did not have a finalized and approved Project Management Plan as it was dependent on timely delivery of the ASP environment. As such, the audit team noted in our first advisory briefing that management could have considered a contingency plan in the event the ASP was unable to deliver. Since that time, the ASP environment has been made available to NRCan, albeit delayed, allowing the project to proceed and the PMP to be finalized and approved.

Another issue related to tracking and reporting, was that there were some inconsistencies noted in the completion of weekly project team status updates by the team leads. Specifically, on some occasions, project leads did not formally prepare regular written updates to brief the Project Director.

RECOMMENDATIONS

3. The Project Director should ensure approval of the Project Management Plan in order to provide a clear/established timeline for tracking and monitoring of project deliverables and milestones.
    
4. The Project Director should ensure that a more consistent and timely tracking and reporting of project activities is performed within the project team.

MANAGEMENT ACTION PLAN

3. The Project Director will ensure completion and approval of the Project Management Plan by June 30, 2013. The Project Management Plan will include the revised scope, schedule and costs resulting from the delay in obtaining the Government of Canada Electronic Document Record Management Solution (GCDOCS) environment from the Application Service Provider (ASP).
   
   This MAP has been fully implemented as of September 24, 2013.
    
4. The Project Director has ensured that Project Team Leads update the weekly status report on time. It should be noted that team leads verbally report progress on activities at the weekly Team Leads meetings, as well as at the weekly Status meetings. Tracking of project activities is documented in the project schedule by the Project Manager based on progress reported by Team Leads.
   
   This MAP has been fully implemented as of September 24, 2013.

APPLICATION SERVICE PROVIDER

The ability for the GCDOCS project to proceed on-time requires the successful delivery of the ASP’s production environment. This environment represents the core functionality of GCDOCS and is critical to running the system. The absence of this environment prevented the use the GCDOCS application, as originally planned.

As of May 2013, during the first advisory briefing, the production environment had not yet been delivered to NRCan by the ASP and no formal commitment existed with a firm delivery date. The provision of an environment for GCDOCS by the ASP was delayed from January 7 to June 3, 2013. This resulted in an overall project schedule delay of 4 months. It should be noted that only one environment was delivered – the production environment, instead of the two or three environments originally anticipated by the NRCan GCDOCS project team.

Furthermore, a key element to ensure the ASP provides adequate levels of support to NRCan is the existence of an approved SLA. As previously identified, an SLA is a beneficial agreement between two or more departments. Such an agreement is considered a good practice, as it helps ensure that issues are resolved in a timely manner and allows the recipient an opportunity to measure performance of the service provider.

At the time of the audit, no formal Service Level Agreement had been finalized between NRCan and the ASP to ensure that adequate levels of support were available throughout implementation and post-implementation. This issue was identified and communicated to management during the first set of advisory briefings.

The lack of an approved SLA exposes the Department to additional risk, as ASP delays in resolving incidents and implementing change requests may result in a delay to the project schedule. (See Recommendation #1)

RECOMMENDATION

5. The Chief Information Officer (CIO) should formally document with Citizenship and Immigration Canada (CIC) and Enterprise Project Management Office (EPMO) the date at which the production Application Service Provider (ASP) environment will be available.

MANAGEMENT ACTION PLAN

5. At the request of the CIO, EPMO and CIC have formally confirmed that they are on track to provide Natural Resources Canada’s (NRCan’s) Government of Canada Electronic Document Record Management Solution (GCDOCS) environment by June 1, 2013.
   
   This MAP has been fully implemented as of June 3, 2013.

ORGANIZATIONAL READINESS

Organizational readiness focuses on the human component of change management. It addresses the people aspect of change, which is necessary to build and sustain commitment. In this part of the audit, focus was given to the facilitation of GCDOCS adoption through user engagement and a communications strategy.

In terms of change management and user communications, the audit found that adequate and comprehensive strategies were in place. The change management plan has a particular emphasis on stakeholder engagement, including senior management, regional management, middle managers and end users. Regarding user communications, it includes products and activities targeted to raise awareness, engage stakeholders, obtain feedback, and provide regular status updates. Moreover, the project team works in collaboration with IM group at NRCan in order to align IM Awareness activities with GCDOCS rollout.

Similarly, the training plan and schedule included a clear learning approach, such as a detailed action plan outlining key tasks, resources and timelines linked back to the learning objectives. End-user training will be performed in multiple phases, broken down by business units and with adequate time planned before implementation.

Although the audit team found the approach taken for training adequate, two issues were identified related to training materials and the User Acceptance Testing (UAT) environment.

Online training materials for the project were planned to be developed externally by the Canada School of Public Service (CSPS). The audit found that there was no finalized Memorandum of Understanding (MOU) to formalize the service delivery commitments by the CSPS to NRCan. Although the materials from CSPS were ultimately delivered on September 30, the MOU remained in draft form at the time of the audit. The absence of an MOU increases the Department’s risk that issues related to the training materials provided are not resolved in a timely manner. For example, errors or inconsistencies in the training manuals or technical issues with training modules could also impact project budget or schedule.

Regarding the UAT, as previously stated in the section related to the ASP of this report, the audit found that only one environment for GCDOCS was delivered. This approach does not represent a best practice, as it limits the ability to adequately control required changes to the system before it ‘goes live’. Although the audit team identified this as an issue, no recommendation was made as it was beyond the GCDOCS project team’s scope.

RECOMMENDATION

6. The Project Director should ensure a Memorandum of Understanding (MOU) is finalized that formalizes the service delivery commitments by Canada School of Public Service (CSPS) to Natural Resources Canada (NRCan).

MANAGEMENT ACTION PLAN

6. The Assistant Deputy Minister Corporate Management & Services Sector (ADM CMSS) agrees with the recommendation. The Government of Canada Electronic Document Record Management Solution (GCDOCS) Project Team has already provided comments on a second draft of the MOU to CSPS. It should be noted that the CSPS delivered the on-line course materials on September 30, as planned. The Project Director will ensure that a finalized MOU is approved prior to pilot testing (scheduled for October 31, 2013).

CLASSIFICATION STRUCTURE AND BUSINESS REQUIREMENTS

To facilitate complying with requirements related to the TBS Recordkeeping Directive, the GCDOCS system can leverage existing record keeping classification structures, as well as record retention and disposition schedules/authorities. This approach preserves the departmental record keeping efforts performed to date, while minimizing the volume of integration work required.

The baseline for the design and configuration of GCDOCS at NRCan is provided by the GCDOCS cluster EPMO. This baseline (called the Information Management Common Core) has been designed to manage the retention and disposition of records for common services across the government and can apply to all departments. In addition, the Recordkeeping Accountability Instrument (RKAI) summarizes the results of the RKI project at NRCan and identifies information of business and enduring value.

The audit identified that the GCDOCS project leveraged this information from various sources. These sources included the Information Management Common Core (IMCC), the Library and Archives Canada (LAC) approved NRCan Recordkeeping Accountability Instrument, existing NRCan Retention and Disposition Authorities, and NRCan Retention and Disposition schedules.

Although the GCDOCS folder structure views can be configured differently between each Sector at NRCan, a common classification structure across the Department remains ideal. The creation of such a structure however, will remain a risk that the project team will continue to monitor. The audit team confirmed that the GCDOCS project team has been tracking, monitoring and reporting this risk to senior management on a regular basis, as such no recommendation was required.

RECOMMENDATION

None

LEGACY SYSTEMS REVIEW

One of the GCDOCS implementation requirements included a review of NRCan legacy information systems and repositories, and development of a migration strategy for existing documents. This ensures that legacy information is maintained where appropriate, classified and stored in the new system. This facilitates its retrieval and disposal, in accordance with TBS requirements.

As such, a “Legacy Systems Review” was conducted by an external consultant and involved representatives across all sectors and business units. The Review provided cost estimates for the migration of legacy information and recommendations which were communicated to various stakeholder committees.

The audit found that the extent of work conducted, the sources of input, and the means of collecting the information to identify legacy systems at NRCan were adequate, comprehensive, and in consideration of user representation across all organizational units.

The audit also found that the Review formally identified costs and recommendations for migration of departmental legacy information to affected stakeholders in a comprehensive and timely manner. With respect to cost accuracy, the migration costs were at a high level and provision has been made to provide additional cost analysis as a separate project, distinct from the GCDOCS Project at a later date.

RECOMMENDATION

None

TECHNICAL AND FUNCTIONAL REQUIREMENTS

A critical success factor for a system under development is ensuring that the technical and functional requirements of the systems are adequately recorded, addressed, escalated (as required), and monitored throughout the project lifecycle. For systems used in the Canadian federal government, this generally includes elements such as Certification and Accreditation, configuration in both official languages, and conducting performance testing.

The Certification^{Footnote 4} and Accreditation^{Footnote 5}, (C&A) process confirms that mandatory security requirements established for a given IT system are being met and that the controls and safeguards work as intended; and that any residual risks have been formally accepted. The audit found that the GCDOCS project’s C&A program has an effective monitoring regime in place.

Regarding configuration in both official languages, the audit confirmed that the draft test plan includes a provision for assessing functionality in both English and French. An emerging risk with respect to the bilingualism of the system existed at the time of the audit. This issue is beyond the span of control of the NRCan GCDOCS project team; however, they have effectively identified this risk and are seeking a resolution from EPMO. In the short term, the project team has developed a mitigation approach in order to move forward with the NRCan rollout. As such, no recommendation has been made with regard to this issue.

Performance and Stress Testing

Stress testing helps project teams determine the capability of a system to perform under higher volume conditions. It involves testing beyond normal operational capacity in order to confirm the system’s sustainability during periods of heavy use.

Overall, the audit team found that while the GCDOCS project team has the capacity to perform basic performance tests such as the speed it takes to add, access or search for content in GCDOCS, they do not have the necessary tools to conduct stress testing of the system.

Concerning the responsibility of NRCan’s partners in this area, the Project Charter does not specify whether the ASP or EPMO have any formal obligation to support or stress testing for the NRCan GCDOCS project. The ASP and EPMO are responsible for liaising with Shared Services Canada, who provides the infrastructure for GCDOCS. Therefore, any testing of the system performance under heavy volume and stress would need to be conducted in collaboration with the EPMO, ASP, and Shared Services Canada.

Not testing the system’s performance through a stress test makes it difficult to predict how the system will react when NRCan users across Canada begin using GCDOCS as part of their daily responsibilities. This increases the Department’s risk that GCDOCS will be unable to meet the Department’s needs and may also impact user adoption of the tool.

RECOMMENDATION

7. The Project Sponsor should formally request that stress and load testing is conducted by the Application Service Provider (ASP)/Infrastructure Service Provider (ISP) in consultation with Natural Resources Canada (NRCan) and Enterprise Project Management Office (EPMO).

MANAGEMENT ACTION PLAN

7. The Assistant Deputy Minister Corporate Management & Services Sector (ADM CMSS) agrees with the recommendation. The Project Director working with Chief Information Officer Branch (CIOB) will develop a formal request from the Project Sponsor to be issued to Citizenship and Immigration Canada (CIC) (ASP) and Shared Services Canada (SSC) (ISP) to complete this activity (scheduled for October 31, 2013).

IMPLEMENTATION AND ROLLOUT

To ensure project success, it is critical for a project to use a strategy that the team will use for both implementation and rollout. Given that GCDOCS is a department-wide project, this strategy must encompass the needs of stakeholders across all sectors and regions of NRCan. If users find the tool difficult or time-consuming, they will store their documents elsewhere, despite TBS policy requirements. Thus, NRCan will not be able to manage those information assets, and GCDOCS would therefore fail to fulfill Department’s the business requirements.

The audit found that the project has an established implementation strategy which outlines the approach for developing and implementing the system. To address issues related to user adoption, the Project Team has planned a series of change management activities to be completed prior to and during implementation, and will provide ongoing user engagement and support.

Furthermore, the GCDOCS Migration Strategy identifies a number of information handling tools and software that will be operating simultaneously with GCDOCS for up to two years, as well as the provision of post-implementation support that will be provided from the IM group at headquarters.

From a system readiness perspective, the audit found that the project has chosen to deploy a pilot prior to implementing the system. Upon completion of the pilot, the project team plans to gather pilot results into an assessment report that will be presented to the Steering Committee and Business Transformation Committee for endorsement. A decision involving gating will be made based on the results of the Pilot.

Overall, the GCDOCS implementation and rollout is adequately planned, documented, approved, aligned with other project deliverables, and in consideration of ongoing departmental operational requirements, therefore, there are no recommendations.

RECOMMENDATION

None

APPENDIX A – AUDIT OBJECTIVES AND CRITERIA

The criteria have been developed based on various best practices identified by the Information Systems Audit and Control Association (ISACA), as well as other relevant Treasury Board Secretariat (TBS) guidance. The criteria were approved by management prior to the commencement of the audit.

The overall objectives of the audit were to provide reasonable assurance that the GCDOCS project was adequately managed based on project management risks related to project scope, milestones and costs and accepted best practices for systems development; and to provide independent and timely feedback to the department on any issues that could affect the success of project delivery.

LINES OF ENQUIRY AND CRITERIA	AUDIT SUB-CRITERIA
1. GOVERNANCE An adequate governance structure is in place over the GCDOCS project to ensure that the project is effectively defined, approved, planned, executed and monitored throughout its lifecycle. A plan is in place to define, measure, monitor and report on the expected outcomes and benefits for the project.	1.1 Prior to the project initiation, a project management governance structure is established, appropriate to the project’s size, complexity and risks. 1.2 Roles, responsibilities and accountabilities of the project sponsor, the project manager, the steering committee, project management office, other parties involved in the project and stakeholders are clearly defined, communicated and implemented. 1.3 A governance body responsible to provide oversight over the project scope, budget and schedule is present at all stages of the project. 1.4 The GCDOCS project governance structure and resources are adequately represented across all organizational units and sectors. 1.5 Adequate management support exists across all sectors and regions to assist GCDOCS towards successful delivery. 1.6 A plan is in place to define, measure, monitor and report on the expected outcomes and benefits for the project.
2. PROJECT MANAGEMENT The GCDOCS project management methodology and approach ensure adequate monitoring and oversight over critical aspects of the project (i.e. risks, milestones, costs, resourcing, stakeholder involvement, issue escalation, etc), and follows the NRCan Project Management Framework.	2.1 Cost-management processes are established to ensure that the project is completed within the approved budget. 2.2 Project deliverables and milestones are adequately monitored by project management against predefined dates in the Project Charter in order to ensure timely delivery. 2.3 The project progress is reported to key stakeholders, including deviations from established key project performance criteria. 2.4 Risk management processes are in place to identify, analyze and respond to project risks regularly. 2.5 A comprehensive Human Resources Plan is created and maintained throughout the duration of the project, and which includes details regarding how resources will be acquired and timelines for resources. 2.6 Change requests for requirements that are not within the scope of the project are logged in a Change Log, completed, and approved through the governance model. 2.7 The GCDOCS project is consistent with the NRCan Project Management Framework, (NPMO) and goes through a gating process for management approvals at the end of each phase of the project (i.e. initiation, planning, design, testing, and implementation).
3. APPLICATION SERVICE PROVIDER The Application Service Provider is managed effectively to ensure deliverables are achieved and that adequate levels of support are available throughout implementation and post-implementation. (It has been assumed that the ASP is also responsible to liaise with TBS-EPMO and SSC to ensure that services provided by these organizations to support GCDOCS are adequate.)	3.1 Appropriate measures are taken in order to ensure that delays in the identification and on-boarding of the NRCan GCDOCS Application Service Provider do not affect the overall project delivery. 3.2 An alternative solution for the project is available in the occurrence that the ASP cannot deliver the technical aspects / requirements as expected prior to the initiation of the project. 3.3 A Service Level Agreement (SLA) between NRCan and the Application Service Provider is established prior to the system implementation in order to ensure that adequate levels of support are available throughout implementation and post-implementation.
4. ORGANIZATIONAL READINESS The project has a plan to facilitate the adoption of GCDOCS through user engagement and through the use of a communications strategy – including linkages to training and user acceptance testing.	4.1 A Change Management Strategy and Plan is in place and is adequate, comprehensive, and is aligned with the Implementation Strategy and Plan. 4.2 A communications plan is established to provide stakeholders and project leadership with appropriate information to ensure that the project meets functionality, budgetary and timeline goals, and the plan informs stakeholders and management of the progress of the roll-out in timely manner. 4.3 An adequate a comprehensive training plan is in place to assist with knowledge transfer to end-user and management prior to implementation. 4.4 Training materials are complete, comprehensive, illustrate both new system functionality and new business processes, adapted to reflect the NRCan look and feel, and are drafted, revised, approved, and are available in both official languages. 4.5 Training is planned and scheduled in a timely manner to users prior to implementation. 4.6 A comprehensive User Acceptance Test Plan is developed in accordance to project and enterprise standards and requirements and testing is documented and reviewed. 4.7 The project plan provides adequate time for testing and remediation based upon test results. 4.8 The User Acceptance Testing is carried out in an environment representative of the future operational environment. 4.9 The test scripts and volumes are adequate to ensure accurate, effective and complete results.
5. CLASSIFICATION STRUCTURE & BUSINESS REQUIREMENTS The system is designed through leverage of existing classification structures and business disposition rules, as identified in the Recordkeeping Initiative.	5.1 dequate processes are in place to design/configure the system through leverage of existing classification structures and business disposition rules, as identified in the Recordkeeping Initiative. 5.2 Appropriate mitigating controls are in place for system configuration in the occurrence that the Retention Schedules and Disposition Letter are not approved by LAC in a timely manner before implementation.
6. LEGACY SYSTEMS REVIEW Costs and recommendations for migration of departmental legacy information are formally identified and communicated to affected stakeholders in an accurate, comprehensive and timely manner.	6.1 The Legacy Systems Review is adequate, comprehensive, and is in consideration of user representation across all organizational units. 6.2 The Legacy Systems Review provides detailed and realistic cost estimates for the migration of legacy information post-implementation, and the costs are communicated to affected stakeholder management in a timely manner.
7. TECHNICAL & FUNCTIONAL REQUIREMENTS Technical and functionality requirements are adequately recorded, documented, addressed, escalated as required, and monitored throughout the project lifecycle.	7.1 GCDOCS is certified and accredited prior to implementation into the production environment, in accordance to the NRCan Directive on IT System Certification and Accreditation. 7.2 GCDOCS roles and authorizations are designed, documented and formally approved. 7.3 The system is configured and tested in both official languages, and any potential issues are identified and communicated to the ASP/vendor in a timely manner prior to implementation. 7.4 A comprehensive System Performance and Stress Test Plan is developed, testing is carried out in an environment representative of the future operational environment, and test scripts and volumes are adequate to ensure accurate, effective and complete results.
8. IMPLEMENTATION & ROLLOUT GCDOCS Implementation and rollout is adequately planned, documented, approved, aligned with other project deliverables, and in consideration of ongoing departmental operational requirements.	8.1 Implementation strategy and plan are developed and is adequate, comprehensive, and aligned with the Change Management Strategy and plan, as well as with ongoing departmental operational requirements. 8.2 A readiness assessment is part of the implementation plan to ensure that the system is ready for the implementation phase. 8.3 A backout plan has been prepared with appropriate review, approval and decision points to initiate the plan.

APPENDIX B – REFERENCES

Guide to the Audit of Systems Under Development, Treasury Board of Canada, Office of the Comptroller General	http://www.tbs-sct.gc.ca/pubs_pol/dcgpubs/TB_H4/systems-systemes01-eng.asp
ISACA Systems Development and Project Management Audit/Assurance Program	http://www.isaca.org/Knowledge-Center/Research /ResearchDeliverables/Pages/Systems-Development-and-Project- Management-Audit-Assurance-Program.aspx
NRCan Project Management Framework	https://sp-projects.nrcan.gc.ca/pmo/npmf.aspx
“NRCan Directive on IT System Certification and Accreditation”	http://wwwint.nrcan.gc.ca/ci/ems/2/mc/i-mcpo-itsca-e.htm
Policy on Information Management, Policy on Government Security (PGS), and Policy on Management of Information Technology Security (MITS), Treasury Board of Canada Secretariat	http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12328&section=text http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?section=text&id=12742
ISACA COBIT 4.0, IT Services Management	http://www.isaca.org/Knowledge-Center/COBIT/Pages/Overview.aspx
Computer Objectives for Information Technology (COBIT) Audit Guidelines, 2ndEdition, COBIT Steering Committee and the Information Systems audit and Control Foundation, April 1988	http://www.isaca.org/Knowledge-Center/Standards/Documents /G2-Audit-Evidence-Requirement-12Feb08.pdf
“NRCan Directive on IT System Development”	http://wiki.nrcan.gc.ca/index.php/NRCan_Directive_on_IT_System _Development_-_FAQ
The Office of Government Commerce (OGC) Gateway Process – A manager’s checklist	http://www.scotland.gov.uk/Topics/Government /ProgrammeProjectDelivery/Template/IATemplates/OGCchecklist

APPENDIX C – ACRONYMS USED