JOINT AUDIT AND EVALUATION SUMMARY PLAN 2018-2021
- Risk-Based Audit Process
- Evaluation Process
- Planning Results
- Joint Audit & Evaluation Projects
- Audit of Public Accounts 2017-18
- Follow-up on Previous Audit & Evaluation Recommendations
- Continuous Auditing of Core Controls
- Central Agency Audit Projects for 2018-19 – 2020-21
- Advisory Projects for FYs 2018-21
On December 4, 2017, the Audit Branch and the Strategic Evaluation Division were amalgamated into the Audit and Evaluation Branch (AEB). This was done to capitalize on the natural strengths of each function, leveraging the synergies between them while maintaining a high degree of rigour and professionalism.
Development of the Risk-Based Audit Plan and Evaluation Plan provided an opportunity for joint planning within this new Branch. Joint planning has resulted in the development of a joint plan, even though the Audit and Evaluation plans each have different drivers and information requirements, as per their respective TB policies- i.e. Policy on Internal Audit and Policy on Results. It has also successfully resulted in a more comprehensive review of the Audit and Evaluation requirements that exist within Departmental sectors and their programs allowing for joint engagements, where possible, through improved planning, including consultations with senior management on the full suite of planned audits and evaluations.
The audit and evaluation plans outline the organizational priorities, risks and the tools used to identify those risks, and the opportunities to mitigate them. These priorities and risks help guide the selection of audit and evaluation projects for the Department during the period covered by these plans.
For the audit risk-based planning process, NRCan’s universe of auditable entities includes various programs, activities, processes, policies, and initiatives, which collectively contribute to the achievement of the Department’s strategic objectives. Priorities and Core Responsibilities, such as those identified in the NRCan Minister’s mandate letter, NRCan’s Departmental Results Framework (DRF), and the Department’s inventory of external legislated services, help to assess the completeness of the audit universe, which includes 24 groupings of auditable entities.
For Evaluation, planning is guided by the requirements of the TB Policy on Results introduced in 2016. Priority is given to a) ongoing programs of grants and contributions with average annual expenditures of $5M or more or b) programs with evaluation commitments made as part of Treasury Board submissions. Evaluations of other programs are determined by risk, with the overarching goal being that all departmental programs should be evaluated every five years unless they are low risk. A key factor influencing this year’s planning is the government-wide transition from the Program Alignment Architecture to the DRF.
Under this new structure, the Departmental Audit Committee (DAC) will continue to review all audit reports from AEB prior to their approval by the Deputy Minister. Similarly, all evaluation reports will also continue to be reviewed by the Performance Measurement, Evaluation and Experimentation Committee (PMEEC) prior to Deputy Minister approval. Future joint A&E reports, combining features of both audits and evaluations, will be reviewed by both the DAC and PMEEC.
In developing the plans, AEB considered four priorities as critical to the Department meeting its Core Responsibilities and supporting the Government of Canada’s priorities:
- Working with partners to restore public confidence in developing and bringing natural resources to market;
- Innovating to optimize resource potential and to support Canada’s transition to a low-carbon economy;
- Leveraging NRCan’s expertise to maintain public safety and security; and,
- Enabling NRCan to be a high-performing organization that is innovative and collaborative in delivering on its mandate.
Natural Resources Canada’s (NRCan) Joint Audit and Evaluation (A&E) Plan, also referred to as the “Plan”, is prepared by the Audit and Evaluation Branch (AEB). It contains details on the role of internal audit (IA), evaluation, the AEB’s planning methodology for both functions, and the planned projects for the next three year cycle: 2018-21. Under the new AEB structure, which came into effect on December 4, 2017, the Departmental Audit Committee (DAC) will continue to review all audit reports and the Performance Measurement, Evaluation and Experimentation Committee (PMEEC) will continue to do the same for all evaluation reports from AEB, prior to their approval by the Deputy Minister (DM).
The Plan is developed in accordance with the requirements of the Treasury Board of Canada (TB) Policy on Internal Audit and the TB Policy on Results, along with related directives, guidelines and as it specifically pertains to IA, the Institute of Internal Auditors’ (IIA) International Standards for the Professional Practice of Internal Auditing.
Risk-Based Audit Process
Each year, NRCan’s Chief Audit and Evaluation Executive (CAEE) is required to prepare a risk-based audit plan, which sets out the priorities of the internal audit activity that are consistent with the organization’s goals and priorities. The audit planning process ensures that all internal audit activities are relevant, timely, and strategically aligned with NRCan’s Corporate Risk Profile (CRP) to support the achievement of the Department’s strategic objectives. The input from NRCan’s DAC, along with NRCan’s senior management, is sought and taken under advisement in setting internal audit activity priorities.
The starting point for the risk-based planning process is the identification of the audit universe. This audit universe document was developed by the AEB and is updated annually to reflect the Department’s most current priorities. The audit universe characterizes the array of possible audit activities and is made up of auditable entities identified as relevant to NRCan and its operating context. Auditable entities commonly include programs, processes, policies, management activities and control systems, along with departmental and government-wide initiatives, which collectively contribute to the achievement of NRCan’s strategic objectives. NRCan’s audit universe is made up of 24 groupings of auditable entities.
All auditable entities are subjected to a risk assessment and risk ranking exercise to select audit projects in order of priority. Criteria used for selecting audit projects include- past audit coverage and results; materiality; significance to management; level of risk; auditability; audit projects carried forward from the previous year’s Plan; organizational priorities; high priority areas identified by central agencies, such as the Office of the Comptroller General (OCG) and Office of the Auditor General (OAG), among others; and, legislated or other mandated obligations.
Prioritization of the audit universe is a two-step process. The first step includes management consultation, review, and consideration of the following available documentation: NRCan minister’s mandate letter; the former Program Alignment Architecture (PAA) and its replacement the Departmental Results Framework (DRF), departmental risk information, including NRCan’s CRP; the latest Management Accountability Framework (MAF) assessment; recent departmental-wide assessments of IT and fraud risks, which lead to the identification of audits as part of the AEB’s continuous audit framework; business planning documentation; NRCan’s Departmental Plan (DP); Government priorities; and previous audit results (both internal and external), along with the Department’s most recent financial information and statements.
Other factors are also considered, such as collaboration with NRCan’s Evaluation function to identify opportunities to carryout joint work/efforts in order to improve efficiency and minimize duplication of efforts.
The second step to prioritize the audit universe involves consideration of several factors, including significance to departmental core responsibilities and operational objectives; senior management requests and priorities; the DAC’s advice and recommendations; external audit activities and planned evaluations; readiness of the entity for audit activities; and availability of internal resources to complete the audit on time. Following this step, professional judgement is still required to risk-assess and rank the auditable entities. This is performed through collaborative discussions with NRCan senior management and the DAC, where emphasis is placed on projects planned for 2018-19 (the first year of the three-year plan), given that future projects are reassessed annually. Government and departmental priorities are also validated with senior management and the DAC to ensure planned audits align with higher priority areas. In addition, preliminary audit objectives are developed for each audit selected for the Plan. The final plan is then reviewed by the DAC and approved by the DM.
The following diagram highlights the four key phases used in the selection process for the development of a robust risk-based audit plan.
The Four Key Phases Used in the Selection Process for the Development of a Robust Risk-Based Audit Plan
This figure highlights the four key phases used in the selection process for the development of a robust Risk-based audit plan. It covers the starting point of the selection process that determines potential NRCan auditable entities covering a 3 year period to its final recommendation. The first large block represents the potential range of auditable components, which include departmental programs, activities, processes, structures, and initiatives that collectively contribute to the achievement of the Department’s core responsibilities. It is called the audit universe. The Audit and Evaluation Branch uses the Departmental Results Framework (DRF) as well as NRCan's inventory of external legislated services to ensure the audit universe identified is complete. There are approximately 24 groupings of auditable entities based on the DRF and NRCan’s sectors.
The next stage is to prioritize the audit universe based on a risk-based assessment. This is a two-step process that involves a preliminary and final prioritization based on a number of factors such as likelihood of risk and impact. The final 2 steps are to rank the priority of the proposed projects and to recommend them for approval in the 3 year audit plan (as in the final 2 large blocks).
The CAEE also oversees the development of an evaluation plan that outlines the policy and operational context for evaluation services delivered at NRCan. The 2017-18 PAA was replaced with the 2018-19 DRF and associated Program Inventory, which consists of all possible programs that might be considered for evaluation over the course of the next three years. In selecting programs to be evaluated, priority was first given to all mandatory evaluations, including:
- all ongoing programs of grants and contributions with average annual expenditures of $5M or more;Footnote 1 and
- evaluation commitments made as part of Treasury Board submissions.
Once all mandatory evaluations are accommodated within the evaluation plan, a risk assessment is undertaken of the remaining programs within the evaluation universe. The risk assessment exercise involves joint consultations with the IA function of the AEB to solicit senior management feedback on program risks and priorities for evaluation, as well as a review of departmental documentation (e.g., NRCan’s CRP, business planning documentation, NRCan’s Report on Plans and Priorities, departmental audit plans, 2017-18 departmental evaluation plan). Risks are assessed according to various program characteristics (e.g., upcoming renewal, materiality, visibility), management practices (e.g., structures, delivery complexity, performance measurement), past research findings, and upcoming audits. Evaluations are then planned for some of the remaining units of the evaluation universe according to the availability of evaluation resources and the entity’s relative risk ranking.
The final plan also considers feedback and/or requests from the Treasury Board Secretariat (TBS) and makes accommodation for non-project related advisory and other activities to support the department in responding to broader federal initiatives related to experimentation, cross-cutting program reviews (e.g., Innovation, Science), or special studies (e.g., prospective evaluations to support program design).
For operational reasons, this Plan has been prepared using a 3-year planning horizon. Given that the TB Policy on Results requires departments to submit a 5-year evaluation plan, an evaluation schedule (see appendix) has been included identifying the complete list of planned evaluations. These evaluations will cover approximately 88% of NRCan’s total planned program spending, which are subject to change during the actual scoping of the evaluations and exclude amounts for Internal Services and Statutory Payments.
The following table summarizes the number of new audits, evaluations, and joint A&E projects selected for each year, along with the number of OCG-led horizontal audits. In total, 42 of the highest priority engagements are planned for the next three years. Projects that have been carried forward from FY 2017-18 can be found in the respective detailed plans.
|New AEB Projects||2018-19||2019-20||2020-21|
|Joint A&E Projects (see Table 2)||3||2||3|
|Horizontal Audits (OCG-led)||1||1||1|
|Horizontal Evaluations (OCG-led)||3||-||1|
Joint Audit & Evaluation Projects
The IA and Evaluation functions have held joint consultations with senior management and staff to ensure the most effective and efficient planning process, allowing the AEB to provide coordinated advice. As a result, this year’s update to the Plan includes eight potential future joint projects where collaboration is possible. Table 2 provides a listing of joint A&E projects for FYs 2018-19 to 2020-21. It should be noted that collaborative efforts will range from conducting joint interviews, the collection and sharing of information, to conducting hybrid audit and evaluation engagements.
||4. MC & PM Framework of the Green Infrastructure Initiative (IETS, ES, CFS)||6. NRCan Environment
Studies & Assessment
Process (OCS – All
||5. Transition to the
(SPRS – All Sectors)
|7. MC & PM Framework of the Clean Growth in Natural Resources Sector Program
|8. MC & PM Framework of
Electricity – Emerging
Renewables Program (ES)
Audit of Public Accounts 2017-18
The AEB has received confirmation that the OAG will not perform detailed audit testing of NRCan’s Offshore Revenue activities as part of their annual public accounts audit program. The IA function will continue to perform audit testing on the controls related to this activity in FY 2018-19, given the visibility and importance to the provinces. The audit of Offshore Revenue will be carried out as one element of AEB’s annual continuous audit projects, which is described later on in this Plan.
Follow-up on Previous Audit & Evaluation Recommendations
As per the TB Policy on Internal Audit and International Standards for the Professional Practice of Internal Auditing and the new TB Policy on Results, the CAEE “must establish a follow-up process for each audit and evaluation recommendation and ensure that management actions have been effectively implemented or that senior management has accepted the risk of not taking action.”
The follow-up process at NRCan is a two-phase process, which begins with a management self-assessment of the level of implementation for each recommendation, which is accompanied with a Management Action Plan (MAP). MAPs receive senior management approval and assign detailed tasks to responsible parties that address audit recommendations in a timely manner. In the fall of each FY, the AEB reports on the status of the implementation of recommendations based on management’s self-assessment. Each spring, as part of the second phase, the AEB performs a validation of the full implementation of recommendations, confirming that the MAPs are indeed fully implemented. The validation approach includes the following procedures: conducting interviews; reviewing supporting evidence; and performing analysis and testing based on risk. Once completed, a Follow-Up Report is produced and approved by the DM. Once approved, the portion that is related to the IA function is sent to the OCG, where applicable.
Continuous Auditing of Core Controls
The AEB will continue to undertake assurance-based continuous auditing at NRCan to proactively identify potential control issues and report annually on various processes. This work will be performed in accordance with the IIA Standards (i.e. provide reasonable assurance). The work carried out will address key risks associated with significant departmental expenses that have been identified in part, as a result of NRCan’s updated Fraud Risk Assessment’s MAPs.
The 3 areas selected for continuous audit in 2018-19 are:
- Offshore Revenues; and
- Grants and Contributions Processes.
NRCan’s annual report on continuous audit activities will be completed for the DAC’s fall 2018 meeting for continuous audits completed for FY 2017-18.
Tables 4 and 5, below, present the new “highest priority” audits, evaluations and joint A&E projects for fiscal years 2018-19, 2019-20 and 2020-21.
|1. NRCan’s Emergency
(CMSS – All Sectors)
|7. Management of
(CMSS – All Science Sectors)
|15. NRCan’s Experimentation and
(SPRS – All Sectors)
|2. NRCan’s Strategic &
Operational Planning Process
(SPRS, CMSS – All Sectors)
|8. Lower Churchill Falls Loan
|16. Capital Asset Management
(CMSS - All Sectors)
|3. Federal Geospatial Platform
(SPRS – All Sectors)
|9. Clean Technology Stream
|17. Management of Fed-Prov.
Offshore Agreements (ES)
|4. Management of Aging IT
Systems (CMSS – All Sectors)
|10. Open Government
(CMSS – All Sectors)
|18. Implementation of NRCan’s IT
Strategy (CMSS - All Sectors)
|5. Canada Land Surveys Program
|11. Management of NRCan’s
|19. Transition to Social Media &
Information Sharing (CPS)
|6. Horizontal Audit of Project
Management (CMSS –
|12. Horizontal Audit of
Information for Decision
Making (CMSS – OCG-led)
|20. TBD (CMSS – OCG-led)|
|13. Aligning NRCan’s Culture to
its Mandate (SPRS –
|14. Classification (CMSS)|
|*Annual Continuous Audit Projects (CMSS)*|
|1. Geoscience for Sustainable
Development and Natural
Geoscience 5 (LMS)
|6. Forest Sector Competitiveness:
Forest Sector Products Market
Access and Diversification
|11. Lower Carbon Transportation: Deployment of Electric Vehicles and Alternative Transportation Fuels (ES/IETS)|
|2. Electricity Resources:
Renewable Energy (ES)
|7. Climate Change Adaptation
|12. Electricity Resources: Smart Grid (ES/IETS)|
|3. Energy Innovation Program:
World Class Tanker Safety
System-Phase II (IETS)
|8. Polar Continental Shelf (LMS)||13. Energy Innovation Program: Clean Energy Science and Technology (IETS)|
|4. Explosives Safety and Security:
Single Window Initiative (LMS)
|9. Clean Growth in Natural
|14. Climate Change Adaptation and Clean Growth (OGD-led)|
|5. Clean Growth and Climate:
International Cooperation (ES)
|10. Geoscience to Keep Canada
Safe: Canada Hazards
Information Service (LMS)
Central Agency Audit Projects for 2018-19 – 2020-21
The Department is also subject to audits by other assurance providers. Table 6 provides a listing of known external audit projects planned for FYs 2018-19 to 2020-21, with the expected tabling dates.
Office of the Comptroller General
|Horizontal Audit of Project Management||Winter 2019|
|Horizontal Audit of Information for Decision Making||Spring 2019|
Office of the Auditor General
|Audit of Building and Implementing Phoenix||Spring 2018|
|Coordination of the 2017-18 OAG Audit of Public Accounts||Fall 2018|
|Audit of Financial Assurances (2012)||Spring 2019|
Commissioner of the Environment and Sustainable Development
|Audit of Oil Spills in Marine Waters||Fall 2018|
|Audit of Departmental Progress in Implementing Sustainable Development Strategies||Fall 2018|
|Audit of Corporate Social Responsibility in Natural Resources Sector||Fall 2019|
|Audit of Mitigation of Severe Weather Events / Search and Rescue||Fall 2019|
|Public Service Commission||System-Wide Staffing Audit||Fall 2018|
Advisory Projects for FYs 2018-21
As an adjunct to the assurance role, the TB Policy on Internal Audit indicates that as part of the departmental Plan, the CAEE can consider “other oversight engagements, including, where the necessary expertise and capacity are in place, the option to provide consulting services to the organization.” Provided this, the AEB estimates 2-3 advisory projects per FY, which will be based on senior management needs and the availability of AEB’s resources. It is important to note that the AEB will always continue to maintain its independence, when providing both advisory and assurance types of services. The inclusion of advisory projects into NRCan’s Plan helps to ensure additional value is provided to senior management to complement our regular audit activities.
|Departmental Results Framework Program||Planned Evaluations (next 5 years)||Planned Approval (FY)||2018-19 Estimated Program Spending covered by Evaluation ($)||2018-19 Total Planned Program Spending ($)||Evaluation Driver/Rationale|
|Geological Knowledge for Canada’s Onshore and Offshore Land||Geo-Mapping for Energy and Minerals||2018-19||35,028,007||36,748,755||TB Commitment|
|Canada’s Extended Continental Shelf (Horizontal Evaluation led by GAC)||2018-19||501,788||TB Commitment|
|Core Geospatial Data||Core Geospatial Data/Innovative Geospatial Solutions||2022-23||14,046,011||14,921,984||NRCan Priority|
|Canada Lands Survey System||Canada Lands Survey System program||2019-20||9,560,082||12,172,017||TB Commitment|
|Geoscience for Sustainable Development of Natural Resources||Targeted Geoscience Initiative Program||2019-20||11,087,543||26,548,273||TB Commitment|
|Climate Change Adaptation||Climate Change Adaptation Program||2020-21||9,578,265||11,026,199||TB Commitment|
|Climate Change Adaptation Theme (Horizontal led by ECCC)||2021-22||TB commitment|
|Explosives Safety and Security||Single Window Initiative (Horizontal Evaluation led by CBSA)||2018-19||1,532,812||6,328,863||TB Commitment|
|Explosives Safety and Security Program||2022-23||4,796,051||NRCan Priority|
|Geoscience to Keep Canada Safe||Canadian Hazards Information Systems||2019-20||14,126,747||17,175,143||TB Commitment|
|Targeted Geoscience Initiative||2019-20||115,495||TB Commitment|
|Climate Change Adaptation (NRCan)||2020-21||2,932,901||TB Commitment|
|Climate Change Adaptation Theme (Horizontal led by ECCC)||2021-22||TB Commitment|
|Polar Continental Shelf Program||Polar Continental Shelf Program||2019-20||7,392,750||7,392,750||NRCan Priority|
|Clean Energy Technology Policy, Research and Engagement||World Class Tanker Safety System (Horizontal Led by TC)||2018-19||5,528,894||77,517,143||TB Commitment|
|Clean Energy Technology Policy, Research and Engagement program||2019-20||71,988,249||NRCan Priority|
|Clean Growth in Natural Resource Sectors||Clean Growth in Natural Resource Sectors||2019-20||82,486,581||82,563,058||FAA Requirement|
|Energy Innovation Program||Energy Innovation Program||2018-19||41,605,238||97,828,054||FAA Requirement|
|Deployment of Electric Vehicles and Alternative Transportation Fuels||2020-21||10,876,444||FAA Requirement|
|Smart Grids Program||2021-22||14,036,755||FAA Requirement|
|Electric Vehicles Infrastructure Demonstration Program||2021-22||22,344,000||FAA Requirement|
|Clean Energy for Rural and Remote Communities||2022-23||3,957,720||FAA Requirement|
|Energy Efficient Buildings Programs||2022-23||4,038,225||TB Commitment|
|Green Mining Innovation||Green Mining Innovation Program||2022-23||15,713,887||15,713,887||NRCan Priority|
|Fibre Solutions||Forest Sector Innovation Program||2018-19||3,442,733||11,717,311||TB Commitment|
|Electricity Resources||Regional Electricity Cooperation and Strategic Infrastructure (RECSI)||2018-19||0||158,778,058||TB Commitment|
|Renewable Energy Deployment||2020-21||108,852,705||TB Commitment|
|Smart Grids Program||2021-22||10,869,837||FAA Requirement|
|Emerging Renewables Program||2022-23||24,939,778||FAA Requirement|
|Clean Energy for Rural and Remote Communities||2022-23||4,729,135||FAA Requirement|
|Energy Efficiency||Energy Efficiency Program||2018-19||37,547,263||53,003,633||TB Commitment|
|Energy Efficient Buildings Programs||2022-23||15,456,370||TB Commitment|
|Energy and Climate Change Policy||Energy and Climate Change Policy Capacity||2018-19||7,563,627||7,563,627||NRCan Priority|
|Lower Carbon Transportation||Energy Efficiency Program||2018-19||5,743,040||24,735,147||TB Commitment|
|Deployment of Electric Vehicles and Alternative Transportation Fuels||2020-21||18,992,107||FAA Requirement|
|Innovative Geospatial Solutions||Core Geospatial Data/Innovative Geospatial Solutions||2022-23||14,367,311||14,367,311||NRCan Priority|
|Forest Sector Competitiveness||Management Control & Performance Measurement Framework of Forest Sector Competitiveness||2018-19||98,742,643||98,742,643||NRCan Priority|
|Forest Sector Innovation||2018-19||FAA Requirement|
|Expanding Market Opportunities||2019-20||FAA Requirement|
|Clean Energy for Rural and Remote Communities||2022-23||FAA Requirement|
|Provision of Federal Leadership in Minerals and Metals Sector||Extractive Sector Transparency (ESTMA)||2019-20||10,068,295||10,743,532||NRCan Priority|
|International Energy Engagement||Energy and Climate Change Policy Capacity||2018-19||1,233,153||4,626,787||NRCan Priority|
|Clean Growth and Climate Change - International Cooperation Theme (Horizontal led by ECCC)||2020-21||3,146,731||TB Commitment|
|Indigenous Engagement Office – West||West Coast Energy initiative / Indigenous Partnerships Office - West||2018-19||20,139,828||20,139,828||TB Commitment|
|Major Projects Management Office||Major Projects Management Office Initiative (Horizontal led by NRCan)||2018-19||6,101,598||6,101,598||TB Commitment|
|Youth Employment Strategy||Youth Emplyment Strategy (Horizontal led by ESDC)||2019-20||9,298,120||9,298,120||TB Commitment|
|Canadian Geodetic Survey: Spatially Enabling Canada||Not Applicable (NA)||NA||NA||5,625,504||NA|
|Canada-US International Boundary Treaty||NA||NA||NA||1,980,413||NA|
|Pest Risk Management||NA||NA||NA||23,724,920||NA|
|Wildfire Risk Management||NA||NA||NA||6,929,115||NA|
|Forest Climate Change||NA||NA||NA||10,584,648||NA|
|Sustainable Forest Management||NA||NA||NA||14,211,829||NA|
|Energy Safety and Security and Petroleum Resources||NA||NA||NA||7,902,284||NA|
|Percentage of 2018-19 Total Planned Program Spending
Covered by 5-year Evaluation Plan
Table Notes: N/A- Programs denoted Not Applicable (NA) indicate no planned evaluations, being low risk and having no other driver.
- Date modified: