Table of Contents
- EXECUTIVE SUMMARY
- 2.0 BACKGROUND
- 2.1 Introduction
- 2.2 MPMOI Mandate
- 2.3 Context and History
- 2.4 MPMOI Stakeholders
- 2.5 Governance and Administration
- 2.6 Resources and Funding
- 2.7 MPMOI Logic Model
- 2.8 Overview of MPMOI Activities and Delivery
- 2.9 MPMOI Projects and Potential Investment
- 3.0 EVALUATION OBJECTIVE, SCOPE, AND METHODOLOGY
- 4.0 EVALUATION FINDINGS
- 4.1 Relevance
- 4.1.1 Continued Need for the Program (Issue 1)
- 4.1.2 Alignment with Government Priorities (Issue 2)
- 4.1.3 Alignment with Federal Roles and Responsibilities (Issue 3)
- 4.2 Performance (Effectiveness, Efficiency and Economy)
- 4.2.1 Achievement of Expected Outcomes (Issue 4)
- 4.2.2 Demonstration of Efficiency and Effectiveness (Issue 5)
- 4.1 Relevance
- 5.0 CONCLUSION AND RECOMMENDATIONS
- ANNEX A – DEPT/AGENCY FINANCIAL PROFILES: 2007-08 TO 2011-12
- ANNEX B – STAFFING LEVELS: MPMOI-FUNDED FTES: 2007-08 TO 2010-11
- ANNEX C – PROGRAM LOGIC MODEL DESCRIPTION
- ANNEX D – DETAILED MPMOI ACTIVITIES
- ANNEX E – CHARACTERISTICS OF CASE STUDY PROJECTS
Tables and Figures
- Table 1 Participating departments/agencies: funded and MOU signatories
- Table 2 Balance Sheet: MPMOI financial profile, 2007-08 to 2011-12 ($’000)
Table 3 Staffing Levels: MPMOI-funded FTEs, 2007-08 to 2011-11 ($’000)
- Table 4 Overview of department/agency responsibilities under the MPMOI
- Table 5 Active and completed projects under the MPMOI, by phase, 2008 to 2011
- Table 6 Distribution of projects for the comparative case study design
- Table 7 Distribution of internal and external interviews across methods
- Table 8 Summary of response ratings
- Table 9 Overview of mandates of MPMOI-funded departments and agencies
- Table 10 Basic characteristics of projects examined in the case studies
- Figure 1 Overview of the federal regulatory system
- Figure 2 Key issues related to systemic and capacity issues
- Figure 3 MPMOI logic model
- Figure 4 Cross-cutting issues discussed at DM Committee meetings
- AANDC Aboriginal Affairs and Northern Development Canada (formerly Indian and Northern Affairs Canada [INAC])
- ADM Assistant Deputy Minister
- CAU Consultation and Accommodation Unit (at AANDC)
- CCC Crown Consultation Coordinator
- CEA Act Canadian Environmental Assessment Act
- CEA Agency Canadian Environmental Assessment Agency
- CanNor Canadian Northern Economic Development Agency
- CNSC Canadian Nuclear Safety Commission
- DFO Fisheries and Oceans Canada
- DG Director General
- DM Deputy Minister
- DOJ Department of Justice
- EA Environmental Assessment
- EAC Evaluation Advisory Committee
- EAPC Environmental Assessment Project Committee
- EC Environment Canada
- EIA Environmental Impact Assessment
- ENGO Environmental Non-Government Organization
- EPA Environmental Protection Authority
- FEAC Federal Environmental Assessment Coordinator
- GiC Governor in Council
- GoC Government of Canada
- HC Health Canada
- IEAC Interdepartmental/Interagency Evaluation Advisory Committee
- JEGA Jobs and Economic Growth Act
- LAF Lead Agency Framework (Western Australia)
- MOU Memorandum of Understanding
- MPMO Office Major Projects Management Office
- MPMOI Major Projects Management Office Initiative
- MRP Major Resource Project
- NEB National Energy Board
- NPMO Northern Projects Management Office
- NRCan Natural Resources Canada
- ODAC Office of Development Approval Coordination (Western Australia)
- OEPA Office of the Environmental Protection Authority (Western Australia)
- PA Project Agreement
- PC Parks Canada
- PCO Privy Council Office
- PD Project Description
- QMS Quality Management System
- RA Responsible Authority
- ROI Record of Information
- TC Transport Canada
- WA Western Australia
Evaluation Scope and Issues
This report summarizes the results of the evaluation of the Major Projects Management Office Initiative (MPMOI) conducted by Natural Resources Canada (NRCan) with the input of departments and agencies participating in the MPMOI. This evaluation encompasses the five-year period from fiscal year 2007-08 to 2011-12 and covers all objectives and activities of the horizontal initiative, including those undertaken by the Major Projects Management Office (MPMO) and by MPMOI-funded and other participating federal departments and agencies. The primary objective of the evaluation is to assess issues relating to the relevance and performance of the MPMOI and provide recommendations as necessary.
The MPMOI was established in 2007 to improve the efficiency and effectiveness of the federal regulatory review system while ensuring careful consideration of environmental protection, Aboriginal consultation obligations, and enhancing the competitiveness of Canada’s resource industries.
Currently, funding under the MPMOI enhances the scientific and technical capacity and capacity for Aboriginal consultations in five federal departments and agencies: Aboriginal Affairs and Northern Development Canada (AANDC), the Canadian Environmental Assessment Agency (CEA Agency), Fisheries and Oceans Canada (DFO), Environment Canada (EC), and Transport Canada (TC). In addition, the Canadian Nuclear Safety Commission (CNSC), National Energy Board (NEB), and NRCan signed a Memorandum of Understanding associated with the Initiative. NRCan acts as lead department for the MPMOI and hosts the MPMO. Other significant contributors to the Initiative include the Department of Justice (DOJ) and the Privy Council Office (PCO). Health Canada (HC) and Parks Canada (PC) often interact with the MPMOI in the context of project reviews.
All participating departments/agencies contribute to a range of activities and processes under the MPMOI, including those relating to projects in which they are involved (i.e., project agreements, project tracking and monitoring), as well as the collaborative development of tools and procedures, informing and supporting policy research, and participating in the MPMOI’s governance structure. Each of them has specific roles and responsibilities, and may act as a regulatory authority for the environmental assessment (EA) and/or the regulatory decision-making phase, or provide expert advice depending on their statutory responsibilities.
The MPMO was established (and funded) through the Initiative to conduct a range of activities that are intended to improve the accountability, transparency, timeliness and predictability of the federal regulatory review process for major resource projects. The mandate of the MPMO is two-fold: 1) major project management and coordination; and 2) policy leadership (problem-solving of short- and longer-term systemic issues).
At the time that the evaluation was carried out (2011-12), the MPMOI had 70 active projects. Three had been completed, and none had yet been implemented.
The evaluation was extensive and complex given the horizontal, multi-department/agency nature of the MPMOI. The scope included all the activities of the MPMOI, including those undertaken by the MPMO and by individual departments/agencies – individually or in collaboration.
Four main methods were used in this evaluation: (1) an extensive review of over 1,000 internal files and documents provided by the MPMO and the participating departments/agencies as well as external literature; (2) structured interviews with 32 internal and 14 external MPMOI stakeholders (e.g., provincial and federal government, program managers/staff, industry, ENGOs); (3) a comparative analysis of MPMOI and six other international jurisdictions with a more detailed analysis of the project review system in Western Australia, including a review of internal and external documents and data, and four additional interviews with representatives of Western Australia; and (4) case studies of ten major resource projects (based on a comparative case study design), seven MPMOI projects and three non-MPMOI projects, including 41 additional interviews with departmental/agency staff as well as 14 external stakeholders (including project proponents). A total of 105 interviews were conducted across methods.
Evaluation Issue 1—Continued Need for Program
The evidence clearly demonstrates a continued need for the MPMOI – that is, in addition to a number of recent operational and policy improvements consistent with the Initiative’s objectives, such as legislative changes that are attributable to the Initiative. There is a strongly articulated need for system-wide coordination and governance – with senior management involvement – that supports policy leadership, as well as performance monitoring and management.
The Initiative was responsive to at least some of the needs of its stakeholders. In particular, the needs of departments and agencies were most strongly met through the MPMOI funding for science and technical (S&T) capacity and for Aboriginal consultation capacity. It is expected that the demands for S&T capacity and capacity for Aboriginal consultations will only continue to increase as a result of the anticipated surge in the number of major natural resource projects entering the federal regulatory system. Moreover, industry perceived the MPMOI as a “step in the right direction” in meeting its needs, but expected further major improvements that could likely be delivered through the Initiative.
Evaluation Issue 2—Alignment with Government Priorities
There is strong alignment between MPMOI objectives and commitments made by the Government of Canada (e.g., past budgets, speeches, Cabinet directives, etc.). Budget 2010 and the 2011 Speech from the Throne confirmed that the MPMOI’s objectives remain a key priority for the federal government. Additionally, the Initiative was found to be consistent with the currently formulated mandates and strategic objectives of participating departments/agencies (whether funded or unfunded via the MPMOI).
Evaluation Issue 3—Alignment with Federal Roles and Responsibilities
The government’s role in the Initiative is appropriate given the government’s statutory and regulatory responsibilities relative to EAs and regulatory reviews, as well as a constitutional duty to consult Aboriginal groups.
Performance Issues (Effectiveness, Efficiency and Economy)
Evaluation Issue 4—Achievement of Expected Outcomes
Participating departments/agencies and the MPMO, through the MPMOI, effectively implemented a wide range of processes that clearly contributed to a high level of achievement of outcomes relating to Aboriginal consultations and collaborative horizontal policy research. The integration and federal coordination of EAs and regulatory reviews were also found to have increased under the Initiative.
However, the demonstration of the extent to which these improvements have, to date, had an impact on timelines of the process is somewhat limited. In summary, federal timelines were viewed by internal and external stakeholders consulted to be improving due to increased capacity and improved integration and coordination (as a result of MPMOI procedures, coordination, and legislative improvements). Efforts to quantitatively demonstrate to what extent these improvements had translated into increased overall predictability of the federal project review process under the MPMOI were limited and resulted in mixed findings.
Given that none of the MPMOI projects have been implemented to date (noting the early stage of the MPMOI as of 2011-12), data did not exist regarding the impact of the MPMOI on environmental outcomes and hence no conclusions or findings are presented in this report on this aspect of the MPMOI.
A number of internal and external factors such as concurrent activities (e.g., development of Aboriginal consultation procedures, project management approaches) or changes within the federal government and beyond (e.g., court decisions) were found to have had a significant influence on these outcomes, creating challenges and opportunities for the Initiative.
Transparency and accountability of the federal regulatory process within the federal government increased significantly through the MPMOI. However, the impacts of efforts to increase process transparency for external stakeholders, particularly the provinces and Aboriginal groups, were less clearly demonstrated.
The MPMOI also led to benefits that extended beyond the Initiative, such as a “catalyzing” effect for further process improvements within federal departments/agencies, as well as developing an exceptional level of horizontal dialogue at senior management levels.
Evaluation Issue 5—Demonstration of Efficiency and Economy
Based on the available data, the evaluation found that operation and delivery of MPMOI activities, outputs and outcomes would be difficult to achieve at a lower cost under the current regulatory system. However, the demonstration of cost-effectiveness, efficiency and economy was limited by the lack of baseline financial data and detailed data on MPMOI resource allocation and use (i.e., by type of activity or by project; non-MPMOI contributions; across departments and agencies).
Nonetheless, areas for improvement were identified. For instance, while transaction costs were associated with a number of MPMOI processes (e.g., reporting and coordination) that clearly contributed to the achievement of the Initiative’s objectives, there was room to review related procedures.
Improvements were found to have been made in other areas related to MPMOI outputs and activities. The evaluation also found, however, that having initially focused on establishing efficiency measures, there is a growing need to balance MPMOI performance measurement to encompass outcomes related to environmental assessments, regulatory review and Aboriginal consultations to a greater extent.
Enhanced coordination and communication among federal departments/agencies led to increased efficiency of the federal process overall. Reduction of duplication with the provincial processes was clearly an area where significant future gains could be achieved.
The international comparative analysis revealed that the MPMOI performed quite well compared to similar initiatives. The MPMOI implemented several best practices and offered some unique features not identified elsewhere. Certain practices seen in Western Australia could also inform future improvements to the Canadian regulatory system for major resource projects.
Note Regarding Recommendations
The MPMOI is a horizontal government initiative that focuses the work of many federal departments and agencies to achieve the objectives and outcomes of the MPMOI, using horizontal governance structure and collaborative activities. In this context, the NRCan officials involved with the MPMO are not accountable for the delivery of other government departments and agencies active in the MPMOI. NRCan is responsible for the MPMO and activities delivered within the Department; NRCan’s Deputy Minister also chairs the Deputy Ministers’ Committee. Thus, the recommendations made by this evaluation are directed to MPMO, in coordination with other departments and agencies participating in the MPMOI.Footnote 1
Conclusions and Recommendations
While recognizing a number of recent improvements, this evaluation concludes that there is a need to ensure that the federal government continues to address the ongoing system-wide issues and capacity constraints within the federal regulatory system for major resource projects.
MPMOI capacity funding was found to be essential to ensure the efficiency and effectiveness of the federal regulatory system and, as such, remains necessary to address ongoing capacity needs within departments/agencies under the current federal regulatory system. The procedures and practices implemented across federal departments and agencies under the MPMOI have begun to show results in terms of performance improvements; however, further outcomes in this regard are expected by all stakeholder groups. Thus, there is an ongoing need for the MPMOI’s capacity, tools, procedures, and senior-level coordination to help address issues and challenges related to the review of major resource projects in a coordinated manner until key system-wide issues and capacity constraints impacting major project reviews are addressed.
The evaluation confirmed that significant and long-lasting improvements in efficiency and effectiveness of the federal regulatory system will only come from major changes to the system (e.g., further legislative amendments). The horizontal policy leadership, engagement and decision-making approach demonstrated through the MPMOI could be an effective vehicle through which to achieve these goals.
Recommendation 1: MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI maintains its horizontal policy and governance mechanisms as a vehicle to further advance options for system-wide improvements.
Multiples lines of evidence also indicated that overlapping or misaligned federal and provincial review processes were an ongoing barrier to efficient project reviews. Thus, further work to improve coordination and reduce duplication of these processes would likely lead more efficient project reviews.
Recommendation 2: MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI increases its focus on improving the alignment and reducing the overlap of federal and provincial review processes.
The evaluation found that the consistency, integration and coordination of Aboriginal consultations had improved considerably for major resource project reviews since the implementation of the whole-of-government approach under the MPMOI. However, this approach remained a “work in progress” and the delivery of consistent, adequate and meaningful Aboriginal consultations continued to be a challenging process. Thus, ongoing work on this component of the federal regulatory system would be required.
Recommendation 3: MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI continues to further improve the horizontal coordination, integration, communication and effectiveness of Aboriginal consultations.
Observations of overlap of some project coordination, coordination, and monitoring activities, as well as the perception of transaction costs associated with these activities led to the identification of suggestions of how to increase the efficiency of the Initiative. For department/agency staff in particular, the perception of transaction costs also affected their buy-in with regard to MPMOI’s project monitoring, tracking and reporting activities, and posed a barrier to the performance of this horizontal initiative. As the MPMOI has moved beyond its initial learning curve, it would be an appropriate time to revisit its tools, processes and other operational procedures to maximize the cost-effectiveness of the federal review process. When implementing changes to MPMOI tools and processes in order to improve the efficiency of the Initiative, the MPMO would need to monitor the impact of these changes on the system to ensure that major project reviews are conducted in a timely and coordinated fashion.
Recommendation 4: MPMO, in coordination with participating departments and agencies, should review current operational procedures, including transaction costs, with a view to improving their efficiency.
In the absence of baseline financial data, more comprehensive assessments of transaction costs – as well as the assessment of efficiency and economy issues in future evaluations – would require improvements in the availability of detailed financial and full-time equivalents (FTE) data across departments and agencies (e.g., by type of activity or by project, as well as non-MPMOI contributions).
Recommendation 5: MPMO, in coordination with participating departments and agencies, should collect financial data (including non-MPMOI resources) on resource use by project and activity within the Initiative.
Whereas the MPMOI implemented a robust data collection and performance monitoring system for objectives relating to timeliness of project reviews, the evaluation found that a similar level of performance monitoring does not exist to assess the achievement of outcomes relating to effectiveness and quality of EAs, regulatory reviews and Aboriginal consultations. The evaluation identified a growing need to ensure that the demonstration of progress toward achieving these objectives – including the continued maintenance of environmental protection and the mitigation of social and health effects – are supported by improved performance measurement tools and metrics at the level of the Initiative.
Recommendation 6: MPMO, in coordination with participating departments and agencies, should improve the MPMOI performance measurement strategy by enhancing monitoring and reporting on the quality and effectiveness of project review processes and outcomes.
The current public MPMO Tracker website does not appear to provide sufficient information to ensure the usability of this website by external stakeholders. A clear understanding of this issue and of options to address it was hampered by a lack of information on the use and users of the MPMO Tracker. Opportunities to increase the linking and alignment between the MPMO Tracker and other available tools (e.g., CEA Registry) could also be considered to improve the overall transparency of the federal regulatory system for external stakeholders.
Recommendation 7: MPMO, in coordination with participating departments and agencies, should further improve the transparency of project review processes for external stakeholders, notably by identifying public data users’ needs and exploring potential linkages among existing public information tools (including the MPMO Tracker).
The evaluation found that external stakeholders (industry associations) and some internal stakeholders were unaware of the MPMOI’s policy role, or any role outside the operational aspects of the Initiative (e.g., DM Committee, project coordination). As a result, internal and external stakeholders did not necessarily attribute specific changes of the review process to the MPMOI. The lack of awareness also affected the buy-in of department/agency staff with regard to project monitoring, tracking and reporting activities. External stakeholders expressed an interest in participating in efforts to improve the federal regulatory system. Formal engagement through a stakeholder reference group was found to be useful in this regard in Western Australia. The Initiative would thus benefit from increased engagement – including better communication of its results – with a wider range of internal and external stakeholder groups.
Recommendation 8: MPMO, in coordination with participating departments and agencies, should enhance its stakeholder outreach and engagement activities, and communicate its results more clearly within and outside the federal government, nationally and regionally.
Recommendations and Management Response and Action Plan Table
|Recommendation||Management Response and Action Plan||Responsible (Target Date)|
|1. MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI maintains its horizontal policy and governance mechanisms as a vehicle to further advance options for system-wide improvements.||Management agrees with the recommendation
Departments and agencies will continue to leverage the MPMOI horizontal governance mechanism to address project related issues and to advance system-wide improvements.
MPMO will continue to provide secretariat support for regular meetings of the Major Projects DMs’ Committee and supporting governance bodies, including working in close collaboration with parties in identifying priorities, developing options for further system improvements, and setting the forward agenda.
|MPMO with AANDC, CEAA, CNSC, EC, DFO, TC, and NEB
|2. MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI increases its focus on improving the alignment and reducing the overlap of federal and provincial review processes.||Management agrees with the recommendation
Departments and agencies will:
|MPMO with AANDC, CEAA, EC, DFO and TC
|3. MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI continues to further improve the horizontal coordination, integration, communication and effectiveness of Aboriginal consultations.||Management agrees with the recommendation
Departments and agencies will continue to improve Aboriginal consultation for major project reviews by:
MPMO with AANDC, CEAA, DFO, EC, TC
|4. MPMO, in coordination with participating departments and agencies, should review current operational procedures, including transaction costs, with a view to improving their efficiency.||Management agrees with the recommendation
As part of their commitment to continuous improvement, departments and agencies will undertake a review of existing operational tools, processes and procedures, to identify possible improvements or efficiencies. Findings and recommendations will be presented for consideration by the Major Projects DMs’ Committee.
|MPMO with AANDC, CEAA, CNSC, DFO, EC, NEB and TC
|5. MPMO, in coordination with participating departments and agencies, should collect financial data (including non-MPMOI resources) on resource use by project and activity within the Initiative.||Management agrees with the notion of tracking resources allocated to core business activities but finds that the recommendation of collecting financial data on a project-by-project basis would be administratively cumbersome with limited contribution towards the core objectives of the initiative.
Departments and agencies will work collaboratively to conduct a detailed assessment of the overall financial implications of the federal review process for major projects. Findings will be presented to the Major Projects DMs’ Committee.
|MPMO with AANDC, CEAA, CNSC, DFO, EC, NEB and TC
6. MPMO, in coordination with participating departments and agencies, should improve the MPMOI performance measurement strategy by enhancing monitoring and reporting on the quality and effectiveness of project review processes and outcomes.
Management agrees with the recommendation
|MPMO with AANDC, CEAA, CNSC, DFO, EC, NEB & TC
|7. MPMO, in coordination with participating departments and agencies, should further improve the transparency of project review processes for external stakeholders, notably by identifying public data users’ needs and exploring potential linkages among existing public information tools (including the MPMO Tracker).||Management agrees with the recommendation
MPMO will enhance the level of information on major resource project reviews publicly available on the MPMO Tracker. MPMO with AANDC and CEAA will explore potential linkages between existing information tools.
|MPMO, September 2012
MPMO with AANDC and CEAA
|8. MPMO, in coordination with participating departments and agencies, should enhance its stakeholder outreach and engagement activities, and communicate its results more clearly within and outside the federal government, nationally and regionally.||Management agrees with the recommendation
MPMO will work with departments and agencies to implement a proactive communications and outreach plan that will describe concrete actions to ensure consistent and on-going engagement with key partners and stakeholders in all regions of the country.
|MPMO with AANDC, CEAA, CNSC, DFO, EC, NEB and TC January 2013|
This is the first evaluation of the Major Projects Management Office Initiative (MPMOI), a horizontal initiative established by the Government of Canada (GoC) to improve the performance of the federal regulatory system for major resource projects. This evaluation encompasses the five-year period from fiscal year 2007-08 to 2011-12 and covers all objectives and activities of the MPMOI, including those undertaken by the Major Projects Management Office (MPMO) and by MPMOI-funded and other participating federal departments and agencies (total budget of $150 million for the five-year evaluation period). The MPMO is housed within Natural Resources Canada (NRCan), which is the lead department for this initiative.
2.2 MPMOI Mandate
Considering the current growth of Canada’s natural resource sector – more particularly major resource projects – the main objective of the MPMOI is to improve the federal regulatory system so as to have an efficient process, capable of reaching decisions on major resource projects in a more timely and effective manner, while ensuring careful consideration of environmental standards, Aboriginal consultation obligations and technical requirements.Footnote 2 These efforts are expected to improve the efficiency and effectiveness of the regulatory system, enhance the competitiveness of Canada’s resource industries and mitigate undesired social, human health and environmental effects of resource development.Footnote 3
Five departments/agencies are funded under the MPMOI, and an additional three have signed the Memorandum of Understanding (MOU)Footnote 4 associated with the Initiative; together, these are referred to as “participating departments and agencies” (Table 1). The MPMO is located within NRCan and funded via the MPMOI, but NRCan’s environmental assessment (EA) and regulatory activities relating to major projects have not received additional capacity funding through the MPMOI. The Department of Justice (DOJ) and the Privy Council Office (PCO) have also been involved in various MPMOI activities; whereas Health Canada (HC) and Parks Canada (PC) often interact with the MPMOI in the context of project reviews.
Table 1 Participating departments/agencies: funded and MOU signatories
|MPMOI-funded||Other MOU signatories
(not funded by the MPMOI)
Participating departments/agencies and the MPMO have agreed to implement this initiative, both individually and in collaboration. Each of the departments and agencies participating in the MPMOI has a mandate relating to federal EAs and regulatory reviews that pre-date the MPMOI. In this context, the Initiative provides supplemental capacity funding and clarifies certain roles and responsibilities of departments and agencies relating to EAs and regulatory reviews for major resource projects. These roles and responsibilities are presented for each department/agency in Table 4 and focus on the effective, coordinated and concurrent discharge of their statutory duties and functions. In addition, departments and agencies have committed to participate in collaborative efforts to find and implement solutions in order to improve the performance of the federal regulatory system for major resource projects.Footnote 5
The MPMO was established within the Initiative to conduct a range of activities that are intended to improve the accountability, transparency, timeliness and predictability of the federal regulatory review process for major resource projects.Footnote 6 The mandate of the MPMO is two-fold: 1) to provide overarching project coordination, management and accountability for major resource projectsFootnote 7 within the context of the existing federal regulatory review process; and 2) to undertake research and identify options that could drive further performance improvements to the federal regulatory system for major resource projects.Footnote 8 As the Initiative consists of a collaborative approach between departments/agencies and the MPMO, the latter provides a centralized function and works with federal departments/agencies to develop guidelines, procedures and service standards that promote early engagement with proponents and ensures a clear, consistent and coordinated federal approach to the review of major resource projects. The MPMO also leads horizontal collaborative policy research work.
2.3 Context and History
Canada’s natural resource sector continues to undergo rapid growth. Currently (2010–11), major resource projects – those that are subject to a comprehensive study, a panel review or a large or complex multi-jurisdictional screeningFootnote 9 – are estimated to account for 80 percent to 90 percent of all major projects in Canada. In the three-year period before the MPMOI was established (2003–06), the number of major resource projects that were subject to an environmental assessment increased by more than 200 percent.Footnote 10 However, the ability of federal regulatory departments and agencies to provide efficient regulatory reviews in the face of this escalating demand has been hindered by:
- systemic constraints associated with the federal regulatory process for large resource projects (e.g., duplication or lack of integration of federal requirements, overlapping federal and provincial processes);
- a lack of scientific and technical capacity (i.e., in terms of resources) needed to meet the increased demand; and
- a lack of capacity to consistently deliver Aboriginal Crown consultation responsibilities.Footnote 11
The complexity of the federal regulatory system associated with the administration of relevant legislation by multiple departments/agencies, as well as key issues and constraints within this system are further illustrated in Figure 1 and Figure 2.
Figure 1 Overview of the federal regulatory system
Source: T. Hubbard, MPMO. (2011). Improving the Performance of the Regulatory System for Major Resource Projects. Presentation to PPX Symposium, available at: http://www.ppx.ca/download/symposiums/2011/concurrent/2011_CS_3_Hubbard_E.pdf
Figure 2 Key issues related to systemic and capacity issues
Note: The MPMOI does not apply to projects North of 60, which are overseen by the NPMO. However, the NPMO uses the same governance structure as the MPMOI to help address systemic issues relating to projects in the North.
Source: MPMO. (2008). Horizontal Results-Based Management and Accountability Framework (updated final version). Initial program documentation, page 4.
By 2007, it was widely recognized that the federal regulatory review system required a more coherent management structure and horizontal coordination. Alongside a range of related efforts external to or involving the federal government, an interdepartmental task force was created to address this issue Its reports and recommendations served to inform a number of new regulatory improvement initiatives announced in Budget 2007. This included the Cabinet Directive on Improving the Performance of the Regulatory System for Major Resource Projects (the directive that established the MPMOI, launched on October 1, 2007).
Under this Cabinet Directive and its associated MOU,Footnote 12 the Minister of Natural Resources was to establish the MPMO within NRCan, and participating departments/agencies agreed to work collaboratively to identify short- and long-term improvements to the federal regulatory system for major resource projects. Thus, the MPMOI represented a new overarching approach to the federal regulatory review process for major resource projects, with the MPMO assuming a central role in terms of the coordination and support for this approach.
The MPMO began operations on February 26, 2008. The supplemental capacity funding provided through the MPMOI to departments/agencies for scientific and technical activities, as well as Aboriginal consultation responsibilities, was used to gradually staff positions starting in 2007-08, although this element of the Initiative was not fully implemented until 2009-10 (see Section 2.6.2).
Since the launch of the MPMOI, the January 2010 Supreme Court ruling with respect to the Red Chris mine (MiningWatch) expanded further, in an unforeseen manner, the broad approach to the scoping of EAsFootnote 13 already promoted under the Cabinet Directive on Implementing the Canadian Environmental Assessment Act (CEA Act). In its decision, the Court ended up limiting the government discretion in defining the scope of a project by establishing that, at minimum, the project to be reviewed must be the project as proposed by the proponent. The CEA Act was amended in the wake of this rulingFootnote 14 to clarify how the authority over the scope of project could be exercised. Regulated timelines were also established for EAs conducted by the CEA Agency as of July 2011.Footnote 15,Footnote 16 Budget 2010 also included the establishment of participant funding programs for the NEB and CNSC.Footnote 17,Footnote 18
The Canadian Northern Economic Development Agency (CanNor) was created in 2009 to help deliver federal programs tailored to the needs of northern Canada, specifically Yukon, the Northwest Territories and Nunavut. The Northern Projects Management Office (NPMO) was established in 2010 as a core CanNor program to help coordinate federal regulators, track projects, and maintain the Crown consultation record for northern resource development projects. The NPMO uses the same governance structure as the MPMOI.Footnote 19
All of these recent changes, some of which are a direct result of MPMOI-led policy research, have had implications for the way in which EAs and regulatory reviews are conducted by federal departments and agencies.
The Initiative is currently funded until March 31, 2012. Prior to that date, the Minister of Natural Resources is required to review the progress made in achieving the MPMOI’s objectives, and to seek approval for the continued operations of the MPMOI.
2.4 MPMOI Stakeholders
The MPMOI’s primary stakeholders have been identified as: federal departments/agencies which signed the MOU associated with the Initiative (see Table 1); other federal departments/agencies involved in the regulatory review processes for major resource projects, such as Health Canada (HC), Parks Canada (PC), the Canadian Transportation Agency (CTA), the Department of Justice (DOJ) and CanNor; provincial and territorial governments; municipalities; port authorities; Canada’s natural resource industries; environmental non-governmental organizations (ENGOs); Aboriginal communities; northern and rural resource-dependent communities, as well as the Canadian public.Footnote 20
2.5 Governance and Administration
The lead minister for the Initiative is the Minister of Natural Resources. The Minister reports annually to Cabinet and parliament on progress made under the Initiative towards achievement of its objectives.Footnote 21
Executive coordination for the implementation of the Cabinet Directive for the MPMOIand the related MOU is provided by the Major Projects Deputy Ministers’ (DM) Committee. The Committee is chaired by the Deputy Minister of NRCan and comprises DMs from key federal regulatory departments and agencies (AANDC, DFO, EC, and TC), the presidents of the CEA Agency and the CNSC, and the Chair of the NEB. Representatives from CanNor, the DOJ and PCO or other relevant bodies (e.g., Treasury Board Secretariat, Industry Canada) periodically attend meetings, as do representatives from the MPMO. As the MPMOI’s governing body, the Committee members: ensure that the departments/agencies involved in the MPMOI meet their mandate and objectives; provide direction for the resolution of project-specific issues; oversee key policies, processes and procedures; track and monitor projects; and facilitate clear and open lines of communication among departments and agencies.Footnote 22 Individual DMs are responsible for communicating Committee decisions to their respective departments and agencies.Footnote 23
All participating departments/agencies are expected to work collaboratively with the MPMO and with other federal departments and agencies to carry out their statutory functions in a coordinated manner, to identify areas of improvement, and to develop and implement improvements to the regulatory system.Footnote 24 However, the MPMO has no legislative authority in the regulatory review process, and regulatory departments and agencies are not accountable to the MPMO or the DM Committee.Footnote 25 To further enhance communication on key issues and facilitate collaboration among departments, interdepartmental committees have also been established at the assistant deputy minister (ADM), director general (DG), and director levels.Footnote 26
The MPMO is housed within NRCan and has its own ADM, who is supported by two DGs.Footnote 27 One of the roles of the MPMO is to provide secretariat support to the DM Committee, as well as data, analysis and other information for this and other committees.Footnote 28
2.6 Resources and Funding
Funding for the MPMOI amounted to $150 million over five years (2007–08 to 2011–12). A significant portion of the funding, $130 million, was allocated to building scientific and technical capacity within five federal regulatory departments involved in the regulatory review of major resource projects and in the related Aboriginal consultations. Approximately $20 million was allocated to establishing and maintaining the MPMO within NRCan. Annual funding and expenditures over the evaluation period are presented in Table 2. Financial profiles for each MPMOI-funded department/agency and for the MPMO are presented in Annex A.
The beginning of the Initiative was marked by a progressive use of allocated funding. For example, most of the funding and full-time equivalents (FTEs) allocated to the 2007-08 fiscal year were reallocated to 2008-09 as no project activities occurred in 2007-08. The variances between 2007-08 and 2008-09 reflected in Table 2 and Table 3 are attributable to the start-up process of the MPMO Initiative. Full level of staffing was not achieved until the end of 2008-09.
|Operating & Minor Capital||8,150||11,759||8,394||8,550||7,505|
|Employee Benefit Plan (EBP)||1,770||3,287||3,020||3,027||3,034|
|Operating & Minor Capital||4,164||9,445||6,065||7,671|
|Employee Benefit Plan (EBP)||507||2,311||3,134||3,069|
Note: [a] MPMOI-funded FTEs only, other FTEs dedicated to the MPMO process not included.
[b] Grants & contributions and real property enablers.
Totals in ‘000 may be +/-1 the sum of the column due to rounding.
Source: Compiled from financial profiles provided by individual department/agencies and by the MPMO.
2.6.2 Personnel and In-Kind Contributions
For the first four years of the Initiative (2007–08 to 2010–11), salaries accounted for approximately $54.6 million (including EBP), or 60 percent of MPMOI expenditures. The allocation of personnel across the MPMO and funded departments and agencies is presented in Table 3.
In 2010-11, a total number of approximately 184 FTEs were supported by the MPMOI funding, which represents 97 percent of the allocated FTEs. Half of the MPMOI-funded FTEs (51 percent in 2010–11)Footnote 29 were allocated to positions in regional offices. For further details on departmental staffing levels, please refer to Annex B.
According to the representatives of several funded departments and agencies interviewed as part of this evaluation, non-MPMOI resources have also contributed to the Initiative. Examples of in-kind contributions include:
- the deputy ministers serving on the DM Committee (or other senior management participating in senior-level committees);
- additional personnel (i.e., scientific, technical, regulatory, policy and planning staff) participating in the EAs and regulatory reviews for MPMOI projects (including Aboriginal consultations); and
- personnel in units funded by A-base resources who have participated in the development of guidelines for EA practitioners and for Aboriginal consultations used within the MPMOI (internally or shared with other departments).
For instance, TC has estimated that an additional 10 FTEs are currently supporting the MPMO Initiative. The additional 10 FTEs equate to an approximate salary expense of $1 million/year and $110,000/year in other operating costs. These FTEs are not coded to the MPMOI and are therefore not reflected in the numbers listed in Table 3.
Table 3 Staffing Levels: MPMOI-funded FTEs, 2007-08 to 2011-11 ($’000)
|HEADQUARTERS (National Capital Region)|
Note: HQ and regional allocated and staffed positions do not add up to the total sum as HQ/regional breakdown data were not available for certain departments/agencies.
*Total FTEs allocated and positions staffed in these departments/agencies are, however, reflected in the total sums.
Source: Compiled from staffing profiles provided by individual departments/agencies and by the MPMO Office
AANDC created its Consultation and Accommodation Unit (CAU) in 2007-08, which played a key role in the MPMOI (e.g., participating in MPMOI committees, creating internal consultation guidelines for federal officials in all departments, conducting training for federal officials, functioning as the liaison between AANDC and the MPMO). However, the CAU was not funded through the Initiative as it was not in place when the MPMOI funding was received. Instead, the unit was funded through AANDC’s A-base as part of its in-kind contribution to the Initiative. At least one other funded department identified in-kind funding used to deliver MPMOI activities.
Departments and agencies which do not receive MPMOI funding (i.e., NEB, CNSC and NRCan) also support MPMOI-related achievements by conducting project-level activities (i.e., EAs and regulatory reviews for major resources projects), and by participating in policy research and other collaborative work related to the MPMOI, such as attending DM and other senior-level committees. For example, the NEB estimated that the additional work relating to the MPMOI amounts to about 1.5 FTEs.
The MPMO is located in Ottawa, Ontario. Although there are no regional MPMO offices, MPMO staff maintain a network of regional contacts and travel to the regions on an as-needed basis. Each participating department/agency delivers the Initiative from its existing facilities in the Ottawa area (i.e., headquarters) and from regional facilities.
2.7 MPMOI Logic Model
The logic model of the MPMOI (Figure 3) was developed in consultation with the Evaluation Advisory Committee (EAC)Footnote 30 and the Interagency/Interdepartmental Evaluation Advisory Committee (IEAC).Footnote 31
It presents the main inputs, activities, outputs, and expected outcomes of the Initiative, as well as the main external factors and influences within the context of the Initiative. The timeframe for expected short-term outcomes is one to three years, four to seven years for intermediate outcomes and eight or more years for long-term outcomes. Thus, over the five-year period encompassed by the evaluation, the activities undertaken by the MPMOI were primarily focused on the following short-term outcomes (4.1 to 4.4):
- consistent whole-of-government approach to Aboriginal consultations;
- integrated and well-coordinated environmental assessments and regulatory reviews;
- increased transparency and accountability of the federal regulatory process; and
- improved federal regulatory review process.
A detailed narrative explaining each of the elements in the logic model is presented in Annex C.
2.7 Overview of MPMOI Activities and Delivery
MPMOI activities are delivered by participating departments/agencies, as well as by the MPMO. An overview of these activities is presented below, with additional details in Annex D.
Figure 3 MPMOI logic model
Note: The immediate outcomes 4.1, 4.2 and 4.3 relate to major resource projects under the MPMOI.
Source: Based on the Logic Model in the MPMO’s Horizontal Results-Based Management and Accountability Framework, revised in consultation with the IEAC.
2.8.1 Department/Agency Specific Activities
All participating departments/agencies contribute to a range of activities and processes under the MPMOI, including those relating to projects in which they are involved (i.e., project agreements, project tracking and monitoring), as well as in the collaborative development of tools and procedures, informing and supporting policy research, and participating in the MPMOI’s governance structure (i.e., senior-level committees). Table 4 provides an overview of the specific roles and responsibilities of each department and agency participating in the MPMOI, as well as the Acts for which they have statutory responsibilities to deliver related to MPMOI processes.
With regard to specific projects, each of these departments may act as a responsible authority (RA) for the EA and/or the regulatory decision-making phase, or a federal authority that provides expert advice, depending on its statutory responsibilities with regard to that specific project. See Annex D for further details on the regulatory process for major projects, as well as on the roles of each department/agency.
(funded for MPMO Office, not funded for its EA or RA roles)
Source: Compiled by using information retrieved from official websites of the departments/agencies.
2.8.2 MPMO Activities
With regard to project management and coordination, the MPMO’s role includes:Footnote 32
- coordination of the development of project agreements (PAs), which clearly articulate the roles and responsibilities of each federal department/agency and includes target timelines (PAs are signed by relevant DMs);
- overseeing the implementation of a whole-of-government approach to Aboriginal/Crown consultation on major resource projects, including maintaining the record of Aboriginal-Crown consultations;Footnote 33
- ongoing project and performance monitoring, tracking and reporting (including circulating reports of project progress to participating departments/agencies);
- administering the MPMO Tracker,Footnote 34 a publicly accessible web-based monitoring system for major resource projects that can be updated in real time; and
- presenting regular project updates to the DM Committee and other senior-level committees.
The second part of the MPMO’s mandate involves policy leadership, including problem-solving of short- to medium-term issues that are identified through its ongoing project management and coordination efforts. In particular, the MPMO (in collaboration with other departments) is responsible for:Footnote 35
- conducting research and policy analysis to develop new and innovative approaches to address systemic challenges that impede the federal regulatory process for major resource projects, in collaboration with participating departments/agencies;
- establishing the whole-of-government approach to Aboriginal Crown consultation;
- piloting new approaches to test and implement solutions; and
- seeking continual feedback from both internal and external stakeholders.
According to the MPMO, resources are allocated almost equally between management/ coordination and policy research activities.
2.9 MPMOI Projects and Potential Investment
As of July 31, 2011, 70 active projects were being managed under the MPMOI. New major resource projects entering the federal regulatory system under the MPMOI undergo four key phases: Project Description/Project Agreement (PD/PA); Environmental Assessment (EA); Regulatory Decision-Making (Reg.); and Follow-Up and Monitoring. The distribution of active MPMOI projects across these phases, as well as completed projects, is presented in Table 5. The anticipated capital investment of these projects is $120 billion to $140 billion. It was estimated that these projects would create more than 100,000 jobs in Canada.Footnote 36
Some projects enter the process via a pre-submission phase, and are being overseen through the MPMOI process even before they formally enter the federal regulatory system. See Annex D for a detailed description of each of the phases.
|Phase||June 2008||Jan 2009||June 2009||Jan 2010||June 2010||Jan 2011||Apr 2011||Aug 2011|
Source: Compiled from DM Committee Meeting “Project Issues and Updates” decks (various months).
It should be noted in Table 5 that at the time of this evaluation (2011-12) three of 70 projects had been completed and none of the three completed projects had been implemented.
The main objective of this evaluation was to assess the performance and relevance of the Initiative according to the Treasury Board Policy on Evaluation with regard to the Initiative’s targeted outcomes, and provide recommendations as necessary. The evaluation results will also provide input to the renewal of the Initiative.
The evaluation of the MPMOI covered the five-year period from fiscal year 2007-08 to 2011-12. The scope included all the activities of the MPMOI, including those undertaken by the MPMO and by individual departments/agencies – individually or in collaboration. The evaluation also took into account the horizontal, multi-department/agency nature of this initiative, as outlined in its Horizontal Results-Based Management and Accountability Framework.Footnote 37 In particular, it examines the relevance and performance of the MPMOI by addressing five main issues:
- Issue 1: Continued need for the program (R1);
- Issue 2: Alignment with government priorities (R2); and
- Issue 3: Alignment with federal roles and responsibilities (R3).
Performance (effectiveness, efficiency and economy) (P):
- Issue 4: Achievement of expected outcomes (P1); and
- Issue 5: Demonstration of efficiency and economy (P2).
These five evaluation issues are addressed through a series of specific evaluation questions, which are answered in Section 4.0.
The evaluation approach and methodology were informed by the results of an evaluation assessment, which included a data quality and availability assessment, as well as in-depth consultations with internal MPMOI stakeholders. In addition, the methodology took into account feedback from the Evaluation Advisory Committee (EAC)Footnote 38 and from the MPMOI Interagency/Interdepartmental Evaluation Advisory Committee (IEAC).Footnote 39 During both the evaluation assessment and the development of the methodology, feedback was collected from the EAC and IEAC through a comprehensive consultation process which included frequent meetings, a half-day roundtable workshop, a series of individual consultations with participating departments/agencies, as well as written input. The IEAC agreed to an evaluation approach and methodology option proposed in the evaluation assessment report. This option and its combination of methods was the basis for the approach and methods used in the evaluation.
The key findings and recommendations presented in this report were drawn from multiple lines of evidence collected and analyzed using the following four methods:
- Document, file, and data review: a review of more than 1,000 documents, files and data sets collected from the MPMO and participating departments and agencies;
- Stakeholder interviews: telephone and in-person interviews with 1) internal stakeholders (senior management including MPMOI governance committees, MPMO senior staff, and department/agency managers) and 2) external stakeholders (industry associations, provincial representatives, Aboriginal groups, other stakeholders i.e., environmental non-government groups (ENGOs).
- Literature review and international comparative analysis: a review of literature on international practices relating to regulatory/approval systems for major resource projects, followed by a detailed comparative analysis with the approval system of Western Australia, which included a review of documents and data (both internal and public) and interviews with selected representatives;
- Case studies (comparative design): detailed reviews of ten major resource projects, (seven MPMOI and three non-MPMOI projects), half of which were initiated pre-MPMOI and half during the MPMOI era. The distribution of projects for the comparative design is shown in Table 6 (the basic characteristics of the ten projects are presented in Table 10, Annex E). Each case study included several interviews (internal and external stakeholders) and the review of project-level documents, files, and data.
Table 6 Distribution of projects for the comparative case study design
|MPMOI projects||Three MPMOI projects initiated before the Initiative and subsequently included in the MPMOI process:
||Four MPMOI projects initiated during the Initiative:
|Non-MPMOI projects||Two non-MPMOI projects initiated before the Initiative (would have qualified if the MPMOI had existed at the time):
||One non-MPMOI project initiated during the Initiative (would have qualified to be an MPMOI project but was not included for other reason):
In total, 105 interviews with senior management, MPMO senior staff, department/agency managers, other staff from participating federal departments/agencies, industry representatives, proponents, Aboriginal representatives, and provincial representatives were conducted, as well as interviews with representatives of the government of Western Australia. The distribution of internal and external interviews across methods is shown in Table 7.
Table 7 Distribution of internal and external interviews across methods
|Comparative analysis with Western Australia||-||4||4|
The findings and conclusions of this evaluation are presented by evaluation question. A summary response is presented for each evaluation question, using a rating as appropriate (see Table 8), followed by an overview of the supporting evidence.
Table 8 Summary of response ratings
|Great||Exceeds targeted results/expectations (A+)|
|Large||Meets targeted results/expectations (A)|
|Good||Meets most targeted results/expectations, some gaps/inadequacies (B)|
|Moderate||Meets some targeted results/expectations, gaps/inadequacies (B-/C)|
|Low||Significant gaps and inadequacies (D)|
|Very low||Fundamental gaps and inadequacies (F)|
3.4 Evaluation Challenges, Limitations and Mitigation Strategies
Complexity and nature of the Initiative
- Integrating evidence on a complex and multi-faceted horizontal initiative: The horizontal initiative represents a complex national system with significant variability across provinces, involving a wide range of key actors and stakeholders groups.
- MPMOI's short history and demonstration of outcomes: Only a few projects have completed the full review under the Initiative. Consequently, the methods used and evidence collected through this evaluation revealed that it is too early to demonstrate the achievement of expected intermediate and long-term goals of the Initiative. Data availability and quality issues have prevented full implementation of a quantitative comparative design to assess the level of Initiative achievement towards expected longer term outcomes.
- Lack of guidance, standard approaches and/or models for evaluating government policy functions: The evaluation of policy programs is a relatively new field. The federal evaluation community is currently working collaboratively to share and compare emerging applications and solutions in order to better serve this new “evaluation client”.Footnote 40
- Attribution of MPMOI inputs, activities and outputs to expected results: The expected results of the MPMOI are influenced by many internal and external actors and factors. The horizontal nature of MPMOI activities, as well as the fact that there are often concurrent initiatives or changes within the federal government and beyond (e.g., court decisions) that influenced implementation of the MPMOI, make it difficult for the evaluation to clearly (or fully) attribute measured outcomes to MPMOI activities and funding. As such, the evaluation, when possible, assessed the contribution of the Initiative to the achievement of the expected results shared with or affected by other initiatives.
- Literature review: One limiting factor in the literature review process was the accessibility of information. For example, data was only available in languages other than Canada’s official languages in some jurisdictions, and the quality of publicly available data, as well as across the reviewed jurisdictions, were recognized challenges. A second limitation was that some jurisdictions provided a coherent overview of their environmental regulations and review processes but offered very little integrated information on the remainder of the regulatory process for major resource projects.
- Comparative analysis with Western Australia: With regard to the comparative analysis between Canada and Western Australia, a fundamental limitation is that, despite significant similarities between the Canadian MPMOI and the Western Australian EA/regulatory review system, these two initiatives are not perfectly comparable. Other than the fact that the MPMOI is a federal initiative and the Australian system operates at the state level, the main difference is that there is no true equivalent to the Major Project Management Office in Western Australia. Rather, the various roles of the MPMO are shared across a number of different departments and agencies. This limitation was acknowledged throughout the preliminary and final comparative analysis. It should be noted that the United States (U.S.), the United Kingdom (U.K.) and New Zealand were also considered for this analysis; however, the context and characteristics of reforms in Western Australia were found to be the most comparable to the MPMOI.
Efforts to relate quantitative data between these jurisdictions were also challenging, as processes were often defined differently (e.g., the definition of the EA/EIA phase in Canada vs. Western Australia) or are simply different. Additional key challenges relating to the comparability of data included:
- Different contexts in which the EA and regulatory processes occur in each jurisdiction (e.g., federal vs. state; different roles and responsibilities of departments/agencies; multiple responsible authorities vs. single decision-making authority; different legislation; presence/absence of the duty to consult Aboriginal groups; different prevalence in the use of litigation by special interest groups).
- Recent changes to the regulatory processes: In both jurisdictions, it is likely too early to quantitatively measure if a consistent change has occurred as a result of improvements to review processes starting in 2008-09. These reforms have also improved the collection/quality of data, which further complicates the comparison of data before and after the changes, as well as between the two jurisdictions.
- Different governance structure and responsibilities: There are a number of important differences between the MPMO and governance mechanisms overseeing the regulatory system in Western Australia.
- The detailed financial data necessary for an in-depth cost-effectiveness analysis across the regulatory system (i.e., including EAs and regulatory/permitting processes) was not available for either the MPMOI or from Western Australia.
Finally, it was not possible to include a second comparison with a similar Canadian initiative within the scope of the evaluation, such that comparisons with other jurisdictions with regard to performance largely rely on those with Western Australia.
Document and file review
- Quantitative data and compilation across participating departments/agencies: For the evaluation of horizontal initiatives, the collection of documents, files and data from several sources (including, in some cases, from several locations within individual departments/agencies, e.g., regional offices) often involves challenges associated with obtaining information that is collected, compiled or identified in the same way across various departments. For the evaluation of the MPMOI, the collection of data on capacity (i.e., resources, FTEs) across the Initiative was the main challenge in this regard. Because of the large variations in the availability/quality of data across departments and agencies, it was not possible to collect and compile adequate quantitative data (financial/FTEs) for the detailed analysis of resource use (e.g., by type of activity, by project) across the Initiative.
This task might have been possible with an enormous amount of effort to collect, compile and validate the data, but this level of effort was not within the scope and timeline of the evaluation. Financial/FTE data could not be disaggregated by type of activity and/or by project managed under the MPMOI, much less for projects managed in parallel or previous to the MPMOI, which would have been ideal for comparative purposes.
In addition, neither baseline information on resources that was complemented by MPMOI funding nor estimates of in-kind contributions to MPMOI outcomes could be obtained from each participating department/agency. These data would have enabled a convincing assessment of the contribution of MPMOI funding and non-MPMOI funded activities to MPMOI outputs and outcomes. Thus, despite early efforts in the evaluation assessment phase and additional follow-up with individual departments/agencies to obtain such data, only high-level data relating to MPMOI funding (planned/actual) could be obtained and used in this evaluation.
Other data that could be obtained (e.g., estimate of MPMO expenditures per project) were used to the extent possible, but with the clear acknowledgement that these were not comprehensive. The lack of availability of financial and resource data limited the evaluation team’s ability to conduct several measures or analyses related to efficiency and cost-effectiveness.
- Lack of baseline data/comparable data: As noted above, there was also a lack of baseline data and comparable performance/timeline measures collected previous to the MPMOI, which limited the strength of conclusions based on quantitative data with regard to timeline improvements seen for projects under the Initiative. Program and project documents and files did not necessarily provide information or evidence to show: a) what activities actually occurred; b) whether these activities had had an impact (including both expected and unexpected outcomes); and c) the relationship between activities and outcomes (attribution). Thus, information from the document and file review was considered in combination with primary data from other lines of evidence (e.g., interviews, case studies) in order to adequately address the evaluation issues and questions.
Stakeholder interviews (and case study interviews)
- Short interview campaign and summer period: Delays encountered throughout the extensive consultation processes during the evaluation assessment phase and design phases delayed the initiation of evaluation fieldwork by several months. Consequently, the main challenge encountered with regard to the interviews was the short timeframe for both scheduling and interviewing, which also fell during the summer period. As a result, a number of interviewees were not available during the fieldwork period, declined to be interviewed or did not respond promptly to interview invitations. Additional challenges were encountered with regard to turnover of individuals (typically because of retirement and/or changes in their position/organizations). In all cases, back-up candidates were rapidly identified and invited to participate. Thus, the 46 interviews were completed as planned during the fieldwork period.
- Wide range of stakeholders groups and planned number of interviews: This limitation related to the range of different stakeholder groups for the MPMOI and the high number of potential interviewees, both within and outside the federal government. The number of suggested interviewees (many of whom were identified as “priority” candidates by IEAC members) far exceeded the number of planned interviews, particularly for department/agency representatives and industry associations. Thus, several individuals who would have been able to provide views on the MPMOI could not be consulted. However, given that – when case study and comparative analysis interviews were included – 105 interviews were conducted across the range of identified stakeholders groups, this limitation was not expected to affect the overall findings of the evaluation. A survey would have helped to mitigate this limitation; however, this method option was not selected by the evaluation team and the IEAC because of the complexity of administering a survey to such different groups of stakeholders.
- Interview/data bias: The wide range of MPMOI stakeholders had another related implication regarding potential bias among stakeholders selected to provide evidence/data for the evaluation. However, this was largely mitigated by ensuring that different types of internal and external stakeholders were interviewed (including during the case studies). Moreover, internal stakeholders included primarily individuals outside of the MPMO, representatives from both funded and unfunded departments/agencies, and across levels within individual departments. While contrasting views were often obtained from within certain sub-groups (which provided insight into the various contexts in which the MPMOI operates), similar information was often obtained across groups (e.g., internal vs. external, funded vs. unfunded, senior management vs. technical staff). In the latter case, the findings were considered as stronger than when groups did not provide the same evidence. This was further mitigated by including comparisons with other jurisdictions and by cross-referencing evidence from the literature/file review and case study document review. That said, most of the quantitative data regarding the performance of the MPMOI was, of necessity, obtained from the MPMO.
4.0 EVALUATION FINDINGS
4.1.1 Continued Need for the Program (Issue 1)
All lines of evidence support a strong need for the continuation of the Initiative as many of the systemic and capacity issues that triggered the creation of MPMOI are ongoing and significant, despite recent improvements to the federal review system. When expressing their support for the renewal of the Initiative, internal and industry stakeholders cited the ways in which the MPMOI had been meeting their current and future needs (e.g., addressing systemic issues within the federal regulatory system through system-wide approaches; capacity funding to ensure the efficiency and effectiveness of EAs, regulatory reviews and Aboriginal consultations). The continued need for the Initiative is also especially critical given the projected increase in the number of major resource projects entering the federal review process in the near future.
Question: Is there a continued need for the MPMOI in its current form?
Yes, if ongoing focus on continued improvement is maintained. Many of the systemic issues that triggered the creation of MPMOI are ongoing and significant, despite a number of recent operational and policy improvements, as well as legislative changes that are attributable to the Initiative. To address these ongoing issues, there is a clearly articulated need for continued system-wide coordination and governance – including senior management involvement – that supports policy leadership, as well as performance monitoring and management. Meanwhile, the demands for S&T capacity and capacity for Aboriginal consultations remain and are expected to increase as a result of the foreseen surge in the number of major resource projects entering the federal review process within the next five to ten years.
There is strong evidence across all lines of evidence of the need to further improve the federal regulatory system for major projects, including Aboriginal consultations, and therefore on the need for the Initiative to continue its work. Despite recent improvements that have been attributed to the MPMOI and/or to recent legislative changes (e.g., JEGA, CEA Act amendments), many of the systemic issues that triggered the initiation of this Initiative are ongoing and are significant in the sense that fundamental changes to the federal regulatory systems will be needed to address them.
For example, the case studies clearly illustrate that coordination and timely delivery of EA and regulatory review processes across multiple departments/agencies remains challenging due to the multiplicity of internal and external factors involved (including multiple RAs). Case studies also illustrate that the integration of Aboriginal consultations is a critical element of the system, but the whole-of-government approach is still a “work in progress”. Moreover, case study data demonstrates that overlapping or misaligned federal and provincial processes are a major barrier to efficient project reviews. Fundamental changes moving toward a “one project, one review” process would help address these issues.
The Initiative has received strong support for its renewal efforts, both from participating departments/agencies and from industry. Several associations have signed or written letters to encourage the federal ministers to support its renewal.Footnote 41,Footnote 42 The majority of consulted industry stakeholders articulated a strong need for more extensive systemic and/or legislative changes to the regulatory review process, whereas internal stakeholders and internal policy research suggested that both legislative and non-legislative changes would be necessary. Several internal interviewees also noted that if a number of fundamental changes took place, some of the aspects of the Initiative would become less relevant. In that case, they suggested that the Initiative be re-negotiated or an “exit strategy” developed. In the meantime, ongoing and additional work with provincial jurisdictions was seen as highly necessary by both internal and external stakeholders (especially industry), and some believed that the MPMOI, and the MPMO in particular, should play a considerable role in harmonizing the federal and provincial project review processes.
System-wide collaborative efforts involving horizontal policy leadership are generally perceived as an effective approach to realize further improvements. The Initiative’s policy-related achievements to date strongly support this finding. The need for and relevance of this approach is further supported by a current international trend that was observed in the comparative analysis: system-wide reforms of EA/regulatory review processes – many of which draw on collaborative/horizontal approaches – have been underway in jurisdictions such as Western Australia, South Africa, New Zealand, and the United Kingdom.
Related to this point, interview data strongly indicated that one of the MPMOI’s most important roles was providing system-wide coordination of the federal review process. The most frequently cited achievements of the Initiative often related to the horizontal senior governance mechanisms – the DM Committee in particular. For example, policy leadership, priority-setting, performance monitoring, and issue identification and management were frequently-mentioned benefits of the Initiative for which system-wide coordination and governance was viewed as essential. The vast majority of internal and external stakeholders stated that this feature of the Initiative should be sustained in the future to ensure effective and efficient project reviews.
In addition to the continued relevance of the broad objective of the Initiative, multiple lines of evidence indicated that there was a clear need to maintain each of the three main MPMOI components: S&T capacity funding and Aboriginal consultation capacity funding (see below); policy leadership; and project management, tracking and reporting.
Related to capacity needs, industry stakeholders as well as economic projections foresee an increase in the number of major resource projects undergoing federal review within the next five to ten years. The number of MPMOI projects doubled between 2008-09 and 2010-11 and the 73 active projects managed under the MPMOI in August 2011 represented approximately $120 billion to $140 billion in potential capital investment.Footnote 43 In the next ten years, estimates by the MPMO suggest that there may be up to 490 new mining and energy projects entering the federal regulatory system, representing a projected investment of about $400 billion. A similar exercise of the Mining Association of Canada outlining the “Canadian Mining Opportunity 2011-2016” for major mining projects estimates that these represent at least $115 billion in capital investment.Footnote 44 In light of these projections and of the MPMOI’s recognized role in improving the effectiveness and efficiency of the federal regulatory systems, the Initiative is seen as necessary to support the competitiveness of the Canadian resource industry.
Capacity needs were seen to be ongoing and growing. MPMOI funding for S&T capacityFootnote 45 to administer the CEA Act was repeatedly cited by internal and industry interviewees as essential to achieve the Initiative’s efficiency and effectiveness objectives. The benefits of capacity funding and the effects of capacity constraints were also observed in the case studies. Capacity requirements in the future were expected to increase in light of the current and anticipated increase in major resource projects, not only for S&T capacity, but also for Aboriginal consultations.
Capacity funding for Aboriginal consultations in the context of EAs and regulatory reviews would also continue to be particularly critical to ensure these responsibilities were adequately fulfilled given that this evaluation’s case studies and interviews revealed that capacity funding demands were increasing. This growing demand was not only caused by the increasing number of projects, but also by the higher than expected workload related to Aboriginal consultations. Several interviewees also noted that recent adjustments to departmental resources (i.e., budgets, staff) placed added pressure on internal capacity to deliver the Initiative’s outcomes.
As discussed in later sections, tools and processes developed under the MPMOI were found to have supported various improvements in the federal regulatory system, having undergone continuous improvement in the initial years of the Initiative. Several internal interviewees stressed that ongoing continuous improvement of these tools and processes was needed to ensure that they remained appropriate in light of the growing maturity of the Initiative and recent changes to the federal regulatory system (e.g., new regulated timelines for CEA Agency-led comprehensive studies). In general, the tools and processes that were suggested for review were often the same as those seen as duplicative or having high transaction costs.
For example, project monitoring/tracking on a weekly basis – and at the level of detail requested under the Initiative – was seen as being comparatively less relevant, particularly in the context of milestones: (a) not on the critical path; (b) those outside the control of the federal government; or (c) those being tracked separately under the new timeline regulations. In its management response to NRCan’s 2010 Audit of the MPMO, the MPMO had “committed to reviewing the adequacy and effectiveness of project management tools and approaches such as project agreements and the Project Description Guide on a regular basis, and as may be required to respond to important changes in requirements.”Footnote 46
Question: To what extent were the Initiative’s objectives, approach, activities and capacity funding tailored to address and satisfy the needs of participating departments/agencies and of the other main stakeholders?
To a good extent. The Initiative has been for the most part responsive to at least some of the needs of its stakeholders. However, the assessment of the responsiveness depended on stakeholders’ level of awareness of the Initiative’s objectives, activities and outcomes. MPMOI funding for scientific, technical and Aboriginal consultation capacity was the most frequently cited example by department/agency representatives. Industry perceives the Initiative as a “step in the right direction” in giving industry a voice and meeting its needs, but expects further major improvements. Documents confirmed that the MPMOI responds in several respects to industry needs. Aboriginal groups and provincial representatives usually perceived less of a connection between the MPMOI and their needs.
The Initiative’s objectives, approach, activities and capacity funding are responsive to at least some of the needs of the majority of organizations and groups representing both internal and external MPMOI stakeholders. Interview and case study data indicated that the extent to which the Initiative was perceived to be responsive to the needs of interviewees’ organizations depended primarily on their position relative to the Initiative (e.g., funded versus unfunded departments/agencies, National Capital Region versus regions, industry association versus project proponent). This relationship determined the extent to which they were aware of the Initiative and its range of activities, as well as whether they had observed first-hand positive changes or benefits that they attributed to the Initiative.
Department/agency representatives were generally quite positive about the MPMOI’s responsiveness to their needs, particularly if they had been directly involved in collaborative work on operational or policy aspects. Most of the representatives of funded departments and agencies also indicated that MPMOI capacity funding had significantly enhanced their ability to deliver their department/agency mandate (and MPMOI efficiency and effectiveness outcomes) with regard to EAs, regulatory reviews and Aboriginal consultations. By bringing policy-makers and regulators from key departments and agencies to the same table, the Initiative was also seen to address and satisfy their policy needs.
This view was not shared by several internal interviewees who were not aware of the Initiative’s policy activities or role and who therefore did not attribute any related outcomes to the MPMOI. Similarly, department/agency staff working in regional offices evaluated the responsiveness of the Initiative relative to their day-to-day needs, such as those pertaining to EAs, regulatory reviews and Aboriginal consultations at the project level. In other words, if they did not see concrete benefits of the MPMOI for the projects in which they were involved, they generally did not view the Initiative as responsive to their needs.
Some internal stakeholders indicated that the Initiative’s responsiveness could improve if it was less process-driven and more outcomes-focused (including environmental outcomes), if some of the reporting and meeting requirements were reduced (allowing for more time and resources to be allocated to project or policy work), or if more resources were used for the education of proponents. Some interviewees (including external interviewees) who only perceived benefits associated with the supplemental capacity funding suggested that all MPMOI funding should be allocated to supporting S&T and Aboriginal consultation capacity within departments and agencies. However, this view was not supported by other findings of this evaluation.
The MPMOI – and the MPMO in particular – has received considerable praise and support from industry, which clearly considers that the Initiative has and will continue to respond to its needs. Industry representatives often have described the Initiative as a “step in the right direction” in terms of meeting its needs and in giving industry a voice, and have strong expectations as to further improvements and fundamental changes that they believe could be delivered through the Initiative.
These views were often heard in the interviews and were clearly articulated in a letter addressed to the Minister of Natural Resources Canada and signed by nine resource industry associations, among other documents.Footnote 47 In this letter, industry associations advocated for the renewal of the Initiative, including the following elements: 1) funding for the MPMO, as well as capacity funding for departments/agencies (particularly the CEA Agency); 2) continued DM Committee coordination; 3) continued capacity funding and improved processes relating to Aboriginal consultations; and 4) further process and timeline improvements. Industry associations also stressed the need for the MPMOI to continue work towards “one project, one review”, by addressing fundamental issues with the CEA Act and other Acts (e.g., the Fisheries Act), reducing duplication with provincial processes, and working on a new integrated framework of modern environmental legislation.
A recent Red Tape Reduction Commission report that examined industry views on how to promote the growth and competitiveness of Canadian businesses through the streamlining of government processes, confirmed that the Initiative aligned with several industry needs, such as greater transparency, predictability and accountability; increased interdepartmental coordination (and reduced duplication); and provision of an effective point of contact to resolve issues that span multiple regulating bodies.Footnote 48
The case studies included in-depth interviews with project proponents, who provided mixed evidence regarding the Initiative’s responsiveness to their needs. The fact that the Initiative was able to engage various departments/agencies and enhance communication among them contributed to a regulatory system that better met the needs of proponents. Additional capacity funding was also seen as a positive aspect.
However, some proponents indicated that processes for federal project reviews “added a layer of bureaucracy” and did not think that timelines were significantly different from those occurring pre-MPMOI. These individuals pointed to ongoing issues within the system and considered that the process had not been adequately coordinated or sufficiently streamlined (including coordination with provincial processes), that the predictability of the process had not increased, or that the roles of individual departments and agencies had not been significantly clarified or more focused on their mandates.
Provincial representatives and Aboriginal groups did not think that the Initiative had been addressing their needs, especially if they did not see any connection between the MPMOI activities and their needs. Most of these interviewees indicated they had little contact with the MPMO, and often limited awareness of the objectives and activities of the Initiative. Aboriginal groups identified specific needs with regard to the federal regulatory system, a number of which they consider as not being met by the current consultation approach (i.e., process concerns and consultations that address their constitutional rights).
4.1.2 Alignment with Government Priorities (Issue 2)
Question: To what extent is the Initiative consistent with:
a) current government priorities?
To a great extent. There is strong alignment between MPMOI objectives and commitments made by the GoC (e.g., past budgets, speeches, Cabinet directives, etc.). Budget 2010 and the 2011 Speech from the Throne confirm that the MPMOI’s objectives remain a key priority for the federal government.
In the 2007 Speech from the ThroneFootnote 49 and in Budget 2007,Footnote 50 the government established its priorities to streamline the review of large resource projects and increase the competitiveness of the Canadian resource industry. These goals became the task of the MPMOI, which was launched in late 2007 as per the Cabinet Directive on Improving the Performance of the Regulatory System for Major Natural Resource Projects.Footnote 51 This Directive further articulated the government’s commitment to address “the challenges facing the federal regulatory system for major resource projects in order to help improve Canada’s competitiveness and enable more effective examination and mitigation of environmental, human health and social impacts.” This Directive indicated strong government support for the MPMOI’s efforts “to improve the effectiveness and efficiency of the regulatory system for major resource projects.”Footnote 52
Documents (including internal reviews) and interviews provided ample evidence to confirm the ongoing alignment of the Initiative with government priorities since 2007. In Budget 2010 and in the 2011 Speech from the Throne, the government confirmed that improving the EA and regulatory review process for resource projects, while ensuring meaningful consultation with affected groups (in particular Aboriginal communities), remained key ongoing government priorities.
For example, Budget 2010 provided additional resources to participant funding for Aboriginal consultations and announced a number of measures to help “modernize” the federal regulatory system for project reviews, given that “major resource-based and energy projects are significant economic investments, creating direct and indirect jobs and providing important economic development opportunities for many communities in Canada.”Footnote 53 The 2011 Speech from the Throne also highlighted the need to “engage the provinces, territories and industry on ways to improve the regulatory and EA process for resource projects”.5454
Internal interviewees pointed to the above evidence and stressed that major resource projects – often worth billions of dollars – represented an important part of the Canadian economy (i.e., in terms of jobs and gross domestic product), which had been and remained key government priorities. The MPMOI was also perceived as being aligned with other important government goals and commitments through its focus on reducing timelines (i.e., streamlining regulatory processes), on improving Aboriginal consultation practices, on ensuring environmental protection, as well as on engaging external stakeholders (e.g., industry and provinces) to improve the regulatory system.
b) the mandates and strategic objectives of participating departments/agencies?
To a large extent. The Initiative is consistent with currently formulated mandates and strategic objectives of participating departments/agencies (both funded and unfunded via the MPMOI). The MPMOI provided capacity funding to support or fulfil the regulatory and statutory responsibilities for major resource projects of these departments/agencies, and was also designed to assist them to improve related processes. Some evidence suggests that the Initiative could increase its alignment with the departmental and agency objectives if it continues to seek a greater balance of the three core mandates – environmental, economic and Aboriginal.
A comprehensive document review of MPMOI objectives with regard to the mandates, objectives and responsibilities of participating departments and agencies showed that the Initiative was generally strongly aligned with the departmental and agency mandates. This was consistent with the interview data of internal interviewees. The MPMOI supported funded participating departments/agencies in fulfilling their mandates through capacity funding for major resources project EAs and regulatory reviews. The fact that the activities supported by MPMOI capacity funding would have had to be carried out even in the absence of the MPMOI also underlined the Initiative’s consistency with department/agency mandates and strategic objectives.
Furthermore, legislative changes such as the CEA Act amendments and substitution provisions introduced through the Budget 2010 helped enhance the roles of the CEA Agency, NEB, and CNSC. The MPMOI also assisted the unfunded departments and agencies (CNSC, NEB and NRCan) by providing a forum for collaborative operational and policy improvements.
Several internal interviewees noted that adjustments to the Initiative could help better address certain elements of their department’s mandates (i.e., sustainable development; environmental protection; more attention to the regulatory decision-making phase as well as EAs). For example, several interviewees stressed that the Initiative should place a greater focus on effectiveness outcomes (e.g., environmental performance measures, follow-up and monitoring activities) as well as process efficiency.
More generally, consistent with other findings discussed in this report, they expressed concern that the environmental mandates of federal departments were being overshadowed by economic priorities (and industry interests) and indicated that the Initiative should strive to establish a greater balance between these two mandates. In the words of one senior-level department representative, “All departments have an environment[al] protection mandate, so the Initiative has to embody both efficiency and effectiveness parts. Now, because it’s so focused on efficiency, many departments come with their efficiency lens, not their environmental management lens. We have to refocus on environmental effectiveness.”
As Table 9 indicates, all of the departments and agencies have environmental protection either as their core mandate or as a part of their responsibilities. Almost all of the departments and agencies also play an important role in the economic development of Canada. Finally, most of them are mandated to consult or collaborate with Aboriginal groups to help protect and develop Aboriginal communities and resources. This overview of the mandates supports the view of some interviewees regarding the need to focus equally on all three areas of responsibilities.
|CEA AgencyFootnote 56|
Note: - core mandate; - a part of the mandate
Source: Compiled from information retrieved from official websites of the departments/agencies.
4.1.3 Alignment with Federal Roles and Responsibilities (Issue 3)
Question: Given the federal responsibilities with respect to the administration of the
Acts involved in the regulatory processes for major resources projects, is there a legitimate and necessary role for government in this Initiative?
Yes. The government’s role in the Initiative is fully appropriate given the government‘s statutory and regulatory responsibilities relative to EAs and regulatory reviews, as well as a constitutional duty to consult Aboriginal groups. The GoC is also responsible for improving its policies and processes relative to these tasks. In terms of the clarity of roles and responsibilities, some confusion persists regarding Aboriginal consultations and the role of AANDC. NRCan is confirmed to be the most appropriate lead department for the Initiative, given its comparatively lower level of regulatory enforcement and assuming a continuing neutral role for the MPMO.
The majority of internal and external stakeholders agreed that the government’s role in the Initiative was both legitimate and necessary, as it was the government’s responsibility to ensure that the numerous and complex federal Acts relating to EAs and regulatory reviews (Figure 1) were being administered in an effective and efficient manner. Departments and agencies were therefore responsible for the processes put into place to do so.
Document review and interview data showed that the key federal departments/agencies involved in the federal regulatory system for major resources projects also had a legitimate and necessary role in policy decisions, by coordinating and informing efforts to improve these Acts. The government had similar responsibilities relative to the Crown’s duty to consult Aboriginal groups (as a result of Section 35 of the Constitution Act (1982) and related rulings from the Supreme Court of Canada): more specifically, it had to act to ensure that these consultations were conducted in an adequate and meaningful manner. Some interviewees also noted that the government needed to ensure that the review process was coordinated across all the federal departments and agencies involved, and – to the extent possible – with provincial organizations as well.
The interviews and case study data showed that there was a frequent lack of mutual understanding regarding federal roles and responsibilities relating to Aboriginal consultations, particularly around accommodation, Constitutional rights issues, as well as AANDC’s role more generally. Relating to specific elements of the Initiative, the roles and responsibilities of the MPMO and CEA Agency with respect to project management and coordination were also unclear for many interviewees (both internal and external), although this appeared to have improved as the Initiative matured.
Other issues raised in the interviews and observed in the case studies regarding the legitimacy of federal roles and responsibilities relating to the federal review process for major projects often centered on areas of shared jurisdiction (e.g., water), and highlighted certain tensions in areas under the joint responsibility of federal departments and provincial governments (e.g., environment, wildlife).
Finally, NRCan was considered by most internal interviewees as an adequate lead department for the MPMOI, and suggested alternatives were generally presented as imperfect in some way. A key characteristic of NRCan that supported its role as lead department was that it had a more limited regulatory function than other participating departments and agencies, allowing it (and particularly the MPMO) to play a more “neutral” role.
That said, some external stakeholders (especially those outside the NCR or who had little contact with the MPMO) had traditionally had more contact and a closer relationship with the CEA Agency. They indicated that the Agency would be a better place to host the Initiative as “it already does a lot of the work.”
It is important to note, however, that the Agency is only involved in EAs (and has an increasingly limited role in EAs led by the CSNC and NEB), so it is not as well placed as the MPMO within NRCan to address system-wide issues that include the regulatory decision-making phase.
Another alternative identified by several interviewees was the Privy Council Office (PCO), but few argued that PCO should now take over the Initiative from NRCan. Others pointed out that PCO’s position within government might pose a disadvantage. Regardless of who hosts the Initiative’s centralized function, many stakeholders cited the need to for the MPMO to maintain an independent position similar to that offered by the current MPMO, as essential to ensure the outcomes of the Initiative.
4.2 Performance (Effectiveness, Efficiency and Economy)
4.2.1 Achievement of Expected Outcomes (Issue 4)
Participating departments/agencies and the MPMO, though the MPMOI, have effectively implemented a wide range of processes, most to a large or great extent and some to a good extent, compared to initial plans. The implementation of these processes has clearly contributed to a high level of achievement of outcomes relating to Aboriginal consultations (e.g., improved consistency) and collaborative policy research. The integration and federal coordination of EAs and regulatory reviews were also found to have increased under the Initiative, but the demonstration of the extent to that these improvements had, to date, had an impact on timelines of the process was somewhat limited. A number of internal and external factors were found to have had a significant influence on these outcomes, creating both challenges and opportunities for the Initiative (i.e., to address and/or benefit from these factors).
Meanwhile, transparency and accountability of the federal regulatory process within the federal government was found to have increased significantly through the MPMOI, whereas the impacts of efforts to increase transparency outside the federal government were less clearly demonstrated. Additional data was also needed to confirm preliminary findings that improvements to process efficiency had also supported the maintenance or enhancement of the quality and effectiveness of EAs and regulatory reviews. The MPMOI was seen to have led to benefits that extended beyond the Initiative, such as a “catalyzing” effect for further process improvements within federal departments/agencies, as well as developing an exceptional level of horizontal dialogue at senior levels.
Overall, the short-term outcomes of the MPMOI are generally positive, but both internal and external stakeholders expect further results as the Initiative gains momentum.
Question: To what extent have the processes implemented under the Initiative been effective in:
a) providing a single point of entry into the federal regulatory process?
To a good extent. Although the original intent was to have the MPMO act as single point of entry for all stakeholders, this role was adjusted to accommodate existing relationships (e.g., with the CEA Agency). The MPMO mainly acted as a contact point for industry stakeholders to facilitate issue resolution, and supported early engagement with proponents.
The role of the MPMO as a central point of contact for stakeholders of major resource projects has evolved from a broad to a more focused role since its inception. Original program documentation cites that the MPMO would represent “a single window into the federal regulatory process for industry” Footnote 61 and provide “all interested parties with timely information relevant to the federal government’s regulatory responsibilities for major resource projects.”Footnote 62 Internal and external interviews, as well as the case studies, suggest that early lack of clarity around this role has been largely resolved as other departments and agencies (particularly the CEA Agency) have effectively continued to leverage their existing relationships with – and in some cases, their regional proximity to – external project stakeholders (e.g., proponents, provincial organizations, Aboriginal groups) to ensure timely provision of relevant information. It is also worth noting that, under the whole-of-government approach to Aboriginal consultations, Aboriginal groups are encouraged to contact the Crown consultation coordinator assigned to specific major resource projects rather than the MPMO.
Having accommodated these new and existing relationships within the federal regulatory system, the nature of the MPMO’s role as a point of entry into the federal regulatory process has become clearer and more specific over time. In addition to offering the MPMO Tracker, the MPMO has been found to act mainly – and effectively – as a contact point for industry stakeholders to provide guidance, and particularly to facilitate issue resolution.
In the words of one interviewee: “Before, sometimes, we or others in the industry weren’t sure who to talk to. Now, if we are not sure, we know we go to the MPMO.” The MPMO has fostered relationships with industry associations, contributing to greater recognition among proponents of the assistance that it can provide (although proponents usually refer first contact to the CEA Agency or responsible authorities for project-specific issues; departments and agencies generally keep the MPMO informed). Case study data confirm other lines of evidence that the MPMO has conducted and supports early engagement with proponents (when possible). This includes the provision of guidance (verbally and through written materials) and assistance in setting up early contact and discussions between proponents and federal departments/agencies.
b) implementing a consistent, coordinated approach to environmental assessment and regulatory reviews (project agreements and service standards)?
To a large extent. A variety of tools and processes have been collaboratively developed and implemented by the MPMO and by participating departments/agencies, in order to support a more consistent and coordinated approach to EAs and regulatory reviews.
Multiple lines of evidence demonstrate that the implementation of a variety of tools and processes to support a more consistent, coordinated approach to EAs and regulatory reviews for major resource projects has been achieved through the efforts of the MPMO and – just as importantly – through the collaboration and individual efforts of participating departments/agencies.
For example, the document review and interview data identified the development of timeline-based service standards, a wide range of tools (e.g., project agreement templates, process maps, guidance materials,Footnote 63 internal and public project tracking tools, an early warning system, post-review evaluations), project management training, as well as detailed procedural documents for use both across the Initiative and within individual departments and agencies as MPMOI-originated coordination and performance tools. The development and implementation of these tools and procedures involved clarifying and formalizing the roles and responsibilities and lines of communication within the federal regulatory process.
The implementation of this approach necessitated a number of adjustments during the initial “learning curve” of the Initiative (e.g., project agreements), but the approach as a whole was found by interviewees and observed in the case studies to have become fairly consistent across major resource project reviews managed under the MPMOI. For example, as of July 31, 2011, at least 75 percent of active project reviews at the EA stage were supported by a signed project agreement. According to the MPMO website, six additional project agreements had since been signed.Footnote 64 To help address the challenges involved in implementing a consistent approach, the MPMO had conducted periodic reviews of service standards and of specific aspects of the federal process to identify areas for improvement. As a result, it operationalized more effective and efficient practices. These efforts at continuous improvement benefited from the support and guidance of other departments/agencies – in particular, that of senior level committees.
c) integrating Aboriginal consultation responsibilities into the overall regulatory process and maintaining Crown records?
To a large extent. Strong evidence shows that Aboriginal consultations activities have been better integrated into the overall process via the “whole-of-government” approach. Crown record management has been a greater challenge than expected and is gradually improving.
The whole-of-government approach implemented through the MPMOI (with the involvement of AANDCFootnote 65 and the DOJ) has significantly improved the integration of Aboriginal consultation responsibilities into the overall regulatory process. Multiple lines of evidence (i.e., document review, interviews, and case studies) indicated that this had been achieved through:
- the development of consultation work plans;
- identification of Aboriginal consultation responsibilities and tasks in project agreements;
- increased guidance and training relating to consultations (including guides for federal officials and for proponents of major resource projects); and
- the development of new roles: the Crown Consultation Coordinator (CCC) and the Crown Oversight (CO) Committee.Footnote 66
Moreover, a dedicated Aboriginal Funding Envelope was created to support the participation of Aboriginal groups in comprehensive studies of MPMOI projects (funding was already available through the envelope for participation in review panels). In addition, legislative changes that received Royal Assent in 2010 would allow the NEB and CNSC to offer funding for Aboriginal participation in public hearing processes, thus better integrating consultations into their processes and reducing duplication by facilitating the substitution of NEB and the CNSC review panels instead of joint review panels with the CEA Agency.
The whole-of-government approach remains fairly new and is still evolving in order to meet a number of remaining internal and external challenges to the delivery of consistent, adequate and meaningful Aboriginal consultations. One process that has yet to be fully implemented in light of internal challenges is the management of Crown consultation records by the MPMO as the records for many projects are not yet housed by it. The reasons behind this are well-documented (e.g., issues with the quality, management and transfer of records, resulting in inconsistent documentation). A new online database developed by the MPMO has recently resulted in partial achievement of this outcome (especially for NEB projects for which the MPMO chairs the Crown Oversight Committee).
d) monitoring, tracking & reporting on performance?
To a large extent internal to the federal government; to a moderate extent external to the federal government. Internal to the federal government, monitoring, tracking and reporting of the federal review process have greatly increased through the Initiative. From an external standpoint, the establishment of the public MPMO Tracker was an important first step, but a number of limitations were observed regarding the usefulness and use of this tool. Performance information is needed on outcomes related to environmental assessment, regulatory review and Aboriginal consultation.
Internal to the federal government, through the MPMOI, a range of tools to monitor, track and report on performance have been implemented for a range of audiences within the federal government (horizontal and vertical). These include the Weekly Tracker reports (disseminated across all levels of departments/ agencies), monthly project-related decks for the DM Committee, and quarterly reports to ministers.
The MPMO Tracker, which offers both public and internal password-protected access, displays the start and completion dates for each project’s task and milestones are also publicly available on the MPMO website. Following early efforts to generate awareness of the Initiative (2008-09), the MPMO has more recently developed a Communications/Outreach Plan that includes supporting materials for dissemination. This plan has been put on hold during the evaluation period.
These tools support a much more extensive effort to gather and actively use performance data relating to the federal regulatory review of major resource projects than previously existed, allowing for more timely and informed decision-making. Data collection for this purpose is often not conducted through project management systems existing within participating departments/agencies. Thus, MPMOI projects involve additional reporting requirements as well as the work of project officers within the MPMO.
From an external standpoint, the MPMO website and public tracker have also increased the amount of publicly available information about the federal review process for individual projects. However, consistent with the findings of NRCan’s 2010 audit of the MPMO,Footnote 67 the evaluation found that there remained some limitations in terms of the information provided in the Tracker that reduced the ability of users to determine whether a particular review was meeting target timelines, and which department, agency or external party was responsible for each task/milestone. Other limitations outlined in the audit report (e.g., inaccuracies, poor alignment with other tools/sources of information) were largely addressed by the MPMO as part of the strengthened quality management system implemented by it in response to the audit’s recommendations.
Although visits to the Tracker website have clearly increased over time (from less than 1,000 visits per month in 2008 to nearly 5,000 visits per month in the first half of 2011), the limited web analytics data made available did not allow for the characterization of its use by external stakeholders for the purpose of understanding Tracker website visitor characteristics (i.e., one-time compared to recurring visitors, location) and behaviour (i.e., pages visited, time spent on pages). Thus, the increase of Tracker website visits cannot be used to conclude that the public is making greater use of this performance reporting tool. The case study findings suggested that this tool might not be widely used or found useful by stakeholders outside the federal government. Instead, external (and some internal) stakeholders reported that they generally consulted other sources (e.g., the CEA Agency or responsible authorities) to determine the current status of particular reviews.
Performance data is collected on various aspects of MPMOI activities, transactions and processes. Given that the Initiative is entering a more mature phase, the timing is appropriate to develop and implement performance measures related to MPMOI outcomes (e.g., environmental assessment, regulatory review and Aboriginal consultation).
e) establishing and supporting senior governance and oversight mechanisms?
To a great extent. All lines of evidence strongly supported the finding that effective senior governance and coordination mechanisms had been established through the MPMOI.
The range of new senior governance and coordination mechanisms – particularly the DM Committee – have been one of the most recognized and successful aspects of the Initiative. All lines of evidence used in this evaluation support the finding that the monthly DM, ADM and DG-level meetings have been effectively used to discuss and provide direction on project-level, strategic and policy issues.
More broadly, interviewees within and outside the federal government also strongly praised these committees as a means to ensure greater levels of awareness and accountability within the federal regulatory system, and to achieve changes both horizontally and vertically. Industry associations were strongly supportive of the MPMO and DM Committee as coordination and governance mechanisms. This was explained in part by the fact that representatives from industry associations were in contact with senior MPMO staff and DMs to regularly discuss project-specific and policy issues.
With regard to issue resolution at the project level, the tracking and reporting mechanisms administered by the MPMO – including the early warning system established in 2010 – have helped identify issues and propose short- and long-term solutions. For example, the DM Committee has discussed approximately 70 project-specific and 50 cross-cutting issues (Figure 4) since the establishment of monthly meetings in 2008. The MPMO has also been found to be effective in providing secretariat and information support to senior-level committees.
Several internal interviewees – from senior management to regional staff – saw opportunities for future adjustments to the governance structure to ensure the optimal use of both senior-level committees and coordination functions delivered by the MPMO. This was often framed as an efficiency issue, considering the level of administrative “burden” associated with this structure and the meeting frequency. Interviewees provided suggestions on ways to ensure the continued effectiveness of this structure that would also optimize the frequency and/or strategic focus of the meetings, including resolving issues at the lowest level possible rather than “bumping them up” to ADMs or DMs. Western Australia also used senior-level committees to increase accountability and resolve issues in a coordinated manner. However, the focus of these committees was more on performance and accountability, as well as on policy/strategic issues, rather than on project- and operational-level issues.
Figure 4 Cross-cutting issues discussed at DM Committee meetings
Source: MPMO (2011). Performance Measurement for the Major Projects Management Office Initiative. (Confirmed by a scan of DM Committee meeting agendas and projects/issues decks.)
f) conducting and coordinating policy research (to identify, evaluate and implement process improvements)?
To a great extent. Extensive policy research and collaborative work has been conducted on both short-term and long-term options for process improvements. In response to government needs, there has been some work done to explore topics and develop options beyond original plans/expectations.
From its inception, the Initiative involved concerted efforts – across departments/agencies and involving the active participation of senior-level committees (including a half-day DM workshop) – to identify and develop short- and longer-term options for process improvements relating to the federal regulatory system for major resource projects. This work comprised extensive horizontal collaboration involving participating departments/agencies, as well as input from external studies and stakeholder groups (e.g., industry, Aboriginal). The MPMO provided a central point for managing and coordinating this policy research. As noted in the interviews and document review, a number of legislative and non-legislative options had been collaboratively identified, developed and assessed with the support of the Initiative (the DM Committee in particular), and several had already been implemented.
The original plans for the Initiative focused primarily on the identification of short- and longer-term solutions from a federal standpoint, mainly through policy research and analysis.Footnote 68 In response to the initial “diagnostique” developed through the MPMOI’s policy research efforts, the Initiative was called upon by the government to support the development of concrete options on the path to a “one project, one review” process, including recent collaborative work with provincial governments regarding federal/provincial coordination.
Finally, both interviews and the review of internal documents confirmed that many of the policy achievements of the MPMOI had surpassed the initial expectations for the Initiative.
g) determining the eligibility and selection process of projects to be administered through the Initiative?
To a good extent. The selection criteria for “major resource projects” is now well documented and, for the majority of projects, well communicated and understood. However, the rationale supporting the inclusion of some “atypical” projects and existing projects brought into the Initiative was not always as clearly communicated or understood by stakeholders.
The eligibility and selection process for the Initiative was formalized first in initial program documentationFootnote 69 and further refined in the 2010 Guide to Identifying a Major Resource Project.Footnote 70 The latter was developed by the MPMO in collaboration with participating departments/agencies. This Guide clarified the term “major resource project” (MRP) and “provide[s] information on how the MPMO determines if a particular proposed project qualifies as an MRP.” Case study and interview data revealed that most projects administered through the Initiative were appropriately selected, mostly as a consequence of:
- the Guide;
- the growing level of familiarity with the Initiative; and
- the 2010 Supreme Court of Canada MiningWatch ruling: this ruling clarified the scoping for a number of major resource projects that were to be assessed as comprehensive studies (automatically MPMOI projects), rather than as screenings (which have to meet additional criteria for inclusion); given that a number of complex screenings included in the MPMOI became comprehensive studies as a result of this ruling, this supported the observation – as seen in the case studies and interviews – that most projects administered through the Initiative were adequately selected.
However, questions remained, both within departments/agencies as well as external stakeholders, as to why some projects were (or were not) included in the Initiative, especially those identified as “atypical” (i.e., mega-projects or those of a more modest size/complexity, as well as projects with unique characteristics) and a number of existing projects brought into the Initiative in early 2009.
In both cases, the rationale for the inclusion of these projects was often well-articulated in internal documents (e.g., DM Committee decks/minutes, spreadsheets used to identify and select existing projects for inclusion). However, this rationale appeared not to have been clearly and consistently communicated within departments/agencies. This had resulted in a lack of understanding within departments/agencies, as well as within industry, relating to the inclusion (or not) of some projects that might not as clearly meet the criteria outlined in the Major Resource Projects (MRP) Guide (i.e., “borderline” major projects, such that some non-MPMOI projects were seen to be highly similar to projects administered within the Initiative).
Question: To what extent was a consistent, whole-of-government approach to Aboriginal consultations implemented as a result of the Initiative?
To a large extent, but much remains to be addressed. The Initiative’s whole-of-government approach has led to positive changes for Aboriginal consultations. Nonetheless, this approach remains relatively new and must adjust to multiple (and changing) internal and external factors. Meanwhile, Aboriginal groups continue to express concerns with the consultation process.
All lines of evidence, and the case studies in particular, supported the finding that the whole-of-government approach was implemented as intended under the MPMOI and led to positive changes for Aboriginal consultations. This evidence pointed to increased consistency in the process due to integration of consultations in EAs and regulatory reviews, increased capacity to conduct consultations, as well as clearer internal roles (e.g., CCC, Crown Oversight Committee); and procedures within departments and agencies.
Examples of positive changes observed in some of the cases studies as a result of the implementation of this approach included: earlier identification and engagement of Aboriginal groups; better coordination of consultations and reduced duplication of consultation activities; and improved harmonization of consultations with provinces.
The case studies, comparing MPMOI projects to pre- and non-MPMOI projects also identified two strong findings to support the conclusion that the whole-of-government approach (as implemented/informed through the MPMOI) had led to these outcomes. First, uncoordinated Aboriginal consultation was identified as a major problem for pre-MPMOI projects, especially in the absence of the centralized CCC role. Second, two non-MPMOI projects were found to have improved their Aboriginal consultation processes from 2009 onward, when processes largely equivalent to the whole-of-government approach were implemented as under the MPMOI (e.g., CCC, participant funding, provincial harmonization).
However, interviews and document data highlighted the fact that the legal landscape relating to consultation was continually shifting as court decisions were still defining the nature of these consultations. Some recent decisions supported specific elements of the whole-of-government approach, such as the use of NEB processes to fulfil the duty to consult.Footnote 71
In addition, complementary procedures, tools and resources (e.g., efforts led by AANDC, TC’s Aboriginal Consultation Unit, new EC procedures) are being developed and implemented within departments/agencies. These efforts are not necessarily funded by or conducted under the direction of the Initiative, but are seen to be strengthening the approach as a whole and affecting its outcomes. Calculation of the contribution of these factors to Aboriginal consultation outcomes is not possible, but it appears that the Initiative has acted as a forum around which many of these factors and parallel efforts have been discussed, addressed, and/or facilitated – possibly in a more timely manner than if the Initiative had not existed.
While the improvements noted above are clearly significant, interviewees (both internal and external) agree that the federal approach to Aboriginal consultation is still quite new and remains “a work in progress.” Case studies data point to additional challenges relating to consultations that have yet to be fully addressed, many of which are influenced by internal or external factors.
For example, the transition from the EA to the regulatory phases (which includes the hand-off of the CCC role from the CEA Agency [when the Agency is the EA lead] to a responsible authority) has been identified as a key part of the process for which experience and clarity (both within and outside the federal government) are often lacking, resulting in gaps rather than overlap. Similarly, clarification is needed regarding federal processes relating to accommodation. As Aboriginal groups become more involved in the process (through increased interest and experience), “meaningful” consultations require more time and effort, both within the federal government and within Aboriginal communities. This has implications not only for capacity funding and participant funding, but also for the timing of milestones relating to Aboriginal consultations. Another ongoing challenge for the Initiative is finding the appropriate balance between efficiency objectives and adequate and meaningful consultation (i.e., based on strength of claim).
The case studies – which included interviews and the review of several documents relating to Aboriginal consultations – also showed that relationships among Aboriginal groups, the provinces and proponents had strongly influenced the results and other outcomes of the consultation and accommodation process, including the satisfaction of Aboriginal groups.
Integration across federal and provincial processes was cited as another ongoing challenge, especially given process and timing differences, varying interpretations of the “duty to consult” (e.g., strength of claim, inclusion of Métis groups). Thus, duplication with provincial processes was mentioned by many interviewees as an outstanding issue.
The challenges related to Aboriginal consultations were reflected in delays that occurred in past and current MPMOI projects. In the spring of 2010, at least four projects were identified by the MPMO as having experienced major delays on their respective critical paths due to Aboriginal consultation issues. In addition, a review of completed MPMOI project milestonesFootnote 72 revealed that nearly half of the tasks relating to Aboriginal consultations had been completed after the target date, and that the longest delays were observed for the following tasks/milestones:
- send First Notification Letter to Aboriginal Groups;
- distribution of project description to potentially interested Aboriginal groups;
- develop draft Aboriginal consultation work plan; and
- award funding under the Aboriginal Funding Envelope (Phase I).
Senior management involved in the Initiative was aware of these challenges. The need to improve early implementation of the whole-of-government approach was discussed at a DM Committee meeting (September 2010),73 as well as a variety of actions to be implemented to meet this goal.
Given these ongoing issues, many Aboriginal groups and organizations continue to express concerns regarding the federal consultation process, both with the process itself and with the limited opportunities to address broader issues (such as constitutional rights) within this process. Aboriginal groups expressed dissatisfaction with the lack of clarity (i.e., roles, timelines) and the lack of technical capacity (i.e., staff, experience, and funding to review lengthy and technical project documents). The latter was seen to be exacerbated by “imposed” timelines and poor timing of participant funding decisions/delivery (e.g., to hire legal and technical consultants). Despite enhanced participant funding envelopes, lack of participant funding was also viewed as an issue in specific MPMOI projects.
Aboriginal groups generally share the view that “meaningful” consultation should include constitutional rights issues, as some scholars have also argued.Footnote 74,Footnote 75 In the words of one interviewee, “When it came right down to it, there was no one in government with a mandate to engage us in real dialogue on the biggest issues, like the assertion of Aboriginal rights and title and the impact of the project on those rights.” Consultations were also often seen as “one-way”, whereas Aboriginal groups expect more of a dialogue with the federal government, including face-to-face meetings.
Question: To what extent were environmental assessments and regulatory reviews integrated and well-coordinated under the Initiative?
To a good extent. Multiple lines of evidence identified improvements in the integration and coordination of review processes across federal departments/agencies through the tools and processes implemented under the Initiative, as well as 2010 CEA Act amendments. Despite these improvements,the case studies highlighted the influence of internal and external factors (outside the control of the Initiative) on the consistency, timeliness, and predictability of outcomes.
Moreover, efforts to demonstrate quantitatively that the MPMOI had improved the mid-term outcomes of timeliness and predictability resulted in mixed findings. These efforts were limited by the short timeframe of the Initiative and by the lack of baseline and comparative data. Overall, the evaluation found that efforts at the operational level could only go so far: addressing fundamental systemic issues that affect the efficiency and effectiveness of the federal regulatory system would require further fundamental changes to this system.
There is ample evidence of activities and processes that have been implemented under the MPMOI to improve the consistency of the federal regulatory process for major resource projects. This section presents evidence of the achievement of the outcomes that stem from the implementation of these processes, particularly one of the short-term outcomes of the MPMOI (integrated and well-coordinated EAs and regulatory reviews).
Progress in terms of the achievement of improved timeliness of EAs and regulatory reviews, which is a mid-term outcome of the Initiative, is also discussed. This section also includes a detailed discussion of various internal and external factors which influence the achievement of these outcomes, particularly factors outside the immediate control of the federal government.
Improvement of integration and coordination
All lines of evidence pointed to a number of changes to the federal regulatory system associated with the MPMOI that had led to increased integration and coordination of the review process across departments and agencies. These included: the results of the tools and processes implemented under the Initiative including project agreements and service standards, senior-level coordination and issue resolution, and increased monitoring, tracking and reporting. Policy work had also supported this outcome, via non-legislative changes and legislative amendments to the CEA Act in 2010. Capacity funding provided through the MPMOI largely contributed to supporting the resources needed to ensure that this outcome could be achieved.
For example, MPMOI-related changes that improved the coordination of EAs included procedures and legislative amendments that contributed to a stronger coordination role/and additional responsibilities of the CEA Agency with regard to EAs, including:
- a more comprehensive administrative role as EA manager (as outlined in the project agreements, and including coordination with provincial processes);
- a new Federal Panel Coordinator role (i.e., to coordinate federal participants involved in review panels); and
- a RA role for comprehensive studies not led by the CSNC and NEB; this is in addition to the Crown Consultation Coordinator role.
Concurrently, to help support these new roles, the CEA Agency enhanced its project management focus, training and tools (e.g., new project repository database).
Generally, the MPMOI was found to have improved communication across federal departments/ agencies. Examples of new or enhanced forms of interdepartmental/interagency communication – and their impacts – were observed in the case studies and included several levels of government, such as project-level working groups, regional working groups, and – most significantly – senior level committees (e.g., director, DG, ADM and DM Committees).
Previous to the MPMOI, there existed relatively few horizontal fora through which to discuss cross-cutting issues relating to EAs and regulatory reviews (the Environmental Assessment Projects Committee, created in 2005, being one exception). The creation of a range of new horizontal senior-level committees was a significant change and was very frequently cited as having had a wide range of positive impacts on the federal regulatory system.
The MPMOI’s governance structure has led to greater senior-level awareness of, and focus on the system, and has been a major facilitator of collaborative work, priority setting, performance management, and issue resolution. As discussed in other sections of this report collaborative work involving senior level-representatives has been found to be a novel and effective approach to address system-wide challenges facing the federal regulatory system, particularly in improving consistency and to achieving policy objectives.
With regard to operational objectives, senior management attention and commitments relating to MPMOI projects (e.g., via project agreements), timelines (e.g., Weekly Trackers), and issues (e.g., senior-level committee meetings) have sent a clear message to department/agency staff that major resource projects tracked by the Initiative represent a priority. This, in turn, has resulted in earlier and more concerted efforts to conduct EA and regulatory review activities for these projects in order to meet timelines and related performance objectives.
Almost all internal stakeholders reported that the increased pressure to meet timelines was a key effect of the Initiative. Examples of this effect were also observed in the case studies. For example, intense interdepartmental coordination of the Government response to the Prosperity Copper-Gold Mine (led by DFO) proposal enabled this step to be completed within the ambitious ten-week timeline contained in the project agreement.Footnote 76
In terms of improved issue resolution, both document and interview data confirmed that, through the enhanced coordination and governance structures, project-level and cross-cutting issues were more often elevated and resolved (when possible) in a timely manner. The DM Committee, in particular, was recognized as a mechanism that had fostered effective discussion and issue resolution. In addition, there had been clear use of performance data and analysis generated to inform decision-making as monitoring and tracking functions of the Initiative had been used to support both operational and policy outcomes.
Other than the key role of senior-level coordination, findings from the interviews and the case studies provided rather mixed evidence regarding the impact of other specific tools and processes developed and implemented under the MPMOI, particularly project agreements, in terms of improving project-level integration and coordination. Project agreements were sometimes seen to have helped clarify roles and responsibilities and outlined milestones and target timelines (as well as ensure accountability. But quite often, project agreements were rarely consulted after they had been signed, and the milestones and target timelines they contained became outdated in light of project changes, updated work plans, or processes outside direct federal control (i.e., panel reviews or tasks under provincial responsibility, see below). Project agreements were rarely updated once signed.
Some interviewees therefore questioned the level of effort invested in the development of project-specific project agreements and suggested that alternative approaches could be considered (e.g., generic project agreements signed by DMs and more flexible project-specific project charters than could be “evergreened”, and possibly include project stakeholders outside the federal government).
It was also found that few internal interviewees (other than those directly involved and/or consulted in policy research and related activities), knew how the MPMOI was contributing to short- and long-term policy outcomes. This lack of awareness was particularly prevalent among department/agency staff involved primarily in project reviews and/or those located in offices outside of the National Capital Region. For example, many interviewees were familiar with the 2010 CEA Act amendments but did not attribute these changes to the MPMOI (e.g., these amendments were presented as elements outside the Initiative that had improved the federal regulatory system).
While it should be acknowledged that these individuals might have limited exposure to details behind policy work, the evaluation found indications that, because of this lack of exposure and awareness, these individuals did not necessarily feel a strong degree of buy-in for the Initiative or see how the project-level work (e.g., data collection through the project monitoring and tracking activities of the MPMOI) supported policy-level discussions towards future improvements.
The “distance” factor contributed not only to a lack of awareness about the Initiative and its range of activities and outcomes, but was also found to be related to a weaker level of support for the Initiative. For instance, some individuals expressed more resistance towards the Initiative because they saw mainly the administrative tasks associated with it and only a portion of the benefits (i.e., specific project-level impacts, which might not be obviously attributable to the Initiative).
While it was too soon for the MPMOI’s “culture shift” to be felt across federal system (i.e., four years is a very short timeframe in which to observe significant cultural change throughout organizations, especially around a system as broad/complex as the federal regulatory system for major projects), it was possible that additional communication and engagement efforts could help accelerate this shift.
Although the evidence confirms that there have been a number of improvements in the integration and coordination of the federal regulatory system for major resource projects that can be attributed to the MPMOI, some outcomes of the Initiative have been influenced by a number of internal and external factors. These factors are discussed below, followed by a review of the early performance of the Initiative towards timeliness and efficiency outcomes.
Factors impacting the achievement of expected outcomes
Several issues outside the immediate control of federal departments/agencies have posed a challenge to the achievement of short-term outcomes, with implications for the longer-term efficiency and effectiveness objectives of the Initiative. Confirmed by the case studies and interviews, these include both factors external to the federal government, such as issues in the coordination with provincial processes, the quality of information provided by proponents, and court decisions, and internal factors, such as the timing of legislative changes and the difficulties in achieving a wide-scale cultural shift within and across departments/agencies (e.g., both vertical and horizontal, as well as across regions). As the MPMOI is a relatively new Initiative, the impact of these issues can presently be assessed over the short-term; but it is entirely possible that many of these issues could be resolved or become less problematic over a longer timeframe.
A key contextual factor noted across multiple lines of evidence, particularly the case studies, was the Supreme Court decision with regard to the Red Chris mine (i.e., the MiningWatch case)Footnote 77 which resulted in the clarification of a number of scoping issues (i.e., “the scope of future projects is expected to be confirmed promptly and efficiently”Footnote 78), but created delays for active screenings that needed to be re-scoped part way through as comprehensive studies. Similarly, adjustments that needed to be made mid-EA for a number of comprehensive studies after the 2010 CEA Act amendmentsFootnote 79 designated the CEA Agency as the RA for these reviews led to delays for some MPMOI projects.
At the project level, the case studies indicated that the highly variable combination of key project characteristics (e.g., sector, province, type of EA, involvement of and relationships among the range of departments/agencies and external project stakeholders) – as well as project changes made by proponents – often posed a challenge to the consistent and successful application of the MPMOI approach. Of the approximately 75 MPMOI projects, about one third were identified by the MPMO as being “atypical” projects – based on their size, on the inclusion of new technologies, on delegation and/or pilot approaches or other distinctive characteristic – which could present unique challenges.
Specific to EAs, another notable challenge is posed by review panels, which are not governed by MPMOI timelines but rather by their own terms of reference.Footnote 80 In fact, striking the terms of reference, establishing review timelines, and submitting additional information requests are largely determined independently from the MPMOI (although the Initiative monitors and tracks these processes), resulting in a less predictable process.
This challenge is compounded in the case of joint review panels with provinces. While these panels ensure that both federal and provincial requirements are met (thereby reducing duplication), the need to align and coordinate with provincial processes can create additional timeliness issues. Generally, issues with the coordination of federal and provincial processes are often associated with delayed timelines, low process predictability, and the negatively impacted satisfaction of external stakeholders (e.g., proponents, provinces, Aboriginal groups).
All these factors are recognized within the MPMOI, and efforts have been made by the MPMO and participating departments/agencies to address them to the extent possible. This includes measures to anticipate and/or reduce the impact of delays caused by contextual factors, increasing communication and engagement with provinces, proponents and Aboriginal groups, as well as the development of pilot projects and options that may help resolve these issues in the longer term.
Looking toward mid-term outcomes: timeliness
As most of the same factors were present prior to the creation of the Initiative and also influenced the outcomes of non-MPMOI projects, this evaluation’s case study methodology included a comparison of non-MPMOI projects (pre-MPMOI and during the MPMOI era) to gather data regarding the ability of the MPMOI to improve process consistency, and regarding the achievement of mid-term outcomes of timeliness and predictability. As observed in the non-MPMOI projects examined as part of the case studies, it was clear that many of the same factors similarly influenced the coordination, efficiency and effectiveness of the project reviews.
However, it was not possible to conclusively determine to what extent the MPMOI had contributed to better addressing these challenges, partly because most (seven out of ten) of the case study projects were ongoing and the ultimate outcomes of the examined project reviews were yet to be seen. In addition, two of the three non-MPMOI projects had actually benefited from the existence of the MPMOI (e.g., additional capacity in regional offices, strengthened EA coordinating/RA role within the CEA Agency, whole-of-government approach to Aboriginal consultations).
Thus, while it was clear that the MPMOI’s capacity funding and systemic improvements to date had been beneficial to project reviews (including those not administered under the Initiative), the comparative case study design could not provide strong counterfactualFootnote 81 evidence as to the net effect of the MPMOI on timeliness and predictability outcomes.
Complementary efforts to conduct a quantitative analysis to assess early timeliness outcomes of the MPMOI were also hampered by the short timeframe within which to assess the success of the MPMOI (i.e., only three completed projects, plus five having completed the EA phase, Table 5), and by the lack of baseline data on project reviews conducted before the MPMOI was implemented. While the Initiative had a stated goal to reduce the EA and regulatory review stages of major resource projects “[from] upwards of four years to an average of about two years”,Footnote 82 regular monitoring or reporting of timelines was not conducted for pre-MPOMI projects. As well, no comprehensive data had been collected across projects on the time required for the process as a whole (much less for individual stages or milestones) previous to the Initiative, so that “upward of four years” was the only available measure for comparative purposes.
Data limitations also prevented carrying out comparative analyses that considered important variables, such as type of EA, sector, province, departments/agencies involved. It was not clear to what extent pre-MPMOI timeline assessments considered the difference between “federal time” (i.e., time required for milestones under federal responsibility) and the time required for tasks under the responsibility of external parties (e.g., proponent, province), including those for which the clock was “stopped” under current MPMOI tracking procedures. Thus, any attempts to demonstrate that current MPMOI project review timelines had improved compared to those pre-MPMOI had to acknowledge these limitations.
The MPMO assessed the performance of projects managed under the Initiative vis-à-vis timeline objectives based only on federal time. Using this approach, early results suggested that these objectives were being achieved. To date, the three completed projects reviews (EAs and regulatory reviews) were conducted within the two-year target (i.e., within 104 weeks of federal time), as were the EA and regulatory steps for two active projects that had not yet been granted all their permits/licences but had obtained those required to begin construction.
It should be noted, however, that of these five projects, three are NEB-led screenings (which followed a fairly straightforward process, especially compared to panel reviews), and one did not proceed to the regulatory decision-making phase in accordance with the related EA decision. The anticipated completion time for the remaining active MPMO projects (which considered delays that had already occurred in specific projects) was typically longer than for these five projects, but remained below the average two-year target.Footnote 83 These were very early results; stronger conclusions could be made when a larger number of projects would have completed the process under the MPMOI.
Since many MPMOI projects to date experienced a “stopped clock” during the EA or regulatory phase and most included various tasks under the control of external parties, the actual total time (i.e., calendar time) for project reviews was longer than the target two-year period. The MPMO reported that for about half of the projects, a number of elements not in the original plan had contributed to “extra time,” which was defined as unplanned federal time that had been added to the critical path of the federal review.Footnote 84 The most common reasons for “extra time” included: additional “sufficiency review of information” requests (made by review panels or by departments), project changes, additional public consultation, court decisions or legislative changes.
Thus, as many factors outside federal control added “extra time” or otherwise affected total calendar time for milestones not under federal control, it appeared that improving federal timelines did not necessarily contribute to significantly greater process certainty or predictability for project reviews. Such concerns were voiced by some industry associations and by several project proponents that found that project reviews were not necessarily more timely (or if so, not by a great extent) or predictable than previous to the Initiative. It should also be noted that proponents valued process predictability as much or more than the reduction of review timelines, since much of their planning efforts depended on knowing when the review process would be completed.
Another approach used to assess timeliness outcomes was to review data collection on individual and/or project-level target timelines conducted under the MPMOI. This data was used to assess whether target timelines were being met (including lateness with respect to established service standards, aspects such as “extra time” and time taken by external parties), and to track timeline performance indicators over time. Additional analyses were conducted by the evaluation team on raw data from the MPMO Tracker. Overall, these analyses showed that target timelines were met to a moderate extent, that certain timelines under federal control had improved since 2009 and that others had shown no change over time.
For example, the MPMO assessed “lateness”, in which federal target timelines were not met for reasons within federal control. This was found in nearly half of the projects.Footnote 85 The most common reasons for lateness included requests for legal advice, ministerial decisions and policy decisions.
Considering only lateness, 86 percent of project reviews were presently within eight weeks of their federal target timelines. When “extra time” was taken into account, 55 percent of projects were within their federal target timelines.Footnote 86 These results were fairly consistent over the 18 months of the Initiative; they were not available previous to 2010-11.
In contrast, one clear example of measurable improvement over time was that, according to the MPMO, the mean development time for project agreements had decreased by half, from 20 weeks in 2009 to ten weeks in 2011. This finding was confirmed by the calculation of the proportion of milestones completed within the target time using MPMO Tracker data.Footnote 87 For milestones within the PA development phase, the proportion completed after the target date decreased between 2009 and 2011. This improvement was not noted for milestones within the EA phase, and too few milestones had been completed under the regulatory decision-making phase to support additional findings.
In summary, federal timelines were viewed by internal and external stakeholders consulted to be improving due to increased capacity and improved integration and coordination (as a result of MPMOI procedures, coordination, and legislative improvements). However, efforts to demonstrate from a quantitativestandpoint to what extent these improvements had translated into increased overall timeliness and predictability of the federal project review process under the MPMOI were limited and resulted in mixed findings to date.
The MPMOI has assumed a very challenging task – i.e., to improve a complex process that is influenced by a large range of internal and external factors – and it would be unrealistic to expect full achievement of this outcome at the present. In addition, the international comparative analysis has clearly indicated that timeliness of the project review process in Canada has been influenced by factors similar to that of other countries, and in similar ways.
As a final point, these findings supported the views expressed by several interviewees across internal and industry stakeholder groups that the integration and coordination of the federal regulatory system under the present set of policies and regulations could only go so far. Specifically, it was clear that fundamental systemic issues still needed to be addressed and that addressing these issues would require fundamental changes to the system. For example, the broad objective leading to a “one project, one review” process, which could potentially include delegation to provinces, was often cited as the only means by which to achieve significant and long-lasting improvements to the federal regulatory system.
Question: To what extent has the Initiative increased the transparency and accountability of the federal regulatory process?
To a large extent internally, to a good extent externally. The case study, interview, file and document review found that increased transparency and accountability of the federal regulatory process for major projects were attributable to a number of tools and processes developed and implemented under the MPMOI, particularly project agreements, the Initiative’s governance structure, as well as performance monitoring, tracking and reporting activities. This finding primarily applied to stakeholders within the federal government, as external stakeholders provided mixed evidence on this outcome.
Internal documents and interview data strongly supported the finding that the additional project monitoring, tracking and reporting tools and processes implemented through the MPMOI had contributed to improving the transparency and accountability of the federal regulatory process for major resources projects. For example, interviewees within the federal government – particularly those at management levels – identified the contribution of senior-level committees, project agreements, Weekly Trackers, MPMO Tracker and supporting performance monitoring, tracking and reporting activities towards these objectives.
Not only do these tools and activities exist, but they are cited by interviewees and referred to in internal documents (including those reviewed as part of the case studies) in ways that demonstrate that individuals within the federal government are very aware of them and regularly consult them for various purposes, including accountability. For example, project agreements (being signed by DMs) are used as “leverage” to encourage federal authorities to submit technical reviews in a timely manner and, more generally, to hold departments/agencies accountable for fulfilling their outlined roles and responsibilities and meeting target timelines. Weekly Trackers are used in a similar manner, and have been implemented across all levels (i.e., working level, managers, DMs) to verify how well a certain project review or a particular department/agency is performing. The role of the DM Committee and other senior level committees has already been mentioned with regard to other outcomes, and remains a key element to ensure both transparency and accountability of the federal regulatory system for major projects.
As discussed earlier, findings with regard to reporting processes suggested that transparency outcomes had been achieved to a lesser extent for stakeholders outside the federal government. External stakeholders reported modest use of the publicly available information offered through the MPMOI (i.e., project agreements and MPMO Tracker). Even when they were aware that these tools were available (and a few were not, especially provincial representatives and Aboriginal groups), they generally relied on the CEA Agency (including the CEA Registry website) and responsible authorities to obtain timely and accurate information about the review process. In particular, while industry associations (and some proponents) acknowledged that departments and agencies were more consistently held accountable within the system (many referring to the DM Committee in this regard), and that the system should be considered to be more transparent due to the existence of these tools, their actual use of these tools remained relatively rare.
The increase in the number of visits to the public MPMO Tracker website since 2008 – given the modest use of the MPMO Tracker reported by external stakeholders might be explained by a potential increased use of the public MPMO Tracker website by department/agency staff, by external project stakeholders that were not interviewed, or by other stakeholders not widely consulted as part of the evaluation (e.g., public interest groups). When the number of monthly visits is divided by the number of active MPMOI projects for that month, the trend over time almost disappears (i.e., there is a positive correlation between the number of MPMOI projects and the number of Tracker visits). While this last finding may suggest that, as new projects are added, new project stakeholders subsequently begin to consult the Tracker, it still does not indicate whether these visitors are internal or external to the federal government. Further research (e.g., review of web analytics data, survey of MPMO Tracker visitors) would be useful to provide better business intelligence regarding who is using the MPMO Tracker and for what purpose.
The modest use of the MPMO Tracker may be explained at least in part by some limitations identified with regard to this tool. For instance, the fact that the MPMO Tracker does not show targets dates (as explained in NRCan’s 2010 Audit of the MPMO OfficeFootnote 88) makes it difficult to determine “whether or not the project is on track”. Furthermore, given that the milestones in the MPMO Tracker are sometimes different from those identified in the PA (i.e., have been adjusted to reflect project changes), there are often projects for which one can no longer compare milestones in the MPMO Tracker with those in the PA in order to identify the target timelines for this milestone (which are provided in the PA).
The MPMO Tracker also does not always identify which milestones are outside of federal control (i.e., those that depend on provinces, review panels, and proponents, or those that result from project changes, court decisions or legislative changes). In several cases, only those with access to supplementary information (such as that available within departments, agencies and the MPMO) could accurately identify the current status of an MPMOI project.
In addition, some external interviewees pointed out that the MPMO Tracker was not always updated in real-time, so that it was not always a reliable source of information for current milestones. For these reasons, interviewees often found that a visit to the CEA Registry or a call to the CEA Agency EA manager or to the responsible authority were more effective ways to obtain timely and contextual information on the federal review process for a specific project.
Although the interview, case study and document review data revealed that there was more information available to external stakeholders under the MPMOI than before, these findings, in combination with findings about the certainty of project timelines, suggested that more work remained to be done. The federal regulatory process had yet to achieve the mid-term outcome of being significantly more predictable for external stakeholders (mid-term outcomes were expected four to seven years after the establishment of the Initiative; see Figure 3).
While some industry associations reported seeing progress toward this objective, almost none of the proponents interviewed as part of the case studies considered the process to be more predictable. However, several interviewees (internal and external) noted that it was still quite early to be seeing concrete evidence of this outcome either at the project level and across the Initiative (e.g., only three completed reviews). Moreover, recent changes such as the 2010 CEA Act amendments and the new regulated timelines for comprehensive studies conducted by the CEA Agency were elements that would likely contribute to this outcome in the near future.
Question: To what extent is the Initiative achieving its policy and research objectives with regard to the continuous improvement of the federal regulatory review process?
To a great extent. The achievements stemming from the collaborative policy research and development activities under the Initiative exceeded those originally planned. These incremental changes on the path to a “one project, one review” process have had a number of positive implications in terms of supporting the goals of the Initiative. Thus, the MPMOI provided an effective forum through which to deliver these achievements, as well as to pursue ongoing efforts to improve the federal regulatory process.
The MPMOI contributed to a number of concrete policy changes and improvements. Interviews and an extensive document and file review clearly showed that the collaborative policy research and development activities conducted across departments and agencies under the policy leadership and/or within the forum of the Initiative contributed to this targeted outcome.
These included both non-legislative improvements (e.g., the whole-of-government approach to Aboriginal consultations, policy direction related to EAs and regulatory processes) and, on the legislative side, changes under Budget 2010 (i.e., amendments to the CEA Act and participant funding for EAs led by the CNSC and the NEB). Ongoing work through this approach included the development of further policy options and collaborative work with the provinces to support greater harmonization and integration of federal and provincial processes.
All of these changes, as well as other policy options under development, are incremental steps on the path to the ultimate goal of a “one project, one review” process. As these incremental steps address a number of fundamental systemic issues (e.g., diffuse responsibility, federal-provincial overlap), they are referred to within this evaluation as “fundamental changes”.
Some of these achievements, such as policy research work (i.e., “research and analysis of options for legislative change”)Footnote 89 and the piloting of new approaches, were outlined in the original plans for the MPMOI. In light of initial achievements, the government (via DMs) requested that the Initiative also support the development and implementation of concrete policy packages and enhance its efforts to improve provincial coordination. These elements were added to the Initiative after its launch, and the Initiative delivered them within a short timeframe. As such, the Initiative achieved more than its original policy and research objectives, while contributing to its overall goal of improving the federal regulatory system.
The implications of the legislative changes to which the Initiative contributed are consistent with the objective of continuous improvement of the federal regulatory system for major projects. For example, the amendments to the CEA Act made as part of Budget 2010 include several changes that “will improve timeliness of federal environmental assessment, establish clear accountability and focus resources where they would produce the greatest benefit to the environment and the economy.”Footnote 90
In particular, these amendments strengthen the role of the CEA Agency, which assumed the role of responsible authority for comprehensive studies of major projects (except those regulated by the NEB and CSNC). These changes were expected to reduce delays and duplication and potentially also allow for more effective collaboration with the provinces. The changes came into effect on July 12, 2010, and some early effects were noted in some of the case studies where this change applied, as well as in interviews and letters from stakeholders.
While the changes created some initial delays in terms of incorporating the CEA Agency’s EA role into EA work plans, there is early evidence to suggest that – in combination with new regulated timelines for comprehensive studies conducted by the CEA Agency (another change supported by the MPMOI) – these change, now that they have been implemented, are likely to result in reduced duplication, more timely EAs and improved provincial coordination.
In response to needs brought forward through the MPMOI, Budget 2010 provided the NEB with the legislative authority to establish a Participant Funding Program (PFP) under the National Energy Board Act,Footnote 91 and provided the equivalent authority to the CNSC under the Nuclear Safety and Control Act.Footnote 92 This not only expanded the total amount of funding available to support the participation of Aboriginal groups in federal review processes, but also had important implications in terms of the role that NEB and CNSC play in EAs. This amendment, in combination with existing substitution provisions of the CEA Act, would allow the NEB and CNSC to deliver EAs for projects within their jurisdictions that normally would have been assessed by a Joint Review Panel with the CEA Agency.Footnote 93,Footnote 94
A number of interviewees pointed to the expected impact of the new timelines regulations for comprehensive studies delivered by the CEA Agency (i.e., excludes NEB- and CNSC-led comprehensive studies), which were introduced in June 2011. These regulations required the CEA Agency to complete the comprehensive study in 365 days. They complemented the 2010 CEA Act amendments “by ensuring that the Agency is accountable for conducting these environmental assessments within a prescribed timeline and in a predictable manner.”Footnote 95,Footnote 96
As noted in a previous section, the extensive policy research and collaborative work conducted under the Initiative involved the active participation of senior-level committees, which supported horizontal collaboration involving regulators and policy-makers from key departments. The MPMO acted as key resource (i.e., a “centre of expertise” in the words of some internal interviewees) to facilitate and inform this work.
This demonstrates how the Initiative provided a forum through which policy options were developed and discussed, taking into account factors that might influence policy decisions, both internal to the federal government (e.g., departmental/agency mandates) and external (e.g., events affecting public opinion, such as the Gulf of Mexico oil spill in the summer of 2010).
In the case of policy options being implemented (e.g., whole-of-government approach to Aboriginal consultations, 2010 CEA Act amendments), this structure also provided direction as to the implementation of these changes from a system-wide perspective. As such, the Initiative also was found to provide an effective mechanism through which to deliver policy improvements, as well as to pursue ongoing efforts to improve the federal regulatory process.
Question: To what extent has the Initiative achieved a balance between the improvement of efficiency in the regulatory process (e.g., timeliness) and the quality of environmental assessment and regulatory reviews?
To a good extent, but additional data are needed. Multiple lines of evidence suggest that the initial efforts of the Initiative have been directed towards improving and monitoring efficiency outcomes relating to the federal regulatory system for major projects (e.g., timeliness), while simultaneously supporting high-quality processes (i.e., well-conducted EAs, regulatory reviews and Aboriginal consultations) that are effective (i.e., that protect the environment and support the purposes of other applicable regulations). The available data suggests that the quality and effectiveness of reviews have been maintained under the Initiative. However, while abundant performance data has been collected under the MPMOI with regard to efficiency outcomes, there has been a more limited focus in terms of monitoring quality and effectiveness outcomes at this stage of the Initiative. Recent efforts were identified to address the task of developing effectiveness metrics.
The present discussion of the extent to which the MPMOI achieved a balance between the improvement of efficiency in the regulatory process (e.g., timeliness) and the quality of environmental assessment and regulatory reviews, first, considers evidence of whether the MPMOI approach and capacity funding effectively support these outcomes; second, reviews the effects of these activities (available evidence relative to achievement of these outcomes).
There is ample evidence illustrating how the MPMOI implemented activities and processes directed towards improving the efficiency of the federal regulatory system for major projects. This includes the development of service standards, procedures and coordination tools, increased project coordination, tracking of project milestones, and mechanisms to ensure timely resolution of project-level and cross-cutting issues. MPMOI capacity funding provided to departments/ agencies helped address capacity issues that were limiting the efficiency of the federal regulatory system. Meanwhile, many policy improvements were designed with efficiency outcomes in mind (e.g., improved timelines and reduced duplication).
The effect of this combination of tools, processes, and capacity funding on the efficiency of project reviews has been carefully monitored, through weekly tracking, regular and frequent performance updates to senior-level committees, periodic reviews of services standards and other general project tracking and reporting mechanisms. There is an abundance of available performance measures relating to process efficiency. As discussed in previous sections, these measures suggest that the Initiative has already contributed to improvements in federal timelines for EAs and regulatory reviews.
Multiple lines of evidence also indicated that many of the MPMOI’s tools, processes and capacity funding supported the quality and effectiveness of major resource project reviews. These two concepts included both the quality of the processes(also called process effectiveness, i.e., well-conducted EAs, regulatory reviews and Aboriginal consultations) and their effectiveness (i.e., that protect the environment and support the purposes of other applicable regulations).
For example, capacity funding helps ensure that sufficient department/agency staff are in a position to conduct scientific and technical reviews and Aboriginal consultations, whereas improved procedures, training and senior-level direction helps them deliver high-quality EAs, regulatory reviews and Aboriginal consultations. Another frequently cited example in this regard is the MPMOI’s Guide to Preparing Project Descriptions for a Major Resource Project,Footnote 97 which provides guidance to proponents regarding what information they should include in their project descriptions in order to meet federal department/agency requirements. Some internal interviewees have reported an increase in the quality of the information provided by project proponents who had used this guide. This subsequently has improved the quality of the scientific and technical reviews and decisions made by departments.
However, relatively few formal tools and processes implemented thus far under the MPMOI seek to monitor the quality and effectiveness of EAs, regulatory reviews and Aboriginal consultations conducted under the Initiative – certainly fewer than those implemented to monitor efficiency outcomes. It was widely recognized that such measures and indicators were much more challenging to develop and implement than those relating to efficiency (e.g., the same challenge was observed as part of the comparative analysis with the approvals process in Western Australia). However, the evaluation found that there was a lack of methods, tools, procedures or clear roles within the MPMOI to collect, compile, review and report data on quality and effectiveness outcomes, and the mandate to do so was unclear. For example, the MPMO prepared a document in August 2011 entitled Performance Measurement for the MPMOI, in which the only evidence provided regarding the quality of EAs and regulatory reviews was the implementation of “effective project management and issue resolution mechanisms.”
Furthermore, the only quantitative indicator in the HMRAF that was designed to assess the quality or effectiveness of the EA and regulatory reviews was the percentage of time that mitigation strategies were implemented and follow-up conducted (the only other proposed indicator for this issue was based on stakeholder views).Footnote 98 This evaluation was conducted at too early a point in the MPMOI’s existence to assess this indicator, as only three project reviews had been completed under the Initiative, and two of these appeared to be still under construction. The third project, the Prosperity Copper-Gold Mine, was not approved for construction following the EA decision, which found that “the project is likely to cause significant adverse environmental effects that cannot be justified in the circumstances.”Footnote 99 This therefore represented a case in which the environment was protected through an EA process conducted within the Initiative. This should not be taken to mean that this decision was a result of the MPMOI, but rather that the MPMOI supported the processes which led to this decision.
In the case of other MPMOI projects, no specific tasks or milestones had been identified in the project agreements or MPMO Tracker relating to the follow-up and monitoring phases of completed projects. Furthermore, it was not clear who was responsible for collecting data on the percentage of time that mitigation strategies were implemented and follow-up conducted in order to inform this indicator.
There was limited evidence of potential effects (positive or negative) of the MPMOI process on the quality and effectiveness of EAs and regulatory reviews. The assessment of performance with regard to these outcomes as part of this evaluation was based primarily on the evidence collected during interviews and explored in depth in the case studies. Most interviewees (both internal and external) had observed no change in the quality and/or effectiveness of project reviews under the MPMOI and considered that project reviews had been of high quality before the MPMOI (with the possible exception of follow-up and monitoring processes, which were sometimes cited as being underfunded). Thus, the Initiative was viewed as having maintained the quality of EAs and regulatory reviews. Additional data and specific indicators or measures to support this view were not provided.
The case studies, interviews and internal documents provided a variety of examples in which target timelines were missed or extra time was added to reviews in order to ensure that the quality of the EAs, regulatory reviews and Aboriginal consultations was maintained. For example, certain milestones were completed after their target timeline in order to take the time to obtain legal advice or develop a position on a specific issue. In other cases, “extra time” was added to the federal process when it was deemed necessary to submit additional/unplanned project information requests to proponents (i.e., additional time was also required to review this information) and to conduct additional/unplanned public and/or Aboriginal consultations.
A small number of internal interviewees also identified circumstances in which they perceived an actual or potential effect of the Initiative’s efforts to improve the efficiency of federal processes, as making “compromises” to meet target timelines (e.g., accepting information that was incomplete, not asking certain questions or requesting additional revisions that could add time, submitting partial comments or reviews on documents). This also had important implications with regard to Aboriginal consultations, where timing issues were noted by Aboriginal groups and in the case studies. Thus, there was a potential that strict adherence to target timelines on such concerns would pose a risk to the Initiative if they were not acknowledged, investigated and mitigated as appropriate. This risk was also not considered under the Initiative’s risk management approach.Footnote 100
As the Initiative matured, more frequent references were observed with regard to the need to pay closer attention to quality and effectiveness outcomes, as well as efforts to identify and develop effectiveness measures that could be implemented.Footnote 101 In fact, the MPMO was in the advantageous position of having already implemented project monitoring and reporting tools and processes for efficiency outcomes so that it could readily add quality and effectiveness measures (e.g., on the effectiveness of mitigation measures) to these processes.
Question: Has the Initiative produced additional impacts (e.g., unintended [positive or negative] outcomes), and if so, what were these impacts?
Yes. Three main additional beneficial impacts were observed that extend beyond the Initiative itself: 1) the Initiative’s role in “catalyzing” or even directly supporting the improvement of non-MPMOI processes; 2) the role of the MPMOI to help inform and support the launch of the Northern Projects Management Office (NPMO); and 3) the creation of an exceptional level of horizontal dialogue at senior management levels. However, the evaluation also found some evidence that MPMOI processes, while supporting the achievements of the Initiative, occasionally had some unintended consequences (e.g., diverting time and resources from important non-MPMOI projects and departmental activities, reduced independence at lower levels, increased risk aversion and reduced flexibility to make adaptive changes to EAs and regulatory reviews).
Some of the additional effects that have occurred as a result of the MPMOI have been mentioned in previous sections and are briefly reviewed here, as well as other unanticipated impacts observed in multiples lines of evidence.
The Initiative was often referred to by federal government representatives interviewed as a “catalyst” that prompted departments/agencies to review and improve their own processes relating to EAs and regulatory reviews (or similar processes), not only for MPMOI projects but also for non-MPMOI projects (including pre-existing projects and those outside the natural resource sector, such as infrastructure projects).
These efforts were not necessarily conducted as part of the Initiative as they did not receive MPMOI funding or coordination through the Initiative. However, since the MPMOI operated within the broader context and management of the federal regulatory system – these parallel improvements were seen to be in some way inspired by the existence of the MPMOI. While possible that these changes might have happened in the absence of the Initiative, they would not have benefited from the experience gained from the MPMOI or the new horizontal structures implemented to support the Initiative, such as the DM Committee (see below).
Similarly, tools and procedures developed as part of the MPMOI (or adapted in parallel) were found to have benefited non-MPMOI project reviews. Several concrete examples of this type of additional impact were seen in the context of the case studies, where the whole-of-government approach for Aboriginal consultations was found to have been applied to non-MPMOI projects, as were some of the MPMOI-supported tools (e.g., Project Description guide, the CEA Agency’s project management approach/procedures). MPMOI capacity funding also directly and indirectly benefited non-MPMOI projects by bringing additional resources and expertise into regional offices.
Another key example of additional benefits (although these have been largely intended) is the Northern Projects Management Office (NPMO), administered by CanNor. While not expressly part of the MPMOI’s operations, it is supported by the MPMOI in various ways. The MPMO serves as a model for many of its processes. NPMO projects will be publicly tracked in the MPMO Tracker, the DM Committee provides guidance and coordination on matters related to the NPMO, and some of the capacity funding for the MPMOI has been used to support resources that play a role within the NPMO (i.e., within AANDC).
There were also unintended benefits associated with the senior governance structure of the MPMOI. Members of senior-level committees reported that the MPMOI senior governance structure had generated an exceptional level of horizontal dialogue across department/agency senior management and created networks that extended beyond the Initiative. This structure was also viewed by many as a horizontal framework model and example on how to achieve collaboration and teamwork across departments and agencies.
While the effectiveness of MPMOI tools and processes was the focus of previous sections, these were also found to have had unanticipated consequences. A number of interviewees – particularly those at the working level and in regions, and some proponents – described the Initiative to be a “mixed blessing” or a “double-edged sword”. For example, allocating higher priority to MPMOI projects might result in less time and resources available for EAs and regulatory reviews of important non-MPMOI projects (e.g., comprehensive studies and panel reviews for pre-MPMOI and infrastructure projects) and other departmental mandates, such as research and activities necessary to maintain scientific and technical expertise. This was a more significant concern in the context of capacity limitations (e.g., in unfunded departments).
In addition, the emphasis on consistency associated with enhanced senior coordination was linked with many positive outcomes, but was also seen by some interviewees and in the case studies to reduce independence at lower levels, increase risk aversion, and reduce the flexibility to make changes to EAs and regulatory reviews that would have been appropriate or beneficial in the context of specific projects.
These constraints were reportedly manifested as overly cautious decision-making (e.g., requesting additional information/review, legal advice or sign-off from senior staff/managers, resulting in slower decisions) in the avoidance of innovative or adaptive approaches, and in the unnecessary elevation of issues to senior levels. The latter effect was reported by some internal interviewees at all levels. Working-level interviewees considered that some issues could be adequately resolved within working group/teams rather than always through obtaining management or director-level approvals. ADMs felt that some issues could be resolved during their meetings instead of reviewing decks that would be tabled for discussion at DM Committee meetings. Some DM Committee members suggested that some decisions could be made without ministerial approval.
4.2.2 Demonstration of Efficiency and Effectiveness (Issue 5)
Based on the available data, the evaluation found that it would be difficult to achieve more at lower cost under the current regulatory system, but areas existed for improvement. Quantitative analysis of efficiency was precluded by the lack of baseline financial data (pre-MPMOI) and the lack of detailed data on MPMOI resource allocation and use (i.e., by type of activity or by project; non-MPMOI contributions; across departments and agencies).
Transaction costs were associated with a number of MPMOI processes (e.g., reporting and coordination). Although these clearly contributed to the achievement of the Initiative’s objectives (as reviewed in previous sections) there remained room to review a number of coordination and reporting processes to further increase the efficiency of the MPMOI. The evidence indicated that efforts to improve the efficiency of the regulatory system (i.e., reducing the timelines), all the while maintaining its effectiveness, required additional resources such as those provided by the MPMOI. Thus, MPMOI-funded departments/agencies stressed the importance of capacity funding to support the Initiative’s efficiency and effectiveness objectives with regard to the federal regulatory system for major resource projects. Both funded and some unfunded departments highlighted ongoing and growing capacity constraints.
Duplication was managed through increased coordination within the federal government. Enhanced coordination and communication among federal departments/agencies led to the increased efficiency of the federal process. However, the reduction of duplication with the provincial processes was recognized as an area where significant future gains could be achieved.
The international comparative analysis conducted as part of this evaluation, which included emphasis on Western Australia, revealed that the MPMOI performed quite well compared to similar initiatives. The MPMOI had been implementing several best practices with respect to transparency (e.g., public tracking, senior management coordination) and policy research, and offered some unique features not identified elsewhere, particularly the legislated duty to consult Aboriginal groups, and – to some extent – the concept of single point of entry. Some practices seen in Western Australia might inform future improvements to the Canadian regulatory system for major resource projects, such as bilateral/substitution agreements with states, the introduction of a Lead Agency Framework model (instead of a central office), and formal and earlier proponent/stakeholder engagement.
Question: Is the Initiative the most efficient means of achieving its intended objectives?
Based on the available data, the evaluation found that it would be difficult to achieve more at lower cost under the current regulatory system, but areas for potential improvements exist. Quantitative analysis of efficiency was precluded by the lack of sufficient baseline financial data (pre-MPMOI) and the lack of detailed data on MPMOI resource allocation and use (i.e., by type of activity or by project; non-MPMOI contributions; across departments and agencies). However, MPMOI-funded departments/agencies stressed the importance of capacity funding to support the Initiative’s efficiency and effectiveness objectives with regard to the federal regulatory system for major resource projects, highlighting ongoing and growing capacity constraints.
It was also found that the Initiative represented a minor investment compared to the projected capital investment and job creation potential of the major resource projects reviewed under the Initiative. Transaction costs were associated with a number of MPMOI processes (e.g. coordination and tracking). Although these contributed to the achievement of the Initiative’s objectives (as reviewed in previous sections), opportunities remain to review some processes to further increase the efficiency of the MPMOI.
The unavailability of sufficient quantitative data on resources (i.e., financial and FTEs) used for pre-MPMOI (i.e., baseline) and post-MPMOI processes prevented this evaluation from assessing the efficiency and economy of the Initiative using detailed quantitative analyses.
For example, the format and availability of financial and FTE data across departments and agencies varied considerably, including planned, actual and in-kind (non-MPMOI) contributions according to type of activity (e.g., EAs, regulatory reviews, Aboriginal consultations, policy work and senior governance) or per project under the MPMOI. Combined with the lack of baseline data on pre-MPMOI project reviews, this also limited the analysis of how MPMOI funding contributed to the improvement of timelines (e.g., ratio between resources and timelines) and other outcomes across phases of the project review process. The following section thus relies on a combination of quantitative and qualitative information obtained from departments/agencies, as well as interviews.
In accordance with their mandates and regulatory/statutory responsibilities (e.g., triggers under the CEA Act), departments and agencies would have been required to conduct project reviews for major resource projects currently overseen by the MPMOI even if the Initiative had not been implemented. Thus, the key question in this section is whether the MPMOI represented the most efficient means to achieve its objectives, i.e., to improve the federal regulatory system for major resource projects to support the competitiveness of Canadian resource industries.
A large proportion of MPMOI resources, about 85 percent ($130 million of the $150 million MPMOI funding from 2007-08 to 2011-12) was allocated to departments and agencies to increase their scientific, technical and Aboriginal consultation capacities. This was necessary to address capacity issues encountered during project reviews and to support additional MPMOI-led activities (e.g., development and implementation of tools and procedures, and policy work). The remainder ($20 million) was used to establish and support the activities of the MPMO.
With regard to capacity funding provided to departments/agencies, internal interviewees identified the vital nature of this funding in terms of achieving target timelines for MPMOI project reviews. In the case studies, this was commonly demonstrated by identifying the specific resources contributing to the EA, regulatory reviews and Aboriginal consultations whose positions were supported by MPMOI funding. The case studies also indicated that non-MPMOI projects whose EAs were conducted pre-MPMOI had experienced significant delays because of insufficient staff; whereas, MPMOI projects (and one non-MPMOI project conducted during the MPMOI era that had directly benefited from the presence of MPMOI-funded resources in the region) had experienced only minor challenges in meeting timelines because of capacity constraints.
While the number of major projects managed under the MPMOI was fewer than originally expected (i.e., 75 projects, as opposed to 100 anticipated projects in the original program documentation),Footnote 102MPMOI capacity resources were nonetheless fully utilized to support MPMOI project reviews, since funds were required to meet new responsibilities. Interviewees highlighted a number of factors that placed increased pressure on MPMOI resources. For instance, largely as a result of court decisions, more resources were required by departments and agencies to meet evolving and expanding Aboriginal consultation responsibilities. As well, the January 2010 Supreme Court decision on the MiningWatch caseFootnote 103 resulted in the re-scoping of several EAs from screening-type assessments to more resource-intensive comprehensive study EAs. The Initiative also took on mega-projects (e.g., Northern Gateway) that required enhanced coordination, Aboriginal consultations and other related activities compared to more typical MPMOI projects. Finally, support for some MPMOI processes, such as collaborative work and administrative mechanisms outlined in the Cabinet Directive on Improving the Performance of the Regulatory System for Major Resource Projects, was also found to have strained departments’ and agencies’ capacities.
Moreover, it was widely reported that, even with MPMOI funding, departments/agencies were facing ongoing and significant resource and capacity constraints in the case of some projects. In particular, as seen in the case studies, certain regions reported a lack of capacity given that the number and complexity of projects in their regions were greater than anticipated.
Capacity constraints were also found to be an ongoing and a growing concern in terms of limiting the timely involvement and responses of some unfunded departments (e.g., NRCan) in EAs and regulatory reviews, as well as of federal departments that were not signatories of the MPMOI’s MOU, but participated in some federal project reviews (e.g., Health Canada, Parks Canada). These reported capacity constraints encompassed both S&T capacity and capacity to carry out meaningful Aboriginal consultations.
Almost all funded departments/agencies also reported having used additional resources (from other sources, such as A-base funds) to complement MPMOI funding and conduct activities that supported the outcomes of the Initiative – not only project-related activities, but also those involving policy work. This was most often observed in the case of Aboriginal consultations (e.g., development of procedures, training, advice and guidance).
Furthermore, collaborative policy work and supporting processes (e.g., development of the whole-of-government approach, collaborative policy research, senior-level coordination and guidance) often involved the participation of senior-level management, including unfunded departments/agencies and entities not supported by the MPMOI (e.g., DOJ, PCO).
Thus, interviewees consistently reported that it would be difficult to realize the same level of achievement of objectives as attained under the MPMOI with fewer resources. Importantly, these objectives included not only those stemming from EA and regulatory review activities (i.e., that would have occurred whether these were funded through the MPMOI or other sources), but also project coordination and monitoring activities as well as collaborative policy work. Similarly, senior management interviewees within the MPMO perceived that resources within the MPMO had been fully utilized to implement activities under both of its mandates (i.e., operations and policy). Given the high level of achievement for these two mandates, reducing the MPMO’s resources would result in a lower level of achievement.
Efforts were also made to analyze the delivery cost per project as a measure of efficiency. As of July 31, 2011, there were 70 active projects managed under the MPMOI. While the cost associated for each project review was not available (nor the cost per EA), the MPMO estimated that expenditures associated with project-level operations were approximately $11,000 per project, per year.
This estimate, however, includes only the activities conducted within the MPMO and does not include resources applied to EAs and regulatory reviews of MPMOI projects across departments and agencies, which received the bulk of MPMOI funding.Footnote 104 Furthermore, this estimate does not include operations and maintenance costs (e.g., supplies, information technology, training) and other corporate funding provided to central agencies/corporate services (e.g., accommodation costs to Public Works and Government Services Canada). Thus, while a more comprehensive estimate is not possible, the cost associated with providing project coordination by the MPMO is relatively modest, especially when considering the value of the proposed projects.
The capital investment of about 70 active MPMOI projects was estimated at $120-140 billion.Footnote 105 These projects have the potential to create more than 100,000 jobs in Canada.Footnote 106 Given the size and potential socio-economic impact of these projects, even if a single project were to be abandoned as a result of a lack of efficiency or effectiveness within the federal regulatory process that the MPMOI would have otherwise resolved, the lost opportunity cost of the average MPMOI project (nearly $2 billionFootnote 107) would outweigh the entire cost of the MPMOI by approximately 13 times.
In light of the figures, the cost of the Initiative did not seem to be substantial to some interviewees. Senior management at the ADM and DM levels in particular stated that the Initiative was a worthwhile and “essential” investment. These interviewees also stated that some of this investment was used to support administrative efforts (i.e., tracking, reporting, additional processes and coordination, including but not limited to the average expenditures of $11,000 per project per year by the MPMO) as opposed to only supporting resources that deliver project reviews. “We are now more on top of things, and able to let different levels know where things are at. We can identify and pinpoint problems in areas, address them. That investment has paid off.”
It should be noted, however, that data collected from several departments/agencies clearly illustrated that processes related to the administration and coordination of MPMOI projects represented additional transaction costs. In particular, the MPMO-led focus on process and timelines and the broader range and number of people/groups overseeing MPMOI projects required additional processes to support project monitoring, tracking and issue resolution (e.g., information for Weekly Tracker, project decks and support for monthly DM and other senior-level committees, participating in committees, preparation and review of project agreements) compared to that required for non-MPMOI projects.
The Initiative was frequently referred to as “an additional layer of bureaucracy” by both internal and external interviewees. Most used this phrase when discussing the extent to which these additional processes had been beneficial. Many (but not all) recognized that these processes effectively contributed to achieving the objectives of the Initiative. However, although these costs were meant to be addressed by MPMOI capacity funding, they were found to be higher than anticipated by some participating departments/agencies (funded and unfunded). Thus, both internal and external interviewees frequently questioned whether greater efficiencies could be found in order to reduce these costs.
Data to quantify the workload associated with these “transaction costs” was limited to estimates provided by three departments/agencies, and the perceptions of interviewees involved in the coordination and delivery of EAs and regulatory reviews of MPMOI and non-MPMOI projects. These project reviews were required as per the responsibilities and mandates of departments/agencies. Moreover, major resource projects would often require greater levels of coordination (and associated briefings) that would add to their workload even in the absence of the MPMOI.
The data collected by the MPMO from two funded and one unfunded department/agency suggests that transaction costs are approximately one to 1.5 FTE per funded and unfunded department/agency (not including the CEA Agency, which received the highest level of funding under the Initiative), plus about 2.5 FTEs within the MPMO. While the total FTEs required to support the MPMOI’s transaction costs likely represents a relatively small proportion of the total FTEs supported by the Initiative (perhaps a dozen or so FTEs within a total of nearly 200 FTEs, or about five percent), the fact that this includes FTEs within unfunded departments/agencies and groups currently facing growing capacity constraints indicates that attention could be paid to this situation in the context of efficiency issues.
Comparing the responses of department/agency staff collected as part of case studies, the evaluation team found that MPMOI projects generally required more staff time than non-MPMOI projects, especially in terms of reporting and briefing of senior management. In some cases, this increased workload was seen to be caused at least in part by the nature of and public interest in certain projects, as well as by specific characteristics of the project and related complications (e.g., litigation, requirements of the Joint Review Panel), rather than because the project was managed under the MPMOI.
In these cases, having senior management involved from the beginning of the process was often viewed as beneficial despite the amount of tracking, reporting and briefing that the project entailed. Rather, it was usually in the context of more straightforward projects (i.e., in which there were fewer issues) that department/agency staff saw fewer benefits (or a lesser degree of return) stemming from the increased MPMOI-related workload.
In the case of the 30 department/agency staff ableFootnote 108 to comment on resources used for MPMOI and non-MPMOI projects, two-thirds considered that this workload was greater for MPMOI projects, whereas the other third indicated that it was about the same. However, four of the individuals in the former group found that the increase in workload was insignificant in relation to their other project tasks and responsibilities. Some of those who reported a greater workload associated with MPMOI projects estimated the difference to represent an increase of about 5-10 percent.
Most indicated that the reporting and briefing for MPMOI projects took an almost negligible proportion of their time (about one percent to three percent), while others were more involved and reported that these tasks took about ten percent or more of their time, particularly certain tasks or during some phases of a project (e.g., development of project agreements and government responses). An example of a fairly typical response among those who observed an increase in their workload was as follows, “Absolutely, I’ve seen a huge difference [between MPMOI project and non-MPMOI projects]. Reporting to the MPMO, the Weekly Tracker, and then there are the monthly DM meetings, DG and ADM meetings. We’ve got to prepare materials for each of those meetings and if a project is on the agenda, there is extra work. Another part is the MPMO follow-up, when they are asking questions about a project status.”
As seen in previous sections, the present governance structure of the Initiative has been found to be an efficient approach to improving the integration and coordination of project reviews under the Initiative, particularly to address project and/or policy-related issues under the Initiative. Thus, while interviewees generally agreed with the findings of this evaluation that this governance structure should be maintained, calls for fewer meetings and reduced/streamlined reporting requirements were heard across groups of internal stakeholders (including some, but not all DMs).
Specific suggestions included reducing the number of committees and/or the frequency of meetings, as well as other ways to reduce the large amount of time spent collecting and reviewing project-level data in order to prepare for these meetings (e.g., focusing more on strategic and policy issues rather than on project-level issues).
In addition, it was often reported that some issues previously incorporated into the DM Committee meeting agenda could be resolved by lower-level committees. Some interviewees also reported that the strong focus on DM Committee meeting preparation (i.e., reviewing the deck) took time away from substantial discussions of other issues that might lead to the resolution of these issues.
Additional suggestions included: preparing and circulating fortnightly trackers instead of weekly trackers; taking advantage of existing project management and tracking systems (e.g., those operated by the CEA Agency and TC, through automated processes) rather than requiring additional spreadsheets or manual data collection; and reducing the number of milestones that are tracked (especially those not on the critical path or outside the control of the federal government).
One such innovation, the early warning system, was identified by a number of interviewees as a positive first attempt to reduce some of these transaction costs. Evidence from interviews and the document and file review indicated that the amount of time and effort required to prepare and review project agreements had decreased since the initial implementation of the early warning system. However, there remained room for ongoing efforts to develop more cost-effective ways of delivering/designing MPMOI processes that would take into account the transaction costs involved (which would require detailed information and input from departments/agencies) and critically assess related benefits. The impact of any changes to MPMOI processes would need to be assessed using a risk management approach, which balanced the costs savings against the lost benefits associated with, for example, less frequent meetings and reduced reporting requirements.
Efficiency gains were also expected to occur as a result of the 2010 CEA Act amendments, as well as from further fundamental changes that might occur as a result of the Initiative. It would be of interest to ensure that such gains could be measured and assessed, as they would represent an important contribution of the MPMOI to the efficiency of the federal regulatory system. Moreover, both internal and external stakeholders indicated that cost-effectiveness could be enhanced by reducing duplication through a higher level of integration and coordination of processes within the federal government, but also between federal and provincial entities. As mentioned earlier, efforts were underway within the MPMOI to improve coordination and collaboration with provinces.
Question: How was duplication of effort managed (or avoided) in order to ensure effective use of resources within this Initiative and with other jurisdictions for the federal regulatory review of major resource projects?
Duplication was managed through increased coordination within the federal government – but duplication with the provinces is recognized as an area where significant future gains could be achieved. Enhanced coordination and communication among departments/agencies led to an increased efficiency in the federal process. Some overlap remained in the roles and responsibilities of the MPMO and CEA Agency with respect to coordination and related coordination. A lack of coordination was found to occur frequently between the federal and provincial processes. This was where duplication was seen by the majority of internal and external stakeholders. A better integration of the federal-provincial review processes would lead to efficiency gains.
Despite varying perceptions among both internal and external stakeholders, the majority agreed that the duplication of federal efforts had been reduced with respect to Aboriginal consultations and NEB/CNSC-led processes; this was confirmed by case study findings. Several tools developed under the Initiative (e.g., memoranda of agreement and project agreements) and enhanced communication through the DM Committee and other interdepartmental working groups were cited as the vehicles of improved coordination and greater clarity in the process. In addition, several interviewees emphasized that rather than reducing duplication, the MPMOI helped to fill gaps in the federal system.
On the other hand, some duplication was frequently cited by both internal and external stakeholders in the roles and responsibilities of MPMO and CEA Agency, especially when the CEA Agency was the EA lead coordinator of a project. Although this issue had improved to a certain degree over time, some still perceived the MPMO as another layer of bureaucracy and felt that the CEA Agency could perform most of the tasks that the MPMO carried out during the EA phase of a project review.
This opinion was expressed most strongly by regional staff, and supported by external stakeholders (e.g., provincial and Aboriginal representatives, some project proponents). Other departmental staff pointed out the duplicative tracking process of the MPMO and the CEA Agency as a considerable source of inefficiency. Both the case studies and interviews also indicated that relatively minor duplication persisted with regard to elements under the mandate of multiple departments (e.g., hydrology and hydrogeology issues and the processes relating to the Metal Mining Effluent Regulations).
Moreover, despite the considerable improvements in horizontal coordination of Aboriginal consultations, case study reviews and interviews with both internal and external stakeholders suggested that this part of the process still required further refinement to prevent overlap and duplication. The tools developed under the MPMOI for more effective and efficient Aboriginal consultations were well appreciated by most internal stakeholders as well as some project proponents. However, most still considered some parts of the process as challenging, such as timely identification of all Aboriginal groups that might be affected by the proposed project, or coordination of consultations with other departments and provinces. Greater efficiency might be achieved by further clarification of federal roles and responsibilities as well as coordination and integration of the process with provinces.
Some interviewees also advocated for the development of a system that would facilitate early identification of all affected Aboriginal groups with land rights in the area of a proposed project. Based on their experience, the current unavailability of this information repeatedly resulted in adding Aboriginal groups to be consulted later in the process which created further resource and time inefficiencies.
This last issue has been recognized within the federal government, and the Aboriginal and Treaty Rights Information System (ATRIS) being developed by AANDC, and is expected to address it. The ATRIS offers centralized information on the nature and location of treaty rights. This information, combined with enhanced services offered by AANDC to facilitate departments’ consultation efforts, would make it easier to identify which Aboriginal groups needed to be consulted and would assist departments in carrying out strength of claim analysis.
Coordination of the EA and permitting processes between the federal government and individual provinces was generally cited as far more challenging. Although there were cases where this process was viewed as well managed, many interviewees across stakeholder groups mentioned this aspect as a major source of process discord that frequently led to duplication and delays, and sometimes also resulted in gaps in the process. As such, proponents and some federal government representatives advocated for delegation and/or substitution arrangements with provinces that would allow for simplification of the process and having a true “one project, one review” approach. However, that delegation and/or substitution would require mechanisms to ensure that provincial processes meet federal requirements.
Under the current system, several processes within project reviews were found to be duplicated at the federal and provincial level, which could result in proponents providing identical or similar information to each level of government. Although efforts to use the same information request processes were often implemented, these were not always successful. Furthermore, as the two processes were frequently unsynchronized, proponents were required to provide this information at different times and at different levels of detail. As such, a greater level of coordination and collaboration between the federal and provincial governmental bodies would bring further efficiencies.
Delegation of EA responsibilities from the central/national government to regional governments was identified through the comparative analysis in Australia: the Government of Australia accredited EA/regulatory processes in some of its states and subsequently delegated its review responsibilities to the state governments for projects occurring within the states’ jurisdictions.Footnote 109 More details regarding the substitution agreement between the Australian Government and the Government of Western Australia are provided in the following section.
Question: How does the MPMOI’s performance compare to that of similar initiatives or alternative models (in other national or international jurisdictions)?
Quite well. The MPMOI is implementing several best practices seen in other jurisdictions and also has some unique features that may be considered best practices. Features consistent with other jurisdictions include a system-wide approach to reforms, senior management coordination, public tracking, policy research to streamline processes, legislative and non-legislative amendments to environmental protection Acts and related regulations, earlier stakeholder engagement and early public consultations. A more detailed discussion of project review processes in Western Australia identified other practices that Canada might consider implementing to increase efficiencies in its own process.
The MPMOI’s activities were compared to EA/regulatory systems in six international jurisdictions (Western Australia, New Zealand, United Kingdom, South Africa, Hong Kong, and the Netherlands). One of the six jurisdictions, the State of Western Australia (WA), was selected and analysed in greater detail to help answer the question about how well the MPMOI compared to another jurisdiction in terms of effectiveness, efficiency and economy. This section first presents findings from the review of selected international jurisdictions followed by a more detailed comparison of the MPMOI and regulatory reform initiatives in WA. It concludes with an overview of promising international practices identified through the comparative exercise.
International comparison overview
The examination of EA and regulatory review related practices in six international jurisdictions indicates that the MPMOI has been implementing several processes, tools and activities that can be considered as best practices. These include: system-wide approach to reforms, senior management coordination, public tracking, policy research to streamline processes (leading to legislative amendments to environmental protection Acts and related regulations, as well as non-legislative changes), earlier stakeholder engagement, and early and multiple public consultations. Each of these aspects is discussed in greater detail in the conclusion of this section. In addition, the review demonstrates that other jurisdictions around the world are identifying and addressing – through ongoing reforms and targeted initiatives – similar issues to those identified in Canada.
A more detailed comparison of the approvals system (i.e., project review system) in WA, which has faced similar internal and external challenges as the MPMOI, demonstrated that the achievement of outcomes by the MPMOI was similar to the WA approvals reforms. In WA, the Environmental Protection Authority (EPA)Footnote 110 is responsible for the EIA of resource projects, including several large projects. The EPA is supported by the Office of the EPA (OEPA).Footnote 111
The OEPA was, however, not fully comparable to the MPMO because the OEPA (as well as the EPA) engages solely in the EIA phase of the project reviews, whereas the regulatory phase is carried out by individual state departments. Coordination of the overall process is provided by the Department of Premier and Cabinet under the Lead Agency Framework (LAF; discussed below).Footnote 112 While the OEPA’s mandate includes policy support, policy leadership is a key role of the DPC, which coordinates collaborative policy work across relevant departments. Thus, there is no single organization in WA that delivers the dual mandates (operations and policy) of the MPMOI.
For these reasons, as well as data limitations (availability and quality), the comparative analysis was limited to the two offices and the EA/EIA phase. In particular, direct cost-effectiveness comparison could not be performed due to a lack of financial data for both jurisdictions and core differences in their delivery models and regulatory systems. Although a direct comparison of system costs is precluded for reasons discussed above, available expenditure data are presented below to illustrate the use of efficiency indicators and to inform future discussion about the development of efficiency indicators.
The volume of projects should be taken into account when discussing timelines and cost-efficiency across the two jurisdictions. The EPA completes an average of 35-45 project reviews per year, whereas the MPMOI oversaw approximately 70 active projects in 2010-11.
Timeliness: The performance of the MPMOI and the WA review system in terms of total time taken to complete the EA/EIA phase cannot be directly compared because of differing processes and contexts. However, the timeliness of the EA/EIA phase was affected by several similar factors in both jurisdictions, which were often related to the preparation of supplementary information by proponents and the review of information by government department/agencies. These activities typically introduced extra time and variability into the process. On average, EA/EIA phases that were largely under control of the proponent were the longest and most variable in both jurisdictions, whereas phases devoted to public reviews tend to be the shortest.Footnote 113
Both the MPMOI and the OEPA also collected data on the difference between actual and targeted timelines. For each stage of the EA/EIA review, both offices determine whether the review is “on time” or “late” (although the way in which this status is determined varies to a certain extent between the two offices). In both jurisdictions, the rate of timely completion has been between 80 percent and 90 percent since the last quarter of the 2009-10 fiscal year. That said, EIA timelines increased in WA following the implementation of reforms in 2009. Since then, the EPA has recorded some improved efficiency, suggesting future promising results. However, only future performance reports will be able to better establish the level of improvement.
System costs: The MPMO’s expenditures associated with project-level operations (including both EAs and regulatory reviews) have been estimated to be $11,000 per project. However this is only a fraction of the total cost per project, as it does not include expenditures of departments/agencies participating in the project review, for which data were not available. Generally, the cost per project has not been used as a performance indicator by the MPMOI and as such has not been collected by participating departments and agencies.
In contrast, the OEPA has begun to assess, measure and report on (in the EPA/OEPA annual reportFootnote 114) various efficiency performance indicators. These include the average cost per EIA, per environmental policy and per environmental audit completed in the previous 12-month period. The average cost per EIA was nearly AUD$40,000 in 2010-11, which is roughly equivalent to CAD$40,000 over 2010-11.Footnote 115
Comparisons with project reviews under the MPMOI should be made with care for the following reasons: 1) the MPMO estimate does not include a portion of the MPMO operating overhead costs, corporate funding sent to central groups, and costs incurred by other departments/agencies involved in the EA; 2) the MPMO includes costs related to the regulatory/permitting stage and some coordination costs (e.g., PA development) that are not incorporated into the OEPA figure; and 3) the OEPA indicator includes costs relating to conducting the EIA incurred by the OEPA and EPA. Similarly to the MPMO estimate, the EPA figure does not incorporate costs incurred by other departments who provide expert advice or S&T information during the EA/EIA process. However, it is likely that the involvement of the state departments in WA’s EIA process is considerably lower than that of federal departments/agencies in Canadian EAs because the EPA is the only decision-making authority for EIAs in WA, whereas EAs under the MPMOI often involve multiple responsible authorities. Moreover, authorizations relating to Aboriginal rights are not integrated in the EIA process in WA (where there is no equivalent duty to consult), but are examined during the regulatory review phase once the EIA has been completed.
Thus, rather than comparing these figures, the OEPA indicators may serve as input to the development of future performance indicators under the MPMOI.
Unique aspects of the MPMOI
The comparative analysis identified some unique aspects of the MPMOI that were not identified in other jurisdictions. These were the “single-window” office and integration of Aboriginal consultations into the EA and regulatory phases.
Single point of entry: As discussed previously, the MPMO was intended to function as a single-window entry into the federal review system, but its role in this regard was adjusted to accommodate existing relationships between proponents and the federal government (e.g., with the CEA Agency). The MPMO thus acted as a contact point for industry stakeholders to facilitate issue resolution and supported early engagement with proponents.
In 2005, WA had attempted to implement a similar single-window entry into the regulatory process by establishing the Office of Development Approval Coordination (ODAC). However, in 2008, the Auditor General for WA reported that, despite the fact that the ODAC and other approvals agencies had acted on the government’s commitments, the intended outcomes had not been delivered.Footnote 116 In particular, the approval agencies’ compliance with set timelines did not improve despite the ODAC monitoring them. This office was subsequently replaced by the current Lead Agency Framework in 2009.Footnote 117 The LAF is described in more detail below.
Unlike the MPMO, neither the ODAC nor the other agencies conducted any analysis of the timeline data to identify opportunities for improvement.Footnote 118 Nor did the ODAC have a transparent process for determining which proponents (or which projects) would receive assistance, creating inequity in the system. Interviewees from WA also indicated that the ODAC had not sufficiently engaged with various stakeholders to achieve its objectives.Footnote 119
In light of the WA experience, the MPMOI’s focus on increasing the clarity and transparency of the process (e.g., criteria for project inclusion) for both internal and external stakeholders may be considered a best practice.
Aboriginal consultations: Only three of the examined jurisdictions (Canada, Western Australia, New Zealand) have Aboriginal peoples with whom engagement and/or consultation are required through various types of laws and treaties. However, there are significant differences among the formal obligations in each jurisdiction and the subsequent level of engagement.
Canada is the only jurisdiction where the government has a formal duty to consult, as a result of Section 35 of the Canadian Constitution Act (1982) and rulings of the Supreme Court of Canada in 2004 and 2005.Footnote 120 The implementation of the whole-of-government approach under the MPMOI also represents a key difference, as it seeks to fully integrate Aboriginal consultations into the EA and regulatory review processes, rather than have these occur in a parallel process.
In WA, indigenous groups are expected to be consulted by proponents, rather than by Crown representatives. Project proponents are responsible for negotiating agreements (including substantial financial agreements) with Aboriginal groups for projects that will occur on native title land in order to obtain regulatory approvals under the Aboriginal Heritage Act.Footnote 121
In New Zealand, indigenous peoples are explicitly considered to be one of the affected groups that should be consulted by proponents. To assist proponents with this process, the Ministry for the Environment provides detailed guidelines on the tangata whenua (people of the land) consultations, as well as a website providing information on the various groups that could be consulted.Footnote 122,Footnote 123 This information is provided in accordance with the Resources Management Act, which identifies a duty to keep records about indigenous groups and the areas over which they have kaitiakitanga (guardianship), but this does not extend to a specific duty to consult these groups.Footnote 124 In this sense, then, Canada may be considered as a leader in developing a special duty to consult Aboriginal peoples.Footnote 125
Unique practices of Western Australia
The review identified other practices that have been implemented in WA and may be of interest to Canadian policy-makers and regulators in the context of the future improvements to the efficiency and effectiveness of the federal regulatory system.
Bilateral/substitution agreements with states: In 2002, the Government of Australia accredited the WA EIA process in order to minimize duplication and strengthen intergovernmental cooperation. This resulted in a bilateral agreement between the two jurisdictions.Footnote 126 This agreement was conditional to the State of Western Australia’s amending its Environmental Protection Act to conform to the Government of Australia’s Environment Protection and Biodiversity Conservation Act 1999.
As a result of this agreement, WA is now largely responsible for environmental assessment of major projects carried out in the state territory.Footnote 127 The agreement is limited to certain types of projects, such that dual assessments by both the state and the federal government still occur. More specifically, the instances in which assessments are required by both levels of the government are usually large complex projects with strategic national significance,Footnote 128 as well as projects conducted on federal land or proposed by the Australian Government.Footnote 129
In these instances, the Government of Australia may:
- allow the WA-EPA to carry out the assessment;
- decide to collaborate on a joint assessment; or
- decide to conduct its own assessment parallel to that of the WA-EPA.
Interestingly, in the two latter cases, coordination issues were reported which are similar to those occurring between federal and provincial EA processes in Canada. As such, substitution/ delegation agreements with states have been recognized as a best practice in EIA in the review of Australian EIA practices.Footnote 130
Lead Agency Framework (LAF): The LAF was introduced in 2009 to improve the coordination of the approvals processes and to provide clarification of the administrative procedures for proponents. The LAF provides a pre-determined single agency point of entry for project proposals to the industrial sector (i.e., the framework is not designed only for resource projects but also applies to infrastructure). Project proponents therefore work with the agency most appropriate to their type of project (e.g., oil and gas, mining, infrastructure). The level of assistance provided to proponents varies based on the complexity and anticipated impact of a given project (i.e., not just for major projects).Footnote 131
Formal stakeholder engagement: In 2008, the EPA established a Stakeholder Reference Group (SRG) comprising representatives from the industry, non-government organizations and the WA government. The SRG was established to provide input into reforms of the EIA system. After the reforms were implemented, the group was maintained on “an as needed basis” and was reported as being very useful in maintaining a dialogue among various groups of stakeholders. Currently, the SRG meets quarterly to discuss and work through a range of issues, and is used to help inform the drafting of policies, as well as provide guidance to proponents. The SRG acts an informal liaison between the EPA and the industry and environmental organizations, rather than a formal advisory or decision-making role.
Additional International Best Practices
The literature review of major project review practices in six international jurisdictions was also helpful in identifying best practices relating to the regulatory system for major projects. As these practices were mostly changes implemented as part of recent and ongoing reforms, it was too early to determine their outcomes and benefits. However, they could represent promising practices that might bring further efficiency and effectiveness gains to regulatory systems and could be considered in the context of the MPMOI.
Transparency was one of the key – if not the first – elements cited by each of the interviewees from WA when they were asked to identify best practices pertaining to regulatory systems. Transparency was central to best practices relating to public participation, tracking, stakeholder engagement, and the provision of guidance to proponents. Moreover, it was seen as necessary to demonstrate that a credible approvals process is in place, which placed the public interest above industry demands. The MPMOI pursued a number of practices to increase the transparency of the federal regulatory system for major resource projects.
Earlier proponent/stakeholder engagement has become an important element of modern regulatory review systems and was a growing practice across many of the jurisdictions examined (e.g., WA, South Africa, U.K.). Enhanced stakeholder engagement was seen as necessary to achieve future improvements to the effectiveness of the approvals system. In WA, the SRG had been involved in preparing the system reforms and drafting policies, while the proponents had been gradually engaged in the review process following the 2009 reforms. Early evidence suggested that this increased proponent engagement contributed to reducing the number of appeals and increased the overall efficiency of the review process.Footnote 132
Public review is another integral part of EA processes. In all jurisdictions examined, the public has at least one designated opportunity to review and comment on major project proposals, and may have additional opportunities as well.Footnote 133,Footnote 134,Footnote 135,Footnote 136,Footnote 137 The involvement of the public is seen as necessary to ensure the transparency, accountability and credibility of the EA process.Footnote 138,Footnote 139 Consequently, in most of the jurisdictions examined, in the case of complex projects, the public is invited to provide its input at multiple stages of the review process (e.g., before the start of the EA and once the EA report has been released, as well as during public hearings).
Importantly, in countries that have indigenous peoples, Aboriginal consultations can also be viewed as a particular sub-type of public participation (see above for discussion of this element).
Public tracking, including publicly-available target timelines and/or public data on projects currently undergoing EAs, is common in several of the reviewed jurisdictions (e.g., New Zealand, Hong Kong, WA). However, in most of the systems, it is difficult to compare how long a given phase/stage is supposed to take (as indicated by service standards/target timelines) and how long the actual processing time is for individual projects, such that performance and predictability are difficult for users to assess.
WA is currently moving to an improved tracking system. The new Statutory Approvals Tracking System will present information regarding the status of specific applications in a “common-user” platform. In this system, users will be able to see whether the application has been referred to another agency for advice, whether it is under a “stopped clock”, or whether a decision is pending.Footnote 140 The information will be accessible in real-time to government staff (e.g., project managers), proponents and the public.
Of the jurisdictions reviewed for this evaluation, only WA established senior management coordination mechanisms similar to those seen in the MPMOI. With the support of the Department of Premier and Cabinet, two main senior-level committees were created. The first, a horizontal Ministerial Taskforce (involving ministers of relevant departments in WA), provided direction on strategic and policy issues such as the review and implementation of approval reforms, including legislative amendments and the establishment of the new Lead Agency Framework. The second, a Director General Working Group, discussed project-level performance and coordination issues. Interestingly, a senior management coordination aspect relating to regulatory reforms affecting large infrastructure projects is also present in the U.K. As a whole, this evidence indicated that senior-level committees were likely an effective means to identify, communicate and resolve strategic issues pertaining to large, nationally significant projects.
Collaborative policy research, as incorporated into both the MPMOI in Canada and recent reforms in WA, was found to drive legislative and non-legislative amendments to regulations and policies in both jurisdictions. These amendments and policy changes were viewed as promising steps toward a more integrated, modern project review system by a broad community of stakeholders both in Canada and WA. Interestingly, many of the jurisdictions that had recently implemented changes to their major project review systems (Canada, WA, New Zealand, and gradually also South Africa) have increasingly adopted a broad systemic approach to their reforms.Footnote 141,Footnote 142,Footnote 143,Footnote 144 The goal of these reforms was to achieve system-wide changes in processes – often across a range of departments/agencies and/or levels of government – rather than piece-meal modifications of current processes.
CONCLUSION AND RECOMMENDATIONS
This section summarizes the evaluation’s findings, discusses opportunities for improvement, and presents related recommendations that have been developed based on multiple lines of evidence. Overall, the evaluation finds that the MPMOI is relevant, and generally achieved its short-term target outcomes. As such, the nine recommendations are formulated with the view to ensure continuous improvement of the federal regulatory system and achievement of the MPMOI targeted outcomes. The discussion is organized according to the two main groups of the core evaluation issues, relevance and performance.
Note Regarding Recommendations
The MPMOI is a horizontal government initiative that focuses the work of many federal departments and agencies to achieve the objectives and outcomes of the MPMOI, using horizontal governance structure and collaborative activities. In this context, NRCan officials involved with the MPMO are not accountable for the delivery of other government departments and agencies active in the MPMOI. NRCan is responsible for the MPMO and activities delivered within the Department; NRCan’s Deputy Minister also chairs the Deputy Minister’s Committee. Thus, the recommendations made by this evaluation are directed to MPMO in coordination with other departments and agencies participating in the MPMOI.Footnote 145
5.1 Relevance Issues
The evaluation demonstrates strong consensus on the need to further improve the federal regulatory system for major resource projects. This need has been clearly expressed by internal and external stakeholders and observed in all other lines of evidence. Moreover, addressing these issues and ensuring adequate capacity funding are seen to be particularly relevant in the next five to ten years as the number of major resource projects in mining and energy industries (and the potential economic benefits for Canada) are expected to increase, in order to ensure the efficient and effective reviews for these projects. The evidence also indicates that the MPMOI’s mandate and objectives are well aligned with government priorities, as well as with the individual mandates and strategic objectives of the participating departments.
As the systemic issues that prompted the creation of the Initiative are ongoing, there is a continuing need for tools, mechanisms and policy research that help address these issues, at both the operational and policy levels. System-wide approaches, horizontal policy leadership and senior management coordination are an effective means to address these issues and achieve systemic changes across departments/agencies. Thus, the evaluation evidence clearly demonstrates the need to sustain these mechanisms in order to achieve further improvements to the efficiency and effectiveness of the federal regulatory system.
Furthermore, MPMOI capacity funding is found to be essential to ensure the efficiency and effectiveness of the federal regulatory system and thus, remains necessary to address ongoing and growing capacity needs within departments/agencies under the current federal regulatory system (including the improvements implemented to date). The capacity to deliver EAs and regulatory reviews, as well as growing responsibilities relating to Aboriginal consultation will be even further stretched if the number of major projects entering the federal regulatory system grows as foreseen by industry representatives and the MPMO.
The significant achievements of the MPMO in the short timeframe since its creation also demonstrate a continued need for a central function within the Initiative. A central office is necessary to review the MPMOI approach and ensure ongoing continuous improvement to the Initiative, as well as incremental achievements of the Initiative’s longer-term outcomes. The evidence indicates that it is critical that this central entity remains as independent as possible to ensure buy-in across participating departments and agencies. NRCan has been confirmed as the most suitable department to host the central function due to its relatively minor regulatory role in the major project review process.
Although the MPMOI’s objectives as a whole meet those of its key internal and external stakeholders, several findings indicate that the Initiative could, in some cases, better meet the particular needs of departments/agencies and key external stakeholders and its long-term outcomes have not yet been measured. Thus, the Initiative should remain responsive to changes external to the federal government, as well as the current and developing needs and mandates of participating departments and agencies. This involves sustained communication and outreach, as well as continuous improvement of MPMOI tools and processes; these elements are discussed at greater length with their associated performance issues. The horizontal model, supported by a central function and senior-level coordination, is a strong structure through which to implement the related recommendations and ensure the responsiveness of the Initiative to the needs of its various stakeholders.
5.2 Performance Issues
Multiple lines of evidence established that the MPMOI has achieved many of its targeted objectives for the period under evaluation. On the operational side, the Initiative has developed and effectively implemented a broad range of processes to improve the federal regulatory system for major resource projects based on a system-wide perspective, which involves enhanced communication and collaboration among participating departments and agencies. Similarly, the horizontal collaborative approach has allowed the MPMOI to support exceptional policy work, which has led to both legislative amendments and non-legislative changes. When the assessment of the MPMOI’s performance takes into account the challenging and complex nature of the system it has sought to improve (i.e., number of stakeholders involved, within and external to the federal government), it makes its achievements within a relatively short timeframe all the more remarkable.
The tools, procedures and practices implemented under the MPMOI have begun to show results in terms of performance improvements. However, further outcomes in this regard are expected by all stakeholder groups now that the Initiative has matured and is gaining momentum. For example, significant achievements – such as the 2010 CEA Act amendments, the improved consistency and coordination of EAs, regulatory reviews and Aboriginal consultations, both of which have been related to some recent improvements in timelines for milestones under federal control – are seen as the first of several incremental steps towards a more efficient and effective federal regulatory system for major resource projects.
A key element of the MPMOI approach involves horizontal collaboration across departments/agencies at a number of levels, such as through working groups and the DM Committee. The DM Committee in particular, has been found to act as one of the most important drivers of collaboration among departments/agencies contributing not only to policy work (e.g., policy options, whole-of-government approach for Aboriginal consultations) but also to effective short-term issue resolution. Senior management involvement has been a key enabler of both policy and operational improvements under the MPMOI, as well as leading to a number of additional benefits (e.g., networking). It has been cited as a model for horizontal initiatives and will be essential to support further improvements to the federal regulatory system. It has both informed policy options, including fundamental changes and improved coordination with provincial processes. It also has overseen and provided direction with regard to the implementation of future policy changes and pilot approaches.
The evaluation also confirmed that – as stressed by a wide range of stakeholders – significant and long-lasting improvements in efficiency and effectiveness of the federal regulatory system will only come from major changes to the system. These fundamental changes would include further legislative amendments, as well as better coordination and/or other strategies to reduce duplication with provincial processes, ultimately leading to the “one project, one review” goal.
In light of results to date, the horizontal policy leadership approach demonstrated through the MPMOI would be an effective vehicle through which to achieve this goal. This collaborative approach brings both regulators and policy-makers to the same table, as well as creating opportunities for collaborative policy work with external stakeholders (e.g., provinces).
Recommendation 1: MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI maintains its horizontal policy and governance mechanisms as a vehicle to further advance options for system-wide improvements.
Multiples lines of evidence indicated that overlapping or misaligned federal and provincial review processes were an ongoing barrier to the efficiency of overall project reviews. Issues in the coordination with provincial processes – the projects involving Joint Review Panels, being particularly clear examples – were often observed to be associated with delayed timelines, low process predictability, and the negatively impacted satisfaction of external stakeholders. As such, further work to improve coordination and reduce duplication of these processes would likely lead to more efficient project reviews.
Recommendation 2: MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI increases its focus on improving the alignment and reducing the overlap of federal and provincial review processes.
Similarly, multiple lines of evidence indicate that the consistency, integration and coordination of Aboriginal consultations have improved considerably for major resource project reviews since the implementation of the whole-of-government approach under the MPMOI. This approach has reduced the duplication of federal consultation processes. However, this approach remains a “work in progress” and the delivery of consistent, adequate and meaningful Aboriginal consultations remains a challenging process. For example, the transition between the EA phase and regulatory reviews has been identified as an area for improvement. In addition, the provincial and federal consultations are frequently duplicative and Aboriginal groups continue to express concerns about the process through which they are consulted. The MPMOI thus needs to continue to be involved in collaborative efforts to coordinate and improve consultation processes, particularly to ensure their integration into EAs and regulatory reviews from a system-wide perspective.
Recommendation 3: MPMO, in coordination with participating departments and agencies, should ensure that the MPMOI continues to further improve the horizontal coordination, integration, communication and effectiveness of Aboriginal consultations.
With regard to efficiency issues examined in this evaluation, there was evidence of reduced duplication within the federal government that could be attributed to the MPMOI, particularly through the implementation of the whole-of-government approach for Aboriginal consultations. Moreover, further reduction of duplicative processes is expected following the creation of participant funding programs administered by the NEB and CSNC under Budget 2010, which will allow for the substitution of CEA Agency processes (and thus reduced duplication).
However, numerous internal interviewees reported issues with regard to overlap in the project coordination and tracking offered by participating departments and agencies, as well as perceived additional “transaction costs” for certain project coordination, monitoring, and reporting processes implemented under the MPMOI. For department/agency staff in particular, the perception of such transaction costs affected their buy-in with regard to MPMOI to project monitoring, tracking and reporting activities, and as such posed a barrier to the performance of this horizontal Initiative. These costs were reported by a number of departments and agencies (both funded and unfunded) but could not be comprehensively assessed based on the available quantitative resource data.
These comments were usually accompanied by suggestions as to ways to increase the efficiency of the Initiative. This includes continuous improvement of MPMOI tools/processes that seeks to reduce transaction costs and align project coordination/management mechanisms with those offered by participating departments/agencies in such a way that maximizes the cost-efficiency of these processes. Frequent suggestions included reviewing the frequency/level of detail of internal reporting (e.g., Weekly Tracker), and the number of senior-level committee meetings.
Although the involvement of senior management in the MPMOI’s coordination structure was found to contribute to effective issue resolution, it was also identified as creating additional transaction costs. Given the importance of senior-level committees to policy work and strategic issue resolution, this suggests an opportunity to further reserve the DM Committee meetings for strategic and policy issues and reduce its focus on routine operational and project-level issues. The resolution of such issues could be increasingly addressed at lower-level committees and working groups. A similar practice was observed in Western Australia, where the horizontal Ministerial Taskforce provided direction on strategic and policy issues such as the review and implementation of approval reforms, including legislative amendments and the establishment of the new Lead Agency Framework. Meanwhile, project-level performance and coordination issues were discussed by the Director General Working Group.
Limited availability of quantitative resource data impeded efforts to determine the extent to which the efficiency of MPMOI processes could be improved by implementing these suggestions. A more comprehensive assessment of transaction costs would be required to identify which procedures and practices could be modified to increase the efficiency of the MPMOI.
As the MPMOI matures and gains momentum, opportunities will arise to ensure that the Initiative is as cost-effective as possible. Furthermore, recent changes to the federal regulatory system, such as the 2010 CEA Act amendments and enactment of regulated timelines in 2011 place a greater responsibility on the CEA Agency with regard to comprehensive studies. Thus, opportunities exist to review the appropriateness of existing project monitoring/coordination tools in the case of CEA Agency-led comprehensive studies and improve their efficiency through the implementation of revised and integrated tools and processes.
It is also important to stress that, when implementing changes to MPMOI tools and processes in order to improve the efficiency of the Initiative, the MPMO should monitor the impact of these changes on the system. This ongoing exercise would ensure that the long-term objectives of the Initiative (i.e., an efficient and effective regulatory system) continue to be met. Meanwhile, options for alternative/streamlined project coordination, monitoring and management practices that allow both a more efficient use of resources and the achievement of MPMOI objectives should be investigated.
Recommendation 4: MPMO, in coordination with participating departments and agencies, should review current operational procedures, including transaction costs, with a view to improving their efficiency.
The unavailability of sufficient quantitative data on resources (i.e., financial and FTEs) used for pre-MPMOI (i.e., baseline) and post-MPMOI processes prevented this evaluation from assessing the efficiency and economy of the Initiative using detailed quantitative analyses. In particular, the availability of financial and FTE data across departments and agencies varied considerably. Therefore, it was not possible to compile data at the level of the Initiative for planned, actual and in-kind (non-MPMOI) contributions by project or according to type of activity (e.g., EAs, regulatory reviews, Aboriginal consultations, policy work) under the MPMOI.
In the absence of baseline financial data, the assessment of efficiency and economy issues in future evaluations would require improvements in the availability of detailed financial and FTE data across departments and agencies (e.g., by type of activity or by project, as well as non-MPMOI contributions).
Recommendation 5: MPMO, in coordination with participating departments and agencies, should collect financial data (including non-MPMOI resources) on resource use by project and activity within the Initiative.
Project coordination and monitoring practices implemented under the MPMOI to date have a range of benefits in terms of the Initiative’s achievements, such as consistency, integration, coordination and recent improvements to federal timelines. To successfully implement the MPMOI approach and gain the necessary buy-in from participating departments/agencies, a strong emphasis on process and efficiency was essential and contributed to achieving the short-term goals of the Initiative. The MPMOI implemented a robust data collection and performance monitoring system for objectives relating to timeliness. The evaluation found that it needed to consider doing the same for objectives relating to effectiveness and quality of EAs, regulatory reviews and Aboriginal consultations.
The initial focus of the Initiative was on improving the efficiency of EAs, regulatory reviews and Aboriginal consultations, all the while supporting their continued effectiveness. Thus, both the efficiency and effectiveness objectives were supported by the MPMOI approach. Footnote 146 However, multiple lines of evidence indicated that less emphasis was placed on monitoring and reporting effectiveness performance metrics than efficiency metrics. As observed in the comparative analysis and recognized by interviewees, measuring effectiveness poses greater challenges than measuring efficiency. There is a growing need to ensure that the demonstration of objectives relating to the quality and effectiveness of these processes, including environmental protection and mitigation of social and health effects – which also relate departmental/agency mandates – are supported by improved performance measurement tools and metrics at the level of the Initiative. The MPMO has recently supported a study that sought to identify best practices in terms of measuring both efficiency and effectiveness of EAs and regulatory reviews.
A useful framework through which to begin to develop such metrics is through a formal risk assessment approach, in order to identify how best to demonstrate that high-quality EAs, regulatory reviews and Aboriginal consultations are maintained or improved while achieving efficiency gains. As such, this would help ensure that future improvements and fundamental changes to the federal regulatory system have a beneficial effect on not only the efficiency of this system, but also on the quality of EAs, regulatory reviews and Aboriginal consultations, as well as effectiveness outcomes. This will support the accountability of the Initiative on the achievement of all targeted outcomes. Similar to the data collection procedures relating to the efficiency and timeliness of the project review process, the MPMOI should consider leveraging the effectiveness metrics of participating departments and agencies, as well as the results of recent studies on international practices regarding such metrics.
Recommendation 6: MPMO, in coordination with participating departments and agencies, should improve the MPMOI performance measurement strategy by enhancing monitoring and reporting on the quality and effectiveness of project review processes and outcomes.
Relating to transparency outcomes, the evaluation has also found that the current public MPMO Tracker website does not appear to provide sufficient information to ensure the usability of this website for external stakeholders. For example, the current project status (i.e., whether the project is “on schedule”) is difficult to interpret using this tool in the absence of target timelines. The available evidence also suggests that some external stakeholders continue to refer to other sources to obtain project information.
A clear understanding of this issue is hampered by a lack of information on the use and users of the MPMO Tracker, including web analytics data, the characteristics of visitors and their behaviour. It is also unclear for what purpose visitors are accessing the MPMO Tracker, and, as such, what level of detail would be appropriate to meet their information needs (e.g., what type of timeline information and how many milestones). For instance, it is possible that increasing the usability could be achieved by presenting more details on a smaller number of milestones. The MPMO should, therefore, consider gaining greater business intelligence in this regard and subsequently apply the knowledge to increase the usability of the tracker.
The interaction of users with other available tools – such as the CEA Registry (managed by the CEA Agency) Footnote 147 – that provide project information on federal EAs and regulatory reviews should also be considered to identify opportunities to increase the linking and alignment between these tools to further increase transparency of the federal regulatory process and promote ease of use of information about MPMOI projects by external stakeholders.
Given the evolution in the MPMOI, the timing is appropriate to develop and implement performance measures for longer-term MPMOI outcomes (e.g., environmental assessment, regulatory review, and Aboriginal consultation).
Recommendation 7: MPMO, in coordination with participating departments and agencies, should further improve the transparency of project review processes for external stakeholders, notably by identifying public data users’ needs and exploring potential linkages among existing public information tools (including the MPMO Tracker).
The MPMO is one of the most widely recognized features of the Initiative, particularly among participating departments/agencies and industry stakeholders. Industry representatives are increasingly using the MPMO as a point of contact with the federal government with regard to issues facing individual major resource project reviews, or with the regulatory system more broadly. However, the role of the MPMO is not as well known among other external stakeholders, particularly provincial organizations and Aboriginal groups. Confusion regarding the roles of the MPMO versus those of the CEA Agency was also frequently cited with regard to the coordination and monitoring for individual projects.
More generally, few external stakeholders (outside industry associations) and a large proportion of internal stakeholders were aware of the MPMO or the MPMOI’s policy role. For example, within departments/agencies, the perception of the MPMO’s role was largely limited to operational aspects, particularly project monitoring, tracking and reporting of whether project reviews were meeting their target timelines, as well as the development of Project Agreements and supporting tools (e.g., Guide to Preparing a Project Description, MPMO Tracker). This perception was greatest among department/agency staff working in regional offices as well as those involved primarily in project-level work. External stakeholders, proponents, provincial organizations and Aboriginal groups had varying levels of contact with and information about the MPMOI. Their understanding of the Initiative was largely influenced by their knowledge of the MPMOI’s mandate and activities.
As a result, internal and external stakeholders consulted did not necessarily attribute specific changes to the review process to the MPMOI, much less recognize the involvement of the MPMOI in supporting the collaborative work that contributed to these improvements. Examples included: the 2010 CEA Act amendments, the implementation of the whole-of-government approach, and more recent efforts to improve collaboration with provinces. This and the confusion about the MPMO’s role had an impact on the stakeholder’s perception as to the performance and relevance of the MPMOI (i.e., they were less likely to support the renewal of the Initiative if they saw only a limited range of activities, outcomes or benefits).
For department/agency staff in particular, this also affects their buy-in with regard to project monitoring, tracking and reporting activities (especially given their perception of additional “transaction costs” associated with these activities). Some have a limited understanding of how the collected data is used to identify issues, track system-level performance, and inform further improvements. Meanwhile, several external stakeholders (e.g., industry, provincial organizations, and Aboriginal groups) expressed an interest in working and/or communicating more closely with the federal government to improve the regulatory system for major resource projects. However, most – especially those having limited contact with or knowledge of the MPMOI – did not have a clear perception of which forum they could use to participate in such a process.
Overall, these findings indicate that the MPMOI’s performance could be improved if there was increased communication, outreach and engagement – including better communication of its results – with a wider range of internal and external stakeholder groups. Within the federal government, increased awareness of the MPMOI, its activities and achievements would help generate greater buy-in and support for the Initiative, as well as promote the ongoing culture shift the MPMOI is seeking to achieve, particularly outside the National Capital Region. For external stakeholders, more formal stakeholder engagement would provide a means by which to inform future changes and build stronger relationships among the various groups involved in the federal regulatory process across Canada and the federal government.
While these activities should be supported by the MPMO, all participating departments/ agencies are also expected to be involved in outreach and engagement activities in some capacity (as appropriate), and have a role to play in ensuring that information pertaining to the MPMOI is adequately disseminated within their organization. The added-value of formal stakeholder engagement was identified through the comparative analysis.
The evaluation findings found that transparency of the federal regulatory system had improved because of the MPMOI but suggested that further improvements might be warranted. Greater outreach, engagement and communication would help address some of the confusion as to the roles of the various players in the federal regulatory system, including that of the MPMO.
Recommendation 8: MPMO, in coordination with participating departments and agencies, should enhance its stakeholder outreach and engagement activities, and communicate its results more clearly within and outside the federal government, nationally and regionally.
ANNEX A – DEPT/AGENCY FINANCIAL PROFILES: 2007-08 TO 2011-12
|Operating & Minor Capital||149||2,401||1,401||1,401||901|
|Employee Benefit Plan (EBP)||200||466||466||466||466|
|Operating & Minor Capital||825||1,422||1,675||1,512|
|Employee Benefit Plan (EBP)||38||345||448||453|
|Operating & Minor Capital||3,874||4,523||3,098||3,298||2,798|
|Employee Benefit Plan (EBP)||542||1,083||1,083||1,083||1,083|
|Operating & Minor Capital||1,379||4,823||1,933||3,049|
|Employee Benefit Plan (EBP)||184||831||1,217||1,094|
|Operating & Minor Capital||1,508||467||467||467||467|
|Employee Benefit Plan (EBP)||149||306||306||306||306|
|Operating & Minor Capital||500||252||467||498|
|Employee Benefit Plan (EBP)||56||144||194||271|
|Operating & Minor Capital||1,636||1,636||1,636||1,636||1,636|
|Employee Benefit Plan (EBP)||719||719||719||719||719|
|Operating & Minor Capital||813||2,133||991||1,634|
|Employee Benefit Plan (EBP)||86||589||799||751|
|AANDC Balance Sheet: MPMOI Financial Profile: 2007-08 to 2011-12|
|Operating & Minor Capital||520||520||520||520||520|
|Employee Benefit Plan (EBP)||112||112||112||112||112|
|Operating & Minor Capital||444||354||331||166|
|Employee Benefit Plan (EBP)||92||99||125||125|
|Operating & Minor Capital||464||2,212||1,272||1,228||1,183|
|Employee Benefit Plan (EBP)||49||602||335||342||349|
|Operating & Minor Capital||204||461||667||812|
|Employee Benefit Plan (EBP)||51||303||350||374|
Note: [a] MPMOI-funded FTEs only, other FTEs dedicated to the MPMO process not included.
[b] CEAA: Grants & Contributions for Participant Funding Program (PFP).
[c] AANDC: Grants & Contributions; [d] DFO: Real Property, Enablers.
Source: Compiled from data provided by each department and agency.
ANNEX B – STAFFING LEVELS: MPMOI-FUNDED FTES: 2007-08 TO 2010-11
|HQ FTEs Allocated||25||8.5||5||1||4.9||30.9|
|HQ Positions Staffed||1.43||4.7||1||1||3.1||5.53|
|HQ % Staffed||6%||55%||20%||100%||63%||18%|
|Region FTEs Allocated||0||0||37||6||17.5||23.5|
|Region Positions Staffed||0||0||4.5||3||19||22|
|Region % Staffed||0%||0||12%||50%||109%||94%|
|Total FTEs Allocated||25||8.5||42||7||22.4||62.9|
|Total Positions Staffed||1.43||4.7||5.5||4||22.1||32.23|
|Total % Staffed||6%||55%||13%||57%||99%||51%|
|HQ FTEs Allocated||25||29||13.5||5||1||4.9||59.9|
|HQ Positions Staffed||17.22||20||8||3.15||1||3.1||41.32|
|HQ % Staffed||69%||69%||59%||63%||100%||63%||69%|
|Region FTEs Allocated||0||45||5||37||6||17.5||68.5|
|Region Positions Staffed||0||30||4.5||28||3||18.7||51.7|
|Region % Staffed||0%||67%||90%||76%||50%||107%||75%|
|Total FTEs Allocated||25||74||18.5||42||7||22.4||146.9|
|Total Positions Staffed||17.22||50||10.6||31.15||4||21.8||103.62|
|Total % Staffed||69%||68%||57%||74%||57%||97%||71%|
|HQ FTEs Allocated||25||31.5||13.5||5||1||8.2||65.7|
|HQ Positions Staffed||22.25||28||14.5||4.5||1||13.9||65.15|
|HQ % Staffed||89%||89%||107%||90%||100%||170%||99%|
|Region FTEs Allocated||0||42.5||5||37||6||17.5||66|
|Region Positions Staffed||0||28.5||5||31||5||20.1||53.6|
|Region % Staffed||-||67%||100%||84%||83%||115%||81%|
|Total FTEs Allocated||25||74||18.5||42||7||25.7||150.2|
|Total Positions Staffed||22.25||56.5||16.2||35.5||6||34.0||135.0|
|Total % Staffed||89%||76%||88%||85%||86%||132%||90%|
|HQ FTEs Allocated||25||31.5||-||5||1||8.2||65.7|
|HQ Positions Staffed||22.83||28||-||5.26||1||14.2||66.03|
|HQ % Staffed||91%||89%||-||101%||100%||173%||101%|
|Region FTEs Allocated||0||42.5||-||37||6||17.5||66|
|Region Positions Staffed||0||32.5||-||33||6||21.5||60|
|Region % Staffed||0%||76%||-||88%||100%||123%||91%|
|Total FTEs Allocated||25||74||18.5||42||7||25.7||150.2|
|Total Positions Staffed||22.83||60.5||19.5||38.26||7||35.7||145.53|
|Total % Staffed||91%||82%||105%||91%||100%||139%||97%|
Note: MPMOI-funded FTEs only, other FTEs dedicated to the MPMO process not included.
[a] TC only: Numbers do not include FTEs allocated Legal Services, as limited resources were required at that time and T C used its internal Legal Services resources.; [b] TC only: Numbers do not include FTEs allocated Legal Services, for dedicated staffing and related legal activities.
Source: Compiled from data provided by each department and agency.
ANNEX C – PROGRAM LOGIC MODEL DESCRIPTION
Logic Model Narrative
The logic model uses a top-down diagrammatic structure to represent the Initiative’s composition and process flow (Figure 3). The following narrative describes the program theory for the MPMOI, as illustrated in the logic model. The top row, i.e., external factors and influences that affect the MPMOI (0.1 to 0.7), bring attention to the context in which the Initiative operates, including factors that led to its creation. These factors/influences can have either positive or negative effects on the Initiative. For instance, changes in the economic context/markets, regional trends & technology (0.1) could have an influence on the number (either increasing or decreasing) or on the type of major resource projects planned or developed in Canada.
Major resource projects are subjected to a complex regulatory/system (0.2) involving several players throughout the regulatory process, including both federal departments and other stakeholders, namely provincial jurisdictions that have shared responsibilities for resource projects (0.3). Changes in government priorities across the various jurisdictions (0.4) can thus affect the MPMOI in various ways.Footnote 148 Similarly, legislative changes (0.5; e.g., Jobs and Economic Growth ActFootnote 149) or court decisions regarding EA and Aboriginal consultations (AC) can have important implications on the regulatory review process for major resource projects. Capacity issues for the federal regulatory review process and for Aboriginal consultations (0.6) were an important consideration in the establishment of the MPMOI (and allocation of funding). The extent to which capacity needs are being met could influence the effectiveness of the Initiative. Changes in stakeholder and public awareness and expectations relating to EA and Aboriginal consultations (0.7) have several implications with regard to the MPMOI, particularly in terms of transparency, but potentially also leading to changes in the scrutiny of the effectiveness of EAs and regulatory controls (e.g., increased public interest in policies regulating offshore drilling following the Deepwater Horizon oil spill in 2010).
The resources section of the logic model includes not only funding obtained through the MPMOI (1.1 to 1.3), but also in-kind contributions that have supported the delivery of the Initiative’s activities, outputs, and outcomes (1.4). These in-kind contributions also include the work conducted by non-federal government stakeholders (i.e., other participants/contributors in the EA process, including proponents, provincial government departments, Aboriginal community members, etc.). Note that capacity funding for Aboriginal consultations (1.3) is qualified as “interim” in the Horizontal Management and Results-based Framework, indicating that a general approach to Aboriginal consultation would be implemented on an interim basis within the MPMOI while a broader federal policy is developed.Footnote 150 Some MPMOI funding has also been provided to Aboriginal groups, which is included under 1.3. The scientific/technical expertise and transactional support provided by personnel involved in the MPMOI, including MPMO staff (1.5; also includes infrastructure), full-time equivalents funded by the MPMOI (1.6) and FTEs supported by in-kind contributions (1.7), are also illustrated. Note that 1.7 includes the DMs and other senior staff involved in governance/coordination of the MPMOI.
No distinction is made between programs and activities within the MPMOI that are delivered by the MPMO and those that are delivered by the participating departments and agencies although it is understood that certain activities are primarily led by specific groups. Activities are listed in two rows (2.1 to 2.11); the top represents activities specific to the MPMOI whereas the bottom row (in orange) represents activities within the regulatory review process that are supported by the MPMOI (via capacity funding, and following the processes put into place for major resource projects). The activities are described in detail in Section 2.7.
Turning to the outputs, the service standards (3.1) are informed by policy and research work (2.6), whereas the achievement of these service standards (i.e., reduced timelines, as measured by the MPMO) are expected to result from several of the other activities, including the provision of a single point of entry for proponents (2.1), the consistent and coordinated approach that has been implemented (2.2), the integration of the Aboriginal consultations (2.3), and the monitoring, tracking (2.4) and resolution of impasses identified (possibly via the DM Committee, 2.5). Similarly, the implementation of a consistent, coordinated approach to EA and regulatory reviews includes primarily the development of project agreements (3.2), but also leads to inter-departmental/agency collaboration (3.5). The latter also results from the activities relating to senior governance and coordination mechanisms (2.5). Activity 2.4 generates the production of progress reports for internal project management purposes (including weekly, monthly and quarterly reports), as well as publicly-available performance information via the MPMO Tracker (3.4). Meanwhile, activity 2.6 includes not only the policy and research work led by the MPMO, but could also involve intra or inter-departmental initiatives to improve processes, such as training programs or pilot programs to delegate certain responsibilities to provinces. The results, findings and reports relating to such initiatives would therefore be captured in box 3.6.
The activities in the logic model, in producing the outputs 3.1 to 3.6, are expected to result in the immediate, intermediate and long-term outcomes shown in the logic model. Conceptually, several activities and outputs will lead to a consistent, whole-of-government approach to Aboriginal consultations for major resource projects under the MPMOI (4.1), such as 2.3 and 3.3, policy and research work (2.6 and 3.6), and Aboriginal consultation milestones and work plans outlined in the project agreements (3.2). All of the activities listed (and their outputs) will contribute to having well-integrated and well-coordinated EAs and regulatory reviews for major resources projects under the MPMOI (4.2). The increased transparency and accountability of the federal regulatory process (4.3) are expected to be linked to the service standards (3.1), project agreements (3.2), and particularly to the progress reporting mechanisms and publicly-available information (i.e., MPMO Tracker; 3.4). Policy and research activities will be key to the continuous improvement of the federal regulatory review process (4.4); however, all of the lessons learned from other activities should feed into this outcome as well.
In the more intermediate term – if the short-term outcomes are realized – this is expected to contribute to Aboriginal consultations which are fulfilled in a consistent, adequate and meaningful manner (5.1), and EAs and regulatory reviews which are high-quality, predictable and timely (5.2). It should be pointed out that the program theory indicates that the quality of the EAs and regulatory reviews should be at the very least maintained as a result of the MPMOI. Improvements to their quality and effectiveness would also likely be possible. The paired effectiveness and efficiency of the regulatory review system (6.1) are defined as a long-term outcome of the MPMOI, coupled with an increase in the competitiveness of the Canadian resources industries (6.2). Major resource projects are expected to benefit from a federal review process that would be more predictable and less prone to delays.
ANNEX D – DETAILED MPMOI ACTIVITIES
New major resource projects entering the federal regulatory system under the MPMOI undergo four key phases: Project Description/Project Agreement, Environmental Assessment, Regulatory Decision-Making, and Follow-Up and Monitoring. Some projects enter the process via a pre-submission phase.Footnote 151
- Development of the Project Agreement (PA): Once a project proponent submits a project description, the MPMO and potentially involved federal departments/agencies begin developing a PA. PAs are consensus-based agreements identifying the roles, responsibilities and statutory obligations of the entities involved and the guidelines and procedures to be followed as per the project scope. PAs are developed based on consultations with involved federal departments and agencies, with the MPMO being responsible for the coordination of the process.
- Environmental Assessment (EA): The key steps to be undertaken in the EA phase are governed by the CEA Act, with activities being largely delivered by the CEA Agency and the relevant departments/agencies. The EA identifies possible environmental effects, proposes measure to mitigate adverse effects and predicts whether there will be significant adverse environmental effects, even after the mitigation is implemented.Footnote 152
- There are three types of EAs conducted in Canada – screening, comprehensive study, and panel review. It is during this phase when the public has an opportunity to participate in the review and provide their comments on the proposed projects. The CEA Agency acts as the Crown Consultation Coordinator for Aboriginal consultations, while the MPMO maintains all official Crown records of Aboriginal consultations.Footnote 153 When the NEB is the EA lead, the MPMO plays a larger management/lead role in the establishment and chairing of a Crown Coordination Committee for the consultation process. The EA phase is concluded when an EA report is finalized.
- Regulatory Decision-Making: If the EA decision is favourable to the project, permits, licenses and authorizations are then completed by relevant departments/agencies (having generally been initiated during the EA stage), which then allow physical work to be initiated or operations to commence.
- Follow-Up & Monitoring: Once the project is underway, the RAs have responsibilities under the CEA Act to ensure the implementation of any mitigation measures and the design and implementation of a follow-up program.
Department/Agency Specific Activities
Each department and agency involved in the MPMOI has specific roles and responsibilities relating to its statutory duties, functions and obligations, as outlined in the MOU and (for specific projects) in the project agreements.
Canadian Environmental Assessment Agency (CEA Agency)
The CEA Agency has administrative and coordinating responsibilities pursuant to the Canadian Environmental Assessment Act (CEAA). Under the MPMOI, the Agency plays an enhanced coordination role, facilitating communication and cooperation among federal authorities in the EA process.Footnote 154 Its activities include organizing public participation activities; preparing EA documents such as work plans, scoping documents, and reports for approval by RAs; managing the public Canadian Environmental Assessment Registry; and conducting training sessions.
The CEA Agency also acts as the federal Crown Consultation Coordinator, leading the integration of Crown consultation activities into EAs for major resource projects. In addition, as a result of targeted modifications to the CEAA that came into effect in July 2010, the CEA Agency is now the responsible federal authority for most comprehensive study assessments (excluding those for large energy projects led by the NEB and CNSC),Footnote 155 resulting in greater decision-making responsibility for the Agency. Prior to these changes, the department(s) responsible for the major project review in question were also responsible for leading the comprehensive study (i.e., as the lead RA).
Environment Canada (EC)
In fulfilling its role as an expert federal authority under the CEA Act, EC provides advice, guidance and information to federal departments and agencies carrying out EAs. EC also regularly acts a responsible authority under the CEA Act and so has additional responsibilities to ensure it meets the Crown duty to consult and effectively discharges its regulatory decisions. When acting as a federal authority in MPMOI projects, EC is often asked to provide advice on environmental issues to responsible authorities, provide technical information for panel hearings or discuss work on a previous EA. With respect to EC’s regulatory function, the Department issues various licenses, permits and authorizations for major resource projects (e.g., ocean disposal permits, International River Improvement Act licenses).
With the new funding received through the Initiative, EC has increased capacity for the provision of expert advice on EAs for major projects, developed tools that increase the efficiency and coordination of major project EAs (e.g., systems for streamlining and tracking expertise engagement, project approval, tracking and follow-up/monitoring) and furthered the development of an Aboriginal consultation planning and expertise.
Fisheries and Oceans Canada (DFO)
DFO has regulatory responsibilities relating to the Fisheries Act (for which it issues authorizations following the EAs, when necessary and appropriate), the CEA Act and the Species at Risk Act. The Department is a highly active RA on major resource projects, primarily due to the many authorizations issued. These efforts are led by DFO’s Major Projects Group. Meanwhile, DFO’s Policy Group, which focuses primarily on the streamlining of reviews under the Fisheries Act, has also worked with the MPMO on policy issues related to major resource projects and in furthering the Initiative’s regulatory improvement strategy.
Moreover, DFO carries out Aboriginal engagement and consultation activities for the regulatory review associated with the Fisheries Act and for EAs more generally. The Department also coordinates processes with EC for tailings impoundment areas under the Metal Mining Effluent Regulations, as well as other processes with MPMO and provincial/territorial processes.
Aboriginal Affairs and Northern Development Canada (AANDC, formerly INAC)
In fulfilling its role as an expert federal RA under the CEA Act, AANDC provides advice, guidance and information to federal departments and agencies carrying out Aboriginal consultations, as well as internal EA expertise to other AANDC sectors, as required (e.g., the treaties and Aboriginal government group). This includes providing guidelines, training and advice (e.g., which communities to consult) to help other departments meet the “duty to consult”.
AANDC’s Consultation and Accommodation Unit (CAU), created in 2007-08 under the Cabinet Directive on Consultation, is a discrete unit at AANDC that has played a key role in the MPMOI. It provides support to federal departments to increase their consultation capacity, participates in all MPMOI committees, has created the internal guidelines for consultation for federal officials in all departments and conducts training for federal officials. It functions as the liaison between AANDC and the MPMO even though the CAU was not funded through the Initiative, as it was not in place when the MPMOI funding was received. Instead, the unit is funded through the AANDC A-base as part of its in-kind contribution to the Initiative.
It is also AANDC’s legal duty to determine the appropriate accommodation measures when the rights of Aboriginal groups are found to have been infringed. Under the MPMOI, AANDC staff members assist departments in developing consultation work plans.
AANDC also received funding under the MPMOI for Aboriginal capacity development (i.e., grants and contributions for Aboriginal groups) supporting AANDC’s operational policy to work closely with Aboriginal peoples directly impacted by major resource projects. Initiative funding has also allowed AANDC to focus on improvements of the northern regulatory regime and the development of a comprehensive approach for projects in the North.
Transport Canada (TC)
TC issues many approvals under the Navigable Waters Protection Act (NWPA) and, therefore, carries out many EAs as a RA under the CEA Act, in addition to performing technical reviews, compliance monitoring, and assessments of cumulative effects. EAs for MPMOI projects are conducted by staff in TC regional offices, with coordination by TC headquarters, as required.
TC engages in a number of activities that may trigger the legal duty to consult with Aboriginal groups, including regulatory approvals, environmental remediation, contribution funding, land development, property management, disposal of lands, policy and regulation development. Created on April 1, 2010, TC’s Aboriginal Consultation Unit provides operational and functional support to TC and works collaboratively with other departments, such as AANDC and the Department of Justice, to ensure that the legal duty to consult with Aboriginal groups is met in a consistent manner throughout the Department.
Canadian Nuclear Safety Commission (CNSC) and National Energy Board (NEB)
The NEB and the CNSC act as the EA lead for projects falling within their area of regulatory responsibilities. Both the NEB and the CNSC are also involved in Aboriginal consultations; however, because the NEB cannot fully represent the Crown in these processes, it works with the MPMO (which chairs the Crown Oversight Committee) to ensure that consultations are consistent and fulfill the Crown’s duty to consult. The CNSC is an agent of the Crown and as a "cradle-to-grave" regulator, acts as the Crown Consultation Coordinator for nuclear projects throughout their life-cycle. Moreover, CNSC and the NEB are active participants in related MPMOI-related activities to improve the federal regulatory system for major pipeline projects (NEB) or uranium and nuclear projects (CNSC). Note that neither the NEB nor the CNSC are supported by MPMOI funds.
Natural Resources Canada (NRCan)
Specific to EAs under the CEA Act, NRCan is an RA for major projects primarily as a result of its regulatory responsibilities (e.g., the Explosives Act) and through the provision of incentive funding to enable certain projects to proceed. NRCan also plays a key role in providing its available specialist or expert knowledge on topics such as earth sciences (e.g., groundwater, seismic hazards, permafrost, landscape process), mining (e.g., acid rock generation and mine waste management), and forestry, when requested by a review panel or other RAs. Note that its departmental responsibilities (as opposed to the MPMO) are not supported by MPMOI funding.
NRCan is actively involved in MPMOI project agreements, interdepartmental working groups, and in regulatory and policy research, as well as in the various Aboriginal consultation groups chaired by MPMO (e.g., the Crown Oversight Committee).
ANNEX E – CHARACTERISTICS OF CASE STUDY PROJECTS
|1||Darlington New Nuclear Power Plant||Ontario Power Generation Inc.||ON||Nuclear||EA||Review Panel||CNSC, DFO, TC, CTA||Yes|
|2||Detour Lake Gold Mine||Detour Gold Corp.||ON||Minerals/ Metals||EA||Comp. Study||CEAA, DFO, NRCan||Yes|
|3||Groundbirch Pipeline||TransCanada Pipelines||AB/BC||Pipelines||F-UP||Screening||NEB, TC||Yes|
|4||Jackpine Oil Sands Mine Expansion||Shell Canada||AB||Oil & Gas||EA||Review Panel||DFO, TC (likely)||Yes|
|5||Labrador-Island Transmission Link||Nalcor Energy||NL||Electricity†||EA||Comp. Study||CEAA, EC, DFO, TC||Yes|
|6||Prosperity Gold-Copper Mine||Taseko||BC||Minerals/ Metals||REG||Review Panel||DFO, NRCan, TC||Yes|
|7||Star-Orion South Diamond Mine||Shore Gold Inc.||SK||Minerals/ Metals||EA||Comp. Study||AANDC, CEAA, DFO, NRCan, TC||Yes|
|8||Joslyn North Mine Project||Total E&P Canada Ltd||AB||Oil & Gas||EA||Review Panel||DFO (originally TC)||No|
|9||Keystone (original)||TransCanada Pipelines Ltd||AB/SK||Pipelines||F-UP||Screening||AANDC, DFO, NEB, TC||No|
|10||Schaft Creek Mine Project||Copper Fox Metals Inc.||BC||Minerals/ Metals||EA||Comp. study||CEAA, DFO, TC, EC, NRCan||No|
Notes: †Electricity: Wind, Hydro & Transmission projects have been grouped under the sector “Electricity”.
*Status: as of April 15, 2011; EA: Environmental Assessment phase; REG: Regulatory Decision-Making phase; F-UP: Follow-up and Monitoring phase; Comp. Study = Comprehensive Study
- Date Modified: