Evaluation of the Environmental Studies and Assessments Sub-Activity (2007-08 to 2011-12)

Executive Summary

Introduction

This report presents the findings of an evaluation of the Environmental Studies and Assessments (ESA) Program Sub-Activity 2.3.3 in Natural Resources Canada’s (NRCan’s) 2012-13 Program Activity Architecture.^{Footnote 1} The ESA is administered by NRCan’s Earth Sciences Sector (ESS) - Central and Northern Canada Branch within the Geological Survey of Canada (GSC). The evaluation covered NRCan’s direct program spending of $21.6 million over the period from 2007-08 to 2011-12.

The overall objective of the ESA Sub-Activity is to provide scientific knowledge and expertise that allows federal regulators and stakeholders to make informed decisions about responsible natural resources management including ecosystem risk and protected areas management.

The three components of ESA are described below:

• Environmental Geoscience Research Program (EGP): This program focuses on characterizing environmental impacts of metals mining, northern energy development, renewable energies, shale gas, northern mineral development, carbon capture and storage and the oil sands development. It was designed to generate new geoscience knowledge to support the development of guidelines and best practices that allow federal regulators and industry to make informed decisions about ecosystem risk management in resource development projects.
• Environmental Assessments service (EA):Through this service, NRCan provides the geoscience expertise and information to federal departments for the purposes of reviewing projects undergoing federal environmental assessment reviews as required under the Canadian Environmental Assessment Act [CEA Act 1992, Section 12 (3)^{Footnote 2}]; Inuvialuit and Nunavut Land Claims Agreement Acts; Yukon Environmental and Socio-economic Act; and the Mackenzie Valley Resource Management Act. It was designed to protect the environment by ensuring decisions on projects subject to environmental assessment reviews (e.g., mines, hydroelectric projects, linear infrastructure, nuclear facilities, and pipelines) are informed by NRCan geoscience advice and information.
• Resource Assessments Service (RA): Through this service, ESS-GSC conducts the scientific studies to qualitatively and/or quantitatively assess the resource potential of areas under federal jurisdiction that are proposed for protected status. ESS-GSC then publishes the results as peer-reviewed GSC Open Files. ESS-GSC experts also participate in community and stakeholder consultations to respond to any questions regarding the resource assessments done under the Mineral and Energy Resource Assessment (MERA) process. Furthermore, ESS-GSC along with NRCan’s Minerals and Metals Sector (MMS) and Energy Sector (not covered under this evaluation) contribute to the economic and strategic analyses conducted once the potential has been assessed and published. These reports, which are not formally published, are used to inform the Senior MERA Committee and may also be informally presented during the community consultation process. This service was designed to ensure that the withdrawal of land or water for national park or marine protected area purposes is balanced between ecosystem values and potential for exploration, development and use of mineral and energy resources.

NRCan’s Strategic Evaluation Division (SED) conducted this first evaluation of ESA over the period from April 2012 to February 2013, addressing the issues of relevance and performance through multiple lines of evidence. These consisted of a document and literature review, a file review, 32 interviews (16 internal to ESS and 16 external to ESS^{Footnote 3}), 13 case studies (4 projects for each component and one comparison of service models between ESS EA service and Fisheries and Oceans Canada and Environment Canada) and an online survey with ESS geoscientists who have contributed their expertise to the EA and/or MERA processes. The methodology was designed based on a risk assessment that the EA service was higher risk (increasing demand and high public profile) and therefore received the most attention, followed by RA service, and EGP.

Key Evaluation Findings

Relevance

Overall, the evaluation finds that the components of ESA are relevant. They respond to a demonstrated need to support federal decision-makers in making environmentally responsible decisions on mineral and energy resource development and land use. They are aligned with NRCan Strategic Objective # 2: “natural resource sectors and consumers are environmentally responsible” and support other federal departments in fulfilling their regulatory and monitoring roles (e.g., CEAA, CNSC, Parks Canada, AANDC, DFO, and EC^{Footnote 4}). There is a necessary and appropriate role for the federal government, and NRCan in particular, in these activities. ESS-GSC is the federal government centre for geoscience expertise, possessing a staff of very high-calibre geoscientists, and NRCan is seen as a neutral scientific honest broker in the federal system.

The EGP program responds to the need to fill scientific information gaps for regulators and industry (i.e., project proponents) when making decisions on development of natural resources. This includes environmental science and federal policy initiatives and industry best practices to characterize and mitigate environmental risk. In some cases, it includes decisions made based on the EA service. The EA service responds to legislated requirements under CEA Act 1992 (now CEA Act 2012) and Inuvialuit and Nunavut Land Claims Agreement Acts; and Nunavut Impact Review Board); Yukon Environmental and Socio-economic Act; and the Mackenzie Valley Resource Management Act to provide expertise. The MERA service responds mainly to obligations under the 1980 Policy/Process articulated in the MERA Terms of Reference (1980 TOR), the 2002 National Marine Conservation Areas Act, and the 1996 Oceans Act.

Both EA and RA services have legislated mandates. The demand for EA services has increased in terms of volume of requests (314 requests during 2007-08 to 2011-12 vs. 198 in 2002-03 to 2006-07), complexity and duration of the reviews. The number of requests for resource assessments under the 1980 MERA TOR appears to be diminishing due to devolution of land management to the Territories. There are currently no new national parks or protected areas being proposed in the North that would require the 1980 MERA process. However, future federal needs will likely focus on informing the establishment of marine protected areas. Although relevant, there are also some questions with respect to the appropriateness of certain aspects of NRCan’s role.

ESS RA service has been asked to provide resource assessments to national marine conservation areas, administered by Parks Canada, and marine protected areas (MPAs), administered by DFO. MPAs are not part of the MERA process since DFO is not a party to the 1980 TOR. Constraints arising from an unclear decision-making process and resources available to ESS have limited the service’s ability to inform federal decisions on these protected areas. ESS has engaged with DFO at various times to clarify what NRCan’s obligations and role should be when requests for resource assessments for MPA establishment processes have been made.

Senior ESS representatives participated in the work of the interdepartmental Resource Assessment Task Group (including NRCan, Parks Canada, DFO, EC, and AANDC) to explore this issue. Overall, the May 2012 Task Group report recommended that departments with relevant expertise (including NRCan) provide input to MPAs, and that a formal process for MPAs should be established. NRCan agreed with the findings of the Task Group’s report and continues to work with DFO to identify measures for involving ESS-GSC in marine resource assessments and implementing the recommendations of the Task Group. Note that the Oceans Act (1996) identifies DFO as the lead department for coordinating implementation of MPAs.

While the EGP program appropriately engages potential users in developing projects, there are opportunities to strengthen coordination with policy groups (i.e., those tasked with assessing and monitoring environmental effects) given that policy needs can evolve ahead of the science. ESS documents indicated that some potential federal environmental assessments or regulatory decisions in rapidly developing areas of in-situ oil sands and shale gas may be undertaken before ESS could develop the capacity to address the related environmental geoscience knowledge gaps.^{Footnote 5} Although ESS has made connections and has added oil sands and shale gas research to the Program, several ESS interviewees raised concerns that policy groups can become disconnected from emerging science, and that EGP projects should try to strengthen those connections as much as possible.

Performance – Effectiveness

The ESA Sub-Activity has performed very well at meeting its legislated obligations and objectives of identifying ecosystem risks of environmental impacts from development (EGP), informing stakeholders of the validity of proponent environmental impact claims (EA), and the resource potential of proposed protected federal lands (RA).

The EGP component, while still a new program (originating in 2007 as the Environment and Health Program and restructured in 2009 as EGP), has demonstrated strong performance at developing geoscience evidence to identify and characterize risks of environmental impacts from resource development. Stakeholder interviews and case studies show that the innovative methods being developed by EGP research projects have strong potential for characterizing impacts from natural resource development. Examples of strong progress identified in interviews and ESS documents include projects on geohazards mapping, metals mining contamination, risks to seabed from tidal energy development, quantifying and characterizing contaminants in oil sands development, and understanding land subsidence risks from natural gas projects.

EGP projects are generally on track and preliminary results are encouraging. For example, research on isolating natural from anthropogenic sources of nitrogen has shown strong promise for oil sands monitoring, while research on characterizing lake sediments is valuable for assessing cumulative environmental effects from mining. These EGP projects have been facilitated by their focus on combining geoscience with other disciplines such as ecology. For example, the Nitrogen and Sulphur project case study showed strong progress on using an innovative approach to isolating anthropogenic nitrogen and sulphur contamination using isotope deposits in tree-rings for assessing cumulative environmental impacts from hydrocarbon extraction. This technique has never been used in this way before.

The unique and innovative nature of the research has also meant that adoption by industry of new techniques and guidelines is still five to ten years away in most cases. The Program’s permafrost research is a notable exception because of its use by partners in planning pipeline projects, and the long-standing industry relationship that predates the EGP. The Permafrost research case study suggests that information provided through this research allowed the project to make substantial progress to meet final outcomes to both mitigate conditions that may be hazardous to pipelines and to reduce the impact of pipelines on the environment.

The EA service has performed well on informing stakeholders of the validity of proponent conclusions and proposed mitigation strategies in proposed projects undergoing federal environmental assessment review. ESS-GSC researchers have provided strong technical reviews of proposed projects, advice, and presentations to regulators, responsible authorities, First Nations, provinces and project proponents. ESS-GSC reviews are widely seen to be of high quality and greatly valued by decision-makers. Scientific and technical presentations to review panels and community stakeholders have been credited with distilling complex geoscience for decision-makers, and informing stakeholders of the validity of project impact claims and mitigation strategies.

ESS-GSC reviews of the validity of proponent conclusions have led to project improvements, and crucial changes to proposed projects. The evaluation found many examples of crucial changes made to proposed projects resulting from ESS-GSC review of environmental impact statements that proponents were able to correct early in the project to avoid issues at a later date. For example, ESS-GSC review helped the Victor Diamond Mine project proponent to better monitor methyl-mercury impacts. On another project, ESS-GSC reviewers discovered historic landslides during the course of their own previous research that could have generated tsunami waves. ESS-GSC was able to submit additional evidence that the proponent will consider in its design. External interviewees also described shortcomings in proponent hydrogeological models in mining projects identified by ESS-GSC researchers, resulting in proponents doing further groundwater monitoring work based on ESS-GSC requests.

ESS-GSC has informed stakeholders of mineral and energy resource potential for federal lands (onshore and offshore) proposed for protection with high-quality studies and presentations. The evaluation found ESS contributions to the terrestrial MERA process for northern national park establishment to be very well delivered in terms of quality, detail and appropriateness of the information provided. All interviewed stakeholders in the MERA process indicated that ESS resource assessment studies are of very high quality and are well used by federal authorities in park boundary decisions. Conclusions reached by ESS-GSC on the mineral potential within the areas of interest were described as scientifically defensible, and have gone unquestioned by decision-makers within MERA due to the high quality work behind them. ESS-GSC presentations to the MERA working group were described as clear, well prepared, and bringing to bear a wealth of subject matter knowledge.

Parks Canada organizes community consultations in national park establishment processes as part of the MERA process. External non-NRCan interviewees reported that ESS-GSC staff have done well at distilling potential mineral findings for Aboriginal communities in the North. ESS-GSC researchers were said to have been well prepared at these sessions with plain language assessments of the mineral potential.

However, concerns have been raised over GSC’s ability to respond to some non-terrestrial resource assessment needs mainly due to an undefined process and unclear obligations. While national marine conservation areas (administered by Parks Canada) are covered under the 1980 MERA TOR, the process and obligations for GSC in providing assessments of DFO-administered marine protected areas are not clear. This has led to an uneven response from GSC to requests for assessments. In cases where human resources were strained, GSC has nevertheless committed to support these assessments by developing statements of work for assessments to be contracted out by DFO. When GSC has contributed to resource assessments for MPAs, those assessments have been of high quality and information value.

Federal decisions on resource development and land use have taken into account sound geoscience evidence and advice provided through EGP, EA, and RA. EGP has provided new geoscience evidence and guidance to federal regulators (e.g., mercury guidelines to HC, AANDC, and EC). There are numerous instances where federal decisions took into account geoscience evidence and advice provided by the EA and RA services, and stakeholders consider GSC input to these projects and review processes as a key part of their decision-making process.

Major examples of the impacts of GSC research and advice on federal decisions include:

• The Mackenzie Gas Pipeline review where ESS-GSC research and advice was an essential part of decisions made on, and the government’s response to, the Joint Review Panel (JRP) recommendations. The Permafrost research now part of EGP has had significant impacts on government decision-making in this review. Evidence from case studies, interviews and documents showed that EGP Permafrost research allowed the federal authorities to know what issues to include in the Environmental Impact Statement, to have the knowledge and expertise to review the proponent proposal and to respond to JRP recommendations.
• As well, two large northern terrestrial parks were established during the evaluation period directly using GSC RA studies (i.e., Nahanni and Nááts’ihch’oh). The results of GSC’s RA study contributed to an expanded Nahanni Park boundary that maximizes mineral potential and conservation values. The accuracy and detail in GSC’s analysis helped to shape the final boundaries and significantly informed the deliberations of the Working level and Senior MERA committees. The Nááts’ihch’oh Park’s boundary decision was based in part on the Nahanni research and on a boundary proposal developed by the ESS-GSC and Parks Canada, MERA Working Group co-chairs.

While ESS performance has been good in terms of informing federal decisions, challenges with the late timing of recent RA results contributed to delays in the Parks Canada consultation process (e.g., the current Thaidene Nene Open File was delivered three years behind schedule) with Aboriginal stakeholders and consideration of final results by the MERA Committee.

A key factor in the results achieved by the ESA component programs has been the depth of ESS-GSC expertise and the availability of researchers. ESS has been able to provide expertise to EA reviews and to terrestrial MERA. However, ESS has contributed less to some MPA assessments than in the past due to resource constraints. ESS has contributed to all MPA resource assessments by at least providing advice to DFO on setting up external contracts and providing critical reviews of assessments that were completed under those contracts.

The survey of ESS-GSC researchers indicates that the two services are at risk of losing much of their expertise over the next 5 to 10 years. In total, 49% of EA reviewers plan to retire in under 10 years, including 23% within the next 5 years. Similarly, 7 out of 13 surveyed researchers that contribute to the RA service plan to retire within 5 years.

Performance – Efficiency and Economy

Overall, the ESA Sub-Activity is efficient and economic, employing good management practices and leveraging internal ESS expertise and external resources. By all accounts, the EGP program is operating well. The EA and RA Services are also delivered economically using a lean service delivery mechanism to administer the processes they inform and leverage expertise already employed by GSC to respond to service requests as needed (all contributors are practicing research scientists). The office of ESS EA Coordinators efficiently assigns reviewer expertise to EA files, and manages the process with NRCan’s Science and Policy Integration Sector (SPI). This is consistent with practices identified in other departments with similar obligations.

EGP and the RA service have leveraged expertise and in-kind resources from NRCan, other government departments, and private sector partners to accomplish research agendas. EGP, an on-going A-base program, acquired funding from NRCan’s Program of Energy Research and Development and Clean Energy Fund, and project managers have invested in developing relationships with industry to leverage significant financial and in-kind support for permafrost monitoring projects. Recently completed MERA studies for Thaidene Nene and Nahanni National Parks effectively shared Government of Canada equipment and resources to complete their work. This includes benefitting from logistical support from the ESS Geo-mapping for Energy and Minerals (GEM) and National Geochemical Reconnaissance Program (funded by the Yukon Geological Survey).

Notwithstanding these good management practices, the evaluation identified some risks to the efficiency of the ESA Sub-Activity, particularly with respect to delivery of EA and RA services. These are described below.

As ESS experts are not hired specifically for responding to EA requests, issues have been identified with prioritizing EA/RA service obligations over other ESS program activities. Concerns were raised in interviews and case studies that prioritizing among these commitments is a challenge for researchers. This may contribute to inefficient delivery of EA reviews and the late timing of recent MERA studies noted earlier.

A more structured planning process to identify potential upcoming requests for expertise would be consistent with the process in place at DFO, identified in the EA service comparison case study. Annually, a secretariat within DFO’s Ecosystems and Oceans Science Sector undertakes a planning process to develop the sector’s research plan for the upcoming year and includes known requests for advice when prioritizing planned work. This allows the Ecosystems and Oceans Science Sector management to plan and prioritize the overall work of the sector, including both direct research and EA support work.

The EA review timelines are currently being met; however, evidence suggests that there are ongoing challenges with efficiency of NRCan (i.e., ESS and SPI) EA review process. ESS reviewers are experiencing challenges with the time pressures in the EA review process structure at NRCan and even stricter timelines are being implemented with CEAA 2012. Evidence from the survey and interviews indicates that the time allotted for expert review of proponent project documents is short. According to the survey of EA reviewers, 43% of respondents noted challenges with meeting reviewer deadlines. Interviewed expert reviewers, coordinators, regulatory authorities and members of SPI all suggested that there were opportunities to improve efficiency of how reviewer comments are approved and provided to CEAA.

Most internal and external stakeholders noted that in the timelines the federal government has to review an environmental impact statement (EIS), the balance between expert reviewer time and other aspects of administering the EA review process at NRCan could be improved. Evidence from interviews suggests that efficiency of manager approvals of reviewer comments has improved recently with a new project tracking process introduced in the ESS EA office in 2012-13 and the introduction of project agreements in place for approximately 85% of EA reviews. The comparison case study indicates that other departments providing expert review to EA process are revisiting their approval structures amid concerns over time constraints as well.

Conclusions and Recommendations:

ESA activities are relevant, meet an ongoing need for geoscience research and advice, and are uniquely provided by ESS. The ESA components have performed well at meeting obligations and objectives of identifying risks of environmental impacts, and informing stakeholders on validity of environmental impact claims and the resource potential of proposed protected federal lands.

The many successes of the Sub-Activity include informing the Joint Review Panel with new science and expert review on the Mackenzie Gas Pipeline project, informing the establishment of two large national parks (Nahanni and Nááts’ihch’oh), and furthering the development of environmental effects monitoring using geoscience.

The evaluation also identified opportunities for maintaining and improving ESA performance. These are described in detail below.

Recommendation 1: Ensure ESS is Organized to Meet Future EA Review Needs

• ESS should work with partners to assess if it is organized to deliver on future environmental assessment review requirements given the increase in complexity of EA reviews, the availability of expertise in ESS, and the time available to expert reviewers within the recent implementation of CEA Act 2012.

The volume of EA service requests has increased along with the complexity and duration of the reviews compared to the five years before the evaluation. EA reviewers report that the volume of material that they must look at to understand and make determinations about the adequacy of an environmental impact statement can be immense, especially if they are unfamiliar with the project. Early involvement of EA reviewers prior to triggering the EA process appears to be a practice employed successfully in several projects, but there are concerns over sustaining such a practice. While timelines have so far been met, there is a risk that this could result in inadequate reviews.

Provision of a strong review by ESS experts has proven to be an essential part of the EA process. The evaluation found that due diligence reviews of proposed projects has helped to limit or prevent uncontrolled environmental impacts. ESS expert reviewers are experiencing challenges with the time pressures in the current EA review process structure at NRCan. Evidence from the survey of ESS-GSC researchers and interviews indicates that the time allotted for expert review of proponent environmental impact statements (EIS) is short.

The evaluation identified challenges among research scientists in managing priorities of EA review and program commitments. This may also be related to challenges in planning resources available for EA reviews due to the reactive nature of the service. Interviewed expert reviewers, coordinators, regulatory authorities and members of SPI all suggested that there were opportunities to improve efficiency of how expert comments are reviewed and approved within ESS and SPI and provided to CEAA.

While ESS EA office provides direction to reviewers to assist them in their work once engaged in a project, the evaluation found that there are differences of opinion between some expert reviewers and those involved in managing the EA process as to what information is needed at the EA review phase of a project. On both sides of this, however, there was agreement that the reviewers job was to assess the reasonableness of the proponent’s conclusions on environmental effects and mitigation strategies.

The main factors facilitating the achievement of results for the EA and RA services are the expertise depth, high quality ESS-GSC researchers and the availability of those researchers. So far, ESS has been able to respond to requests for CEAA reviews and to provide researchers with expertise in mineral deposits to terrestrial MERA processes. However, some expertise areas are thinly stretched (e.g., hydrogeology and geotechnical engineering for EAs). Challenges in responding to EAs tended to be linked to constraints on expertise availability due to retirement, promotion, and competing priorities. The difficulty in permanently filling the EA Service Manager position (vacant since 2011) illustrates the unique expertise and experience needed for this role.

The amount of experience doing EAs is also a factor in the quality and efficiency of the review. The more experienced reviewers were seen by stakeholders to more easily focus their comments on the big picture with regard to environmental effects of the project, and tend to provide a more solid review. The survey of ESS-GSC researchers indicates that the services are at risk of losing some of their expertise over the next five to ten years. In total, 49% of EA reviewers plan to retire in under 10 years, including 23% within the next 5 years. Key expertise areas will be affected by this.

Recommendation 2: Address Timeliness Challenges in RA Service

• ESS should assess protocols for delivering resource assessments to identify and mitigate factors that have contributed to delays in some RA studies.

The evaluation found ESS contributions to the terrestrial MERA process for northern park establishment to be very well delivered in terms of quality, detail and appropriateness of the information provided. All stakeholders indicated that ESS RA studies are of very high quality and are used by federal authorities in park boundary decisions. While highly valuable, evidence from case studies and stakeholder interviews suggests that the decision-making and consultation processes that RA studies inform were negatively impacted by the lateness of recent ESS Open File Reports. Thaidene Nene was provided three years behind schedule and consultations were impacted, and the Laurentian Channel and Hecate Strait MPA desktop studies were not delivered in time for DFO to establish the MPAs through their process. One of the contributing factors to this appears to be the challenge in balancing researcher commitments in ESS programs and delivery of Open File reports for MERA. Changes in requirements from Parks Canada were also a factor in the delays on Thaidene Nene.

Recommendation 3: Maintain and Strengthen Connections with Policy Groups

• ESS should continue to maintain and strengthen connections to federal and provincial policy and regulatory groups at the EGP program level.

The goal of the EGP is to generate new geoscience that allows federal regulators and industry to make informed decisions about ecosystem risk management in resource development projects. The EGP case studies and interviews suggest that the adoption of new geoscience information and techniques to answer ecosystem effects questions is facilitated best when research has specific connections to policy groups (e.g., new guidelines adopted by CNSC) and to the EA process where decisions about the acceptability of impacts from resource development projects are made (e.g., Alberta Environment or project proponents in Mackenzie Gas Pipeline project).

Recommendations and Management Response and Action Plan Table

Recommendations	Management Responses and Action Plans	Responsible (Target Date)
ESS should work with partners to assess if it is organized to deliver on future environmental assessment review requirements given the increase in complexity of EA reviews, the availability of expertise in ESS, and the time available to expert reviewers within the recent implementation of CEA Act 2012.	ACCEPTED GENERAL COMMENT: It should be noted that since April 2012, ESS has streamlined the management of the Environmental Assessment Service (EAS) and Resource Assessment Service (RAS), incorporating them into a single unit with the objective of increasing efficiency. In April 2014, the management of the unit was assigned to a REM-02. Also, in January 2013, ESS ADM tasked DG CNCB with the development of an EAS and RAS management plan. This work is currently underway and will address recommendations 1 and 2.	ADM Earth Science Sector
	Environmental Assessment Service (EAS) ESS will undertake the following actions to ensure that it is organized to deliver on future EA review requirements.   ESS will work with partners to assess the likely future EA expertise requirements: As part of the EAS and RAS management plan (see general comment above), ESS will review and assess the GSC’s current and near future EA expertise. Discussions will be maintained and enhanced with SPI and developed with other groups (e.g., CEAA, CNSC, NEB, Nunavut Impact Review Board) to examine the potential required expertise for environmental assessment over the next five years. Preliminary analysis indicates a reduction in capacity in some internal areas of expertise due to retirements; this is the same group of key experts that is required in approximately 50% of all environmental assessments. Further gap analyses will be performed to inform the planning and resource allocations for the related ESS program research activities.	(March 31, 2014).
	ESS will ensure that it has the necessary capacity to continue to carry out EAS and RAS. ESS has developed and implemented a long-term Human Resources plan that is aligned to the sectors business lines. The implementation of this HR plan will allow for the allocation of resources to critical program areas including EAS and RAS: This review of potential required expertise will be part of the annual review of the ESS Human Resources Plan. NRCan (GSC) develops an annual HR plan that includes allocation of staff or new staffing for critical positions to ensure the overall success of all programs and services. The GSC Strategic HR plan’s main objective is to maintain sufficient expertise and geoscience capacity to deliver on key commitments and priorities. Linkages with program research activities (e.g., current Environmental Geoscience, Groundwater Geoscience, Public Safety Geoscience and Climate Change Geoscience Programs) ensures the expertise requirements are maintained and developed in support of future EA requirements. ESS is currently undertaking the design and implementation of the next phase (April 1, 2014 to March 31, 2020) of the Environmental Geoscience, Groundwater Geoscience, Public Safety Geoscience and Climate Change Geoscience Programs, which will also consider expertise requirement for the EAs. (September 30, 2014).	(March 31, 2014). (March 31, 2014).
	ESS will review current EA processes and continue to implement project tracking to ensure experts have adequate time to respond by streamlining management approvals and providing advanced notice to expert reviewers of upcoming EA requirements. Since the adoption of the new regulations, including processes and timelines, as spelled out in the Canadian Environmental Assessment Act (2012), in May 2012 EAS developed and implemented a tracking tool to monitor progress of all active Environmental Assessments to address the issue of timeliness of reviews. In addition to the weekly tracking reports that are circulated to the Geological Survey Directors General offices, reports will now be sent on a monthly basis to the ESS ADMO. (January 2014) Working with the Science and Policy Integration Sector, ESS will develop and implement an improved advance notice protocol for the EAS. Review and approval processes will be examined, and best practices from other departments such as Fisheries and Oceans Canada will be incorporated. (December 2014) This improved protocol will:   ensure reviewers have adequate time to complete their reviews of proposed projects currently undergoing environmental assessment reviews; and reduce the time required for management approval.	(January 31, 2014). (December 31, 2014)
ESS should assess protocols for delivering resource assessments to identify and mitigate factors that have contributed to delays in some RA studies.	ACCEPTED In 2012, the RAS developed and implemented a tracking tool to monitor progress of active MERAs to address the issue of timeliness of assessments. A monthly report will continue to be distributed to the GSC DGO. In addition, ESS will review MERA projects to identify factors responsible for encountered delays. The service will develop and implement a process map that will prevent future delays as well as planning and reporting tools. (December 2014).	ADM Earth Science Sector (December 31, 2014).
ESS should continue to maintain and strengthen connections to federal and provincial policy and regulatory groups at the EGP program level.	ACCEPTED In 2013, the Environmental Geoscience Program (EGP) staff were key members of the NRCan Horizontal Task Teams on mine tailings and shale oil and gas as well as the Interdepartmental working group on shale oil and gas. The recommendations from the Interdepartmental working group were addressed by the EGP in 2012-13 through the implementation of a new two-year project on shale gas (action completed in April 2012). In addition, ESS will continue to consult with Environment Canada and other federal and provincial policy groups to foresee science gaps in emerging federal science and policy issues. These will inform the design and implementation of the next phase of its environmental geoscience activities. Consultation of policy groups and design of program activities and outcomes are to be completed by March 31, 2014. Implementation of research projects in the renewed program is to be completed by March 31, 2015.	ADM Earth Science Sector (March 31, 2015).

1.0 Introduction and Objectives

This report presents the findings of the evaluation of the Environmental Studies and Assessments (ESA) (Program Sub-Activity 2.3.3 in Natural Resources Canada’s (NRCan’s) 2012-13 Program Activity Architecture).^{Footnote 6} The ESA is administered by NRCan’s Earth Science Sector (ESS) - Central and Northern Canada Branch within the Geological Survey of Canada (GSC). This first evaluation covers research project activities and services provided with NRCan’s direct program spending of $21.6 million over the period from 2007-08 to 2011-12.

The overall goal of the ESA Sub-Activity is to provide scientific knowledge and expertise that allows federal regulators and stakeholders to make informed decisions about responsible natural resources management (ecosystem risk and protected areas management).

The ESA has three components, described below:

• Environmental Geoscience Research Program (EGP): This research program, identified as the Environment and Health Program prior to 2009, consisted of $17.5 million from 2007-08 to 2011-12, and focuses on characterizing environmental impacts of metals mining, northern energy development, renewable energies, shale gas, northern mineral development, carbon capture and storage and the oil sands. It was designed to generate new geoscience knowledge to influence and support the development of guidelines, and best practices that allow federal regulators and industry to make informed decisions about ecosystem risk management in resource development projects.
• Environmental Assessments service (EA): This service provides ESS-GSC geoscience expertise and information to federal departments for the purposes of reviewing projects undergoing federal environmental assessment reviews as required under the Canadian Environmental Assessment Act [(CEA Act 1992, Section 12 (3)]. It was designed to ensure that adverse environmental impacts of projects subject to environmental assessment reviews (e.g., mines, hydroelectric projects, linear infrastructure (e.g., highways), nuclear facilities, and pipelines) are identified and can be mitigated before projects receive federal approval to proceed.
• Resource Assessments service (RA): Since 1980, this service conducts, upon request, geoscience research to support decision-making of other federal departments by preparing an inventory of non-renewable natural resource potential for proposed protected areas under federal jurisdiction. It was designed to ensure that the withdrawal of land or water for terrestrial national park national marine conservation area purposes is balanced between the ecosystem values of the land and the potential for exploration, development and use of mineral and energy resources. The service has also provided assessments (on a non-mandatory basis) to support the establishment of Marine Protected Areas and Marine Conservation Areas, which occur outside the formal MERA process.

The logic models for the EGP program and the EA/MERA service components of the ESA, found under Appendix A, demonstrate the expected causal relationships between the program’s activities, outputs, and intended immediate, intermediate and ultimate outcomes.

1.1 Background and Context

Exhibit 1 presents the ESA as part of the Responsible Natural Resource Management Program Activity (PA 2.3) that feeds into NRCan’s Strategic Outcome 2: Natural Resource Sectors and Consumers are Environmentally Responsible.

Exhibit 1: Environmental Studies and Assessments Sub-Activity within the context of approved NRCan's 2012-13 Program Activity Architecture (2012-13 PAA)

Text version

Exhibit 1: Environmental Studies and Assessments Sub-Activity within the context of approved NRCan's 2012-13 Program Activity Architecture (2012-13 PAA)

Strategic Outcome 2: Natural Resource Sectors and Consumers are Environmentally Responsible

Program Activity 2.3: Responsible Natural Resource Management

Expected results: Natural resource sectors manage impacts to the environment.

Program Sub-Activity: Environmental Studies and Assessments (PAA 2.3.3)

• Environmental Assessments Service (1992)
• Mineral and Energy Resource Assessments Service (1980) 
• Environmental Geoscience Program (2009)  

Through the ESA, NRCan’s ESS provides geoscience expertise and information required by government departments, regulatory bodies and industry in order to reduce the environmental impacts that may occur in the development of major resource projects.

The EGP program replaced ESS’s Environment and Health program that operated from 2007 to 2008-09. The EGP is different from its predecessor in that it focuses exclusively on natural resources development and on new methodologies, rather than risk to human health and environment. This program was established in 2009 to provide science and develop guidelines and best practices needed to address the environmental impacts of metals mining, northern pipelines and the oil sands. The program was developed in consultation with industry and regulators and involves priority-driven collaboration with partners under the following principles: 1) regulatory decisions must consider cost-effective, performance-based approaches to reduce environmental impact; 2) industry will need to select the best available approaches to limit the impacts of development; and, 3) ESS will generate new science required by regulators and project proponents to make informed decisions on ecosystem risk management.

With regard to the EA Service, NRCan has statutory obligations under the Canadian Environmental Assessment Act [(CEA Act 1992, Section 12 (3)] to provide information and expertise, when directed to do so, as well as obligations under provincial and territorial EA regimes (e.g. Part 5 under the Nunavut Land Claims Agreement and the Mackenzie Valley Resource Management Act) with federal input. Section 12 (3) of the CEA Act [1992] states that “Every federal authority that is in possession of specialist or expert information or knowledge with respect to a project shall, on request, make available that information or knowledge to the responsible authority or to a mediator or a review panel.^{Footnote 7}” Under CEA Act 1992, which was in effect during the evaluation period until it was replaced in 2012; there were three levels of federal Environmental Assessment (EA) reviews, each with a more detailed review requirement:

• Screenings: a systematic approach to documenting the environmental effects of a proposed project and determining the need to eliminate or minimize (mitigate) the adverse environmental effects, to modify the project plan or to recommend further assessment through mediation or an assessment by a review panel.
• Comprehensive Studies: typically used for large-scale or complex projects likely to have significant adverse environmental effects. These include large-scale natural gas developments, mining projects, nuclear power developments, hydroelectric-generation projects, linear-infrastructure (e.g., highways) projects and certain projects in national parks.
• Review Panels: a group of independent experts selected on the basis of their knowledge and expertise and appointed by the Minister of the Environment to review and assess, in an impartial and objective manner, a project with likely adverse environmental effects and significant public concern. A review panel submits its recommendations to the Minister of the Environment and to the responsible authority.

A Mineral and Energy Resource Assessment (MERA) process must be completed for proposed protected areas in the Northwest Territories and Nunavut (1980 to present), and in the Yukon (1980 to devolution in 2002). The MERA process is outlined in the 1980 MERA Terms of Reference and establishes a formal, mandatory framework for Aboriginal Affairs and Northern Development Canada, Parks Canada, NRCan and the territorial governments to cooperate on development of boundary recommendations for proposed northern protected areas. Within this process, NRCan’s GSC prepares an inventory of the non-renewable natural resource potential in the Area of Interest for a proposed protected area, and supports Parks Canada in the public consultation process for boundary options. Also within this service, but on a non-mandatory basis, NRCan may support other federal departments with the preparation of an inventory of non-renewable natural resource potential for protected areas in federal jurisdiction (e.g., Marine Protected Areas administered by DFO).

1.2 ESA Component Activities

The ESA is delivered through three program/service components, each with their own set of activities. The table below provides a summary of the activities undertaken by the ESA components.

EGP :

This component carries out research on environmental impacts of metals mining, northern energy development, renewable energies, shale gas, northern mineral development, carbon capture and storage and the oil sands. Program activities are divided into five areas:

• Program management and transition Activities: Conduct research to support appropriate remediation of gold and uranium mines, and conduct research to distinguish local and development driven mercury effects in northern populations. This includes all activities related to the former Environment and Health program and activities that arise from transitioning to the current Environmental Geoscience Program, and participation in science and policy integration forums.
• Tools for Metal Mining Environmental Impact Assessment: Conduct research to support development of new resources and to maintain expertise for environmental impact assessment reviews in mineral development. In particular, activities include the development of models to predict source, transport and fate of contaminants, and development of protocols to assess background conditions in regions subject to mineral development.
• Coal and Oil Resources Environmental Sustainability (CORES): Activities use geoscience to fill knowledge gaps needed to improve the efficiency of remediation efforts in the oil sands and develop approaches to quantify and limit contamination from the use of bitumen and coal. Two activities within this project are financially supported through the Clean Energy Fund (CEF).
• Renewable Energies: These activities are developing the geoscience knowledge required for sustainable development of tidal, wave and geothermal energies. Three activities within this project are financially supported through the CEF.
• Northern Development: Research is conducted related to the northern environment in order to facilitate the environmental impact assessment for energy and mineral development in the North. One activity in this project is financially supported through the Program of Energy Research and Development (PERD). The project develops methodologies for assessing the environmental impacts of pipelines (geohazards and terrain conditions) and mining development in the North (naturally occurring contamination and permafrost conditions).

EA Service:

This service provides geoscience expertise in reviews of resource development projects undergoing federal environmental assessment (EA), including assessments under provincial and territorial jurisdictions. Participating ESS experts review technical and regulatory documentation, review research and analyses, disseminate information, contribute to technical working groups, participate in technical workshops, and testify in formal EA hearings/venues.

The service also provides expertise in public information sessions, liaises and meets with internal and external clients/proponents, and manages and maintains the ESS EA records and documents. Activities under the EA service are designed to provide expert geoscience advice on physical environmental baseline conditions and environmental impact assessments. ESS responds to requests on more than 50 resource development project activities each year, and provides approximately 70% of NRCan’s environmental assessment technical expertise. Each environmental assessment activity generally involves one or more request for input, one or more expert, and may extend over several months to years throughout the many phases of the federal EA review process.

Resource Assessment Service:

In response to proposals for establishing new national parks, NRCan’s GSC conducts research, including field research (when necessary), to develop an inventory of the non-renewable natural resource potential of areas in the Northwest Territories and Nunavut prior to their formal establishment as national parks. NRCan may also support other federal departments with the preparation of an inventory of non-renewable natural resource potential for protected areas in federal jurisdiction (e.g., Marine Protected Areas administered by DFO). In these cases, NRCan’s role could range from preparation to the non-renewable resource inventory, to advising on work plans and review of inventories prepared by third parties.

This service develops terms of reference and workplans with clients and partners, for mineral and energy resource assessments, delivers the resource assessments (geological field surveys and laboratory studies, geological data collection, compilation, integration and analyses, and application of mineral and energy potential models for resource evaluation), conducts community and client consultations/outreach, co-chairs the intergovernmental and interdepartmental MERA Working Group, supports the ADM level Senior MERA Committee, and manages and maintains the MERA records.

Since 1980, GSC has conducted and published formal resource assessment reports for 12 proposed northern park establishments or expansions. RAs under the MERA process for northern terrestrial parks have generally included large field components (taking up to 4 years to complete and publish), and in the last decade received substantial Parks Canada funding for field work.

1.3 Resources

The ESA primarily consists of A-base funding for salaries and operating and maintenance (O&M) expenditures from NRCan and other federal government departments (i.e., EC, HC, and AANDC). Over the period from 2007-08 to 2011-12, NRCan’s ESA expenditures totaled approximately $21.6 million including Program Energy Research and Development (PERD) and Clean Energy Fund (CEF) associated with the EGP program.

Exhibit 2 shows the financial resource profile for the EGP. The EGP replaces the Environment and Health Program (EHP), which was operating in the first two years of the evaluation coverage period (i.e., in 2007-08 and 2008-09). The EGP expenditures totaled approximately of $17.5 million over the period from 2007-08 to 2011-12.

Exhibit 2: EGP Program Resource Profile, 2007-08 to 2011-12 ($000)

Funding Source	2007-08	2008-09	2009-10	2010-11	2011-12	Total
NRCan ESS A-base $	$2,744	$2,539	$3,070	$2,830	$3,351	$14,534
NRCan PERD	$0	$0	$782	$516	$190	$1,488
NRCan CEF	$0	$0	$0	$826	$629	$1,455
Total NRCan $	$2,744	$2,539	$3,852	$4,172	$4,170	$17,477
SPA	$48	$89	$19	$0	$0	$156
OGD	$980	$1,373	$1,645	$1,255	$50	$5,303
ESS Leveraged $ (PERD, CEF, SPA, and OGD)	$1,028	$1,462	$2,446	$2,597	$869	$8,402
Total	$3,772	$4,001	$5,516	$5,427	$4,220	$22,937
Leverage Ratio*	2.7:1	1.7:1	1.3:1	1.1:1	3.9.1	1.7:1

*Leverage calculated based on ESS A-base leveraging of all other funds (i.e., PERD, CEF, SPA, and OGD)

The Program also receives funding from a Specified Purpose Account (SPA) from Imperial Oil Resources Ventures Limited for collaboration in borehole thermal instrumentation and data compilation and analysis for description of soil thermal conditions along the Mackenzie pipeline corridor. In addition, other government departments sometimes provide funds to EGP. Between 2007-008 and 2009-10, this included funds from Health Canada to complete the transition activities of the EHP and from EC for activities related to the oil sands.

Exhibit 3 details the financial resource profile for the EA and RA Services. These two components are legislated services to Government. Until the fiscal year 2010-11, these two services were managed together and therefore financially tracked as one unit namely the Legislated Environmental and Resource Assessment (LERA). As of 2012-13, they are once again managed as a single unit.

Exhibit 3: EA and RA Services Resource Profile, 2007-08 to 2011-12 ($000)

Funding Source	2007-08	2008-09	2009-10	2010-11	2011-12	Total
NRCan ESS A-base $	$731	$732	$920	$900	$855	$4,138
C-base (MGP review and CEAA)	$222	$121	$208	$100	$80	$731
Parks Canada funds for MERA	$80	$1,048	$278	$0	$0	$1,406
ESS Leveraged $ (C-base +Parks Canada)	$302	$1,169	$486	$100	$80	$2,137
Total	$1,033	$1,901	$1,406	$1,000	$935	$6,275
Leverage Ratio*	2.4:1	1:1.6	1.9:1	9:1	10.7:1	1.9:1

*Leverage calculated based on ESS A-base leveraging of all other funds (i.e., C-base and Parks Canada)

For EA service, even though other government departments do not generally contribute funds to EAs when service is requested, it does happen on rare occasions. The EA service received C-base funding from AANDC for the Mackenzie Gas Pipeline (MGP) review process which ended in 2010-11, and the service also receives funding from the Canadian Environmental Assessment Agency to support operating expenses associated with expert appearances at Panel reviews.^{Footnote 8} For MERA projects, Parks Canada provides funding to ESS when they request a resource assessment.

2.0 Evaluation Scope and Methodology

2.1 Evaluation Scope

The evaluation covered activities of the ESA Sub-activity over the period from 2007-08 to 2011-12, including $21.6 million in NRCan expenditures.
The evaluation was designed to assess the issues of relevance and performance of the ESA in accordance with the requirements of the 2009 Treasury Board Policy on Evaluation,^{Footnote 9} and was part of NRCan’s approved Departmental Evaluation Plan 2012-13 to 2016-17.

The Strategic Evaluation Division conducted the evaluation fieldwork from April 1, 2012 to February 2013 using in-house resources.

This evaluation study was designed using a calibrated risk-based approach as per Treasury Board Guidelines on identifying and understanding risk when planning evaluations so as “to prioritize units of evaluation based on risk”.^{Footnote 10}

The risk assessment exercise identified the following factors that contributed to an overall medium risk rating for the Environmental Studies and Assessments Sub-Activity (ESA) and an evaluation design that took into account the areas of highest risk. Overall, the EA service was deemed the highest risk component, followed by RA and EGP. The following were key risk factors considered in the evaluation design:

• The ESA Sub-Activity is low materiality, with an average annual expenditures of $4.3 million in ESS A-base funds
• The ESA Sub-Activity has not been evaluated or audited previously;
• Changes to program context for Environmental Assessments triggered under the CEA Act and Resources Assessment components suggest higher evaluation risk levels given their markedly increased workloads in recent years, and the introduction of Marine Protected Areas on a non-mandatory basis for the RA service; and
• The EA component is also higher risk due to the visibility of Environmental Assessments in terms of recent amendments to the CEA Act and the high profile of resource development projects in the media.

This report is organized at the Sub-Activity level, recognizing that there are three distinct but related components. When developing the research instruments, the evaluators worked with the program to operationalize the outcomes identified in the separate EGP and EA/RA logic models. These were combined in a single logic model for reporting on the Sub-Activity.

2.2 Evaluation Methodology

The ESA evaluation employed multiple lines of evidence. This included a document and literature review, 32 interviews, and 13 case studies covering all ESA components. In addition, a sample file review of EA files (20%) was conducted, and an online survey with ESS geoscientists involved in EA and MERA services. The evaluation team developed the methodology and the data collection instruments/tools in consultation with ESA program managers.

Document and Literature Review

The document and literature review examined approximately 30 documents provided by ESA management and others identified during the evaluation planning stage and through interviews. Examples of documents reviewed include NRCan’s Departmental Performance Reports, NRCan’s Reports on Plans and Priorities, NRCan’s Integrated Business Plans, Federal Budgets, Speeches From the Throne, Program Plans, NRCan-ESS document on Implementing Federal Marine Protected Areas, NRCan’s ESS Year-End and Mid-Year reports, applicable federal legislation ( e.g., The CEA Act, Oceans Act),MERA Terms of Reference, and other internal NRCan Review documents.

File Review

The file review included a detailed look at a sample of 20% of approximately 100 EA project files under the federal EA review process and 12 EA projects under the provincial/territories regimes (e.g., Mackenzie Valley and Nunavut EA regimes) in which ESS has provided geoscience expertise and advice over the evaluation period. The file sample was randomly selected among the projects with the highest level of involvement of NRCan’s geoscientists. The sample was selected from among the project files to reflect the distribution of the levels of federal EA review process and provincial/territories EA regimes, and the frequency of ESS involvement.

Exhibit 4 shows the sample of 20 EA project files (18 under the federal EA and 2 under the provincial/territories EA regimes with federal input) as compared to the population of project files.

Exhibit 4: Distribution of EA project files reviewed by type of EA review and Regime

Regime	Type of EA	Population	Sample
Federal EA	Comprehensive Study	49	10
	Review Panel	12	2
	Screening	32	6
Sub-total		93	18
Territorial EA	Mackenzie Valley and Nunavut	12	2
Total		105	20

Documents used in the file review were found using the online registry found on the Canadian Environmental Assessment Agency website and the progress reports (i.e., NRCan’s ESS Mid-Year and Year-End review reports) provided by the Program. A database was developed and populated by compiling information on the type of expertise provided, whether the geoscience has influence, if NRCan’s ESS committed to a follow-up review, and whether timelines were met.

Key Stakeholder Interviews

A total of 32 individuals (16 internal to ESS and 16 external) were interviewed across the components of ESA from June 2012 to February 2013. By component, there were 13 interviews conducted for EGP, 10 for RA, and 18 for EA. Interviewees were identified and selected by the evaluation team in consultation with ESS to ensure knowledgeable informants in each of the stakeholder groups. Due to the interrelated nature of these program/service components several interviews covered more than one program, which means that the total number of interviews is therefore not the sum of the individuals interviewed across components. It is important to note that most of the interviewees are federal government employees, which is appropriate given the nature of these components (i.e., informing federal decision-makers).

Exhibit 5 provides a breakdown of interviews by component and stakeholder groups across ESA.

Exhibit 5: Distribution of ESA Interviews by Type and Program/Service

Type	EGP	RA	EA	Total # of Interviews
Internal: NRCan-ESS (researchers and managers)	8	2	10	16
NRCan SPI and MMS	1	2	4	4
OGDs (Parks Canada, AANDC, DFO, EC)	2	5	4	9
Provinces/territories	1	1		2
Private sector	1			1
Sub-total External (non-ESS)	5	8	8	16
Total	13	10	18	32

Case Studies

Thirteen case studies were completed for this evaluation; they were selected in consultation with ESA program and service managers. Twelve of the case studies were of individual projects within each of the three components and were selected to cover the range of project types involved. The four EGP case studies were selected to cover one project area each. The four RA cases were selected to cover terrestrial MERA studies and marine protected area studies. The five EA case studies included two Panel level reviews, one comprehensive study review, and a screening project, as well as one comparative case study of ESS’s EA service model to that of other federal departments with similar obligations under CEA Act [1992, now 2012] (i.e., DFO, EC).

Each case study consisted of a review of background documents, and two to five interviews with project leaders, researchers, partners and other stakeholders involved (some of these overlapped with the key informant interviews). Exhibit 6 shows the distribution of case studies.

Exhibit 6: ESA Evaluation Case Studies by Component

Component	Case Studies				Total
EGP Activity type:	Indicators of Ecological Integrity for Canada’s Parks Environment and Health Project	Permafrost and terrain information for design and environmental management of northern onshore pipelines Northern Development project	Natural and anthropogenic perturbations of the nitrogen & sulphur cycles Coal and Oil Resources Environmental Sustainability project	The Lac Dasserat study Tools for Metal Mining Environmental Impact Assessment project	4
RA Activity type:	Nahanni National Park Terrestrial National Park; Parks Canada/AANDC	Thaidene Nene (East Arm/Great Slave Lake) Terrestrial National Park; Parks Canada/AANDC	Laurentian Channel Marine Protected Area Special request from DFO	Hecate Straight Marine Protected Area Special request from DFO)	4
EA Activity type:	Mackenzie Gas Project Panel Review	Deep Geological Repository project Panel Review	Victor Diamond Mine project Comprehensive Study	Port Hope project Screening	4
EA	Service comparison to other departments with similar obligations ((i.e., DFO and EC)				1
Total					13

On-line Survey

An on-line survey was conducted with ESS-GSC researchers (i.e., geoscientists) that have contributed expertise to EA and RA projects from 2007-08 to 2011-12. The survey was available for approximately five weeks in October and early November 2012, and was programmed using NRCan’s Departmental license from Fluidsurveys.com (server is hosted in Canada and conforms to Canadian privacy laws). In advance of the survey, an email letter from the Director General of Central and Northern Canada Branch GSC was sent to all researchers to introduce the evaluation and encourage participation. Respondents were then invited to the survey via an email containing a link and a unique password, and were sent weekly reminders to encourage response. The survey was available in both English and French, and the survey content was developed in consultation with ESS.

Exhibit 7 describes the response rate to the survey:

Exhibit 7: EA-RA Geoscientists Survey Completion Rate

Total population	69
Total Responses	46
Valid responses (at least ¾ complete)	42
Completion rate	61%
Average time taken	13.5 minutes

Survey response data was analyzed separately for EA (n = 35) and RA (n = 13), including 6 respondents who had contributed to both EA and RA services. Due to the small sample size of RA researchers (n = 13), percentages were not reported. Instead, the actual number of respondents for each response is presented.

2.3 Limitations and Mitigation Strategies

The overall evaluation methodology was calibrated based on the evaluation risk assessment described in section 2.1 of this report. It provides the basis for addressing all evaluation issues through multiple lines of evidence. However, there are some limitations that should be considered when interpreting the findings. These are outlined below:

• The evaluation unit was the ESA Sub-Activity, which includes EA and RA services provided by ESS geoscience experts, however this does not include related services provided by NRCan experts in other sectors: Since the EA and RA services were part of the ESA Sub-Activity with EGP, the evaluation assessed them as services provided by ESS to OGD clients when in fact they are one component of a whole of NRCan service provided to other federal departments and agencies. For example, the EA service is led by NRCan SPI EA Branch and coordinates the provision of expertise under CEAA and northern review regimes from all NRCan sectors (though ESS is the largest provider of this expertise). Similarly, NRCan’s contributions to the federal MERA process include the geoscience from ESS as well as economic overlay of mineral potential from NRCan MMS and energy potential from NRCan ES. As a result of this scoping, the evaluation did not look closely at SPI, MMS, or ES in providing these services.
• Small sample size of EA and RA survey: While the sample size (n = 42) for the survey was small, this simply reflects the reality of a small population to begin with (n = 69). This was a census survey (i.e., no sample frame, so all population members were invited) of a small and targeted known population, and the response rate was high (61%). The results can therefore be considered indicative of the views of the population.
• Interview sample was identified by SED in consultation with NRCan-GSC: Contact information for potential interview respondents was compiled by SED through documents provided by ESA management and staff and in consultation with NRCan-GSC. While interviews were ultimately the decision of the evaluation team (subject to willingness and availability of interviewees), this may have resulted in a selection bias.
• Small number of interviews conducted within each ESA sub-component: For this evaluation a total of 32 interviews were conducted with stakeholders. Broken down across the three Sub-Activity components, this is a small sample for each component. However, this is mitigated by the careful targeting of interviewees in consultation with ESS-GSC, the online survey of researchers, and by the selection of case studies (which include separate interviews) that covered significant portions of work. For example, the four RA case studies included documents and interviews covering nearly all component activities for the evaluation period.

The limitations identified above with respect to individual lines of evidence are mitigated through the use of a multiple lines of evidence approach that allows the evaluation team to verify the strength of findings across lines of evidence.

3.0 Evaluation Findings

3.1 Relevance

3.1.1 Continued Need for the Program

Evaluation Question	Lines of evidence	Assessment
1: Is there an ongoing need for the program/services and activities? How are target groups served by the program/services?	Document/literature review Interviews Cases	Clear ongoing need for the Sub-Activity. EA requests steadily increasing, MERA evolving towards marine studies and EGP continues to fill information gaps for regulators.

Both the RA Service and the EA Service respond to federal legislated and policy requirements

For the EA Service, NRCan has a legal requirement under CEA Act and other federal regimes to provide its unique geoscience expertise when requested by federal decision makers: The requirement for the ESS EA Serviceto provide expertise is well supported by interviews and documents. Internal and external interviewees all recognized that the ESS EA service was needed in order for NRCan to fulfill its legal obligations. The service addresses the need for geoscience expertise and advice on natural resource development projects under the federal environmental assessment review process. This was outlined in the CEA Act 1992, Section 12(3) which was in effect during the evaluation period, and in the new CEA Act 2012, Section 20. The 2012 Act states that “Every federal authority that is in possession of specialist or expert information or knowledge with respect to a designated project that is subject to an environmental assessment shall, on request, make that information or knowledge available, within the specified period.”^{Footnote 11} Also, the EA Service is needed under the legislated provincial and territory EA regimes that require federal input (e.g., the Inuvialuit and Nunavut Land Claims Agreement Acts andthe Mackenzie Valley Resource Management Act).^{Footnote 12}

The RA Service responds to the legislated and policy needs for geoscience expertise and knowledge on the establishment of national parks and other protected areas: Interviewees noted that the RA Service is necessary in order to respond to the requirements for mineral and energy resource assessments from ESS-GSC to inform the MERA process articulated in the 1980 TOR and various legislated needs in the federal government. Documents show that the RA Service responds to legislated and policy obligations for mineral and energy resource assessments of geographic areas proposed for federal protected status. It is driven by legislation and policies, which include the 1980 MERA Policy/Process articulated in the MERA Terms of Reference, the 1980 National Policy on Creation of Parklands, the 1994 Canada Wildlife Act, the 1996 Minerals and Metals Policy, the 2001 Canada National Parks Act, the 2002 National Marine Conservation Areas Act, the 2003 Parks Action Plan, and the 1996 Oceans Act.^{Footnote 13}

EA, RA Services and EGP provide the geoscience expertise and knowledge needed to inform federal decisions and industry

Mineral and energy resource assessments are necessary for federal authorities to make informed decisions about the removal of land from development: According to ESS internal documents, the RA Service allows regulatory authorities (i.e., AANDC, EC, and DFO) to determine the economic and strategic significance of mineral and energy resource potential in proposed protected areas as part of the feasibility assessment process and public consultation process on a wide range of issues related to park or protected area establishment (e.g., boundaries, restrictions on activities, regulations).^{Footnote 14}

All interviewees agreed that ESS-GSC’s RA Service provides essential information to decision-makers regarding the mineral and hydrocarbon potential of land being considered for conservation. Several interviewees explained that the decision to remove land from development permanently (in the case of national parks) is an important decision that must be based on solid understanding of its potential given the economic implications for Canada (i.e., foregone economic gains). Particularly with regard to the North, there is normally limited information about the sub-surface when an area is selected for a national park, so resource assessment studies are needed to fill that information gap.

ESS EA Service provides federal and provincial responsible authorities with geoscience information and advice to assess the validity and appropriateness of proponent environmental impact claims and proposed mitigation strategies needed to make sound regulatory decisions: Internal and external interviewees indicated that there was a need for ESS-GSC experts to vet the environmental impact claims made by project proponents. Interviewed RAs reported that NRCan’s experts were responsive to their needs and helped considerably to inform their decisions. GSC staff involved in EA reviews tended to see this as one of the most important roles they can play as a public sector research scientist. Their unique expertise allows them to see impacts from projects that the proponent and their consultants may not have considered, assess the validity of the assumptions in their models, and make judgments about the likely effectiveness of proposed mitigation strategies for known environmental effects. Nearly all interviewees noted that the risks to the environment and to human health that can be created by a development project can be high if mistakes are made.

EGP responds to the need to fill scientific information gaps for regulators and industry (i.e., project proponents) when making decisions on development of natural resources: According to documents, interviews, and case studies, EGP addresses gaps in knowledge, tools, and expertise for assessing impacts of projects proposed in the EA process and the effectiveness of proposed mitigation measures. Documents show that consultations with industry (i.e., project proponents) and regulators revealed a need for greater scientific certainty on the impacts of resource development to improve Canada’s environmental regulatory quality.^{Footnote 15} Interviewees supported this assessment, noting that most ESS-GSC programming focuses on finding and characterizing natural resources, while EGP looks at the environmental impact. They noted that these figure most prominently in EA and other regulatory reviews that are the principle means for federal decision-making on resource development. Case studies show that projects were developed based on a need to distinguish between natural and anthropogenic effects from development to help regulators and proponents accurately distinguish between manmade and naturally occurring contaminants in the environment.

EGP projects respond to the need to have baseline knowledge of the impacts of development and changes in the environment. Priorities are generally appropriate, covering areas where gaps exist to make informed decisions about resource development, although additional areas for the program to focus on were identified.

Current priorities are seen as generally appropriate and necessary given the types of projects that enter the federal regulatory process. Many interviewees identified a need to expand EGP focus to accommodate EA needs in shale gas that will arise in the future: Most interviewees see the current priorities of EGP to be the right ones. ESS-GSC researchers and managers note that the program focuses on what proponents and regulators know the least about that enters the EA review process and requires decisions from federal regulators. The most commonly cited examples were projects that related to Northern development and those related to the oil sands. In fact, one oil sands research partner noted that the environmental effects work being done is vitally important to regulators and would not have been possible without ESS-GSC.

Most interviewed researchers noted that the work they are doing is on developing new methods that will take a number of years to refine before they are likely to be used more widely. With this in mind, they noted that ESS-GSC should consider future EGP research needs that are likely to arise. Most indicated that research needed to understand shale gas projects should be added as a priority in the next round of project planning. These interviewees noted that these projects are less well understood, and that interest will grow in the future.

Baseline information is needed to determine impacts and facilitate better engineering. Often, the tools to do this do not exist: Documents show that the EGP responds to the scientific needs of the industry and regulators for targeted research by providing science, developing guidelines and identifying best practices needed to address the environmental impacts of metals mining, northern pipelines and the oils sands.^{Footnote 16} Projects featured in the evaluation case studies all responded to a need to have baseline knowledge of some sort on the impacts of development and changes in the environment. In each of these cases, projects were designed to be exploratory (except for permafrost) to develop new tools and indicators at some point in the future that would help target research to track these changes. These projects all responded to identified gaps in science that are crucial to making informed decisions about the use of land for development.

The demand for EA services has increased in terms of volume of requests, while the character of requests for RA studies appears to be evolving towards more studies of marine areas rather than terrestrial parks.

The demand for EA reviews, the complexity, and duration of those reviews has steadily increased: Interviewees involved in coordinating ESS-GSC reviewers as well as the reviewers themselves generally noted a significant increase in requests to provide expertise on projects undergoing EA review in the last five years. Most pointed to increased economic development in natural resources sectors as the explanation, noting increases in proposed mining projects, which normally require the more in-depth comprehensive or panel level reviews (compared to the less in-depth screening level reviews). This is supported by the survey in which the majority (51%) of surveyed ESS expert reviewers reported an increase in requests for EA review since 2007-08.

Over the period from 2007-08 to 2011-12, data provided by ESS indicated that NRCan-ESS has responded to 314 requests for geoscience expertise to EA review process, an increase of 58% over the 198 requests from the five year period (2002-03 to 2006-07) preceding the evaluation (note that individual projects have multiple requests for expertise). Approximately 40% of these requests relate to projects undergoing comprehensive studies and 20% to review panels.

Exhibit 8 shows the number of requests for ESS EA reviewer expertise over the evaluation period and the five years prior to the evaluation.

Exhibit 8: Requests for ESS-GSC Expertise on EA Reviews from 2002-03 to 2011-12

Text version

Exhibit 8: Requests for ESS-GSC Expertise on EA Reviews from 2002-03 to 2011-12

Exhibit 8 shows a line graph depicting the trend in the number of requests for ESS EA reviewer expertise from 2002-03 to 2011-12.

In 2002-03, there were 40 requests.

In 2003-04, there were 54 requests.

In 2004-05, there were 58 requests.

In 2005-06, there were 50 requests.

In 2006-07, there were 59 requests.

In 2007-08, there were 56 requests.

In 2008-09, there were 56 requests.

In 2009-10, there were 54 requests.

In 2010-11, there were 77 requests.

In 2011-12, there were 71 requests.

Source: ESS-GSC expertise requests (compiled by service management)

Federal need for terrestrial MERA’s appears to be diminishing, while future federal needs will focus on marine protected areas: Most interviewees noted that Parks Canada’s terrestrial national park system is nearing completion and that there are no additional parks being considered at this time (except for current active MERAs). With devolution of land and resource management responsibilities to the Government of Yukon (2003), and upcoming devolution of responsibilities to the Government of Northwest Territories in April 2014, and to Nunavut (to be determined), there are unlikely to be many more terrestrial MERAs.^{Footnote 17}

By contrast, stakeholders pointed to a rise in federal requirements and requests for resource assessments to inform offshore conservation areas that are not included under the 1980 MERA TOR such as marine protected areas administered by DFO. Some noted that the skill set that GSC has traditionally brought to terrestrial resource assessments is applicable to marine areas as well. Most interviewees agreed that the decision-maker needs for information on resource potential are the same in both land and marine cases. That is, these decisions should be informed by geoscience data. Evidence from the Hecate Strait and Laurentian Channel MPA case studies supports this position given that GSC researchers were able to provide assessments of the mineral and hydrocarbon potential of both regions based on analysis of existing geological and geophysical GSC data. So far, GSC has provided, on a non-mandatory basis, varying degrees of assessment to MPAs ranging from review and analysis of existing data (e.g., in the Hecate Strait and Laurentian Channel MPAs) and support for DFO in reviewing work of third party contractors (e.g., St. Ann’s Bank, Shediac Valley and American Bank MPAs).

3.1.2 Alignment with Government Priorities

Evaluation Issues	Lines of evidence	Assessment
2: Are programs and activities consistent with government priorities and NRCan strategic objectives?	Documents Interviews Case studies	Yes. Consistent with NRCan objectives and federal priorities. ESA supports decision-making by other federal departments.

Evidence:

ESA Sub-Activity is consistent with NRCan’s Strategic Objective # 2: Natural Resource Sectors and Consumers are Environmentally Responsible.

The mandate and objectives of all three Sub-Activity components are consistent with NRCan’s Strategic Objective 2: Natural Resource Sectors and Consumers are Environmentally Responsible. According to NRCan’s 2011-12 Report on Plan and Priorities, NRCan provides knowledge and expertise on the impacts of resource development on the environment and ecosystems in order to develop projects in a sustainable and environmentally responsible manner and create federally protected areas. Through RA the department informs decision-making on the establishment of national parks and other protected areas, and through the EA service contributes to environmental assessments for natural resources projects as required under the Canadian Environmental Assessment Act [(1992, Section 12(3)] and for all federally triggered or regulated projects/reviews.^{Footnote 18} NRCan’s 2011-14 Integrated Business Plan indicated that through EGP the department also provides science, develops guidelines and identifies best practices needed to address the environmental impacts of metals mining, northern pipelines and the oil sands.^{Footnote 19} All interviewees indicated that the three components of ESA are well aligned with government and NRCan priorities. They mainly noted alignment with economic development and responsible/sustainable development priorities of the department.

EA/RA are consistent with federal priorities as described in Budgets and Speeches from the Throne during the evaluation period

ESA EA and RA activities are identified as federal government priorities in recent speeches from the Throne and Budgets. For example, in the 2011 Speech from the Throne the Government of Canada committed to expanding protected lands and marine protected areas, so that current and future generations can continue to enjoy them.^{Footnote 20} As well, the federal government has made references to the EA review process in Federal Budgets 2009, 2010, 2011, and most substantively in 2012. In Federal Budget 2012, the Government committed to improving the review process for major economic projects to accelerate investment and job creation.^{Footnote 21} This budget lead to changes to the 1992 CEA Act, resulting in an updated Act – CEA Act 2012.

ESA activities support other federal departments in fulfilling their mandates.

Interviews and documents also indicate that ESA supports the mandates of other federal departments and agencies: The EA service has provided expertise to 18 federal departments and agencies to fulfill their regulatory roles (e.g., CEAA, CNSC and DFO). RA services have been requested by Parks Canada, AANDC, EC, and DFO to make boundary decisions on protected areas. EGP research has been used by other departments in environmental effects monitoring (e.g., EC).

EGP case studies show that the Program changed in 2009-10 to focus on resource development has improved alignment with NRCan strategic outcomes: The three cases from the current EGP are all aligned with the sustainable development priorities of NRCan. The Eco Indicators case from the Environment and Health era, while providing valuable work, is less well aligned with NRCan priorities and fits better with Parks Canada mandate (the project authority). This refocus of the program to look at regulatory improvement appears well supported by the three current EGP case studies that focus on developing indicators for assessing cumulative effects form development projects and understanding engineering needs in the north.

3.1.3 Alignment with Federal Roles and Responsibilities

Evaluation Issues	Lines of evidence	Assessment
3: Is there a legitimate, appropriate and necessary role for the federal government in these programs and activities? Is NRCan’s role appropriate in the context of the role of others?	Interviews Cases Documents	Yes. Legitimate and necessary role for NRCan and federal government. NRCan’s role generally appropriate. Concerns raised over connecting EGP to policy groups and EA service, and on role of NRCan in marine resource assessments.

ESA Sub-Activity appropriately provides the unique and neutral geoscience expertise of ESS-GSC researchers to inform federal decision-makers and regulators

ESS’ role is appropriate because of its unique geoscience expertise to do this research, and its neutral role in the federal regulatory environment: NRCan-ESS is the only organization within the federal government with the geoscience expertise to carry out the environmental geoscience research projects, RA studies, and geoscience-related EA reviews. Nearly all internal and external interviewees explained that ESS-GSC provides neutral science and advice to regulators and acts as a scientific honest broker among federal departments, provinces, territories and industry when developing methods for answering environmental effects questions, informing boundary discussions for protected areas, and assessing the environmental impacts of proposed development projects.

NRCan GSC appropriately provides neutral, credible EA reviews and information for park establishment processes: ESS-GSC’s advice on EAs and provision of resource assessments are seen by external stakeholders as integral to making informed decisions, and to ensuring the legitimacy of the decision-making process. ESS-GSC’s researchers are highly respected among stakeholder communities and federal decision-makers for their high quality work, depth of expertise, as well as their neutral provision of geoscience information. Most interviewees noted that the public interest was served best by the EA service because it is advice to government from within government and does not come with the perception of possible conflict of interest. NRCan is usually not a Regulatory Authority (RA), and is therefore strictly providing expert advice to RAs rather than making decisions about the proposed projects. Case studies also showed that where an NRCan sector was an RA (e.g., MMS in Victor Diamond Mine case study), ESS continued to provide credible advice into the follow-up and monitoring of the project. Several MERA working group interviewees noted that ESS-GSC’s resource assessments are the cornerstone and the strongest part of the terrestrial MERA process. Evidence from the four MERA case studies, interviews and documents demonstrates that ESS-GSC is seen as a key provider of expertise for supporting conservation decisions because of its qualifications and neutral position in the federal system.

Unique expertise of ESS-GSC researchers and their positions as public sector researchers is used to fill important knowledge gaps that would otherwise not be studied by industry: Interviews with ESS-GSC researchers and partners as well as all four EGP case studies demonstrated that ESS-GSC researchers’ were appropriately engaged in these studies. For example, the Nitrogen and Sulphur Cycles and Lac Dasserat cases each demonstrated that the research undertaken to isolate the differences between natural and anthropogenic contaminants was uniquely something that government researchers would explore and not something that industry would spend resources developing. Interviewees explained that if the science does not exist to easily isolate the differences, industry will not try to invent it. However, EGP projects are a public good in that knowing the answers to these questions will make for more informed regulatory decisions, and potentially better methods for isolating and predicting effects.

EGP program appropriately engages users in developing projects; need to sustain coordination with policy groups to proactively address emerging needs.

ESS has engaged potential users as partners in planning individual projects but there is potential for better coordination with provinces and federal policy groups: According to interviewees, EGP projects are designed through engaging with potential users of the research in industry, other federal departments (as well as NRCan Energy and MMS sectors), and the provinces. Interviewed provincial and federal partners noted that ESS-GSC has been a valuable partner and that collaborations have yielded valuable information for provincial and federal monitoring needs in some cases (e.g., oil sands, Mackenzie Delta).

Documents, interviews and findings from EGP case studies also suggested that coordination with the provinces and developing closer relationships with federal policy groups could be improved. ESS documents indicated that some potential federal environmental assessments or regulatory decisions in rapidly developing areas of in-situ oil sands and shale gas may be undertaken before ESS could develop the capacity to address the related environmental geoscience knowledge gaps. Although ESS has added oil-sands and shale gas research to the Program, several ESS-GSC interviewees raised concerns that policy groups can become disconnected from emerging science and EGP projects should try to strengthen those connections as much as possible. Where there has been good collaboration with provinces, EGP researchers have been able to answer particular data needs concerning impacts of resource development and emerging provincial regulations. According to the Lac Dasserat case study, a lack of EGP connection to EA service users and to industry was seen as a challenge as the EGP Program itself does not have linkages to CEAA or the EA service at NRCan. The links that exist are developed by the individual project leaders.

RA Service has experienced challenges with regard to NRCan’s role in the provision of resource assessments and expertise to non-terrestrial conservation processes.

Challenges regarding clarity of ESS role observed in creation of MPAsand NMCAs in Southern Canada; some suggest updating the MERA TOR: While there is a formal process that delineates NRCan’s role and obligations in providing expertise to terrestrial MERAs and NMCAs, there are no such terms of reference for other federal resource assessment needs. The terms of reference, work plans and budgets for resource assessments undertaken for DFO Marine Protected Areas (MPAs), and EC Wildlife Management Areas (WMAs), and Parks Canada National Marine Conservation Areas (NMCAs) in southern waters, have been developed on a case by case basis. ESS documents note concerns that there is a lack of clear terms of reference and framework for resources assessments of marine protected areas (especially south of 60). Those involved in the MERA process also noted that the roles of various stakeholders and the requirements for ESS-GSC are unclear for the introduction of MPAs, WMAs and even NMCAs.

There is a debate as to whether ESS-GSC should be involved in MPAs because DFO (the responsible department) is not party to the MERA terms of reference. Those who believe ESS-GSC should have a role point to their current role as the impartial federal expertise in providing resource assessments for National Parks through the MERA process. They explain that if the federal government is to make a decision to limit development in a marine protected area, an assessment similar to that done for land-based decisions should be completed. In the Laurentian Channel and Hecate Strait MPA case studies, this rationale was the underlying reason for requesting ESS-GSC involvement.

NRCan continues to work with DFO to identify appropriate measures for involving ESS-GSC resource assessments in MPAs:Review of NRCan communication with DFO indicates that ESS has engaged with DFO at various times to clarify what NRCan’s obligations and role should be when requests for resource assessments for MPA establishment processes have been made. These memos indicate that Assistant Deputy Minister level agreements were reached in 2010 and 2011 to provide analysis of existing ESS-GSC data for the Laurentian Channel and Hecate Strait MPAs (2010), and then to provide a reduced role of support for DFO in reviewing work of third party contractors (e.g., St. Anns Bank, Shediac Valley and American Bank MPAs).

To study this issue, a Resource Assessment Task Group (including NRCan, Parks Canada, DFO, EC and AANDC) was convened in 2011 with a mandate to propose a process for formalizing departments’ involvement in and approach to determining the level and delivery of renewable and non-renewable resource assessment for MPAs. At the time of the evaluation research, the work of the task group had not been completed. The 2012 report of the Task Group recommended that, “NRCan-Geological Survey of Canada, as the Government of Canada expert, provide analysis and compilation of existing data and information on geoscience and mineral and energy potential” and recommended establishing a process similar to the formal MERA process that is in place for National Parks – i.e. create a working group and senior governance body composed of Parks Canada, Environment Canada, DFO, NRCan, and AANDC.^{Footnote 22} NRCan formally agreed with the report recommendations in October 2012 and ESS officials continue to work with DFO through the Interdepartmental Committee on Oceans to identify appropriate measures to implement the recommendations. At the time of the evaluation, this recommended formalized process had not yet been established. However, DFO does not currently have plans for establishing MPAs in the next three to five years.

3.2 Performance (effectiveness, efficiency and economy)

3.2.1 Achievement of Expected Outcomes

Evaluation Question	Lines of evidence	Assessment
4: To what extent have intended outcomes been achieved as a result of the program/services?	Document/literature review Interviews Case studies File review Survey	Overall, the ESA Sub-Activity has made good progress, performing well on its intended outcomes. Some challenges were identified for each ESA component.

Immediate Outcomes

Evidence:

EGP research projects showed strong progress on isolating natural from anthropogenic contaminants

EGP research projects are seen by stakeholders to demonstrate important progress on isolating natural from anthropogenic risks from development: Both ESS and non-NRCan interviewees identified important progress made by EGP projects in identifying and characterizing risks of environmental impacts from development. Most noted that EGP projects have helped to better understand and isolate natural from anthropogenic contamination in the environment. Examples include projects on geohazards mapping, metals mining contamination, risks to seabed from tidal energy development, quantifying and characterizing contaminants in oil sands development, and understanding land subsidence risks from natural gas projects. Many of these research areas were identified as having made good progress in ESS Mid-Year and Year End documents as well.

In the EGP Permafrost case study, the increased availability of ground thermal data resulting from this research has facilitated further research, which has led to a better understanding of geomorphic processes operating in the region. This understanding is considered a key aspect of engineering safer northern pipelines, mining and infrastructure projects in the north. Similarly, the Eco Indicators case study (a project from the Environment and Health Program that preceded EGP) shows that the project lead to new methods used to monitor ecosystem health of national parks. The project was led by Parks Canada, who is still using the methodology and data developed through this project to monitor the health of six national parks. Parks Canada plans to expand the application of this technology.

GSC researchers are developing innovative methods and approaches to fill scientific gaps needed to inform regulatory decisions

Evidence from case studies shows that EGP projects have pursued innovative and promising methods for identifying and characterizing risks from development: The Nitrogen and Sulphur case study demonstrated that project deliverables are generally on track and that the innovative approach to isolating anthropogenic nitrogen and sulphur contamination using isotope deposits in tree-rings has demonstrated strong potential so far for assessing cumulative environmental impacts from hydrocarbon extraction. This technique has never been used in this way before. Similarly, the Lac Dasserat case shows that the innovative application of using lake sediment archives and hyperspectral remote sensing to assess cumulative impacts and follow-up programs downstream of contaminated mine sites on surface water has showed promising results for the basis for indicators that could one day be adopted by industry to better study cumulative effects issues in new or reopened mines. According to project partners, if these promising techniques are successful, industry and regulators will have access to new data and capabilities in determining environmental effects from hydrocarbon extraction and mining projects that did not exist before.

Evidence:

GSC researchers have provided strong technical reviews of proposed projects, advice, and presentations to regulators, regulatory authorities, First Nations, provinces and project proponents.

ESS expert reviewers believe strongly that they have informed stakeholders on the validity of proponent conclusions, increased their understanding of environmental effects, and to a lesser degree, understanding of proposed mitigation strategies: Surveyed ESS expert reviewers gave positive (scores of 4 or higher on a 7 point scale where 1 = ‘not at all’, and 7 = ‘to a large extent’) assessments of to the extent to which they inform stakeholders of the validity and appropriateness of proponent conclusions on the potential geoscience-related environmental impacts from proposed development projects (100%), provide increased understanding of environmental impacts (74%), and on the validity of proposed mitigation strategies (68%). Approximately half gave strong (scores of 6-7) ratings (51% and 46% respectively) to these except for the validity of proposed mitigation strategies (28%). Evidence from interviews suggests that this positive, but weaker self-assessment on the validity of proposed mitigation strategies relates to the challenges of assessing the predictive nature of proposed mitigation measures.

Widespread agreement that ESS-GSC does a very good job at informing stakeholders of the validity of project impact claims and mitigation strategies: Both internal ESS and external (including non-NRCan) interviewees gave ESS-GSC high praise for its ability to review projects and assess the validity of proponent claims related to environmental impacts and the effectiveness of proposed mitigation strategies. ESS-GSC reviewers described their role as trying to assess if the proponent’s plan is viable and safe and if the predicted environmental effects are reasonable based on geoscience evidence. Interviewees cited numerous examples of projects in which ESS-GSC provided a solid review of the project. For example several interviewees described instances where communities said they now had a better understanding of the project after hearing testimony from ESS-GSC reviewers. In the MGP review case study NRCan researchers also participated in workshops with the proponent’s geohazards consultants to verify design assumptions of the project. In the Port Hope environmental assessment case study, ESS-GSC influenced criteria for determining the effectiveness of the remediate project.

Presentations by ESS-GSC researchers to review panels, communities and review boards seen as high quality and able to distil complex geoscience issues for decision-makers: External interviewees praised the ability of ESS-GSC researchers to explain the reasons for their concerns or information requests to members of review panels, review boards, and federal regulatory authorities. For example, in the Lower Churchill project, interviewees reported that the ESS-GSC presentation on landscapes and reactions to seismic events given to the Panel had helped them to understand the issues with the project. In the Sydney Tar Ponds project, interviewees reported that there was a presentation on geochemistry for the panel to clearly understand the issues. This was seen as very useful especially on the technical issues of modelling that are challenging for the panel and other participants to understand. Internal interviewees indicated that preparing for panels can easily mean reviewing thousands of pages of documentation. This preparation is usually facilitated by EA coordinators preparing questions and answers documents to assist reviewers.

ESS-GSC reviews of the validity of proponent conclusions have led to project improvements, and crucial changes to proposed projects.

File review and interview evidence shows that due diligence reviews have been conducted carefully by ESS reviewers; though some differences of opinion on what the scope of review should be: The EA file review also showed that ESS expert reviewers provided due diligence reviews of proposed projects, accepting proponent claims when valid, recommending changes to mitigation measures, improvements to monitoring plans, and changes to projects to reduce adverse impacts. Interviewees reported that ESS-GSC researcher contributions to reviewing project proposals provide public accountability to the parties involved. Researchers indicated that they assess the proponent proposals holistically and therefore have to ask for the necessary detail to understand the submission. The thoroughness of the review means that issues can be identified early so that projects do not create unforeseen and uncontrolled environmental impacts when they go forward.

There appears to be some difference of opinion between ESS-GSC research scientists and those involved in managing the EA process outside ESS as to what the burden should be on the project proponent. The concern raised by some of those outside ESS was that reviewers do not always limit their comments to what is needed to make an EA decision, and that they may be requesting information that will be needed down the road in the project implementation and monitoring phases, but may not be needed to determine likelihood of adverse environmental impacts. On both sides of this however, there was agreement that the reviewers job was to assess the reasonableness of the proponent’s conclusions on environmental effects and mitigation strategies.

Evidence from interviews demonstrates that ESS-GSC review has lead to project improvements for proponents: ESS-GSC reviewers and government decision-makers identified a number of projects in which the ESS-GSC reviewers had identified significant errors or shortcomings in environmental impact statements that proponents were able to correct early in the project to avoid issues at a later date. For example, several interviewees identified shortcomings in proponent hydrogeological models in mining projects that were identified as ESS-GSC researchers asked for more information on how the values in the models have been arrived at. These supplemental information requests often resulted in proponents doing further monitoring work that is targeted based on ESS-GSC requests. Since the reviewer is trying to assess whether or not the mitigation strategy will work, their comments often include proposals to improve the mitigation mechanisms. For example, on one project ESS-GSC reviewers discovered historic landslides during the course of their own previous research that could have generated tsunami waves. ESS-GSC was able to submit additional evidence that the proponent will consider in their design.

Evidence from case studies demonstrate crucial changes to projects based on ESS-GSC reviews: Each of the EA case studies showed that ESS-GSC reviewers’ materially impacted the design of proposed projects in a direct and indirect manner. For example, in the review of the Victor Diamond Mine project, ESS-GSC was praised for the reviewer’s ability to distill the complex groundwater modeling issues for the Responsible Authority so that they could understand the implications. ESS-GSC’s expert knowledge allowed for an effective challenge to the initial groundwater modeling interpretation done by the proponent that ultimately resulted in changing the project’s monitoring program to ensure that muskeg was properly re-watered to limit methyl-mercury impacts from the project. ESS-GSC continued to inform the Province of Ontario as it proceeded with its licensing review of the project once approved federally.

Some raised concerns that the meaning of reviewer comments could be changed inadvertently through NRCan’s approval process

Several internal interviewees noted that there may be some confusion about who should have the final word on the content of reviewer comments. In June 2011, NRCan implemented the Directive on Environmental Assessment, which proscribes the levels of management approval for comments as they move through ESS and SPI and then out of the department. Prior to 2011, experts were consulted before changes were made to their comments; however it appears that over the last year this has occurred less often as ESS incorporated more approval levels into their review process. Some expert reviewers expressed concerns that their comments have been altered through editing without notifying them. This is potentially an issue if they are called to testify at proceedings where they have to be able to stand behind their comments. This did not appear to be an issue in the four EA case studies examined more closely during this evaluation, though the cases looked at comments provided prior to 2011. In the case studies, emails between ESS EA Coordinators and reviewers demonstrated that reviewers tended to be asked about changes to their comments prior to sending them to SPI. The comparison case study suggests that making this check is a good practice in EA reviews. DFO, for example, involves reviewers in discussions about their comments prior to their release.

Evidence:

ESS-GSC has informed stakeholders of mineral and energy resource potential for federal lands (onshore and offshore) proposed for protection with high quality studies and presentations

ESS-GSC researchers and external stakeholders involved in MERAs report that ESS-GSC does an excellent job of informing them of the mineral and energy potential of proposed protected lands: Most surveyed ESS-GSC researchers (n = 8 of 13) indicated that their service has at least contributed (scores of 4 or more on a 7 point scale where 1 = ‘not at all’, and 7 = ‘to a large extent’) to clients and stakeholders awareness of the mineral and/or energy resource potential of proposed federal lands (onshore and offshore) being considered for protection. Almost half (n = 6 out of 13) indicated this was a strong contribution (scores of 6-7).

Interviewed decision makers in the MERA working group and users of those decisions in government policy circles all reported that ESS-GSC has done very well at informing them of terrestrial mineral potential. They described ESS-GSC MERA Open File reports as thorough, of high technical quality, providing the necessary information for informed decisions, scientifically credible, and neutral. Conclusions reached by ESS-GSC on the mineral potential within the areas of interest were described as scientifically defensible, and have gone unquestioned by decision-makers within MERA due to the obvious high quality work behind them. ESS-GSC presentations to the MERA working group were described as clear, well-prepared, and bringing to bear a wealth of subject matter knowledge. Consider the four RA case studies of resource assessments for national park processes for Nahanni, and Thaidene Nene/East Arm as well as Marine Protected Areas for Laurentian Channel and Hecate Straite. Workplans for all four cases were developed in consultation with the intended users of the assessments in the MERA working group (terrestrial) and with DFO (marine) to ensure ESS-GSC could meet their needs. The Open File Reports each provided conclusions on the mineral and energy potential of these areas for consideration.

Generally positive perceptions of ESS-GSC ability to inform community stakeholders (e.g., aboriginal communities, the public) of findings: External non-NRCan interviewees reported that ESS-GSC staff have done well at distilling mineral potential findings for aboriginal communities in the North as part of the Parks Canada MERA consultation process. ESS-GSC researchers were said to have been well prepared at these sessions with plain language assessments of the mineral potential. Evidence from documents also shows that ESS experts have communicated research results. For example, Bathurst Island National Park results were presented to community members in Resolute Bay in September 2010 and at two community meetings in Iqaluit in November and December 2009.^{Footnote 23} In the Nahanni case study ESS-GSC participated in Parks Canada community consultations with the Dehcho and Sahtu First Nations in 2008 and 2010. Both AANDC and Parks Canada stakeholders indicated that ESS-GSC researchers have performed very well in explaining their findings at these consultations.

Concerns have been raised over ESS-GSC ability to respond to non-terrestrial resource assessment needs.

ESS-GSC seen as less responsive to providing assessments of mineral potential in non-terrestrial applications; there is confusion as to process, and obligation: While ESS-GSC has always contributed to terrestrial MERA when asked, ESS-GSC’s ability to contribute to non-terrestrial mineral and energy resource assessments has been mixed. At a working level, external interviewees described ESS-GSC staff as extremely knowledgeable and helpful when they were able to be involved in MPAs. Case studies show that ESS-GSC developed new analysis for Open File reports based on their existing data for the Laurentian Channel and Hecate Straight areas of interest. Documents also report that ESS-GSC experts participated in several calls with DFO offices on the selection of candidate areas of interest for MPA status.^{Footnote 24}

Interviews show that NRCan declined to do the assessments of three of the six MPA areas of interest to DFO, due to not having enough resources available. However, ESS-GSC did agree to develop the work plans and to review consultant deliverables when DFO contracted out the resource assessments. While ESS-GSC’s reviews of the work was considered to be of high quality, the process of contracting out the work, developing a statement of work by non-experts at DFO, and the incompleteness of some contracted out assessments meant that DFO was unable to complete work on establishing new MPAs. Both internal and external interviewees see ESS-GSC as a logical choice for providing resource assessments to this process, however there are issues with not having a clear process, limited resources and unclear obligations.

Intermediate Outcomes

Evidence:

EGP undertakes unique research on reducing environmental impacts from natural resource development. The program theory is that this will one day form the basis of new cost-effective performance based methods used by project proponents.

Interviewees provided some evidence of early use of guidelines and best practices evolving from EGP projects; however most considered it to be too early to assess: Both internal and external interviewees noted that it was too early to assess the development of guidelines and best practices that could be used by industry (i.e., project proponents). Some interviewees gave examples of guidelines and best practices that will be developed over the next few years based on EGP research. Examples included draft tidal energy development standards being developed for discussion at the International Electrotechnical Commission (IEC), guidelines being developed by Environment Canada on oil sands monitoring, and data from EGP CORES projects that can be used to develop guidelines in the future. The techniques being developed through EGP are often in partnership with industry to varying degrees, and some project leaders indicated that their industry partners have been able to use data from EGP projects to develop better environmental impact statements for their EA processes. This seems to be mainly a result of industry involvement in the project, and not a broader adoption of the new techniques being developed.

Case studies show good progress on research, but the development of tools and techniques that could be used by industry is still a number of years away in most research projects.

In the Nitrogen and Sulphur case study, partnerships with industry were developed to obtain site access in Fort Saskatchewan and Edmonton to conduct sampling research. However, industry has not been engaged yet with tools and new processes as these are more than five years away from being developed. The Nitrogen and Sulphur project partners (ESS-GSC and EC) have made links with industry associations to keep them informed of the progress, and have a five year plan for informing industry once the research is complete. Similarly, in Lac Dasserat, the research team and partners at EC and NRCan MMS see potential for this study on processes and indicators to give the mining industry and regulators indicators for predicting the impacts of these mines. However, the development of this is still several years away. At the time of the evaluation, data from the first phase of the study had just finished being analyzed.

The EGP Permafrost project has also generated knowledge with respect to permafrost and terrain sensitivity that could inform both the design and environmental management of northern onshore pipelines in the Mackenzie and Alaska Highway corridors. The case study suggests that information provided in a number of reports and databases enhanced knowledge on permafrost and geotechnical conditions and delivered on the early outcome to provide updated Geographic Information System compatible with geoscience databases. This allowed the project to make substantial progress to meet final outcomes to both mitigate conditions that may be hazardous to pipelines and to reduce the impact of pipelines on the environment. This Permafrost research is more advanced than the other case studies in producing outputs to be used by industry. This is in part because of the long standing industry partnerships dating back to the beginning of the Mackenzie Gas Project research in 2001 and involvement in current pipeline proposals (e.g., Alaska Highway). Project researchers see this research as highly applicable to other Northern development projects where roads or pipelines are being built over or through permafrost which can change significantly in short and long periods of time due to environmental conditions.

The evaluation found evidence of federal decisions informed by all three ESA components.

The EGP program has contributed through its focus on improving regulatory quality by providing new geoscience evidence and guidance to federal regulators. The ability to do this requires good project links to policy groups, which some suggest could be improved.

Documents describe a number of instances in which regulatory decisions have taken into account sound geoscience produced through EGP projects: ESS documents show that EGP research that assessed and characterized mercury risks in arctic and oil sands development projects informed Canadian (e.g., AANDC/EC) and international mercury assessments (e.g., Arctic Monitoring and Assessment Program/United Nations Environment Program). Also, Health Canada has implemented ESS recommendations on the use of geochemical data that determines background concentrations of mercury in soils in environmental and human health risk assessments.^{Footnote 25}

EGP case studies and interviews show good potential for research to be used once projects are further advanced providing that projects have connections to policy and regulatory groups: The preliminary results from Lac Dasserat have begun to be presented to users in the public sector. The project manager has engaged the EC Environmental Effects Monitoring (EEM) and CEAA Environmental Effects working groups, and the province of Quebec. CEAA and EC EEM are interested in this project because it will help them to make policy decisions related to mining projects in Canada that are subject to CEAA reviews or monitoring by EC once approved and established. According to internal and external interviews, the direct impacts on informing federal decisions will be longer term and dependent on having EGP researchers that are involved in EA’s and projects connected to policy groups. This is not the case for all projects at this time.

Permafrost research now part of the EGP has had significant impacts on government decision-making in the Mackenzie Gas Project EA review.

In the Mackenzie Gas Project, evidence from case studies, interviews and documents showed that EGP Permafrost research allowed the federal authorities to know what issues to include in the Environmental Impact Statement, to have the knowledge and expertise to review the proponent proposal and to respond to Joint Review Panel recommendations:Overall, the EGP Permafrost research and MGP EA review case studies cover 10 years of ESS-GSC involvement on generating new research to inform federal decision makers. Interviewees noted that the most direct impact on federal decisions from EGP research so far has been with respect to research done in support of the MGP project in which the federal government was a partner and therefore project proponent. According to the Northern Pipelines Permafrost case study, the work done in advance of the MGP review has evolved into the Permafrost work now part of the EGP program.

The EA MGP case study shows that this research was used by ESS-GSC experts to review the MGP proposal and to inform deliberations of the Joint Review Panel (JRP). That same research and expertise was then used by the government to respond to the JRP’s 176 recommendations. Permafrost research conducted under what is now the EGP Pipelines project was used by EC, AANDC, and IC to inform the JRP decision on what the federal government undertakings should be in mitigating environmental impacts and follow-up monitoring requirements. ESS-GSC researchers were called to testify up to four times at hearings that lasted approximately 8 months. By all accounts, ESS-GSC researchers were very well prepared for the hearings, and had gained the confidence of the panel members. This was clear in the official findings from the JRP that made specific reference to NRCan data and quoted NRCan ESS-GSC researcher testimony extensively in their two volume report. NRCan researchers had provided to the JRP members specific comments and analysis of proponent conclusions on many issues including the likelihood of permafrost erosion, slope stability, sediment drainage, and representation of diverse ecological zones along the proposed pipeline corridor.

Decisions made by Federal Regulatory Authorities have direct links to advice from ESS-GSC reviews of project proponent environmental impact statements (EIS) and supporting documents. Interviews and case studies suggest that reviewer due diligence ensures decisions consider geoscience expert views among policy and economic factors

Internal and external interviews, including interviews with regulatory authorities, reported that regulators and panel members almost always take into account ESS-GSC advice because it is trusted and well-respected. Not surprisingly, nearly all surveyed ESS-GSC EA reviewers (97%) indicated that their service has at least contributed (scores of 4 or more on a 7 point scale where 1 = ‘not at all’, and 7 = ‘to a large extent’) to Federal Responsible Authorities using ESS geoscience expertise to carry out their obligations and render their decisions under CEAA and other federally legislated environmental assessment regimes. Just 31% of EA reviewers indicated a strong contribution to this (scores of 6-7), suggesting that they perceive there to be other factors beyond their advice that impact these decisions.

ESS-GSC seen as doing its due diligence by delivering strong technical submissions that allow federal decision-makers to make informed choices: Nearly all interviewees explained that a thorough review is always valuable even when it does not result in changes to the project. It means that decision-makers (Responsible Authorities or panel members) can be sure that ESS-GSC has done its due diligence and proceed with confidence on whatever decision they make. Interviewees explained that the value of EA review is that it acts as a public check for decision-makers since ESS-GSC reviewers have a strong understanding of the science that informs proponent EIS’s. By checking to ensure that the claims are sound, ESS-GSC helps decision makers to prevent projects with unmitigated environmental impacts from going forward, and to ensure that projects that do go forward have adequate mitigation measures and designs. Documents and interviews provided many examples of ESS-GSC advice used in regulatory decisions. For example, DFO representatives stated that the information provided by NRCan-ESS was very helpful in their decision on the Sydney Tar Ponds and Coke Ovens Sites Remediation Project EA review, and they sought out ESS-GSC advice on an alternate beach disposal containment structure once the review was concluded.^{Footnote 26} In the Lower Churchill project, DFO, EC, HC and ESS-GSC collaborated to show how flooding of the soil and understory vegetation for the reservoir could increase the rate of methylation of the mercury present in the Lower Churchill River that would harm fish in a nearby stream. Using ESS expertise on where the mercury was in the environment and how to reduce it, NRCan was able to inform a panel recommendation for further research.

In each of the case studies, reviewers provided due diligence reviews, carefully assessing the validity of proponent claims. The Port Hope environmental assessment case study provides an example of how the environmental assessment of the project evolved through an informed decision-making process. ESS-GSC reviewer comments were considered by CEAA and CNSC licensing authorities in approving the project. ESS-GSC was first asked to review the EIS and project documents at the CEAA screening phase in 2005-06, then to testify at a CNSC panel in 2007, and then review at the follow-up phase in 2009-10. Over this time period, reviewer concerns over establishing criteria for determining the effectiveness of the proposed site remediation activities, establishing a sufficient network of groundwater monitoring sites, and explaining anomalies in initial groundwater monitoring data were incorporated into these decisions. The proponent ultimately committed to addressing these in subsequent phases of the project.

ESS-GSC resource assessments are a key piece of information used by federal decision-makers in boundary decisions for national terrestrial parks and are expected to be used in marine protected areas for those that ESS-GSC has provided assessments

Evidence from documents as well as all interviewed users of ESS-GSC resource assessments reported use in federal decisions on park and conservation area boundaries: Externalinterviewees each reported that ESS-GSC resource assessments for terrestrial MERA’s have always been used as part of the decision on boundaries. It is considered a key part of the decision making process by all involved. Examples of this can be found in the case studies and the document review where ESS informed the boundaries of two large scale national parks during the evaluation period (Nahanni and Nááts’ihch’oh National Parks) by providing geoscience analysis and advice to the Senior MERA Committee through the formal MERA process.

• Nahanni National Park decision: ESS-GSC MERA results contributed to an expanded Nahanni Park boundary that maximizes mineral potential and conservation values. On June 9, 2009, the Government of Canada announced an expanded Park boundary (6 times larger that the existing park). The case study showed that the geoscience information contained in the Open File report was considered as a key piece of data in the final park boundary decision. The accuracy and detail in ESS-GSC’s analysis helped to shape the final boundaries and significantly informed the deliberations of the Working level and Senior MERA committees. The final boundary position allowed for inclusion of 91% of the South Nahanni watershed within the Decho region in the Park, protected 95% of the Upper Nahanni Caribou herd habitat, and met the desired grizzly bear population targets, while 47% of the area with strategic mineral potential (i.e., tungsten) and the areas with hydrocarbon potential were excluded from the expanded park.^{Footnote 27}
• Nááts’ihch’oh National Park decision: Interviews and ESS documents showed that the boundary decision reached on the proposed Nááts’ihch’oh National Park (in the Sahtu region of NWT adjoining the Nahanni National Park) was informed by geoscience from the Nahanni MERA study. Documents indicate that ESS-GSC and Parks Canada MERA Working Group co-chairs lead the development of a compromise boundary option, evaluating the impacts of the proposed option on conservation and resource development potential. This option was presented, considered and agreed to by the Senior MERA Committee in 2010.^{Footnote 28}

ESS-GSC researchers that conduct RA studies gave mixed assessments on the degree to which their geoscience expertise and information informed federal decisions: In total, 5 of 13 ESS-GSC researchers rated positively that their RA research has contributed to Federal decisions to designate land for protection under legislated (e.g., Oceans Act, National Parks Act) or policy (e.g., MERA policy) regimes using geoscience expertise and information. Just 3 gave this a strong rating, while 5 indicated that they did not know. Overall this suggests that RA researchers are well aware of the impact of factors beyond their data and conclusions that inform these decisions. Interviewees noted that use of ESS-GSC data has varied by degrees based on factors such as desired political outcomes, relationships with First Nations, and the timeliness of deliberations of the Senior MERA committee (e.g., decision on Bathurst Island was made 10 years after the MERA study was completed).

Evidence from interviews and case studies suggests that there have been challenges in meeting timelines for RA studies, which affects the decision-making and consultation processes they inform.

Concerns were raised in interviews and case studies over timeliness of RA reports and input; users identified negative impacts on federal decision-making processes: Both internal and external interviewees raised concerns that while ESS-GSC resource assessments are of high quality, they are often significantly late which impacts the broader federal government processes that they are intended to inform. For example, the Thaidene Nene (ongoing) and Nahanni (completed) MERA case studies showed that the final Open File reports were three and two years behind schedule respectively. This impacted timing of Parks Canada consultation processes with First Nations, and to some degree appears to have left the MERA process open to criticism by the mining industry. In the case of informing DFO MPAs, while ESS-GSC RA service did provide desktop studies and a review of consultant deliverables on three MPAs, the timing of the reviews was much later than ESS-GSC had committed to and this impacted DFO’s ability to use the studies in order to establish the MPAs (currently no MPAs have been established). The main reason for delay identified by internal and external interviewees was priority setting within ESS by researchers and managers alike. According to ESS-GSC management, lack of clarity and requested changes by PC concerning planned outputs during the course of the project compounded the delay in product delivery.

Unclear federal process for establishing marine protected areas and national marine conservation areas in Southern Canada have meant limited ESS-GSC input to federal decisions.

The composition of the MERA Committee as a forum for debating proposed marine protected areas and conservation areas, which does not include EC or DFO, has been questioned by almost all interviewees. These interviewees noted that the process for considering RA data in terrestrial national parks is institutionalized through the MERA TOR, but that is not so for MPAs. The Lancaster Sound NMCA was held by many as an example of where the current MERA process is poorly equipped to consider geoscience evidence by the key players in non-terrestrial MERAs. As noted earlier, ESS-GSC has not been able to continue to provide geoscience expertise in support of DFO MPAs, and three of the six have been contracted out. While ESS-GSC provided a strong review of the material, stakeholders noted that because of the difficulty in contracting out these assessments, they have not yet been able to use the data to establish MPAs so far. ESS-GSC management notes that ESS ADM informed DFO ADM of the human resources issue upon receiving the written request from DFO for NRCan participation in the MPAs.

Ultimate Outcome

Overall, EGP conducts research to generate science so that federal regulators can make informed decisions about ecosystem risk management in resource development projects. Evidence from interviews and case studies suggest that research focused in this manner will contribute to balancing development and protection of the environment.

Internal and external interviewees noted that EGP is needed to help ESS-GSC deliver on balancing development and protection of the environment and that the mechanism for doing that is developing techniques and tools that can inform the regulatory process and policy groups. The EGP case studies provide examples of how this logic is applied from identifying the ecosystem risk management knowledge gap, to informing regulators, policy-makers, and proponents. For example, the Permafrost case study shows that the thermal and geotechnical data could inform pipeline routing to avoid hazardous terrain to ensure integrity of both the pipeline and environment are maintained. Similarly, the Nitrogen and Sulphur tracing research, when completed, has implications for environmental effects monitoring in the oil sands (EC and Alberta government initiatives), development of mitigation measures by project proponents and considerations in the EA process when new projects are proposed.

The EA and RA Services provide the geoscience basis for decision-makers to balance development and protection of the environment. ESS-GSC researchers see themselves as one part of this decision-process, and federal decision-makers seem them as a key resource. Evidence from interviews and case studies demonstrates the application of ESS-GSC researcher knowledge and data in decisions to approve or change development projects and to delineate national park boundaries.

Many (60%) ESS EA reviewers report positive impact (scores of 4 to 7 on a 7 point scale where 1 = ‘not at all’, and 7 = ‘to a large extent’) on protecting the environment through informed federal decisions on development and land use, with just 34% giving a strong rating. Similarly, just 6 out of 13 RA researchers felt that they had some impact on achieving this balance. This is in part explained by ESS-GSC interviewees that the point of the EA review is not to prevent development but to ensure that the environmental impact risks are understood and properly mitigated. It is up to regulatory authorities to make decisions on what will be acceptable and what projects will go forward. The evaluation case studies and interviews with external stakeholders for EA and RA demonstrate that ESS-GSC researchers contribute to those decisions.

External interviewees noted a number of instances in which ESS-GSC reviews and testimony was a key consideration in declining proposed projects that would have had unacceptable environmental impacts. Evidence from interviews and case studies also show that many projects have been changed based on ESS-GSC review so that they can go forward with acceptable environmental impact mitigation measures in place where previous measures were not adequate.

ESS-GSC RA studies provide the means of considering empirical information in the balance between conservation and development values. The consideration of strong scientific data on mineral and energy potential from ESS-GSC was said to provide legitimacy to the park establishment process by informing decisions about what potential is being given up through the park boundary process and if it is appropriate. Some external interviewees explained that the processes are a negotiation between conservation and economic development values. ESS-GSC facilitates striking a balance between the two perspectives. The MERA process for terrestrial park establishment requires the consideration of geological, ecological, and economic data in the boundary decisions.

Evaluation Question	Lines of evidence	Assessment
5. Have there been any unintended (positive or negative) outcomes?	Interviews Case studies	Yes. Mainly positive unintended outcomes of follow-on economic development, new research, training and networking opportunities for researchers. Only negative unintended outcome was the perception among ESS-GSC experts that EA reviews are less valued for career progression.

Evidence:

EGP and RA have generated data that supported economic development in the North

The public-private partnerships on EGP permafrost projects have made monitoring data publicly available for use by smaller companies and Northern communities: Evidence from the Permafrost case study suggested that there can be unintended impacts of induced development from EGP work on understanding permafrost conditions. For example, the borehole data that exists as a result of the public-private partnerships between ESS-GSC researchers and larger industry partners is publicly available to smaller resource development companies that cannot afford the baseline studies necessary to develop complete project proposals. As well, Northern communities use this data to assist with their infrastructure development and land planning as conditions in the North can change radically over time.

Economic development may occur due to RA studies: ESS-GSC Open File reports from MERA studies are made public, and industry and others are able to use this information. The areas covered by RAs are generally much larger than the area being considered for national park status which means data is collected for areas outside the final park boundaries. Evidence from interviews and the Nahanni case study showed a number of instances in which RA studies have later supported resource development projects. Interviewees noted that RA studies have been used by project developers in industry to begin exploration and create projects that later stimulate economies. For example, park establishment research for Tuktut Nogait National Park (established in 1996) discovered a large gravity anomaly that indicated copper potential. This prompted additional research projects that eventually found the deposits in the nearby area of Victoria Island, which stimulated exploration activities there years later. Several interviewees noted that examples such as this are commonplace for MERA’s since the process began.

EA and RA activities have resulted in unique researcher networking opportunities and training of highly qualified mineral specialists

Networking of federal researchers to review EAs: In responding to EA requests, a few researchers noted that they have benefitted from collaborating with fellow ESS-GSC researchers and researchers in other federal departments with whom they would not have otherwise had the opportunity to work.

Highly qualified mineral researchers have been trained through RA studies: Those involved in delivering RAs have used professors and their graduate students in their work, in part to complete projects within available budgets. ESS-GSC researchers indicated that students receive valuable training in doing this kind of work due to the variety of techniques needed to answer the question of mineral potential and then are able to fill jobs in government related to mineral exploration.

EA reviews have led to new research projects at universities and access to data produced by proponents that ESS-GSC researchers could use in further studies

Proponent data provided during the EA review process have been made available for further ESS-GSC research in some cases: A few ESS-GSC interviewees noted instances in which datasets from fieldwork done by project proponents was provided by ESS-GSC for EA review, and then was used by ESS-GSC with permission in related research applications.

New research commissioned, often with universities, as a result of information gaps identified by EA review: Evidence from case studies and interviews indicates thatknowledge gaps identified by EA review process has led to new research. A few interviewees noted that this tended to be projects that proponents have commissioned with universities in order to meet undertakings as a result of the EA review, but has also included research done by ESS-GSC in partnership with federal partners such as DFO, and provincial partners including land use planners in Alberta, and research undertaken during the Mackenzie Gas Project review case study. As examples, research projects have included new studies on seismicity and landslides, permafrost conditions, waste water contamination, and oil sands development.

Some ESS-GSC researchers perceive heavy EA review involvement to have negative implications for their career progression

Most interviewed ESS-GSC research scientists involved in EA reviews noted that the time they have to spend on these reviews has had a direct impact on their ability to publish research findings from their work. The more frequently engaged researchers indicated that it was not uncommon for them to spend up to several months a year on EA reviews. In addition to EA reviews, once involved in a project, ESS-GSC researchers may be asked to review follow-up monitoring requirements for projects as they are implemented and projects that go through the panel review process (the highest level of EA) can have undertakings for the proponents, which ESS-GSC researchers are asked to review as well since they possess the necessary expertise in the federal system. This view that EA reviews can impact career progression was also held by 20% of surveyed researchers that contribute to EAs. The EA service comparison case study suggests that this perception may also be a concern to DFO and EC researchers. DFO has experimented with a peer review system of scientist advice for complex EAs so that they may use that as part of their promotion package Since April 1, 2012 ESS-GSC EA service has been keeping track of experts' time spent on EA reviews, by asking researchers to report how much time they spend on a given EA. In 2012-13, the total equivalent FTEs was 2.1 for a total of 37 involved experts (note that the evaluation covered EA reviews up to 2011-12).

Evaluation Question	Lines of evidence	Assessment
6. What are the factors (both internal and external) that have facilitated or hindered the achievement of expected results?	Interviews Survey Case studies Document review	Internal factors: multidisciplinary EGP design, high quality/depth of ESS-GSC experience, researcher availability. External factors: site access/location, relationship with First Nations, EA project sensitivity and complexity.

Evidence:

Internal – EGP collaborative multidisciplinary approach to research questions is a key facilitating factor

EGP’s collaborative multidisciplinary approach to research as part of project design has facilitated progress on expected results: The multidisciplinary approach to the research – combining geoscience with other science disciplines such as ecology – was identified in the Nitrogen and Sulphur Cycles, Lac Dasserat, Permafrost and Eco Indicators case studies as a key factor for success. All four EGP case studies indicate that these collaborations are often the result of the professional networking and experience of the ESS-GSC researchers involved. ESS documents also show that strategic agreements have been signed with key collaborators to deliver the EGP outputs (e.g., three agreements with EC, one with AANDC, and NRCan’s OERD). By combining geoscience with other disciplines, EGP projects are able to develop innovative methods for identifying and assessing environmental impacts.

Internal – High quality and depth of experience of ESS-GSC researcher expertise is a key facilitating factor for the ESA Sub-Activity. The ability of ESA to deliver service has been shown to be highly contingent on the availability of ESS-GSC researchers.

The high quality of ESS-GSC work and the depth of expertise of ESS-GSC researchers is widely acknowledged as a key facilitating factor in delivering high quality RA studies: All interviewed stakeholders indicated that ESS-GSC’s personnel and products to be of very high caliber. The personal integrity of the researchers involved, the internal peer review process of their open file reports, and the unique depth of qualifications found at ESS-GSC were noted as contributing to the high quality of their work. All surveyed RA researchers have 10 or more years’ experience at NRCan ESS, including 8 out of 13 with at least 20 years. All have been working as geoscientists for at least 10 years, with 10 out of 13 having at least 20 years’ experience in the field. Eight out of 13 have been conducting RA assessments for at least 6 years, including 5 who have done so for at least 10 years.

Strong technical expertise and depth of experience of EA reviewers and EA coordinators are key facilitating factors in the achievement of expected results: Interviewees within ESS and users of ESS reviews indicated that the efforts of the EA coordination office have been a key factor in successfully administering the service. In particular, interviewees noted that the coordinator office, due to their experience with the EA process (and northern processes) and knowledge of geosciences, were good at identifying the appropriate technical expertise within ESS to review projects. The survey of ESS-GSC researchers also identified possessing the appropriate scientific skills (34%) and the professionalism and performance of ESS EA coordinators (31%) as the top factors facilitating their ability to deliver on EA objectives.

Interviews within ESS and external decision-makers, case studies, and the researcher survey each identified the depth of reviewer expertise as the most important factor in achieving intended outcomes. All ESS reviewers are practicing research scientists, meaning that they are familiar with current practices in their respective fields, are part of the published research community, are highly knowledgeable about their subject matter, and have a network of colleagues in the discipline that they can draw on. Users of the service all indicated that because of this, the reviewers comments were seen as highly credible.

EA coordinators, reviewers, and users close to the EA process all noted that the amount of experience doing EA’s is a factor in the quality and efficiency of the review. The more experienced reviewers more easily focused their comments on the big picture with regard to environmental effects of the project, and tend to provide a more solid review. Also, when NRCan is asked to send experts to testify at a panel hearing, the expert’s resume is important. They have to have significant experience in order to properly represent the Department. The Mackenzie Gas Project and Deep Geological Repository case studies highlighted the importance of this experience in testifying before panels. Nearly all surveyed EA reviewers (94%) have 10 or more years of experience at NRCan ESS, including 40% with 20 years or more. All ESS reviewers have been working as geoscientists for at least 10 years, with 63% having at least 20 years of experience in the field. Just under half (49%) have been doing EA reviews for at least 10 years.

Internal – Availability of ESS-GSC expertise is a key factor for success of the ESA, and is increasingly at risk in the EA and RA components as key expertise areas retire

Human resource constraints appear to be hindering the ability to commit to projects and therefore the achievement of expected results: ESS documents indicated that constraints on resources, such as availability of ESS-GSC researchers and field work issues hindered the achievement of expected results.^{Footnote 29} Several internal interviewees found it difficult to attract and retain researchers within ESS for EGP projects. They noted particular challenges with regard to research on closed mines, and finding researchers with experience in the oil and gas sector. Some interviewees within ESS-GSC added that the horizontal management structure, while having significant benefits, has also meant that researchers report to more than one Director General and that has made it difficult to get decisions on assigning researchers to EGP projects. It was also noted that these issues have meant delays in some projects.

The ability to respond to requests is directly influenced by expertise availability. Some experts and expertise areas are thinly stretched and this is expected to continue: ESS has so far been able to meet requests for review. However, internal and external interviewees expressed concerns about ESS-GSC’s future capacity to do this. Most interviewees expressed concern that in recent years ESS has been strained to provide reviews. The file review identified over 10 geoscience research fields in which ESS-GSC provides expertise to EA requests. The most frequently identified were geology, hydrogeology, geochemistry, seismic hazards, and geotechnical engineering.

Nearly all interviewees expressed concerns about the availability of hydrogeologists, and those close to the service noted an ongoing shortage in geotechnical engineering as well. These interviewees explained the shortage as a result of promotion and retirement of experienced reviewers, and ESS priorities in programming that deal with mineral exploration rather than environmental issues (approximately 70% of ESS-GSC programs). This focus and the smaller funding for environmental issues compared to mineral exploration (e.g., TGI, GEM) were said to not easily attract researchers in hydrogeology, or geotechnical fields to ESS-GSC. The survey of EA reviewers indicates that 23% plan to retire within the next 5 years, and a further 26% within the next 6 to 9 years. Put another way, 49% of ESS reviewers plan to retire within 9 years. Respondent expertise areas where one-third or more plan to retire withinthe next 5 years include: geophysical (shallow terrain, deep crustal and marine), geological engineering and geotechnical properties, and marine environmental geoscience, and glaciology.

Difficulty in appointing a permanent successor to the long-time EA service manager was perceived by most internal and external interviewees as having negatively impacted the service: Nearly all interviewees mentioned the broad geoscience knowledge and familiarity with the federal review processes (e.g., CEAA, northern processes) of the former service manager as a key factor facilitating the successful delivery of the EA service. This position has been vacant on a permanent basis since June 2011, and has been filled temporarily while a full-time replacement was being found. The difficulty in filling this key position illustrates the challenge of replacing the broad expertise, judgment and experience needed for this role.

Constraints on availability of ESS-GSC experts have limited ability to contribute to MERA process: Interviewees generally acknowledged that it is a unique combination of skills that allow ESS-GSC to carry out resource assessments needed for these decisions. Interviewees within ESS-GSC noted that this expertise base is critical to the ability to deliver on RA and that ESS should plan carefully to maintain that critical mass. The Laurentian Channel and Hecate Straight case studies illustrated that ESS-GSC had been able to work with DFO to varying degrees to deliver on these and one other marine resource assessment. However, the success of ESS in securing additional C-base funding through the Geomapping for Energy and Minerals program has impacted the availability of researchers meaning that ESS-GSC has been unable to contribute to the extent requested by DFO to MPA resource assessments. ESS-GSC has since limited its involvement to providing advice on setting up external contracts and providing critical reviews of assessments that were completed under DFO contracts. Researcher survey results suggest this could be a greater challenge in the near future as researchers retire. In total, 8 out of 13 surveyed RA researchers indicated that they plan to retire in less than 10 years, including 7 that plan to do so within the next five. This accounts for at least half of surveyed experts involved in RA with expertise in geological hazards (e.g., earthquakes, landslides, deep water hazards, flooding, subsidence/heave, tsunamis), geophysical (shallow terrain, deep crustal, and marine), geological engineering and geotechnical properties, landscape processes and stability (e.g., coastal, fluvial, Aeolian, slope). Interviewees noted that ESS has recently started planning its HR around deposits known to be found in Canada, which they felt should be continued.

External – The size and type of study areas of the research commissioned under EGP and RA components impact the level of effort/resources needed

Obtaining site access from industry partners has been a critical challenge to EGP projects: Most interviewed ESS-GSC researchers reported that one of the main challenges they have encountered with EGP projects is getting site access agreements with companies. They noted that companies are often concerned over the potential liability of having researchers taking samples and investigating the land where they have conducted their resource development projects (e.g., closed mines, oil sands operations). This has resulted in project delays of over a year in some cases. Although there are examples of good industry cooperation, researchers explained that to get this requires a lot of relationship building and developing trust which takes a lot of time. Across case EGP studies, there is general agreement that developing partnerships takes considerable effort but is also a key factor for success of these research projects.

The large geographic size and location of the areas being assessed impacts the cost and timing of the resource assessment study: Those who have been involved in the MERA processes made note of the fact that over the last ten years, the actual size of the conservation areas being studied has increased considerably. Recent examples of Nahanni and Thaidene Nene were covered as case studies in the evaluation and were each over 40 000 square kilometers. As well, the locations of the areas being studied are North of 60 (as per MERA TOR), and prone to uncertainty in timing of field work due to weather. Several interviewees made note of the fact that helicopter time in these areas costs about $500/hour.

External – Federal government relationships with First Nations and the consultation process affects the use of MERA results

The short duration of the First Nations communities’ consultation process has limited ESS-GSC ability to inform community stakeholders in MERA. Concerns were raised among external interviewees with the shortness of the community consultation process (typically one day per community) and the limited time to consider the information provided from the MERA study of the area of interest. The actual length of the consultation process is not within NRCan control, but is a product of Parks Canada and AANDC budgets and mandates. The Thaidene Nene Case study shows that the relationships between the federal government and First Nations were key factors in the process. In 1970, an area of 7407 km2 was set aside for a national park in the east arm of Great Slave Lake. Progress on the park did not occur for another 30 years due in part to unresolved aboriginal land claims. Negotiations have taken place among the federal government, the Northwest Territory Metis Nation and the Lutsel K’e Dene First Nation to settle these issues and in 2005 the Dene delineated an area of just over 57,000 km2 called ‘Thaidene Nene’ as a part of its traditional territory that would be protected throught the establishment of a national park.

External – the level of project proponent cooperation during EA reviews affects the level of effort and length of time taken to review

The level of cooperation of project proponents is a key factor influencing the achievement of expected EA Service results.Evidence from all four EA case studies and interviews shows that the level of cooperation from project proponents is a key factor in the achievement of expected results as it affects the level of effort requires by reviewers and the length of time taken to review the project. The case studies demonstrate that in projects where proponents were able to cooperate with reviewer requests for information or were forthcoming with baseline technical results and complete reports, the review process moved more smoothly. For example, in the Port Hope case study, the proponents generally responded with the requested documentation to answer ESS-GSC reviewer concerns or support their conclusions when asked.

External – the deliberations of review panels, regulatory authorities, and the level of public sensitivity can significantly influence the level of effort required by ESS-GSC as well as the impact of its review

Interviews with researchers and evidence from case studies suggested that the questions and decisions of review panel members as well as the level of public sensitivity has a direct impact on the effort of ESS-GSC in responding to the EA requests. For example, in high profile EA projects, ESS managers were reported to spend more time examining comments and understanding the proposed projects. Also, where projects were particularly complex, reviewers could spend more time responding to Panel questions or justifying their positions at Panel hearings.

3.2.2 Demonstration of Efficiency and Economy

Evaluation Question	Lines of evidence	Assessment
7. Are the program and its activities the most efficient and economic means of making progress towards intended outcomes?	Interviews Survey Case studies Document review	Overall, ESA is efficient and economic, though some systematic challenges have been identified with respect to the MERA and EA services.

Evidence:

The environmental effects (i.e., ecosystem risk) regulatory improvement foci of EGP allows researchers involved in key EA areas to undertake projects to continue to develop expertise that can be used in EAs

The design of EGP to focus on environmental risk characterization has allowed some researchers that are more heavily involved in EA reviews to pursue research that compliments and supports that work: Evidence from interviews suggests that EGP has allowed some ESS-GSC researchers that are heavily involved in EA reviews to leverage their research agendas more in fulfilling their EA duties by having research projects that address EA relevant issues such as environmental risk characterization. Others explained that the methodologies they are working on will ultimately be very portable to other similar types of projects that enter the EA review process such as mines and roads. The Lac Dasserat, Nitrogen and Sulphur, and Permafrost case studies each demonstrated the actual use (i.e., the work done in Permafrost grew from the need to respond to the MGP EA review) or potential application in EA reviews. For example, permafrost studies are relevant to virtually all Northern projects; work done on the chemical signatures of different types of ore deposits is relevant to assessing impacts from mining projects; and work done on sulphur and nitrogen signatures is relevant to cumulative effects monitoring in the oil sands.

EGP and RA leveraged expertise and in-kind resources from NRCan, other government departments, and private sector partners to accomplish research

EGP projects have leveraged PERD, CEF funding and NRCan A-base to augment the ESS A-base committed to the program: Documents, interviews, and case studies show that EGP has leveraged funding and complementary expertise through Clean Energy Fund (CEF) and PERD projects, collaborations with other government departments, NRCan’s Metals and Mineral Sector, and provincial partners. Both interviews and documents show that direct support from the CEF provided the opportunity to extend the scope of EGP work on oil sands monitoring research to include groundwater and surface water, and that PERD has been a valuable funding source for EGP projects on permafrost. EGP has also secured funding from AANDC and Health Canada to facilitate the completion of commitments on long range transport of mercury and radon respectively. Furthermore, documents show that EGP has involved outside organizations in developing clean energy research initiatives including Environment Canada, and Alberta Environment.

Many EGP projects leverage significant financial and in-kind support from industry partners; though these relationships were said to require considerable effort to develop: Evidence from case studies and interviews shows that EGP projects have received financial and in-kind support from industry. Several interviewed EGP project leaders said that partnering with industry to get access to research sites has allowed them to leverage logistical support for their research. In the Lac Dasserat case study, the Program leveraged existing partnerships between Lac Dasserat and Institut national de la recherche scientifique (INRS) in Quebec. As well, the Permafrost case study showed very good leveraging of partner resources at almost three dollars for each ESS dollar. The project partners provided equipment, shared data, field site establishment, maintenance, logistic support, and field study in the Mackenzie Delta region, as well as in the Alaska Highway corridor. By working as partners with industry already doing work in the North, the project has influenced the type of data being collected, the database used, and the locations of the permafrost monitoring boreholes.

RA studies have been able to leverage equipment and data from other geological research ongoing in study areas:Evidence from the Thaidene Nene and Nahanni case studies shows that these assessments effectively leveraged other programs to complete their work. For example, the Thaidene Nene MERA team worked with ESS’s Geo-mapping for Energy and Minerals (GEM) Program to use resources from a geophysical project to map part of the Thaidene Nene MERA study area. The GEM Program collected teleseizmic data collected in the Park area and the MERA study was able to use that data for their report. The costs were split between the RA Service, covering site access and installation, and the GEM Program covering the purchase of the teleseizmic units themselves. The Nahanni MERA team also worked with the National Geochemical Reconnaissance Program in doing work in the Yukon where they shared helicopter and base camps which realized cost savings for both research teams.

The RA and EA Services are delivered economically using a lean service delivery mechanism to administer the services and leverage expertise already employed by ESS-GSC to respond to service requests as needed.

The administration of the EA and RA services on an as needed basis has kept operating costs low, while leveraging expertise that ESS already employs to meet EA review and resource assessment needs: The EA and RA services are delivered for approximately 800,000 $ a year to cover the salaries of the EA and RA service leader, EA coordinators and related operational expenses. Interviewees often noted that the service is well managed under this model. This allows ESS to maintain a core group that is familiar with EA processes, and the mineral and energy resource assessment process that can leverage specific ESS researcher expertise as needed. All ESS interviewees said that the services were highly efficient in how it uses expertise because it does not retain research scientists expressly for the purposes of doing EA reviews or mineral and energy resources assessments. These researchers are practicing research scientists, allowing them to develop expertise and credibility in their fields.

EA service delivery through office of ESS EA Coordinators efficiently assigns reviewer expertise to EA files, and manages the process with SPI. That said, some challenges identified with workload planning between ESS and SPI.

Delivery of the EA service through a central ESS EA Coordinator mechanism was said to efficiently manage EA reviews and is consistent with practices in other federal departments, though some indications that ESS review office is overburdened: Each of the EA case studies demonstrated that the ESS EA coordination office reviewed materials to determine what experts were needed, sought out the experts, and provided guidance on the EA review process via email. The office also follows up with reviewers and negotiates timelines with SPI based on reviewer availability. The comparison case study showed that EC is moving towards a similar mechanism to better manage EA requests. Despite the efficiency of this mechanism, several internal interviewees observed that the managers and coordinators have very high workloads and often put in overtime in order to complete the work on time. Several interviewees noted that at minimum, managing the service is a full time commitment, making it very difficult to deliver on other responsibilities (i.e., as a research scientist).

Use of SPI Environment Assessment Division as single window for CEAA (Agency) requests widely seen as appropriate and necessary, though some noted issues with prioritizing requests from SPI: Nearly all interviewees noted that using SPI EA as a single window into NRCan is appropriate. SPI receives requests for expertise from responsible authorities (RAs) such as CEAA and CNSC, and then sends a request to ESS EA to provide expertise. ESS EA coordinators then use their judgment to determine if ESS-GSC has expertise to contribute to this review. External interviewees felt that it was reasonable for NRCan to have a single window into the department through SPI, while internal interviewees noted that this was essential for protecting ESS reviewers from being overburdened by direct requests from proponents or responsible authorities. A few ESS representatives also suggested that SPI should try to prioritize the requests it receives and could do a better job of keeping the ESS EA unit updated on what requests are likely to come in the near future to assist them with planning. Nearly all recognized that since the EA service is reactive to requests for expertise, it is difficult to plan for.

The use of a more structured planning process for identifying potential upcoming expertise requests would allow for management to plan for at least some of this work in advance, and it is more likely that commitments could be managed in a timely and satisfactory manner. Developing a more structured process for the EA office with NRCan’s Science and Policy Integration (SPI) sector to identify, in advance, potential upcoming requests, would facilitate planning expertise availability. It would be consistent with some aspects of the National Science Advisory Process in place at DFO, identified in the EA service comparison case study. At DFO, the Ecosystems and Oceans Science Sector actually conducts the research, while the Habitat Management Program is responsible for coordinating advice from scientists to respond to EA requests.

Annually, a secretariat within the Ecosystems and Oceans Science Sector undertakes a planning process to develop the sector’s research plan for the upcoming year. Their process includes a call to other sectors to identify their needs, including expected or potential specific advice requests from Habitat Management. This allows the Ecosystems and Oceans Science Sector management to plan and prioritize the overall work of the sector, including both direct research and EA support work, and the resources needed to undertake this work in the upcoming year. For more complex upcoming EA requests, the plan also includes the identification of the team of individual scientists with the expertise needed to support the project. This ensures that scientists are engaged early in all aspects of the project and their work plan reflects this work.

No duplication found in provision of ESS EA expertise with provinces and OGDs; ESS found to compliment expertise gaps.

Users of EA reviewer advice do not think that the advice is duplicated in the federal EA system, and most interviewees have identified collaborative review efforts across federal authorities: All EA service users indicated that there was very minimal, if any, duplication in the federal EA review system when it comes to expert reviewers. The main reason given was that there are so few of the experts to review these projects that RA’s tend to find the expertise wherever it exists. Others noted that the division of mandates among FA departments means that expertise is siloed among departments.

Some interviewees also noted that even between federal and provincial governments the expertise is not normally duplicated, especially with respect to ESS-GSC expertise in hydrogeology, geochemistry, and geotechnical engineering where it has very unique capacity. Interviewees noted many instances where reviews were coordinated among NRCan, DFO, EC and HC experts working on different aspects of a review. For example, on the same mining project ESS-GSC could be reviewing the likelihood of wastewater moving through rock, while EC is looking at the contamination, DFO the impact on fish and wildlife habitats, and HC the implications for human health. In the MGP and Victor Diamond Mine case studies, the contributions of ESS-GSC researchers informed both federal and provincial/territorial review mechanisms (e.g., licensing). In the Victor Diamond Mine case, the Province of Ontario continued to rely on ESS expertise in hydrogeology as it reviewed the permitting requirements of the project once it passed the federal EA stage.

Concerns raised in interviews and case studies that prioritizing between ESS research programs and EA and RA service delivery is a challenge for researchers to navigate and may contribute to inefficient delivery of EA reviews and late timing of RA studies.

EA and RA studies not always seen as a high priority across ESS management and researchers, which has impacted timelines and service delivery: ESS-GSC researchers are involved in several limited duration programs as part of their core research duties at NRCan. The survey of EA and RA researchers indicated that most are part of two or more ESS research programs. Both internal and external stakeholders perceived these programs as taking priority within ESS over RA studies and EA reviews. All interviewed ESS-GSC researchers noted that there is often a conflict between their work on EAs and responding to the priorities of the programs to complete their research projects. These interviewees suggested that ESS could make a more explicit commitment across all managers and researchers that completing the RA studies and performing EA reviews is a high priority for the sector. It was also acknowledged by ESS-GSC researchers and managers that they have to respond to the priorities of programs that usually have 5 years of funding to be completed.

This tension between program and service priorities has proven difficult to navigate as nearly all interviewed ESS-GSC researchers that contribute their time to RA and EA Services indicated that they had competing pressures from managers of the programs they work in for their time. A total of 23% of surveyed EA researchers indicated that the time they spend on this service is at the expense of their other duties, and a further 14% reported that they should spend more time on EAs but cannot because of their other duties. Interviewees noted involvement in a MERA study could take over a researcher’s time very quickly because they are such large undertakings.

Early EA reviewer involvement prior to reviewing the EIS appears correlated with efficient EA review process. However, the sustainability of such practices has been questioned.

Case studies and interviews suggest that early involvement of EA reviewers is correlated with efficient review process: Most internal and external interviewees felt that involving ESS-GSC experts earlier in the project rather than at the environmental impact statement (EIS) review stage would make the review process more efficient. In some cases where this has happened, through informal requests from a responsible authority (RAs), reviewers and RAs noted that it was easier for the review process ultimately because the reviewers could inform the terms of reference and therefore the scope of research and data needed for a satisfactory EIS. Reviewers explained that the current practice is to request their expertise when reviewing the EIS after all of the preparation work has been done, so they come to a project they know nothing about and have to catch-up. Then, if they have questions about the methodology or scope of the EIS it could mean that the proponent has to go back and do work that could have been done already if the expert had informed the TOR.

The EA case studies also demonstrated efficiencies in having early reviewer involvement in complex projects. For example, in the Deep Geological Repository case study (a panel review), CNSC and ESS worked proactively together to provide ESS reviewer comments on the development and scoping of the EIS five years before fieldwork actually began. The project proponent, the CNSC, and ESS indicated that this allowed for a more thorough EIS to be developed and a smoother review process. By contrast, in the Victor Diamond Mine comprehensive study, reviewers only became involved once the fieldwork for the EIS had been completed, and found significant gaps in information and construction of hydrogeological models that ultimately had to be addressed by the proponent.

This efficiency gain notwithstanding, several ESS reviewers noted that to become involved early on every project or even every comprehensive level review, would mean a tremendous (some said impossible) commitment of their time. The service comparison case study found that having “class practice guidelines” similar to those used by DFO could assist in ensuring EIS’s are focused on the right issues and following acceptable standards by the time ESS reviews them. A set of “class practice guidelines” were developed by DFO to manage the time commitment of their researchers when receiving EA review requests. These guidelines provide project proponents with a checklist of what their environmental impact statements should include for particular types of projects to meet DFO reviewer information needs when assessing proponent conclusions. While the contexts for ESS-GSC and for DFO are different, this practice has helped DFO to manage workloads. At the time of the evaluation, EC has also begun to develop a similar practice in an attempt to streamline the review process by making sure information is complete. This appears to also have a benefit of ensuring consistency of advice to proponents across the various departmental scientists.

EA review timelines are currently being met; however evidence suggests ongoing challenges with efficiency of NRCan (i.e., ESS and SPI) oversight to EAs. New project tracking processes have been introduced at ESS, but there is a widespread perception that broader reforms are needed.

ESS reviewers are experiencing challenges with the time pressures in the current EA review process structure at NRCan: Evidence from surveys and interviews indicates that the time allotted for expert review of proponent EIS’s is short. According to the survey of EA reviewers,43% of reviewers noted challenges with internally tight reviewer deadlines, and 37% noted time constraints arising from availability of experts and management understanding of the time needed in the review process. Asked what could be done to improve delivery of EA reviews, surveyed researchers identified having a better consideration of EA resourcing needs (29%), and streamlining the governance of the EA process at NRCan (23%).

Interviews with expert reviewers, coordinators, responsible authorities and members of SPI all suggested that there were opportunities for improved efficiency of how reviewer comments are approved and provided to CEAA. Most of these stakeholders perceived that in the allocated timelines to review an EIS, the balance between expert reviewer time and other aspects of administering the EA review process at NRCan could be improved. Most ESS interviewees had specific concerns that the ESS process for approving reviewer comments posed a challenge given tight timelines to review, adding that they expect even shorter timelines under the 2012 CEA Act. Others suggested that while most SPI coordinators get documentation to ESS quickly, there have been instances where documents have been delayed at SPI for over a week or not all documents were sent at once. Interviewees generally agreed that the increasing amount of time taken in moving reviewer comments from ESS to SPI and then to CEAA has significantly increased two risks. The first is that comments on Panel proceedings could be late and therefore not considered. The second and potentially more severe risk is that reviewers miss something important because they do not have enough time to properly review. Many interviewees noted that this has over the last year and a half become a major concern.

This can be compounded if ESS reviewers are not given complete information and have to make requests back to CEAA through SPI or back to the proponent through SPI and then CEAA. Everyday a federal organization has EA documents, the review time is shortening. Interviewees in ESS and SPI suggested that the process as currently constructed may not be nimble enough to respond to these needs. Most agree that it is important for managers to be aware of the responses, but the time needed for approvals between ESS and SPI is generally seen as too much. The EA comparison case study found that other departments providing expert review to EA process are revisiting their approval structures amid concerns over time constraints.

Some interviewees noted that efficiency of management review and approval has improved with recent use of project agreements and EA tracker tool: Those close to the service have said that the signing of project agreements by the ADM, directors, and DGs involved has meant that all levels of management are at least aware of the EAs NRCan is working on the deadlines. It was noted in interviews that as much as 85% of active EAs have such an agreement right now. In addition, beginning in FY 2012-13 the EA service began using an EA Tracker tool identifying when approvals are needs within the next month and what steps are coming up. ESS also now has an EA tracker for non-active, active, and future EA’s where they will need new reviewers. The current service manager also has a weekly standing meeting with the Director to discuss or resolve issues and keep up to date on the EAs including DG approvals. The implementation of a tracking tool is consistent with practices identified in the EA service comparison case study, where EC is also considering implementing similar mechanism to better manage its EA process.

4.0 Conclusions and Recommendations

4.1 Conclusions

Overall, the ESA Sub-Activity components are relevant. They support NRCan strategic objectives, legislated and policy obligations. ESS is uniquely capable to deliver the components of this Sub-Activity because it is the federal government centre of expertise on geosciences, and does not have a role in making decisions about projects or protected areas informed by its geoscience research and advice. This impartiality is an important part of the legitimacy of the EA and RA services in particular.

The evaluation raised some questions about role from an NRCan perspective on delivery of the RA service. It does appear that there will be a defined requirement in the future for ESS to provide resource assessments to the MPA process to support DFO decision-making. At the moment, NRCan provides support on a non-mandatory basis to the MPA process. The degree to which the service is able to impact decisions in the terrestrial MERA process has been facilitated by the existence of a formal process that outlines roles and responsibilities for all parties, as well as the provision of funding to ESS from Parks Canada to support conducting the resource assessments as part of the parks establishment process. By contrast, the establishment of DFO administered MPA’s does not yet have a formal agreement with clear obligations for NRCan nor a mechanism for considering the resource assessment.

The EGP program has performed well in its so far short existence. With a small A-base budget, the program has developed projects with early promising results for isolating natural from anthropogenic environmental contamination from development projects using new and innovative techniques. The research projects are relevant, focusing on areas of resource development encountered in the EA process. However, there is no explicit link to EA or policy groups at the EGP program level. Project leaders appear to have made these links as part of resourcing their research and looking for partners, but a formal process of engaging users from a program level may enhance the use of the research in informing regulators and federal authorities.

The EA and RA services have been performing well at informing stakeholders and providing due diligence reviews of proposed projects. RA studies and EA reviews are seen as credible and valued by decision-makers. The many successes of the services aside, there appear to be increasing risks to the ability of ESS to deliver. These stem mainly from issues with availability of experts and the time available to reviewers in the process. The process between SPI and ESS and the timing of EA reviewer involvement appear to need reconsideration in the face of pressures on service delivery versus ESS research program delivery.

Fundamentally, this Sub-Activity relies on the availability of strong expertise in key areas of geoscience. Similar to other science based departments, NRCan ESS is experiencing attrition in key areas of expertise which will put service delivery at risk in the future.

4.2 Draft Recommendations

Recommendation 1: Ensure Timeliness for EA and RA Services

• ESS should work with partners to assess if it is organized to deliver on future environmental assessment review requirements given the increase in complexity of EA reviews, the availability of expertise in ESS, and the time available to expert reviewers within the recent implementation of CEA Act 2012.

The volume of EA service requests has increased along with the complexity and duration of the reviews compared to the five years before the evaluation. EA reviewers report that the volume of material that they must look at to understand and make determinations about the adequacy of an environmental impact statement can be immense, especially if they are unfamiliar with the project. Early involvement of EA reviewers prior to triggering the EA process appears to be a best practice employed successfully in several projects, but there are concerns over sustaining such a practice. While timelines have so far been met, there is a risk that this could result in inadequate reviews.

Provision of a strong review by ESS experts has proven to be an essential part of the EA process. The evaluation found that due diligence reviews of proposed projects has helped to limit or prevent uncontrolled environmental impacts. ESS expert reviewers are experiencing challenges with the time pressures in the current EA review process structure at NRCan. Evidence from the survey of ESS-GSC researchers and interviews indicates that the time allotted for expert review of proponent environmental impact statements (EIS) is short.

The evaluation identified challenges among research scientists in managing priorities of EA review and program commitments. This may also be related to challenges in planning resources available for EA reviews due to the reactive nature of the service. Interviewed expert reviewers, coordinators, regulatory authorities and members of SPI all suggested that there were opportunities to improve efficiency of how expert comments are reviewed and approved within ESS and SPI and provided to CEAA. Most of these stakeholders perceived that in the allocated timelines to review an EIS, the balance between expert reviewer time and other aspects of administering the EA review process at NRCan could be improved.

The main factors facilitating the achievement of results for the EA and RA services are the expertise depth, high quality ESS-GSC researchers and the availability of those researchers. So far, ESS has been able to respond to requests for CEAA reviews and to provide researchers with expertise in mineral deposits to terrestrial MERA processes. However, some expertise areas are thinly stretched (i.e., hydrogeology and geotechnical engineering for EAs). Challenges in responding to EAs tended to be linked to constraints on expertise availability due to retirement, promotion, and competing priorities. The difficulty in permanently filling the EA Service Manager position (vacant since 2011) illustrates the unique expertise and experience needed for this role.

The amount of experience doing EA’s is also a factor in the quality and efficiency of the review. The more experienced reviewers were seen by stakeholders to more easily focus their comments on the big picture with regard to environmental effects of the project, and tend to provide a more solid review. The survey of ESS-GSC researchers indicates that the service is at risk of losing some of their expertise over the next five and ten years. In total, 49% of EA reviewers plan to retire in under ten years, including 23% within the next five years. Key expertise areas will be affected by this.

Recommendation 2: Address Timeliness Challenges in RA Service

• ESS should assess protocols for delivering resource assessments to identify and mitigate factors that have contributed to delays in some RA studies.

The evaluation found ESS contributions to the terrestrial MERA process for northern park establishment to be very well delivered in terms of quality, detail and appropriateness of the information provided. All stakeholders indicated that ESS RA studies are of very high quality and are used by federal authorities in park boundary decisions. While highly valuable, evidence from case studies and stakeholder interviews suggests that the decision-making and consultation processes that MERA studies inform were negatively impacted by the lateness of recent ESS Open File Reports. Thaidene Nene was provided three years behind schedule and consultations were impacted, and the Laurentian Channel and Hecate Strait MPA desktop studies were not delivered in time for DFO to establish the MPAs. One of the factors contributing to this appears to be the challenge in balancing researcher commitments in ESS programs and delivery of Open File reports for MERA.

Recommendation 3: Maintain and Strengthen Connections with Policy Groups

• ESS should continue to maintain and strengthen connections to federal and provincial policy and regulatory groups at the EGP program level

The goal of the EGP is to generate new geoscience that allows federal regulators and industry to make informed decisions about ecosystem risk management in resource development projects. The EGP case studies and interviews suggest that the adoption of new geoscience information and techniques to answer ecosystem effects questions is facilitated best when research has specific connections to policy groups (e.g., new guidelines adopted by CNSC) and to the EA process where decisions about the acceptability of impacts from resource development projects are made (e.g., Alberta Environment or project proponents in Mackenzie Gas Pipeline project).

Appendix A: Logic Models for the Environmental Studies and Assessments Sub-Activity