Language selection

Search

Evaluation of the Geo-Mapping for Energy and Minerals (GEM-2) Program

Evaluation Division
Audit and Evaluation Branch
Natural Resources Canada

June 2018

Table of Contents

  • Executive Summary
  • 1 Introduction
  • 2 Methodology
  • 3 Findings: Relevance
  • 4 Findings: Performance

    Tables

    Figures

    Abbreviations, acronyms and initialisms

    ADM Assistant Deputy Minister
    AGN Advisory Group of Northerners
    FAA Financial Administration Act
    F/P/T Federal ,Provincial and Territorial
    GC Government of Canada
    GEM-2 Geo-Mapping for Energy and Minerals program, phase 2
    GSC Geological Survey of Canada
    IGA Intergovernmental Geoscience Accord
    LMS Lands and Minerals Sector
    NRCan Natural Resources Canada
    NTGS Northwest Territories Geological Survey
    P/T Provincial or Territorial
    SED Strategic Evaluation Division
    SRRB Sahtú Renewable Resources Board
    TSA Task-shared Agreement

    Executive Summary

    Introduction

    This report presents the results of the evaluation of Phase II of the Geo-Mapping for Energy and Minerals (GEM-2) program. The evaluation covered GEM-2 program activities from 2013–14 to 2016–17, worth $54.3 million in program expenditures. Its main purpose was to assess the relevance and performance (i.e., effectiveness, efficiency and economy) of GEM-2, as required by the 2009 Treasury Board Policy on Evaluation and the 2016 Policy on Results.

    GEM-2 is a $100 million program delivered by the Geological Survey of Canada (GSC). Its main goal is to advance geological knowledge of the North in order to support increased exploration of natural resources and inform decisions on land use that balance conservation and responsible resource development. To achieve this goal, GEM-2 conducts geoscience research activities in collaboration with provincial/territorial (P/T) geological surveys and academic geoscience researchers. GEM-2 seeks to engage northerners and their institutions in the conduct of its activities through approaches and tools that facilitate the use of GEM data and knowledge by Northerners.

    Findings

    Relevance

    Based on the evidence, GEM-2 is generating basic geoscience research that would be unavailable otherwise. In a context where mineral and energy exploration and production remain key drivers of Canada’s northern economy, the products generated help identify areas of resource potential and help industry identify targets for exploration.  Increasingly, they are used by public agencies and Indigenous stewardship organizations for land-use planning and management. The evidence also points to a growing need for geoscience information that can be used by territorial and Indigenous authorities to address climate change impacts and inform decision-making regarding environmental planning and protection, and infrastructure planning. Key informants and survey respondents anticipated that significant gaps would remain beyond 2020.

    The impact of changing technologies and methods for collecting and interpreting geoscience data, and the varying capacity of the northern regions to deliver full geoscience research programs also support  the need for the program. There is evidence to suggest that emerging needs (e.g., related to climate change, environmental protection) may require different products (e.g., smaller-scale mapping) than are currently being produced.

    The GEM-2 program is consistent with federal government priorities regarding support for economic and social development of the North, as set out in the 2014 Budget Plan. It is a legitimate and appropriate role for the federal government to deliver the GEM-2 program, and the role is clearly delineated in the Intergovernmental Geoscience Accord (IGA). In all lines of evidence, program stakeholders confirmed that the intervention of the federal government is needed to provide the provinces/territories, academia and industry with essential and complementary scientific skills and resources. In addition, the role is consistent with the role of NRCan as per the authority provided to the Minister in the Department of Natural Resources Act and the Resources and Technical Surveys Act.

    Performance – Effectiveness

    Overall, the program made moderate progress toward the achievement of its immediate, intermediate and long-term outcomes. Evidence from all sources indicated that, at the mid-point of its program cycle, GEM was on track to add significantly to the geoscience knowledge base of northern Canada.

    Knowledge transfer activities mainly took the form of including GEM information on GEOSCAN, the online NRCan repository, and presentations by GEM-2 researchers at geoscience conferences and community engagement activities. GEOSCAN downloads by external sources increased each year, suggesting an increase in awareness over time.  However, interview evidence suggested that the GEOSCAN site was difficult to search and information was not well-organized, thereby limiting accessibility of the information. In addition, the low bandwidth in the north made it challenging for northerners and their institutions to access the geoscience information stored online and acquire data for community-use purposes. Interview and survey evidence indicated that industry particularly valued the presence of GEM-2 researchers at conferences and fora because they provided direct and preliminary access to GEM-2 research results. Documentary evidence as well as some interviewees and case study participants indicated that limitations on scientists’ travel to these events reduced the effectiveness of GEM-2 presentations at these fora.

    Applying learnings from GEM-1, the program expanded its community engagement activities in recognition of the need to engage northerners and their institutions to achieve GEM-2 objectives. All lines of evidence indicated that community engagement efforts were particularly effective in obtaining the social licence to operate and ensuring ongoing support and participation for the projects.  Interviewees and case study informants noted that these efforts were undertaken in conditions where there was limited receptivity and/or understanding of GEM-2 research work by local communities. In other areas related to community engagement, there was a marked difference in perception about the effectiveness. Sources internal to the program viewed them as largely effective. In contrast, external Interviewees and survey respondents pointed to a need for improved engagement activities aimed at increasing awareness of GEM-2 outputs and at developing tools and facilitating their use by northerners and their institutions in the longer term.

    All evidence pointed to the value of the multidisciplinary grants, and the utility of the projects themselves. Documentary evidence suggested that projects funded through the multidisciplinary grants achieved their intended project-level objectives. For example, the SRRB project successfully integrated GEM-2 and traditional Indigenous knowledge into their Atlas of the Sahtú region. However, limited data were available to assess the impact of these projects in terms of GEM-2 outcomes because the grant agreements did not ask recipients to report on the deployment of their facilitating tool. Other suggested areas of improvement included more realistic timelines, addressing northern capacity gaps and providing more program assistance to proponents to undertake planned work and maximize participation.

    Given that the lead time from initial prospecting through to the construction of new mines is 10 to 20 years in Canada’s North, and that GEM-2 products only began to be released in 2014–15, it was too early in the program lifecycle to observe any meaningful impacts of the program on GEM-2 products’ use and on the northern economy overall. However, early signs of the potential impact exist. For example, several exploration companies expressed interest in research data from the South Rae project and expressed a clear interest in following up. Also, some interviewees indicated that provincial, territorial and local governments used GEM-2 information to complete or refine their own maps for land use planning purposes, including future exploration and infrastructure development. In the long-term, internal and external interviewees, as well as survey respondents, were reasonably confident that GEM-2 would generate meaningful economic impacts. In fact, 40% (n=25) of the survey respondents expected the longer-term contribution of the GEM program to a stronger northern economy to be large or very large.

    Performance – Economy and Efficiency

    The program design provided an appropriate foundation for conducting geoscience research and reaching knowledge production outcomes. However, as designed and implemented, the program provided insufficient support to facilitating use and access to GEM-2 knowledge, particularly for northerners and their institutions. Some case study participants and interviewees suggested that the short timelines and the lack of program assistance for multidisciplinary projects limited the effectiveness of the funded projects. Evidence also suggested that program planning and delivery could leverage AGN’s strategic advice more effectively, and ongoing consultation with external stakeholders to inform project planning, design and coordination could be strengthened. Challenges in assessing the program design arose because the program’s performance measurement strategy omitted outcomes for the core geoscience research business of the program.

    Lessons learned during GEM-1 were used to improve program administration and support. Major actions taken included establishing the GEM Coordination Office and standardizing the approach to project-related community engagement. In addition, the GEM-2 program was developing a project management database, which, as conceived, could provide opportunities for cross-project sharing of information and improved sharing of resources. Further improvements to administration, particularly in obtaining permits for fieldwork, community engagement, support and coordination across projects, as well as increased collaboration across projects were the most commonly suggested means of achieving more cost-effective program delivery.

    Recommendations

    1. In light of identified and emerging needs for geoscience data and knowledge beyond 2020, it is recommended that NRCan conduct a needs assessment with a full range of external stakeholders to inform future programming in northern Canada.
    2. To build on the success of its community engagement efforts in obtaining the social license to operate in northern Canada, it is recommended that the GEM program conduct a review of its northern engagement strategy.
    3. Given the challenges with respect to accessing GEM knowledge, the development, distribution and use of tools, and the role of research scientists in the dissemination of research and knowledge transfer, it is recommended that the GEM program develop and implement a strategic plan to enhance knowledge transfer/uptake by industry and the community.
    4. Given the challenges to operating in the north it is recommended that NRCan identify a plan to address the barriers (i.e. travel, low bandwidth) to successful program delivery in northern communities Given the need for clarity in identifying and representing the intended outcomes of geoscience research programs, it is recommended that subsequent programs develop a performance measurement strategy that reflects the program design and rationale more accurately.
    5. Given the need for clarity in identifying and representing the intended outcomes of geoscience research programs, it is recommended that subsequent programs develop a performance measurement strategy that reflects the program design and rationale more accurately.

    Management Response and Action Plan

    Recommendation 1

    In light of identified and emerging needs for geoscience data and knowledge beyond 2020, it is recommended that NRCan conduct a needs assessment with a full range of external stakeholders to inform future geoscience programming in northern Canada.

    Management Response and Action Plan Responsible Manager, Branch, Sector Target Date
    Management concurs with this recommendation.

    Building on the work currently being conducted by NRCan on a national Arctic Policy Framework, which will encompass the activities for the next generation of Northern geoscience data and knowledge beyond 2020, as well as the engagement efforts undertaken as part of the Canadian Metals and Mining Plan; LMS intends to:
    • Engage the diverse group of stakeholders throughout the north to identify future geoscience needs (summer and fall 2018). The assessment will be presented at the Policy and Science Implementation Committee of NRCan for concurrence (by December 2018).
    LMS ADM December 2018

    Recommendation 2

    To build on the success of its community engagement efforts in obtaining the social license to operate in northern Canada, it is recommended that the GEM program conduct a review of its northern engagement strategy.

    Management Response and Action Plan Responsible Manager, Branch, Sector Target Date
    Management concurs with this recommendation.

    LMS will undertake a review of its engagement strategies and activities to ensure a more robust process is in place for future Northern activities. This will allow alignment with the current reconciliation process between government and communities. The review will be done through the following:
    • Through an external supplier, LMS will assess the impact of and the lessons learned from the GEM Northern engagement strategy and activities. The results of this assessment will be used to address what needs to be improved and changed for the next generation of Northern engagement activities. The findings will be presented to LMS management for future engagement activities (March 2019). Working with NRCan’s Indigenous Affairs and Reconciliation Sector, lessons learned will be shared across NRCan.
    LMS ADM March 2019

    Recommendation 3

    Given the challenges with respect to accessing GEM knowledge, the development, distribution and use of tools, and the role of research scientists in the dissemination of research and knowledge transfer, it is recommended that the GEM program develop and implement a strategic plan to enhance knowledge transfer/uptake by industry and the community.

    Management Response and Action Plan Responsible Manager, Branch, Sector Target Date
    Management concurs with this recommendation.
    • LMS had already identified the need and is engaged with NRCan Communications to develop and implement a strategic communication plan with the goal of enhancing knowledge transfer from GEM to industry and communities. This will include the GEM syntheses and community close out sessions which is an opportunity for LMS to reach out and directly share with the community the scientific results and products. The program will also continue efforts to develop and improve tools that will allow local communities to access and use GEM knowledge (ex. multidisciplinary grants).
    LMS ADM March 2020

    Recommendation 4

    Given the challenges to operating in the north it is recommended that NRCan identify a plan to address the barriers (i.e. travel, low bandwidth) to successful program delivery in northern communities

    Management Response and Action Plan Responsible Manager, Branch, Sector Target Date
    LMS Management concurs with this recommendation and in working with CMSS and other sectors will implement in future programs adapted measures that address the challenges of working in Northern Canada. 
    • As for travel, NRCan is currently in the process of reviewing the management of travel activities. By enhancing flexibility on the allocation of funding and use of travel. It is expected that these changes will provide LMS with the capacity and flexibility to adequately engage with communities and secure more representation at conferences and other venues such as open houses with stakeholders to transfer and deliver geoscience knowledge. 
    • Concerning access to GEM knowledge as a result of limited bandwidth, LMS will include measures that will address these challenges when preparing the communication plan in recommendation 2. 
    LMS ADM March 2019

    Recommendation 5
    Given the need for clarity in identifying and representing the intended outcomes of geoscience research programs, it is recommended that subsequent programs develop a performance measurement strategy that reflects the program design and rationale more accurately.

    Management Response and Action Plan Responsible Manager, Branch, Sector Target Date
    LMS has learned from this evaluation process and through the Policy on Results and will be ensuring that all future programs develop performance measurement strategies that reflect the program design and are aligned with intended outcomes and deliverables based on the mandate letter of NRCan and program funding.
    • LMS will consult with NRCan expertise on performance measurement to ensure the performance information collected is aligned with program outcomes and sufficiently demonstrates actual program results that can be used for future evaluation purposes.
    LMS ADM N/A

    Acknowledgements

    The Evaluation Team would like to thank those who contributed to the Evaluation of the Geo-mapping for Energy and Minerals Program (GEM), particularly program managers and coordinators, as well as others who provided significant insights and comments to this evaluation.

    The evaluation project was managed by Jamie Riddell with evaluation support from Michelle Brazil. Mark Pearson, Christian Asselin, David Ash and William Blois provided Senior Management oversight. Evaluation services were provided by Science-Metrix Inc.

    1.0 Introduction 

    This report presents the results of the Evaluation of the Geo-Mapping for Energy and Minerals (GEM-2) program of Natural Resources Canada (NRCan). The evaluation covered GEM-2 activities over the four-year period 2013-14 to 2016-17, representing expenditures of approximately $54.3 million. The evaluation was identified on the five-year NRCan Evaluation Plan, and examined the core issues of relevance and performance in accordance with the requirements of the 2009 Treasury Board Policy on Evaluation, the 2016 Policy on Results and the Financial Administration Act (FAA).

    The report provides a brief program profile and evaluation methodology, followed by detailed findings on each evaluation issue by evaluation question. The last section of the report presents the conclusions and recommendations to GEM-2 management.

    1.1 Program profile

    1.1.1 GEM-2 objectives

    The GEM program was launched in 2008 as a five-year, $100 million geological mapping initiative, with the purpose of addressing insufficient geological understanding of vast tracts of Canada’s northern territory. In 2013, the Government of Canada (GC) allocated an additional $100 million over seven years (2013–2020) for a second phase of GEM to provide modern regional geo-mapping of Canada’s North. The primary purpose of the GEM-2 program is to produce geological knowledge for the benefit of the minerals and energy exploration sectors, residents of the North and their institutions, as well as P/T jurisdictions with resource management responsibilities.

    GEM-2 conducts scientific field research and data collection activities through six projects aimed at addressing specific geoscience questions in the Baffin, Hudson-Ungava, Rae, Western Arctic, Mackenzie and Cordillera geologic regions. Figure 1 presents the project locations.

    Figure 1: GEM-2 Research Project Areas

    Figure 1: GEM-2 Research Project Areas
    Text version - Figure 1: GEM-2 Research Project Areas

    Figure 1 shows a map of northern Canada with coloured shapes corresponding to the six GEM-2 project areas. From west (left) to east (right), they are as follows:

    The Codillera project area is located in the southwest part of the Yukon and extending slightly into northern British Columbia. It is the smallest project area and is coloured reddish-brown.

    The MacKenzie project area is depicted to the right of the Codillera project area and is loosely correlated with the MacKenzie River area in the Northwest Territories (NWT). The project area covers the land area of western NWT, extending from the southern NWT boundary and stopping at the northern NWT land boundary. The northwestern project area extends across the NWT boundary into the Yukon. The eastern boundary extends slightly into Nunavut. The southern boundary of the project area dips slightly into northern Alberta. The shape is coloured purple.

    The Western Arctic project area is located immediately to the north of the MacKenzie project area. The Western Arctic project encompasses the ocean and land area of the northwestern portion of northern Canada extending towards Greenland. The Western Arctic project boundary extends into Nunavut, particularly in the northeast and southeast. It is coloured taupe.

    The Rae project area is shown as a narrow shape in the middle of northern Canada. Its southernmost boundary is located in northern Saskatchewan and slightly into Manitoba. It extends northwards through the eastern NWT and the central part of Nunavut to the southeastern part of Ellesmere Island near Greenland. It is coloured pink.

    The Hudson-Ungava project area corresponds roughly with Hudson Bay and its shores in the west and south. In the east, the project area extends across the Ungava Peninsula of northern Quebec to encompass the northern tip of Labrador. In the north, the project boundary extends to Baffin Island. The shape is coloured ivory.

    The Baffin project area corresponds roughly to the land area of Baffin Island. Its eastern boundary includes the sea area south along Canada’s marine boundary with Greenland to the Labrador Sea. The shape is coloured blue.

    The six projects are conducted collaboratively with provincial/territorial (P/T) geological surveys and academic geoscience researchers. Task-Shared Agreements (TSAs) define the objectives and scope of research activities within projects. Research activities undertaken by university-based academics are funded by GEM-2 research grants.

    1.1.2 Outcomes and Activities

    According to the program logic model (see Appendix A), GEM-2 has the following outcomes:

    • Immediate outcomes:
      • Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM.
      • Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM.
    • Intermediate outcome: Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making.
    • Long-term outcome: A strong northern economy through stable, long-term investment in responsible resource development.

    To achieve these outcomes, the GEM-2 program undertook activities as follows:

    1. Increasing publicly available geoscience information about Canada’s North—including the identification of areas of high potential for gold, nickel, platinum-group elements, rare metals, base metals and diamonds;
    2. Making the data, maps and knowledge available and accessible via the Internet to decision-makers ranging from government and community agencies, to industry investors and land use planners.
    3. Collecting new field data, applying modern techniques in airborne geophysics and using cutting-edge geochemistry methods.
    4. Using state-of-the-art geological science and technologies to document geological structures, create new maps, and develop geological models and regional frameworks;
    5. Engaging communities and local governments to participate in field projects; and
    6. Collaborating with Northern institutions to develop innovative approaches and tools that facilitate the use of GEM data and knowledge by Northerners.

    1.1.3 Resources

    Table 1: GEM-2 Budget 2013-14 to 2016-17
      2013–14 2014–15 2015–16 2016–17 Four-Year Total
    Operating and Maintenance  $ 6,288,600  $ 11,057,160  $ 15,342,799  $ 16,222,425  $ 48,910,984
    Capital    $   1,000,000  $   1,000,000  $       824,950  $   2,824,950
    Grants & Contributions  $    140,000  $       800,000  $       800,000  $       800,000  $   2,540,000
    Total  $ 6,428,600  $ 12,857,160  $ 17,142,799  $ 17,847,375  $ 54,275,934

    Source: GEM 2 Program

    1.1.4 Governance

    Within NRCan, the Assistant Deputy Minister (ADM), Lands and Minerals Sector (LMS) is responsible for the GEM-2 program. The Director General of the Geological Survey of Canada (GSC) is responsible for guidance and oversight of the planning, management and delivery of GEM-2. The Director, Northern Division, GSC, manages the GEM-2 program operations and delivery. Support and coordination are provided through two working committees:

    1. The GEM Steering Committee is responsible for the GEM-2 program overall including the six GEM-2 projects. It is composed of four GSC directors and supported by the GEM Coordination Office (GCO).
    2. The GEM Management Committee focuses on operational management and coordination matters. It is composed of the GEM project managers and GCO management.

    2 Methodology

    2.1 Evaluation objectives and scope

    The objective of the evaluation was to assess the relevance and performance (effectiveness, efficiency and economy) of the GEM-2 program, in accordance with the requirements of the 2009 Treasury Board Policy on Evaluation and the 2016 Policy on Results. It examined activities over the four-year period from 2013–14 to 2016–17.

    2.2 Evaluation issues and questions

    The evaluation addressed the issues of relevance and performance as follows:

    Relevance:

    • Is there an ongoing need for the GEM-2 program?
    • Is the GEM-2 program consistent with government priorities and NRCan strategic objectives?
    • Is there a legitimate, appropriate and necessary role for the federal government in the GEM-2 program? Is NRCan’s role appropriate in the context of the role of others?

    Performance (effectiveness, efficiency and economy):

    • To what extent have intended outcomes been achieved as a result of the GEM-2 program?
    • Have there been any unintended (positive or negative) outcomes?
    • What are the factors (both internal and external) that have facilitated or hindered the achievement of expected results?
    • Is the GEM-2 program the most economic and efficient means of making progress towards intended outcomes?

    An evaluation matrix outlining sub-questions, indicators and data sources was developed to guide data collection and analysis. (See Appendix B: Evaluation Matrix.)

    2.3 Data collection

    Four lines of evidence were used to collect data. Each is briefly described below.

    Document/project file review: Content analysis was conducted on over 50 documents to address all of the evaluation questions.

    Key Informant Interviews: Interviews were a critical line of evidence and addressed all evaluation issues. Twenty-two semi-structured interviews were held with external and internal stakeholders. In the report, the descriptors in the table below were used to describe interview findings.

    Table 2: Descriptors for strength of interview findings
    Ordinal scale Response count range
    Few The shared views, opinions or experiences of ≤ 25% of the respondents
    Some The shared views, opinions or experiences of 26%–50% of the respondents
    The majority The shared views, opinions or experiences of 51%–75% of the respondents
    Consensus/almost all The shared views, opinions or experiences of >75% of the respondents

    Case studies: Two case studies were conducted to examine areas of particular interest more closely. Both involved interviews with internal and external individuals directly involved, and detailed a review of related document and project files. The first case study focussed on the two multidisciplinary grants received by the Sahtú Renewable Resources Board (SRRB). Its objective was to obtain more detailed insights into the SRRB projects’ rationale and key achievements and the effectiveness of the GEM-2 program’s support. Case Study #2 examined the collaboration between GEM-2 and the Northwest Territories Geological Survey (NTGS). The purpose of this case study was to obtain detailed insights into the effectiveness and efficiency of four research projects carried out jointly with the NTGS.

    Survey: An electronic survey was conducted to assess the extent to which expected outcomes were realized. Using a purposive sampling method, a sample of 200 was selected from internal and external stakeholder groups that would potentially have an interest in resource development, exploration and land use in northern Canada. A response rate of 39% was obtained.

    In presenting the survey findings throughout the report, representatives of minerals and energy exploration / development sectors are referred to as “industry respondents.” All other external respondents (academic, provincial/territorial) are referred to as “non-industry respondents.” NRCan representatives are categorized as “internal respondents.” The survey findings presented in this report are based on the number of responses received for each group. Non-responses are excluded from the tabulations.

    The breakdown is as follows:

    Table 3: Number of Responses by Stakeholder Category
    Group n # % of total responses
    Internal 30 38.5
    External - Industry 18 23
    External – Non-industry 30 38.5
    Total 78 100

    Source:   GEM-2 survey data compiled by Science-Metrix

    2.4 Challenges and limitations

    The following challenges and limitations were encountered during the course of the evaluation:

    • Both the key informant interviews (n=22) and the survey (n=78) obtained responses from a relatively small number of program and external representatives across a diverse range of stakeholder groups. This approach provided a breadth of coverage of the various groups that participated in or may have benefited from the work of the GEM-2 program, but the depth of coverage from each participating group was not extensive. It may be that the views expressed were not representative of the total mix of views held by stakeholders.
    • In conducting the survey, the population of external stakeholders, particularly related to industry, was not possible to identify. To mitigate, a sample was constructed using a purposive sampling method. The survey targeted respondents who would more likely be familiar with GEM-2 by virtue of (1) membership in one of the northern Chamber of Mines industry associations, (2) interaction with a project or activity of GEM-2, or (3) role in participating groups from other jurisdictions. While this sampling method is suitable when the population size is unknown, it is vulnerable to self-selection bias. While the response rate of 39% was higher than expected for an online survey, the results cannot be considered statistically reliable. Therefore, the survey findings should be considered indicative only.
    • Each of the case studies provided detailed insights into specific instances of program delivery arrangements and results which may not reflect the broader program perspective. To mitigate this inherent challenge, case study evidence was mainly used to provide examples and illustrate findings from other lines of enquiry.
    • The evaluation was conducted in year four of a seven-year program. As it was still early in the GEM-2 program lifecycle, few final research and synthesis products had been released. While interviewees were familiar with GEM, they had difficulty differentiating between GEM-1 and GEM-2 outputs. While efforts were made to have interviewees clarify which GEM phase they were referencing during interviews, interviewees could not always make a clear distinction.
    • To reflect management interest in the northern engagement activities, the evaluation team collaborated with the program to collect data related to the northern engagement strategy through an external contract. However, a report was not produced, and attempts to retrieve the data collected were unsuccessful. To mitigate, the evaluation team re-examined evidence from the four lines of enquiry to identify data related to the engagement strategy. As well, one of the case studies was selected to obtain data on the implementation of the multidisciplinary grants.
    • The immediate outcomes in the program logic model were related to the knowledge transfer and outreach and engagement activities undertaken by the program. However, the bulk of the program operations and financial resources were applied to geoscience research causing a disconnect between the outcomes relative to the activities and outputs where most of the program resources were applied. To mitigate, the evaluation investigated the production of outputs in relation to planned geoscience research activities.

    3 Findings: Relevance

    3.1 Ongoing need for the program

    Finding #1 : The need for geoscience research and products to facilitate the identification and assessment of resource potential, as well as to inform land use, resource management and environmental protection decision-making continues. At the end of the current GEM program, gaps will remain with respect to geoscience knowledge of large parts of the north. A growing need for the production of maps at a lower scale (i.e., higher level of detail) and for geoscience information that could be used to address climate change impacts has also been identified. As well, increased capacity for community and regional organizations to use geoscience products is needed.

    Mineral and energy exploration and production are primary economic drivers in Canada’s North. They provide opportunities for sustainable employment and skills development, and the development and expansion of infrastructure to support wider economic growth.Footnote 1 To undertake exploration activities in the North, companies rely on pre-competitive and foundational geoscience maps and information to help them identify prospective areas of resource potential. Interview and documentary evidence indicated that companies would typically not invest in the production of such basic geoscience research,Footnote 2 and even less so in the northern context where the costs of conducting exploration are very high.Footnote 3

    The federal government’s production of “low-cost, high-resolution, and high-quality geological information in northern Canada’s vast territory and remote environment [was thus] fundamental to reducing the investment risks and uncertainties associated with resource exploration and development in the north.Footnote 4 Survey data confirmed this need. When asked to rate the need for geoscience to help identify areas of mineral or energy potential, 38 responses from external respondents were received, and the majority of them agreed. Of the 11 industry responses, 82% (n=9) rated the need for geoscience information to help identify areas of mineral or energy potential, and the need for geoscience information to reduce the cost of minerals or energy exploration activities, as “very” or “extremely” relevant to them or their organization.

    Documentary and interview evidence indicated that a significant portion of the North (i.e., above 600 north or the area north of the limit of discontinuous permafrost) had not been mapped in 2008 (when GEM-1 was launched). Following the completion of GEM-1 in 2012, an area equivalent to the total area of Quebec, Ontario and Manitoba remained to be mapped to a level appropriate to assess resource potential. Phase 2 of the GEM program was expected to fill this identified gap by 2020.Footnote 5 As well, interviewees noted that geological knowledge was a rapidly evolving area since modern geological concepts, methods and technologies (e.g., geochemistry, age-dating techniques and tectonics) created a need for new and updated maps.

    Survey and interview evidence suggested that the needs of the GEM-2 program would continue beyond 2020. Some internal and external interviewees and a few internal survey respondents believed that the program would not cover all of the strategic areas of Canada’s North by 2020. Additionally, a few interviewees and a few survey respondents stated that the need for more detailed/lower-scale maps and updated versions of earlier rounds of geological maps (including GEM-1 maps) would remain after 2020. The need for more detailed maps was expressed primarily by representatives of the three territories, and by an industry key informant interviewee representing the views of several mining companies, who stated that the “…north should be mapped completely at 1:100 000 and after that [GEM 2 staff should] pick out interesting spots and map at 1:50 000.” Given the national focus of the GEM 2 program, a few internal stakeholders questioned whether the need for smaller-scale maps was consistent with the role of the Government of Canada and NRCan.

    Documentary and interview evidence showed that updated geoscience products were necessary to inform land-use planning and management by public agencies and Indigenous stewardship organizations in order to facilitate responsible resource development. Some interviewees explained that the three northern territories had limited financial and scientific human resource capacities to produce the geoscience maps that were needed to inform decision-making in these areas. In Nunavut for example, the Canada–Nunavut Geoscience Office supported government geoscience activities. In this context, some external interviewees saw GEM-2 as an opportunity to build capacity in the North in the form of partnerships between the Geological Survey of Canada (GSC) and territorial surveys, as well as scientific training and job opportunities for Northerners.

    Survey and interview evidence pointed to a growing need for geoscience information to help address climate change impacts. Sixty-three percent of the external survey respondents rated the need for geoscience information to inform Indigenous authorities’ and governments’ decision-making regarding environmental planning and protection as “very” or “extremely” relevant. Industry respondents were more likely than non-industry respondents to point to the need at 82% (n=9) and 56% (n=15) respectively. Some interviewees stated that GEM-2 geoscience products could provide highly useful baseline knowledge to inform infrastructure planning in the North (e.g., industry platforms; local buildings, roads and ports) and to study the effects of climate change on terrain stability (e.g., effect of melting permafrost).

    The use of geoscience products by regional stewardship organizations for land-use and environmental protection purposes would also require the development of capacity aimed at combining geoscience data with traditional knowledge. The SRRB case study found that the support from the GEM-2 program facilitated initial capacity development in combining geoscience and traditional knowledge.

    While the GEM-2 program was generally perceived as contributing to meeting the capacity gap in the North, interview and survey data suggested that more could be done. Survey data indicated there were further opportunities for developing tools and methods to facilitate the use of GEM-2 products by community and regional organizations. While 44% (n=12) of the non-industry respondents reported that such tools were “very” or “extremely” important to them or their organization, only 8% (n=2) reported that the program had addressed the need for such tools to a “large extent or entirely”. A further 29% reported the program had not addressed this need at all or had addressed it only to a small extent (n=7). The program life cycle may be a factor in the lack of tools and methods to facilitate the use of GEM-2 products by community and regional organizations, since the evaluation covered the first four years of a seven-year program. However, the program will need to consider this data as it continues its community engagement efforts.

    3.2 Consistency with government priorities and NRCan strategic objectives

    Finding #2:     GEM-2 program objectives were consistent with Government of Canada (GC) priorities regarding support for economic and social development of the North, and were also aligned with NRCan’s Strategic Objective 1: Canada’s Natural Resource Sectors are Globally Competitive.

    Alignment with federal government priorities

    Initial funding for the GEM program (GEM-1) was announced by the federal government in its 2008 Budget, at which time it noted that “publicly available geoscience maps help companies target new exploration activity to the areas of highest mineral potential.Footnote 6 ; A year later, in Canada’s Northern Strategy, the GEM program was identified as a component of the action to be taken to support economic and social development in the North, one of four priorities in the strategy.Footnote 7 The policy statement noted that the program’s results would “highlight areas of mineral and petroleum potential, [and] lead to more effective private sector exploration investment and create employment opportunities in the North.Footnote 8 A subsequent publication, Achievements under Canada’s Northern Strategy, 2007–2011, saw the value of geo-mapping data as not only contributing to private sector exploration activities, but also for informing land use decision-making, such as the creation of protected areas and parks.Footnote 9

    In its 2012 report, Resource Development in Northern Canada, the House of Common’s Standing Committee on Natural Resources recommended that:

    . . . the Government of Canada continue to support geoscience programs in order to improve the development prospects of northern energy and mineral resources, and specifically renew the geo-mapping for energy and minerals (GEM) program to another productive five-year term.Footnote 10

    In response to the Standing Committee’s report, the GC noted that “renewed investment in GEM will be considered through the process of priority setting and fiscal planning of the Government.” In Budget 2014, the federal government committed a further $100 million over seven years (2013–2020) for GEM as part of a package of initiatives to support the economic and social development of the North by investing in public infrastructure.Footnote 11

    Alignment with NRCan’s strategic objectives

    The Prime Minister’s mandate letters to ministers translate the government’s overall vision into sets of specific policy objectives and challenges to be addressed as a matter of priority by each minister. In 2015, the Minister of Natural Resources’ letter identified that “getting natural resources to market” was a “core responsibility of the federal government” but had to be achieved by “addressing environmental, Indigenous Peoples’, and local concerns.” In doing so, the “overarching goal is to ensure that the resource sector remains a source of jobs, prosperity, and opportunity within the context of a world that increasingly values sustainable practices and low carbon processes.”

    During the period of the evaluation (2013–14 to 2016–17), the GEM-2 program was one of the four programs expected to contribute to the achievement of Strategic Outcome #1: Canada’s Natural Resource Sectors are Globally Competitive in the NRCan Program Alignment Architecture (PAA).Footnote 12 More specifically, the GEM-2 program was a stand-alone sub-program (1.3.3) in Program 1.3, Investment in Natural Resource SectorsFootnote 13 and was expected to advance Strategic Outcome 1 by contributing to the following result: “Natural resource sectors have increased investment.Footnote 14 The intent was to make the GEM 2 products available to industry investors, land use planners, governments and community agencies to inform resource exploration and development which would help increase exploration investment in Canada’s north.

    Besides the investment through direct program spending to increase geoscience knowledge, investment in public geoscience programs such as GEM-2, which help to identify areas for minerals and energy potential, is deemed to spur private sector investment in exploration.Footnote 15 A 2010 study conducted by the Prospectors and Developers Association of Canada (PDAC), indicated that for every $1 million invested in public geoscience, $5 million in exploration expenditure would be generated.Footnote 16 GEM-2 could therefore be expected to generate more than $500 million in economic activity within 10 to 15 years, through private sector exploration for new energy and mineral resources.Footnote 17

    3.3 Alignment with federal roles and responsibilities

    Finding #3:     Generating geoscience data and knowledge in the North is a legitimate and appropriate role for the federal government. The Department of Natural Resources Act and the Resources and Technical Surveys Act provide the legal authority for the Geological Survey of Canada (GSC) and, by extension, GEM-2. As per the Intergovernmental Geoscience Accord (IGA), the federal government is authorized to deliver research programs that are thematic and national in scope and that can collaborate with the provinces and territories on programs that have common objectives. The federal government complements rather than duplicates the work of the provinces/territories, academia and industry, and helps to fill their capacity gaps. It has done so by providing substantial funding, appropriate expertise, and by covering different areas, considering different scales and timelines and producing cross-border geoscience knowledge.

    Both the federal and provincial/territorial jurisdictions have a role to play in the provision of geological survey and geoscience services in Canada. The Department of Natural Resources ActFootnote 18 and the Resources and Technical Surveys ActFootnote 19 provide the legal authority for the Geological Survey of Canada (GSC) and, by extension, GEM-2. The respective roles and responsibilities of the F/P/T geological surveys are defined in the Intergovernmental Geoscience Accord (IGA).Footnote 20

    According to the IGA, the GSC role is “to carry out geoscience programs that are typically thematically based and national in scope and significance,” whereas the P/T surveys would carry out programs “at scales appropriate to addressing provincial or territorial geoscience priorities.” Furthermore, the P/T geological surveys are the “principal stewards of and resident authorities for public geoscience in their jurisdiction” and “contribute to a systematic description of the geology of the province or territory which may include its mineral, energy and groundwater endowment.” As a result, geoscience activities in Canada, including the production of GEM-2 geoscience products, are undertaken on a partnership basis through work which is coordinated among and between the F/P/T geological surveys. With respect to the GEM-2 program, the scope and activities of the respective parties are defined in task-shared agreements (TSAs), with each party funding and staffing their activities.

    The majority of external and internal interviewees confirmed that the federal government worked in concert with the provinces and territories, with no major overlap or duplication. They said that the GSC brought different expertise, covered different areas and considered different scales and timelines. It also produced cross-border geoscience knowledge, including looking at frontier areas, which provided provinces/territories with useful cross-jurisdiction information/learning.

    Some interviewees noted that the F/P/T roles in the production of geoscience information might continue to evolve as the devolution of federal powers and authorities progress with regard to land and resource management to territorial governments. They believed that the federal government would remain an essential player because of the scientific, human resource and financial capacity limitations of the territories as well as the provinces. For instance, an internal interviewee noted that work in the Arctic was too complex and expensive for any other jurisdiction or organization to invest in without the substantial involvement of the federal government.

    StudiesFootnote 21 have also shown that, from an economic perspective, the generation and dissemination of geoscience data and knowledge was a legitimate, appropriate and necessary role for governments. Pre-competitive government geological information typically addressed market limitations, because the production of geological maps and data would not be undertaken to the same degree by the private sector in the absence of government geological research.Footnote 22,Footnote 23 In the context of the GEM-2 program, this meant that there would be less industry interest and investment in the North, which would in turn affect the northern economy as it is largely driven by mineral and energy exploration and development, as previously discussed.

    4 Findings: Performance

    4.1 Achievement of expected outcomes

    4.1.1 Outputs

    Finding #4: Overall, outputs related to collaboration agreements and northern geoscience products were produced as planned. However, outputs related to knowledge transfer and dissemination, community-level stakeholder engagement, and tools and methods to facilitate the use of geoscience by northerners were more limited in terms of availability and effectiveness.

    1. Collaboration agreements with academic researchers and P/T geological survey partners
      Documentary, interview and case study evidence indicated that GEM-2 established and implemented collaborative agreements aligned to GEM-2 priority research themes and project areas as planned. Interviewees and case study participants involved in these agreements expressed positive views on the process that led to the signing of the agreements, whether with provincial and territorial surveys or with academia. A few interviewees suggested that the Task-Shared Agreements (TSAs) be streamlined to reflect their purpose. As well, one key informant suggested that the TSA be regularly reviewed and updated to reflect changes in priorities, and human and financial resources.

    2. Knowledge transfer to promote and disseminate northern geoscience products  
      All lines of evidence indicated that knowledge transfer consisted primarily of presentations at industry and geoscience fora and symposia, as well as the placement of knowledge products on GEOSCAN and the publication of articles in academic/industry journals. Survey evidence indicated that industry particularly valued scientists’ presentations at conferences and geoscience fora because they could discuss the implications and possibilities of the research directly. The absence of scientists responsible for the research being presented was noted as a limitation on the effectiveness of GEM research dissemination. As well, challenges accessing information on GEOSCAN were identified in the interview evidence with the suggestion that better organization of information would facilitate accessibility.

    3. Community-level stakeholder engagement
      Evidence from the various lines of evidence showed mixed viewpoints on the success of community engagement activities. On the one hand, project staff were largely successful in obtaining the social license to operate as a result of community engagement activities conducted at the beginning of research activities. Efforts made to hire local people contributed to this success. Interview and survey evidence on the other hand, showed a marked difference between internal and external viewpoints with respect to the effectiveness of the community engagement activities. Internal respondents tended to view these activities as being more effective than external sources. Interview and survey data also pointed to a need for improved engagement activities aimed at increasing awareness of GEM-2 outputs and at developing tools and facilitating their use by northerners and their institutions in the longer term.

    4. Production and release of northern geoscience products
      Principal geoscience outputs from the GEM-2 program were for the most part produced or undertaken as planned. There were some delays experienced in the fieldwork activities and the production of GEM-2 products due to the availability of scientific resources, weather and logistical factors, and the ensuing limited time to prepare final deliverables. While some activities and deliverables experienced delays, mid-year and year-end project reports indicated that these factors did not significantly impact overall project timelines.

    5. Tools and methods to facilitate the use of GEM-2 geoscience products by Northerners
      Nine multidisciplinary projects were funded to develop tools to facilitate the use of geoscience information by Northerners. All evidence pointed to the value of the multidisciplinary grants, and the utility of the projects themselves.  However, tight timelines, northern capacity gaps and insufficient program assistance limited the ability of proponents to undertake planned work and maximize participation.

    Collaboration agreements with academic researchers and P/T geological survey partners
    Geoscience research activities carried out in collaboration with P/T geological surveys and university-based researchers had to comply with the terms of the IGA and were formalized through the establishment of a Task Shared Agreement (TSA) with participating stakeholder organizations. Each party to the TSA agreed to independently fund and perform the identified activities. The resulting research findings were to be shared between the participating organizations and jointly published. Research findings of the joint projects were expected to be presented at the Geological Open Houses of the P/T geological surveys, as per commitments made in the IGA. Also, the knowledge products were made publically available at no charge through the publications websites of the GSC and its partners.

    Interview and case study evidence indicated that GEM-2 established and implemented collaborative TSAs aligned to GEM-2 priority research themes and project areas as planned. Interviewees and case study participants involved in these agreements expressed overall positive views on the process that led to the signing of the agreements.

    There were no major issues raised regarding the timeliness, complexity or fairness of the process. In the context of the NTGS case study, the TSA was generally considered to reflect the discussions between the GSC and the NTGS to set the parameters of the projects. When prompted on the matter, interviewees felt the whole process was extremely effective and simply formalized what was already occurring.

    The TSA process was the same for collaborations with other jurisdictions as it was for collaborations with academics. A few internal interviewees suggested streamlining the TSA process to include only those paragraphs that were relevant to the other party to the agreement. Several issues that applied to academia for example, did not apply to other jurisdictions. Further, an NTGS case study informant suggested that TSAs with P/T geological surveys should be regularly reviewed and updated to reflect changes in budgets, priorities, and/or the availability of staff with the required skills.

    Knowledge transfer activities to promote and disseminate northern geoscience products

    According to interview and case study evidence, the primary means of promoting GEM-2 geoscience products to potential users in industry and, to a lesser extent, government ministries, Northerners and their institutions, was via presentations by GEM-2 staff at P/T geological survey geoscience fora, industry and academic research conferences.  The NTGS case study data suggested that geoscience conferences/fora were highly regarded by industry representatives because they were able to obtain advance information on GSC activities and pending outputs. They particularly valued the opportunities for direct interaction with the researchers at these events. However, documentary evidence,Footnote 25 as well as some interviewees and case study informants, indicated that the absence of the scientists who conducted the research reduced the opportunities for interaction and thereby the effectiveness of GEM-2 presentations at these fora. The evidence indicated that travel restrictions/caps imposed by NRCan were the reason for the absences.

    In some cases, only a single or sometimes no GEM-2 scientist, was present to deliver research results of a project. For instance, external case study interviewees noted the absence from the Yellowknife Geoscience Forum of GSC scientists involved in collaborative work with the NTGS, stating that this impeded the dissemination of results. In the case of the Hudson-Ungava project, mid-year and year-end reports noted that a travel cap and ensuing limitation on domestic travel had a negative impact on the dissemination of results. In that regard, NRCan appears to have applied a restrictive interpretation of the Treasury Board Directive on Travel, Hospitality, Conference and Event Expenditures, which stipulates that departments are responsible for “limiting the number of public servants attending a conference from a department to the minimum necessary to achieve the departmental objective.Footnote 26

    All GEM-2 open file and data products were available and downloadable from the online GEOSCAN database of publications. A key informant representing the views of several exploration companies mentioned that this search engine provided too many results for a given search enquiry. This made it difficult for industry users to identify relevant information and therefore affected companies’ ability to access GEM-2 information. The informant suggested that search results should be distinguished using different categories (e.g., geological studies, thematic geological maps, large-scale geological maps, geochemical studies, environmental studies, ground water studies). The same key interviewee also suggested building on Ontario’s “highly regarded” survey tool, which exported data to Google Earth.Footnote 27

    Community-level stakeholder engagement activities

    A 2012 evaluation of the first phase of GEM recommended that “NRCan should clarify the program objective with respect to the role of community engagement and developing appropriate strategies and approaches.Footnote 28 As a result, the GSC developed a northern engagement strategy which established the framework through which GEM-2 would operate in relation to communities. In accordance with the GEM-2 Engagement Strategy, community engagement activities would occur at the beginning, throughout, and at the end of the projects being undertaken. 

    Based on documentary evidence to this point in the program cycle, community-level engagement heavily focused on explaining and getting approval for planned fieldwork, presenting the results of prior season fieldwork, and resolving issues affecting support and participation by communities and residents.Footnote 29 There was some limited evidence of engagement activities aimed at increasing and sustaining awareness of GEM-2 outputs and at facilitating their use by community or regional organizations in the longer-term. All lines of evidence indicated that the GEM program successfully created awareness of the field research activities resulting in the social licence to operate. However, in other areas related to community engagement, there was a marked difference in perception about the effectiveness of the community engagement activities. Sources internal to the program viewed them as largely effective while external sources did not.

    Survey respondents who participated in community engagement activities were asked to rate the effectiveness of these events in achieving elements of community engagement. Eleven non-industry and 17 internal responses were received. Results are presented below in Table 4. While the activities were rated as effective for the most part, marked differences in viewpoints emerged between the internal and non-industry stakeholders (no industry respondents indicated participation). 

    Overall, the groups rated the activities related to employing community members and/or companies to assist with the conduct of GEM field work as effective, at around 43%. However, 29% of the internal respondents rated the practices as “very effective”, whereas only 9.1 % of non-industry respondents did so. In considering the effectiveness of identifying and responding to community concerns and interests regarding proposed GEM activities, 65% of the internal group rated it as effective compared to 46% of non-industry respondents. Again, internal respondents were much more likely than non-industry respondents to rate it as “very effective at 24% and 9% respectively. Both groups’ views coalesced in only one case: that activities in keeping local communities and other stakeholders aware of the progress and results of GEM activities were effective (68% overall, 64% non-industry, and 71% internal).  

    Community engagement activities aimed at providing guidance and assistance to northern community members and organizations to understand and use the GEM geoscience products were more likely to be rated as effective by the non-industry respondents at 46%, versus 18% for internal respondents who were much more likely to respond “don’t know”. Of note in this instance however, is that all of the non-industry respondents who responded very ineffective were from stakeholders with the strongest ties to the community.

    Table 4:  Effectiveness of Community Engagement Activities : Survey Results
        % Response
    Q 12 # of responses Very ineffective Ineffective Effective Very effective Don't know
    /NA
    Total
    Identifying and responding to community concerns and interests regarding proposed GEM activities 28 0.0 14.3 57.1 17.9 10.7 100.0
    Non-industry 11 0.0 18.2 45.5 9.1 27.3 100.0
    Internal 17 0.0 11.8 64.7 23.5 0.0 100.0
    Employing community members and/or companies to assist with the conduct of GEM field work 28 0.0 14.3 42.9 21.4 21.4 100.0
    Non-industry 11 0.0 18.2 45.5 9.1 27.3 100.0
    Internal 17 0.0 11.8 41.2 29.4 17.6 100.0
    Keeping local communities and other stakeholders aware of the progress and results of GEM activities 28 3.6 10.7 57.1 10.7 17.9 100.0
    Non-industry 11 0.0 9.1 54.5 9.1 27.3 100.0
    Internal 17 5.9 11.8 58.8 11.8 11.8 100.0
    Providing guidance and assistance to northern community members and organizations to understand and use the northern geoscience products generated by the GEM program 28 7.1 21.4 28.6 3.6 39.3 100.0
    Non-industry 11 0.0 27.3 45.5 0.0 27.3 100.0
    Internal 17 11.8 17.6 17.6 5.9 47.1 100.0

    Source: GEM-2 survey data compiled by NRCan Evaluation Division, March 20, 2018

    Documentary evidence and some interviewees and case study participants also noted that efforts had been made by the program to ensure that communities were consulted and were aware of, and participated in, fieldwork activities. The following examples were identified:

    • The Advisory Group of Northerners (AGN) continued as an interface between the program and the communities.
    • Efforts were made by program management and scientific staff to introduce the program, discuss the research projects, and present results during community visits.
    • Some GEM-2 researchers conducted more extensive engagement activities, such as handing out reports and a non-scientific photo book to locals, providing data in GIS format to be uploaded on a community web platform, or arranging for a community to visit the research camp.
    • Local businesses and workers were hired whenever possible, although a few interviewees and project managersFootnote 30 noted that it was difficult to find local workers with the appropriate level of qualification, and local businesses were often expensive.

    A few internal and external interviewees and NTGS case study participants noted that the program operated in a context where there was sometimes a lack of interest in, receptivity to, and understanding of GEM-2 geoscience work and the long-term benefits that might result from the research. Local communities could be difficult to reach on the phone and by email, and sometimes very few people attended local events organized by GEM-2 to present the program and projects.
    Interview evidence supported the case study findings. Some internal and external interviewees felt that substantial improvements to the GEM-2 engagement strategy were needed to build stronger and more sustainable relationships with northern communities. The list below reflects suggestions provided by interviewees and survey respondents to improve engagement efforts.  

    • While program documentary evidenceFootnote 31 suggested that AGN’s recommendations were incorporated into GEM-2’s communication plan, a few interviewees felt that further efforts could be made to build on AGN’s advice to improve communications with communities and to hire local people.
    • Engagement officers should have expertise in how to engage with Indigenous communities specifically, not just engagement expertise in general (it was suggested by internal and external interviewees that GEM-2 might consider hiring Northerners to fill that role).
    • More efforts could be made toward capacity building by providing more training and job opportunities to local Indigenous people, so they are better equipped to analyze and use the geological knowledge produced by GEM-2 for land use planning and management purposes. This need was also expressed by a few external survey respondents.
    • There should be more frequent and widespread communication of the projects’ progress at the regional and local level (e.g., expand the letter distribution list to include regional associations; interact with regional and community research-oriented institutions). A few external survey respondents also identified this need.
    • A few interviewees expressed the need for GEM-2 to clearly articulate and communicate the “day-to-day” benefits for Northerners that might result from GEM-2 work. They suggested this should be done early in a project’s lifecycle to ensure community buy-in. A few external interviewees and survey respondents also advocated the production of plain language summaries of project activities that would be regularly updated.

    Production and release of northern geoscience products

    Documentary evidence indicated that data collection and analysis, and the production of related geoscience products (e.g., summary of activities’ reports, research papers, maps), were mostly on track throughout the period from 2014–15 to 2016–17.Footnote 32 Table 5 presents the release of GEM-2 products by year.

    Table 5: Release of GEM-2 Geoscience Products, 2014-15 to 2016-17
      2014–15 2015–16 2016–17
    Number of new GEM-2 products made available for public release on GEOSCAN 120 217 137
    Cumulative number of GEM-2 products made available for public release on GEOSCAN 139* 356 493

    * The cumulative number of GEM-2 products in 2014–15 includes 19 GEM-2 products published by third-party organizations that were listed on GEOSCAN prior to 2014–15.

    Source: GEM-2 data on product releases and downloads provided to Science-Metrix, July 2017

    Documentary, interview and case study evidence pointed to some delays in the fieldwork activities and the production of GEM-2 products. Delays in fieldwork activities were typically caused by challenges in the logistical arrangements and timing (for example, arrangements for fuel positioning, equipment supply), poor weather during planned fieldwork periods, availability of key scientific resources-skills, as well as difficulties in hiring students and local field support. GEM-2 project managers reported that measures were taken in subsequent years to mitigate delays in order to ensure all planned deliverables would be produced by 2020. Challenges encountered were overcome, in some instances, by the willingness of P/T geological survey partners to modify their work programs, logistical support arrangements and/or deployment of staff.Footnote 33

    According to some interviewees and case study participants, delays in the preparation of the final deliverables were very frequent, because GEM-2 scientists had to share their time between the preparation of final deliverables and competing demands from other projects. While such delays were deemed inevitable by some, a few suggested that better planning and better allocation of resources could help accelerate the release of GEM-2 products.

    Tools and methods to facilitate the use of GEM-2 geoscience products by Northerners

    In 2015–16 and 2016–17, GEM-2 funded nine multidisciplinary grants to develop innovative approaches and tools to facilitate the integration and use of GEM-2 knowledge into decision-making regarding land-use planning and resource development.Footnote 34. In relation to the overall GEM grant budget, these funds represented about $400K (or about 25% of the total grant budget over the two-year period referenced). The grant recipients were mostly Indigenous community and regional organizations, but also included the Nunavut Arctic College and the University of Western Ontario. Projects typically sought to achieve combinations of the following objectives:

     

    • Provide training and skills development in the use of GIS tools, software and data
    • Develop tools such as databases and web-based database interfaces (apps)
    • Introduce participants to GEM-2 geoscience products
    • Compile and/or integrate traditional Indigenous knowledge with GEM-2 geoscience knowledge
    • Increase understanding of geoscience data and its application to resource and land-use planning and management

    End-of-project reports submitted by the grant recipients indicated that projects aligned to the objectives listed above. For instance, the SRRB projects led to the production of interactive maps integrating GEM-2 geological, archaeological and industry activity information and traditional Indigenous knowledge about the Sahtú region. The project was designed to increase communities’ awareness of their land and contribute to more informed land use discussions and decision-making.Footnote 35 To develop such maps, the SRRB identified, compiled, and analyzed existing traditional knowledge and integrated it with the geoscience knowledge.Footnote 36 Workshops, training sessions and open houses were held to familiarize users with the geoscience information and maps that were produced.Footnote 37 Members of the Tulit’a community, one of the five Sahtú communities, also actively participated in the project planning and methodology, and in the preparation of the traditional knowledge narratives of the different Sahtú locations represented on the maps.

    Despite such positive project results, a majority of interviewees were unclear as to whether and what tools were being developed. A few internal and external interviewees expressed the need to have more tools developed and to increase their visibility/accessibility. Interviewees involved in the AGN noted that a communication plan, including suggestions on tools to facilitate the use of geoscience data, was submitted to GEM-2 by the AGN, but they could not tell if this plan was implemented. A few interviewees also mentioned that, whenever possible, GEM-2 maps should be available in GIS format, which was a standard format that could be more easily accessed and used by the communities and their institutions. A few internal interviewees explained that GEM-2 needed to increase efforts in the planning and development of tools. They suggested that such projects be planned annually and prepared at the same time as the technical geoscience products.

    Documentary and interview evidence indicated that there was a short lead time to develop proposals for the Multidisciplinary Grants program, and impacts of funded projects could have been increased if the program’s two calls for proposals had been undertaken earlier in the fiscal year.Footnote 38 Proposals for both rounds of projects, in 2015–16 and 2016–17, were due late in their respective calendar years (e.g., December 9, 2016 for the 2016–17 projects). Furthermore, projects had to be completed in the same fiscal year that the Call was issued, i.e., no later than the end of March in both 2016 and 2017.Footnote 39

    Participation in projects was reported to be lower than anticipated due to the short notice for submitting proposals, as well as competing demands on the capacity of prospective participants or weather-related constraints on travel to carry out the projects. Some of the internal interviewees suggested that many communities did not submit proposals because they could not meet the short timelines for proposing and completing projects. This evidence was supported by the documentary evidence found in proponent project reports as well as the case study.

    Two of the four 2016–17 recipients also noted that the program did not have an assigned contact available to provide assistance when needed. In that regard, the SRRB case study illustrated further how the proponent experienced two different levels of support from the GEM-2 program between 2015–16 and their second project of 2016-17.

    In 2015–16, the SRRB project benefited from administrative and technical guidance of a GEM-2 officer during the application and data collection process of the project. This enabled the proponent to complete the project within the grant timelines. The officer answered proponent questions and explained the grant terms, helping to get the project started and provide meaningful reports so as to comply with the grant terms and conditions. The officer also provided relevant information about GEM-2 research conducted in the Sahtú area, including research reports, data sets, lists of publications and the contacts of GEM-2 researchers to the SRRB proponent. However, this type of support was not provided during the second SRRB project of 2016–17.  

    A key challenge arising from the lack of support provided in 2016-17, was timely access to information about studies being conducted in the Sahtú area in 2016–17, particularly studies that were not yet available publicly. According to information from the program, the grants were administered at arms-length in order to avoid any perception of conflict of interest. Subsequent clarification of the conflict of interest guidelines with respect to multidisciplinary grant recipients was expected to alleviate this circumstance with respect to the administrative and technical support available to future multidisciplinary grant recipients.

    4.1.2 Immediate outcome: Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM

    Finding #5:     Overall, industry and Northerners were aware of and had access to GEM-2 products to a moderate extent. While GEM-2 publications and open data file were posted online immediately as available in a downloadable format, interactive engagements were noted as being more effective at increasing awareness and access. Presentations to geoscience conferences and community engagement activities were the primary means of actively disseminating and sharing new GEM-2 products, and were of particular importance to industry. Three improvements to knowledge transfer to community and to industry were suggested by interviewees and survey respondents as follows: 1) ensuring GEM-2 lead scientists’ presence during geoscience conferences generally and related to northern geoscience specifically, 2) improving the dialogue with communities and 3) increasing the availability or visibility of tools to facilitate uptake.

    All lines of evidence showed that the level of access and awareness varied by audience and level of engagement with the program, as well as method of dissemination. As noted in section 4.1.1 however, there were limits on the effectiveness of the knowledge transfer activities in the form of absence of scientists from geoscience fora, organization of knowledge products on GEOSCAN, and the need to emphasize the “day-to-day” utility of the science in a layperson’s format.

    Most interviewees believed that industry was aware of and had access to GEM-2 products primarily through presentations on project activities and emerging results . Documentary and survey evidence also supported this view.

    The majority of external interviewees stated that Northerners and their institutions were unfamiliar with GEM-2 products, and were not exposed to them. They cited reasons such as the engagement-related issues (discussed previously): a possible lack of interest in the topic; limited understanding of the relevance of geoscience information to their local or regional interests: Internet access barriers; and the limited availability or visibility of facilitating tools as contributing factors.

    The analysis of GEOSCAN data on product downloads provided a proxy measure of accessibility. GEOSCAN data showed that the program generated approximately 500 geoscience products to the end of 2016–17. Two-thirds of these products were accessed and downloaded at least once during 2016–17 (this figure was 42% in 2014–15), and the total volume of downloads increased from 2,370 in 2014–15 to 12,084 in 2016–17.

    Information based on users accessing and downloading the GEM-2 products is shown in Table 6. While imprecise as to user, the data for GEM-2 is available for 2015, 2016, and 2017 and show totals for each category. The data suggest that both the number of users (particularly the target audience of northeners) accessing the documents and the number of downloads increased over time among users external to the public sector.

    Table 6: GEOSCAN Data of Organizations and Downloads of GEM-2 Products by Category and Year, 2015 to 2017
    GESCAN Data for Geo-Mapping for Energy and Minerals (GEM-2) Products
    User Category 2015 2016 2017
    # Organizations # Downloads # Organizations # Downloads # Organizations # Downloads
    Public Sector 17 573 22 1,681 20 304
    Private Sector 37 99 48 154 51 429
    Academic Sector 48 134 57 277 74 1,862
    Generic / General Public 125 864 127 1,495 146 2,415
    Other* 20 734 31 1,637 41 7,387

    Source: As provided by GEM-2 program, February 5, 2018.
    * Includes the GEOSCAN tracking categories: Other Sectors, No Category for Now, and No ID possible. For the purposes of this discussion, they are grouped in the Other category.

    Figure 2 shows that the targets for the number of unique GEM-2 products downloaded were exceeded each year. The patterns of growth in the numbers of new products released, numbers accessed and volumes of downloads for each project area were consistent with the program-wide patterns.

    Figure 2: GEM Products Downloaded and Released, 2014 to 2017

    Figure 2: GEM Products Downloaded and Released, 2014 to 2017
    Text version

    Figure 2 is a clustered column chart depicting two series in columns side by side. Series One is the number of GEM-2 products downloaded and shown in a blue column. Directly alongside the blue column is an orange column showing series two. Series Two is the cumulative number of GEM-2 products released on GEOSCAN. The actual number of products for each series is located just above its respective column.

    The horizontal X-axis depicts both series for each of the following three years: 2014-15, 2015-16, and 2016-17. The vertical Y-axis at the left side of the figure is a range from 0 to 550 at intervals of 50. A red line across the blue column showing the number of downloads represents the target number of downloads. The target line is places at 50 in both 2014-15 and 2015-16, and at 100 in 2016-17.

    Overall, the number of products downloaded exceeded the target for each year, and the cumulative number of GEM-2 products released on GEOSCAN increased each year. On an annual basis, the numbers for each series were as follows:

    • In 2014-15, the number of GEM-2 products downloaded at least once from GEOSCAN was 59, while the cumulative number of products released on GEOSCAN was 139. [An asterix located by the number 139 denotes that 19 of the 139 products were listed on GEOSCAN prior to 2014-15 by third-part organizations and thus GEOSCAN was unable to collect download data on those.]
    • In 2015-16, the number of GEM-2 products downloaded at least once from GEOSCAN was 329, while the cumulative number of products released on GEOSCAN was 356.
    • In 2016-17, the number of GEM-2 products downloaded at least once from GEOSCAN was 197, while the cumulative number of products released on GEOSCAN was 493.
     

    Note: *19 GEM-2 products were listed on GEOSCAN prior to 2014–15. These documents were published by third-party organizations and thus GEOSCAN was unable to collect data on the download volumes.
    Sources: Numbers of product releases and downloads are from data on GEM-2 product releases and downloads provided to Science-Metrix in July 2017; targets for GEM-2 product downloads are from the NRCan Reports on Plans and Priorities for 2014–15, 2015–16 and 2016–17

    4.1.3 Immediate outcome: Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM

    Finding #6:     Funding support to community and regional organizations through multi-disciplinary grants was the principal means by which the program sought to provide northerners and their institutions with access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM. While there is some evidence that the multidisciplinary grants were helpful in creating tools to facilitate the use of the GEM knowledge, the extent to which the proponents successfully applied the tools developed could not be determined because grant recipients were not required to report on the deployment of their tools. Consequently, the GEM-2 program did not track their use. Bearing in mind that it was still early to observe meaningful impacts on access to facilitating tools, document review and case study results suggested that this outcome had been achieved to a moderate extent. 

    As noted earlier, GEM-2 aimed to increase access to tools by providing multidisciplinary grants to community and regional organizations in the North, including organizations with stewardship authority for various lands and regions in the North. The grant recipients would in turn develop the tools that would facilitate use of the geoscience data generated by GEM. Information about the multidisciplinary grants indicated that the projects were successfully completed in spite of challenges. However, limited evidence was found of the extent to which these organizations had successfully implemented these tools for several reasons.

    First, grant recipients were not required to report on the deployment of their tools.Footnote 40 Therefore, the GEM-2 program could not track their use. In this regard, GEM should apply provisions in the Policy on Transfer Payments and the Treasury Board Directive on Grants and Contributions that permit the collection of performance measurement and results information from grant recipients.Footnote 41

    Second, the majority of interviewees reported moderate access to such tools by Northerners and their institutions because few tools had been produced or were visible. Survey evidence corroborated this finding. Among external survey respondents (n=35 responses), 37% (n=13) reported that Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge to a moderate or large extent. However, 23% (n=8) of them reported that this immediate outcome had been achieved to a small extent or not at all, while an additional 40% (n=14) of them responded “did not know/not applicable.” When combined, the evidence suggested a low level of access to facilitating tools.

    Third, documentary and SRRB case study evidence also suggested that tools created during the period under evaluation were conceived to reach a limited targeted audience (i.e., one or a few communities) and would not be widely available. As such, multidisciplinary grant projects alone could not be expected to generate widespread access to facilitating tools beyond the communities they served.

    Fourth, because of the challenges described previously (e.g., short timeframes), they were used mainly by those directly involved in the development of the method or tool.Footnote 42 In the case of the SRRB projects, for instance, awareness and access to GEM data increased primarily among the participants in workshops and training sessions and among students exposed to classroom curricula that incorporated GEM data—all members of the same Sahtú community. At the time this evaluation was conducted, the SRRB had yet to expand their tool to the other four communities in the region, as intended.Footnote 43

    4.1.4 Intermediate outcome: Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making

    Long-term outcome: A strong northern economy through stable, long-term investment in responsible resource development

    Finding #7:     Opportunities for northerners, their institutions and the exploration industry to use GEM-2 products in decision-making have been limited given that GEM 2 research was on-going and the release of such products only commenced in 2014-15. The long lead times from initial prospecting to the development of new mines meant that it was premature to expect the program to have made extensive contributions to the intended intermediate and long-term outcomes. However, stakeholders were reasonably confident that GEM-2 would generate meaningful impacts on exploration and resource planning/management decision-making and on the northern economy in the longer term.

    Documentary evidence indicated that the timelines from prospecting and exploration for new mineral resources to the development and construction of new mines and supporting infrastructure varied from at least 10 years to over 20 years in Canada’s North, with initial prospecting and exploration taking approximately three to five years.Footnote 44 The generation of GEM-2 outputs only effectively commenced in 2014–15, and production was planned to continue through to the program’s end in 2020. Therefore, opportunities for the use of GEM -2 geoscience products to inform exploration planning were relatively limited as of 2017. Hence, it was premature to expect extensive evidence of the contribution by the program to the intended intermediate and long-term outcomes. Views expressed by interviewees and the NTGS case study participants confirmed these findings.

    The majority of interviewees noted that the use of GEM-2 products by northern communities and institutions was limited due to the fact that research was ongoing with final deliverables to be produced. Other factors impacting the use of GEM products by northerners and their institutions included: of low awareness of the GEM-2 program and products, limited access to GEM-2 data, limited skills in applying geoscience data and the low demand for such products by these stakeholders. The impact of facilitating tools on the use of GEM-2 data was also minimal during the period covered by this evaluation because the multidisciplinary grants program started in 2015–16, and few projects had been funded as of mid-fiscal year 2017–18. In the case of the SRRB case study, interviewees reported that most impacts on decision-making would likely be observed in the longer term, when the technical challenges experienced during the project were solved and their tool was extended to cover all Sahtú communities.

    Nevertheless, documentary, interview and case study evidence identified early examples of use of GEM-2 data by industry and provincial, territorial and local governments. Three research projects carried out in collaboration with the NTGS resulted in interest in or use of the projects’ results by exploration companies and an organization serving the exploration industry. In the context of the South Rae project, several exploration companies, including Areva and Rio Tinto, requested access to the results before they were released and expressed a clear interest in following up. Following the completion of the Banks Island project, the NTGS had informal discussions with the Inuvialuit Development Corporation, which expressed interest in the Banks Island work being pursued to assess the potential for Kimberlite Indicator Minerals on the island. One mineral exploration company also expressed strong interest in a geophysical survey conducted in the western part of the Brock Inlier map area and asked to access the data before they were published in an open file.

    Some interviewees indicated that provincial, territorial and local governments used GEM-2 information to complete or refine their own maps for land-use planning purposes, including future exploration and infrastructure development. The Northwest Territories Department of Transport used GEM-2 “published Surficial Geology maps/data … to assess all-season road alternatives to reduce the reliance on winter roads,” and the municipalities of Yellowknife and Dettah both used GEM-2 products to “map permafrost and terrain hazards related to community infrastructure and development.”

    Footnote 45

    Almost all of the internal and external interviewees were confident that GEM-2 would generate meaningful economic impacts in the medium to longer term. Some thought that this might require extending the program beyond 2020. Among the survey respondents overall, 56% (n=35) believed that the GEM-2 program would make a large or very large contribution to the use of geoscience knowledge by Northerners, their institutions and the exploration industry to inform their decision-making. The breakdown among sub-groups mirrored the overall finding, with 50% of non-industry respondents (n=12), 55% of industry respondents (n=6), and 61% of internal respondents (n=17) believing the GEM-2 program would make a large or very large contribution. 

    Of the total survey respondents, 40% (n=25) expected the longer-term contribution of the GEM program to a stronger northern economy to be large or very large. Among the sub-groups, industry respondents were more likely to expect the contribution to be large or very large at 45.5%, whereas 42% of non-industry and 36% of internal respondents held that view.

    4.2 Factors impacting the achievement of intended outcomes

    Finding #8:  Three key factors were identified as facilitating progress on the program’s intended outcomes. They are: 1) engagement with local communities prior to data collection to obtain the social licence to operate; 2) geoscience collaborations with provincial/territorial geological surveys, academic researchers, and exploration companies, and 3) logistical and administrative support from the GEM Coordination Office and the Polar Continental Shelf Program.

     

    On the other hand, impediments to achieving intended outcomes can be grouped in four key areas: 1) technical barriers in electronic access to information; 2) lack of a clearly articulated dissemination and knowledge transfer strategy; 3) capacity challenges; and 4) exploration for minerals and energy was at a low point due to depressed demand for energy and minerals and uncertainty over opportunities for energy exploration in the North. As a result, the uptake of GEM geoscience information was somewhat diminished, as least in the near term.

    Facilitating Factors

    Documentary, interview and case study evidence pointed to three key factors that contributed to achieving GEM-2 projects’ objectives. They are as follows:

    1. Well-established and effective relationships and working arrangements with P/T geological surveys that facilitated collaborative work on project activities.

      GEM-2 project managers indicated in their mid-year and year-end reports that collaborations with P/T geological surveys and academic researchers enabled the program to benefit from the complementary scientific skills of collaborators and to address gaps in the program’s own expertise in some project areas.Footnote 46 In particular, collaboration between the Quebec Geological Survey and GSC was viewed by external key informants as complementary and generally effective. Other external interviewees mentioned that the collaboration between GSC, the Yukon Geological Survey and the German Federal Institute for Geosciences and Natural Resources in the Arctic was very effective and complementary: “all those people bring different perspectives, focus on different scales and there is no competition.Footnote 47

      NTGS case study participants also mentioned that the NTGS brought a first-hand and in-depth understanding of the local geology, while the GSC contributed a regional and cross-border perspective. NTGS participants involved in South Rae, Banks Island and Brock Inlier projects emphasized the positive influence a good relationship with NTGS and their active involvement in project planning and execution had on the success of the project. For example, the involvement of the NTGS facilitated contacts with the local communities.  As well, collaborative arrangements enabled the deployment of skilled personnel that helped to mitigate skilled capacity gaps within projects.

    2. Effective engagement with local communities to obtain consent for work and to identify and respond to concerns regarding proposed data collection activities (i.e., social licence to operate).

      Interviewees and case study participants stressed the importance of frequent and adequate engagement activities with local communities to identify and address their needs. The South Rae project illustrated this type of engagement. In this project, the GSC and NTGS researchers visited the communities closest to their research field several times to present the project and its benefits for the communities and discuss collaborative opportunities (e.g., hiring of local workers).

      According to case study participants, these engagement activities exceeded expectations as previously members of the same communities were reluctant to communicate with federal government representatives. In response to the interest and support for the GEM-2 project communications efforts from one of the communities, the project team arranged to fly community members to the camp to observe the field work, and subsequently sent them project-related reports. The community then invited the GEM-2 team to present the project at their annual general meeting.

      In the context of the Brock Inlier project, a magnetotelluric survey that required helicopter flights was planned for summer 2014. However, according to mid-year and year-end project reports, this survey was postponed “due to perceived disturbance to caribou in the proposed transect area.” Additional engagement activities were carried out and the Inuvialuit Land Administration agreed to provide access to the area by helicopter for the 2015 season. According to the data from internal interviews, because this challenge and the possibility of conducting a two-year rather than a one-year project had been anticipated at project onset, it did not meaningfully affect its overall timeline or scope.

    3. Logistical and administrative support from the GEM Coordination Office and the Polar Continental Shelf Program.

      All project field season, mid-year, and year-end reports cited the importance of administrative and logistical support in enabling data collection to be carried out. The GCO role in arranging travel and accommodations, as well as in handling issues related to travel expenses, was a factor that enabled the scientists and field work personnel to focus on data collection. The logistical support provided by the Polar Continental Shelf Program (PCSP) was also mentioned in most of the reports as a factor facilitating achievement of objectives. Arrangements for on-site logistical support and services for activities such as wildlife monitoring, boat operation, fuel and equipment handling enabled the scientists to maximise short field seasons. In one project for example, the arrangement for a turn-key camp operation provided an extra seven days for data collection.

    In the annual mid-year and year end project reports, project managers also identified other supporting factors that facilitated the conduct of the GEM-2 projects. They included the quality of support from transport providers and opportunities to showcase GEM-2 results through participation in industry, science and other events which had high levels of interest from attendees.

    Hindering Factors

    Achievement of program objectives was impeded by the following factors:

    1. Technical barriers
      • Low bandwidth
        Four of the nine multidisciplinary grant project reports noted that technical barriers created inefficiencies in accessing GEM information. Within GEM, the open data was placed on GEOSCAN, but northerners’ access to it was limited by the low band-width and inefficiencies in downloading the information. For example, one of the projects reported that it could not show live simulations of the data that was available, reducing student opportunities to gain familiarity and understanding of the online placement. Low bandwidth and charges for exceeding usage would also make it challenging to communicate by email and by videoconferencing, also limiting access.

      • Inconsistent data formats
        In addition to the low bandwidth challenges, another impediment was the inconsistent formats of the datasets produced by GEM. This created challenges for integrating GEM data the tools that were being developed in two of the multidisciplinary projects. One of the proponents suggested that all data be in GIS format.

    2. Lack of a clearly articulated dissemination and knowledge transfer strategy;
      • Poor organization of information on GEOSCAN
        GEOSCAN is the online repository for all GEM knowledge products. However, as noted earlier, the database was described as being poorly organized and its search function limited.
      • Absence of researchers from northern mining fora and geological conferences
        Industry respondents in the survey identified Geoscience Fora and geological conference as the key means of acquiring the geoscience information related to the north, and particularly appreciated the face-to-face interaction. Both internal and external sources identified a mixture of travel restrictions (the application of “event” limits on number of NRCan attendees) and travel caps (expenditures exceeding planned amounts for a sector) as contributing to the absence of research scientists at such events. The issue was specifically noted by external informants in the case study evidence. In that case, GSC scientists were absent from the presentation on their research at the Yellowknife Geoscience Forum.
      • Research results insufficiently translated to community interests
        Some internal and external interviewees felt that there was a disconnect between GEM-2 and the communities. The nature of the projects and geoscience information produced was not always appropriately communicated to and easily understood by the communities, which limited the communities’ uptake of information.

    3. Capacity Challenges
      • Capacity challenges with respect to the development of tools that use GEM geoscience

        Within the program, the main capacity gap was the support available to the multi-disciplinary grants proponents, and the limitations of the multidisciplinary grants program as discussed in section 4.1.1 and 4.1.3 earlier. One interviewee suggested that a person with community engagement experience in the north would positively address the capacity issue. In addition, some interviewees and case study participants explained that the high staff turnover and staff shortfall, including project activity leaders, impeded GEM-2 production and support to stakeholders.

      • Science resource gaps
      • Documentary evidence also pointed to difficulties in addressing scientific resource gaps in certain fields due to retirements and workload demands on staff with specialized skills (e.g., palynology in the Cordillera, conodont biostratigraphy in the Hudson-Ungava), although the strength of the collaborations mitigated the overall impact on the project timelines.
        Permitting Issues
        Project managers reported that the complexity and timeframes associated with obtaining permits for fieldwork activities impeded the conduct of projects. An interviewee suggested that project officers dedicated to obtaining permits and licences be located in the regional offices rather than in Ottawa. The interviewee believed that it would be more effective, in that it would ensure permitting was not performed by project managers, which was time-consuming and not part of their responsibilities.
    4. Low point in the exploration cycle
      • Depressed world prices for mineral commodities limited the level of mineral and energy exploration that would otherwise rely on GEM geoscience products.Footnote 48 A few interviewees, including a key informant representing several exploration companies, mentioned that the mining industry experienced a very strong downturn, resulting from a decrease in the demand for mineral commodities. According to the informant, market capitalization in junior companies was reduced by 90%, resulting in their inability to raise money to do prospecting work, and therefore reducing the demand for GEM-2 geoscience. Data published by NRCan on trends in exploration and development spending and outlooks for future spending confirmed the significance of the downturn, albeit with some expectation that spending would increase in 2017.Footnote 49
      • Another external factor impacting the program was that the energy industry investments had stalled following the December 2016 moratorium on Arctic oil and gas exploration. The joint announcement by the United States and Canada made Arctic Canadian waters indefinitely off limits to future offshore Arctic oil and gas licensing, subject to a five-yearly “climate and marine science-based life-cycle assessment.Footnote 50 In addition, the moratorium on fracking in the Yukon also decreased demand for new geoscience products in the area.Footnote 51

    Other challenges included weather and transport availability issues during fieldwork in some project areas, higher costs of shipping than budgeted/expected, and problems with proper payments to students and temporary help due to the Phoenix pay system.Footnote 52

    4.3 Unintended outcomes

    Finding #9:     The evaluation found no evidence of any major positive or negative unintended outcomes.

    4.4 Demonstration of efficiency and economy

    4.4.1 Program design

    Finding #10:   The program design provided an appropriate foundation for conducting geoscience research outputs and outcomes. However, as designed and implemented, the program provided insufficient support to facilitating use and access to GEM-2 knowledge, particularly for northerners and their institutions. Some case study participants and interviewees suggested that the short timelines and the lack of program assistance for multidisciplinary projects limited the effectiveness of the funded projects. Evidence also suggested that program planning and delivery could leverage AGN’s strategic advice more effectively, and ongoing consultation with external stakeholders to inform project planning, design and coordination could be strengthened. The program’s performance measurement strategy did not clearly depict the intended program outcomes.

    As noted previously, evidence from the GEM-2 project managers’ year-end reports, interviews and case studies indicated that research project activities were generally undertaken as planned and that the generation of program outputs was largely on track. Additionally, documentary evidence indicated that .  The program’s approach to data compilation and analysis also built on well-established working arrangements with P/T geological surveys enabling the program to obtain additional leverage on its resources and funds. These findings suggest that the program design provided an appropriate foundation for achieving the geoscience objectives as presented in the program rationale.

    However, interviewees and case study participants identified possible opportunities to improve the GEM-2 program design, as follows:

    1. Consultation with external stakeholders: While the process for evaluating research activities was standard, survey, interview and case study evidence suggested that earlier and sustained collaborative program and project planning and fieldwork coordination with GEM-2 stakeholders (i.e., industry, provinces/territories, communities) would make it possible to identify and deliver projects that generate the most relevant outcomes per input for all stakeholders.

    2. Improvements to the Multidisciplinary Grants Program - Timeliness of the call for proposals and program assistance for multidisciplinary grants: The multidisciplinary grants program was welcomed by northerners and their institutions who viewed it as a positive initiative overall. However, interviewees and grant proponents recommended that GEM-2 issue the call for proposals for multidisciplinary grants earlier in the fiscal year and provide more lead time for grantees to submit their application (the decision to increase the length of 2017–18 projects to two years was a step in the right direction). The planning and delivery of multidisciplinary grant projects would also benefit from personalized and sustained assistance from program staff.

    3. Use of AGN’s advice: The Advisory Group of Northerners (AGN) was established to provide advice to GEM-2 management to maximize the benefits for the Northerners resulting from the GEM activities and knowledge. This consultative body is chaired by the LMS ADM, with the membership composed of representatives from territorial governments, the private sector and Indigenous socio-economic development organizations. In particular, the AGN identified the following five themes as the basis for its activities and provision of advice:
      1. community engagement and stakeholder identification
      2. capacity and skills development
      3. integration of knowledge and data into other decision-making processes
      4. messaging and communication with northerners
      5. environmental impactFootnote 53

    However, evidence provided mainly thorough the external interviewees indicated that the full value of the AGN was not being realised. External interviewees suggested that GEM-2 consider revisiting AGN’s mandate so its advice is better integrated in areas such as program priority setting, the selection of projects, community engagement in relation to knowledge dissemination, and how capacity could be developed in the territories. Interviewees also felt that the program could have greater transparency in how advice provided during AGN meetings was translated into decisions or actions.

    4.4.2 Measures taken to improve the efficiency of program delivery

    Finding #11:   Lessons learned during GEM-1 have been used to improve program administration and support. Major actions taken included establishing the GEM Coordination Office, standardizing the approach to project-related community engagement, and developing a project management database.

    Documentary evidence indicated that lessons learned during GEM-1 were applied to the design of GEM-2 to improve both the effectiveness and efficiency of key activities. Little evidence was found of the impact of these measures on the costs and efficiency of GEM-2 operations during the evaluation period.

    Of particular note was the creation of the GEM Coordination Office (GCO) to manage administrative activities that were common to, or supported, all project areas. The mandate of the GCO was to facilitate the implementation of GEM program activities and enable GEM staff involved in project activities to focus on operations. The GCO was responsible for coordinating the logistics for fieldwork; providing a consistent basis for, and support to, planning and budgeting processes; standardizing tracking and reporting; and providing a consistent framework and support processes for community and stakeholder engagement.Footnote 54 Internal interviewees stated that the GEM-2 coordination structure contributed to reduced operational costs through increased administrative support, monitoring and cross-project coordination. 

    Progress in obtaining the social license to operate was notable in GEM 2 due to steps taken to improve engagement activities, as illustrated in the following examples:

    • In the wake of legal challenges to planned seismic work in Lancaster Sound (under GEM-1), changes were made to community engagement processes to improve awareness of and support to GEM activities, better train program staff to engage with community representatives and increase the program’s internal capacity to undertake engagement activities. Engagement was recognized as an integral element in the planning and conduct of project activities.Footnote 55
    • According to program documentation, advice from the GEM-1 AGN informed the development and provision of awareness training to project leaders, ensuring that the content and language of materials were relevant to the communities with which the program interacted; maps and other resources were provided to communities to facilitate their understanding of planned activities and utilization of GEM products; and close-out engagement was conducted, including explaining the geoscience information produced.Footnote 56
    • Internal interview data indicated that at the time the evaluation was conducted, the GCO was leading the development and deployment of a project database designed to consolidate and track the progress of GEM-2 project activities and products and the assignment of human resources to project activities.

    4.4.3 Opportunities to achieve more cost-effective program delivery

    Finding #12:   Further improvements to administration, community engagement, support and coordination across projects, as well as increased collaboration across projects were the most commonly suggested means of achieving more cost-effective program delivery.

    A few interviewees and case study participants suggested a number of possible improvements to GEM-2 activities that could result in efficiency and cost-effectiveness gains. The three most common  suggestions are listed below.

    • Inter- and intra-project collaborations: Survey and case study evidence indicated that increased collaborations between multidisciplinary grants and GEM-2 research projects could contribute to more efficient and cost-effective project delivery through sharing of information and resources.  Survey data indicated researchers had an interest in improving intra-project collaboration, including more face-to-face meeting opportunities for project participants, e.g., workshops, field trips, mini-symposia. Sahtú case study participants suggested that GEM-2 support the creation of a network of all GEM-2 multidisciplinary grant projects to enable project participants to learn from each other’s experiences (i.e., what worked well, what did not).
    • Increased program planning and fieldwork coordination with external GEM-2 stakeholders: Such collaboration and cooperation could generate efficiencies in the field through the sharing of resources and use of common tools (e.g., dissemination platforms and activities). This would require sustained communication and consultation over the life of the projects.
    • Streamlining and further developing the coordination structure: All lines of evidence indicated that the GCO structure enabled more efficient program delivery of phase 2 of GEM. However, room for further improvement was noted in documentary evidence and by some survey respondents and interviewees with respect to timeliness and communication of project decisions, both internally and with external project collaborators and partners.  Interview and NTGS case study evidence suggested that effectiveness and efficiency would be improved if the GSC scientists received more support from GEM-2 officers for administrative (including permitting) and engagement activities. They would then be able to focus on the scientific work and the production of geoscience deliverables, which is their area of expertise.

    4.4.4. Alignment between planned and actual spending

    Finding #13:   Annual program expenditures have been closely aligned with budgets each year since the start of GEM-2. Some project areas have had significant variances in some years as a result of changes to project activities.

    Table 7 shows the breakdown of approved funding for the GEM-2 program, by Vote and year, covering the program’s seven-year time frame. These data indicated that GEM-2 activities were anticipated to progressively ramp up from $6.43 million in 2013-14 to a peak of $17.86 million in 2016–17 and 2017–18 and tapering to $12.14 million in 2019–20. This pattern of planned spending was implemented to allow a phased approach to data collection and analysis activities in each project area, and accommodate differing rates of data synthesis and publication.

    Table 7 Approved GEM funding, 2013–14 to 2019–20
      2013–14 2014–15 2015–16 2016–17 2017–18 2018–19 2019–20 TOTAL
    Vote 1                
    Personnel $1,072,400 $1,534,100 $1,557,100 $1,580,500 $1,604,200 $1,628,200 $1,577,400 $10,553,900
    O&M $5,001,720 $10,216,240 $14,474,280 $15,160,500 $15,132,060 $12,960,440 $9,810,080 $82,755,320
    EBP (@20%) $214,480 $306,820 $311,420 $316,100 $320,840 $325,640 $315,480 $2,110,780
    Sub-Total $6,288,600 $12,057,160 $16,342,800 $17,057,100 $17,057,100 $14,914,280 $11,702,960 $95,420,000
    Vote 10                
    Gs&Cs $140,000 $800,000 $800,000 $800,000 $800,000 $800,000 $440,000 $4,580,000
    Total New Funding $6,428,600 $12,857,160 $17,142,800 $17,857,100 $17,857,100 $15,714,280 $12,142,960 $100,000,000

    Note:                  O&M amounts include provisions for $6.03 million in minor capital.

    Table 8 below shows the breakdown of budgeted and actual GEM expenditures for the period examined by the evaluation. These data show that actual program expenditures have been closely aligned with annual budgeted amounts since the launch of GEM-2. The variance ranged between 2.7% and 3.4% in each of 2014–15, 2015–16 and 2016–17. Variances were more significant in some years for individual project areas as a result of reallocations between project areas to accommodate planned and unplanned changes in activities, including delays in activities due to factors such as equipment availability (for example, a planned aeromagnetic survey did not go ahead in 2016–17 due to late approval).Footnote 57

    Table 8 Budgeted and actual GEM-2 expenditures, 2013–14 to 2016–17
      2013–14 2014–15 2015–16 2016–17
      Budget Actual Budget Actual Budget Actual Budget Actual
    O&M                
    GCO $3,336,706 $2,177,423 $1,376,954 $1,265,794 $1,516,600 $1,484,809 $2,459,258 $1,515,833
    Baffin     $1,220,747 $1,166,186 $1,176,538 $1,161,877 $284,032 $263,957
    Hudson-Ungava     $773,574 $787,741 $1,525,065 $1,504,612 $1,126,146 $1,115,878
    Rae   $1,154,187 $1,806,945 $1,793,909 $2,197,663 $2,155,877 $2,982,417 $3,231,972
    Western Arctic     $582,900 $526,967 $1,444,621 $1,423,588 $1,203,784 $1,102,570
    Mackenzie     $611,913 $630,011 $1,081,092 $1,040,900 $829,866 $891,700
    Cordillera     $456,239 $380,364 $926,040 $915,472 $538,461 $881,673
    Sub-Total $3 336 706 $3 331 610 $6 829 272 $6 550 972 $9 867 619 $9 687 136 $9 423 964 $9 003 583
    Personnel $1,302,940 $1,306,294 $1,445,264 $1,451,596 $1,994,842 $1,783,910 $3,149,344 $3,109,000
    Capital $0 $0 $1,072,837 $1,072,232 $1,028,128 $1,010,354 $824,950 $807,234
    Grants & Contributions $129,350 $86,400 $800,000 $799,993 $800,000 $789,652 $800,000 $800,000
    TOTAL $4,768,995 $4,724,304 $10,147,373 $9,874,793 $13,690,589 $13,271,051 $14,198,258 $13,719,817
    Variance   0.94%   2.69%   3.06%   3.37%

    Note:                   Data excludes 20% EBP provision for Vote 1 Personnel amounts.
    Source:                GEM2 year-end reports; GEM2 program dashboards

    5 Conclusions and recommendations

    Relevance

    Based on the evidence, GEM-2 is generating basic geoscience research that would be unavailable otherwise. In a context where mineral and energy exploration and production remain key drivers of Canada’s northern economy, the products generated help identify areas of resource potential and help industry identify targets for exploration.  Increasingly, they are used by public agencies and Indigenous stewardship organizations for land-use planning and management. The evidence also points to a growing need for geoscience information that can be used by territorial and Indigenous authorities to address climate change impacts and inform decision-making regarding environmental planning and protection, and infrastructure planning. Key informants and survey respondents anticipated that significant gaps would remain beyond 2020.

    The impact of changing technologies and methods for collecting and interpreting geoscience data, and the varying capacity of the northern regions to deliver full geoscience research programs also support the need for the program. There is evidence to suggest that emerging needs (e.g., related to climate change, environmental protection) may require different products (e.g., smaller-scale mapping) than are currently being produced.

    The GEM-2 program is consistent with federal government priorities regarding support for economic and social development of the North, as set out in the 2014 Budget Plan. It is a legitimate and appropriate role for the federal government to deliver the GEM-2 program, and the role is clearly delineated in the Intergovernmental Geoscience Accord (IGA). In all lines of evidence, program stakeholders confirmed that the intervention of the federal government is needed to provide the provinces/territories, academia and industry with essential and complementary scientific skills and resources. In addition, the role is consistent with the role of NRCan as per the authority provided to the Minister in the Department of Natural Resources Act and the Resources and Technical Surveys Act.

    Performance – Effectiveness

    Overall, the program made moderate progress toward the achievement of its immediate, intermediate and long-term outcomes. Evidence from all sources indicated that, at the mid-point of its program cycle, GEM was on track to add significantly to the geoscience knowledge base of northern Canada.

    Knowledge transfer activities mainly took the form of including GEM information on GEOSCAN, the online NRCan repository, and presentations by GEM-2 researchers at geoscience conferences and community engagement activities. GEOSCAN downloads by external sources increased each year, suggesting an increase in awareness over time. However, interview evidence suggested that the GEOSCAN site was difficult to search and information was not well-organized, thereby limiting accessibility of the information. In addition, the low bandwidth in the north made it challenging for northerners and their institutions to access the geoscience information stored online and acquire data for community-use purposes. Interview and survey evidence indicated that industry particularly valued the presence of GEM-2 researchers at conferences and fora because they provided direct and preliminary access to GEM-2 research results. Documentary evidence as well as some interviewees and case study participants indicated that limitations on scientists’ travel to these events reduced the effectiveness of GEM-2 presentations at these fora.

    Applying learnings from GEM-1, the program expanded its community engagement activities in recognition of the need to engage northerners and their institutions to achieve GEM-2 objectives. All lines of evidence indicated that community engagement efforts were particularly effective in obtaining the social licence to operate and ensuring ongoing support and participation for the projects. Interviewees and case study informants noted that these efforts were undertaken in conditions where there was limited receptivity and/or understanding of GEM-2 research work by local communities. In other areas related to community engagement, there was a marked difference in perception about the effectiveness. Sources internal to the program viewed them as largely effective. In contrast, external Interviewees and survey respondents pointed to a need for improved engagement activities aimed at increasing awareness of GEM-2 outputs and at developing tools and facilitating their use by northerners and their institutions in the longer term.

    All evidence pointed to the value of the multidisciplinary grants, and the utility of the projects themselves. Documentary evidence suggested that projects funded through the multidisciplinary grants achieved their intended project-level objectives. For example, the SRRB project successfully integrated GEM-2 and traditional Indigenous knowledge into their Atlas of the Sahtú region. However, limited data were available to assess the impact of these projects in terms of GEM-2 outcomes because the grant agreements did not ask recipients to report on the deployment of their facilitating tool.

    Given that the lead time from initial prospecting through to the construction of new mines is 10 to 20 years in Canada’s North, and that GEM-2 products only began to be released in 2014–15, it was too early in the program lifecycle to observe any meaningful impacts of the program on GEM-2 products’ use and on the northern economy overall. However, early signs of the potential impact exist. For example, several exploration companies expressed interest in research data from the South Rae project and expressed a clear interest in following up. Also, some interviewees indicated that provincial, territorial and local governments used GEM-2 information to complete or refine their own maps for land use planning purposes, including future exploration and infrastructure development. In the long-term, internal and external interviewees, as well as survey respondents, were reasonably confident that GEM-2 would generate meaningful economic impacts. In fact, 40% (n=25) of the survey respondents expected the longer-term contribution of the GEM program to a stronger northern economy to be large or very large

    Performance – Economy and Efficiency

    The program design provided an appropriate foundation for conducting geoscience research and reaching knowledge production outcomes. However, as designed and implemented, the program provided insufficient support to facilitating use and access to GEM-2 knowledge, particularly for northerners and their institutions. Some case study participants and interviewees suggested that the short timelines and the lack of program assistance for multidisciplinary projects limited the effectiveness of the funded projects. Evidence also suggested that program planning and delivery could leverage AGN’s strategic advice more effectively, and ongoing consultation with external stakeholders to inform project planning, design and coordination could be strengthened. Challenges in assessing the program design arose because the program’s performance measurement strategy omitted outcomes for the core geoscience research business of the program.

    Lessons learned during GEM-1 were used to improve program administration and support. Major actions taken included establishing the GEM Coordination Office and standardizing the approach to project-related community engagement. In addition, the GEM-2 program was developing a project management database, which, as conceived, could provide opportunities for cross-project sharing of information and improved sharing of resources. Further improvements to administration, particularly in obtaining permits for fieldwork, community engagement, support and coordination across projects, as well as increased collaboration across projects were the most commonly suggested means of achieving more cost-effective program delivery.

    Recommendations

    1. In light of identified and emerging needs for geoscience data and knowledge beyond 2020, it is recommended that NRCan conduct a needs assessment with a full range of external stakeholders to inform future programming in northern Canada.

    2. To build on the success of its community engagement efforts in obtaining the social license to operate in northern Canada, it is recommended that the GEM program conduct a review of its northern engagement strategy.

    3. Given the challenges with respect to accessing GEM knowledge, the development, distribution and use of tools, and the role of research scientists in the dissemination of research and knowledge transfer, it is recommended that the GEM program develop and implement a strategic plan to enhance knowledge transfer/uptake by industry and the community.

    4. Given the challenges to operating in the north it is recommended that NRCan identify a plan to address the barriers (i.e. travel, low bandwidth) to successful program delivery in northern communities

    5. Given the need for clarity in identifying and representing the intended outcomes of geoscience research programs, it is recommended that subsequent programs develop a performance measurement strategy that reflects the program design and rationale more accurately.

    Appendix A: Program logic model

    Appendix A: Program logic model

    Appendix B: Evaluation matrix

    Evaluation Issues Evaluation Sub-Issues Indicators Lines of Enquiry Data Sources
    Relevance        

    1.  Is there an ongoing need for the GEM-2 program?

    1.1  Who are the principal stakeholders and what are their needs?

    1.2  Are there new or emerging needs?

     

     

     

    1.3  What factors are driving demand?

    • Principal stakeholder groups

       
    • Current needs for geoscience data, maps, tools and knowledge (“geoscience information”) among stakeholders
    • Gaps in program coverage of needs
    • New or emerging needs

     

    • Factors driving demand
    • Document review
    • Policy and program documents presenting the rationale for GEM-2, identifying principal stakeholders and evidence of their needs
    • Key informant interviews

     

    • GEM program management
    • Stakeholders:

    -  Indigenous organizations with land use oversight roles-       Community organizations in the north
    P/T geological surveys
    - Academic researchers
    -  Industry

    • Survey of program staff and stakeholders
    • GEM program management
    • Stakeholders:
    • Indigenous organizations with land use oversight roles
    • Community organizations in the north
    • P/T geological surveys
    • P/T government ministries with northern land use responsibilities
    • Academic researchers
    • Industry
    • Case studies (topics to be decided with SED and EAC)

    Depending on case study topics:

    • Engagement planning and conduct document/file reviews
    • Project/activity-specific documents/files
    • Interviews with program and stakeholder participants
    2.  Is the GEM-2 program consistent with government priorities and NRCan Strategic Objectives?

     

    2.1 To what degree are the GEM-2 intended outcomes aligned with federal government priorities?
    2.2 To what degree are the GEM-2 intended outcomes aligned with NRCan’s strategic objectives?

    • Alignment/consistency of GEM-2 activities, outputs and outcomes with:
    • Federal priorities
    • Applicable NRCan Strategic Outcome(s)
    • Document review

     

    • Policy documents – Speeches from the Throne, Budget Plans, policy statements, Ministers of Mines and Energy statements/papers, PAA, DPRs, RPPs/Departmental Plan

     

    3.  Is there a legitimate, appropriate and necessary role for the federal government in the Sub- Program? Is NRCan’s role appropriate in the context of the role of others?

    3.1 Is the generation of geoscience data and knowledge in the North a legitimate and appropriate role for the federal government?

    3.2 Are there other organizations that produce geoscience products similar to those produced by the GEM program? If so, is NRCan’s role appropriate in the context of the role of others?

    • Program mandate aligned with federal government jurisdiction


       
    • Degree of overlap or complementarity of roles of other federal departments, other levels of government and/or industry
    • Alignment of GEM-2 activities with NRCan’s core capabilities and functions
    • Document review
    • Policy/program documents – Inter-governmental Geoscience Accord (IAC), Natural Resources Act and the Resources and Technical Surveys Act, program descriptions – other similar/ equivalent programs
    • Key informant interviews
    • GEM program management
    • Stakeholders:
    • Indigenous organizations with land use oversight roles
    • Community organizations in the north
    • P/T geological surveys
    • P/T government ministries with northern land use responsibilities
    • Academic researchers
    • Industry

    Performance - Effectiveness, Efficiency and Economy

         

    4.  To what extent have intended outcomes been achieved as a result of the GEM-2 program?

    4.1 To what extent have the program’s principal outputs been produced or undertaken as planned?

    • Collaboration agreements with academic researchers and P/T geological survey partners (TSAs and grants)
    • Production and release of northern geoscience products versus plan
    • Knowledge transfer activities to promote and disseminate northern geoscience products versus plan
    • Community-level stakeholder engagement activities versus plan
    • Tools and methods to facilitate use of GEM geoscience products by northerners versus plan
    • Document review
    • Program documents and data relating to planned and actual production of geoscience products, rates of product downloads, peer reviewed publications and citation rates, and planned and actual community engagement activities.
    • Key informant interviews
    • GEM program management

     

    • Survey of program staff and stakeholders
    • GEM program management

     

    • Case studies (topics to be decided with SED and EAC)
    • Documents/files relating to production and dissemination of selected GEM-2 products and tools, and delivery of selected engagement activities
    • Interviews with applicable program and/or stakeholder participants

     

    4.2 To what extent is the program achieving intended outcomes:

     

     

     

    Immediate outcomes:
    - Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM

    - Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM

    Intermediate outcome:

    • Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making

     

    Long-term outcome:

    • A strong northern economy through stable, long term investment in responsible resource development

     

    • Rates of awareness of GEM-2 data and knowledge products among primary beneficiaries
    • Satisfaction with GEM-2 data and knowledge products

     

    • Evidence of usage of new tools for accessing regional geoscience data
    • Satisfaction levels with new tools

     

    • Estimates/evidence of usage and/or anticipated future usage of GEM products by stakeholders
    • Satisfaction levels with usefulness of GEM-2 data and knowledge products

     

     

    • Qualitative assessments or examples or examples of economic and/or social benefits of future resource developments in the North

     

    • Document review

     

    • Program documents relating to the results of awareness building, product uptake, provision on training and support for usage of tools, application of GEM-2 products and tools by beneficiaries
    • Key informant interviews
    • GEM program management
    • Stakeholders:

    -  Indigenous organizations with land use oversight roles
    -  Community organizations in the north
    -  P/T geological surveys
    -  Academic researchers
    -  Industry
    associations/ companies

    • Survey of program staff and stakeholders
    • GEM program management
    • Stakeholders:
    • Indigenous organizations with land use oversight roles
    • Community organizations in the north
    • P/T geological surveys
    • P/T government ministries with northern land use responsibilities
    • Academic researchers
    • Industry
    •  
    • Case studies (topics to be decided with SED and EAC)
    • Documents/files relating to uptake and use of GEM-2 products and tools
    • Data on download volumes, participation in training and product use/ promotion events
    • Interviews with applicable program and/or stakeholder participants

    5.  What factors – internal and/or external to the program – have enabled or limited the achievement of the intended outcomes?

    5.1 What factors have enabled or limited the achievement of the a) intended program outputs b) intended GEM-2 outcomes?

    • Impacts on performance of the enabling and/or limiting factors
    • Document review
    • Program documents – program/ project/activity reports, mid-year and year-end review reports
    • Key informant interviews
    • GEM program management
    • Stakeholders:

    -  Indigenous organizations with land use oversight roles
    -  Community organizations in the north
    -  P/T geological surveys
    -  Academic researchers
    -  Industry

    • Survey of program staff and stakeholders
    • GEM program management
    • Stakeholders:
    • Indigenous organizations with land use oversight roles
    • Community organizations in the north
    • P/T geological surveys
    • P/T government ministries with northern land use responsibilities
    • Academic researchers
    • Industry
    •  
    • Case studies (topics to be decided with SED and EAC)
    • Applicable project/activity documents and performance data
    • Interviews with applicable program and/or stakeholder participants

    6.  Have there been any unintended (positive or negative) outcomes?

    6.1 Have there been unintended outcomes that can reasonably be attributed to GEM-2 activities?

    6.2 What actions have been taken in response to any unintended outcomes?

    • Unintended outcomes partially or fully attributable to program activities

     

    • Actions taken to address unintended outcomes (e.g., integrate positive outcomes into the program’s design or mitigate impact of unintended negative outcomes)
    • Document review
    • Program documents – program/ project/activity reports, mid-year and year-end review reports
    • Key informant interviews
    • GEM program management
    • Stakeholders:

    -  Indigenous organizations with land use oversight roles
    -  Community organizations in the north
    -  P/T geological surveys
    -  Academic researchers
    -  Industry

    • Survey of program staff and stakeholders
    • GEM program management
    • Stakeholders:
    • Indigenous organizations with land use oversight roles
    • Community organizations in the north
    • P/T geological surveys
    • P/T government ministries with northern land use responsibilities
    • Academic researchers
    • Industry
    •  
    • Case studies (topics to be decided with SED and EAC)
    • Applicable project/activity documents and performance data
    • Interviews with applicable program and/or stakeholder participants

    7.  Is the GEM-2 program the most economic and efficient means of making progress toward the intended outcomes?

    7.1 Is the program’s design appropriate for achieving the intended outcomes?





    7.2 What measures have been taken to improve the efficiency of program delivery in recent years? How significant were the efficiency benefits or cost savings?

    7.3 Are there opportunities to achieve more cost-effective delivery of program activities and production of outputs?

    7.4 Has actual spending been in line with planned spending?

    • Program design issues encountered in program delivery
    • Levels of satisfaction with the way the program was designed and is being delivered
    • Efficiency and/or cost saving initiatives implemented and their impacts


       
    • Potential opportunities to improve program efficiency and/or economy

     

    • Comparison of actual versus planned spending
    • Explanations for variances and/or budget adjustments
    • Document review
    • Program documents – program/ project/activity reports, mid-year and year-end review reports
    • Key informant interviews

     

    • GEM program management
    • Stakeholders:

    -  Indigenous organizations with land use oversight roles
    -  Community organizations in the north
    -  P/T geological surveys
    -  Academic researchers
    -  Industry

     

    • Survey of program staff and stakeholders
    • GEM program management
    • Stakeholders:
    • Indigenous organizations with land use oversight roles
    • Community organizations in the north
    • P/T geological surveys
    • P/T government ministries with northern land use responsibilities
    • Academic researchers
    • Industry

    Appendix C: Case study approach

    Case study of the collaboration between the Geological Survey of Canada (GSC) and the Northwest Territories Geological Survey (NTGS)

    Background

    Provincial and territorial (P/T) geological survey organizations and university geoscience researchers collaborate with the program on the planning and performance of geoscience research activities, and the production and dissemination of the resulting geoscience products. The scope and activities of the respective parties are defined in Task Share Agreements (TSAs), with each party funding and staffing their respective activities. Geoscience products generated may be published jointly, published in scientific journals or made available for free public dissemination via publication websites. The resulting research findings – new geoscience knowledge and data – are shared between the participating organizations and jointly published. Joint results of collaborative work with P/T geological surveys are presented at P/T “geological open houses”, as per commitments made in the IGA as well as other outreach events, such as industry and university-sponsored conferences and workshops.  

    The Northwest Territories Geological Survey (NTGS) is a division of the Department of Industry, Tourism and Investment, Government of Northwest Territories. The mission of the NTGS is to:

    (P)rovide geoscience information about the Northwest Territories (NWT) to inform decisions by governments, industry, and the public concerning the responsible development of mineral and energy resources, use of the land, and environmental stewardship. The NTGS also plays an important role in delivering geoscience education and outreach services. Other than strategic advice provided to the Minister, geoscience information provided by the NTGS is freely available to all.Footnote 58

    The GSC and NTGS currently had two TSAs in place at the time of this evaluation and two that expired on March 31, 2017. The applicable GEM-2 project areas and objectives for each of these are listed below. 

    Table 1 GEM-2 Task Shared Agreements between the GSC and NTGS
    Project Areas Activity Areas Term Objectives
    Rae South Rae June 2016 – March 2018 South Rae Geoscience Bedrock and Quaternary mapping activity: Update the geoscience knowledge of southeastern Northwest Territories to a modern level at regional scale
    Western Arctic Banks Island June 2016 – March 2017 Undertake kimberlite indicator mineral (KIM) sampling and analyses in the Nelson head – Rufus River are, southeastern Banks Island
    Mackenzie Mackenzie-Selwyn Geotransect June 2017 – March 2020 To place Proterozoic to Cenozoic strata of the Mackenzie Platform, Selwyn Basin, and adjacent regions into a modern tectono-stratigraphic and metallogenic framework.
    Mackenzie Shield to Selwyn Geotransect – Brock Inlier/ Darnley Bay June 2015 – March 2017 Improve geological maps of the area by increasing the density of geological field observations in NTS 97A and 97D. Existing geology maps of the area are out of date and of low resolution. Data relevant to the (GEM-2 Mackenzie Project) will be obtained from detailed documentation of outcrop exposures of lower Paleozoic and Proterozoic rocks along Hornaday River Canyon. A side benefit of the project will be to provide more current interpretive information for the area within the adjacent Tuktut Nogait National Park.

    Source:               TSAs for collaboration between the GSC and NTGS in the Rae, Western Arctic and Mackenzie project areas.
    Planned deliverables under these TSAs are combinations of:

    • scientific / Yellowknife Geoscience Forum presentations;
    • current research summaries and external journal papers;
    • open File reports and data (Geoscan and/or NTGS); and
    • geological maps

    Objective of the NTGS case study

    The objective of this case study was to obtain more detailed insights into the effectiveness and efficiency of the GEM-2 program’s collaborative arrangements with the NTGS. Conduct of the case study provided information that contributed to the assessment of the following evaluation issues and questions (see also the logic model in Appendix A and evaluation matrix in Appendix B for details).

    Table 2            Case study coverage of the evaluation issues and questions

    Evaluation Issues & Questions
    Sub-Questions Case Study Focus

    1. Is there an ongoing need for the GEM-2 program?

    1.1 Who are the principal stakeholders and what are their needs?

    • Nature of the NTGS and GSC’s expectations of their collaborative arrangements

    1.2 Are there new or emerging needs?

    • Reasons for the selection of the agreed activities and deliverables

    1.3 What factors are driving demand?

    3. Is there a legitimate, appropriate and necessary role for the federal government in the GEM-2 program? Is NRCan’s role appropriate in the context of the role of others?

    3.2 Are there other organizations that produce geoscience products similar to those produced by the GEM program? If so, is NRCan’s role appropriate in the context of the role of others?

    • Rationale for the selection of the respective activities and deliverables of the GSC and NTGS
    • Potential for other organizations to undertake these activities and generate the associated deliverables

    4. To what extent have intended outcomes been achieved as a result of the GEM-2 program?

    4.1 To what extent have the program’s principal outputs been produced or undertaken as planned?

    • Extent to which agreed activities and deliverables have been realized or are on track for realization

    4.2 To what extent is the program achieving intended outcomes:
    Immediate outcomes:
    -   Industry and Northerners are aware of and have access to new regional geoscience data and knowledge generated by GEM
    -   Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM

    • Contribution (or expected contribution) of the deliverables to the achievement of:
    • GEM-2 immediate outcomes
    • NTGS goals and strategies

    5. What factors – internal and/or external to the program – have enabled or limited the achievement of the intended outcomes?

    5.1 What factors have enabled or limited the achievement of the:
    a) intended program outputs?
    b) intended GEM-2 outcomes?

    • Extent to which processes involved in establishing TSAs, undertaking planned activities, and generating agreed deliverables met or exceeded expectations, or could have been improved.

    6. Have there been any unintended (positive or negative) outcomes?

    6.1 Have there been unintended outcomes that can reasonably be attributed to GEM-2 activities?

    • Nature of any unintended or unexpected results
    • Responses to these results

    6.2 What actions have been taken in response to any unintended outcomes?

    7. Is the GEM-2 program the most economic and efficient means of making progress toward the intended outcomes?

    7.1 Is the program’s design appropriate for achieving the intended outcomes?

    • Lessons learned and/or best practices that could be applied to future collaborations with P/T geological surveys

    7.3 Are there opportunities to achieve more cost-effective delivery of program activities and production of outputs?

    • Opportunities to improve the efficiency (cost-effectiveness and/or timeliness) of activities and generation of deliverables

    7.4 Has actual spending been in line with planned spending?

    • Planned and actual data on respective levels of effort and expenditures on agreed activities.

    Case study of the Sahtú Renewable Resources Board (SRRB) Multidisciplinary Grants

    Background

    Two of the nine grants were awarded to the Ɂehdzo Got’ı̨nę Gots’ę́ Nákedı / Sahtú Renewable Resources Board (SRRB). The SRRB was created by the Sahtú Dene and Métis Comprehensive Land Claim Agreement to be the main instrument of wildlife management within the Sahtú Settlement Area. Its Missions is to:

    • Protect, conserve, and manage, in a cooperative spirit, all renewable resources within the Sahtu Settlement Area in a sustainable manner to meet or exceed the needs of the public today and in the future for generations to come, and
    • Assist communities with the management of wildlife and habitat for the benefit of the people of the Sahtu Region.Footnote 59

    In carrying out its mandate, the SRRB works closely with Renewable Resources Councils (Ɂehdzo Got’ı̨nę) in the five communities of the Sahtú Region. The role of the Renewable Resources Councils is to “encourage and promote local involvement in conservation, harvesting studies, research and wildlife management in the community”.Footnote 60

    The SRRB’s 2015/16 project:

    enabled the research, acquisition, cataloguing, preparation, and integration of a sampling of available GEM GIS spatial and documentary datasets into the online Atlas: a series of data conversions (kml, shapefile, pdfs) of the GEM data was conducted to make the GEM data technically compatible with existing traditional knowledge (TK) GIS datasets and attribute information. […] The resulting product now enables users to access TK concurrently with existing GEM studies and independently research energy and mine data within the context of their own cultural knowledge and environment. Further, decision-makers, educators, and students are now able to consider cross-cultural interpretations of geoscience and TK and consider mechanisms to tailor future Atlas versions with geoscience knowledge for their specific needs.Footnote 61

    The “Atlas” referred to in the above statement is the Sahtú Atlas, a compilation of maps and supporting descriptions of the region’s heritage, natural history (climate, ecology, wildlife and resource distribution), and state of resources and development.  

    In carrying out the project, the proponents conducted over 20 workshop presentation/training sessions and community open houses that demonstrated the geoscience datasets and trained participants to use the geoscience component of the Atlas. Participants in these sessions spanned “primary and high school students and teachers, adult learners and adult learning instructors/program heads, Renewable Resource Councils (RRCs), co-management boards, elders, and community members”.Footnote 62

    A second project in 2016–17 was used to:

    explore and showcase Dene and Métis relationships with their traditional territory via the interplay between indigenous/community knowledge and western science (geology), using material culture (archaeology) as a bridge between the two. Current resource interest and exploration activity datasets (fossil fuels, minerals) are also used to present contemporary land interest and use, as are classroom curriculum aids designed to assist teachers and educators promote student knowledge and reflection.Footnote 63

    GIS data sets were acquired and incorporated into the Sahtú Atlas. Landscape and community knowledge was then compiled using research sessions with Tulit’a community knowledge holders and high school students (Tulit’a (formerly Fort Norman) is a hamlet in the Sahtú region). Locations of stories obtained through this line of enquiry were integrated into the digital atlas, with text in both English and Slavey. Special attention was paid to capturing “information related to geological and resource activity considerations, as well as areas of key importance to the Tulıt́ ’a people.”Footnote 64

    The end-product from the project formed the basis for the preparation of materials for use in the local high school’s curricula, consisting of “explanatory documents, reference material, and discussion questions aimed for both independent study and group discussion”. The project proponents also indicated in their end-of-grant report that they intend to showcase the project results more widely within the Sahtú region and potentially seek to develop similar integrated content for other communities in the region (Délınę̨, Fort Good Hope, Colville Lake, and Norman Wells).Footnote 65

    Objective of the SRRB case study

    The objective of the present case study was to obtain more-detailed insights into the SRRB projects’ relevance and key achievements and the effectiveness of the GEM-2 program’s support.

    Table 3 provides more details on the evaluation issues and questions that were addressed by this case study (see also the logic model in Appendix A and evaluation matrix in Appendix B for details).

    Table 3 Case study coverage of the evaluation issues and questions
    Evaluation Issues & Questions Sub-Questions Case Study Focus

    1. Is there an ongoing need for the GEM-2 program?

    1.1 Who are the principal stakeholders and what are their needs?

    • Nature of the SRRB’s and GSC’s expectations of the support for tool development and deployment

    1.2 Are there new or emerging needs?

    • Extent to which GEM geoscience products can be utilized
    • Types of decisions where GEM geoscience products can make a difference

    1.3 What factors are driving demand?

    3. Is there a legitimate, appropriate and necessary role for the federal government in the GEM-2 program? Is NRCan’s role appropriate in the context of the role of others?

    3.2 Are there other organizations that produce geoscience products similar to those produced by the GEM program? If so, is NRCan’s role appropriate in the context of the role of others?

    • Extent to which other organizations provide support to integrate and use geoscience products with other GIS-based data
    • Degree of complementarity or duplication

    4. To what extent have intended outcomes been achieved as a result of the GEM-2 program?

    4.1 To what extent have the program’s principal outputs been produced or undertaken as planned?

    • Extent to which activities and commitments by the SRRB and GEM program have been realized or are on track for realization

    4.2 To what extent is the program achieving intended outcomes:
    Immediate outcomes:
    -   Northerners and their institutions have access to tools that facilitate use of new regional geoscience data and knowledge generated by GEM
    Intermediate outcome:

    • Northerners, their institutions and exploration industry use modern regional geoscience knowledge to inform their decision-making
    • Contribution (or expected contribution) of GEM-supported tools to the achievement of:
    • GEM-2 immediate and intermediate outcomes
    • SRRB’s goals

    5. What factors – internal and/or external to the program – have enabled or limited the achievement of the intended outcomes?

    5.1 What factors have enabled or limited the achievement of the:
    a) intended program outputs?
    b) intended GEM-2 outcomes?

    • Extent to which the GEM program’s approach to supporting tool development and deployment has aided, or limited, the achievement of the GEM programs and the SRRB’s expected outputs and outcomes

    7. Is the GEM-2 program the most economic and efficient means of making progress toward the intended outcomes?

    7.1 Is the program’s design appropriate for achieving the intended outcomes?

    • Lessons learned and/or best practices that could be applied to improve the effectiveness of GEM support for tool development and deployment

Report a problem on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: