Table of Contents
- Executive Summary
- 1.0 Introduction and Background
- 2.0 Evaluation Approach and Methodology
- 3.0 Evaluation Findings
- 3.1 Relevance
- 3.2 Performance - Effectiveness
- 3.3 Performance - Efficiency
- 4.0 Conclusions and Recommendations
- Annex A: Natural Resources Canada Program Alignment Architecture 2013-2014
- Annex B: Evaluation Matrix
The Evaluation Project Team would like to thank NRCan’s Minerals and Metals Sector and in particular CanmetMINING for its collaboration during the evaluation. We would also like to acknowledge the contribution of all those individuals who provided insights and comments crucial to this evaluation.
The Evaluation Project Team was under the direction of Jennifer Hollington, Head of Evaluation, as well as Gavin Lemieux, Director of Evaluation. The evaluation was managed by Jamie Riddell, with the support of Jarret Boon and Michelle Brazil. Evaluation services were provided by TDV Global Inc.
This report presents the results of the evaluation of the Green Mining Initiative (GMI), Sub-Program 2.2.2 in Natural Resources Canada's (NRCan) Program Alignment Architecture,Footnote 1 which was undertaken from August 2013 to September 2014.
The mining industry faces ongoing challenges to reduce its environmental footprint. In response, NRCan’s Minerals and Metals Sector (MMS) established the GMI to accelerate the research, development and deployment of green mining technologies and practices; enhance innovation and productivity in mining; improve environmental performance; and improve the competitiveness of the Canadian mining industry thereby positioning Canada's mining sector as the global leader in green mining technologies and practices.
From 2009-10 to 2012-13, all projects carried out under the Sub-Program fell under four main pillars: 1) Footprint reduction; 2) Innovation in mine waste management; 3) Mine closure and rehabilitation; and 4) Ecosystem risk management. During 2012-13, work conducted under the Green Mining Initiative at CanmetMINING was reorganized into five priorities: 1) Clean Water; 2) Best Practices in Mining Environment Management; 3) Energy Efficiency in Mining and Milling; 4) Northern Mineral Development; and 5) Mining Extraction Innovation.
The objective of the evaluation was to assess the relevance and performance (efficiency, economy, effectiveness) of the activities within the Green Mining Initiative (GMI), according to the Treasury Board’s Policy on Evaluation (2009). The evaluation covered the period 2009-10 to 2012-13 and included all activities of the GMI. The program expenditures over the four-year period were budgeted at $37.5 million.
Evaluation Issues and Methodology
The evaluation was completed as per the requirement of the Treasury Board Policy on Evaluation (2009) that all direct program spending be assessed every five years. The evaluation objective was to assess the relevance and performance (effectiveness, efficiency and economy) of the GMI and to make recommendations for consideration by NRCan program and senior management.
The study consisted of document, file and literature review, industry survey, 27 in-depth interviews with key stakeholders as well as eight case studies.
Need for the Program
Program documentation and interview findings illustrate that there is a well-defined need and role for the federal government in accelerating the research, development and deployment of green mining technologies and practices, as well as in reducing the environmental impact or footprint associated with mining. GMI is well aligned to government and NRCan objectives and priorities. NRCan has the mandate and expertise to deliver on the objectives of the GMI.
Industry expressed a need for research-centric green mining support from the Government of Canada (GC). GMI’s pillar/priority activity areas are well aligned with industry needs, as validated by the annual Energy and Mines Ministers’ Conference (EMMC) and the Green Mining Initiative Advisory Committee (GMIAC). GMIAC meets quarterly to discuss priorities and the research agenda and to obtain updates. GMI is pivotal in addressing a range of economic, environmental, social, regulatory and science needs associated with green mining and, in turn, is instrumental in achieving a competitive advantage for the country.
Alignment with Government Priorities and NRCan Strategic Objectives
GMI is consistent with the 2010, 2011 and 2013 Speeches from the Throne, Canada’s Economic Action Plan as provided in Budget 2010 and 2011, and is linked to the Northern Strategy. It supports federal government priorities including strengthening Canada’s research advantage, fostering commercialization and business innovation, protecting Canada’s natural environment, supporting Aboriginal and Northern communities, and strengthening Canada’s international engagement.
GMI is aligned to NRCan’s Strategic Outcome 2 (Natural Resource Sectors and Consumers are Environmentally Responsible), Program 2.2: Technological Innovation; and Sub-Program 2.2.2: Green Mining.
Alignment with Federal Roles and Responsibilities
There is a legitimate and appropriate role for the federal government’s support of such green mining activities. The mandate of the GMI accrues from the role of the Minister of Natural Resources as defined under the Department of Natural Resources Act, S.C. 1994.
The government has a legitimate role in leading an initiative that enables mines to be greener and in leading projects that can have tangible environmental and economic benefits for Canadians. The GMI is the only federal initiative in place that looks at the entire mining lifecycle through four comprehensive pillars: Footprint Reduction; Mine Waste Management; Mine Closure and Rehabilitation; and Ecosystem Risk Management. These comprehensive pillars and five related priorities focus on reducing the amount of waste produced in mining and minimizing its impact on the environment at all stages of the mine lifecycle.
Achievement of Expected Outcomes
The evaluation found that the program had effectively produced a wide range of project outputs and had been successful in achieving some of its immediate outcomes. In particular, the vast number of projects had contributed to an expanded knowledge base of innovative mining technologies. Examples include Green Mines Green Energy, Mine Environment Neutral Drainage (MEND), bio-waste for remediation of mine tailings, ventilation technologies and alternative vehicles for use underground.
At the intermediate outcomes level, the adoption of green mining innovation technologies leading to commercialization is difficult to assess as this is dependent on economic realities, time needed to adopt technologies (e.g., purchasing, retrofitting) and reluctance by industry to be first in implementing new technologies (i.e. to disproportionately assume the risk for the industry). While green mining practices are viewed as a desirable outcome, the advantage in adopting innovative technologies lies in being “first to be second” when the technology is proven as there is then a smoother regulatory approval process, fewer costs, and lower associated risks.
The achievement of the ultimate outcomes of “an innovative, environmentally and socially conscious industry” and a “positive public perception and image of the industry” has yet to be measured and would likely need a baseline assessment to better understand the scope and reach of the GMI.
Demonstration of Economy and Efficiency
GMI successfully leverages NRCan funds at the project level thereby increasing the resources available to complete project activities. Interview and case study evidence indicates that the practice is central to the program structure, and continues at ratios estimated to be 6:1 overall, and even as high as 7:1 in the case of the Green Mines Green Energy project. However, while successful leveraging occurs at the project level, the absence of clearly articulated research priorities may have resulted in missed opportunities for further leveraging.
The program could take advantage of opportunities and efficiencies with other federal entities (e.g., Environment Canada (EC), Industry Canada (IC), Foreign Affairs, Trade and Development Canada (DFATD)) and regional development agencies (e.g. Atlantic Canada Opportunities Agency (ACOA), Western Economic Diversification Canada (WD)) that might have a vested interest in the success of the GMI or the achievement of joint objectives (e.g., trade, innovation).
While significant effort has been devoted to collecting broader CanmetMINING information, within the key tool for performance data (i.e., the “dashboard”), there does not appear to be a standard for data collection, performance information or metrics across GMI projects. Lack of consistency in project data made it challenging to identify performance at the GMI program level. This resulted in a limited assessment of GMI during the evaluation. An improved performance measurement and reporting strategy would allow CanmetMINING to identify GMI specific-efficiency and economy outcomes in the future.
Recommendations and Management Response
|Recommendations||Management Response||Responsible Official/Sector (Target Date)|
|1. Opportunities exist to engage other federal entities as the research conducted under the GMI extends across several federal initiatives. To take full advantage of this, it is recommended that GMI conduct a review of federal programs to identify potential efficiencies and areas for collaboration.||Accepted. A formal environmental scan of GMI will be done by CanmetMINING that will include other government departments such as the National Research Council, specialized funds like Sustainable Development Technology Canada, academia and Canada Mining Innovation Council.||ADM Minerals and Metals Sector
|2. Given the evidence highlighting the need to attract new partners through clearly articulated program priorities, it is recommended that GMI establish an overarching internal green mining research agenda that will guide long-term priority setting.||Accepted. CanmetMINING will develop a broad research agenda in consultation with stakeholders to guide priority settings and project decisions for internal projects. We are also developing a Green Mining narrative to help frame and communicate the initiative.||ADM Minerals and Metals Sector
|3. Given that no GMI-specific performance measurement and reporting strategy exists, it is recommended that a performance measurement and reporting strategy for the GMI be developed by the Program to systematically track progress on outputs and outcomes and ensure consistent performance data are collected across projects. This PMRS should include a logic model and associated narrative.||Accepted. CanmetMINING will put in place a performance measurement and reporting strategy by integrating a logic model for its GMI operations. It will encompass indicators of outputs and outcomes.||ADM Minerals and Metals Sector
1.0 Introduction and Background
The following is an evaluation of Natural Resources Canada's (NRCan) Sub-program 2.2.2, Green Mining. The evaluation covers the period 2009-10 to 2012-13 and $37.5 million in NRCan expenditures over the four-year period.
The evaluation’s objective was to assess the relevance and performance (effectiveness, efficiency and economy) of the GMI and to make recommendations for consideration by NRCan program and senior management.
1.2.1 Overview of the Green Mining Initiative
The Green Mining Initiative (GMI) was announced in 2009Footnote 2, and endorsed at the Energy and Mines Ministers’ Conference by federal, provincial and territorial mines ministers. The GMI brought together green mining Science and Technology (S&T) and Research and Development (R&D) under the Green Mining Sub-Program, work which had already been undertaken with various stakeholders in the mining sector by the Mines and Minerals Sector (MMS).
Building on the efforts of the National Orphaned and Abandoned Mines Initiative and Green Mines Green Energy project (GMGE), the Green Mining Initiative was designed to develop collaborative approaches and share relevant scientific knowledge in key research areas across the mining sector. The Director General (DG) of CanmetMINING was responsible for the Green Mining Initiative program and ensuring that the objectives were met.
The “GMI targets the development of innovative energy-efficient technologies required for mining to leave behind only clean water, rehabilitated landscapes and healthy ecosystems. It aims to improve the mining sector's environmental performance, to promote innovation in mining and to position Canada's mining sector as the global leader in green mining technologies and practices.”Footnote 3
In recent years, NRCan commissioned two studies, Delphi Group (2010)Footnote 4 and Meyers Norris Penny (2011)Footnote 5, to determine the state of green mining technology in the Canadian mining sector and to identify barriers to the uptake of green mining innovations. While both reports noted that Government regulations were not barriersFootnote 6 unto themselves to the adoption of green mining technologies and practices, some derivative issues could in essence become barriers. As stated in the Delphi Group ReportFootnote 7 “the process of assessing and approving environmental performance is resulting in challenges for regulators, communities and mining companies, including:
- Lack of information/evidence on the performance of a novel or new mining technology increases perceptions of risk;
- Capacity to substantiate/verify the performance of new technologies for regulators and communities; and
- The ability to define the stringency of environmental standards to address local eco-system concerns.”
The Meyers Norris Penny (2011)Footnote 8 report further noted that there was significant risk aversion to implementing new technologies in the mining sector due to associated costs and the competitive nature of the companies. The GMI was designed to address the issues raised in this report by undertaking pre-competitive research and development and pursuing pilot and demonstration projects to help reduce the risks associated with new technologies. In addition, GMI works to address key competitive challenges in the mining industry by helping to maintain and expand Canada’s mineral resource advantage and leading role in the global arena.
Canada’s key competitors in the global mining sector are investing in green mining technologies as resource-rich countries place a greater emphasis on environmental performance to allow investment in mining and project development. Reducing the environmental impacts of mining and related activities continues to be one of the most significant and important challenges for the sector. The industry’s public image is closely tied to its environmental performance, and the industry recognizes it must continue to improve to ensure it remains responsible, competitive and productive, in Canada and internationally.
At the Energy and Mines Ministers’ Conference in September 2009, the GMI was endorsed by federal, provincial, and territorial mine ministers.Footnote 9 Furthermore, the Canada Mining Innovation Council (a consortium of industry, academic and government leaders) endorsed the GMI and included it as part of its national Mining Research and Innovation Strategy.
The objectives for the first five years of the Initiative were to:
- Improve environmental performance within the mining sector;
- Promote innovation within the mining sector through new technologies; and
- Realize market opportunities in green mining technologies.
From 2009-10 to 2012-13, all projects carried out under the Sub-Program fell under four main pillars, namely:
- Footprint reduction;
- Innovation in mine waste management;
- Mine closure and rehabilitation; and
- Ecosystem risk management.
Within the 2012-13 timeframe, work conducted under the Green Mining Initiative at CanmetMINING was reorganized into five priorities:
- Clean Water;
- Best Practices in Mining Environment Management;
- Energy Efficiency in Mining and Milling;
- Northern Mineral Development; and
- Mining Extraction Innovation.
More specifically, GMI is intended to:
- Target the development of innovative energy-efficient technologies required for mining so that only clean water, rehabilitated landscapes and healthy ecosystems remain. It aimed to improve the mining sector's environmental performance, to promote innovation in mining and to position Canada's mining sector as the global leader in green mining technologies and practices;
- Realize market opportunities in green mining technologies;
- Undertake pre-competitive research and development and pursue pilot and demonstration projects to help reduce the risks associated with new technologies; and
- Raise industry confidence to implement new green technologies to reduce the environmental impacts of mining and related activities.
1.2.2 The PAA
In NRCan’s Program Alignment Architecture (PAA), the Green Mining Initiative contributes to NRCan’s Strategic Outcome (SO) 2: Natural Resource Sectors and Consumers are Environmentally Responsible, Program 2.2: Technological Innovation; and Sub-Program 2.2.2: Green Mining.
1.2.3 GMI Logic Model
The logic modelFootnote 10 depicted in Figure 1 demonstrates the causal relationships between the GMI Program’s activities and the intended immediate, intermediate, and final outcomes. This logic model was developed by the evaluation team and the program during the initial phase of the evaluation.
Figure 1: Green Mining Initiative Logic Model
Figure 1: Logic Model
Figure 1 presents a logic model of the Green Mining Initiative (GMI). The logic model is organized in a series of activities, outputs, and outcomes that are linked in a chain of results. The two Activities have directly associated Outputs that are targeted to those identified in the Reach column. On the figure, Reach refers to the target audiences of the GMI Outputs and is depicted between the Outputs and Immediate Outcomes.
Completing the Activities and producing the Outputs is expected to result in the achievement of the Immediate Outcomes. Achieving the Immediate Outcomes is expected to contribute to the achievement of Intermediate Outcomes, which in turn will contribute to the achievement of the Ultimate Outcomes.
|Activities||1. Funding of S&T activities in the areas of:
||2. Funding of Green Mining Stewardship Activities in the areas of:
The current logic model acknowledges two broad ranges of activities underpinning the GMI namely, science and technology (S&T) activities as well as activities related to green mining stewardship. The first set of activities comprises S&T activities in the following six key areas :
- Northern Mineral Development: focuses on mine waste management, ecosystem protection and remediation of contaminated sites and mine design for the north. This segment of activities is focused on northern sites and is a pivotal Government of Canada priority.
- Mining Extraction Innovation: activities and projects focus on improving and modernizing underground mining processes at the core of mining operations.
- Energy Efficiency in Mining and Milling: activities and projects focus on novel technologies for improving mining efficiency and energy usage as well as reducing energy consumption in mining operations. Energy usage is a key cost driver in mining operations.
- Clean Water: activities and projects focus on assessing and addressing environmental risks of mine-related contaminants as well as fostering the development of technologies for sustainable use of water in the mining industry.
- Critical and Strategic Minerals Processing: activities and projects focus on bioleaching, a process of using bacteria instead of chemicals to dissolve metals. When compared with chemical methods of separation, bioleaching is slower but cleaner.
- Mining Environmental Management: activities and projects focus on topics such as mine waste management and reclamation as well as regulatory input and technology transfer.
The outputs generated from this activity set range from the development of new scientific knowledge, new technologies, new processes, pilots, demonstrations, best practices, validated approaches right through to scientific publications. These outputs are aligned with the needs identified by partners and stakeholders that are also derived from the Mines Ministers’ Conference and Green Mining Committee as well as departmental priorities.
The second activity set, green mining stewardship activities, aim at mobilizing stakeholder engagement, gleaning a better understanding of regulatory affairs, as well as providing a platform for advocacy for green mining technologies, research and development. The outputs achieved cover those ranging from regulatory input, to the creation of stakeholder engagement mechanisms, communications and knowledge exchange/transfer and Information products.
The delivery of these activities and the sustained delivery of outputs are intended to culminate in the achievement of the following immediate outcomes:
- Expanded knowledge base of innovative mining technologies and practices that eliminate or reduce environmental impacts and risks;
- Expanded understanding of regulatory, stakeholder and environmental concerns and requirements; and
- Expanded understanding of Canada’s Green Mining Initiative.
From an intermediate outcomes perspective, the program aims to foster the industry’s adoption of green mining innovation technologies as well as enhance stakeholders’ understanding of Canada’s commitment to environmentally sound and economically viable mining practices. These aspects are important to achieving the ultimate goal of creating/reinforcing an innovative, environmentally and socially responsible industry as well as achieving better public perception of the mining industry.
1.2.4 Resources at the Program and Component Level
Annually, GMI financial resources include approximately $8.5MFootnote 11 in NRCan A-Base funding, of which 75% is dedicated to salaries, and an additional $2-3MFootnote 12 in cost recovery from projects. Over the four-year evaluation period, GMI resources totalled $37.5 million. Resources were also leveraged from partners (industry, academia, NGOs) on a project-by-project basis through in-kind and cash contributions (e.g., $47.2MFootnote 13 in leveraged project funding for 2011-2012).
1.2.5 Program Structure and Governance
GMI activities are carried out as part of the Canada Centre for Mineral and Energy Technology (CANMET) by approximately 70 FTEs based in the CanmetMINING facilities in Ottawa, Sudbury, and Val-d’Or. Project leads are responsible for the day-to-day management of their specific activities. Four Program Managers in CanmetMINING are responsible for selecting projects depending upon which Pillar/Priorities these projects fall. The Project Managers administer these projects and report to the Director General (DG) of CanmetMINING. The DG directs the overall Green Mining activity, is responsible for strategic planning, and ensures that the objectives are met.
In addition, the GMI also receives oversight and guidance from two committees:
- Green Mining Initiative Advisory Committee (GMIAC): Co-chaired by the Canada Mining Innovation CouncilFootnote 14 (CMIC) and NRCan, this committee provides overall guidance for the Green Mining Initiative in order to ensure that it maintains its relevance to key stakeholders such as the provinces and territories, industry and academia; and
- Federal Provincial Territorial Committee on Green Mining: This committee is tasked to identify further science and technology needs which can contribute to an effective and efficient regulatory system; and to identify opportunities to accelerate the development and implementation of green mining technologies.
The following figure illustrates the organizational chart for GMI:
Figure 2: Green Mining Organizational ChartFootnote 15
Source: GMI Evaluation Assessment, 2013
Figure 2: Green Mining Organizational Chart
Figure 2 is an organizational chart showing a three-level management structure for GMI.
The top level is the Director General who has five direct reports. The five direct reports are as follows:
- Science Policy Analyst – no direct reports
- Executive Assistant – no direct reports
- Regional Director, Green Mining Innovation – Extraction
- Director, Green Mining Innovation – Processing
- Director, Business and Operations
The following four positions report to the Regional Director, Green Mining Innovation – Extraction:
- Program and Site Manager – Ground Control (Bells Corner)
- Program Manager – Alternative Energy Technologies (Sudbury)
- Program and Site Manager – Mining Extraction Technologies (Val d’Or)
- Program and Site Manager – Radioactive Waste Stabilization (Limebank)
The following three positions report to the Director, Green Mining Innovation – Processing:
- Program Manager, Processing and Mine Waste Management
- Program Manager, Mine Closure and Ecosystem Risk Management
- Program Manager, Analytical Services and CCRMP
The following four positions report to the Director, Business and Operations:
- Manager, Business Affairs and Communication
- Manager, Finance and Administration, Health, Safety and Security
- Manager, Quality Systems and Engineering and Technical Services
- Manager, Information Systems
Source: GMI Evaluation Assessment, 2013
2.0 Evaluation Approach and Methodology
2.1 Evaluation Scope and Objectives
The Treasury Board Policy on Evaluation (2009) requires that Government of Canada (GC) evaluations examine a program’s relevance and performance (effectiveness, efficiency, and economy). In 2013-14, the NRCan Strategic Evaluation Division (SED) undertook an evaluation of GMI. The evaluation covered the activities and $37.5 million in NRCan expenditures over the four-year period 2009-10 to 2012-13, and considered the following five evaluation issues:
- Is there a continued need for the Program? (Assessment of the extent to which the Program continues to address a demonstrable need and is responsive to the needs of Canadians).
- Alignment with Government of Canada (GC) priorities. (Assessment of the linkages between program objectives and federal government priorities and departmental strategic outcomes).
- Alignment with federal roles and responsibilities. (Assessment of the role and responsibilities for the federal government in delivering the GMI).
- Achievement of Expected Outcomes. (Assessment of progress towards immediate, intermediate and ultimate outcomes).
- Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes.
This report provides the findings for each of these defined evaluation issues and questions. In Section 3, findings are organized and discussed by evaluation question and associated corresponding sub-questions. Conclusions with recommendations for the GMI are presented at the end of the report in Section 4.
2.2 Evaluation Methodologies and Limitations
For this evaluation, data collection consisted of gathering information from the following sources:
- document/file/literature review;
- online survey of GMI stakeholders (industry, academia, provinces/territories); and
- case studies (8).
Summary technical reports were developed for each line of evidence. Further, a logic model was developed by the evaluation team in conjunction with the Program.
Over 786 files in 16 folders were identified in the GMI SharePoint directoryFootnote 16 during the Evaluation AssessmentFootnote 17 and the development of the Evaluation Methodology report. For the purposes of the Document Review, additional documents were identified by the program and through an extensive web-search. Together, these documents provided a comprehensive record of the Initiative including background information, terms of reference, financial information, internal and external reports and presentations, program updates and other relevant documents. Table 1 summarizes the nature of the most salientFootnote 18 types of documents per type:
|Type of Document||Number|
|100 – Governance and Administration Data
Includes program history and evolution, program structure, reporting structure, program accountability, organizational chart, performance management framework (indicators – data collection, etc.), logic model, policy documents and legislation, other related issues
|200 – Financial and Resource Data (Including Materiality)
Includes average annual expenditures, expenditure breakdown per activity type (pillars or priorities), average staffing levels (FTEs), industry participation – cost sharing (financial or in-kind) per pillar or by priority), general funding information, other related issues
|300 – Program Data
Includes GMI projects per pillar, GMI projects per priority focus, general information on projects
|400 – Stewardship Data Includes advocacy, regulatory affairs, stakeholder engagement||12|
|500 – Program Output Data (Key Performance Indicators (KPI)
Includes new scientific knowledge, new technologies, new processes, pilots, scientific publications, validated approaches, demonstrations, best practices, dashboard
|600 – Stewardship Output Data
Includes regulatory input, stakeholder engagement mechanisms, communications, knowledge exchange and transfer, information products
|700 – Outcome Data
Includes expanded knowledge base of innovative mining technologies and practices that eliminate or reduce environmental impacts and risks, expanded understanding of regulatory, stakeholder and environmental concerns and requirements, expanded understanding of Canada’s Green Mining Initiative
|800 – Performance Measurement Data (Efficiency and Economy)
Includes business cases, action plans, critical path
|900 – Other Documents
Includes published articles on green mining, annual reports and financial reports, literature, supplemental information of interest to the evaluation (e.g., evaluation assessment of the GMI, methodology report/detailed work plan for the evaluation of the GMI)
With the help of program staff and project/program records, a total of 41 interviewees were identified across four categories: NRCan Senior Management, NRCan Program Researcher, Other (Federal) Government Departments and Provinces/Territories, and Industry representatives from the mining industry. Twenty-seven interviews were completed for the evaluation. Table 2 outlines the categories of interview groupings that were used for analysis:
|Grouping||Description||Number of Interviewees|
|NRCan Senior Management||Includes NRCan Senior and Program Managers||4|
|NRCan Program Researchers||Includes Program Managers and Research Scientists||12Footnote 19|
|Other Government Departments and Provinces/Territories||Includes OGDs (Environment Canada, Sustainable Development Technology Canada, and the PCO); and Provinces(Alberta and Québec)||6|
|Industry||Includes representatives of the mining industry and CMIC||5|
Online Survey of GMI Stakeholders
The purpose of the stakeholder survey was to reach GMI stakeholders to assess the relevance of the Program to their needs and priorities, as well as its effectiveness in addressing those needs. The 20 survey questions were developed by the evaluation team with input from the Evaluation Advisory Committee (EAC). The survey consisted of a rating scale and/or Likert questions with some open questions to collect individual respondent’s comments.
Based on consultations with program staff, it was determined that the best target population for the survey was the membership of the Canadian Mining Innovation Council. Due to confidentiality concerns, CMIC was required to administer the distribution of the survey. The bilingual survey was launched with the support of the Director General (DG), CanmetMINING on Monday, March 17, 2014 and closed on Thursday, March 27, 2014. According to the CMIC’s Executive Director & CEO, there were 568Footnote 20 recipients with a 24.5% open rate (i.e., 138 people opened the email). Of this open rate, 18 people out of 568 recipients completed the questionnaire for a completion rate of 3%. Survey results are considered indicative only given the low response rate.
Case studies were chosen by the evaluation team, in conjunction with the Program based on the types of partner organizations, project delivery dates, funding and resourcing levels and degree of adoption. To identify case studies, projects were assessed against the criteria in order to determine those most suitable to inform the evaluation. Eight were selected as appropriate. The case studies were conducted using interviews and document reviews. Findings were identified per case study and then combined for analysis to create one technical report to inform the evaluation.
Based on the criteria, the selected case study topics were:
- Case Study #1: Alternative Binder
- Case Study #2: Ventilation-on-Demand
- Case Study #3: Transformation & Dissolution Protocol
- Case Study #4: Green Mines Green Energy
- Case Study #5: Hybrid Scoop
- Case Study #6: Inter-Governmental Collaboration
- Case Study #7: MEND and NOAMI
- Case Study #8: ECOEII Hybrid Filtration System
The case studies were conducted using interviews and document review as lines of evidence.
At the beginning of the evaluation, the GMI lacked an agreed upon logic model. The evaluation team, with the support of Program stakeholders, developed a draft logic model for the Initiative to identify program outputs, and immediate, intermediate and ultimate outcomes for use in the evaluation.
2.2.2 Limitations and Mitigation Strategies
The evaluation methodology provided the evidence required to conclude on all issues and questions. However, there were some limitations associated with the evaluation as outlined in Table 3.
|The industry survey response rate was much lower than anticipated, affecting the reliability of the survey results. Accordingly, survey evidence is indicative only. It is possible that the low response rate may be attributable to a few factors, namely:
||A reminder was sent to industry survey recipients midway through the survey period.
The number of interviews from the industry stakeholder group was increased in order to mitigate this limitation; however, no additional industry stakeholders were available to provide input.
|At the program level there was limited financial/HR data (e.g., financial breakdown beyond planned vs. actual spending, complete business case financials, HR levels and apportionment to the various pillars/priorities, etc.).||The Program was asked to address high level gaps. In other instances, where it was believed that the level of effort and time required to manipulate financial systemsFootnote 21 to achieve the stated aim would have been too onerous, a recommendation was made to the Program to refine its tracking and reporting in the future.|
|A performance management framework was not in place during the evaluation period. The program lacked a logic model, performance indicators and data collection strategy which impacted the ability of the evaluators to assess the true performance of the GMI.||The Evaluation Team worked with the program to develop a logic model and related indicators for the evaluation.|
|It is important to note that while the evaluation questions focus on the GMI as a program (pillars/priorities), the vast majority of responses/perspectives were provided at the project level.||Interviewees were asked to provide program level responses but mainly stayed within the project realm.|
3.0 Evaluation Findings
3.1.1 Is there a continuing need for the GMI?
There is a continued need for a program such as the Green Mining Initiative (GMI). All lines of evidence indicate a need for activities/projects pertaining to Green Mining. Key industry representatives agreed on the need for a program that was research centric, thereby allowing NRCan researchers to conduct necessary research, and industry to benefit from its expertise and knowledge.
The GMI is instrumental in addressing a range of economic, social and environmental needs associated with green mining.
What needs are program activities addressing?
- The primary beneficiary of the GMI is the mining and exploration industry. The mining industry sectorFootnote 22 includes industry associations and research organizations (e.g., Mining Association of Canada, Canadian Mining Innovation Council (CMIC)Footnote 23, Prospectors and Developers Association Canada (PDAC)). Their needs pertain to:
- Support for mining-related innovation;
- Identification of gaps in R&D and technology validation and adoption;
- Assistance in addressing regulatory inefficiencies impeding the timely adoption of mining-related innovation;
- Identification of current and emerging research trends and key issues that need to be addressed in “green mining” research in Canada;
- Focusing research efforts on the development of technologies and procedures to reduce the environmental costs of mining activities in Canada, and the economic costs of environmental good performance; and
- Helping to improve the image of the industry with respect to environmental performance and earning a social licence to operate (SLO).
- A significant portion of industry’s needs have/are being filled through the research projects that have been/are being initiated by GMI and the dissemination of results.
- Provincial and Territorial Governments have been interested in increased federal, provincial and territorial regulatory harmonization. The GMI has aligned to recent provincial mining strategies (i.e., in BC, Quebec and Ontario) and continues to endeavour to foster regulatory harmonization.
- Other Government Departments (OGDs) have identified a need to improve the regulatory processes in order to address challenges impeding industry innovation (e.g., lack of awareness/understanding of new technologies; delays and uncertainty in costly, time consuming regulatory processes). NRCan (MMS) is a responsible authority for the majority of environmental assessments of mining projects involving Tailings Impoundment Area (TIA) proposals. These assessments are carried out with OGDs which include Department of Fisheries and Oceans Canada (DFO), Transport Canada (TC) and the Aboriginal Affairs and Northern Development (AANDC) in the case of Nunavut projects. MMS’ scientific expertise in the area of mine effluent management participates in Environment Canada (EC) initiatives aimed at improving the administration of Mines and Metals Effluent Regulations (MMER). MMS works closely with EC and DFO to identify further opportunities to improve the efficiency and effectiveness of the TIA scheduling process through regulatory amendments and/or legislative changes.
- The industry’s short-term research needs for solutions to specific industrial problems often conflict with the long-term nature of university research. The GMI has a dual role as the developer and provider of knowledge as well as having expertise in innovative technologies. Applied research has historically bridged this dichotomy.
- Aboriginal communities have identified needs concerning economic development and sound stewardship of resources. Mining companies have become Canada’s largest private sector Aboriginal employer. GMI has carried out projects (e.g., ventilation on demand, tailing rehabilitation, etc.) that benefit local communities, worker health and safety, as well as many projects that enhance the economic viability of the industry.
- The General Public is interested in reducing the environmental impact of mining activities:
[One] of the greatest areas of public interest in the mining industry is the environmental impact of mining activities. Concerns about acidic drainage, heavy metals contamination, releases of tailings or other wastes into natural waters, and the general issue of environmental footprint have led to a widespread negative public image of mining.Footnote 24
Are the needs continuing?
- Needs are identified through input from the CMIC and an advisory committee. This practice remains in place and continues to be the main vetting mechanism for ongoing needs. As a demonstration of continued need, the GMI successfully leveraged significant resources (financial and in-kind) from industry and other stakeholders.
- Interviewees unanimously stated that there was a continuing need for the GMI. Some stated that the need would increase based on population expectations and more strenuous regulations. The GMI was instrumental in addressing a range of economic, environmental, social, regulatory and science needs associated with green mining which in turn were instrumental in contributing to a competitive mining advantage for the country.
- The survey results indicated a gap between the areas of need that were the most important to the survey respondents, and the level to which GMI was meeting these needs. While it was acknowledged that most of the respondents did not know whether the GMI and its activities were addressing their needs in the priority areas identified, it was suggested that the areas of need reflect a desire for weighted support.
- All case study projects linked clearly to a pillar or stated NRCan priorities. Some linked to more than one. Upon review it was observed that all case study projects were conducted in areas of identified need. Furthermore, the eight case study project interviewees indicated an on-going need for further work and support.
Has the GMI adapted to meet new or changing needs of stakeholders?
- According to interviewees, from its inception in 2009, the GMI worked toward responding to stakeholder needs as they arose and changed over time (e.g., ranging from addressing regulatory barriers in 2011 to a renewed focus on emerging issues). The literature did not contain any Public Opinion Research or satisfaction level research to corroborate the extent to which it had adapted to meet the new challenges.
- Almost all of the stakeholders indicated that the GMI had adapted to meet the new or changing needs of stakeholders. From a delivery perspective, projects were adjusted to reflect specific stakeholder needs and availability, on a case-by-case basis.
3.1.2 Are the GMI program and activities consistent with government priorities and NRCan strategic objectives?
GMI is consistent with Speeches from the Throne for 2010, 2011 and 2013 and Canada’s Economic Action Plan as provided in Budget 2010 and 2011 and is linked to the Northern Strategy. It supports federal government priorities of strengthening Canada’s research advantage, fostering commercialization and business innovation, protecting Canada’s natural environment, supporting Aboriginal and Northern communities and strengthening Canada’s international engagement.
GMI is aligned to NRCan’s Strategic Outcome 2 (Natural Resource Sectors and Consumers are Environmentally Responsible), Program 2.2: Technological Innovation; and Sub-Program 2.2.2: Green Mining.
To what extent is the GMI linked to federal government priorities?
- The GMI is consistent with the 2010, 2011 and 2013 Speeches from the Throne, Canada’s Economic Action Plan as provided in Budget 2010 and 2011, and is linked to the Northern Strategy. GMI directly supports the Federal Sustainable Development Strategy (FSDS) which outlines goals, targets and implementation strategies under four themes. Under the theme of “Addressing Climate Change and Air-Quality,” GMI supports the goals of reducing greenhouse gas emissions and mitigating the impacts of climate change as well as minimizing threats to air quality so that the air Canadians breathe is clean and supports healthy ecosystems.
- NRCan contributes to Green House Gas (GHG) mitigation and air quality through Ecosystem Risk Management.
- GMI is consistent with NRCan objectives. For instance, NRCan outlined the following government priorities, strategies and frameworks to which the GMI contributes:Footnote 25
- Supporting the Federal Government’s Economic Action Plan through responsible and sustainable resource development. It also fostered the support of communities through employment and healthier Aboriginal and Northern communities/environments in alignment with the Northern Strategy);
- Strengthening Canada’s research advantage through the enhanced competitiveness of Canadian industries; and support for R&D and innovation;
- Fostering commercialization and business innovation through improving demonstration and adoption of new technologies;
- Protecting Canada’s natural environment; and
- Strengthening Canada’s international engagement in areas such as helping address issues related to Corporate Social Responsibility (CSR) as well as the Arctic.
To what strategic departmental objectives and priorities are GMI activities, outputs, and outcomes linked?
- GMI is aligned to NRCan’s Strategic Outcome 2 (Natural Resource Sectors and Consumers are Environmentally Responsible), Program 2.2: Technological Innovation; and Sub-Program 2.2.2: Green Mining.
|SO 2||Natural Resource Sectors and Consumers are Environmentally Responsible|
|Program 2.2 Technological Innovation||Solutions to the environmental challenges faced by the natural resource sectors require sustained efforts in research, development and demonstration because the current level of science and technology is inadequate to address these concerns. However, the natural resource sectors neither have all the necessary knowledge nor make the necessary investments in innovation due to the potential poor return on investment. The objective of this program is to encourage academia, industry and the public sector to research, develop and demonstrate innovative solutions to environmental challenges encountered in the natural resource sectors. This objective is achieved through the generation and dissemination of scientific knowledge, and the development and demonstration of new technologies.|
|2.2 Result:||Stakeholders invest in research, development and demonstration to address environmental challenges|
|Sub-Program 2.2.2 GMI||Mining and processing have impacts on land, water and air. Technology development and commercialization entail significant financial, market, and technical risk, because uptake is dependent upon regulatory requirements, business investment priorities and availability of funding. This program reduces business risks by developing and demonstrating innovative mining technologies and practices that eliminate or reduce environmental impacts and risks. Business needs, technology gaps and priorities are identified with input from the Canada Mining Innovation Council and an advisory committee. Applied research aims to a) reduce land disturbance; water, energy and hazardous chemical use; waste volumes; and releases to the environment and b) accelerate site restoration.
By responding to business needs and seeking out synergies, the program focuses development efforts on technologies with the most potential to reduce environmental impacts and risks and to expand domestic and international business opportunities for mining companies, technology developers and consultants. Findings also a) contribute to the scientific, technological and socioeconomic basis for updating federal, provincial and territorial mining and environmental regulations and policies and b) inform policy priorities in other countries that drive demand for technologies and services.
|2.2.2 Results:||Technology developers increase demonstration of environmental technologies; Industry partners increase financial and in-kind contributions; and academic, government and other non-industry partners increase financial and in-kind contributions|
Source: NRCan Performance Measurement Framework 2012-13
3.1.3 Is there a legitimate, appropriate and necessary role for the federal government in the GMI program?
At the program level, there is a legitimate, appropriate and necessary role for the federal government in carrying out a Green Mining Initiative (GMI). By leading initiatives that minimize the environmental impact of mining operations, government can facilitate the industry’s ability to operate by helping it reduce the environmental impact of its operations, thus gaining greater support from the population. While some parties could deliver various aspects of the Program, none could deliver the scope of the whole program.
Should the Government of Canada be involved in Green Mining, S&T and R&D?
- The mandate of the GMI accrues from the role of the Minister of Natural Resources as defined under the Department of Natural Resources Act, S.C. 1994.
- The mining sector is a core component of the Canadian economy. All lines of evidenceFootnote 26 indicate a role for the Government of Canada (GC) in Green Mining S&T and R&D. The government has a legitimate role in leading an initiative that enables mines to be greener and in leading projects that can have tangible environmental and economic benefits for Canadians.
- While indicative only, the vast majority of respondents (15 respondents) indicated the GC should be supporting the GMI. While three individuals indicated that they did not know, there were not any “No” responses.
- The GMI is the only federal initiative that considers the entire mining lifecycle. It does so through four comprehensive pillars: Footprint Reduction, Mine Waste Management, Mine Closure and Rehabilitation, and Ecosystem Risk Management. These pillars focus on reducing the amount of waste produced in mining operations and minimizing its impact on the environment at all stages of the mine lifecycle. GMI also promotes innovation (development and deployment) in the mining sector.
Are there other parties who could deliver aspects of the GMI that are now delivered by NRCan?
- Almost all of the interviewees agreed that there might be other parties such as universities, CMIC, etc., that could deliver aspects of the GMI; however, none had the mandate, impartiality, capability or capacity to deliver the entire program.
- A majority of respondents (9 respondents) indicated that they did not know if there were other parties that could deliver the GMI. There were 5 respondents who indicated “yes” and 2 respondents who stated “no.”
3.2 Performance - Effectiveness
3.2.1 To what extent have intended outcomes been achieved as a result of the GMI?
GMI activities/projects focus on pillar/priority activity areas and have been validated by the Energy Mines Ministers Conference (EMMC) and the Green Mining Advisory Committee (GMIAC) as being aligned with industry needs.
It should be noted that case study projects were generally very successful and that the outcomes on a CanmetMINING level were being achieved. Projects could be linked to individual pillars/priorities and interviewees noted the tangible outcomes achieved.
The lack of a performance management framework from the onset of the Initiative hindered the collection of data to support a full assessment of the outcomes achieved at the program level. A summary assessment of the achievement of intended outcomes is provided in the table below:
|Level and Outcome||Comment||Assessment|
|Immediate Outcome: Expanded knowledge base of innovative mining technologies and practices that eliminate or reduce environmental impacts and risks||Achieved to a certain extent with stakeholders that have either participated in the project(s), attended workshops, etc. The reach of the expanded knowledge was not systematically captured by the program and as such is difficult to assess.||Partially Demonstrated|
|Immediate Outcome: Expanded understanding of regulatory, stakeholder and environmental concerns and requirements||A better understanding of the regulatory environment and barriers has been fostered by GMIFootnote 27. Some understanding of environmental concerns has been achieved from the Mines Minister Conference and liaison with Industry.||Partially Demonstrated|
|Immediate Outcome: Expanded understanding of Canada’s Green Mining Initiative||Communication of results has in certain instances been good, especially at the project level. However, communication of the “program” outcomes requires improvement. Results have been communicated to industry or subsets of industry.||Partially Demonstrated|
|Intermediate Outcome: Adoption of green mining innovation technologies leading to commercialization||Some progress has been demonstrated toward the achievement of this outcome. Some green mining innovation technologies have led to commercialization, however their scope and reach could not be quantified.||Opportunity for Improvement|
|Intermediate Outcome: Greater understanding by stakeholders of Canada’s commitment to environmentally sound and economically viable mining practices||From a stakeholder perspective, GMI has increased stakeholders’ understanding of environmentally sound and economically viable mining practices.||Opportunity for Improvement|
|Ultimate Outcome: An innovative, environmentally and socially conscious industry||It is too early to measure results.||Too early to measure|
|Ultimate Outcome: Positive public perceptions and image of the industry||It is too early to measure results.||Too early to measured|
Immediate Outcome: To what extent has GMI led to an expanded knowledge base of innovative mining technologies and practices that eliminate or reduce environmental impacts and risks?
All completed projects have been congruent with pillars/priorities, more specifically:
- GMI has carried out a number of projects that led to an expanded knowledge base of innovative mining technologies and practices. This knowledge has been shared through a number of venues, workshops, publications and face-to-face interaction with stakeholders.
- There are a number of GMI projects that have led to an expanded knowledge base of innovative mining technologies and practices that eliminate or reduce environmental impacts and risks including:
- Green Mining Green Energy; MEND; bio-waste for remediation of mine tailings; ventilation technologies; alternative vehicles for underground; etc.
- Three survey respondents indicated that GMI somewhat/mostly had led to an expanded knowledge base of innovative mining technologies and practices that eliminated or reduced environmental impacts and risks.
- The case studies revealed that GMI should or had led to an expanded knowledge base of innovative mining technologies and practices that eliminated or reduced the environmental impacts and risks. Stakeholders and workshop attendees indicated that they benefited from the information produced by GMI.
- While performance information is being collected at the project level, it is not linked to a larger, GMI-level repository that can allow the assessment of the extent to which GMI has led to an expanded knowledge base.
Immediate Outcome 2: To what extent has the GMI fostered an expanded understanding of regulatory, stakeholder and environmental issues, barriers and requirements
- GMI has carried out activities, Delphi Group 2010 and Meyers Norris Penny 2011), to determine the state of green mining technology in the Canadian mining sector. These studies identified barriers to the uptake of green mining innovations that enhanced the understanding of regulatory issues that need to be addressed.
- The GMI Working Group of the Energy Mines Ministers Conference has contributed to providing an understanding of stakeholder issues and requirements.
- The GMI has engaged stakeholders in workshops to help address barriers to the adoption of green mining technologies and practices in the Canadian mining sector.
- Five survey respondents indicated that GMI somewhat/mostly led to an expanded understanding of regulatory, stakeholder and environmental concerns and requirements.
Immediate Outcome 3: To what extent has the GMI been effective in expanding awareness of green mining?
Communication of results has in certain instances been good, especially at the project level. However, communication of the program outcomes has been lacking. Results have been communicated or targeted to industry or subsets of industry.
- At the project level, GMI has expanded the awareness of green mining among a targeted set of stakeholders (e.g., federal, provincial and territorial Energy and Mines Ministers, leaders from industry, academia, provinces and territories, NGOs, and some federal departments such as Environment Canada (EC) and Fisheries and Oceans Canada (DFO)) through presentations, workshops, etc.
- From a program perspective, initial information materials about the pillars contributed to expanding awareness of green mining. However, it was unclear whether similar materials pertaining to priorities were disseminated.
- Although indicative only, five survey respondents indicated that GMI was somewhat/mostly effective in expanding awareness of green mining.
Intermediate and Ultimate Outcomes
At its inception, the GMI did not establish a performance management framework that would allow for the design of a comprehensive logic model and performance indicators and a related data collection strategy. While a logic model was created for the purposes of evaluation, the data to support an assessment of intermediate and ultimate outcomes were challenging to identify.
- The GMI has been managed as an initiative that has been more focused on the short-term project-based outcomes, rather than longer-term program outcomes. While it is acknowledged that most, if not all, activities link to pillars and priorities, linking outputs to intermediate or ultimate outcomes presents a greater challenge.
- Intermediate Outcome: Adoption of green mining innovation technologies leading to commercialization: Some green mining innovation technologies have led to adoption and commercialization. However, it should be noted that while NRCan has an Intellectual Property Office with some patents and licenses, GMI does not appear to be actively engaged in commercialization and appears to be more focused on publication and research.
- Intermediate Outcome: Greater understanding by stakeholders of Canada’s commitment to environmentally sound and economically viable mining practices: The GMI itself is a demonstration of commitment to environmentally sound and economically viable mining practices. Among its direct partners and contributors, it thus demonstrates a greater understanding of these issues. Beyond this group within the mining sector as a whole, it is difficult to comment on the extent that GMI has specifically contributed to this increased understanding. Attribution for this level of outcome will always be difficult to obtain, though the continued involvement of GMI partners in the Initiative does show that some progress has been made.
- Ultimate Outcome: An innovative, environmentally and socially conscious industry: Some interviewees mentioned that while these objectives were desirable, economic imperatives and implementation concerns (e.g., regulatory, risk of failure, costs, etc.) affected the implementation of novel solutions. The program would need to collect data that would validate the achievement of this ultimate outcome. It was deemed too early to measure results.
- Ultimate Outcome: Positive public perceptions and image of the industry: The evaluation of this ultimate outcome would require consultation with the public as well as non-government organizations representing the environment (directly and/or indirectly) over a long-time scale to set baselines and find attributable progress as a result of GMI activities. It was deemed too early to measure results based on the timeframe required for industry to validate and endorse new technologies/practices.
3.2.2 Have there been any unintended (positive or negative) outcomes?
There were no unintended outcomes identified.
No positive or negative unintended outcomes were identified by the evaluation through the lines of evidence.
3.2.3 What are the chief internal and external factors influencing achievement of the GMI program objectives?
The chief internal and external factors influencing achievement of GMI expected results were:
- Regulatory inefficiencies;
- Lack of communication and knowledge transfer processes/protocols; and
- Financial barriers.
A number of internal and external barriersFootnote 28 to implementing GMI program objectives were observed and noted from the lines of evidence. These barriers focused on the following three themes:
1. Regulatory inefficiencies
Regulatory issues: As noted in the documents as well as corroborated by interviews, duplication and inconsistencies lead to confusion and uncertainty in issues such as lack of federal-provincial-territorial harmonization; lack of clear, consistent interpretation guidelines; unclear data requirements; some plans/agreements required prior to completed assessments; and regulatory standards reflective only of traditional, polluting technology;
Lack of knowledge among regulatory stakeholders: Interviewees noted that a significant number of regulators lacked awareness, knowledge, and understanding of the latest and evolving technologies and were reluctant to interpret regulations to factor in this evolution;
Interviewees noted that regulators were reluctant to accept alternative validation practices for new technologies/practices leading to cumbersome validation requirements, as well as expensive, time consuming pilots ; and
First to be second: Interviewees noted that companies were interested to be “first to be second” in innovation, as this tended to be proven technology; smoother regulatory approval process; fewer costs, delays, and risks.
2. Lack of Communications and Knowledge Transfer
According to the literature reviewFootnote 29, there have been some challenges surrounding communications and knowledge transfer. It should be noted that GMI has placed considerable emphasis on conducting knowledge exchange and communications with project stakeholders and has also participated in a number of exchange fora to convey and transfer knowledge. While some interviewees noted that GMI had been successful in sharing, others sought more visibility on the scope and extent of discoveries at a program level, which in turn could assist in identifying relevant knowledge. Two areas of improvement highlighted focused on green mining itself as well as the availability of government support.
Green mining: Lack of awareness, communication, knowledge transfer, and understanding of green mining among stakeholders (including financial institutions/markets), impeding innovation efforts and technology adoption. In particular, interviewees noted a lack of awareness and knowledge regarding new developments (e.g., industry members not aware of new technologies if they were not directly involved in a project).
Awareness of Government support: Some interviewees observed that it was likely that a number of mining stakeholders might not be aware of the GMI and Canada’s progress in green mining leadership and global competitiveness and productivity. There could be value in integrating efforts of other government departments.
3. Financial Barriers
The last category of barriers noted in literature and commented on by some interviewees pertained to the financial risks and limited financial resources to trigger adoption of nascent technologies.
High financial risks: Mining is a capital and energy-intensive industry operating in a high risk environment, which can lead to slow changes (e.g., need to maintain compliance, financial performance, reputation and relationships with Government and communities; more interest in adopting new technologies if proven to be substantially better). As such, industry is cautious to embark on absorbing innovation costs that will take a significant amount of time to recover through increased productivity or reduced operating costs. The payback period is a crucial consideration that affects the decision to adopt or not adopt proposed technologies.
Limited financial resources: Limited financial resources in a current climate of uncertainty and depressed ore prices mean that companies must prioritize their investments, thus hindering the adoption of “unproven” technologies.
Lack of effective financial incentives: The literature review highlighted that fiscal and other incentives were viewed as being too low to entice investments and steer Canada towards green mining leadership and continued global competitiveness. One interviewee noted that despite the economic significance of mining across Canada, including Northern communities, there is limited Government funding available to the industry compared to other sectors.
As such, there are real and perceived barriers to the adoption of green mining technologies. The program is a vehicle that can alleviate these barriers by forging ahead with state of the art green mining projects intended to increased efficiency and reduce operating costs/footprint.
3.3 Performance - Efficiency
3.3.1 Is the GMI program the most economic and efficient means of achieving outputs and progress toward outcomes?
Overall, GMI is a well-managed program. At the project level, financial and performance information is collected and monitored on a regular basis and its multi-stakeholder, collaborative design enables the leveraging of resources and in-kind contributions. The GMI has emphasized the practice of leveraging funds and receives in-kind resources to achieve results. Most of the NRCan interviewees were familiar with the practice of leveraging funds and often quoted an average of 6 to 1 leveraging overall. The roll-up of program information was more challenging given that the performance dashboard cannot easily separate GMI information from other programs.
In order to fully assess whether the GMI was the most economic and efficient means of achieving outputs and outcomes, program design, priorities and “costed” outputs would have had to be aligned to an agreed upon program logic model. As such, without the performance data linked to outcomes, it was challenging to measure the overall effectiveness of the program.
What is the difference between planned and actual spending? What is the explanation for this difference?
There was little variance between planned and actual spending between 2010-11 and 2012-13 which suggested that good financial management was in place during those years. In 2009-10, the significant difference between planned and actual spending was attributable to changes in the PAA structure of CanmetMINING which created some inconsistencies as to what was included under GMI.
What steps has the GMI taken in order to minimize the use of resources in the achievement of results?
The GMI has emphasized the practice of leveraging funds and has received in-kind resources to achieve project results. While it was noted that Green Mines Green Energy (GMGE) project leveraging was 7 to 1, most of the NRCan interviewees were familiar with the practice of leveraging funds and often quoted an average of 6 to 1 leveraging. The document review found evidence of leveraged and in-kind funding. For instance, in 2011 it was reported that research funding totalled $9.2 M. Of that, there were $6 M in leveraged funds (65%), with a MMSL contribution of $3.2 M. GMI projects are able to attract outside resources thus enabling more work to be completed for the same amount of NRCan resources.
What, if anything, about the GMI needs to be adjusted?
- The document review highlighted the following areas of adjustment::
- The need for more financial resources and strategic partnerships;
- The need to enhance/establish partnerships with regulators;
- The need to communicate more effectively with all stakeholders, particularly regulators who may not be familiar with the mining industry;
- The need to consider options for networking and maintaining contact with the industry given the current budgetary constraints; and
- The need to address the issue of replacing departed Highly Qualified People (HQP) and/or options for delivering the program.
- Interviewees suggested:
- Fewer priorities;
- Better communication of the good work internally and externally;
- Refining the plan that the GMI is measured against and identifying annual KPIs, milestones, etc.; and
- Refining the prioritization of effort/funding based on the relative importance of the pillars.
- When asked if there was anything about the GMI program they would like to change, six respondents stated “yes.” Some of the respondents provided ideas for change including using the CMIC “as a delivery vehicle with increased industry participation.” It was also suggested that the GMI should try to “achieve a higher profile” and “seek more industry involvement.”
Among the adjustments identified above, a consistent theme was the need to communicate more effectively with all stakeholders, particularly regulators (and also citizens), who might not be familiar with the mining industry. Also, there was a need to refine the plan that the GMI was measured against (e.g., identify annual KPIs, milestones etc.).
To what extent have the GMI outputs been produced and at what cost?
The GMI is producing project outputs; however, dedicated information on the exact quantity for all years of the evaluation period was not available at the program level. Interviewees and NRCan Program Researchers were in agreement that GMI outputs had been produced at the individual project level. As for the cost at which the outputs were produced, these were in accordance with project budgets allocated. It is important to note that while the evaluation question focuses on the GMI as a program (pillars/priorities), the vast majority of answers/perspectives were provided at the project level.
3.3.2 Is performance information being collected and used on an ongoing basis?
Performance information is being collected at the project level. There is evidence that, at the project level, performance information is used in executive decision-making. There is no standard information gathering of program performance.
To what degree is GMI performance information used for decision-making by the GMI program?
The general view among NRCan Senior Management interviewees is that GMI performance information is being collected and used on an ongoing basis for decision-making. One NRCan Senior Management interviewee noted, “GMI has been recognized for having good performance measurement information and data. This information is needed to make multi-million dollar decisions.”
It was unanimously agreed among NRCan Senior Management interviewees that the DashboardFootnote 30 was an extremely valuable tool which enabled them to monitor and track projects daily. Based on this information, “the program is always adjusting. It is web-based and enables ongoing departmental reporting.” The CanmetMINING Business Office staff indicated that the Dashboard was developed internally and had been up and running for almost a year and a-half. They described it as “work-in-progress” and noted that it did not track in-kind contributions.
While there were tools in place to collect performance information, namely the Dashboard, a performance management framework was not established from the commencement of the GMI. While it was observed that performance and budgets were closely monitored at the project level, the collection and oversight of program level performance information were problematic. It was observed that the information reported on the Dashboard was not GMI-specific and it would have been time consuming to attempt to extract program level performance information. While a logic model was derived during the course of this evaluation, performance data and a related data collection strategy were not implemented at the beginning of the GMI, making it very difficult for the program to “reconstruct” the GMI performance story.
Performance information was collected and used at the project level as noted. Individual projects were presented at mid-year and end-of-year assessments, and outputs were tracked at the project level. Files reviewed for the case studies indicated that projects were managed individually, and project files assembled based on the initiative and leadership of the respective project managers. Some project files contained a fair amount of performance information, whereas others contained less.
Overall, from a program standpoint, the absence of a clear GMI program logic model and related performance indicators mean that the program is managed at the project level. The Dashboard appears to be a step in the right direction.
4.0 Conclusions and Recommendations
- There is a need and role for the federal government in Green Mining R&D and S&T, and the focus of the work carried out under GMI is aligned with GC priorities and mandate.
- There are indications that the GMI is producing industry-relevant outputs. While a significant level of activities and projects are carried out, it is difficult to determine the extent to which program effectiveness has been achieved as results/outputs tend to be linked to projects, rather than pillars or priorities.
- There is a lack of performance data to measure the achievement of outcomes at the program level.
- While there is some evidence of industry adoption, the GMI does not track or report on uptake. It is recognized that industry adoption may take a significant amount of time and that companies are not compelled to report uptake or attribute uptake to the GMI.
- Some green mining innovation technologies have led to commercialization. However, it should be noted that while NRCan has an Intellectual Property Office, with some patents and licenses, it does not appear that GMI is active in commercialization and appears to be more focused on publication and research.
- It was expected that the extent to which outcomes had been achieved would have been tracked by “the dashboard.” However, it appeared that while “the dashboard” can track CanmetMINING information, the systems could not single out GMI-specific information.
Efficiency and Economy
- While significant effort has been devoted to collecting broader CanmetMINING information, there does not appear to be a standard for data collection, performance information or metrics across projects. Information was not collated in terms of percentage of completion or progress reports that outlined the status of outcomes. The link between project deliverables and project outcomes in relation to the GMI program was not established when the project was submitted. Reporting of performance should be at the program level as well as at the individual project level.
- From a project documentation standpoint, files could be organized in terms of project intent (objective), proposed resourcing level, performance measurement and link to performance management framework, project milestones and level of effort (schedule), proposed project outcomes, results (expected vs. achieved), lessons learned and deliverables. This approach would greatly assist in program management and evaluation.
Recommendation 1: Opportunities exist to engage other federal entities as the research conducted under the GMI extends across several federal initiatives. To take full advantage of this, it is recommended that GMI conduct a review of federal programs to identify potential efficiencies and areas for collaboration.
Given the breadth and range of research topics and partners within GMI projects, there is an opportunity to work with other federal entities [Environment Canada, (EC), Industry Canada (IC), Department of Foreign Affairs Trade and Development (DFATD) etc.] to achieve larger joint objectives (e.g., trade, innovation). Collaboration for mutual benefit also would increase the potential to attract more resources to the Program. Such a review should aim to ensure that the outcomes identified for GMI are not being fulfilled by others and vice versa and/or that support is better leveraged /coordinated. There are benefits of performing a horizontal review, namely:
- identifying shared objectives and leveraging funding and resources more effectively among federal entities;
- Enhancing program delivery effectiveness of respective partners; and
- Facilitating the measurement of results for multiple entities by delivering coordinated support to a concrete project.
Recommendation 2: Given the evidence highlighting the need to attract new partners through clearly articulated program priorities, it is recommended that GMI establish an overarching internal green mining research agenda that will guide long-term priority setting.
One of GMI’s strengths is the systematic tracking of outputs at the project level. The evaluation found however that there were issues with tracking at the higher program/initiative level. While CanmetMINING has been successful in managing and tracking below and above this level, without information specific to GMI it becomes difficult to tell the performance story of the Initiative, to highlight successes, to attract new partners and to develop unified program level objectives. The Initiative should consider establishing a research agenda, akin to Canadian Safety and Security Program (CSSP) Call for Proposals,Footnote 31 for innovative science and technology (S&T) projects. A comprehensive Call for Proposal program allows an entity to:
- Identify capability deficiencies, gaps and priorities with the coordinated assistance from stakeholders;
- Set the strategic course of projects and activities that best address capability gaps;
- Identify program priorities to guide investment decision making;
- Align program investments to strategic objectives;
- Enhance tracking and monitoring of progress against identified gaps and expected outcomes; and
- Facilitate the identification of leveraging opportunities among federal partners.
Recommendation 3: Given that no GMI-specific performance measurement and reporting strategy exists, it is recommended that a performance measurement and reporting strategy for the GMI be developed by the Program to systematically track progress on outputs and outcomes and ensure consistent performance data are collected across projects. This PMRS should include a logic model and associated narrative.
A performance measurement strategy should be established to systematically track progress on outputs and outcomes. Technical projects (including deliverables and outcomes) should be appropriately linked to the pillars and priorities and consistent performance data collected for these. For specific GMI projects, each project file should include a clear statement as to the linkage and importance of the proposed work to pillars/priorities, and should highlight the differentiating input/contribution that NRCan provides to the project.
Annex A: Natural Resources Canada Program Alignment Architecture 2013-2014
As depicted in the NRCAN 2013-2014 Strategic Outcomes and Program Alignment Architecture/Program Activity Architecture (PAA), for the Mining and Minerals Sector (MMS), GMI is currently situated as follows:
2013-2014 Strategic Outcomes and Program Alignment Architecture/Program Activity Architecture
Strategic Outcome 1: Canada’s Natural Resource Sectors are Globally Competitive
1.1 Market Access and Diversification
1.1.1 Mineral and Metal Markets Access and Development
1.1.2 Forest Products Market Access and Development
1.1.3 Energy Market Regulation and Information
1.2 Innovation for New Products and Processes
1.2.1 Mining Innovation
1.2.2 Forest Sector Innovation
1.2.3 Geomatics Innovation
1.3 Investment in Natural Resource Sectors
1.3.1 Mineral Investment
1.3.2 Forest-based Community Partnerships
1.3.3 Targeted Geoscience Initiative 4 (TGI4)
1.3.4 Geo-mapping for Energy and Minerals
1.3.5 New Energy Supply
1.3.6 Major Projects Management Office Initiative
1.4 Statutory Programs – Atlantic Offshore
Strategic Outcome 2: Natural Resource Sectors and Consumers are Environmentally Responsible
2.1 Energy-efficient Practices and Lower-carbon Energy Sources
2.1.1 Renewable Energy Deployment
2.1.2 Support for Clean Energy Decision-making
2.1.3 Alternative Transportation Fuels
2.1.4 Energy Efficiency
2.2 Technology Innovation
2.2.1 Materials for Energy
2.2.2 Green Mining
2.2.3 Clean Energy Science and Technology
2.3 Responsible Natural Resource Management
2.3.1 Forest Ecosystems Science and Application
2.3.2 Groundwater Geoscience
2.3.3 Environmental Studies and Assessments
2.3.4 Radioactive waste management
2.3.5 Earth Observations for Responsible Development of Natural Resources
Strategic Outcome 3: Canadians have Information to Manage their Lands and Natural Resources, and are Protected from Related Risks
3.1 Protection for Canadians and Natural Resources
3.1.1 Explosives Safety and Security
3.1.2 Materials and Certification for Safety and Security
3.1.3 Forest Disturbances Science and Application
3.1.4 Climate Change Adaptation
3.1.5 Geohazards and Public Safety
3.2 Landmass Information
3.2.1 Essential Geographic Information
3.2.2 Canada’s Legal Boundaries
3.2.3 Polar Continental Shelf Logistics Support
The expected results from the Performance Measurement Framework (PMF) are provided in Table 5.
|PAA Level||Order||Program Name||Expected Results|
|Strategic Outcome||2||Natural Resource Sectors and Consumers are Environmentally Responsible|
Annex B: Evaluation Matrix
|Evaluation Question||Evaluation Sub-Question||Indicator||Lines of Evidence||Data Sources|
|Evaluation Issue 1: Continued Need for Program|
|R1. Is there a continuing need for the GMI?||R1 (a). What needs are program activities addressing?||
|R1 (b). Are the needs continuing?||
|R1 (c). Has the GMI adapted to meet new or changing needs of stakeholders?||
|Evaluation Issue 2: Alignment with Government Priorities|
|R2. Is the GMI program and activities consistent with government priorities and NRCan strategic objectives?||R2 (a). To what extent is the GMI linked to federal government priorities?||
|R2 (b). To what strategic departmental objectives and priorities are the GMI activities, outputs, outcomes linked?||
|Evaluation Issue 3: Alignment with Federal Roles and Responsibilities|
|R3. Is there a legitimate, appropriate and necessary role for the federal government in the GMI program?||R3 (a). Should the Government of Canada be involved in Green Mining, S&T and R&D?||
|R3 (b). Are there other parties who could deliver aspects of the GMI that are now delivered by NRCan?||
|Performance – Effectiveness|
|Evaluation Issue 4: Achievement of Expected Outcomes|
|PS1. To what extent have intended outcomes been achieved as a result of the GMI?||PS1 (a). To what extent has GMI led to an expanded knowledge base of innovative mining technologies and practices that eliminate or reduce environmental impacts and risks?||
|PS1 (b). To what extent has the GMI fostered an expanded understanding of regulatory, stakeholder and environmental issues, barriers and requirements||
|PS1 (c). To what extent has the GMI been effective in expanding awareness of green mining?||
|PS2. Have there been unintended (positive or negative) outcomes?||PS2 (a). Identify the unintended outcomes.||
|PS3. What are the factors (both internal and external) that have facilitated or hindered the achievement of expected results?||PS3 (a). To what extent factors within or beyond the GMI program outputs contributed/hindered outcomes.||
|Performance – Efficiency|
|Evaluation Issue 5: Efficiency and Economy|
|PE1. Is the GMI program the most economic and efficient means of achieving outputs and progress toward outcomes?||PE1 (a). What is the difference between planned and actual spending? What is the explanation for this difference,||
|PE1 (b). What steps has the GMI taken in order to minimize the use of resources in the achievement of results?||
|PE1 (c). What, if anything, about the GMI needs to be adjusted?||
|PE1 (d). To what extent have the GMI outputs been produced and at what cost?||
|PE2. Is performance information being collected and used on an ongoing basis?||PE2 (a). To what degree is GMI performance information used for decision-making by the GMI program?||