Evaluation of the Geohazards and Public Safety Program Sub-activity (PAA Sub-activity 3.1.5)

Table of Contents

• 1.0 Introduction
• 2.0 Background and Profiles
  • 2.1 Context
  • 2.2 Mandate and Objectives
    • 2.2.1 Public Safety Geoscience (PSG)
    • 2.2.2 Canadian Hazard Information Service (CHIS)
  • 2.3 Activities, Delivery and Organization
    • 2.3.1 PSG Program.
    • 2.3.2 CHIS
  • 2.4 Governance and Administration
    • 2.4.1 PSG Program.
    • 2.4.2 CHIS
  • 2.5 Funding and Resources
    • 2.5.1 PSG Program.
    • 2.5.2 CHIS
  • 2.6 Stakeholders
  • 2.7 Logic Model
• 3.0 Evaluation Objectives, Scope and Methodology
  • 3.1 Objectives and Scope
  • 3.2 Methods
  • 3.3 Evaluation Challenges, Limitations and Mitigation Strategies
    • 3.3.1 Challenges
• 4.0 Evaluation Findings
  • 4.1 Relevance
    • 4.1.1 Continued Need for the Program (Issue 1)
    • Question: Is there an ongoing need for the two Sub-activity components?
    • 4.1.2 Alignment with Government Priorities (Issue 2)
    • Question: Are the two Sub-activity components consistent with federal government priorities and NRCan strategic objectives?
    • 4.1.3 Alignment with Federal Roles and Responsibilities (Issue 3)
    • Question: Is there a legitimate, appropriate and necessary role for the federal government in each of the two Sub-activity components?
  • 4.2 Performance
    • 4.2.1 Achievement of Expected Outcomes (Issue 4)
    • Question: To what extent have the activities implemented for each Sub-activity component produced the intended outputs and generated the intended outcomes?
    • Question: What are the major internal and external factors contributing to or constraining the performance of the Sub-activity?
    • Question: To what extent did the level of interaction and integration between the two Sub-activity components (as well as with other Sub-Activities and stakeholders) contribute to the production of measured outputs and the achievement of outcomes?
    • Question: Have the allocated financial resources and human resources (FTEs) been adequate for the activities of each Sub-activity component?
    • Question: Are there alternative approaches to the current design and delivery of the Sub-activity that would allow it to meet its objectives and mandate in a more effective and efficient manner?
• 5.0 Conclusions and Recommendations
• 6.0 Annexes
  • 6.1 Annex A – Characteristics of Case Study by Component and Program Activity
  • 6.2 Annex B – Service standards and performance metrics implemented by Geoscience Australia

Tables and Figures

• Table 1 Public Safety Geoscience (PSG) Program Expenditures, 2008–09 to 2012-13 ($’000)
• Table 2 Canadian Hazard Information Services (CHIS) Expenditures 2008-09 to 2012–13 ($’000)
• Table 3 Overview of evaluation data collection methods
• Table 4 Top downloaded of GPS and ESS publications, 2012
• Table 5 Current and ongoing needs for CHIS activities, services and products
• Table 6 Sample of PSG scientific outputs (peer-reviewed publications and scientific/technical presentations) for three projects, 2011-12
• Table 7 Queries made to PSG by sector, earthquake location queries, presentations/workshops and tours, 2007-08 to 2012-13
• Table 8 Types of CHIS information products accessed by users/clients surveyed (N=33)
• Table 9 Rating of adequacy of CHIS information products used by respondents’ organization (N=33)
• Table 10 Extent respondents agreed or disagreed with statements about their interactions with CHIS products and services? (N=33)
• Table 11 Overview of Geoscience Australia funding in 2011-12
• Table 12 Characteristics of Case Study by Component and Program Activity
• Table 13 Geoscience Australia Key Geohazards Performance Indicator (2011-12)
• Figure 1 Logic model of the Geohazards and Public Safety (GHPS) Sub-activity
• Figure 2 Expenditures of GHPS and by component, 2008-2009 to 2011-12
• Figure 3 External revenues* as Percentages of GHPS Expenditures, 2008-2009 to 2011-12

Acronyms

Executive Summary

Introduction

This report presents the findings, conclusions and recommendations for the evaluation of the Natural Resources Canada (NRCan) Earth Sciences Sector (ESS) Geohazards and Public Safety (GHPS), Sub-activity 3.1.5 of the 2012-13 Program Activity Architecture (PAA). The evaluation covers the period of 2008-09 to 2012-13 and $65 million in expenditures. The objective of the Sub-activity is to provide information to reduce the risk of natural hazards, identify effective risk mitigation options and provide geographical hazard information on demand to front-line responders in the event of an emergency.

The GHPS Sub-activity comprises two components: the Public Safety Geoscience (PSG) Program, a research component; and the Canadian Hazard Information Service (CHIS), an operational service component.

The mandate of the PSG Program is to contribute to the reduction of future losses from natural hazards events by developing improved scientific understanding of the underlying causes (and impacts) of geohazards and their probability of occurrence in Canada; providing input to the development of regulations, policies and techniques to mitigate hazard impacts; and creating new tools and methodologies to improve the assessment and communication of hazard and risk information for decision making. The PSG Program focuses on geological hazards, including landslides and slope instabilities, earthquakes, tsunamis, space weather and volcanic hazards. Public Safety Geoscience is overseen from the ESS offices located in Ottawa, with activities managed by Geological Survey of Canada (GSC) offices in Ottawa, Quebec, Sidney, Vancouver, and Halifax.

The mandate of CHIS is to assure that the appropriate emergency mapping and hazard information is available to the right people, in the right manner, at the right time to support decision making for emergency management. CHIS comprises  a wide range of activities, including detecting and providing alerts and other information on earthquakes, volcanic eruptions, geomagnetic storms (relevant for trans-polar airline navigation as well as power system, pipeline and satellite operations) and radiological and nuclear incidents; supplying data for tsunami alerting; and maintaining technical capacity for emergency management of landslides. A significant portion of CHIS activity is dedicated to maintaining, upgrading and improving geohazard monitoring networks and infrastructure. CHIS hazard monitoring is mainly accomplished through a national seismic network for earthquake and volcano monitoring and tsunami alerting. A national geomagnetic observatory network also serves as an input to a space weather forecasting system. CHIS plays an important role in relation to the Comprehensive Nuclear-Test-Ban Treaty (CTBT) that Canada ratified in 1998. CHIS operates the 16 International Monitoring System stations and laboratories hosted by Canada. CHIS is overseen from the ESS offices located in Ottawa, with activities carried out in facilities in Ottawa, Sidney, and Yellowknife.

The main stakeholders and client groups of the Sub-activity are emergency management organizations, building standards organizations, professional associations, critical infrastructure owners and operators and related industries, international and governmental organizations, media and the Canadian public.

Evaluation Scope and Objectives

The purpose of the evaluation was to assess the relevance and performance of the Geohazards and Public Safety (GHPS) Sub-activity (PAA Sub-activity 3.1.5) and provide recommendations as necessary. The five core evaluation issues under relevance and performance (as defined by the Treasury Board Secretariat (TBS) in the Directive on the Evaluation Function) are addressed through a series of specific evaluation questions.

The activities carried out under the predecessor to the PSG Program (the Reducing Risk from Natural Hazards (RRNH) Program) in 2008-09 are also included in the evaluation.

Five methods were used to collect and analyze evidence: a program document file and data review, a literature review (including analysis of similar programming in four other jurisdictions), 38 stakeholder interviews (internal and external to NRCan), seven case studies of PSG and CHIS projects and initiatives, and a web survey of CHIS users.

Evaluation Findings

Relevance

Continued Need for the Program: The evaluation findings demonstrate that there is an ongoing need for both Sub-activity components. The magnitude of disaster risks in Canada is higher than it has ever been due to aging infrastructure and climate change, and the financial costs resulting from natural hazards in Canada are expected to increase. In response to rising hazard risks, there is an increased need for cost-effective, proactive and mobilized efforts in the areas of hazard monitoring, emergency management, risk reduction and loss mitigation activities. The evidence suggests that NRCan, through this Sub-activity, should maintain its unique scientific, technical and monitoring capabilities related to various natural and human-made hazards. The PSG and CHIS are either the sole or the principal sources of research, knowledge, guidelines, tools, analysis, monitoring and alerts for the hazards covered under their respective mandates. The evaluation also found that the PSG and CHIS addressed stakeholders’ needs by making important contributions to a diverse range of Canadian international organizations.

Alignment with Government Priorities: The two Sub-activity components are directly aligned with federal government priorities and NRCan strategic objectives related to public safety and security. They also contribute to federal and NRCan mandates related to S&T, economic growth and sustainable development.

Alignment with Federal Roles and Responsibilities: There is a legitimate and appropriate role for the federal government in each of the two Sub-activity components. The federal government has a legislated mandate related to hazards monitoring, national security, public safety and the provision of disaster financial assistance to the provinces and territories. A number of national coordination mechanisms are in place, including the Federal Emergency Response Plan; departmental and strategic emergency management plans; the Federal Nuclear Emergency Plan, the Disaster Financial Assistance Arrangements; the National Platform for Disaster Risk Reduction (led by Public Safety Canada [PSC]), the Canadian Safety and Security Program (led by DRDC and PSC) and the Canadian Risk and Hazards Network). The federal government is also in a unique position to produce standard and consistent national information. Additionally, the PSG and CHIS clearly complement similar programming and services delivered by other organizations.

The evaluation revealed that the roles and responsibilities of the PSG and CHIS are not always clearly outlined or well understood. Consulted stakeholders expressed a need for greater horizontal leadership for the federal involvement in geohazards research and emergency management, as well as greater guidance from PSC, which could contribute to greater understanding.

In particular, clearer dialogue and guidance from key federal organizations would articulate the role of GHPS and others that contribute to emergency management, risk reduction and loss mitigation efforts in Canada. Greater communication of GHPS roles and activities would support this by improving the awareness of stakeholders and other federal organizations. In fact, while external stakeholders consulted were aware of specific activities or services provided by PSG and CHIS, they were not aware of the complete activity profile and initiatives of the GHPS components..

Performance

Achievement of Expected Outcomes: Despite the delay in the achievement of some outputs due to resource constraints and staffing shortages, the evaluation results indicated that the activities implemented for each Sub-activity component successfully produced the intended outputs through alternative innovative partnerships. PSG and CHIS stakeholders consulted and surveyed were in general very positive on the quality of outputs and of the services delivered to partner organizations and users. As a result, both the PSG and CHIS made progress towards achieving their respective short-term and intermediate outcomes.

In the case of PSG, outcomes were achieved in the areas of improved knowledge and awareness of geohazards occurrence, risks and impacts; increased understanding of risk reduction and loss mitigation options; and adoption of geohazards risk reduction and loss mitigation measures in the development of relevant policies, standards and regulations. A few examples of the PSG’s work include:

• The collaborative process for PSG research to inform the National Building Code has been effective in ensuring that the best scientific information is embedded into building design and engineering at regular intervals to minimize seismic risks.
• Collaboration with the PSG helped the District of North Vancouver better understand potential earthquake risks and impacts in terms of physical damage to buildings and infrastructure as well as financial losses and public safety losses.
• PSG space weather research resulted in the use of a real-time geomagnetically induced current (GIC) simulator prototype by power utility companies to mitigate the impacts of geomagnetic storms on their critical infrastructure.
• The PSG has developed a national probabilistic tsunami hazard map for all three Canadian coasts, one of the first of such maps in the world, to aid hazard assessments and inform policy-making.
• The National Energy Board (NEB), the Canada Newfoundland Labrador Offshore Petroleum Board (C-NLOPB) and Aboriginal Affairs and Northern Development Canada (AANDC) rely on PSG information in fulfilling their respective mandates.
• PSG products and expertise are commonly included in the regulatory decisions associated with natural resource and critical infrastructure developments – especially as they relate to their environmental assessments (e.g., proposed pipelines).
• BC Hydro and Power Authority applied updated seismic hazard information generated by the PSG to retrofit its dams and related infrastructure.

For its part, CHIS achieved outcomes in the areas of:

• providing more customized and timely emergency preparedness, response and recovery resources;
• providing more timely and open access to geohazards information;
• increasing the use of emergency preparedness, response and recovery resources and information by government, industry, and the media;
• fulfilling federal legislative and treaty obligations as they pertain to emergency preparedness, response and recovery; and
• providing support for more proactive actions on coordinated and timely emergency preparedness, response and recovery efforts.

Evidence of achievement was comparatively scarce with respect to some of the broader, longer term outcomes, namely the optimization of risk reduction and loss mitigation potential for more resilient built environment and more proactive, coordinated and timely emergency preparedness, response and recovery efforts. Limited information was available on the direct contributions of CHIS to the work of the organizations directly involved in natural hazards emergency response and recovery (e.g., provincial emergency management organizations).

The evaluation identified the major internal and external factors contributing to or constraining the performance of the Sub-activity. The key success factors included collaboration with partners and stakeholders, an open and flexible approach, qualified and committed staff, an understanding of user needs, and senior management support. Conversely, constraining factors related to limited internal resources and staffing (particularly high management turnover), the need for a more coordinated federal approach, a changing federal focus and a lack of integration between PSG and CHIS.

Strong interaction between the two Sub-activity components and other Sub-activities and stakeholders was found to contribute to the production of measured outputs and the achievement of outcomes. The PSG and CHIS collaborated with external partners, both within Canada and internationally, including different levels of government, academia, industry and research networks. Many of the collaborations took place because PSG/CHIS staff on an individual basis were personally engaged with partner organizations or came about as a result of planned encounters such as conferences, committees, or the NRCan website. These types of interactions were useful for leveraging funding, as well as knowledge, data and instrumentation.

The evaluation also found that both the PSG and CHIS had been more effective in collaborating with external partners than with one another. Promoting greater alignment and integration between the two Sub-activity components was in fact the most frequent suggestion made by both PSG and CHIS internal stakeholders for improving the effectiveness of the Sub-activity. Stakeholders noted that there was the potential for PSG to conduct more research to support CHIS information products, services and tools.

Demonstration of Efficiency and Economy: The evaluation found that the allocated financial resources and human resources (FTEs) were adequate for the activities of each Sub-activity component, but that internal funding decreases had been offset by increased external funding sources. While Sub-activity total expenditures increased 35% over the evaluation period, the evidence suggests that the resources internally allocated to the Sub-activity were inadequate compared to the significant amount of research work carried out and the infrastructure maintenance needed and upgrade investments made by both components. The Sub-activity increased its reliance on external revenues and leveraging during the evaluation period (increasing as a portion of total expenditures from 14% to 19% between 2008-09 to 2011-12). This reliance on external sources, while successful, has made year-to-year planning for the Sub-activity much more difficult to complete.

Within the limited resources, both components were able to produce a sizeable number of outputs. The internal funding of the Sub-Activity has remained at a level where the PSG and CHIS have provided value to Canadians and results against departmental priorities and mandated responsibilities through the leveraging of external contributions. The level of external contributions, both leveraged funding and in-kind, have increased over the review period. While internal funding has remained at a level allowing the components to achieve their core mandated aims, there is a possibility that the effectiveness of PSG and CHIS to achieve greater departmental results could be affected if external contributions were to decrease significantly. Therefore, it will be important to retain the current level of leveraging. The research and outputs produced under the Sub-activity have generated considerable external benefits in terms of cost-effective prevention measures (a prime example being the seismic research conducted by the predecessor to the PSG and CHIS that informed the National Building Code). Some efficiency gains could be achieved by closer integration and coordination between the two Sub-activity components and among the various PSG and CHIS locations.

Both the PSG and CHIS were found to have formal processes in place to plan and prioritize projects and activities, which contributed to the efficient use of resources. For the PSG, these processes included mid-year and year-end reviews, annual review of program activities, and monthly meetings on projects. For CHIS, they included weekly and annual staff meetings, production of capital and ongoing operational plans with input from all staff, and creation of annual work plans for each CHIS group. Each component had also developed performance metrics and service standards. Despite these efforts, there was room for improvement in the mechanisms and processes that both components used to collect and utilize performance data.

As part of the evaluation, alternative approaches to the current design and delivery of the Sub-activity were reviewed in order to determine how it could more effectively and efficiently meet its objectives and mandate. Four international jurisdictions were examined—Australia, New Zealand, Switzerland and the United States—and several common themes were identified. Other jurisdictions have been more proactive and effective than Canada in coordinating key emergency management players across multiple jurisdictions and engaging a broader range of community stakeholders. In terms of available resources, Canada resided in the middle of the reviewed countries, spending more than Switzerland and New Zealand but less than Australia.

A more detailed review of the Australian system and the work of a similar agency, Geoscience Australia, revealed a range of useful performance metrics and service standards that NRCan could consider adopting to enhance its performance tracking.

Compared to revenues from OGDs and other external organizations, CHIS and PSG conducted minimal cost-recovery for information services and products. Such additional potential revenues were considered by CHIS for specific services; however, internal stakeholders mentioned that users were used to having access for free and consider such as a public good.

Interestingly, like the GHPS at NRCan, Geoscience Australia’s revenues from external sources were primarily comprised of cost recovery for work performed on behalf of other agencies and other governments (inter-agency agreements). Geoscience Australia conducted minimal cost-recovery from the private sector. Several issues were identified during this Geoscience Australia review, such as: i) other investments were overlooked due to the focus on the products; and ii) the reliance on periodic inter-agency agreements had an impact on staffing and staff development policies. This Geoscience Australia review suggested that the agency seek to recover more of its costs from commercial users of information in order to reduce its dependency on budget funding.

Conclusions and Recommendations

In conclusion, the evaluation findings provide clear evidence that the Sub-activity addresses relevant needs related to geohazards and emergency management in Canada for a diverse range of stakeholders and that PSG and CHIS activities are aligned with both federal government and NRCan priorities. NRCan also maintains unique and necessary scientific, technical and monitoring capabilities related to various natural and human-made hazards under its mandate.

Evaluation results indicated that the activities implemented for each Sub-activity component successfully produced their intended outputs. PSG and CHIS stakeholders were very pleased with the quality of outputs and of the services delivered by the Sub-activity. Both the PSG and CHIS made progress towards achieving their respective short-term and intermediate outcomes. Evidence of achievement was comparatively scarce with respect to some of the broader, longer term outcomes. The performance of the Sub-activity could be enhanced with a few improvements, namely, (1) government-wide collaboration, (2) communication, (3) internal coordination, (4) strengthened performance measurement.

Based on these findings, the following five recommendations are made:

Recommendations	Management Responses and Action Plans	Responsible Official/Sector (Target Date)
1.  NRCan should identify opportunities to develop a horizontal policy dialogue among key federal disaster organizations to improve communications, achieve a common understanding of roles and responsibilities, consider legislative and regulatory adjustments, enhance collaboration, and thereby increase effectiveness.	Accepted The Earth Sciences Sector (ESS) has identified opportunities to engage with federal disaster management organizations and has taken steps to ensure that its role in disaster mitigation and emergency preparedness is clearly communicated to Public Safety Canada (PSC) – the leading federal agency for emergency management in Canada – and other federal stakeholders through ESS’s participation in meetings, committees, and other fora including: Participating at the Assistant Deputy Minister-level interdepartmental Emergency Management Committee (ADM-EMC) established in April 2006; Participating in Public Safety Canada’s Interdepartmental Advisory Committee formed to advise on the design of the National Disaster Mitigation Program and a National Disaster Resilience Strategy established Spring 2011; Contributing as an active member to Canada's Platform for Disaster Risk Reduction (established June 2009), the national venue for policy coordination among government agencies, industry, academia, and NGOs; Contributing through its continued collaboration with Public Safety Canada to the development of Emergency Management Plans for the PSC’s Government Operations Centre (GOC), and participating in the development and execution of interdepartmental emergency management protocols and exercises. Through the leadership of ESS, NRCan has established a horizontal Hazards Working Group in February 2013 to facilitate improved exchange of science and policy information and to provide a forum for the common discussion of targeted hazard-risk issues with OGDs and industry. This working group reports to the ESS Executive Policy Committee.	ADM, ESS (completed)
2.  NRCan should develop and communicate a comprehensive GHPS profile that describes all activities of PSG and CHIS, for activities conducted individually and in collaboration, as well as for any service or expertise-dependent activities. Such improved communication of the GHPS profile should primarily aim at facilitating the horizontal dialogue with other key federal organizations.	Accepted. GHPS stakeholders have a diversity of interests and needs (e.g., earthquake hazard, marine geohazards, space weather events). ESS will undertake measures to better communicate to stakeholders the suite of activities under the Sub-activity profile. These measures include: The launch of a new top-level "Hazards" page on the new NRCan web architecture – which will be implemented in stages starting in January 2014 – to provide better visibility and easy user access to NRCan’s hazards-related web pages; Development of a presentation annex that includes an overview of activities in both the Public Safety Geoscience (PSG) Program and the Canadian Hazards Information Services (CHIS), for use by NRCan staff in presentations and for sharing with partner organizations (by April 2014). Presentation of an overview and profile of the GHPS Sub-activity to the interdepartmental ADM EMC (by September 2014). These communication tools will be used to promote the GHPS profile and suite of activities among its clients, stakeholders and partners on an on-going basis and through the participation of GHPS personnel in fora and events such as the Canadian Risk Hazards Network (CRHNet) and the Platform for Risk Reduction.	DG, AWCB, ESS (March 31, 2015)
3.  NRCan should improve the priority setting process for GHPS to ensure coordination and integration between the Canadian Hazard Information Service and Public Safety Geoscience (e.g., planning; communications; collaboration).	Accepted. ESS commits to improving the coordination between both GHPS Sub-activity components in the process of annual strategic planning exercises.  GHPS will put in place a collaborative framework that will make provisions for: Bi-annual integrated planning meetings chaired by the DG will include the Sub-activity management to set broad strategic directions for program and services, including the identification of areas for collaboration and interdependency. These will be highlighted in a DG-level approved summary document that will be shared with GHPS personnel and used to guide the development of annual PSG and CHIS operational plans; CHIS and PSG Program will each elaborate its requirements for service or support from the other in the form of a prioritized list of requests and deliverables. Monthly operational-level meetings between the PSG Program and the CHIS will continue, during which progress against agreed commitments will be reported and variances identified and reconciled in a documented process;  Communications will be strengthened through the provision of meeting agendas and minutes and the implementation of a dashboard summarizing and highlighting the progress on joint projects. These documents will be accessible on a shared space for GSC senior management and Sub-activity staff; The renewal of the GHPS Sub-activity will follow established GSC processes. As part of this process, GHPS will continue to consult with representatives from its diverse stakeholder groups (e.g., NRCan, key federal players, provincial and territorial EMOS, municipal planners, industry) and S&T experts for input into priorities and to validate proposed research and service directions.	DG, AWCB, ESS (March 31, 2015)
4.  NRCan should continue to improve the performance measurement and financial tracking of CHIS and PSG (e.g., research and service standards) so that reliable information is available to manage the Sub-activity.	Accepted. GHPS management is currently working with NRCan’s Strategic Policy and Operations Branch to update and refine the GHPS Sub-activity’s performance measurement metrics in NRCan’s PAA. The GHPS Sub-activity will review and improve its procedures for tracking financial and in-kind contributions from all partners and will provide the Geological Survey of Canada DGs with a quarterly summary through a dashboard. The GHPS Sub-activity has implemented a new tracking mechanism to report efficiently on outputs, and will continue to collect information as per Sub-activity requirements to be used in mid-year and year-end reporting.	DG, AWCB, ESS (March 31, 2014)

1.0       Introduction

This report presents the findings, conclusions and recommendations for the first evaluation of the Natural Resources Canada (NRCan) Earth Sciences Sector (ESS) Geohazards and Public Safety (GHPS) Sub-activity 3.1.5 of the 2012-13 Program Activity Architecture (PAA). The objective of the Sub-activity is to provide information to reduce the risk of natural hazards, identify effective risk mitigation options and provide geographical hazard information on demand to front-line responders in the event of an emergency.^{Footnote 1} This Sub-activity comprises a research component, the Public Safety Geoscience (PSG) program, as well as an operational service component, the Canadian Hazard Information Service (CHIS).

This evaluation encompasses the five-year period from 2008-09 to 2012-13 and covers all objectives and activities of the Sub-activity, including those undertaken by PSG or CHIS individually or in collaboration. The Sub-activity had approximately 85 FTEs over the five-year evaluation period. Given that the PSG component was launched in 2009-2010, the activities carried out under the predecessor to the PSG Program, the Reducing Risk from Natural Hazards (RRNH) Program, in 2008-09 are also included in the evaluation.

2.0       Background and Profiles

2.1         Context

As is the case for other countries around the world, Canada is increasingly required to adopt comprehensive and integrated emergency management schemes due to complex factors such as climate and environmental change, demographic transition, urbanization and the interdependency of critical infrastructure. Studies have found that governance approaches to disaster risk reduction greatly impact the success of such efforts.^{Footnote 2}

Most disasters in Canada are local in nature and are managed at the municipal or provincial/territorial levels, as provided by the Emergency Management Act (EMA) and The Canadian Constitution. The Government of Canada (GC) plays a large role in building disaster resilience capacity. The public sector, for example, plays a role in the funding of research on effective building techniques in specific regions and the promulgation and enforcement of building codes that reduce the vulnerability of built structures to natural disasters.

Disaster-related efforts in Canada have historically focused on preparedness, response and recovery rather than long-term benefits accrued by mitigation. One major step forward indicating the ongoing need for risk reduction and loss mitigation vis-à-vis geohazards in Canada was the establishment of the An Emergency Management Framework for Canada^{Footnote 3} in January 2011. The Framework establishes clear governance and responsibility delegation mechanisms for all hazards for the various federal, provincial and territorial emergency management initiatives. It focuses on prevention/mitigation, preparedness, and response and recovery measures.

The ongoing operation of Canada’s Platform for Disaster Risk Reduction^{Footnote 4} further harmonizes all elements of the Canadian emergency management system. The structure and work of Canada’s platform are likely to become increasingly prominent in Canada and disaster risk reduction will see ever greater integration throughout emergency management at all levels.^{Footnote 5}

In order that Canadians are protected from natural hazards, constant monitoring and effective planning to reduce the impact of adverse events are required.^{Footnote 6} Hazards under the responsibility of ESS at NRCan include earthquakes, volcanic eruptions, landslides, geomagnetic storms, radiological and nuclear incidents. The provision of hazard information and products helps the various stakeholders (e.g., governments in Canada, international agencies and the international community, private sector operators of critical infrastructure and professional organizations) to mitigate, prepare for, respond to, and recover from emergency events. Informed investment in mitigation, planning and response measures can significantly reduce the extent of loss when disasters occur as well as the cost of relief and recovery from such disasters.^{Footnote 7}

Within the GHPS Sub-activity, ESS is responsible for five of NRCan’s eleven emergency management plans defined by NRCan’s risk analysis under the Emergency Management Act (EMA).  Work undertaken within this Sub-activity also serves to meet Canada’s obligations under the Comprehensive Nuclear Test Ban Treaty (CTBT).^{Footnote 8} The GHPS Sub-activity (3.1.5) falls under Program Activity 3.1 "Protection for Canadians and Natural Resources" and NRCan’s Strategic Outcome 3, "Canadians have information to manage their lands and natural resources and are protected from related risks."^{Footnote 9}

The GHPS Sub-activity comprises a research component, along with an operational component:

• Public Safety Geo-Science (PSG); and
• Canadian Hazard Information Service (CHIS).

Other ESS Branches at NRCan are involved in the delivery of the GHPS Sub-activity, such as the Mapping Information Branch (MIB) and the Canadian Centre for Remote Sensing (CSRS). The Energy Sector (e.g., Office of Energy Research and Development [OERD] and the Science Policy Integration (SPI) (i.e., Environmental Assessment Division) are also involved in specific activities.

The history and rationale as well as the mandate and objectives of each of the GHPS Sub-activity components are presented in the next section.

2.2         Mandate and Objectives

2.2.1        Public Safety Geoscience (PSG)

The mandate of the PSG Program is to contribute to the reduction of future losses from natural hazards events^{Footnote 10} by developing improved scientific understanding of the underlying causes (and impact) of geohazards and their probability of occurrence in Canada (e.g., the development of a National Tsunami Hazard Model for Canada), providing input to the development of regulations, policies and techniques to mitigate hazard impacts (e.g., the design, location and maintenance of built structures and technical guidelines and best practices on landslides) and creating new tools and methodologies to improve the assessment and communication of hazard and risk information for decision making (e.g., tools and methods to reduce earthquake and space weather risks, as well as geohazards assessment to inform the Environmental Assessments (EAs) of proposed energy projects).

History and Rationale

The Public Safety Geoscience (PSG) Program began in 2009-10 as a re-scoped version of its predecessor, the Reducing Risk from Natural Hazards (RRNH) Program. Beginning in 2004-05, the RRNH Program assessed natural hazards and developed methods to reduce losses resulting from these hazards. The principal long-term outcome of this Program was reduction in the loss of life and damage to the economy of Canada and its critical infrastructure in the event of natural disasters. At the end of the RRNH Program’s five-year funding cycle in 2009-10, it was reorganized into the PSG Program. The Program was re-scoped in order to provide a greater emphasis on mitigation and preparedness while still building on the assessment achievements of the previous Program.^{Footnote 11}

Major natural disasters are rare but inevitable, while lesser natural incidents occur more frequently and pose a recurring threat to Canadians and critical infrastructure. The cost to Canadians in the event of natural hazard events is significant and increasing over time due to aging infrastructure, increased development in hazard prone areas, climate change and low levels of personal preparedness.

While geological natural hazards are a constant threat, the risk they pose to Canadian infrastructure is rapidly increasing. The exposure and vulnerability to hazards grows in conjunction with increasing economic development and the concentration of people in hazard prone areas, as well as increasing dependence on fragile and aging infrastructure (e.g., highways, pipelines, power lines, railways and telecommunications).

These changes are reflected in the realignment of the research component of the PSG Program, which was implemented in 2009-10. This new research focus seeks to enhance NRCan’s contribution to mitigating and reducing future losses from natural hazard events.

The Federal Government is liable for approximately 90 percent of disaster relief funding under the Disaster Financial Assistance Arrangements (DFAA) that is administered by Public Safety Canada (PSC). In the event of a large-scale natural disaster, the GC provides financial assistance to provincial and territorial governments through the DFAA. When response and recovery costs exceed what individual provinces or territories could reasonably be expected to bear on their own, the DFAA provides assistance to the province or territory–not directly to affected individuals, small businesses or communities. Since the inception of the DFAA program in 1970, the GC has paid out more than $2.0 billion in post-disaster assistance to help provinces and territories with the costs of response and of returning infrastructure and personal property to pre-disaster condition.^{Footnote 12} Examples of recent payments include those for the 2003 British Columbia wildfires, the 2005 Alberta floods, and the 2006 flood in Newfoundland.

Since 2006, an interdepartmental Assistant Deputy Ministers’ committee for emergency management (ADM-EMC) has met regularly to discuss emergency management priorities and to make decisions to guide federal government actions during emergencies. The committee may make decisions or refer issues to the Federal Coordinating Officer (usually the Deputy Minister of Public Safety), who may refer the issue to a committee of Deputy Ministers. Similarly, the issue may be referred to Cabinet or to the Prime Minister. ADM-EMC is co-chaired by Public Safety Canada and the Privy Council Office to facilitate the sharing of information. The ADM-EMC has served as the coordinating body for events such as the 2007 floods in British Columbia and the H1N1 virus pandemic in 2009.^{Footnote 13}The PSG Program contributes to the reduction of future losses from natural hazard events by providing the knowledge and products required to increase the resilience of built structures; inform the location, design and operations of new structures; and motivate Canadians to prepare appropriately for natural hazard events. Informed investment in mitigation, planning and response measures can significantly reduce the extent of loss when disasters occur and the cost of relief and recovery from such disasters.

2.2.2        Canadian Hazard Information Service (CHIS)

The mandate of CHIS is to assure that the appropriate emergency mapping and hazard information is available to the right people in the right format and at the right time in order to support decision making for emergency management.^{Footnote 14} CHIS provides real-time monitoring and alerting for geohazards, including earthquakes, volcanic activity, tsunamis, landslides and geomagnetic storms. CHIS disseminates risk-related hazard information to support mitigation, preparedness, response and recovery and various phases of emergency management. CHIS also provides specialist expertise to support intelligence, security and emergency management activities related to nuclear and radiological events.

CHIS is the vehicle by which NRCan satisfies mandated responsibilities under the Emergency Management Act (EMA),^{Footnote 15} the Comprehensive Nuclear Test Ban Treaty Implementation Act^{Footnote 16} and the Federal Nuclear Emergency Plan by providing information related to hazards on an ongoing basis, as well as in response to crisis situations. CHIS also develops and maintains emergency response plans and has a response function based on these plans to rapidly provide information to support decision-making  as well as the public in emergency situations. It also provides oversight to other areas of ESS responsibility, such as support for emergency management, specifically the  management of remote sensing capabilities.

History and Rationale

For many years the monitoring of natural hazards and providing relevant information to the government and other parties has been part of NRCan’s mandate. The 2005 Auditor-General report^{Footnote 17} criticized NRCan for its lack of emergency plans, leading the Department to develop high-level plans in response to the Emergency Preparedness Act (EPA) requirements. CHIS was created out of this process, transitioning to a Service to Government role, in order to provide products and services and discharge mandated responsibilities, including emergency mapping capacity and the running of geophysical networks.

NRCan’s plans deal with emergencies within the scope of Geohazards and Public Safety; emergency mapping and remote sensing, geological incidents, extreme space weather, nuclear explosion and radiological incidents. The plans include clear roles and responsibilities while providing for an effective response to emergencies. Mitigating the vulnerability of a community to reduce the impacts of a disaster is also consistent with the objectives of the Emergency Management Act. In addition, the events of September 11, 2001, have focused attention on potential human threats and hazards.

CHIS also has responsibilities under the Federal Nuclear Emergency Plan (FNEP) developed in 1984 as a response to the crash of nuclear satellite COSMO 954 in the Northwest Territories (NWTT) and the accident at the Three Mile Island nuclear generating station in Pennsylvania. The FNEP outlines the federal role, organization and capability in responding to a nuclear emergency. This includes: the federal government’s aim, authority, emergency organization and concept of operations for handling a nuclear emergency; the framework of federal emergency preparedness policies and the links with other relevant documents; the federal responsibilities of participating organizations; and the interface between the federal and provincial emergency management organizations. Since the creation of the National Counter-Terrorism Plan in 1993, the FNEP also covers the provision of similar services in the event of a terrorist act involving radiological or nuclear devices.

The Comprehensive Nuclear-Test Ban Treaty (CTBT) calls for a full ban of all military and civilian nuclear explosions. It was adopted by the United Nations General Assembly in 1996, but as of 2012  has not entered into force. ^{Footnote 18} The Preparatory Commission for the Comprehensive Nuclear-Test-Ban Organization (CTBTO) was founded in 1996 to promote the treaty while developing and maintaining verification measures when the treaty enters into force. The organization, based in Vienna, has over 260 staff from over 70 countries.^{Footnote 19} Canada ratified the Treaty in December 1998 when Parliament passed the Comprehensive Nuclear-Test-Ban Treaty Implementation Act, creating the CTBT National Authority. The National Authority Steering Committee is comprised of representatives from the Foreign Affairs and International Trade Canada (DFAIT), NRCan, Health Canada (HC) and Environment Canada (EC). NRCan, through CHIS, is responsible for technical implementation, such as operating the 16 International Monitoring System (IMS) stations and laboratories hosted by Canada, transmitting the data gathered by these stations to the CTBTO's International Data Centre (IDC) and attending meetings.

2.3         Activities, Delivery and Organization

The objectives of the GHPS Sub-activity are realized through two streams of activities. The PSG Program is entirely dedicated to research and development (R&D), while CHIS is an operational component dedicated to the provision of geohazards information on an ongoing basis (including the operation and maintenance of the networks and infrastructure equipment).  About 5% of CHIS work is R&D focused on nuclear explosion and radiological monitoring issues, both of which are outside the scope of the PSG Program.^{Footnote 20}

2.3.1        PSG Program

The PSG Program conducts geohazards research and provides targeted information to government and industry on risk reduction and loss mitigation. The Program involves a wide range of staff led activities to develop national-scale hazard products and validated hazard assessment methodologies.^{Footnote 21} These activities can be grouped into two main categories: mitigation and preparedness activities.

• Mitigation activities are focused on the generation of natural and targeted hazard models.^{Footnote 22} This process involves:
  • designing and validating methods for developing hazard models;
  • establishing the location, magnitude, frequency and extent of hazards;
  • developing testable hypotheses, consistent with available data, to account for the processes and conditions that generate hazards; and
  • validating existing and implementing new methods to resolve uncertainties in existing hazard models for priority urban areas and corridors of potential new development.
  Mitigation activities also generate quantitative and semi-quantitative natural hazard risk models for the built environment.^{Footnote 23} This work requires the identification, modification and development of exposure databases required to assess natural hazard risks and the execution of quantitative evaluations of natural hazard risks based on probabilistic and scenario-based hazard models.
• Preparedness activities^{Footnote 24} were discontinued in 2012-13. These focused on the evaluation of existing strategies for modifying human behaviour, based around the development and implementation of data management and communication plans. These activities included the publication of reports on hazard processes, conditions for occurrence, consequences and predictability; the creation of hazard inventories and execution of hazard assessments; and the creation of methodologies for assessing vulnerability, defining risk assessment and estimating risk reduction. All activities had a communication and education strategy, which includes development of outreach strategies, work plans and training materials for those who planned for and responded to geohazard event. Whenever appropriate, new hazard information products were transferred to CHIS for incorporation into its routine client delivery system.

The PSG Program activities focus principally on five types of hazards: landslides and other slope instabilities, earthquakes, tsunamis, volcanic and space weather hazards.  .

For much of the period covered by this evaluation, PSG was structured in the following five sub-program/project areas:

• Targeted Hazard Assessment: Eastern, Western and Northern Canada: This sub-activity generates scientific reports to provide hazard assessment information on priority topics relevant to three regions in Canada:
  • Eastern Canada: This area addresses the most significant geo-hazard issues in eastern Canada as part of the national suite of activities. The scientific results provide a greater understanding of the seismic hazard for eastern Canada, and the response of glaciomarine sediments to seismic ground motion. The Sub-activity also develops techniques for monitoring high-risk landslides using Interferometric Synthetic Aperture Radar (InSAR)^{Footnote 25} remote sensing.
  • Western Canada: This area generates scientific reports to provide hazard assessment information on priority topics relevant to western Canada. These reports focus on the impact of natural hazards mitigation through the use of ESS knowledge and products in order to update and create new standards, policies and regulations governing the design and operations of critical infrastructure and other built structures.
  • Northern Canada: This area is designed to minimize the impact of earthquakes and geomagnetic storms by providing the seismic and space weather information to develop effective response and mitigation methods to be used by industry and national, provincial, regional and municipal governments. This will reduce the economic, social and environmental losses from natural hazard events in northern Canada. Specific knowledge and products generated by this Sub-activity are designed to be adopted by industry as ‘best practices’ and feed into the development of standards, policies and regulations governing the design and operations of built structures.
• National-Scale Hazard Assessment Project: The National-Scale Hazard Assessment project produces new or improved national assessments for earthquakes, tsunamis, volcanoes, landslides and space weather. Information gaps exist for most if not all activities and for many hazards the outputs are ‘first generation’ products that may need further work/investment depending on user needs. One of the project outputs is information on where targeted assessment work should take place in the future. As appropriate, national-scale assessments of other geohazards or refinements of initial outputs or derivative products may be considered in later years of the project. The National Scale Hazard Assessment project  synthesizes the results from three regional hazard assessment activities  that address the key geohazards affecting all Canadians.
• National Guidelines for Natural Hazard Assessment and Mitigation: This project develops technical and non-technical guidelines for users and clients to influence Canadian standards, policies and regulation. This suite of guidelines includes: technical space weather guidelines to reduce the impact of space weather phenomena on infrastructure; seismic site assessment guidelines to reduce the impact of seismic risk or assessment cost on new infrastructure; technical and non-technical landslide guidelines to reduce the impact of unstable slopes on people and property; and a technical evaluation of existing slope instability monitoring techniques.
• Quantitative Risk Assessment: Through the development of quantitative risk assessment models, this project demonstrates and promotes the methods and products required to increase the resilience of built structures; to inform the location, design and operations of new structures; to update and develop best-practices, guidelines and policies for managing threats associated with natural hazards; and to assist Canadians in preparing appropriately for natural hazard events.
• Increasing Public Preparedness for Geohazards: The goal of this project is to use NRCan’s geo-hazard knowledge to motivate Canadians to prepare appropriately for natural hazard events. This project works closely with the public safety and emergency management organizations responsible for preparedness in order to ensure that they have access to the best science in a format that can be used effectively to prepare Canadians for geo-hazard events.

2.3.2        CHIS

CHIS is intended to ensure that ESS Emergency Management responsibilities can be met in a robust and reliable manner.^{Footnote 26} In particular, CHIS provides geohazards information on an ongoing basis and in response to crisis situations to satisfy ESS obligations under the Comprehensive Nuclear Test Ban Treaty (CTBT),^{Footnote 27} the Federal Nuclear Emergency Plan (FNEP)^{Footnote 28} and the Emergency Management Act (EMA).^{Footnote 29}

CHIS involves a wide range of activities, including detecting and providing alerts and other information on earthquakes, volcanic eruptions, geomagnetic storms (relevant for trans-polar airline navigation, power system operations, pipeline operations, satellite operations, etc.) and radiological and nuclear incidents; supplying data for tsunami alerting; providing emergency geomatics for flood extent mapping; and maintaining technical capacity for emergency management of landslides.^{Footnote 30}

A significant portion of CHIS activity is dedicated to maintaining, upgrading and improving geohazards monitoring networks, infrastructure/equipment/hardware/software, survey equipment, remote sensory components and map and data repositories.

CHIS hazard monitoring is accomplished through a national seismic network for earthquake and volcano monitoring and tsunami alerting. In addition, a national geomagnetic observatory network serves as an input to a space weather^{Footnote 31} forecasting system.

CHIS plays an important role in relation to the Comprehensive Nuclear-Test-Ban Treaty (CTBT) that Canada ratified in 1998. NRCan, through CHIS, is responsible for carrying out technical implementation, such as operating the 16 International Monitoring System stations and laboratories hosted by Canada; transmitting the data gathered by these stations to the Comprehensive Nuclear-Test-Ban Treaty Organization’s (CTBTO) International Data Centre (IDC); and attending relevant meetings. Nationally, CHIS also feeds data directly into the Federal Nuclear Emergency Plan (FNEP).

Through these networks of monitoring systems, CHIS provides authoritative geohazards information and alerts to media and the public and provides remote sensing, geomatics and material support to government agencies responsible for emergency response and to industries. All CHIS information is also disseminated using a variety of methods including websites, emails, File Transfer Protocol (FTP),^{Footnote 32} Twitter and RSS feeds, formal reports and on-demand access to catalogues and databases.^{Footnote 33}

CHIS divides its work into sections based on specific types of natural hazards. The sections function largely independent of one another, although they pool all information for use across CHIS and share the same management structure. CHIS activities are carried out under the  following areas.

• Earthquake: CHIS provides continued monitoring of all seismic activity within Canada, producing reports on all significant earthquakes. The earthquake monitoring activity is hosted by CHIS at two locations (a western office in Sidney, British Columbia and an eastern office in Ottawa, Ontario). CHIS operates the Canadian National Seismograph Network (CSNS) of over 200 seismographic stations in Canada, acquires and analyzes seismographic data in real-time, produces hazard maps and calculators and collects accounts of seismic events from visitors to its website.
• Geomagnetism: CHIS is responsible for the operations of the Geomagnetic Laboratory located in Ottawa. The laboratory serves as the headquarters for the Geomagnetic Monitoring Service and the Geomagnetic Hazards Project. Staff at the Laboratory operate the Canadian Magnetic Observatory network, including engineering, technical support, data processing and data dissemination.
• Space weather: The Canadian Space Weather Forecast Centre in Ottawa is operated by NRCan under CHIS, with support from the Canadian Space Agency (CSA). This centre operates from NRCan’s Geomagnetic Laboratory, located in Ottawa, with its mission-critical IT component hosted at the two CHIS data centres in Ottawa and Sidney.
• Nuclear Emergency Response (NER): The Nuclear Emergency Response (NER) component of CHIS is responsible for assembling a response group, deploying the remote sensing units and providing technical expertise, as required under the Federal Nuclear Emergency Plan (FNEP). The NER team, located in Ottawa, has developed the ability to detect, identify and delineate radioactive materials in remote, urban and marine environments.
• Nuclear explosion monitoring: NRCan, through CHIS activities in Ottawa, is designated the National Authority responsible for the operation, installation and maintenance of all seismological, infrasonic and hydro-acoustic monitoring stations in Canada contributing to the International Monitoring System (IMS)  of the Comprehensive Nuclear Test Ban Treaty Organization (CTBTO).

2.4         Governance and Administration

PSG and CHIS are directed by the ESS Executive Committee responsible for the ESS Program Activity 3.1 (Protection for Canadians and Natural Resources). ^{Footnote 34} The Sub-activity is managed within the Atlantic and Western Canada Branch of the Geological Survey of Canada (GSC), ESS, NRCan. ThePSG Program Director and CHIS Program Manager report to the Director General of the Atlantic and Western Canada Branch (GSC-AWCB) in charge of the entire GHPS Sub-activity.

2.4.1        PSG Program

PSG Program is administered from the ESS offices located in Ottawa, with activities managed by GSC offices in Ottawa, Quebec, Sidney, Vancouver, and Halifax. Program activities fall under two themes: mitigation and preparedness.^{Footnote 35} PSG Program is managed by a Program Director based in Sidney, British Columbia, who oversees the direction and management of the Program with the support of a PSG Program Manager.^{Footnote 36} Over the last five years, the Program averaged approximately 37 Full-time Equivalents (FTEs) per fiscal year. The number of FTEs increased after the (RRNH) Program transition period, from 33 FTEs in 2009-10 to 42 FTEs in 2012-13.

Each project area has a Project Leader who reports to the PSG Program Manager and Director, and is tasked with coordinating the day-to-day activities of the sub-program/project.

2.4.2        CHIS

CHIS is overseen from the ESS offices located in Ottawa, with activities carried out in facilities in Ottawa, Sidney, British Columbia and Yellowknife, Northwest Territories.^{Footnote 37} Over the last five years, the Program generally averaged approximately 49 Full-time Equivalents (FTEs) per fiscal year. The yearly number of FTEs significantly decreased over the evaluation period from 52.5 FTEs in 2008-09 to 43 FTEs in 2012-13.

Managers for each CHIS sub-area manage the day-to-day activities of researchers and other staff. These managers report to the Program Manager of CHIS, based in Ottawa, Ontario, who is responsible for the direction and management of the Program. ^{Footnote 38}

2.5         Funding and Resources

The Sub-activity expenditures represented approximately $13 million per year on average for the five-year evaluation period (excluding 2012-13)^{Footnote 39}. In 2012–13, the administration and operation of the Sub-activity was supported by approximately 85 full-time employees (FTEs).

2.5.1        PSG Program

The expenditures for the PSG Program for the first four years of the evaluation period (2008-09 to 2012-13) were approximately $26.9 million. The four-year period includes the last year of the predecessor of the PSG Program, the RRNH Program. The transition period to the current form of the PSG Program was accompanied by an increase in expenditures (and number of FTEs) from an annual average of $4.3 million (2008-09 to2009-10) to $6.2 million annually (2010-11 to 2011-12). The number of FTEs increased from 32 in 2008-09 to 42 in 2012-13. Salaries represented the majority of the expenditures (75%) between 2008-09 and 2011-12. Table 1 presents a detailed breakdown of the expenditures of the Program from 2008-09 to 2012-13.

Table 1 Public Safety Geoscience (PSG) Program Expenditures, 2008–09 to 2012-13 ($’000)

	2008–09	2009–10	2010–11	2011–12	2012–13
A-baseFootnote 40	$3,371	$4,554	$5,403	$4,617	$5,001
Salaries from A-base (incl. EBP)	2,937	3,875	4,787	3,907	4,051
Vote-Netted Revenue (VNR)
B-base				$130
C-base
Capital and Major Capital
Other Government Department (OGD)	$276	$303	$696	$919	$1,846
Specified Purpose Accounts (SPA) [a]	$140			$635
Grant and Contributions (G&C)
Cost Recovery (CR)				$6
Total Expenditures	$3,788	$4,857	$6,099	$6,307	$6,847
% Salaries	77.5%	79.8%	78.5%	61.9%	59.6%
FTEs	32	33	35	41	42

Note:  [a] Research Agreement and Cost Shared Agreement; Totals in ‘000 may be +/-1 the sum of the column due to rounding.

Source: Compiled from financial information provided by program management

2.5.2        CHIS

Expenditures for CHIS totalled $37.8 million from 2008-09 to 2012-13. The service spent an average of $7.6 million annually. On average, about $4.3 million (51%) of CHIS expenditures were for salaries and $3.0 million was for O&M.

CHIS had 41 full-time employees (FTEs) in 2012-13.^{Footnote 41} Table 2 presents a detailed breakdown of the expenditures of CHIS for the fiscal years of 2008-09 to 2012-13.

Table 2 Canadian Hazard Information Services (CHIS) Expenditures 2008-09 to 2012–13 ($’000)

	2008–091	2009–101	2010–111	2011–122	2012–132,3
A-base	$3,932	$5,432	$5,504	$5,413	$5,617
Vote-Netted Revenue (VNR)	$1,006	$1,182	$1,011	$1,995	$450
B-base
C-base	$61	$86
Capital and Major Capital	$766	$498	$788	$348	$220
Other Government Department (OGD)	$182	$197	$1,060	$975	$870
Specified Purpose Accounts (SPA)4	$24	$10	$12	$27
Grant and Contributions (G&C)	$25	$55	$58	$33
Cost Recovery (CR)
Total Expenditures	$5,996	$7,460	$8,433	$8,791	$7,157
% Salaries	43.0%	55.1%	50.3%	48.0%	57.4%
% Operations & Maintenance (O&M) 5	53.2%	41.4%	36.3%	40.2%	30.4%
FTEs6	52.5	50.5	48.5	49.5	43

Notes:   1. Derived from NRCan GFS User Friendly - Management Reporting Module; 2. Derived from NRCan’s SAPLPD version 730 financial database; 3. 2012-13 figures represent projected expenditures as of March 6, 2013; 4. Research Agreement and Cost Shared Agreement; 5. O&M includes funding from A-Base, VNR, Capital (2001/201), Major Capital (3002/302) and C-Base; 6. Provided by CHIS staff for the evaluation; Totals in ‘000 may be +/-1 the sum of the column due to rounding.

Source: Compiled from financial information provided by program management

2.6         Stakeholders

PSG Program stakeholders include:

• other federal government departments and agencies (e.g., the Canadian Space Agency [CSA], Defence Research and Development Canada [DRDC], the National Research Council Canada [NRC], Environment Canada [EC], Public Safety Canada [PSC] and others);
• provincial, regional and municipal government bodies (e.g., City of North Vancouver) and provincial and territorial emergency preparedness and management organizations;
• foreign government bodies, such as the US Geological Survey (USGS) and the US National Oceanic and the Atmospheric Administration (NOAA);
• critical infrastructure operators/industries/industry associations, such as the property and casualty insurance industry, electrical power utilities and pipelines operators;
• professional associations, such as the Associations of Professional Engineers and Geoscientists (APEGS), and the Geological Association of Canada (GAC);
• academic research organizations, such as the Canadian Seismic Research Network (CSRN);
• international research organizations, such as the Japan Agency for Marine-Earth Science and Technology (JAMSTEC); and
• international bodies, such as the International Consortium on Landslides (ICL) and the International Civil Aviation Authority (ICAO).

CHIS main stakeholders include:

• the Government Operations Centre of PSC and several federal government departments and agencies (including Health Canada [HC], Canada Space Agency [CSA], Fisheries and Oceans Canada [DFO], Department of Foreign Affairs and International Trade [DFAIT], National Research Council [NRC], Environment Canada [EC], Transport Canada [TC], Defence Research and Development Canada [DRDC] and others);
• municipalities, provincial and territorial Emergency Measures Organizations (EMO);
• Nuclear Waste Management Organization (NWMO);
• critical infrastructure operators and power generation and transmission companies, industry/professional associations;
• international government agencies (e.g., US National Oceanic and Atmospheric Administration (NOOA) Tsunami Warning centers) and international organizations (e.g., CTBTO); and
• media and the Canadian public.

2.7         Logic Model

GHPS logic model (Figure 1) presented below illustrates the influences addressed by the Sub-activity, the types of resources, its components, outputs and intended outcomes.  The program theory represented by the logic model Figure 1 was refined from the following logic models in cooperation with GHPS staff:

• Logic model for the Reducing Risk from Natural Hazards Program;
• Logic model for the Public Safety and Geoscience Program;

Figure 1  Logic model of the Geohazards and Public Safety (GHPS) Sub-activity

Text version

Figure 1 Logic model of the Geohazards and Public Safety (GHPS) Sub-activity

0.0 Influences and External Factors:
0.1 Increasing concentration of Human Activities in Hazard Prone Areas
0.2 Increasing dependence on aging infrastructure
0.3 Climate change/sustainable economic development
0.4 Increased chemical, biological, radiological and nuclear (CBRN) risks / increased threats of terrorism
0.5 Low level  of government, industry and public emergency preparedness

All of these feed into the following three categories, which combined create a feedback loop.

1.0 Resources:

1.1 A-Base funding
1.2 B/C-Base funding
1.3 Other government department (OGD) funding, vote netted revenues (VNR) and revenues
1.4 Specified purpose account (SPA)
1.5 Full-time equivalent (FTE) employees
1.6 Facilities and Laboratories
1.7 Specialized systems and equipment
1.8 Earth Sciences Sector (ESS) Program Activity Board.

2.0          Programs/Activities

2.1 Public Safety Geoscience Program (PSG)

Conducts geohazards research and provides targeted information to government and industry on risk reduction and loss mitigation.
Sub-Program/Project areas: Targeted hazard assessments (Eastern, Western and Northern Canada); national-scale hazard assessment project; national guidelines for natural hazard assessment and mitigation; quantitative risk assessments; increasing public preparedness for geohazards.

2.2 Interaction and integration between 2.1 and 2.3
2.3 Canadian Hazard Information Service (CHIS)
Maintains geohazards monitoring networks, survey equipment, remote sensory components and map and data repositories. Provides geohazards information on an ongoing basis and in response to crisis situations.

Components/Hazard Areas: earthquake, space weather, geomagnetism, emergency radiation mapping, and nuclear explosion monitoring.

3.0 Outputs

3.1 Assessments: national and regional hazard assessments, risk assessments, risk assessments for the built environment, and scenario based risk assessments (PSG only).
3.2 Guidelines: standards-based technical and non-technical loss reduction guidelines for government and industry decision makers (PSG only).
3.3 Geohazards resources and preparedness products: maps, forecasts, reports, methodologies, analysis, data streams and archives, websites, and other information products (mix of PSG and CHIS).
3.4 Emergency management plans: geological hazards, space weather, nuclear explosions monitoring, nuclear and radiological incidents (CHIS only).
3.5 Geospatial data (where appropriate): earthquake, tsunami, landslides, geomagnetic, volcano, space weather, radiation and nuclear explosion data (CHIS only).

4.0 Immediate Outcomes

4.1 Improved knowledge of the occurrence and impacts of geohazards as well as mitigation strategies (PSG only).
4.2 Increased awareness of the risks posed by geohazards among government and industry decision-makers (PSG only).
4.3 Provision of more customized and timely emergency preparedness, response and recovery resources to public safety organizations (CHIS only).
4.4 More time and open access to geohazards information and advisories for the public/media (CHIS only)

5.0 Intermediate Outcomes

5.1 Improved understanding of geohazards risk reduction and loss mitigation options among government and industry decision-makers (PSG only).
5.2 Increased adoption of geohazards risk reduction and loss mitigation measures in developing relevant policies, standards and regulations (PSG only).
5.3 Greater use of emergency preparedness, response and recovery resources to public safety organizations (CHIS only).
5.4 Fulfilment of federal government obligations under legislation and international treaties (CHIS only).

6.0 Long-term Outcomes

6.1 More resilient built environment in terms of location, design and operations as risk reduction and loss mitigation potential is optimized (PSG only).
6.2 More proactive, coordinated and timely emergency preparedness, response and recovery efforts (CHIS only).

7.0 Ultimate Outcomes.

7.1 Improved public safety and security due to reduction in economic, social and environmental losses from geohazards (mix of PSG and CHIS).

Source: Developed by SED in consultation with PSG and CHIS management.

3.0       Evaluation Objectives, Scope and Methodology

3.1         Objectives and Scope

The purpose of this evaluation was to assess the relevance and performance of the Geohazards and Public Safety (GHPS Sub-activity (PAA Sub-activity 3.1.5) and provide recommendations as necessary. This evaluation was designed to meet evaluation standards and guidelines set out by the Treasury Board Secretariat’s Policy on Evaluation.^{Footnote 42}

The evaluation covered the period from 2008-09 to 2012-13 and included all activities under the GHPS Sub-activity. The activities carried out under the predecessor to PSG, the Reducing Risk from Natural Hazards Program, in 2008-09 and 2009-10 were also included.

This evaluation addresses the five core evaluation issues defined by the Treasury Board in the Directive on the Evaluation Function:

Relevance of the GHPS Sub-activity:

• Continued need for Program
• Alignment with government priorities
• Alignment with federal roles and responsibilities

Performance (effectiveness, efficiency and economy) of the GHPS Sub-activity:

• Achievement of expected outcomes
• Demonstration of efficiency and economy

3.2         Methods

The evaluation approach for this project was developed based on an Evaluation Assessment based on available program documentation and data. The methodology was validated at a workshop with program staff. A data collection matrix (DCM) was used to cross-link evaluation questions with associated performance indicators, data collection methods and data sources, allowing data to be triangulated and compared for each question during the analysis.

Five methods were used to collect and analyze evidence:

1. Program document file and data review (including secondary data from NRCan and other external sources;
2. Literature review (including analysis of similar programming in other jurisdictions);
3. Stakeholder interviews (internal and external to NRCan);
4. Case studies (a review of six projects/operational processes and a review of the transition from the RRNH to PSG Program; and,
5. Web survey with CHIS users.

Table 3 presents additional details on each data collection method. An overview of the characteristics of the case studies conducted by component and Program Activity is presented in Annex A (Table 12).

Table 3 Overview of evaluation data collection methods

Data Collection Method	Details
1. Document, file and data review	A review of approximately 200 documents, files and Program data, including: Departmental documents (strategic plans, annual reports, audits, etc.) Program documents (financial or administrative databases, performance monitoring, communication outputs, etc.)
2.   Literature review	A review of documents and literature on similar programming, including: Primary data from Australia (interviews) Secondary data from Australia, New Zealand, Switzerland and the United States Grey literature and documents/reports from international organizations Journal articles (e.g., geohazards management)
3. Stakeholder interviews (internal and external)	Total number of stakeholder interviews ...38 Internal (NRCan and Program management and staff) ...  17 External (clients/partners/collaborators outside of NRCan)... 21
4.   Case studies (four PSG activities, two CHIS activities and a transition case study)	Project/activity case studies...6 Examined 6 projects/activities encompassing different design and delivery features and partner organizations: PSG ...4 CHIS ...2 Review of approximately 370 project-level documents Interviews (between 2 and 6 interviews per case)...21 Internal ...12 PSG ...6 CHIS ...6 External...9 PSG ...5 CHIS ...4 Transition case study (RRNH to PSG) ...1 Examined the transition from the RRNH to PSG (focussing on the decision-making process and the outcomes for stakeholders) Review of program documents and files 3 interviews
5.   Web survey	Online survey with users of CHIS information products Total survey population (initial and snow-ball surveys) ...88 Number of responses (initial and snow-ball surveys)...44 Number of valid responses (n) ...33 (**) Response rate (†); Margin of error (‡)...41.3%;10.5%

Note: † Valid response rate = Number of completed surveys, divided by the valid sample, which excludes unreachable potential respondents; ‡ Calculated for a response distribution of 50% (i.e., 50% yes/50% no); 95% confidence level (19 times out of 20).

(*) Note that some of the case study interviews were conducted as part of the stakeholder interviews.

(**) A total of 44 questionnaires were received from the initial and snow-ball surveys. Only 33 respondents indicated they used CHIS products during the period (25 respondents from an initial survey and 8 respondents from a subsequent snowball survey).

3.3         Evaluation Challenges, Limitations and Mitigation Strategies

The challenges and data limitations encountered during this evaluation and the mitigation strategies adopted to counter them are discussed below. Generally, challenges were anticipated early in the process and associated mitigation strategies were proactively built into the evaluation design. The evaluation was designed to use multiple lines of evidence to answer each evaluation question. Findings could then be validated over different sources. Specific data limitations were identified, including some that were linked to the scope of the evaluation project.

3.3.1        Challenges

Evaluation design: The initial phase of the evaluation faced multiple delays related to establishing the Evaluation Advisory Committee and obtaining the names of external stakeholders. While delayed, the necessary data were received in the end. Nevertheless, in some instances and particularly in the case of CHIS, the number of identified stakeholders was insufficient to meet the interview and survey quota (see below). The delay incurred during the evaluation design phase was recovered during the fieldwork stage and thus did not affect the delivery of the preliminary findings.

Data collection: Typical evaluation challenges were apparent throughout the data collection stage. These included scheduling of interviews over the winter holiday period and prioritizing the wealth of program data/documents provided for the evaluation under a constrained timeframe caused by the initial delay in the design phase. Additionally, the late reception of outstanding data and documentation impacted the ability of the evaluation team to initiate case studies and to complete stakeholder interviews, particularly for CHIS, on schedule. In addition, some of CHIS stakeholders that were proposed could not be included in more than one method (e.g., as survey respondents or key interviewees for the case studies). This resulted in a lower number of CHIS interviews than originally planned. Nevertheless, given that CHIS stakeholders were also consulted via survey, the evaluation collected evidence for CHIS and PSG that was sufficiently balanced.

Document and literature review: While a large number of program documents and files were provided, these documents did not cover all areas equally. Information was lacking in areas such as the current governance mechanisms and performance management practices of both components. The formatting and reporting of the data were inconsistent. Findings from the document review thus were analyzed in combination with the information collected from the other lines of evidence to fill data gaps and cross-validate data with only little/inconsistent documentary support. There was also a significant potential for bias as the majority of documents on activities, outputs and outcomes provided were self-reported, i.e. files created by NRCan. Therefore, wherever feasible, relevant external sources/websites were reviewed in order to verify information and facts reported in program documents and files.  

Finally, the literature review could not secure detailed quantitative information (e.g., funding, expenditures, staffing, etc.) on the four jurisdictions (Australia, New Zealand, Switzerland, and the United States) included in the comparative review from public sources. In particular, a comprehensive comparative perspective on efficiency and economy was not possible given the evaluation scope. Some quantitative data were obtained for Australia through consultation with Geoscience Australia as part of a more detailed review of this jurisdiction. 

Interviews: The target number of CHIS stakeholder interviews could not be reached due to the low number of stakeholder names provided but the target number of PSG interviews was exceeded (23 conducted versus 15 initially provided). Given that the methodology included a survey of CHIS information users, the smaller number of interviews was supplemented to some extent by the views and perspectives of survey respondents. The evaluation obtained balanced interview data for both components. Nevertheless, certain groups of CHIS stakeholders, such as provincial emergency management organizations, were underrepresented. Where possible, information collected from other sources was analyzed to supplement the underrepresented perspectives.

Survey (CHIS only): The initial survey population was small and although the response rate reached 50% (considered positive for this type of survey); the final number of respondents was still low. From the initial list of 82 CHIS users provided, 73 were invited to participate in the survey and 9 e-mail addresses were invalid and removed from the list. A total of 34 respondents submitted a response. More than one-quarter of those indicated either that they had not received and/or accessed CHIS information products since 2008-09 (4 respondents) or that they did not know about CHIS information products (5 respondents). To mitigate the small population number, a snowball sampling technique was used to generate additional potential respondents. Respondents from the initial survey were asked to provide the contact information of relevant contacts in their organization. As a result of this technique, 15 additional user names were provided and 10 responses were received. Of these, 2 respondents did not know whether they had used CHIS information products during the period. 33 respondents indicated that they had used CHIS products during the period (25 respondents from an initial survey and 8 respondents from a subsequent snowball survey).

20% to 25% of survey respondents reported not having used CHIS information products during the period or not knowing about CHIS products. This might indicate that the system used to track the client base was inconsistent or that users did not recognize CHIS products.

While the survey results were quite positive, the small survey population and number of respondents (n=33) did not allow for any analysis beyond descriptive statistics and qualitative analysis. Finally, provincial emergency management organizations were not highly represented in the survey population. Importantly, the group of survey respondents was mainly comprised of users of seismic hazard/geomagnetic information and advisory services and they were in a better position to assess the adequacy of these types of information products. In contrast, between 35% and 45% of respondents were not opposed to rating the adequacy of CHIS resources for emergency preparedness, response and recovery but simply answered "Don't know/Not applicable". Potential bias: As in all consultation-based data collection in which stakeholders provide perspectives and experiences, there is a potential for bias. This is the case particularly for both the interview and survey data. The qualitative information and views collected through these methods have been balanced by quantitative/documentary data on program outcomes where possible.

Case studies: The case study analysis encountered a number of challenges. Firstly, a relatively low number of potential interviewees were identified per case study for PSG component. As a result, the target number of interviewees for case study interviews could not be reached.

Secondly, many activities selected as case studies were actually components of a larger project, making it challenging to gather data specific to the case study activities. In some cases, particularly for activities that spanned a significant period of time or when interviewees were involved in several related activities (e.g., earthquake and tsunami) but interviewed for only one, interviewees tended to refer to their work more generally, deviating from the narrowly defined case study.

To mitigate these limitations, supplementary information was gathered from case study documents and related literature. Also, to validate the information collected, data were triangulated with other methods. Whenever a distinction between the case study activity and the larger project could not be achieved, this was noted under the relevant findings.

Lastly, the examination of the transition from RRNH to PSG encountered challenges related to the time that had lapsed since the RRNH ended, as interviewees were either not available due to retirement or could not recall all of the pertinent details. Information from available documents was used to fill in the information gaps from interviews, wherever possible.

Attribution: The evaluation recognizes that attribution of longer-term and ultimate outcomes (e.g., more resilient built environment as risk reduction and optimized loss mitigation potential, more proactive, coordinated and timely emergency preparedness, response and recovery efforts and improved public safety and security) reduced to a single program/Sub-activity is often difficult as there are other complementary activities and or processes in play. This is especially relevant in attributing outcomes at a national level. Mitigation in this respect relies on the theory of change, to focus on the contribution of the programs/projects, rather than attributing Canada-wide geohazard reduction outcomes to the Sub-activity.

4.0       Evaluation Findings

4.1         Relevance

4.1.1        Continued Need for the Program (Issue 1)

Question: Is there an ongoing need for the two Sub-activity components?

All lines of evidence confirmed there was a continued need for the Sub-activity, as PSG and CHIS were the principal sources of research, knowledge, guidelines, tools, analysis, monitoring and alerts in the Canadian context for the hazards covered under their respective mandates.

Continued risk: The risks posed by earthquakes, tsunamis, volcanoes and space weather are fundamentally present as extremely high-cost, low-probability events in Canada. This low probability leads to decreased attention paid to these events by most sectors of society, making it difficult for planning and loss mitigation work to move fully forward. ^{Footnote 43} Increasing awareness of the risks posed by hazards events is paramount to the success of the Sub-Activity. The risk of these hazards continues to rise due to factors such as the increasing scale of human activities in hazard prone areas, obsolete infrastructure and a growing dependence on technologies susceptible to hazards.^{Footnote 44}  Further, based on a 2010 Institute of Catastrophic Loss Reduction report, public infrastructure in Canada was deemed highly vulnerable following decades of underinvestment and might be severely challenged by a large earthquake.^{Footnote 45} Additionally, increased demand for space weather forecasting has resulted from ever-growing technology developments, thus increasing the number of critical technological infrastructure and systems potentially affected by this hazard.^{Footnote 46}

Historically, among meteorological and hydrological hazards, floods were the most frequently occurring natural hazard in Canada.^{Footnote 47} According to the Canadian Risk and Hazards Network (CRHNet),^{Footnote 48} a significant earthquake is perhaps Canada’s greatest potential natural disaster. A high percentage of urban infrastructure was constructed prior to the introduction of modern seismic construction provisions. Therefore, the CRHNet recommends that the vulnerability of critical older infrastructure such as hospitals, schools and bridges should be addressed before a major earthquake occurs.^{Footnote 49}

Potential for financial loss: The financial cost of natural hazards in Canada is estimated to be large and increasing steadily due to aging infrastructure, increased development in hazard prone areas, climate change and low levels of personal preparedness. According to the Insurance Bureau of Canada, the 1998 ice storm in eastern Canada resulted in $5.4 billion in personal losses, the 1996 Saguenay flood cost $1.6 billion, and the 1997 Red River Valley flood cost $815 million in losses.^{Footnote 50} Furthermore, geomagnetic disturbances could interfere with the operations of infrastructure systems (such as power grids, pipelines, phone cables and railway systems) and, in extreme instances, cause blackouts, again with related financial losses.

Proactive and coordinated approach: Organizations involved in emergency management globally agreed that a proactive approach, such as conducting risk reduction and loss mitigation research and implementing associated measures before hazards occurred, could significantly reduce the extent of damage from hazards.^{Footnote 51} For example, Japan has had high levels of earthquake resilience. After the magnitude 9.0 earthquake in 2011, Japan was able to restore more than 90% of its power supply in ten days, 90% of its telephone lines in two weeks and 90% of its cellular base stations in 19 days.^{Footnote 52}

Japan has strict building codes and has spent billions of dollars to develop the most advanced structural technology against earthquakes and tsunamis.^{Footnote 53} Japan has also spent an estimated $500 million to build a sophisticated early warning system for earthquakes, which, according to experts, helped save millions of lives and mitigated the damage from the 2011 earthquake and tsunami.^{Footnote 54}

Other countries around the world have adopted emergency management (mitigation, preparedness, response and recovery) policies and measures that tend to reflect country specific factors, such as the prevalence of different types of hazards, political environments and socioeconomic factors. A review of the emergency management practices conducted during the evaluation for four jurisdictions (Australia, New Zealand, Switzerland and the United States) highlighted targeted efforts in areas such as hazard research, disaster-resilient design of critical infrastructure, stakeholder engagement and coordination, hazard information dissemination, public education and outreach, active community involvement, and community capacity building. The efforts of some of these jurisdictions might have been somewhat more extensive than those implemented in Canada to date in some areas, particularly in terms of stakeholder coordination and active community involvement.

An Institute for Catastrophic Loss Reduction (ICLR) report Canadians at risk: Our exposure to natural hazards^{Footnote 55} emphasized that implementing a proactive approach would involve developing a better understanding of natural hazards in Canada, building on initiatives such as the Canadian Disaster Database and the Canadian Risk and Hazards Network (CRHNet). The ICLR also noted that better data on the costs and benefits of mitigation activities were still needed to allow communities to identify the range of specific hazards they face and assess their level of vulnerability and risk. According to the report, data collection is uncoordinated across many departments and agencies; therefore, there was limited information on what cost savings might have resulted from such efforts. The report further identified case studies from Asia and North America where local communities, all levels of government and non-governmental organizations worked collectively in a coordinated manner in order to reduce the harm caused by disasters.^{Footnote 56}

Also identified in the 2010 ICLR report was the need for increasing public awareness of hazards, risk and mitigation through federal and provincial programs. The Organisation for Economic Co-operation and Development (OECD) also identifies public awareness as a key factor for effective natural hazards risk management: "In this respect, recent experience has shown that public awareness of natural hazards and disaster risk reduction education constitute a foundation and pre-requisite for effective catastrophic risk management strategies at country and regional levels."^{Footnote 57} Finally, a large number of internal and external stakeholders consulted also confirmed that the need for PSG and CHIS programming and services would increase going forward, citing the growing public awareness of hazard risks/impacts resulting in greater public demand for faster and better emergency management support/services.

PSG and CHIS contribution: The evaluation found that NRCan maintained unique and necessary scientific, technical and monitoring capabilities related to various natural and human-made hazards in Canada. PSG and CHIS made critical and important contributions to a diverse range of stakeholders such as municipal, regional, provincial, territorial and federal government organizations, critical infrastructure operators, emergency management agencies, industry and professional associations, research networks, international organizations, the media and the general public through the provision of research findings, knowledge products, advice, information and alerts.

All internal PSG stakeholders consulted noted an ongoing need for the activities carried out under PSG. The large and diverse number of collaborators and partners that supported PSG projects (financially and in-kind) was cited as an indication that PSG addressed demonstrable needs. For example, in 2012-13, external in-kind commitments for PSG projects were estimated at $1,429,500 and commitments from OGDs were estimated at $924,000. This is equivalent to 36% of the total PSG funding for that year. These estimates were collected from PSG project leaders and activity leads for the year 2012-13 as of March 6, 2013.^{Footnote 58} Interviewees noted the high demand for PSG publications and this was further demonstrated through the volume and frequency of PSG publications downloaded from ESS’ online bibliographic database GEOSCAN relative to other ESS publications. Among the top ESS publications downloaded in 2012, 11 were from PSG and 4 from CHIS, which represented 19% and 12% of ESS downloads respectively (Table 4).^{Footnote 59}

Table 4 Top downloaded of GPS and ESS publications, 2012

Components	Top publications (*)		Downloads (*)
	#	%	#	%	Average
PSG	11	22%	1,548	19%	141
CHIS	4	8%	1,014	12%	254
Other ESS	35	70%	5,780	69%	165
Total ESS	50	100%	8342	100%	167

Source: Compiled from data provided by PSG program from ESS (GEOSCAN) - Publications database; (*): filtered on the sum of downloads, for publications with >80 downloads.

Many geohazards-related media inquiries to NRCan were reported by NRCan interviewees. A media coverage analysis of NRCan showed that earthquakes were consistently in the top five issues. From August 2011 to December 2012, earthquakes were in the top five for seven months over the 17-month period.^{Footnote 60}

All PSG external stakeholders consulted also indicated that there was an ongoing need for the activities carried out under PSG. They also advocated a proactive, as opposed to reactive, federal government stance on geohazards research and emergency management and noted the important role of geohazards research in facilitating resource/economic development in a safe and environmentally responsible manner.

Furthermore, Public Service Canada (PSC) and CHIS external interviewees reported that they relied heavily on NRCan information regarding seismic hazards, flooding and space weather to fulfill their mandates. CHIS data were directly used by the Government Operations Centre (GOC)^{Footnote 61} together with other hazard information provided by different federal operation centres and Canadian and international partners under an information sharing and communication interoperability protocol.

Interviewees stated that even though they also received earthquake reports from the US Geological Survey (sometimes faster than those from NRCan), it was still crucial to maintain Canadian expertise.

All CHIS internal stakeholders consulted indicated that there was an ongoing need for the activities carried out under CHIS. Increasing prevalence of geohazards, reliance on CHIS for mission-critical services on the part of various industries, stakeholder/user and funding/support for CHIS activities and high demand for hazard information were cited as factors that warrant ongoing CHIS work. All CHIS external stakeholders also indicated an ongoing need for the activities carried out under CHIS for similar reasons. CHIS external stakeholders also noted the unique capabilities of CHIS for addressing national and international security concerns and CHIS support for emergency management.

Further, the survey of CHIS external stakeholders revealed several varied uses of CHIS products, tools and services. The majority of survey respondents agreed or strongly agreed (79%) that CHIS information products adequately addressed their organization’s needs. Also, the fact that CHIS products continued to meet stakeholder needs was further confirmed by the majority of surveyed stakeholders indicating that there was an ongoing need for the core CHIS activities, services and products (Table 8). Nearly all survey respondents (91%) either strongly agreed or agreed that there was a continued need for CHIS to maintain its geohazards monitoring networks, survey equipment, remote sensory components and map/data repositories. More than three-quarters of survey respondents also indicated that there was a continued need for CHIS to provide geohazards information on an ongoing basis and in response to critical events (85%), use advanced research and technology to improve geohazards information products (82%) and facilitate more timely and open access to geospatial information (76%).

Table 5 Current and ongoing needs for CHIS activities, services and products

	1- Strongly disagree 2- Disagree	3- Neither agree nor disagree	4- Agree 5- Strongly agree	Don't know/not applicable
There is an ongoing need for CHIS to maintain geohazards monitoring networks, survey equipment, remote sensory components and map/data repositories.	3%	3%	91%	3%
There is an ongoing need for CHIS to provide geohazards information on an ongoing basis and in response to critical events.	3%	3%	85%	9%
There is an ongoing need for  CHIS to use advanced research and technology to improve geohazards information products.	3%	3%	82%	12%
There is an ongoing need for more timely and open access to geospatial information.	3%	9%	76%	12%
There is an ongoing need for more timely emergency preparedness, response and recovery resources.	3%	12%	63%	21%
There is an ongoing need for more customized emergency preparedness, response and recovery resources.	6%	12%	60%	21%

Source: Survey of users of CHIS information products (N=33)

The need for further geohazards knowledge and tools in the Canadian/national context to minimize hazard risks/impacts was also reported by survey respondents. This expectation had placed increasing demand on organizations like CHIS that were charged with monitoring hazardous events to provide this information to the public as quickly as possible, while still ensuring quality and accuracy of content.^{Footnote 62}

Data provided/facilitated by CHIS were also critical to some stakeholders for conducting research and served as a viable alternative to labour-intensive manual surveys necessary for some research programs. Real-time monitoring of geomagnetic activities and earthquakes, and related services enabled some stakeholders to provide quality and timely services to their clients, such as sending alert notifications, while the warnings, knowledge and know-how associated with CHIS information products assisted other stakeholders in carrying out emergency management tasks.

NRCan’s staff expertise was further highlighted by partners and collaborators of the six PSG/CHIS case study projects (see Table 12 in Annex A for an overview of project and partners/collaborators). NRCan was cited as the only organization that had adequate expertise, infrastructure or scientific credibility to carry out critical components of the projects. Further, all PSG and CHIS case studies were considered to be highly relevant to their respective target user groups by stakeholders. It was also noted that all examined projects were highly client-oriented and/or engaged in close collaborations with the project partners to deliver the best results, capitalizing on NRCan’s long experience in the field but drawing also on partners’ and collaborators’ capacity to deliver. Case study activities specific to PSG also noted an increasing need for the  section in the future due to the projected increase in development in northern Canada.

4.1.2        Alignment with Government Priorities (Issue 2)

Question: Are the two Sub-activity components consistent with federal government priorities and NRCan strategic objectives?

Current alignment: PSG and CHIS are directly aligned with federal government priorities and NRCan strategic objectives related to public safety and security. Indirectly, the Sub-activity also contributes to federal and NRCan mandates related to economic growth and sustainable development. On the level of departmental priorities, the Sub-activity is aligned with the NRCan priority to leverage science and technology (S&T) knowledge for safety and security risk management and corresponds to the Strategic Outcome to ensure that Canadians have information to manage their lands and natural resources and are protected from related risks.

The 2011 Speech from the Throne stated that, "[T]he Government of Canada has no more fundamental duty than to protect the personal safety of our citizens and defend against threats to our national security."^{Footnote 63} This overarching priority of the Government of Canada policy is to minimize the vulnerability from natural and other threats to public safety and security, including the threat of terrorism.^{Footnote 64} In addition, the 2007 Government of Canada framework Mobilizing Science and Technology to Canada’s Advantage - to guide Canada’s science and technology policy for the future ^{Footnote 65} notes that S&T is instrumental in modeling and predicting natural disasters, such as earthquakes, tsunamis, floods, landslides and forest fires and helping Canada prepare for and respond to these events. An internal assessment of the Sub-activity conducted in 2009 indicated that PSG and CHIS also highlighted a general alignment with the recommendations from this 2007 S&T framework and from another study^{Footnote 66} on Canada’s S&T role and priorities (both reports noted the important role of science, research and technology in enhancing public safety and security). The assessment further noted that PSG and CHIS contributed to the strategic outcome of NRCan related to safety, security and governance (natural resource and landmass knowledge strengthen the safety and security of Canadians and contributed to the effective governance of Canada).

Nearly all PSG and CHIS internal stakeholders indicated that their respective programming/ service were aligned with both NRCan and federal government priorities and strategic objectives. PSG internal stakeholders noted specific NRCan Strategic Outcomes (Canadians have information to manage their lands and natural resources and are protected from related risks), specific NRCan priorities (leverage S&T knowledge for safety and security risk management) and specific federal government outcome areas (a safe and secure Canada) where the alignment was the greatest. Other internal stakeholders noted that PSG contributed to economic/ resource development by informing environmental assessments and facilitating critical infrastructure (e.g., pipelines) operations. It was also noted that that CHIS supported economic development by assisting industry (e.g., resource exploration) in providing hazard assessments for planned projects.

Similarly, based on documentary and interview evidence, all projects examined as case studies are highly aligned with the federal priorities and NRCan strategic objectives. For instance, the Improved National Earthquake Hazard Model for Canada carried out under PSG’s Assessing Earthquake Geohazards project is required by the Canadian National Committee for Earthquake Engineering (CANCEE) for the 2015 and a subsequent version of the National Building Code of Canada and other national codes/standards (e.g., dams, pipelines). As such, the uptake of the model directly contributes to the following two short-term outcomes: i) Use of ESS knowledge to assess, update and develop standards and ii) ESS products increase hazard mitigation by decision-makers.^{Footnote 67} Further, based on the reports of participating stakeholders, CHIS project concerning the development of a surge capability to address extreme traffic flow on the earthquakescanada.ca website directly addressed NRCan’s national mandate to provide information on earthquakes in a timely and effective manner.

Possibility of shifting priorities: Interview data indicate that the current degree of alignment of the Sub-activity with federal and NRCan priorities could change with a shift in federal priorities. For example, the current federal government priority on developing Canada’s northern region may affect the Sub-activity as PSG and CHIS indirectly contribute to economic growth and sustainable development as noted earlier (e.g., contribution to economic and resource development by informing environmental assessments and facilitating critical infrastructure). The primary focus of PSG and CHIS programming remains on the south, where the majority of the Canadian population lives, where the bulk of Canadian infrastructure is concentrated and, consequently, where the actual risks are greater.

4.1.3        Alignment with Federal Roles and Responsibilities (Issue 3)

Question: Is there a legitimate, appropriate and necessary role for the federal government in each of the two Sub-activity components?

Context: The federal government has a legislated mandate related to hazard monitoring, national security, public safety and the provision of disaster financial assistance to the provinces/territories. Additionally, the Government of Canada (GC) works towards fulfilling the need for uniform and consistent hazard approaches and information across the country, in addition to having jurisdiction over certain regions of the country such as offshore areas.

Coordination across jurisdictions and organizations and minimization of duplication in emergency management is also facilitated by mechanisms such as the Federal Emergency Response Plan (FERP),^{Footnote 68} departmental emergency management plans,^{Footnote 69} strategic emergency management plans,^{Footnote 70} the Disaster Financial Assistance Arrangements (DFAA)^{Footnote 71} and the Canadian Risk and Hazards Network (CRHN).^{Footnote 72}

As for the specific role of NRCan, its involvement in hazard monitoring, emergency management, risk reduction and loss mitigation is mandated by the Emergency Management Act),^{Footnote 73} the Comprehensive Nuclear Test Ban Treaty Implementation Act,^{Footnote 74} the Federal Nuclear Emergency Plan,^{Footnote 75} the Federal Policy for Emergency Management^{Footnote 76} and the Department of Natural Resources Act.^{Footnote 77} Additionally, NRCan possesses relevant geoscience expertise and provides a national perspective necessary to promote a unified and best practice approach to risk reduction and loss mitigation across the country. NRCan is the sole federal department responsible for monitoring, notifying and providing technical information about seismic events within or near Canada.^{Footnote 78} On the other hand, despite its monitoring and information-sharing activities, NRCan is not a first agency in Canada to provide information with respect to some geohazards, such as landslides^{Footnote 79} and space weather.^{Footnote 80}

Legitimate role: The vast majority of internal and external stakeholders of both components indicated that there was a legitimate and appropriate  role for the federal government in the hazard activities carried out at NRCan under PSG and CHIS due to the federal government’s legislated mandate to provide disaster assistance to provinces and territories, its unique position to produce standardized and consistent national information and its jurisdiction over offshore areas.

NRCan was also seen as the federal organization best-positioned to deliver on this role. Based on the interview data, PSG either provided unique programming or was complementary to other organizations that had similar programming. External stakeholders noted that PSG collaborations with government, academia, research and industry organizations/networks on geohazards research minimized potential for overlap or duplication. Likewise, CHIS complemented similar programs/services administered by other organizations or jurisdictions. For example, while provincial emergency management organizations had the first-level responsibility for emergency management within their jurisdictions, they required reliable information in order to carry out their responsibilities. As raised by several interviewees, NRCan provided this key information and facilitated a common, best-practice, standard-based approach to mitigation, preparedness and response strategies across all provincial jurisdictions.

The appropriateness of the federal/NRCan involvement in geohazards risk assessment and mitigation was also corroborated by the case studies, which confirmed PSG’s/CHIS’s unique expertise and NRCan’s position to carry out these projects and support the relevant partners. For example, the work carried out under the "Risk Assessment Case Study in the District of North Vancouver", which helped to enhance and validate the previous work of the district on seismic hazards, would not have occurred without NRCan’s assistance. The Department was seen as the only organization having sufficient expertise, credibility and connections to the geo-science community to engage in this type of assessment.

Similarly, CHIS activities under the project "Development and validation of an operational capability to respond to radiological events" concerns national security and public safety and thus requires the involvement of the federal government to ensure the country’s preparedness to respond to natural and human-induced radiological hazards. Also in this case, NRCan contributed unique expertise^{Footnote 81} and survey capabilities that could not be easily secured otherwise.

Despite the strong support for the appropriateness of NRCan’s initial involvement in the selected case study projects and the perceived increasing need for NRCan’s services in the near future, interviewed internal and external case study stakeholders were uncertain whether the Department should be involved in disaster follow-up activities due to a lack of clarity regarding the alignment of NRCan’s mandate and/or the lack of resources.

Clarity of roles and responsibilities: In general, the activities carried out by PSG and CHIS complement each other, as well as similar programming/services delivered by other actors in the emergency response management community in Canada. The transition case study noted that in designing PSG’s predecessor the RRNH, special care was paid to ensure that the RRNH did not infringe on any of the activities of CHIS. PSG was seen as being complementary to CHIS, providing it with new and enhanced products and techniques in order to support its outcomes. Most case study projects benefitted from collaboration and in-kind scientific support from other NRCan sectors, while almost all of PSG case studies reported a significant contribution from CHIS staff, in addition to other NRCan sectors.

Issues regarding the clarity of PSG and CHIS roles and responsibilities were noted throughout the evaluation. For instance, despite the efforts to firmly separate the RRNH’s work from that of CHIS, interviewees in the RRNH review believed that there was still overlap of work and responsibilities between PSG and CHIS due to the intertwined nature of their work. The evaluation found that the working and historical relationship highly influenced their interactions and contributions to each other.

Several of PSG case study projects and the evaluation fieldwork observation also indicated that the collaboration and communication between CHIS and PSG could be improved. In particular, PSG stakeholders indicated that CHIS had become less responsive to PSG needs, following the separation of the two components. This perception might have been caused by increasingly constrained resources on the side of CHIS. Inversely, the planned contribution of PSG to the improvement of CHIS tools and services was not clearly defined. Opportunities for the clarification of the roles of PSG research in supporting CHIS services/tools and vice versa are discussed in more detail in the performance section.

In addition, a need for clarity of the roles and responsibilities of other federal organizations involved was found throughout the evaluation. The federal government had discontinued some work related to geohazards research and emergency management carried out by PSG’s predecessor RRNH, PSG and other federal programs, as they were deemed to be under the jurisdiction of provincial and municipal governments, as outlined, for example, by the Federal Policy for Emergency Management.^{Footnote 82} However, according to external stakeholders consulted, this approach had created gaps in some areas of geohazards research and emergency management work, as provincial and municipal governments did not always have the resources to become involved and replace the role of the federal government in these areas. Similarly, NRCan no longer engaged in Personal Preparedness activities, which were deemed to be in the mandate of Public Safety Canada (PSC). Nevertheless, some interviewees mentioned that the PSC had to suspend its activities in this area due to recent resource reductions.

While it was observed that the work carried out by PSG and CHIS did not overlap with the mandate of other departments or of the PSC, linkages and coordination between the organizations could be improved. Particularly, it was noted by internal stakeholders that the link between PSG activities and contributions to the emergency management policies, standards and regulations of various stakeholders–as well as the link between CHIS activities and facilitation of more proactive, coordinated and timely emergency preparedness, response and recovery efforts–could be significantly enhanced through greater coordination and guidance from PSC and stronger leadership at the federal level.

Given that the PSC was the lead federal government agency for coordinating emergency management activities in Canada, internal stakeholders stated that guidance from the PSC could contribute to greater clarity, consensus and coordination in the fields of geohazards research and emergency management. However, a number of constraints for the PSC in fulfilling such policy-level leadership and guidance activities were identified including the resource constraints and the complexity of the federal emergency management system. As a result, many interviewees recognized that the federal government faced a challenge in trying to unite all of the entities responsible for emergency management, under one collaborative federal approach.

In addition, a number of interviews highlighted that the operational nature of emergency management issues often guided the relationship and coordination efforts among federal organizations. On the one hand, organizations did not have expertise and resources to engage and sustain such policy-level discussions. Established federal coordinating structures often focus on day-to-day operations and tactical issues rather than developing a horizontal strategic vision for a federal approach to emergency management. For example, some internal and external interviewees reported that the ADM Emergency Management Committee (ADM EMC) (a committee coordinating the GC emergency response and chaired by PSC), often focuses on day-to-day operations rather than developing a horizontal strategic vision for a federal approach to emergency management. The interview data identified an opportunity for the Committee to engage in additional policy discussions.

Awareness and communication issues: The coordination of the federal players in emergency response management also faces communication issues. Given the amplitude of emergency programs and functions across participating federal entities, the current communication mechanisms have been reported by interviewees as not meeting the needs. Interview data suggest that there is a general lack of awareness of the key external players involved in emergency functions and activities (e.g., program, policy) across the federal government due to the limited direct contact with the different functions in each department. In particular, awareness has been low with respect to the key activities and appropriate contacts within NRCan, including PSG and CHIS components.

While PSG and CHIS collaborate frequently with OGDs in various projects and initiatives, the evaluation found that the effectiveness and efficiency of the Sub-activity could be further enhanced by strengthening coordination with OGDs that had a mandate related to geohazards, emergency management and/or national security (e.g., Public Safety Canada ; Environment Canada; Fisheries and Oceans; Defence Research Development ; etc.). The level of coordination between CHIS and the Government Emergency Operation Centre of PSC as well as the NRCan Emergency Operation Centre was perceived by some interviewees to be low.

Further, the interview data identified some specific examples where the lack of multilateral communication and information-sharing resulted in isolated work of some federal entities and potential lost opportunities for the federal government to approach emergency management as a whole. For example, the HAZUS tool adapted and tuned by NRCan for earthquake hazards would be equally useful for hazards related to water, which were outside NRCan’s mandate. Similarly, CHIS data could be integrated into hazard-related information systems operated by other departments such as Environment Canada.

While personal preparedness is not in NRCan’s departmental mandate, PSG and CHIS recognize the need to, "…ensure that public safety and emergency management organizations have the geohazard information they require to motivate Canadians to effectively prepare for natural hazard events".^{Footnote 83} In 2010, NRCan surveyed EMOs across the country to better understand how geohazard knowledge and products are currently used in EMO product design and program implementation. This survey confirmed the need for the GHPS Sub-activity to work closely with other federal departments to ensure EMOs have the natural hazard information they need to continue to work with PSC supporting its "Know the Risk" piece and to make sure that municipalities have the geohazard information they require.

According to internal and external interviewees, closer coordination and collaboration with provincial and territorial EMOs and professional organizations (e.g., engineering and planning associations) would allow the GHPS to reach municipalities and local communities through the design work of engineers and planners.

4.2         Performance

4.2.1        Achievement of Expected Outcomes (Issue 4)

Question: To what extent have the activities implemented for each Sub-activity component produced the intended outputs and generated the intended outcomes?

GHPS outputs: The GHPS planned outputs had been produced during the evaluation period or were being produced at the time of the evaluation. There was adequate evidence of the development, compilation, processing and dissemination of the intended outputs under the  Sub-activity. For example, all planned outputs under the six case study projects had been delivered.

PSG outputs include hazard models and maps, hazard and risk assessments, related methods and guidelines, tools (e.g., the Canadian Hazus tool and related activities such as webinars, workshops, a resource website and course curricula), scientific and technical publications and presentations and lesson plans, posters and media interviews.

Lists of publications and presentations were provided by PSG for all 2011-12 projects (National-Scale Hazards Assessment, Quantitative Risk Assessment, National Guidelines for Natural Hazard Assessment and Mitigation, Landslides and Marine Geohazards, and Targeted Geohazard Assessments in Eastern, Northern, and Western Canada). While all PSG program outputs were captured in the GEOSCAN^{Footnote 84} database, PSG scientific publications were not specifically tracked at the program level prior to 2011-12. Thus, this sample did not constitute the whole body of knowledge disseminated by PSG and impeded the review of such PSG outputs.

Table 6 Sample of PSG scientific outputs (peer-reviewed publications and scientific/technical presentations) for three projects, 2011-12

Project area	Peer-reviewed papers	Presentations*
Space Weather	6	13
National Scale Hazards	2	12
Eastern and Western Canada Geohazards Assessment	30**	31***

Note: * Conference presentations, posters and invited workshop and client presentations on seismic hazard; ** 24 published/in-press, 6 submitted; ***, including 12 invited or keynote

Source: Compiled from publication and presentation lists provided by PSG program staff for the evaluation.

With respect to the response of PSG Program to queries external to NRCan, data compiled by the Program provided further evidence that the GHPS Sub-activity was responding to the need of external stakeholders (Table 7). Over the last 6 years, the PSG Program had responded to 1,461 queries from diverse stakeholders groups and to 8,152 queries about earthquake event locations.

Table 7 Queries made to PSG by sector, earthquake location queries, presentations/workshops and tours, 2007-08 to 2012-13

Year	2007-08	2008-09	2009-10	2010-11	2011-12	2012-13(*)	Total
Queries	255	284	203	256	248	215	1,461
Media interviews	39	68	27	55	44	32	265
Emergency	7	7	15	9	8	15	61
Public	100	104	85	101	95	97	582
Gov't/Int'l	45	45	26	34	65	37	252
Education	13	23	33	27	12	19	127
Industry	51	37	17	30	24	15	174
Earthquake location	1,537	1,818	1,397	1,197	1,318	885	8,152
Presentations/ Workshops	9	10	14	25	17	25	100
Tours of facilities	12	8	29	15	21	7	92

Note: *As of December 2012

Source: Compiled from data provided by PSG program staff for the evaluation.

CHIS outputs comprise departmental emergency management plans, geospatial and geomagnetic data, data archives and catalogues, maps and related analyses, seismic information (provided via staff, website and publications/maps), forecasts, alerts and notifications, presentations, joint peer-reviewed publications and media interviews.

Some PSG planned outputs were delayed primarily due to GSC staffing shortages. These delays were noted, at least to some extent, in all hazard areas (i.e., earthquake, tsunami, landslide, space weather and marine geohazards) and in the case studies.

Similarly, CHIS staff reported an instance in which certain stages of a project were delayed, as the staff did not receive necessary information/support from NRCan’s Shared Services Office (SSO) in a timely and effective manner. The performance of the Earthquakes Canada website was flawed during two earthquakes, which attracted media and public attention, and CHIS has taken steps to improve the server capacity of the site in order to prevent such disruptions in the future..

PSG Outcomes: Although progress toward short-term, intermediate and long-term goals has been noted, it is not possible to adequately capture the long-term results of PSG research within the evaluation timeframe of five years. The collection of data for geohazards research could take between six to nine years in some instances. Nevertheless, PSG has received, on average, a relatively high rating from both internal and external stakeholders in the following five outcome areas. These are discussed in more detail below.

• improving the knowledge of the occurrence and impacts of geohazards, as well as mitigation strategies;
• increasing awareness of the risks posed by geohazards;
• improving understanding of geohazards risk reduction and loss mitigation options;
• increasing the adoption of geohazards risk reduction and loss mitigation measures in the development of relevant policies, standards and regulations; and
• optimizing risk reduction and loss mitigation potential for a more resilient built environment in terms of location, design and operations.

Improving knowledge: The geohazards research carried out by GSC staff generated considerable, important and useful knowledge on hazards such as earthquakes, landslides, tsunamis and space weather events as well as their potential risks and impacts. Interview data pointed out that PSG scientists were held in high regard for their work internationally. An example of a knowledge-enhancing project involved PSG conducting research to improve the seismic site provisions in the National Building Code of Canada for soft soils as well as updating provisions for design in the Canadian Foundation Engineering Manual and the Canadian Standards for Highway Bridges.^{Footnote 85} In one instance, PSG collaborative research led to the discovery of a type of earthquake that was previously unknown. That had important implications for the west coast of Canada in terms of emergency management planning and hazard preparedness. Also, as one of the most effective ways to avoid impacts of extreme space weather events was to provide advanced forecasting, PSG scientists were developing several new forecast schemes in order to provide more robust forecasts 24 hours-several days ahead.^{Footnote 86}

As demonstrated in the interview data, information and knowledge products produced by PSG and CHIS were important to Public Safety Canada (PSC), which relied on the GHPS Sub-activity components to fulfill its mandate. Although a high volume of data and information circulated on a daily basis through the PSC operations centre, data and images on earthquakes, flooding and space weather provided by NRCan had been assigned high importance and value due to the specific technical expertise of NRCan staff and scientists and, in some cases (e.g., space weather), the unique access to this data. The level of awareness beyond specific data or expertise was low at PSC with respect to the roles and responsibilities of PSG and CHIS, as well as of their key emergency management activities.

Increasing awareness of the risks posed by geohazards and understanding of geohazards risk reduction and loss mitigation measures: A wide range of PSG activities carried out independently as well as in collaboration (e.g., simulation exercises; media interviews; presentations to academia, governments, industry bodies and communities; provision of hazard information and advice through website, hazard maps, teaching kits, one-on-one interaction, etc.) contribute to raising the awareness of the risks posed by geohazards, particularly on the west coast of Canada. Internal and external stakeholders highlighted PSG collaborative projects, publications, guidelines and tools that facilitated better understanding of geohazards risk reduction and loss mitigation options, particularly among municipal government staff, critical infrastructure operators and industry professionals.

PSG reaches out to stakeholders through surveys/consultations with target audience groups, customized presentations to targeted audience groups as well as conference presentations, public information campaigns and posters, participation in networking and training events and provision of refereed and open file publications. Awards received by PSG staff and frequent solicitation of information from PSG (and CHIS) staff by the media further confirmed a high level of stakeholder awareness and reach. Multiple PSG (and CHIS) publications were among the most downloaded from the online database of the ESS,^{Footnote 87} further confirming their extensive use.

Interviewed stakeholders reported that PSG work had helped to inform relevant risk reduction and loss mitigation policies, standards and regulations of some provincial and municipal governments, pipeline operators and power utility companies and industry regulators. The provincial emergency program in British Columbia, representing the most seismically active region in Canada, was a key user of PSG hazard assessments and regularly sought advice from NRCan related to hazards.^{Footnote 88} Other partners and stakeholders included Hydro Quebec, TransCanada and the Standing Committee on Earthquake Design. PSG staff members also assisted their stakeholders through participation in committees and workshops, as well as in the provision of research papers and presentations. Finally, PSG’s successful partnership in quantitative risk assessment with the District of North Vancouver was noteworthy, as it led to the creation of a comprehensive disaster risk reduction strategy that incorporated responses to earthquake, landslide, debris flow and hurricane scenarios. This emergency preparedness work earned the District of North Vancouver the prestigious United Nations Sasakawa Award for excellence in disaster risk reduction.^{Footnote 89}Interviewees noted that there was potential for PSG work to increase influence on policies, standards and regulations across the country.

Increasing the adoption of geohazards risk reduction and loss mitigation measures: PSG cannot directly influence the policies, standards and regulations of external organizations as it does not have legislative authority or direct control over the activities of other organizations involved in emergency response management. However, there is evidence that representatives from various levels of government, industry, critical infrastructure operators, research networks, academia, media and international organizations receive and utilize PSG information, advice, guidelines and software/tools related to hazard assessments and models, mitigation plans, risk scenarios and building and other construction codes.

The documentary evidence provides numerous examples of adoption and use of the information. Some of the larger projects, such as the advice provided to the seismic hazard assessment project of BC Hydro and Power Authority, have been initiated under the component’s predecessor RRNH. Some concrete examples of the adoption of information and tools produced by PSG over the five-year period are as follows.

• The collaborative process in place for PSG research to inform the National Building Code of Canada has been effective in ensuring that the best scientific information gets embedded into building design and engineering at regular intervals to minimize seismic risks.
• Collaboration with PSG Program helped the District of North Vancouver better understand potential earthquake risks and impacts in terms of physical damage to buildings and infrastructure as well as financial losses and public safety losses.
• PSG space weather research resulted in a real-time geomagnetically induced current (GIC) simulator prototype that is being used by power utility companies to mitigate the impacts of geomagnetic storms on their critical infrastructure. PSG GIC simulator prototype was adapted by Hydro One and integrated into its system control centre so that system operators had immediate updates on the status of the geomagnetic effects on its power system and took appropriate action to protect the system.
• PSG Program has developed a national probabilistic tsunami hazard map for all three Canadian coasts, one of the first of such maps in the world to aid hazard assessments and inform policy-making.

While PSG Program’s authority to directly influence policies, standards and regulation of other organizations involved in emergency management is limited (affecting the achievement of PSG Program’s long-term outcome), the Program has recorded some significant accomplishments in this area. Examples include: the established highly functional collaborative mechanism to ensure that PSG’s research informs the National Building Code of Canada; and other standards and regulations (e.g., Canadian Dam Safety Guidelines and the Canadian Highway Bridge Design Code [CSA-S6]). PSG Program’s continued participation in the Canadian National Committee on Earthquake Engineering and the provision of an updated seismic hazard model for use in the development of the Code are considered key accomplishments of the Sub-activity.^{Footnote 90} Additionally, PSG Program’s research outputs and scientific expertise are integral to many federal Environmental Assessment (EA) review processes.

Other examples of longer term outcome progress include the collaborative projects to showcase and apply PSG methodologies/ tools, among which the most successful have been the risk assessment work with the District of North Vancouver and other similar work in the Vancouver/BC region. More generally, research produced by PSG Program has also been included in land-use policies of regional and municipal jurisdictions:

• The Integrated Partnership of Regional Emergency Managers (IPREM) used three PSG/CHIS-developed earthquake scenarios in its Hazard, Risk and Vulnerability Analysis (HRVA) of the District of Metro Vancouver, undertaken in partnership with Defence Research Development Canada’s (DRDC).
• The City of New Westminster (CNW) used PSG risk assessment tools and expertise in a risk-based land-use planning scenario that considered flooding hazards. NRCan led the exercise for CNW city planners, engineers, geographic information systems technicians and permitting officers.
• These two examples of activities build on four similar tabletop exercises conducted in 2011-12 for other emergency management organizations in B.C. and Washington state (e.g., Heavy Urban Search and Rescue, Canada-US Northwest Coast Earthquake scenario, TransLink (Metro Vancouver’s regional transportation authority)).

PSG/CHIS earthquake information is used by Emergency Management BC in the design  of its public earthquake mitigation program Shakeout BC. Other examples of outcomes produced by PSG component are listed below.

• The National Energy Board (NEB), the Canada Newfoundland Labrador Offshore Petroleum Board (C-NLOPB) and Aboriginal Affairs and Northern Development Canada (AANDC) rely on PSG information in fulfilling their respective mandates.
  • For example, they use PSG information to identify regions characterized by a high degree of geohazards risk or utilize PSG geohazards information to confirm and verify that industry strategies/plans address all known geohazards in a given region.
• PSG products and expertise are commonly included in the regulatory decisions associated with natural resource and critical infrastructure developments – especially as they relate to their environmental assessments (EAs).
  • PSG experts acted as reviewers in the federal environmental assessment of a proposal for a pipeline construction project submitted by Enbridge-Northern Gateway to determine whether or not the engineering standards and design features for the pipelines were adequate to withstand the impacts of various hazards such as space weather events, earthquakes and landslides.
  • TransCanada Pipelines Ltd. also consulted with PSG’s Space Weather group during the design stages for the Alaska Highway pipeline to assess and mitigate hazards presented by the region's high telluric (geomagnetic) activity. A similar assessment was also conducted for the Mackenzie Valley Gas Project’s proposed pipeline route.
• While no longer provided by PSG Program, the Ottawa microzonation map is still used by the City of Ottawa in relation to issue building permits in order to ensure that building design and materials are appropriate in relation to seismic site categories. The map is also regularly used by geotechnical consulting firms in Ottawa.
• BC Hydro and Power Authority applied updated seismic hazard information generated by PSG to retrofit its dams and related infrastructure.
• PSG risk reduction demonstration projects resulted in hazard policy developments in participating communities (e.g., because of the work that started under the RRNH Program and continued under PSG Program, the official community plan of the District of Squamish now requires hazards to be considered for land use zoning).

Industry consults PSG seismic catalogues (seismic maps that contain information on various Canadian building codes, ground acceleration, etc.) regularly for information on hazard mitigation potential when developing various structures. These catalogues are not available from any other source.

Finally, PSG scientists have been approached to provide input into understanding the impacts of seismic activities around the world. PSG has provided such requested input, whenever feasible and has taken advantage of such occasions to apply PSG hazard modelling to real-life scenarios around the world for validation. As well, some PSG research data (e.g., plate motion and gravity measurements) have benefited NRCan’s Geodetic Systems and Infrastructures Section in fulfilling its non-hazard mandate (e.g., water management in Canada) and the Geodetic Systems and Infrastructures Section continue to leverage PSG data.

CHIS outcomes: CHIS stakeholders interviewed assigned highly positive ratings^{Footnote 91} to the components’ activities and achieved outcomes in five areas that are discussed in greater detail below:

• providing more customized and timely emergency preparedness, response and recovery resources;
• providing more timely and open access to geohazards information;
• increasing the use of emergency preparedness, response and recovery resources and information by government, industry, media;
• fulfilling federal legislative and treaty obligations as they pertain to emergency preparedness, response and recovery; and
• provide support for more proactive actions on coordinated and timely emergency preparedness, response and recovery efforts.

Providing resources: CHIS develops efforts to improve alert systems in order to minimize hazard losses and provide data and related products that are used to facilitate research for improving hazard and risk assessments, situational awareness of hazards, modelling and forecasting methods, emergency response and management plans and structural mitigation solutions, such as shelters and barriers.^{Footnote 92} The component also provides customized earthquake notification alerts to critical infrastructure operators. Progress has also been achieved in improving the time interval within which information about a geohazard occurrence is provided to these operators (e.g., the time span to provide earthquake notification to railway operators has decreased from 10-15 minutes to about 4-6 minutes). Additionally, CHIS is capable of providing important intelligence which informs the official response of the Government of Canada (GC) in the event of nuclear incidents.

Facilitating access to geohazards information and advisories: As noted under PSG outcomes, some CHIS publications belong among the most popular downloads from the ESS website.^{Footnote 93} Evidence also points to CHIS improvement in geohazards information dissemination by incorporating new methods to relay data and alerts over time. CHIS employs a number of measures to facilitate more timely and open access to geohazards information, such as the provision of information through multiple means (media interviews, website, telephone, email, fax, RSS feeds and Twitter); mobile communication devices for on-call CHIS seismologists; service agreements with a content delivery network provider to supplement CHIS’s web infrastructure capacity; an automated earthquake alert system; and integration of select CHIS resources (earthquake and space weather alerts, and satellite imagery based flood mappings) into the Multi-Agency Situational Awareness System Development Initiative. Some external stakeholders interviewed also expressed satisfaction with the speed with which CHIS provided the necessary seismic information. The timeliness and accessibility of CHIS information products were also rated highly by survey respondents.

CHIS is continuously working to improve its services. For instance, following the increased traffic on the earthquakes website after the Val-des-Bois, Quebec earthquake in 2010, CHIS implemented a number of changes to improve the robustness of web availability, as well as to take advantage of other complementary earthquake information dissemination vehicles such as social media. This was ultimately to achieve the goal of faster and more authoritative earthquake information for Canadians.^{Footnote 94} Other ongoing efforts to make further improvements to information-dissemination/access include discussions with The Weather Channel to determine how to use this system to disseminate disaster information and discussions with the Canadian Institute for the Blind (CNIB) to determine how to deliver hazard information on CHIS website in a useful manner for visually impaired Canadians.

In the survey results, 75% of surveyed CHIS stakeholders reported having accessed CHIS information products or services. Table 8 provides an overview of services/information most frequently accessed by CHIS stakeholders that responded to the survey. Information on seismic hazards was by far the most frequently sought. As discussed in the limitations section, the survey results have to be interpreted with care because the number of responses was low (33), and there were representativeness issues. Provincial emergency management organizations were not highly represented in the survey population and a large proportion of respondents were not in a position to rate the adequacy of CHIS resources for emergency preparedness, response and recovery.

Table 8 Types of CHIS information products accessed by users/clients surveyed (N=33)

CHIS Information Products	%
Seismic Hazard	58%
Geomagnetic Services	39%
Alert Services	18%
Nuclear Emergency Response	9%
Other	9%
NR	3%

Note: NR=No response; Respondents could select more than one answer.

Source: Survey of users of CHIS information products

In terms of the adequacy of  CHIS information products, three-quarters of CHIS stakeholders (25 respondents) also considered the quality, timeliness/accessibility and the degree of customization of CHIS services/products to be very adequate or adequate, as seen in Table 9While the ratings on the adequacy of CHIS information products were positive, these results should be interpreted with caution. The survey population was not representative of CHIS users and a sizable number of respondents reported not having used CHIS information products during the period or did not know about CHIS products.

Table 9 Rating of adequacy of CHIS information products used by respondents’ organization (N=33)

	1- Not adequate 2- Somewhat inadequate	3- Somewhat adequate	4-Adequate 5- Very adequate	Don't know/Not applicable
Provision of quality resources for emergency preparedness, response and recovery	3%	3%	58%	35%
Provision of customized resources for emergency preparedness, response and recovery	6%	0%	48%	45%
Provision of timely resources for emergency preparedness, response and recovery	0%	6%	55%	39%
Provision of quality geospatial information and advisories	6%	9%	76%	9%
Provision of timely and accessible geospatial information and advisories	6%	10%	74%	10%

Source: Survey of users of CHIS information products

Finally, a majority of survey respondents also rated the quality and timeliness of CHIS service and the quality of their S&T-based products as adequate or very adequate. The improvement of CHIS products over the last five years was rated somewhat lower. More than one-quarter of the respondents indicated that the information products did not need any further improvements or improvements had already been made/were already underway.

Table 10 Extent respondents agreed or disagreed with statements about their interactions with CHIS products and services? (N=33)

	1- Strongly disagree 2- Disagree	3- Neither agree nor disagree	4- Agree 5- Strongly agree	Don't know/Not applicable
CHIS information products are of high quality as they are informed by the latest scientific research and technological advancements.	6%	3%	85%	6%
CHIS information products have improved in terms of customization, timeliness and/or accessibility over the last five years.	12%	15%	60%	12%
The quality and timeliness of CHIS client service is satisfactory.	9%	0%	88%	3%

Source: Survey of users of CHIS information products

Increasing the use of resources and information: CHIS data and services are utilized by different levels of government, emergency measures organizations, critical infrastructure operators, international organizations, the media and the general public. The research component of the Sub-activity through CHIS and its reach to industry sectors also contributed to mitigation plans for several industries, including power generation (e.g., BC Hydro and Power Authority for seismic hazards related to dams) and power systems, pipelines, air navigation and airlines (for hazards from geomagnetic storms).

While it was noted that government emergency management organizations had begun to recognize CHIS as the source of reliable, robust and authoritative information on the hazards that were under NRCan’s mandate, the level of awareness among those who could benefit from some CHIS emergency management resources (satellite images through CHIS Emergency Geomatics Services) varied and was especially low among some provincial emergency management organizations.

Survey respondents were asked to identify and explain operational changes, benefits or other effects that might have resulted from the use of CHIS information products/services. The contribution of  CHIS information products to research programs and services was cited by four respondents (e.g., data provided by  CHIS were critical for conducting research and served as a viable alternative to the very labour intensive task of performing manual surveys necessary for some research programs). Real-time monitoring of geomagnetic activities, earthquakes and related services were also reported as beneficial to respondents’ needs, particularly as it enabled them to provide quality and timely services to their customers such as sending alert notifications. Some respondents indicated that the warnings, knowledge and know-how associated with CHIS information products assisted them in carrying out emergency management tasks (e.g., setting up response procedures, response systems and staff training). Lastly, CHIS contributions to hazard inspection and facility/dam/installation assessment were highlighted by respondents more than once.

Fulfilling federal government obligations: CHIS has developed and maintained monitoring and radiological surveying networks and expertise in compliance with relevant acts.^{Footnote 95} In particular, it has maintained nuclear incident monitoring stations and provided related intelligence/information to relevant organizations within and outside of Canada in order to fulfill the Government of Canada’s obligations under the Comprehensive Nuclear Test Ban Treaty (CTBT). Similarly, CHIS provides emergency support resources under the Emergency Management Act (EMA). The achievement of this outcome was further corroborated by close to 40% of surveyed CHIS stakeholders who agreed that CHIS information facilitated fulfillment of federal obligations under legislation and international treaties. Another 45% of the surveyed stakeholders could not assess CHIS achievement in this area.

Achievement of long-term outcomes: The evaluation evidence is limited with regard to the achievement of the two long-term outcomes of the component: a) more resilient built environment as risk reduction and loss mitigation potential is optimized and b) more proactive, coordinated and timely emergency preparedness, response and recovery efforts. There is limited information on the extent to which CHIS contributed to the work of organizations directly involved in natural hazards emergency response and recovery, such as provincial emergency management organizations.

Half of the surveyed CHIS stakeholders agreed that CHIS information products facilitated more proactive, coordinated and timely emergency preparedness, response and recovery efforts. However, another third was not able to assess the degree of achievement in this area. Interview data provided very few indications of regular interaction between CHIS and the emergency management community to facilitate greater coordination of efforts. Some stakeholders also noted a lack of leadership from the federal government (and correspondingly weaker CHIS efforts) in this area.

Despite the lack of evidence regarding the long-term outcomes, CHIS has recorded an array of activities, outputs and short term and intermediate outcomes. Other examples of these are listed below.

•  CHIS and its predecessors have augmented existing hazard information dissemination mechanisms with new and varied methods over time for relaying information. For example, introducing online earthquake reports in 1996, delivering mobile/Short Message Service (SMS) earthquake alerts in 2000, incorporating Really Simple Syndication  (RSS) feed in 2006 and issuing Twitter earthquake event  feeds in 2011.^{Footnote 96}
• Upgrades to CHIS web service infrastructure, as well as a service agreement with a content delivery network provider to supplement CHIS IT infrastructure, have improved CHIS’ ability to handle a sudden web traffic surge. Web capacity improved from 1.5 million hits per hour to 5 million hits per hour.
•  As per Canada’s obligations under the Comprehensive Nuclear Test Ban Treaty (CTBT), CHIS hosts the National Data Centre that receives data from 16 primary and auxiliary seismic, infrasound and hydroacoustic stations. The data are then transmitted to the CTBTO’s International Data Centre in Vienna.
•  CHIS successfully provided nuclear intelligence and radiological surveying expertise (e.g., baseline mapping of radiological background, scientific reach back to assist detailed assessments of potential radiological threats, etc.) to other government organizations during key events.
• Examples include the North Korean nuclear tests, the 2010 Winter Olympics, the G8/G20 Summits and the Fukushima nuclear reactor incident in Japan (one case study examined the capability of CHIS to respond to radiological events). 
• The information provided by CHIS in the event of a nuclear incident is integrated with the technical and political information from other federal sources so that policymakers will have a holistic understanding, which, in part, informs the Canadian government’s official response.

Question: What are the major internal and external factors contributing to or constraining the performance of the Sub-activity?

Success factors: The performance of the RRNH/PSG Program was enhanced by collaboration with OGDs, other levels of government, industry and universities. ^{Footnote 97} Collaborations with program partners and stakeholders also helped minimize the effects of internal resource limitations. This finding was corroborated by PSG activities reviewed in the case studies. All of these were highly client-oriented and/or involved close collaborations with the project partners to deliver the best results, capitalizing on NRCan’s longstanding experience in the field but also drawing on partner resources and expertise. Several of the case study projects had a formal outreach strategy in place, or the respective GSC staff stepped out of their more traditional scientific roles and engaged in networking and relationship-building with potential clients or intended end-users.

The Sub-activity also adopted an open and flexible approach towards the delivery of its activities, which allowed for easier adoption/greater scope of projects with varied partners. This was best exemplified in a case study, Geomagnetically Induced Currents Simulator, in which the flexibility of the NRCan management structure allowed staff to identify where they could make the best contribution and left it to staff to interact with the different user communities. In this particular case, this choice was proven to be a key success factor, as staff was able to link closely with the engineering aspects of the work, rather than focusing solely on the geophysical aspects. This flexible approach also facilitated the establishment of common goals and collaborative ties, sharing of best practices and leveraging of complementary resources and expertise vis-à-vis a diverse group of PSG and CHIS stakeholders within and external to Canada.

One of the most significant success factors for both PSG Program and CHIS related to their committed staff and unique expertise. The skills and dedication of the staff were deemed instrumental in achievement of the intended outcomes, particularly as the staff was able to deliver these outcomes with limited resources. The expertise of the staff was particularly valued by external stakeholders or partners who noted that the science produced by NRCan was credible and the scientists who produced it were in some cases world-renowned in their respective fields. The shortage of available qualified staff was also a serious limiting issue to the timely delivery of most PSG activities.

PSG Program maintains a clear focus on user needs in the process of operationalizing its research and applying the research findings. This helps to ensure that the user needs are being adequately addressed and that the knowledge/technology transfer mechanisms are effective. An example of such successful efforts is the validation of hazard modelling/forecasting and simulation tools.

Finally, CHIS mandate related to the provision of hazard information received strong management support, which helped raise the profile of CHIS. As well, the separation of CHIS from the research component of the Sub-activity (PSG) resulted in increased robustness of the 24/7 operations that responded to ministerial responsibilities under several acts, along with greater clarity of objectives and a clearer results chain within the program logic model.^{Footnote 98} While benefitting CHIS’s mandate, as noted above, the separation of PSG and CHIS constrained, to a certain extent, PSG Program’s ability to conduct research. PSG interviewees reported a lack of local technical support from CHIS to maintain and repair research stations and instruments, which impacted the quality, breadth and depth of the seismic data used to develop seismic hazard maps.

The level of interaction and integration between the two Sub-activity components is further discussed in the next section, which is dedicated to this issue.

Constraining factors: Both Sub-activity components, particularly PSG (and previously RRNH) Program, operated with limited internal resources, which was reported to have had a negative impact on the success of their activities. Both internal reports^{Footnote 99} and PSG stakeholders noted that the scope, breadth and depth of the intended PSG outcomes were not reflected in the level of NRCan resources. This had been a constraint for the RRNH Program as well.^{Footnote 100}

In addition, both components experienced staffing constraints, which in some cases resulted in abandoned and delayed projects, particularly in the case of the RRNH and PSG Programs, which experienced key staff turnovers.^{Footnote 101} For CHIS, the shortage of staff contributed to delays in decision-making. From the perspective of some interviewees, the head of CHIS had too many responsibilities and could not always respond to staff queries in a timely manner. Hiring additional staff would expedite decision-making.

Also, CHIS does not have the resources for dedicated day-and-night staff for earthquake notification in real-time. Instead, it uses on-call seismologists, but CHIS has only four staff members to cover the on-call roster.

It was further noted that both Sub-activity components lack effective succession planning and had a limited ability to hire staff (and students on a temporary basis) effectively due to the complex hiring policy and system in place. As a consequence, in some instances, staff left or moved to other positions and the vacant positions were not filled. These constraints also clearly impacted most of the six case study and, in some cases, they reached critical levels. These experienced serious delays in their outcome delivery. The staffing shortage did not compromise the final quality of the outputs as the Sub-Activity was able to leverage additional resources through agreements with external stakeholders.

Finally, the federal web accessibility standard and guidelines^{Footnote 102} restricts the type of hazard information that can be made available online, as the formatting of all web content needs to accommodate a diverse group of users. These guidelines have had an impact on the information traditionally provided on the web to PSG/CHIS stakeholders and limited the dissemination of new information. On a related topic, a PSG website on landslide guidelines and best practices was taken offline to accommodate federal web content accessibility guidelines. It has not been updated in two years.

From the external perspective, a number of stakeholders interviewed identified many of these constraining factors:

• "The Sub-activity seems to have less funding and fewer staff in recent times. CHIS/PSG work has been delayed for six months due to inadequate staffing"
• "CHIS are operating at maximum capacity. Seismic data quality or frequency has not been affected but on the technical support side, it now takes longer for  CHIS to respond to support requests"
• " CHIS seems to have insufficient resources/personnel to make seismic data available to the public  including academia in real-time, compared to the USGS"
• "Due to inadequate PSG staffing, some of the outputs of a collaborative project have taken a lot longer than originally anticipated"
• "CHIS website used to have more detailed earthquake information even two or three years ago. The information has been reduced partly due to federal web content accessibility guidelines and partly due to a past incident when the website could not handle the flow of traffic following an earthquake near Ottawa"
• "A website on PSG landslide guidelines and best practices was taken offline to accommodate federal web content accessibility guidelines and it has not been updated for two years. Many landslide professionals are aware of PSG work in developing landslide guidelines and best practices but they do not have up-to-date information. The website could have been useful in addressing that need."

A lack of coordination among the key federal government departments and agencies that have a mandate related to geohazards, emergency management and/or national security was noted as a constraint for both PSG and CHIS efforts. In particular, the absence of a federal harmonized and coordinated approach to emergency management activities was identified as a constraining factor for CHIS’s ability to facilitate more proactive, coordinated and timely emergency preparedness, response and recovery efforts.^{Footnote 103} The effectiveness and efficiency of the Sub-activity could be further enhanced by strengthening collaborations with other NRCan sectors or programs (e.g., the climate change program) or the academic research community. Some stakeholders across groups and components also noted the need for strengthened PSG and CHIS ties to other federal government departments that had a mandate related to geohazards, emergency management and/or national security (e.g., Public Safety Canada, Environment Canada;, Fisheries and Oceans, Defence Research and Development Canada, etc.). One way to enhance interdepartmental collaboration could be to adopt the staff secondments arrangement currently in place between Public Safety Canada and the Department of National Defence by encouraging staff exchanges among organizations working in related areas. A similar model (e.g., hosting engineers) and closer collaboration could also be applied in relation to professional organizations such as engineering and planning associations so that PSG knowledge could indirectly influence municipal and local government policies through the design work of engineers and planners. Stakeholders across all groups noted that with more resources, both components would be able to provide better services/resources.

Further, PSG internal interviewees noted that PSG contributions to the emergency management policies, standards and regulations of various stakeholders (with the exception of the National Building Code of Canada) were constrained because of a lack of guidance from the GC in general, as well as from the PSC. For example, the PSC’s work on risk mitigation has advanced slowly.^{Footnote 104} This has restricted PSG agenda due to a lack of clear priorities vis-à-vis hazard preparedness and loss mitigation.

Examples of the impacts of scope change on the Sub-activity’s activities include the following:

• PSG lost funding for personal preparedness activities in 2012, which resulted in the phasing out of some activities and terminating some activities earlier than planned;
• a four-year marine PERD geohazard project will end two years in advance as it is no longer deemed relevant to the revised PSG mandate; and
• due to a change in focus, no new microzonation maps will be generated under PSG.

The transition from RRNH to PSG and the accompanying shift of focus towards producing and feeding into national level products resulted in the decreased participation of a number of RRNH stakeholders that represent more regionally specific and targeted outputs. The change in direction and limited resources also led to some lost opportunities to build on previous work. In addition, it was noted by internal representatives interviewed for the RRNH transition case study that the RRNH was constrained by a lack of clear program direction, expected activities and actual allocation of resources from senior management, including the balance and nature of the program’s fundamental and applied science projects.^{Footnote 105} Some stakeholders shared a view that the new program (PSG) was moving away from areas where it could clearly demonstrate reduced risk across the country at the regional/community level (e.g., developing municipal-level seismic hazard maps). It should be noted that the resources available to the Program over the last five years would likely prevent the staff from developing an appreciable number of these maps.

Best practices/lessons learned: Some of the tools and mechanisms developed under PSG Program and CHIS were identified as best practices by organizations external to NRCan that also adopted them to enhance their own hazard-related work. For instance, the ESS structure for emergency plans, which formed the basis for NRCan’s planning framework, was adopted in essentially unmodified form by Public Safety Canada as the best practice and de facto standard for federal emergency plans under the Emergency Management Act (EMA).^{Footnote 106} As well, the hazard assessment methodology developed by PSG Program for the proposed Alaska Highway pipeline was adopted by the U.S. Geological Survey (USGS).^{Footnote 107} On the other hand, NRCan also incorporates lessons learned from dealing with emergencies elsewhere in the world. For example, the Sub-activity’s program plans were adjusted following earthquakes and tsunamis in Chile, New Zealand and Japan.^{Footnote 108}

Finally, for both PSG and CHIS, stakeholders interviewed called for continued, better and increased outreach activities in order to ensure dissemination of targeted information on PSG mandate, priorities and (actual and potential) contributions both within and outside of NRCan. In particular, better dissemination of mandate and priorities to other federal organizations would help GHPS in its efforts to improve coordination of its activities with the key federal government departments and agencies involved in emergency management and/or national security.

Question: To what extent did the level of interaction and integration between the two Sub-activity components (as well as with other Sub-Activities and stakeholders) contribute to the production of measured outputs and the achievement of outcomes?

Multiple lines of evidence describe PSG’s collaborations with different levels of government, academia, industry and research networks. This has facilitated and/or funded research work and joint research outputs, sharing of research data, access to research instrumentation, hiring of research staff and consultation with user groups. CHIS also engages in collaborations with other government programs, academia and research networks in order to facilitate research and monitoring work and improve the provision of hazard information, alerts and forecasts.

The collaborations of both components with external stakeholders enabled significant leveraging of funding, staffing, complementary knowledge and expertise, instrumentation and technology, and data and data networks. As a result, PSG Program was more effective in carrying out research, applying research knowledge to improve geohazards risk/impact assessment and hazard loss mitigation potential and disseminating research findings/outputs and related tools. Likewise, CHIS achieved greater effectiveness in improving hazard monitoring and forecasting, applying uniform data standards, providing emergency management support resources and conducting radiological research because of external collaboration efforts. Many of the collaborations carried out by either component took place because PSG or CHIS staff had good rapport with the partner organization, or they were a result of unplanned interactions with representatives of these organizations at conferences, via participation in a committee, through the NRCan website and other means.

The evaluation data indicate that both PSG Program and CHIS have been more effective in collaborating with external partners than working with one another, despite the fact that there are interdependencies between the work of the two components. For example, access to CHIS seismic network, related instrumentation and technical staff is paramount for PSG research, whereas PSG research is intended to improve CHIS hazard monitoring and forecasting. In addition, it was common for PSG and CHIS external stakeholders to report interacting with both PSG and CHIS staff and/or accessing/utilizing both PSG and CHIS outputs and services.

While there was an annual interaction between CHIS and PSG (in which the managers of both components discuss common directions and how one could support the other in the fulfillment of their respective priorities), the evaluation found, based on interview data and observations, that PSG and CHIS approached issues from different perspectives, resulting in both components setting and implementing their priorities independently of one another.

Interviewees from both components also observed gaps and lost opportunities in the contributions of CHIS to PSG, as CHIS tended to only support the activities of PSG Program to the extent that its resources allowed. Inversely, PSG’s research did not significantly contribute to the improvement of CHIS operations. On a positive note, some interviewees highlighted some instances demonstrating efforts to achieve a higher degree of integration of the activities of each Sub-activity’s components. For example, following the Haida Gwaii earthquake in October 2012, PSG Program planned to work with CHIS and the Canada Centre for Remote Sensing (CSRS) for marine and land-based aftershock surveys.

Promoting greater alignment and integration between the two Sub-activity components are the most frequently cited suggestions by both PSG and CHIS internal stakeholders to improve the Sub-activity. As mentioned earlier, there are opportunities for the clarification of the roles and responsibilities of PSG research in supporting the improvement of CHIS services/tools and, similarly, for the clarification of the role of CHIS in providing services to PSG. On the one hand, PSG is conducting an array of activities from basic to applied research with multiple stakeholders without having a formal joint CHIS/PSG research plan for the improvement of CHIS information products and services. Such internal activity would benefit from being jointly prioritized, planned and formally integrated into the Program Sub-activity logic model. As CHIS is providing services to several external organizations, PSG’s expectations for CHIS internal services should also be managed with a more formal prioritization process and internal service coordination.

Further, with respect to strategic planning, a documented need was found for improved coordination between planning for emergency management within the ESS (the responsibility of CHIS) and business continuity planning of NRCan in order to avoid disconnection.^{Footnote 109}

Question: Have the allocated financial resources and human resources (FTEs) been adequate for the activities of each Sub-activity component?

The evaluation concluded that the Sub-activity operates efficiently given the level of achievements in relation to costs. Both components operate with limited resources while mandated to deliver wide range of products and services which rely on sophisticated technologies.

PSG operated with fewer resources than originally allocated—over the evaluation period, planned expenditures exceeded actual by $6 million. The actual expenditures of CHIS exceeded the planned expenditures by $14.1 million. To examine the costs of producing outputs and outcomes in more detail, five themes are examined below: a) overall expenditures, b) adequacy of resources/staffing levels, c) external revenues/leveraging, d) the cost-benefit of preventative measures and e) suggestions for improved efficiency.

a) Expenditures: Based on available financial data the total expenditures for the Sub-activity increased from $9.8 million in 2008-09 to $15.1 million in 2011-12, which represents a 35% increase. This increase was in part a result of PSG program gaining 10 FTEs with the addition of the Marine Geohazards project, which increased the salary amounts associated with the program in 2011-12. A similar increase was observed for each of the two components as shown in Figure 2. Specifically, CHIS A-base expenditures increased 40% between 2008-09 and 2009-10 while PSG A-base expenditures increased almost 38% between 2008-09 and 2010-11. For CHIS a large but decreasing part of the expenditures was related to O&M ranging from 53% in 2008-09 to a projected 30.4% in 2012-13. Decrease in expenditure from 2011-12 to projected 2012-13 is mainly due to a decrease of vote netted revenue (VNR) and grant and contributions (G&C).

In the case of PSG, this increase is mainly due to the transition from the RRHN Program to the current version, PSG Program. This increase is explained by the increase of FTEs from 32-33 to 40-42 FTEs. In addition, the increased acquisition of external financial resources by both GHPS components contributed to increased total expenditures in 2011-12.

Figure 2 Expenditures of GHPS and by component, 2008-2009 to 2011-12

Text version

Figure 2 Expenditures of GHPS and by component, 2008-2009 to 2011-12

Expenditures for CHIS were just over $6 million in 2008-09, rose to just under $8 million in 2009-10, increased to just over $8 million in 2010-11, and were at $9 million in 2011-12 (approximately).
PSG expenditures were just under $4 million in 2008-09, rose to $5 million in 2009-10, increased again to $6 million in 2010-11, and were at just over $6 million in 2011-12 (approximately).
Expenditures for GHPS were just under $10 million in 2008-09, rose to just over $12 million in 2009-10, increased again to just over $14 million in 2010-11, and were at $15 million in 2011-12 (approximately).

Source: Compiled from financial information provided by program management

b) Adequacy of resources/staffing: In some instances, inadequate staffing caused significant delays (up to a year) in carrying out planned work in all PSG geohazards areas (i.e., earthquake, tsunami, landslide, space weather and marine geohazards). The shortage of staff was noted not only by internal stakeholders, who in several cases took on additional responsibilities to their regular duties, but also by external stakeholders in the case study reviews. In one instance, resources to hire staff for earthquake research became available in the last year of the evaluation period, even though the need was identified in 2009-10. As illustrated in Table 1, more resources to hire staff were available in the later years of the evaluation period, particularly in 2010-11 and 2011-12, partially due to contributions from other federal departments.

Both PSG internal and external stakeholders noted that the resources allocated to PSG were not adequate (particularly for earthquake, landslide and marine geohazard research) given actual mandate and expectations. CHIS stakeholders also noted that the resources allocated to CHIS were not adequate, particularly on the seismic side. It should be noted that despite these reservations, the funding of the Sub-Activity has remained at a level where the core functions of the PSG and CHIS could be completed. Additional research activities and further value-added work done to extend the reach of the two components beyond their core mandates has only been sustainable through the use of external contributions.

c) External revenues and leveraging: The delivery of the planned outcomes, albeit with some delays, was largely possible due to the capability of both PSG and CHIS to leverage significant external funds through their collaboration with diverse partners and stakeholders.^{Footnote 110} PSG projects received external funding from the Canadian Space Agency and the Department of National Defence and in-kind contributions from Public Safety Canada. Resources were also shared in a collaborative project with Ontario Hydro One, which also provided in-kind support. Other projects involved in-kind contributions from provincial emergency organizations such as British Columbia Provincial Emergency Preparedness or financial contributions from other NRCan groups.

CHIS’s closest collaborators were the Royal Canadian Mounted Police (RCMP) and DND; however, some activities also involved Health Canada (HC), Defense Research and Development Canada (DRDC), National Research Council (NRC) and the Canadian Nuclear Safety Commission (CNSC). Almost all activities analyzed in case studies (excluding the RRHN transition case study) for both components leveraged in-kind contributions from other sectors of NRCan and/or external partners. Financial leveraged resources included funding for equipment purchases, research grants or graduate students. 

In 2012-13, the total revenues from other sources for the Sub-activity were approximately $2.6 million, which represented 17% of the total GHPS expenditures (Figure 3). TBC For the same year, the total revenues for PSG and CHIS reached 25% and 12%, respectively. Funding from other government departments (OGDs) is the main source of external revenues for PSG and the sole source for CHIS. It accounted for about 15% of total expenditures of PSG in 2012-13.

CHIS and PSG conducted minimal cost-recovery for information services and products. Such additional potential revenues were considered by CHIS for specific services. While internal stakeholders mentioned that users were used to free access to such services and consider them to be public goods, GHPS could explore cost-recovery opportunities for specific services within the limit of its mandate and NRCan policies.

Figure 3 External revenues* as Percentages of GHPS Expenditures, 2008-2009 to 2011-12

Text version

Figure 3 External revenues* as Percentages of GHPS Expenditures, 2008-2009 to 2011-12

* Includes funds from other government departments, specified purpose accounts and cost recovery.
CHIS percentage of external revenue was 17% in 2008-09, fell to 9% in 2009-10, rose to 22% in 2010-11 and then fell again to 16% in 2011-12.
PSG percentage of external revenue was 12% in 2008-09, fell to 7% in 2009-10, rose to 12% in 2010-11 and then rose again to 25% in 2011-12.
GHPS percentage of external revenue was 14% in 2008-09, fell to 8% in 2009-10, rose to 17% in 2010-11 and then rose again to 19% in 2011-12.

* Includes funds from other government departments, specified purpose accounts and cost recovery.
CHIS percentage of external revenue was 17% in 2008-09, fell to 9% in 2009-10, rose to 22% in 2010-11 and then fell again to 16% in 2011-12.
PSG percentage of external revenue was 12% in 2008-09, fell to 7% in 2009-10, rose to 12% in 2010-11 and then rose again to 25% in 2011-12.
GHPS percentage of external revenue was 14% in 2008-09, fell to 8% in 2009-10, rose to 17% in 2010-11 and then rose again to 19% in 2011-12.

* Includes funds from: other government departments; specified purpose accounts; and cost recovery

Source: Compiled from financial information provided by program management

After the initial increase between 2008-09 and 2009-10, the A-base expenditures as a percentage of total expenditures declined in 2011-12 to 73% (vs. 94% in 2009-10) and 62% (vs. 73%) for PSG and CHIS respectively, a reflection of the increase in resources from external sources.

d) Cost effectiveness of preventive measures: The Sub-activity’s research also generates external benefits, the most salient example being the seismic research conducted under the RRNH Program and CHIS that informed the National Building Code. It is believed that modifications to the Code based on this research resulted in reduced building losses from shaking and fire damage, as well as reduced loss of life and injuries. NRCan estimated that the safety benefits, net of the cost of new construction to comply with the modified building code, represent an annualized value of $42 million (in constant 1992 dollars). This amount represented ten times the cost of the associated earthquake research and did not take into account potential cost savings from modifications to other infrastructure codes (e.g., bridges and dams).^{Footnote 111}

This estimate further supported the conclusion of various international organizations and documents that pro-active, preventative disaster mitigation measures were more cost-effective compared to reactive post-disaster activities. Countries that were well-equipped and prepared for natural disasters ultimately fare better in reducing costs on relief, reconstruction and recovery efforts and they suffer fewer casualties.^{Footnote 112}  One 2009 American study^{Footnote 113} estimated that one dollar spent on prevention was more than ten times more valuable than a dollar spent on relief in net present value. Geoscience Australia estimated that additional investment in natural disaster mitigation by all levels of government provided a 15% rate of return. Another analysis found that for every dollar invested in flood mitigation, more than AUD$2.10 was saved.^{Footnote 114} Finally, the World Bank estimated that for every USD$1 spent in risk mitigation and disaster preparedness, $USD 7 was saved in reduced costs of response and recovery.^{Footnote 115}

Qualitatively, some PSG internal stakeholders/external stakeholders stated that PSG provided the best possible value for money given the level of its achievements with limited resources. They also noted that it was essential for the component that external resources supplement internal resources, but they highlighted that PSG would not be able to effectively absorb any further external funds without a corresponding increase in staff capacity to assume the additional work.

On CHIS side, both internal and external stakeholders stated that CHIS provided the best possible value for money given the level of its achievements with limited resources. It was also highlighted that increasing dependence on external revenues could constitute a risk for the mandatory activities for which CHIS was responsible.

In sum, both components need to obtain additional external funding to supplement internal funding to deliver their programing as actual resources do not allow the flexibility to PSG and CHIS to implement identified best practices and to further engage in horizontal federal initiatives.

e) Improving GHPS efficiency: Despite the generally efficient operations of both components, it was found that nominal efficiency gains could be achieved by closer integration and coordination between the two components and among their locations. Particularly on the PSG side, some internal stakeholders noted the need for closer collaboration, which would include annual common planning of priorities and activities of the two components, as well as a more open exchange of information on the capacities and resources of each of the components.

On CHIS side, internal interviews identified a need for PSG to balance its basic and applied research for the benefit of CHIS operations and services. The interview data also indicated that "talks" and meetings between the leadership of the two components had been scheduled to establish integrated planning and execution of common priorities. As mentioned earlier, the evaluation that PSG and CHIS approach issues from different management perspectives resulting in both components setting and implementing their priorities independently of one another.

A discussion of efficiency and economy must also include an examination of management structures and mechanisms. In this regard, there is evidence of formal processes implemented under both PSG and CHIS to plan and prioritize their respective activities (e.g., through the ESS Executive Committee).^{Footnote 116}

For PSG, these processes included mid-year and year-end reviews, annual review of Program activities and monthly meetings on projects. Several internal PSG stakeholders stated that effective structures and processes were in place to ensure optimal resource allocation and utilization, noting that senior management made resource allocation decisions in the context of ESS/NRCan programming, as well as other relevant federal/provincial government programming.

In addition, at the end of each fiscal year, PSG undergoes a formal program budgeting exercise for the following fiscal year, during which a Work Plan is also established. During this exercise, Project Leaders are asked to evaluate and rank each activity in their project in terms of its relative priority and performance. The priority is established based on criteria such as: contribution of project activity to the PSG program; appropriateness of Federal/GSC Role; relevance; and contribution of activity to GSC Strategic Plan. The performance is assessed against the following criteria: effectiveness; efficiency; management processes in-place & working; and contribution of activity to the advancement of science. During the budgeting exercise, Project Leaders are also invited to table ideas and proposals for new or continuing projects. The selection of supported is then primarily made based on an activity’s potential for geohazards risk reduction.

Planning exercises for CHIS include weekly and annual staff meetings, production of capital and ongoing operational plans with input from all staff, and creation of annual work plans per CHIS group. Consequently, all significant budget variances, major budget changes or staffing issues require discussion with the head of CHIS.

Due to a lack of information, the evaluation could not determine how CHIS projects and activities were prioritized and planned at a more strategic level, such as at the level of ESS as a whole.

Some form of project-level management structure/performance monitoring was also identified in all six case study projects. These included indicators such as:

• the count of publications and presentations;
• monitoring uptake of knowledge/products/tools by clients and international community;
• popularity of website applications or monitoring of activity on  CHIS Twitter account;
• after-action reports; and progress updates during weekly meetings;
• mid-year review checks against project milestones; and
• annual reports to NRCan and other departments that fund joint projects and activities.

Lessons learned from RRNH: The RRNH underwent a performance review in its final year, which was expected to inform the development of a new program. The review advocated for hazards work with a national perspective (as opposed to both regional and national as was the case under RRNH) to achieve greater efficiencies.^{Footnote 117} This shift in program focus was seen as a way to direct limited available resources that would produce results for all provinces and municipalities. Given the potential for funding reductions, management identified key areas where the greatest impacts could be generated. This formed the basis for a shift towards high-level projects which would feed into national codes, standards and regulations. The consultations undertaken as part of the review also led to the development of the four major activity areas of PSG program: national hazard assessments, targeted hazard assessments, natural hazard risk assessments and influencing personal preparedness for natural hazards.

The transition process and planning of PSG Program was supported by several factors. With ESS management taking on a direct role in the development of the Program, the new PSG logic model and performance indicators became much more grounded in the realities of NRCan. Ensuring that the program was clearly tied to the department’s strategic objectives and that there was a clear link between the activities to be undertaken and the larger performance measurement strategy made PSG a much more focused version of RRNH.^{Footnote 118}

Existing performance measures: Finally, in examining efficient and cost-effective Sub-activity operation, the evaluation found that performance measurement, management and reporting were taking place but could be improved as outlined below.

A set of service standards for  CHIS component has been developed and includes measures such as the 24/7 operation of the earthquakescanada.ca website, earthquake notification to emergency measures organizations and critical infrastructure operators sent out within 4-10 minutes after the earthquake, all sensor data from permanent seismic stations available to the public within 30 minutes after acquisition, etc.^{Footnote 119} CHIS collects not only performance information on the timeliness of earthquake alerts and notifications but also the adequacy and effectiveness of monitoring networks. Despite this, the evaluation could not make a conclusive statement about whether or not CHIS had any target benchmarks and/or standards for such performance data and other key performance indicators on CHIS outputs and outcomes.

Some PSG measures reported in the departmental performance measurement framework included: number and scope of new or updated mitigation and emergency plans using ESS hazards assessment information;^{Footnote 120} data and information are accessible electronically;^{Footnote 121} and percentage of outputs completed or on track.^{Footnote 122} Some interviewees also added that the number of PSG peer-reviewed publications served as a proxy performance indicator. The peer-reviewed publications are not systemically complied and analyzed from a performance standpoint and most internal stakeholders found the current performance measures inadequate. A PSG progress report also noted that a new tracking system had been implemented in October 2009 in order to document meetings with stakeholders for communication of hazard information to support risk reduction. Many less formal contacts could not be documented.^{Footnote 123}

Challenges to performance measurement: The evaluation examined whether necessary mechanisms were in place to efficiently manage the two Sub-activity components. It was found that these could be improved with better performance measures and performance management practice. In particular, the mid-year and year-end PSG and CHIS progress reports did not report against targets, specific research performance indicators or service standards. Rather, they summarized the activities carried out during the period. In addition, information was not always collected and reported in a consistent manner to allow for comparisons across periods or against overall objectives.

Internal stakeholders reported a long-term effort to develop sound metrics for both components, particularly the high-level DPR metrics. However, it was considered a challenge to measure the results of activities that focused on the prevention of events that might not happen. Both components seek to avoid costs associated with geohazards and until these hazards have occurred, it is difficult to determine the investment that would be necessary to avoid the cost. The case study in which PSG collaborated with the District of North Vancouver was a good example of a natural disaster scenario that aided in estimating future costs. More scenarios would be necessary to obtain a more accurate estimate.

In addition, given the nature of the Sub-activity’s objectives and activities, the logic model that traditionally serves as the starting point for designing performance indicators may not work well for a service-oriented program such as CHIS. Instead, internal stakeholders expressed the need to align performance metrics with a risk assessment framework that targets emergency management support activities within the department to make these metrics more useful for the management of the components. This avenue would provide value to the management of the component and would improve the compilation and reporting of data from existing databases and systems.

Further, due to the research nature of the programming, challenges and difficulties associated with performance measurement were noted by PSG/RRNH researchers. In particular, their projects were producing information that was capable of influencing products and decision-making across the country but which was almost impossible to trace back to an individual project output in less than five years. Additionally, the RRNH end performance review noted that efforts to increase the number of hazard assessments used in mitigation efforts to reduce risks had not been achieved in a measureable way and was most likely not a workable goal given the budget and research nature of the program.^{Footnote 124}

At the onset of the evaluation period and PSG programming, the component implemented a tracking system^{Footnote 125} and developed noteworthy efforts to formally document how various stakeholders utilized PSG research outputs and identified any associated impacts. However, based on information collected by interviews, such efforts were discontinued as too time-consuming, especially in the context of inadequate staffing and limited resources. As a result, the evaluation found that basic research performance indicators were lacking and not systematically collected, reported and used for the management of PSG.

There is a need to develop a strategy to better track the interaction between the two components to inform a discussion on how to improve mutually beneficial collaboration and efficiency at the level of the Sub-activity.

Lack of clear profiles for GHPS and its components: The evaluation found that the activity profiles of CHIS and PSG documented internally and communicated externally were not complete or up-to-date. Such detailed activity profiles could help clarify the roles and responsibilities internally, for activities conducted individually, in collaboration or any service or expertise dependent activities. Another version of the profile could be developed for external public diffusion in order to improve the lack of awareness from external stakeholders identified during the evaluation.

Question: Are there alternative approaches to the current design and delivery of the Sub-activity that would allow it to meet its objectives and mandate in a more effective and efficient manner?

To address this evaluation question, alternative approaches and best practices in four other jurisdictions were examined (Australia, New Zealand, Switzerland and the United States). It is important to point out that the different international practices best suit each country’s context. Therefore, when making comparisons, the evaluation recognizes the contextual singularities in which entities responsible for emergency management operate. The differences in the scope, breadth and depth of geohazards activities carried out by NRCan’s GHPS, as compared to the other countries examined, were taken into consideration in the following findings.

Coordinated approach

As in Canada, all four of the jurisdictions examined have made efforts to apply integrated, ‘all-hazards risk’ approaches that combine the various elements of hazard risk reduction,( i.e., address various natural and human-made hazards in a uniform manner). The review suggests that other jurisdictions were more proactive and effective than Canada in coordinating key emergency management players and engaging a broader range of stakeholders. While PSG and CHIS have been active in all of these areas, Canada’s efforts lag behind those in other jurisdictions in some of the areas, particularly in terms of stakeholder coordination and active community involvement.^{Footnote 126}

At the federal level, Canada also appears to lag somewhat in the level of coordination of various emergency response actors that can be found in some other jurisdictions, particularly in Australia. Australia’s emergency response system is based on the National Strategy for Disaster Resilience^{Footnote 127}, a comprehensive, long-term, multi-hazard approach to disaster risk reduction that integrates all risk considerations, with a focus on disaster prevention, mitigation, preparedness and vulnerability reduction. The Strategy was introduced as a response to a fractured system in which disparate policy agendas and competing priorities resulted in gaps and overlaps in natural hazard preparedness and management.

In Australia, the Standing Council on Police and Emergency Management (SCPEM )^{Footnote 128} of the Council of Australian Governments and a hierarchy of committees underneath it, bring together the various levels of government, identify specific needs of the government in areas of hazards research and set the collective emergency management agenda. In particular, the goals of the SCPEM are to:

• promote a coordinated national response to law enforcement and emergency management issues;
• provide a framework for cooperation and shared strategic directions for the policing and emergency services of Australia and New Zealand; and
• encourage and share best practice in police policy and operations and in emergency management, across jurisdictions.

Another approach is New Zealand’s National Civil Defence Emergency Management Strategy (CDEM). The supporting CDEM legislation aims to integrate and coordinate hazard and emergency management arrangements, policies and activities across the "4 Rs": reduction and emergency readiness, response and recovery. Recent CDEM reforms have further aimed to integrate the planning and coordination of resources across agencies and service providers. In particular, the CDEM ‘Cluster Approach’ aims to reinforce existing multi-agency relationships by clarifying goals, responsibilities and roles for disaster management; identifying gaps in capability and capacity; and addressing these gaps through action plans. These clusters include: CDEM Groups, Public Information & Education, Emergency Services, Research & Science, Health, Business Community, Welfare, Socio-Economic, Transport, Agriculture & Rural, Lifeline Utilities (by sectors) and International Assistance.^{Footnote 129} The Natural Hazards Research Platform^{Footnote 130} is aligned with this strategy and other initiatives of government agencies responsible for reduction, readiness, response and recovery from natural hazard events.^{Footnote 131}

Another example of a highly integrated national approach to disaster risk reduction is the Swiss National Platform for Natural Hazards (PLANAT). PLANAT assumes an integrated and holistic risk management in which all types of measures for natural disaster reduction - preparedness, response and recovery (reconstruction) are considered, though its primary focus has been on the long-term prevention and mitigation of natural risks. Its mission is three-fold: to advise the federal government on strategic matters related to disaster risk reduction; to coordinate work in this field; and to raise awareness and promote a long-term shift towards averting danger through investment in disaster risk management.^{Footnote 132} PLANAT’s 18 members come from all Swiss regions and sectors and possess complementary expertise across the fields of natural hazards and disaster risk reduction.^{Footnote 133} PLANAT also cooperates with other organizations and institutions that work in the area of natural hazards, including the National Association of Natural Hazards experts Fachleute Naturgefahren Schweiz (FAN), the Working Group on Geology and Natural Hazards Arbeitsgruppe Geologie und Naturgefahren (AGN) and the Natural Hazards Platform of the Alpine Convention (PLANALP).

Stakeholder and community engagement

To various extents and degrees of success, the reviewed jurisdictions have also established mechanisms that facilitate interaction and knowledge transfer among different program components and key stakeholders such as the PLANAT platform in Switzerland. Further, the federal government of New Zealand established the Natural Hazards Research Platform^{Footnote 134} to help research providers and end users work more closely together. Today, the platform is led by a Technical Advisory Group (made up of representatives of Platform partners), a Strategic Advisory Group (made up of end users) and a Platform Management Group (made up of the CEOs of GNS Science, the National Institute of Water and Atmospheric Research and the Ministry of Business, Innovation and Employment).^{Footnote 135}

The review of other jurisdictions highlighted targeted efforts in areas such as active community involvement and community capacity building. Under the Australian National Strategy for Disaster Resilience (NSDR), current efforts towards effective risk identification and mitigation involve cooperation with state governments and building capability at the local government level. Nevertheless, some challenges have been recorded in implementing the NSDR, particularly due to the wide array and large number of stakeholders within and across jurisdictions and beyond the emergency management sector, making it difficult to coordinate efforts and prevent duplication. Other challenges included: the coordination of the information and knowledge generated through the various mitigation and other programs by the states to ensure that best practice approaches are being shared and lessons learned; achieving agreed standardized approaches to information gathering and publication; and ensuring the relevance, accuracy and timeliness of information, both when published and ongoing.^{Footnote 136}

In New Zealand, the vision of the CDEM strategy is resilient communities understanding and managing their hazards. The CDEM supports the government’s broader policy directions for sustainable growth and the safety of citizens and communities.^{Footnote 137}

The US Federal Emergency Management Agency (FEMA) developed several effective strategies for reducing risk, one of which is Risk MAP^{Footnote 138}, which is aimed at delivering quality data to increase public awareness and support community action that reduces risk to life and property. It builds on flood hazard data and maps produced during the Flood Map Modernization program. It is contributing to a multi-hazard, integrated approach to disaster risk reduction for infrastructure and community development and land use planning. Further, the FEMA’s Comprehensive Preparedness Guidance 201: Threat and Hazard Identification and Risk Assessment^{Footnote 139}, published in 2012, provides a common, consistent approach for identifying and assessing risks and their associated impacts. It provides Hazard Mitigation Assistance grants to communities in the pre- and post-disaster environments to reduce risk to undamaged facilities vulnerable to hazards. Finally, the Community Education and Outreach (CEO) group promotes effective hazard mitigation through community education, outreach, training and coordination with the public and private sectors.

Insight from Geoscience Australia

Considering all the international approaches described above, some particular insight from Geoscience Australia (re: stakeholder engagement, resource allocation and performance measurement) may be applied to GHPS as described below.

Resources: According to data provided by Geoscience Australia, funding for geohazards activities in 2011-12 was estimated to be AUD$28 million (approximately CAD$26.5 million) (see Table 11).^{Footnote 140} The current Canadian funding of CAD $15.1 million represents 57% of the funding available to Geoscience Australia. Note, however, that Australia engages also in international work and that the number of Geoscience Australia staff devoted to geohazards activities was estimated to be between 90 and 100 FTEs.

Table 11 Overview of Geoscience Australia funding in 2011-12

Type	Amount in millions		Description
	(AUD$)	%
Salaries	10.4	37%	Including superannuation and leave
Funding for suppliers	9.6	34%	Covered contract services such as R&D collaborations and services provided by external organizations, contract staff such as contractors hired to do specific tasks to support geohazards activities, training/conferences such as professional development activities and presentations at conferences, etc.
Revenues	8	29%	Funds received by Geoscience Australia that were not directly appropriated to the agency by the Government of Australia supported nuclear monitoring work in collaboration with the Australian Department of Foreign Affairs and Trade, regional disaster risk reduction activities in collaboration with the Australian Agency for International Development, climate change adaptation work in collaboration with the Department of Climate Change and Energy Efficiency and some state governments and vulnerability modelling work in collaboration with the Australian Reinsurance Pool Corporation.
Total	28	100%

Cost-recovery: Geoscience Australia underwent a strategic review to determine whether the agency was operated in the most cost-effective manner. Interestingly, like the GHPS at NRCan, Geoscience Australia’s revenues from external sources are primarily comprised of cost recovery for work performed on behalf of other agencies and other governments (inter-agency agreements); it conducts minimal cost-recovery from the private commercial sector. The review found several issues with this funding model. Firstly, due to the focus on the immediate service requirements as a discrete product to be funded, issues of funding capability development and other related investments were overlooked. Secondly, agencies sometimes had difficulty in seeing where their money was being spent in relation to outputs. Thirdly, the reliance on periodic inter-agency agreements had an impact on staffing and staff development policies at Geoscience Australia. The review suggested that the agency seek to recover more of its costs from commercial users of information through targeted, specialized data products, which would reduce its dependency on budget funding to support its products and services.

Stakeholder engagement and collaborations: As in the case of PSG and CHIS, Geoscience Australia collaborates extensively with external stakeholders and leverages considerable external funds in carrying out its geohazards activities. However, Geoscience Australia has closer working relationships with state governments, other jurisdictions and the insurance industry compared to PSG and CHIS, partly due to the strong leadership and coordination role on the part of the Standing Council on Police and Emergency Management (SCPEM) of the Council of Australian Governments. Geoscience Australia is also involved to a greater extent in emergency response and recovery than PSG/CHIS (e.g., through collaborative work aimed at determining post-disaster damage assessment, enhancing community resiliency to hazards etc.). On the other hand, PSG and CHIS tend to work more closely with academia, professional associations and critical infrastructure operators than Geoscience Australia.

Performance measurement: The performance indicators and service standards used by Geoscience Australia to report on its geohazards activities are more specific and more comprehensive than the existing performance indicators and service standards used by PSG and CHIS. The Australians track how the provision of hazard information/advice contributes to the emergency management functions of stakeholders and set specific benchmarks for the provision of geoscientific information. However, in the case of earthquakes, for instance, CHIS delivers alerts within a shorter period of time (4-6) minutes compared to Geoscience Australia (10 minutes) and ESS also uses Twitter to quickly relay hazard information to the public. Geoscience Australia is currently contemplating the use of social media to speed up its alert-issuing activity. An overview of performance measurement (service standards and performance metrics) implemented by Geoscience Australia is provided in Annex B.

Pro-active approach: Similar to Geoscience Australia, Canada focuses on and engages in geohazard risk preventive activities and measures. This constitutes a positive finding, as these were found to comply with the most cost-effective approach to risk and disaster mitigation policies. A general trend evident in the literature that facilitates greater cost effectiveness in disaster risk reduction programs and initiatives in the long-term is the overarching shift in many countries towards preventive approaches. The traditional public sector approach to disaster funding has focused on relief rather than prevention spending. When emergencies happen, funds are often diverted from key development projects in order to pay for relief and recovery efforts. A number of documents^{Footnote 141} reference the fact that due to the growing financial cost of disasters for government (as well as businesses and individuals) a gradual conceptual shift is being made towards proactive, rather than reactive, strategies. Many of these reports have recommended that policy emphasis be shifted from disaster response to pre-disaster mechanisms.^{Footnote 142} Governments and the donor community are now largely aware that it is more cost effective to fund programs that aim to reduce societal damage from disasters than to continually provide assistance for response and recovery.^{Footnote 143}

5.0       Conclusions and Recommendations

Relevance: The Sub-activity addresses important and relevant needs for a diverse range of stakeholders related to geohazards and emergency management in Canada. In particular, the evaluation found that needs for hazard monitoring, emergency management, risk reduction and loss mitigation activities are expected to increase in the future, both in Canada and globally. The combination of PSG and CHIS knowledge, services and networks constitutes an important aspect of Government of Canada’s efforts in science-based geohazards risk preventive activities and measures.

NRCan, through this Sub-activity, maintains unique and necessary scientific, technical and monitoring capabilities related to various natural and human-made hazards under its mandate in the Canadian context. The activities carried out under PSG and CHIS are aligned with NRCan and federal government priorities directly related to public safety and security and beyond including S&T, economic growth and sustainable development priorities.

While a strong federal government mandate and role are envisioned by stakeholders in geohazards research and emergency management, the evaluation of the Sub-activity unveiled a lack of clear and consistent horizontal vision within the federal government. In particular, the enhancement of policy dialogue and policy guidance from key federal organizations, including Public Safety Canada would help clarify the role of GHPS component and other key federal organizations that contribute to the national emergency management, risk reduction and loss mitigation efforts. Such issues are beyond the scope of this evaluation, but must be addressed by NRCan to help clarify where the Sub-activity fits in relation to the other organizations involved and how best to position the development of its expertise, research, services and tools in the future.

Furthermore, the evaluation found that strategic-level discussions are needed internally within NRCan to facilitate policy dialogue with other key federal government organizations. Given the nature of geohazards-related activities, horizontal discussions have often been more operational than policy-focused. As such, the evaluation found that the GHPS would benefit having policy support from within NRCan to facilitate horizontal policy dialogue and initiatives. This would help clarify the role of its components and identify some opportunities where NRCan would have the most beneficial role within the suite of federal activities.

Although mitigating the vulnerability of communities to reduce the impacts of disasters is consistent with the objectives of the Emergency Management Act and the government’s priorities, there is a need to clarify the role of NRCan at the regional and community levels. Given recent shift in PSG’s focus towards national-level priorities away from municipal/provincial outreach, education and extension programs due to resource shifts, such dialogue will help PSG to position its role at the regional and community levels in relation to the activities led by other federal organizations. PSG and CHIS should leverage established relationships and communication channels and may wish to consider seeking assistance from other areas of NRCan.

Several lines of evidence indicated that the GHPS stakeholders (federal organizations in particular) were not aware of the complete suite PSG and CHIS roles, responsibilities and activities. This was also reflected by the review of available activity profiles of CHIS and PSG documented internally and communicated externally. These profiles are not clear, complete or up-to-date. Greater and more detailed communication of GHPS roles and activities would help improve stakeholder awareness. Importantly, such improved communication of GHPS profile should primarily aim at facilitating the horizontal dialogue with other key federal organizations.

Performance: PSG and CHIS produced the planned outputs and have made considerable progress in achieving the intended outcomes in spite of some challenges and constraints. As noted above, both Sub-activity components are constrained by the lack of clarity on the extent of federal government involvement in geohazards research and emergency management. On the positive side, there are several key success factors that enhanced the performance of the Sub-activity or its components, including extensive collaboration with partners and stakeholders, an open and flexible approach, qualified and committed staff and a good understanding of user needs. Both Sub-activity components have been effective in collaborating with external stakeholders and other federal departments for specific initiatives and services.

Internally, the evaluation found evidence of collaboration between PSG and CHIS but an improved mechanism is clearly needed to better integrate the priorities and activities of the Sub-activity components. While there is regular interaction between CHIS and PSG, the evaluation found, based on interview data and observations, that PSG and CHIS approach issues from different perspectives, resulting in both components setting and implementing their priorities independently of one another. This relates to the contribution of PSG research to the development of CHIS services and tools and the contribution of CHIS service to PSG research activities.

Clarifying and formalizing their interactions by establishing common goals would facilitate the relations between PSG and CHIS, help set expectations and improve the performance of the Sub-activity. Formal priority setting mechanisms, project management plans, performance tracking and service standards would also need to be established and overseen jointly. If such mechanisms were developed or overseen externally (see next recommendation), this would have the benefit of ensuring objectivity in the review of research and service priorities. This mechanism would consider the mandate and priorities of NRCan, but also the broader federal context and the perspective of key Canadian stakeholders. It should also consider international research, science and technological development.

GHPS should consider enhancing the role of key external stakeholders and S&T experts (peers) in order to ensure that research priorities are validated to the highest degree possible in order to achieve greater cohesion between the two components. Importantly, this work should define how PSG directly supports the improvement or the development of new services or tools by CHIS. Making greater use of meetings with external stakeholders will help balance PSG research activities and ensure its contribution to applied S&T, for the benefit of both CHIS and external stakeholders.

It would also be important that this body review PSG research needs for CHIS expertise and information by considering the multiple activities and services provided to stakeholders outside NRCan, many of which are mandatory. This oversight body could help inform the GHPS planning in the light of international S&T developments and of the initiatives of other stakeholder organizations (OGDs and key federal players, provincial and territorial EMOs, municipalities, industry).

The evaluation recognizes the challenge in tracking the outcomes of science-based activities, especially for a Sub-activity that aims to mitigate risks from natural hazards. However, there remain opportunities to improve performance and financial monitoring and reporting. The current performance measurement practices used by both PSG and CHIS do not adequately capture their performance against set targets in a meaningful manner in order to support reporting and decision-making. Both components should continue to find opportunities to improve the quality and meaningfulness of existing performance information that is currently collected, including improvement of their reporting (i.e., research and service standards metrics). In addition, as external revenues are becoming increasingly important for both components in order for them to produce expected outputs and outcomes, there is a need to improve information on GHPS clientele and partners in order to support processes that help bring in external support while respecting the need to continue to provide publicly available data. The evaluation also found a lack of structured and complete information on past and current partners, collaborators and clients. PSG and CHIS should work collaboratively at rethinking their performance measurement approach in order to align performance information with existing program delivery and risk-based frameworks that are known and used. GHPS could also consult with performance measurement experts and NRCan SED for support and advice.

6.0       Annexes

6.1         Annex A – Characteristics of Case Study by Component and Program Activity

Table 12 Characteristics of Case Study by Component and Program Activity

#	Public Safety & Geoscience (PSG)
	Project Title (Activity/ Project Type)	Brief summary	Client/ Partner	Duration	Status/Outcomes
PSG1	Risk assessment case study: District of North Vancouver (Quantifying Geohazard Risk - QRA)	Canadian municipalities and other levels of government face a pressing need to perform all-hazard risk assessment in order to meet regulatory requirements and inform planning and mitigation decisions. However, most municipalities presently lack the tools and guidance to appropriately undertake rigorous assessments, particularly where specialized natural hazard is required. This project responds to the need of the municipalities to develop a better practice, standards-based approach to natural hazard risk assessment, which requires expanding beyond traditional emergency management and security threat and risk assessments to include analysis and modelling of accidental and natural hazards. The objective of the Quantifying Geohazard Risk (QRA) is to improve understanding, acceptance and widespread usage of QRA methods and tools through two demonstration studies (North Vancouver and Quebec-Montreal corridor). The risk assessment tool Hazards Assessment United States (HAZUS) was elected by PSG Program as a best practice method to be adapted for use in Canada. The District of North Vancouver (DNV) was selected as a demonstration case of the Canadian application of the tool.	District of North Vancouver	2009-10 to 2012-13 (This is the final year of PSG’s partnership with the DNV)	Documentation of the risk assessment methodology is in progress and will be published as a best-practices piece. The project was very useful to the client, as it could validate and further enhance its work on earthquake hazards carried out prior to the project. The benefits of NRCan’s work with the municipality have been reflected by the UN award the municipality received in 2011.
PSG2	Improved National Earthquake Hazard Model for Canada (Assessing Earthquake Geohazards)	The National Earthquake Hazard Model provides the basis of the seismic design provisions of the National Building Code of Canada (NBCC) and other construction codes and standards such as the Canadian Highway Bridge Design Code and the CSA standards for LNG Production, Storage and Handling. The NBCC will be updated in 2015; the seismic hazard model will be integrated into the updated codes and standards. The development of the National Earthquake Hazard Model is an activity under PSG Project Reducing Geohazard Vulnerability. The specific output for this activity is an improved National Earthquake Hazard Model for Canada. Components of the improved National Earthquake Hazard Model are complete and have been submitted for external review. This document, when completed, will be the main reference vehicle for communicating the hazard model to clients. In addition, outputs and deliverables from PSG Project Assessing Earthquake Geohazards form the basis of input to the development of national earthquake hazard models. This is an ongoing project; four generations of seismic hazard maps for Canada have been produced at approximately 15-year intervals (1953, 1970, 1985, 2005) and a fifth generation was justified for 2015 because there was sufficient new information available to improve the hazard estimates.	Canadian Commission on Building and Fire Codes’ Standing Committee on Earthquake Design	>5 years, ongoing	The development of an Improved National Earthquake Hazard Model is an ongoing activity that, despite not being finished yet, will generate tangible outputs (updated data and maps for the National Building Code) of great importance to all. Canadians and its direct users. Components of the improved National Earthquake Hazard Model are complete and have been submitted for review (NCR external review process).
PSG3	Geomagnetically Induced Currents (GIC) simulator (Assessing Space Weather Hazards)	Geomagnetic disturbances induce electric currents in power systems that can interfere with system operation and, in extreme cases, cause power blackouts. A system has been developed that enables simulations of the geomagnetically induced currents (GIC) produced during magnetic disturbances. This can be used with archived data to evaluate the impact of past extreme events or with real-time data to provide information for the power system control centre. The objective of the GIC Simulator Project is to implement the Real-time GIC simulator developed by the Geomagnetic Laboratory of the ESS into the Company’s Ontario Grid Control Centre to improve its current capabilities and to develop new ones.	Ontario Hydro One; Manitoba Hydro	July 2011 – October 2013	The GIC simulator software has been implemented by the power industry in several provinces and U.S. states and is used to assess the effects of geomagnetic disturbances on power systems.
PSG4	National tsunami hazard map for Canada (Reducing Geohazard Vulnerability)	The Canadian coastline is the longest of any country in the world and is at risk from tsunamis generated in three oceans. The current state of knowledge precludes a complete probabilistic tsunami hazard assessment. This project had the objective to develop a National Tsunami Hazard Map for Canada. The assessment presented a first attempt to quantify the tsunami hazard on the Canadian Pacific, Atlantic and Arctic coastlines from local and far-field, earthquake and large landslide sources. This map represents a summary of geological tsunami sources with the potential to threaten the coasts of Canada in order to quantify tsunami hazard. The Pacific coast is most at risk. Large tsunamis resulting from magnitude 9 Cascadia megathrust earthquakes have impacted the British Columbia coast on average every 500 years throughout the Holocene, most recently in A.D. 1700. The project has been completed. A peer-reviewed journal publication is still pending.	Critical infrastructure owners/ operators; Decision-makers at national level	2009-10 to 2012-13	A preliminary tsunami hazard assessment of the Canadian coastline is complete. It will be published as an Open File and on the Atlas of Canada website.

Here

#	Canadian Hazard Information Service (CHIS)
	Project Title (Activity/ Project Type)	Brief summary	Client/ Partner	Duration	Status/Outcomes
CHIS1	Development of a surge capability to address extreme traffic flow on the earthquakescanada.ca website (Information provision under the Emergency Management Act)	Following the 2010 Vals-de-Bois earthquake of magnitude 5.0, the NRCan website earthquakescanada.ca experienced unusual high traffic (peaking at 150,000 active connections at a time). This significantly limited the functionality of the website and the access to the NRCan earthquake internet domain by interested public and the NRCan seismologists who needed to update information on the website or answer telephonic questions. During the process of recovering the network access and full functionality, the NRCan-CHIS IT team discovered additional IT problems that could not be resolved instantly. The IT team procured a content delivery network provider to deliver critical network capacity and services that NRCan could not design in-house. As a result, a series of steps had to be undertaken to ensure that the website would be able to sustain this high traffic in case of future earthquakes or similar hazard events.	Bell Canada; Limelight; the public	2010 – 2012 (maintenance ongoing)	The project was completed and resulted in superior performance of the website compared to the initial level of service. The effectiveness was tested to a maximum capacity of 150,000 connections, but most importantly by a real event, a strong earthquake that occurred in October 2012 near Montreal, QC. During that event, the provider’s service  handled 6.5 times more traffic than the total capacity of NRCan’s  internet bandwidth alone.
CHIS2	Development and validation of an operational capability to respond to radiological events in Canada (Nuclear Emergency Response Project)	The NRCan-CHIS Nuclear Emergency Response Project, which evolved from a former Emergency Radiation Mapping project, provides response capabilities for real-time aerial mapping of radioactivity dispersed accidentally or intentionally and is part of the Chemical, Biological, Radiological, Nuclear and Explosives (CBRNE) Strategy of the Government of Canada (GC). The project was established as a GC response to the attacks of September 11, 2001 as the government wanted to ensure that Canada was prepared to respond to nuclear/radiological incidents. The initiative encompasses CHIS response to drivers from major external events (e.g., the 2010 Olympics, the G8 Summit) as well as internal drivers in order to create and then validate an operational capability. The existence of the project and the deployment of project personnel and capabilities in support of the security forces during the public events help to ensure that Canadians and international visitors in the course of these events are protected. In addition, there would be a capability for response and potentially prevention if an incident of radiological attack was to occur.	NRCan Airborne Geophysics; Health Canada; DND; Defense Research and Development Canada; National Research Council, CNSC; OGDs	>5 years (2008-09 – ongoing)	This ongoing project has seen some highly positive results in terms of NRCan’s contribution to ensuring national security and preventing public safety threats. NRCan is deemed to possess a unique combination of expertise and equipment that is a real asset to the Emergency Response Management Team comprised of other federal departments, mostly DND, RCMP and HC.

6.2         Annex B – Service standards and performance metrics implemented by Geoscience Australia

Performance monitoring

The progress made on high level Geoscience Australia priorities is monitored on a monthly basis for aggregated reporting to an internal Executive Board and the parent department (Department of Resources, Energy and Tourism). In addition, the 2011-12 Annual Report of Geoscience Australia reported on a performance indicator related to geohazards activities.^{Footnote 144} These indicators are listed in Annex B.

According to agreed delivery timelines for advice and alert activities, Geoscience Australia and the Bureau of Meteorology need to deliver an alert or advice triggering an alert for tsunami within 10 minutes from the time tsunami activity is recorded. The time limit for issuing earthquake alerts is not mandated, but Geoscience Australia endeavours to publish information on any earthquake larger than magnitude 4 within 10 minutes from the time seismic activity is recorded. Geoscience Australia is currently contemplating the use of social media to further reduce this time limit.

Geoscience Australia also has its Website Service Standard (listed in Annex B).^{Footnote 145} Moreover, it undertakes to continually improve service standards in response to customer requirements and ongoing feedback.

Apart from the aforementioned, Geoscience Australia has yet to implement a formal performance measurement regime at the operational level and utilizes ad hoc client and community feedback as proxy for performance. It recently established a Governance, Performance and Planning Section to strengthen performance measurement in support of more effective and integrated project management by following up on the recommendations from three separate audit, program and strategic reviews.

Geoscience Australia maintains a 24/7 hotline for media enquiries. According to the data provided by Geoscience Australia staff, between February 2012 and January 2013, Geoscience Australia received 495 media requests for interviews (including non-hazard related queries).

A preliminary search of Geoscience Australia databases conducted by Geoscience Australia staff identified 20 refereed publications (including abstracts), as well as approximately 65 client reports, abstracts for conferences and papers presented externally between February 2012 and January 2013 that were directly linked to geohazards.

Service standards and performance metrics

The Website Service Standard^{Footnote 146} of Geoscience Australia states the following:

• Geoscience Australia's website is monitored 24 hours a day 7 days a week by an external service. Any faults detected between 7am and 10pm will be investigated within two hours of their occurrence. Any incidents outside these hours will be investigated at the beginning of the following business day.
• Applications on the Geoscience Australia website that depend on the Oracle Database environment will have an availability level of 97%. Static content will have an availability of 99.5%.

In addition, an example of reporting on the key performance indicator from a Geoscience Australia annual report is presented in the table below.

Table 13 Geoscience Australia Key Geohazards Performance Indicator (2011-12)

Key Performance Indicator	Provision of geoscientific information for improved hazard management
Target	Evidence that information provided by Geoscience Australia has improved hazard management
Achievement	Target achieved
Explanation	The agency supported the scoping of the long-term flood modelling and mapping program, led by the National Emergency Management Committee of the Council of Australian Governments. The Sentinel bushfire mapping system functioned effectively throughout the year and now provides a seven-year history of major fires across Australia. Flood mapping from Earth observation supported flood responses in eastern Australia. Wind maps of tropical cyclone Lua off Western Australia were provided to the Australian Government Crisis Coordination Centre on 26 March 2012, a day before the cyclone made landfall, enabling the centre and state agencies to determine the potential impact and focus their response efforts. The Queensland Government used the Tropical Cyclone Risk Model to assess cyclone risk. The model was also used by the Secretariat of the Pacific Nations to assess storm-surge hazard in the region. The Western Australian Government used a study of coastal inundation risk in western Australia (Mandurah and Busselton) to help assess risk from sea-level rise in areas being considered for future development. The agency assessed the earthquake impact in Adelaide in collaboration with the South Australian Government as part of their risk assessment process. The agency advised the Queensland, Western Australian, South Australian, Victorian and Tasmanian State Governments on the development of state natural hazard risk assessments. The agency provided tsunami risk advice to the Attorney-General’s Department and regional Australia for emergency management planning on Norfolk Island.

The Earth Monitoring (Geodesy) Customer Service Standard^{Footnote 147} of Geoscience Australia includes the following:

• 95% availability in Geoscience Australia databases of acquired seismic, infrasound and hydroacoustic data;
• 95% availability in Geoscience Australia databases of geomagnetic data;
• Observed geomagnetic data submitted to INTERMAGNET compliant with INTERMAGNET specifications;
• Calibration of compasses and magnetometers completed within 10 working days of accepted receipt;
• 90% of national geodetic information requests responded to within two working days;
• 90% availability at International GPS Service of total daily and hourly data from Australia Regional GPS Network sites within 24 hours of observation;
• 80% availability at International GPS Service total daily data from Australian Global Navigation Satellite System sites within 24 hours of observation; and
• At least 400 passes per month delivered to the International Laser Ranging Service from each of Mt Stromlo and MOBile LAser Ranging System 5 Satellite Laser Ranging (SLR) observatories.