ARCHIVED - Evaluation of the Energy Efficiency: Transportation Sub-Sub Activity

Information Archived on the Web

Information identified as archived on the Web is for reference, research or recordkeeping purposes. It has not been altered or updated after the date of archiving. Web pages that are archived on the Web are not subject to the Government of Canada Web Standards. As per the Communications Policy of the Government of Canada, you can request alternate formats. Please "contact us" to request a format other than those available.


Table of Contents


Executive Summary

Introduction

This report provides the findings of the evaluation of Natural Resources Canada's (NRCan's) Energy Efficiency: Transportation Sub-Sub Activity, which comprises six programs and the transportation sector regulatory action (to support the regulatory work under the Clean Air Regulatory Agenda), intended to reduce greenhouse gas (GHG) emissions from on-road transportation by encouraging:

  • drivers of personal vehicles and fleets (and fleet managers) to use more fuel-efficient buying, driving and vehicle maintenance behaviours; and
  • manufacturers to produce new vehicles with improved fuel efficiency ratings.

Three programs target drivers of personal vehicles—the Motor Vehicle Fuel Efficiency Initiative, the Marketing of Fuel Efficient Vehicles Program and ecoENERGY for Personal Vehicles. Three programs target fleet managers and drivers—the Freight Efficiency and Technology Initiative, the Commercial Transportation Energy Efficiency and Fuels Initiative, and ecoENERGY for Fleets.

Between 2004–05 and 2008–09, total expenditures for the Transportation Sub-Sub Activity were $53.3 million. Of this, $26.6 million was allocated as transfer payments in the form of contribution funding.

Evaluation objectives and methodology

Covering 2004–05 to 2008–09, this evaluation was undertaken as part of NRCan's ongoing evaluation cycle and to support NRCan's commitment to provide input into two Clean Air Agenda (CAA) evaluations: one of the Clean Transportation Theme and one of the Clean Air Regulatory Agenda Theme. The purpose of this evaluation was to assess issues related to the relevance and performance (in terms of effectiveness, efficiency and economy) of the Transportation Sub-Sub Activity. Evaluation methodologies included a document review, literature review, interviews with 26 key informants and eight case studies.

The evaluation faced two challenges and limitations:

  1. At the outset of the evaluation, two potential methodologies—a file review and a survey of fleet managers—were proposed even though they were beyond the original scope of the terms of reference for this evaluation. The purpose of these proposed methods would have been to identify stronger outcome (i.e., performance) information than might be available through standard evaluation methodologies (such as interviews, literature and document reviews, and case studies). However, given that an evaluation assessment had not been completed (by Strategic Evaluation Division) prior to the start of this evaluation, about half of the evaluation resources were devoted to addressing start-up challenges. Consequently, these two additional methodologies were not feasible.
     
  2. While the Sub-Sub Activity has collected performance information identified for the immediate and intermediate outcomes, the performance indicators tend to be focused on the activity/output-level rather than on the outcome-level information required to fully respond to the summative-type questions posed in the evaluation. This limited the ability of the evaluation to assess the Sub-Sub Activity's effectiveness, efficiency and economy.

Findings

Relevance

The Transportation Sub-Sub Activity programs aim to reduce GHG emissions from on-road transportation by encouraging drivers and fleet managers to use energy-efficient purchasing, driving and vehicle maintenance behaviours. Growth in emissions from on-road transportation, if not contained, could conceivably jeopardize the country's ability to meet current and future GHG reduction targets and to satisfy its commitments to the public and to other nations.

The Transportation Sub-Sub Activity is consistent with federal priorities. The 2008 Speech from the Throne included a commitment to reduce GHG emissions by 20 % by 2020,1 and the 2008 Budget announced new measures to support Canada's ecoACTION plan. Moreover, the Transportation Sub-Sub Activity forms part of the Government of Canada's Clean Air Agenda, which aims to reduce air pollution and GHG emissions. It also supports NRCan's strategic outcome related to environmental responsibility in the development and use of Canada's natural resources.

There is a legitimate role for NRCan to reduce GHG emissions from on-road transportation. The Department undertakes clean transportation activities in accordance with the Energy Efficiency Act, 1992. Given NRCan's mandate to enhance the responsible development and use of Canada's natural resources, this is an appropriate role for the Department to fill. The existence of two market failures—externalities, where decisions and behaviours of road users impose a cost on society, and information deficits, which limit the decision-making capacity of drivers and fleet managers—indicates that there is a necessary role for NRCan in this area. However, because several federal departments are undertaking activities intended to address these market failures, there is some concern among most stakeholders about potential overlap and duplication.

The literature review shows that when drivers and fleet managers lack information on how to reduce GHGs through vehicle purchase and driving, their behaviour not only costs them money, because they use more fuel than needed, but also affects the welfare of others. However, the literature suggests that information is generally necessary but insufficient to induce large changes in vehicle purchasing, maintenance and driving habits that would reduce GHG emissions. Nonetheless, the literature includes several examples where driver training, feedback on driver performance and community-based social marketing have successfully led to changes in driver behaviour. The Transportation Sub-Sub Activity uses a combination of these proven approaches to encourage behaviour change.

Performance

Although there is a rigorous performance measurement strategy for the Transportation Sub-Sub Activity, the indicators included in this strategy do not capture outcome information. Similarly, although the programs systematically collect performance data in an attempt to track performance toward immediate and intermediate outcomes, the performance indicators reported against tend to be focused on the output level, rather than on the outcome level required to measure performance toward outcomes. Thus, the information reported against the indicators is not sufficient to track performance. For example, performance indicators that tend to be outcome-oriented often rely on survey data; however, the Better Energy Efficiency Reporting System (BEERS) indicates only whether or not these surveys have been done rather than the results of the surveys.

The Sub-Sub Activity has collected performance information identified for the immediate and intermediate outcomes; however, the performance indicators tend to be focused on the output level rather than on the outcome level information that is required to fully respond to the summative-type questions posed in the evaluation.

While the Transportation Sub-Sub Activity uses a two-phase modelling approach to calculate achieved GHG emissions reductions, the model is dependent upon assumptions about program reach, implementation of behaviour changes and sustained use of the changed behaviours. The table below provides the estimated GHG emissions reductions achieved between 2007–08 and 2008–09.

GHG emissions reductions achieved through the ecoENERGY programs
Sub-Sub Activity component Estimated GHG emissions reductions (2007–08 to 2008–09) % progress toward 2010 target
ecoENERGY for Personal Vehicles 0.0614 Mt of 0.1 Mt by 2010 61%
ecoENERGY for Fleets 0.1844 Mt of 0.5 to 0.7 Mt by 2010 26% to 37%
Memorandum of Understanding with the Canadian Automotive Industry 3.1 Mt to 3.4 Mt of 5.3Mt in 2010 58% to 64%

The lack of outcome-level performance data prevents the evaluation from assessing the economy of the Sub-Sub Activity. Nonetheless, the evaluation noted that the programs collect data on key indicators about their reach as well as developing partnerships to minimize program delivery costs and maximize their reach. The evaluation also found that government policies such as a moratorium on advertising and lengthy (or unpredictable) contracting processes have reduced the programs' ability to operate in an efficient and effective manner.

The evaluation identified a number of external factors that may influence or confound the performance of the programs, including fuel prices, changes in consumer demand, U.S. and Canadian regulations and privacy considerations.

Key informants identified several possible program improvements, including increased collaboration across and between federal departments; the provision of one-window access to federal government transportation information and programs; increased consistency and regular updating of information; and increased flexibility in government contracts.

Conclusion

The federal government has a role to play in reducing Canadian GHG emissions, since it has the power to regulate GHG emissions, is viewed as a neutral and trustworthy information source, and has the financial resources needed to fund behaviour-change projects. Given its mandate to enhance the responsible use of natural resources, of which fuel is one, it is appropriate for NRCan to deliver fuel efficiency-related behaviour change programming.

The program theory underlying the Transportation Sub-Sub Activity programs is sound. The bulk of the programming uses driver training and information dissemination through publications, websites and trade shows, as well as community-based social marketing (CBSM), to encourage behaviour among experienced or already licensed personal vehicle drivers. CBSM involves identifying the activity to be promoted, along with the barriers to the activity, and then designing a strategy to overcome the barriers, using psychological knowledge regarding the behaviour change.

While recognizing that the programs have evolved and expanded over time, a historical record of consistent, time-series outcome performance data has not been collected. The programs are not capturing sufficient outcome-level data and instead focus on activity indicators. Many of the indicators included in the performance measurement strategy are activity/output-based, and therefore do not measure behaviour change. Although the programs conduct surveys to collect some indication of behaviour, the surveys are not conducted frequently enough to provide evidence of program impact. Further, strong baseline data to support statements relating to changes in awareness and behaviour is not available.

Nonetheless, all groups of key informants believe the programs are beginning to influence behaviour change. Fleet managers have told NRCan representatives about some of the fuel efficiency gains they have made. Additionally, some local-level projects have attempted to systematically measure their impact. The results of these projects offer important illustrations of the effectiveness of the interventions conducted through these programs.

Recommendations, Management Response, and Action Plan
Recommendations Management Response Action Plan Responsible Official / Sector (Target Date)

1. NRCan should overhaul the Transportation Sub-Sub Activity's performance measurement strategy by:

a) establishing detailed and timely data capture for a select number of key behaviour change (outcome) indicators;

Agreed.

Division management agrees that the existing performance reporting in the Results-Based Management Accountability Framework (RMAF) could be improved and has developed a response to address each of the recommendations made in this evaluation.

Since 1994, the Division has undertaken supplementary performance measurement by capturing outcomes data under its personal vehicles programs. This work is done through surveys that are conducted every four to five years to measure awareness, attitudes and self-reported behaviours as they relate to buying, driving and maintaining vehicles. The Division has also routinely captured data on a wide range of activity-based indicators semi-annually to ensure that the program is fulfilling its obligations under its RMAF.

Division management will implement the following changes in any new activity in this area:

a)A single transportation energy efficiency program performance measurement strategy will be developed that identifies activities, outputs and outcomes. The outcomes will be measured through a detailed set of focused and relevant behaviour change performance indicators, which will be tracked over time through surveys. These surveys will be conducted at timely intervals and will focus on probable program outcomes such as driving-style behaviour changes. The timeline of surveys will be adjusted to coincide with the launch of new programs, followed by data collection every two to three years afterwards. By conducting these surveys, the Division will possess timely outcome-level performance data available at the time of evaluation.

 

 

ADM, Energy Sector

To be implemented by April 1, 2012. Strategic Evaluation will follow up on this recommendation in July 2011.

b) ensuring that proponents are accountable for delivering performance data by providing them with technical guidance and/or imposing funding hold-backs where deliverables are insufficient; and   b) While this evaluation was underway, Division management implemented a process in 2009 that replaced generic activity reporting templates for proponents to complete with more focused outcome performance reports. This new approach, which helps to align reporting with program outcome requirements, was not in place for the period covered by this evaluation.

Once a division performance measurement strategy is in place for any new activity in this area, project outcome-level reporting will be better aligned with program performance indicators and entrenched in the proponents' reporting process.

In addition, program managers will be required to apply funding hold-backs in situations where the proponents have provided inadequate reporting on outcomes.

ADM, Energy Sector

To be implemented by April 1, 2012. Strategic Evaluation will follow up on this recommendation in July 2011.

c) ensuring NRCan's ability to track outcome data by requiring proponents to obtain participants' permission to be contacted within two years following the completion of a project.   c) In response to this recommendation, an approach will be developed, with the advice of departmental privacy experts, to require the driver instruction industry that delivers fuel-efficient messaging under the AutoSmart or SmartDriver modules or proponents involved in similar activities to conduct a follow-up with a sample of participants within two years of their participation in driver training activities to determine the extent to which participants have retained and applied what they have learned.

ADM, Energy Sector

To be implemented by April 1, 2012. Strategic Evaluation will follow up on this recommendation in July 2011.

2. NRCan should work with other federal departments to ensure better collaboration, communication and coordination of on-road transportation activities.

Agreed.

Division management agrees that the role of various federal departments that act in the area of transportation can be confusing for stakeholders. The responsibilities and accountabilities of each department stem from departmental authorities and mandates, which are described in legislation.

To ensure a clear understanding of roles within the federal and provincial governments, the Division commits to participating in horizontal committees designed to improve coordination and communication between governments and departments, including committees created under the governance structure of the Clean Air Agenda, the Council of Energy Ministers, the Council of Ministers Responsible for Transportation and Highway Safety, and any future fora that aims to develop a low-carbon strategy for the transportation sector.

The Division further commits to working with federal partners to ensure a collaborative approach to communication with external stakeholders.

ADM, Energy Sector

To be implemented by April 1, 2012. Strategic Evaluation will follow up on this recommendation in July 2011.

1.0 Introduction and Background

1.1 Introduction

This report provides the findings of the evaluation of Natural Resources Canada's (NRCan's) Transportation Sub-Sub Activity, which comprises six programs and the transportation sector regulatory action (to support the regulatory work under the Clean Air Regulatory Agenda), intended to reduce greenhouse gas (GHG) emissions from on-road transportation. Covering 2004–05 to 2008–09, this evaluation was undertaken as part of NRCan's ongoing evaluation cycle and to support NRCan's commitment to provide input into two Clean Air Agenda (CAA) evaluations: one of the Clean Transportation Theme and one of the Clean Air Regulatory Agenda Theme.

1.2 Overview of the Transportation Sub-Sub Activity

As a signatory to the Kyoto Protocol to the United Nations Framework Convention on Climate Change, which came into force in 2005, Canada committed to reduce its GHG emissions to 6% below 1990 levels between 2008 and 2012.2 This represents a 240-megatonne reduction in GHG emissions from projected business-as-usual emissions in 2010.

As shown in Table 1, GHG emissions from transportation, the single largest sectoral contributor to Canada's GHG emissions, have been increasing over time. In 2007, total GHG emissions from transportation, at 200 megatonnes, were almost 38% higher than in 1990. A similar situation exists for on-road transportation, which accounts for 70% of the GHG emissions from all transportation modes. GHG emissions from on-road transportation were 137 megatonnes (Mt) in 2007, which is 39% more than 1990 levels.

Table 1: GHG emissions from Canada's transportation sector, 1990–2006 (megatonnes of equivalent carbon dioxide [CO2e])
  1990 2006 2007 Growth
1990–2007 (%)
All transportation 145.0 191.0 200.0 37.9%
Road transportation 98.4 133.0 137.0 39.2%
Light-duty gas vehicles 45.8 39.9 41.1 -10.3%
Light-duty gas trucks 20.7 43.6 45.0 117.4%
Heavy-duty gas vehicles 7.8 6.4 6.6 -15.4%
Motorcycles 0.1 0.3 0.3 200.0%
Light-duty diesel vehicles 0.4 0.4 0.5 25.0%
Light-duty diesel trucks 0.7 2.2 2.3 228.6%
Heavy-duty diesel vehicles 20.7 38.9 40.1 93.7%
Propane/natural gas vehicles 2.2 0.8 0.8 -63.6%
Road transportation as a % of all transport 67.9% 69.6% 68.5% 0.9%

Source: Environment Canada. (2009b, May). A Climate Change Plan for the Purposes of the Kyoto Protocol Implementation Act – 2009. Retrieved on March 5, 2010, from http://www.climatechange.gc.ca/default.asp?lang=En&n=4044AEA7-1.

The bulk of on-road transportation-related GHG emissions originate from light-duty gas vehicles and heavy-duty diesel vehicles (92% in 2007). Between 1990 and 2007, emissions from light-duty diesel trucks, light-duty gas trucks and heavy-duty diesel trucks increased by 229%, 117% and 94% respectively. These increases constitute most of the growth in emissions from on-road transportation over this period.

The Transportation Sub-Sub Activity, as shown in Table 2, forms part of NRCan's Program Activity Architecture (PAA).

Table 2: Situating energy-efficient transportation within NRCan's Program Activity Architecture

The Transportation Sub-Sub Activity comprises three theme areas:

  • Personal Vehicles (e.g., on-road personal vehicle fleet): NRCan's programs targeted at personal vehicles aim to influence the buying, driving and maintenance behaviours of individual drivers and to improve the fuel efficiency of new vehicles sold in Canada. This area also includes the Memorandum of Understanding between the Canadian auto industry and the Government of Canada to reduce GHG emissions from new light-duty vehicles.
  • Fleets (e.g., on-road commercial [passenger and freight] vehicles): NRCan's programs targeted at on-road fleets aim to influence driver behaviour and fleet management, as well as to improve the fuel efficiency of new vehicles sold in Canada.
  • Regulatory action: NRCan supports other federal departments' work on developing regulations for the transportation sector.

The programs forming the Transportation Sub-Sub Activity span the period of 2001–02 to 2010–11. Within the evaluation period of 2004–05 to 2008–09, some of NRCan's transportation programs were discontinued and replaced by new activities, while others continued under new names or were absorbed as smaller components within larger programs. Table 3 illustrates the recent history of the individual programs. Note that the diagonal shading in 2006–07 represents the implementation of the one-year Climate Change Interim Strategy, which resulted in program extensions or budget reductions.

Table 3: Historical overview of the Transportation Sub-Sub Activity
  2001–
02
2002–
03
2003–
04
2004–
05
2005–
06
2006–
07
2007–
08
2008–
09
2009–
10
2010–
11
Pre-evaluation Evaluation period Post-evaluation
Personal Vehicles
Motor Vehicle Fuel Efficiency Initiative Yes Yes Yes Yes Yes Yes      
Marketing of Fuel Efficient Vehicles     Yes Yes Yes Yes      
ecoENERGY for Personal Vehicles             Yes Yes Yes
Memorandum of Understanding with the Canadian Automotive Industry (part of the Motor Vehicle Fuel Efficiency Initiative and ecoENERGY for Personal Vehicles) Yes Yes Yes Yes Yes Yes Yes Yes Yes
Fleets
Freight Efficiency and Technology Initiative Yes Yes Yes Yes Yes Yes      
Commercial Transportation Energy Efficiency and Fuels Initiative     Yes Yes Yes Yes      
ecoENERGY for Fleets             Yes Yes Yes
Regulatory action
Transportation sector regulatory action (to support the Clean Air Regulatory Agenda)             Yes Yes Yes

1.3 Objectives and Rationale

1.3.1 Objectives

The programs within the Transportation Sub-Sub Activity intend to reduce GHG emissions from the on-road transportation sector by encouraging:

  • drivers of personal vehicles and fleet vehicles (and fleet managers) to use more fuel-efficient buying, driving and vehicle maintenance behaviours; and
  • manufacturers to produce new vehicles with improved fuel efficiency ratings.

The objectives of the programs forming the Transportation Sub-Sub Activity are:

Personal Vehicles
  • Motor Vehicle Fuel Efficiency Initiative (MVFEI): "…achieve an improvement of 25% in fuel efficiency of new vehicles sold in Canada by 2010."3
  • Marketing of Fuel Efficient Vehicles (MEV): complement the Motor Vehicle Fuel Efficiency Initiative by contributing an additional GHG reduction "from personal vehicles by 0.34 megatonne[s] in Canada by 2010."4
  • ecoENERGY for Personal Vehicles: "…improve the energy efficiency of the on-road personal vehicle fleet and reduce associated environmental impacts through voluntary initiatives such as a Memorandum of Understanding (MOU) with the vehicle industry."5
  • Memorandum of Understanding with the Canadian Automotive Industry: In April 2005, the Government of Canada and the Canadian automotive industry signed a Memorandum of Understanding that voluntarily commits the auto industry to achieve a 5.3-megatonne reduction in GHG emissions from light-duty vehicles by 2010. Work on the Memorandum of Understanding began as part of the Motor Vehicle Fuel Efficiency Initiative and is continuing as part of ecoENERGY for Personal Vehicles.
Fleets
  • Freight Efficiency and Technology Initiative (FETI): "…reduce the growth of GHG emissions from [the freight] sector of transportation."6 Its sub-objectives were "i) to increase the freight transportation industry's participation in voluntary climate change initiatives; ii) to increase the operating efficiency and environmental training and awareness amongst freight operators and shippers; and iii) to demonstrate and encourage the take-up of innovative environmental technologies and efficient best practices within the freight transportation sector."7
  • Commercial Transportation Energy Efficiency and Fuels Initiative (CTEEFI): "…reduce the growth of greenhouse gas emissions from the commercial transportation sector (passenger and freight)."8 This Initiative included two components: NRCan was responsible for the On-Road Commercial Transportation Energy Efficiency and Fuels Initiative, and Transport Canada was responsible for the Non-Road Freight Efficiency Program.
  • ecoENERGY for Fleets: "…contribute to reductions in fuel use and related costs, air pollutants, and GHG emissions."9 Its specific objectives are to expand NRCan's fleet training program and accelerate the uptake of energy-conservation technologies and practices.
Regulatory action

One component of the Transportation Sub-Sub Activity support regulatory action in the transportation sector.

  • The Clean Air Regulatory Agenda (CARA) will "enable Canada to establish clear national standards, move industry from voluntary compliance to regulation, monitor progress, and report to Canadians on reductions in the emission of air pollutants and GHGs, as well as improvements in air quality"10. NRCan conducts analysis and research in support of the transportation regulations that fall under the Clean Air Regulatory Agenda.

In October 2006, the Government of Canada announced its intention to regulate the fuel consumption of road motor vehicles under the Motor Vehicle Fuel Consumption Standards Act, effective for model year 2011. However, in the April 4, 2009, issue of the Canada Gazette Part I, the Government of Canada issued notice that "the Department of the Environment is initiating the development of regulations under the Canadian Environmental Protection Act, 1999 to limit carbon dioxide (CO2) emissions from new cars and light-duty trucks to take effect beginning with the 2011 model year."11

1.3.2 Rationale

NRCan "seeks to enhance the responsible development and use of Canada's natural resources and the competitiveness of Canada's natural resources products."12 Thus, the Transportation Sub-Sub Activity promotes and encourages the purchase of energy-efficient vehicles and the use of energy-efficient driving practices in order to foster responsible use of transportation fuels.

NRCan's clean transportation activities are undertaken in accordance with two acts:

  • The Energy Efficiency Act (1992) enables the Government of Canada to prescribe energy efficiency standards for energy-using products, require labelling of energy-using products and provide for testing to determine the energy efficiency of energy-using products. It also gives the Government of Canada the power to promote the efficient use of energy and the use of alternate energy sources.13
  • The Motor Vehicle Fuel Consumption Standards Act establishes the legal authority to regulate fuel consumption for prescribed classes of motor vehicles. Although the Act was passed in 1982, it was not proclaimed until 2007 because manufacturers voluntarily committed to continue to provide vehicles that meet U.S. standards.14

1.4 Key Stakeholders

Various federal departments have transportation-related roles and responsibilities:

  • NRCan's role is to develop and deliver energy-efficient transportation programs for the on-road sector. The Energy Efficiency Act provides the Department with the authority to promote to and educate Canadians about energy-efficient transportation practices. Its role in energy-efficient transportation links to its mandate to protect natural resources, of which fuel is one.
  • Transport Canada has regulatory authority for marine, aviation and rail transportation, as well as responsibility for the safety and security of on-road vehicles. With its authorities, Transport Canada has established tailpipe emissions limits for ships, aircraft and locomotives. Additionally, it delivers the ecoTRANSPORT programs for urban transportation, freight transportation and advanced technology vehicles, which focus on promoting modal shift and testing advanced technologies.
  • Environment Canada is responsible for environmental and air quality issues. It has the authority to regulate light-duty vehicle emissions under the Canadian Environmental Protection Act, 1999. Environment Canada also regulates air pollutants from small and marine engines. It coordinates the overall ecoACTION programs, which promote more environmental choice, and works with Transport Canada on its ecoMOBILITY programs.

Provincial governments and industry also have various roles to play:

  • Provincial governments have the authority to regulate vehicles currently on the road. For example, they can regulate the type of vehicles allowed on the road, vehicle weight and vehicle speed. They are also responsible for licensing drivers. Additionally, provincial governments help deliver federal messaging and may establish and promote their own messages, which in many cases incorporate and expand on federal informational and educational material.
  • Industry associations play an important role in alerting the federal government of transportation issues that need to be addressed, contributing to the development and promotion of energy-efficient transportation messaging and helping design and deliver behaviour change programming.

1.5 Program Structure

1.5.1 Logic model 15

The programs forming the Sub-Sub Activity comprise four core activities and outputs:

  • Liaising and building partnerships with provinces and industry.
  • Issuing contracts and contribution agreements to industry representatives and other stakeholders to conduct various research projects, develop tools to support vehicle purchase decision-making processes and encourage specific driver behaviours, and develop curriculum for and/or deliver driver training sessions.
  • Conducting promotional and communications-related activities such as preparing materials for the ecoENERGY website, staffing a booth at auto/truck shows and hosting conferences.
  • Undertaking negotiations with vehicle manufacturers for the establishment of, and commitment to meet, voluntary fuel efficiency targets.

These activities and outputs are expected to increase individuals' awareness of the impact the transportation sector has on the environment and inform them of actions they can take to reduce the environmental impact of their personal use of transportation. By providing individuals with the tools and training needed to support behaviour changes, these programs are expected to increase the use of fuel-efficient buying, driving and vehicle maintenance behaviours as well as increase the fuel efficiency of cars and light-duty trucks, which will result in reduced GHG emissions from on-road transportation.

1.5.2 Expected results

Table 4 lists the targets for each of the programs forming the Transportation Sub-Sub Activity.

Table 4: Transportation emissions reduction targets by program

Sources: Individual program Results-based Management and Accountability Frameworks.

The Transportation Sub-Sub Activity has limited ability to significantly reduce GHG emissions from road transportation. In 2004, total Canadian GHG emissions from road transportation were 129 megatonnes. The ecoENERGY for Personal Vehicles and ecoENERGY for Fleets are intended to achieve a 6.1-megatonne reduction in GHG emissions by 2010–11, which represents a 4.7 % reduction in GHG emissions from road transportation.

Common to most programs under the Clean Air Agenda, GHG reduction outcomes are estimated and not directly measured. Since it is not feasible to directly measure GHG emissions from on-road transportation at a global level, the models are used to estimate achieved reductions in GHG emissions.

  • Environment Canada's National Inventory Report on Greenhouse Gas Sources and Sinks uses a top-down model to estimate national GHG emissions for six sectors. Estimates of GHG emissions from transportation are included in the energy sector.
  • Like many other departments, NRCan uses a bottom-up model to estimate annual reductions in GHG emissions resulting from the Transportation Sub-Sub Activity.

It is difficult for top-down models to accurately measure changes that are small in magnitude, and these models cannot isolate program-related changes in GHG emissions. According to Environment Canada's National Inventory Report, GHG emissions from road transportation have increased from 129 megatonnes in 2004 to 137 megatonnes in 2007.

NRCan's bottom-up model estimates GHG reductions resulting from the Transportation Sub-Sub Activity. The model calculates the reductions based on program activity and the estimated impact of that activity on GHG emissions.

1.5.3 Activities and outputs

Personal Vehicles – encouraging fuel-efficient driving behaviour

To encourage individuals' uptake of fuel-efficient driving behaviour, the Personal Vehicles programs develop fact sheets, post information on NRCan's website and conduct education campaigns focusing on idling, tire inflation and vehicle maintenance. While NRCan develops and delivers some of this material itself, it also relies on provincial and industry partners to develop and distribute some of the information and deliver training sessions. Some of the activities the Program has completed are described below.

The Program has also developed and distributed its Auto$mart Driver Educator kit to driving instructors who teach novice drivers. The new driver training program:

  • distributed 200 kits to driver educators in 2004 and 2,500 in 2006;
  • held seven driver education workshops in 2004 and four in 2006;
  • trained 16 master trainers in 2004 and 12 in 2008; and
  • provided training to 350,000 novice drivers in 2007 and 447,000 in 2008.

In addition to driver education through Auto$mart:

  • the Idle-Free campaigns have reached under 4 million Canadians;
  • the "Be Tire Smart" tire inflation campaign done in collaboration with the Rubber; and Association of Canada (RAC) has reached nearly over 7 million Canadians.
Personal Vehicles – encouraging the purchase of fuel-efficient vehicles

NRCan, in collaboration with its partners, uses three core methods of encouraging drivers to purchase more fuel-efficient vehicles:

  • vehicle labels;
  • the Fuel Consumption Guide; and
  • ecoENERGY for Vehicles Awards.

This information aims to increase consumer knowledge and awareness of the fuel-efficient features available in new vehicles. Some of the specific activities the Personal Vehicles programs have undertaken are:

  • Providing information and outreach by participating in three auto shows per year to increase consumers' awareness of the impact of vehicle purchase choice on fuel efficiency and GHG emissions.
  • Issuing ecoENERGY for Vehicles Awards to recognize, and identify for consumers, the most fuel-efficient light-duty vehicles in nine categories.
  • Producing an annual Fuel Consumption Guide, which informs Canadians of the fuel efficiency of vehicles by model year. The Program distributes between 300,000 and 350,000 copies of this guide to the Canadian Automobile Association offices and new car dealerships. The guide is also available on NRCan's website.
  • Maintaining a vehicle ranking tool (by fuel efficiency) on NRCan's website, which allows purchasers to compare fuel efficiency of different vehicle models.
  • Engaging partners to deliver vehicle fuel efficiency messages. In 2004–05, NRCan partnered with the RAC, the Road Safety Educators' Association (RSEA) and the Canadian Automobile Association – Quebec (CAAQc) and, in 2006–07, with the New Brunswick Lung Association and the Environmental Youth Alliance (EYA). There were 15 new contribution agreements for FY 2008–2009 that focused on raising awareness about tire maintenance, idle reduction and fuel-efficient driving.
  • Conducting research on consumers' purchasing behaviours. Research efforts include the annual DesRosiers Light Vehicle Study, an EnerGuide label compliance survey and research studies funded through standing offers.
Fleets – encouraging fuel-efficient driving behaviour

The Fleets programs have undertaken various activities to encourage uptake of fuel-efficient driving behaviour by fleets. These activities are conducted and delivered in partnership with provinces and industry though contribution agreements and contracts. Examples of activities that the programs have completed are posting information on NRCan's website; providing information and outreach by attending trade shows/conferences; and conducting education campaigns that focus on training to improve driver behaviour and fleet fuel management practices. More specifically, the programs have:

  • provided information and outreach by participating in 15 trade shows and 40 industry meetings annually;
  • attracted 141 participants to the 2008 Fleet$martWay Conference, which promoted a better understanding on return investment on investing in fuel efficient practices;
  • registered 4,733 members in Fleet$mart, as of 2005;
  • conducted Fuel Management 101 workshops to educate fleet managers on fuel-efficient management practices; in 2007 and 2008, 250 fleets participated in 12 workshops; a survey tool to measure awareness gains from Fuel Management 101 workshops is being developed;
  • conducted six SmartDriver professional development workshops in 2008–09;
  • had about 60 truck stops annually participated in anti-idling campaigns; however, as of 2008, the campaign is on hold due to contracting difficulties; and
  • trained 27,459 commercial drivers in 2007–08 and 2008–09.

ecoENERGY for Fleets is developing a SmartMechanic training module, which is being piloted in Quebec and is about one year away from completion. It is also piloting a Benchmarking Web Tool, which shows fleets how their fuel use operations compare to others.

Fleets – encouraging the purchase of fuel-efficient equipment for vehicles

To encourage the purchase of fuel-efficient equipment, the Fleets programs:

  • Offered a rebate (2004 and 2005) for the purchase of auxiliary power units and heater units through the rebate program, which formed part of the Commercial Transportation Energy Efficiency and Fuels Initiative.
  • Conducted 19 technology transfer workshops in 2008, including 10 workshops with the Same Roads – New Challenges Forum and nine with FPInnovations. Additionally, work is under way to track 150 firms within the forestry sector that adopt technologies to reduce GHG emissions.
  • Conducted demonstrations of three "Star Trucks," which led to specification improvements in 66 trucks. These improvements are removing 2,000 tonnes of GHG emissions from each truck annually.16
  • Funded demonstrations of the cost savings associated with anti-idling devices, aerodynamic devices, low rolling resistant and wide-base tires, and trailer side skirts.
  • Funded demonstrations of SmartWay-certified technologies, which became mandatory under a new California regulation, to show how the technology performs in a Canadian climate.

Some other activities the Fleets programs have undertaken to increase fuel-efficient consumer behaviours by fleets include:

  • Developing a vehicle fuel efficiency benchmarking ranking tool for fleet manager, the Fleet Tool. The Fleet Tool is intended to help fleet managers determine the impact of making various technology changes. The tool facilitates the development of action plans to reduce energy use and includes four components: baseline fuel consumption, technical assessment of possible measures to reduce fuel consumption (training), development of an action plan, and evaluation of actions against the baseline. Related to this activity, NRCan signed a Memorandum of Understanding with the U.S. Environmental Protection Agency to gain access to information that could be integrated into the tool.
  • Developing an Intelligent Transportation Systems (ITS) catalogue that provides fleets with technical and cost information on various vehicle technologies.
  • Joining FPInnovations' membership plan, which provides opportunities to influence the selection of fuel-efficient vehicle technology for testing.

1.5.4 Better Energy Efficiency Reporting System

The Office of Energy Efficiency (OEE) uses the Better Energy Efficiency Reporting System (BEERS) as the main source of performance data for NRCan's energy efficient transportation programs. This system was developed and implemented in 2004–05.

While the BEERS includes some performance information for the Personal Vehicles and Fleet programs encompassed in the Transportation Sub-Sub Activity, there is no consistent set of indicators used to track performance across the various programs. Although data on indicators is collected and reported on bi-annually, some limitations and weaknesses are associated with the data captured in BEERS, including:

  • for 2007 and 2008, the same indicators are used for both outputs and outcomes, resulting in indicators that may measure outputs but not outcomes;
  • many indicators rely on information to be obtained through special studies or exercises, and in many cases the results of these studies are not indicated;
  • in some instances, the BEERS data indicates the number of outputs achieved, but does not describe what the outputs comprise (e.g., one indicator for Fleets is number of awareness campaigns, and the result is four – no information on the nature of the campaigns is provided);
  • in some cases, only a portion of the performance associated with an indicator is discussed; and
  • in some instances, shortfalls have been shifted to the next fiscal year; however, no information is given about whether processes have been put in place to make up the shortfall, and there are instances when these shortfalls span several years.

1.6 Resources

Over the evaluation period of 2004–05 to 2008–09, total expenditures for the Transportation Sub-Sub Activity were $53.3 million. Of this, $26.6 million (50%) was transfer payments in the form of contributions. Total expenditures for Personal Vehicles programs were $19.9 million, including $5.0 million (25%) for contribution funding, and for Fleets programs were $32.4 million, including $21.6 million (67%) for contribution funding. Total expenditures for regulatory action were $1.0 million, solely in the form of operating funds. Tables 5 to 7 show the total expenditures and the amount spent on contributions funding by program and year.

Table 5: Personal Vehicles program expenditures, 2004–05 to 2008–09 ($millions)
Program 2004–05 2005–06 2006–07 2007–08 2008–09 Total
Motor Vehicle Fuel Efficiency Initiative
Total1 $2.9 $2.2 $1.9 n/a n/a $7.0
Gs&Cs $0.5 $0.8 $0.5 n/a n/a $1.8
% Gs&Cs 17% 36% 26% n/a n/a 26%
Marketing of Fuel-Efficient Vehicles
Total $0.9 $1.5 $1.1 n/a n/a $3.5
Gs&Cs $0.3 $0.5 $0.1 n/a n/a $0.9
% Gs&Cs 33% 33% 9% n/a n/a 26%
ecoENERGY for Personal Vehicles
Total n/a n/a n/a $3.8 $5.6 $9.4
Gs&Cs n/a n/a n/a $0.4 $1.9 $2.3
% Gs&Cs n/a n/a n/a 10% 34% 24%
Total – Personal Vehicles
Total $3.9 $3.7 $3.0 $3.8 $5.6 $19.9
Gs&Cs $0.8 $1.3 $0.6 $0.4 $1.9 $5.0
% Gs&Cs 20% 35% 20% 10% 34% 25%

Source: figures provided by the Manager, Financial Planning and Analysis, Corporate Management and Services Sector, Financial Management Branch.
1 Total includes: salaries, EBP, O&M.
Note: Gs&Cs = grants and contributions; n/a = not applicable.

Table 6: Fleets programs expenditures, 2004–05 to 2008–09 ($millions)
Program 2004–05 2005–06 2006–07 2007–08 2008–09 Total
Freight Efficiency and Technology Initiative
Total1 $1.3 $1.2 $0.5 n/a n/a $3.0
Gs&Cs $0.4 $0.1 $0.0 n/a n/a $0.5
% Gs&Cs 31% 8% 0% n/a n/a 17%
Commercial Transportation Energy Efficiency and Fuels Initiative
Total $7.3 $12.1 $2.3 n/a n/a $21.7
Gs&Cs $6.7 $10.9 $1.4 n/a n/a $19.0
% Gs&Cs 92% 90% 61% n/a n/a 88%
ecoENERGY for Fleets
Total n/a n/a n/a $2.7 $5.0 $7.7
Gs&Cs n/a n/a n/a $0.3 $1.8 $2.1
% Gs&Cs n/a n/a n/a 11% 36% 27%
Total
Total $8.6 $13.3 $2.8 $2.7 $5.0 $32.4
Gs&Cs $7.1 $11.0 $1.4 $0.3 $1.8 $21.6
% Gs&Cs 83% 83% 50% 11% 36% 67%

Source: figures provided by the Manager, Financial Planning and Analysis, Corporate Management and Services Sector , Financial Management Branch
1 Total includes: salaries, EBP, O&M.
Note: Gs&Cs = grants and contributions; n/a = not applicable.

Table 7: Regulatory Action expenditures, 2004–05 to 2008–09 ($millions)
Program 2004–05 2005–06 2006–07 2007–08 2008–09 Total
Total1 n/a n/a n/a $0.4 $0.6 $1.0
Gs&Cs n/a n/a n/a - - -
% Gs&Cs n/a n/a n/a - - -

Source: figures provided by the Manager, Financial Planning and Analysis, Corporate Management and Services Sector, Financial Management Branch.
1 Total includes: salaries, EBP, O&M.
Note: Gs&Cs = grants and contributions; n/a = not applicable.

Figure 1 shows the distribution of the Transportation Sub-Sub Activity's $53.3 million expenditures by program.

Figure 1 shows the distribution of the Transportation Sub-Sub Activitys $53.3 million expenditures by program

Figure 2 shows the allocation of the Transportation Sub-Sub Activity's $53.2 million expenditures by expenditure type.

Figure 2 shows the allocation of the Transportation Sub-Sub Activitys $53.2 million expenditures by expenditure type.

2.0 Evaluation Approach and Methodologies

2.1 Evaluation Scope and Objectives

This evaluation assessed issues related to:

  • relevance of energy efficiency programs in the Transportation Sub-Sub Activity in terms of a continued need for the program, alignment with federal government priorities, alignment with federal roles and responsibilities, and validity of the program theory; and
  • performance of the Transportation Sub-Sub Activity programs, including achievement of expected outcomes, and demonstration of efficiency and economy.

The evaluation covered the Department's direct spending on transportation energy efficiency over the period of 2004–05 to 2008–09, with an emphasis on the last two fiscal years.17

2.2 Evaluation Methods

In order to assess these three issues, the evaluation used the following four data collection tasks:

  • document review;
  • literature review
  • key informant interviews; and
  • case studies.

Key informant interviews

Using a list of potential key informants provided by NRCan, interviews were completed with 26 individuals from the following groups:

  • NRCan representatives (n=13);
  • representatives of other federal government departments (n=3);
  • provincial government representatives (n=3); and
  • industry representatives (n=7).

The findings of the key informant interviews are reported using the following scale.

The findings of the key informant interviews are reported using the following scale

It is important to note that, in analyzing the findings from the key informant interviews, two important situations can occur: first, there is high agreement across all groups of key informants, and second, a single respondent offered a perspective, based on experience or a unique position, that materially clarified an aspect of the Program or context, thereby assisting the evaluation to respond to important questions. Further, not all groups of key informants were asked to respond to each evaluation question, and different groups of key informants may have diverging opinions; therefore, where appropriate, findings are linked to the key informant group that provided the information.

Case studies

A total of eight case studies of funded projects were completed, four with Personal Vehicles projects and four with Fleets projects. The case studies, selected in consultation with NRCan, provided coverage of various program areas and components.

The Personal Vehicles case studies included a national vehicle maintenance campaign, a national tire maintenance campaign, a local driving behaviour project and a local anti-idling project.

The Fleets case studies included projects intended to reduce GHG emissions from the trucking sector, taxis, urban transit buses and highway trucking.

2.3 Evaluation Challenges and Limitations

This evaluation faced two challenges and limitations.

  1. At the outset of the evaluation, two potential methodologies, a file review and a survey of fleet managers, were proposed even though they were beyond the original scope of the terms of reference for this evaluation. The purpose of these proposed methods would have been to identify stronger outcome (i.e., performance) information than might be available through standard evaluation methodologies (such as interviews, literature and document reviews, and case studies). However, given that an evaluation assessment had not been completed (by Strategic Evaluation Division) prior to the start of this evaluation, about half of the evaluation resources were devoted to addressing start-up challenges. Consequently, these two additional methodologies were not feasible.
     
  2. While the Sub-Sub Activity has collected performance information identified for the immediate and intermediate outcomes, the performance indicators tend to be focused on the activity/output-level rather than on the outcome level information required to fully respond to the summative-type questions posed in the evaluation. This limited the ability of the evaluation to assess the Sub-Sub Activity's effectiveness, efficiency and economy.

3.0 Evaluation Findings

3.1 Relevance

Summary

The programs within the Transportation Sub-Sub Activity aim to reduce GHG emissions from on-road transportation by encouraging drivers and fleet managers to use energy-efficient purchasing, driving and vehicle maintenance behaviours. Growth in emissions from on-road transportation, if not contained, could jeopardize the country's ability both to meet current and future GHG reduction targets and to satisfy its commitments to the public and to other nations.

The Transportation Sub-Sub Activity programs are consistent with federal priorities. The 2008 Speech from the Throne18 included a commitment to reduce GHG emissions by 20 % by 2020,19 and the 2008 Budget20 announced new measures to support Canada's ecoACTION plan. Moreover, the Transportation Sub-Sub Activity programs form part of the Government of Canada's Clean Air Agenda, which aims to reduce air pollution and GHG emissions. Overall, the programs within this Sub-Sub Activity also support NRCan's strategic outcome related to environmental responsibility in the development and use of Canada's natural resources.

There is a legitimate role for NRCan to reduce GHG emissions from on-road transportation. The Department undertakes clean transportation activities in accordance with the Energy Efficiency Act. Given NRCan's mandate to enhance the responsible development and use of Canada's natural resources, this is an appropriate role for the Department to fill. The existence of two market failures-externalities, where decisions and behaviours of road users impose a cost on society, and information deficits, which limit the decision-making capacity of drivers and fleet managers-indicates that there is a necessary role for NRCan in this area. However, because several federal departments are undertaking activities intended to address these market failures, there is some concern among most stakeholders about potential overlap and duplication.

The literature review shows that when drivers and fleet managers lack information on how to reduce GHGs through vehicle purchase and driving, their behaviour not only costs them money, because they use more fuel than needed, but also affects the welfare of others. However, the literature suggests that information is generally necessary but insufficient to induce large changes in vehicle purchasing, maintenance and driving habits that would reduce GHG emissions. Nonetheless, the literature includes several examples where driver training, feedback on driver performance, and community-based social marketing have successfully led to changes in driver behaviour. The Transportation Sub-Sub Activity programs use a combination of these proven approaches to encourage behaviour change.

Evaluation question: What is the environmental need the programs are trying to address?
  • The Sub-Sub Activity programs are intended to reduce GHG emissions from on-road transportation.

The Transportation Sub-Sub Activity programs aim to reduce GHG emissions from on-road transportation by encouraging drivers and fleet managers to use energy-efficient purchasing, driving, and vehicle maintenance behaviours.

Increases in GHG emissions from on-road transportation arise because of economic and population growth and because of changes in consumer tastes. An increase in population will induce increases in the number of cars and trucks, increases in the total distances driven and commensurate increases in fossil fuel consumption and GHG emissions. There were approximately two million more passenger vehicles on the road in Canada in 2005 than in 1990, and the share of minivans and SUVs-which are generally less fuel-efficient than cars-grew from 19% to 32% of the passenger fleet21; similarly, Canada's freight transportation fleet expanded from 2.0 to 2.7 million trucks between 1990 and 2006.22 Moreover, the average distance travelled by personal vehicles and fleets increased 9% and 24% respectively over that same period.

The fact that total GHG emissions from passenger cars declined even as the number of these vehicles increased points to another important trend in Canadian transportation, namely a gradual improvement (reduction) in GHG intensity over time. Between 1990 and 2006, GHG emissions per passenger- and tonne-kilometre have declined 13% and 24% in passenger and freight transportation, respectively.23 However, even with these improvements in fuel efficiency, given the increasing number of vehicles on the road and a shift towards larger passenger vehicles, total GHG emissions from on-road transportation grew by 39%, from 98 megatonnes in 1990 to 137 megatonnes in 2007.

Growth in emissions from on-road transportation, if not contained, could jeopardize the country's ability both to meet current and future GHG reduction targets and to satisfy its commitments to the public and to other nations. Although Canada produces only 2% of global GHG emissions, and road transportation constitutes just under 20% of this, it might be argued that failure to reduce emissions from road transportation is unlikely to significantly contribute to climate change.24 However, this is a large portion of all transportation emissions. These emissions were expected to "increase by 16 Mt (1.4 % per year) between 2004 and 2010" and are expected to continue to increase "by 25 Mt (1.1 % per year) between 2010 and 2020."25

Evaluation question: Are the programs consistent with government priorities and NRCan strategic objectives?
  • Yes, the Sub-Sub Activity programs directly respond to the federal government's GHG emissions reduction commitments and align with NRCan's clean energy objectives.

Various government publications indicate that the Transportation Sub-Sub Activity programs are consistent with federal priorities.

  • One of the priorities identified in the 2008 Speech from the Throne was to tackle climate change and preserve "Canada's environment by reducing greenhouse gas emissions 20 % by 2020."26 The transportation programs include GHG reduction targets.
  • Supporting the commitments made in speeches from the Throne, several federal budgets have earmarked funding allocations aimed at reducing GHG emissions and/or increasing energy efficiency. Specifically, the 2008 Budget announced "new measures to strengthen and ensure effective implementation of Canada's ecoACTION plan," including funding to implement regulations that will lead to significant reductions in GHG emissions and improvements in air quality,andadditional incentives to advance progress on cleaner energy generation and use.27
  • The Transportation Sub-Sub Activity is part of the Government of Canada's Clean Air Agenda (CAA), which is a four-year (2007–08 to 2010–11) horizontal strategy intended to achieve improvements in Canada's environment, including reduced air pollution and GHG emissions.28 Clean Transportation is one of the themes included in the Clean Air Agenda.

The Transportation Sub-Sub Activity is also consistent with NRCan's strategic objectives. NRCan's annual plans and priorities include measures related to energy-efficient transportation. NRCan's 2007–08 Report on Plans and Priorities indicates that the Department's clean energy objective was to "reduce the harmful effects of energy production and consumption that account for 85 % of smog-causing nitrogen oxide emissions and 80 % of Canada's annual GHG emissions, while generating innovative technologies that Canada can market abroad."29 It also identified a transportation-specific objective, "to improve the energy efficiency of new vehicles, influence vehicle operation and maintenance, and expand the production and use of alternative transportation fuels."30

NRCan representatives—the only group of key informants asked to comment on the alignment between program objectives and NRCan's strategic objectives—did not identify any components of the Department's energy-efficient transportation programs that are inconsistent with government priorities and NRCan strategic objectives.

Evaluation question: Is there a legitimate, appropriate and necessary role for NRCan in the program?
  • Yes; however, stakeholders have limited understanding of each other's roles and responsibilities and therefore expressed concern about the potential for overlap and duplication.

Legitimate Role

NRCan undertakes clean transportation activities in accordance with the Energy Efficiency Act, 1992. Additionally, discussions are under way to determine the Department's role and responsibilities related to the regulations for cars and light-duty trucks that will be implemented, starting with the 2011 model year, under the Canadian Environmental Protection Act, 1999.

Appropriate Role

All NRCan representatives firmly believe the Government of Canada has a role to play in reducing GHG emissions. First, they stated that the Government of Canada is the only body that can regulate GHG emissions. Second, they reported that the Government of Canada is responsible for showing leadership and encouraging Canadians to reduce GHG emissions. Most NRCan representatives believe that the Government of Canada "levels the playing field" by providing all stakeholders with access to reliable information on energy-efficient transportation practices and behaviours. These interviewees noted that there is no entity other than the Government of Canada that has the power or financial resources needed to influence behaviour change.

Given that NRCan "seeks to enhance the responsible development and use of Canada's natural resources and the competitiveness of Canada's natural resources products," promoting efficient use of fuel, which is a natural resource, is well within its jurisdiction.31 All NRCan representatives agreed that the energy-efficient transportation programs directly and completely support and align with NRCan's mandate and firmly believe that, because of this mandate, even if the federal government decided climate change was no longer a priority, the need for the Department to continue delivering its energy-efficient transportation programs would still exist.

Necessary Role

Because fuel consumption is directly related to GHG emissions, and because it is in the self-interest of households and firms to take measures to reduce fuel consumption, it might seem reasonable to expect the market to constrain growth in GHG emissions without public intervention. However, economic theory suggests that various forms of market failure may prevent this from occurring. Externalities and information deficits are two forms of market failure where the market costs fail to capture all of the social costs of using a product:

  • Externalities. Car drivers and fleet managers behave as if the price of fuel at the pump includes all of the costs of vehicle operation. The fundamental problem for road transportation is that the costs of gasoline do not include the social and environmental costs of the by-product GHGs, and therefore consumers consume higher amounts of fossil fuel than they would if the price were set to a level that also covered the social costs.
  • Information deficit. If drivers and fleet managers lack information about how their vehicle purchasing decisions and driving behaviour affect fuel consumption and GHG emissions, they may make choices that both increase their fuel expenses and contribute to climate change. Because of the direct relationship between fossil fuel use and GHG emissions, to the extent that more knowledgeable consumers reduce fuel consumption, they also reduce GHGs. However, policy intervention is not always required to respond to information deficits since markets can, and sometimes do, respond to this demand for information. That said, markets may provide incomplete information or may not effectively reach all market segments.

These two themes underpin the programs in this sub-activity. In particular, this can be clearly seen in the idea of externalities where decisions and behaviours of road users impose costs on society and where information deficits limit the decision-making capacity of drivers and fleet owners, thereby causing them to make vehicle purchases and adopt driving and maintenance behaviours that have higher energy use and GHG emissions.

Several federal departments, including NRCan, Transport Canada, and Environment Canada, provide various aspects of transportation-related information, as well as a range of programs, to Canadians. Most key informants expressed some concern about potential overlap and duplication of effort across these federal departments. For example, a few key informants suggested that Transport Canada, which is responsible for the safety and security of on-road vehicles, is becoming more involved in GHG emissions and on-road transportation programs. Additionally, Environment Canada, which is responsible for environmental and air quality issues, will begin regulating carbon dioxide emissions from cars and light-duty trucks in the 2011 model year.

Most key informants said that there is a lot of confusion about the energy-efficient transportation-related roles and responsibilities across different levels of government and between various departments within each level. They suggested that this confusion exists not only among the drivers and fleet managers who are trying to access transportation-related information and programs but also among those working with federal and provincial government departments. Although they recognized that when multiple stakeholders are working together some duplication is inevitable, they indicated that there is a need to increase collaboration, communication and consistency in messaging among federal departments.

Evaluation question: What connection exists between the production and dissemination of information on energy conservation, energy consumption and GHG emissions reductions?
  • Awareness of and knowledge about the relationship between fuel efficiency, GHG emissions and the environment may encourage drivers to make behavioural changes. Complementary interventions such as provision of training, reminders and incentives may increase the uptake of, as well as the sustaining of, behaviour change.
Why is there a need for information in the marketplace?

The literature review shows that when drivers and fleet managers lack information on how to reduce GHG emissions through vehicle purchase behaviours and driving practices, their behaviour not only costs them money, because they use more fuel than needed, but also affects the welfare of others. Also, the literature and many key informants maintain that consumers may be unable to critically assess the vehicle information provided by a manufacturer concerning the specified fuel efficiency. Consequently, they may not incorporate this factor into their purchasing decision,32 although this is getting less likely as some manufacturers now promote the efficiency of their vehicles. More likely, the driver/operator/owner is unaware of how routine maintenance or driving behaviour affects fuel consumption and GHG emissions. These messages are just starting to emerge in the marketplace, in part because of the Transportation Sub-Sub Activity.

While fuel efficiency information for specific vehicles can theoretically be researched by consumers, this takes time, and information on the relationship between driving behaviour and/or vehicle maintenance practices and fuel efficiency is less widely available. Therefore, a role for provision of information about fuel-efficient vehicles and driving behaviour to consumers exists, to the extent that information deficits would limit the decision-making capacity of drivers and fleet owners, thereby encouraging purchasing, driving and maintenance behaviours that increase energy use and GHG emissions.

Limitations of only providing information

The literature suggests that information on how vehicle choice and driving behaviour influence GHG emissions is generally necessary but insufficient to induce large changes in vehicle purchasing, maintenance and driving habits that would reduce GHG emissions. Two examples are:

  • A survey of Canadian drivers' attitudes, awareness and behaviour found that "[while] many Canadians are aware of the types of actions they could take to address climate change […] this does not necessarily translate into taking those actions."33
  • A study of consumers in Montreal found that while positively disposed to alternative fuel and electric vehicles, they were unwilling to trade vehicle performance for lower environmental impact.34

One explanation for the inability of most information-based approaches to substantively alter consumer behaviour is that individuals face a multitude of barriers in making more sustainable choices in transportation, of which insufficient information is only one.35 For example, it is difficult to "write-off" major capital expenditures, and the longer-term response to this information must await the next purchase cycle. In the short term, the decision to reside in a suburban area means that changes in driving distances and destinations cannot be easily altered.

Benefits of supplementing the provision of information with training

The literature indicates that driver training, which represents a more formal approach to delivering education to targeted audiences, primarily novice drivers, is one form of information provision that has been found to have long-term effects on behaviour. There is some evidence that training in some circumstances may enable significant reductions in GHG emissions from transportation at relatively low cost:

  • The U.K. Energy Research Centre asserts that ecodriving campaigns are capable of producing reductions in emissions of 10–15% at a cost below ₤20 (approximately C$35) per tonne of equivalent carbon dioxide.36
  • An international ecodriving workshop concluded that ecodriving is capable of reducing GHG emissions from road transportation by 10%, suggesting that it has "significant potential to deliver CO2 reductions quickly and cost-effectively."37

However, recent research is mixed and has shown that the extent to which learned behaviours are sustained depends on the driver. A report published by Phase 538 found that new drivers tended to forget the lessons, while Zarkadoula et al.39 showed that in the context of a fleet, learning efficient driving practices was rapid and drivers retained the behaviour. The study undertaken by Zarkadoula et al. found that driver training increased fuel efficiency 10.2% in the short term, and 4.35% up to two months later. Thus, although some of the gains associated with training were eroded over time, the training continued to produce fuel savings months later.40

Further, other research has shown that providing drivers with real-time information about fuel consumption, or about factors that can influence fuel consumption, can have a significant impact on driving behaviour. Barth and Boriboonsomsin41 found that providing drivers with real-time information on traffic conditions can reduce fuel consumption and CO2 emissions by 10–20%, while af Wahlberg42 found that equipping buses with feedback devices showing instantaneous fuel use reduces fuel consumption by 2%. Additionally, Barkenbus suggests that incorporating real-time feedback mechanisms into ecodriving programs could potentially double the fuel efficiency gains obtained through ecodriving programs.43

Using community-based social marketing as an alternative to information provision

The literature also suggests that community-based social marketing can help encourage behaviour change. Community-based social marketing can be defined as an approach to changing behaviour in which practitioners "identify the activity to be promoted and the barriers to this activity, and then design a strategy to overcome these barriers, using psychological knowledge regarding behaviour change."44 This hybrid of psychology and social marketing is underpinned by psychological theories about what induces people to change their behaviour. For example, it incorporates the principle of consistency (which suggests that people try to act consistently) in encouraging community members to commit to sustainable patterns of behaviour, since experiments have demonstrated that after people have committed to a particular course of action, they are more likely to pursue it.45 Similarly, it acknowledges that believing one has the resources (i.e., time and money) to undertake an activity and that the behaviour will produce successful outcomes is important in inducing behaviour change (self-efficacy).46

The literature review identified several examples where community-based social marketing was successful, specifically in altering behaviour related to road transportation:

  • A study of Toronto's "Turn it Off" anti-idling campaign concluded that a combination of prompts (anti-idling signs) and eliciting commitments from drivers reduced the incidence of vehicle idling by 27% and idling duration by 72%, relative to control sites.47 This campaign was designed to address barriers to reducing idling behaviour, including comfort, convenience and the mistaken beliefs that idling was easier on vehicle components and used less gas than restarting the engine.48
  • An analysis of data collected for Seattle's "Neighborhoods in Motion" program revealed a 9% increase in bus boardings in the neighbourhood affected by the intervention, versus a 1% decrease in the control neighbourhood.49 To measure the behaviour changes achieved, 10 days' worth of baseline data was collected at 12 sites, including a group of control sites. After the behaviour change strategies were implemented, trained staff measured the incidence of idling at the same locations for another 10 days.50
  • An evaluation of the Greater Vancouver Regional District's (GVRD's) "Employee Trip Reduction Program" estimated an increase in average vehicle ridership of 1.56 to 1.78. Other results included a decrease in single occupant vehicle travel from 57% to 46%, an increase in carpooling from 15% to 21%, and an increase in public transit from 19% to 22%.51 This Program used financial incentives and disincentives to encourage behaviour change. To measure the success of this Program, surveys were conducted following program implementation.52 This measurement approach had limitations because it relied only on self-reported data. Further, it is unclear if baseline data was collected prior to program implementation.

The Transportation Sub-Sub Activity is consistent with the approaches described in the literature

The programs forming the Transportation Sub-Sub Activityuse a combination of information provision and training to encourage behaviour change.

  • The majority of Personal Vehicles programs provide information to the general public through NRCan's website, attendance at auto shows and national campaigns. This information is supplemented at local levels with training, such as tire inflation clinics and training for novice drivers; requests for personal commitments to make behaviour changes; and the provision of reminders to change behaviours or sustain behaviour changes. While behaviour change data is not collected nationally, some local projects have attempted to collect this information through observations and follow-up surveys.
  • The Fleets programs have posted information about energy-efficient transportation on NRCan's website and make information available at trade shows and through personal interactions with fleet managers. Further, a large component of the Fleets programs is the SmartDriver training modules. However, performance data is not systematically collected.

3.2 Performance

Summary

Although there is a rigorous performance measurement strategy for the Transportation Sub-Sub Activity, the indicators included in this strategy do not capture outcome information. Similarly, although the programs systematically collect performance data in an attempt to track performance toward immediate and intermediate outcomes, the performance indicators reported against tend to be focused on the output level, rather than on the outcome level required to measure performance toward outcomes. Thus, the information reported against the indicators is not sufficient to track performance. For example, performance indicators that tend to be outcome-oriented often rely on survey data; however, the Better Energy Efficiency Reporting System (BEERS) indicates only whether or not these surveys have been done rather than the results of the surveys.

The Sub-Sub Activity has collected performance information identified for the immediate and intermediate outcomes; however, the performance indicators tend to be focused on the output level rather than on the outcome level information that is required to fully respond to the summative-type questions posed in the evaluation.

While the Transportation Sub-Sub Activity uses a two-phase modelling approach to calculate achieved GHG emissions reductions, the model uses information about program reach and is dependent upon assumptions about program reach, implementation of behaviour changes, and sustained use of the changed behaviours. Table 8 provides the estimated GHG emissions reductions achieved between 2007–08 and 2008–09.

Table 8: GHG emissions reductions achieved through the ecoENERGY programs
Sub-Sub Activity component Estimated GHG emissions reductions (2007–08 to 2008–09) % progress toward 2010 target
ecoENERGY for Personal Vehicles 0.0614 Mt of 0.1 Mt by 2010 61%
ecoENERGY for Fleets 0.1844 Mt of 0.5 to 0.7 Mt by 2010 26% to 37%
Memorandum of Understanding with the Canadian Automotive Industry 3.1 Mt to 3.4 Mt of 5.3Mt in 2010 58% to 64%

The lack of outcome-level performance data prevents the evaluation from assessing the economy of the Sub-Sub Activity. Nonetheless, the evaluation noted that the programs are using partnerships to minimize program delivery costs and maximize their reach. The evaluation also found that government policies such as a moratorium on advertising and lengthy (or unpredictable) contracting processes have reduced the programs' ability to operate in an efficient and effective manner.

The evaluation identified a number of external factors that may influence or confound the performance of the programs, including fuel prices, changes in consumer demand, U.S. and Canadian regulations and privacy considerations. Some NRCan representatives suggested that concerns over privacy issues may prevent some stakeholders from sharing data with the department.

Key informants identified several possible program improvements, including increased collaboration across and between federal departments; the provision of one-window access to federal government transportation information and programs; increased consistency and regular updating of information; and increased flexibility in government contracts.

3.2.1 Personal Vehicles

Evaluation question: Have the programs increased fuel-efficient driving behaviour by individuals?
  • The Personal Vehicles Program of the Transportation Sub-Sub Activity has undertaken numerous activities to raise driver awareness and increase driver knowledge of fuel-efficient driving behaviour. Behaviour change is not regularly tracked; therefore, the overall impact is unknown.

Within the timeframe of this evaluation, program-level behaviour change data has not been systematically collected.53 One reason for this, according to many NRCan representatives, is that data collection is technically difficult and costly. Nonetheless, to provide some performance information for its training programs, retention studies will be conducted in 2009–10 and 2010–11. A few NRCan representatives indicated previous studies have shown that about half of the participants in driver training programs will retain the information provided, about half of these will implement what they learned, and about 15% of these will continue to follow the guidelines in the future. Further, previous survey results (2006) show that 60% of drivers believe that changing their driving habits and improving vehicle maintenance will result in reduced fuel costs.54

Some individual projects funded through contribution agreements have attempted to measure behaviour change. For example, two of the four case studies on Personal Vehicles projects found measured evidence of behaviour change.

  • The DriveWiser case study (2008 to 2009) found that one month after committing to a behaviour change, depending on the change committed to, between 15% and 81% of individuals actually made the change (based on self-reported data). Table 9 shows the results for various behaviours.
Table 9: Pledges made at DriveWiser presentations vs. reported behavioural changes one month later
Behavioural change % who committed (FY 08–09)
n=422
% committing who made changes
(FY 08–09)
n=72
% who committed (FY 09–10)
n=469
% committing who made changes (FY 09–10)
n=65
Make an effort to carpool once a week 4% 15% 53% 23%
Reduce car use through better errand planning 53% 33% 43% 55%
Change my air filter according to manual 47% 40% 44% 27%
Reduce or stop using "drive-thrus" 44% 50% 49% 18%
Try not to accelerate uphill 46% 50% 39% 23%
Try to stop "jackrabbit driving" in town 44% 50% 51% 32%
Check my tire pressure every month 64% 50% 56% 45%
Warm up car for no more than 30 seconds 53% 56% 54% 59%
Change my oil every 5,000 km (or per manual) 48% 58% 45% 36%
Drive the speed limit 57% 78% -- 73%
Turn off car when idling for longer than 60 seconds (FY 08–09) 57% 81%  
Turn off car when idling for longer than 10 seconds (FY 09–10)   55% 77%
Number of people with whom each participant said they would share the information 1.7 8.5 8 4.8

Source: Clean Nova Scotia Foundation. (2009). ecoENERGY for Personal Vehicles Program: Final Report 2008/09. pp. 7–8.

The Changing Driver Behaviour in London (Anti-Idling Tools) case study (2008 to 2009) found that, based on observation data, no-idling signs and anti-idling awareness campaigns reduce the incidence of idling at various sites. For example, pre- and post-intervention surveys show that incidents of idling at the hospital dropped from 65% or 83% (depending on the area surveyed) to about 5%, and that incidents of idling at the school dropped from 45% to 4% after no-idling signs were installed and the advertising campaign was conducted.

The Personal Vehicles Program has also collected data on the reach of its targeted campaigns, which are developed and delivered through contribution agreements. In 2008:

  • The idle-reduction campaign, according to preliminary results from a municipality/non-governmental organization (NGO) study, reached 3.9 million Canadians. Additionally, in 2006, 28 municipalities in Canada had anti-idling bylaws and 21 planned to implement an idling control bylaw in the next two years.
  • The tire maintenance campaign, based on estimates from current activities and previous program experience, reached 7.7 million Canadians. Further, according to information included in the Better Energy Efficiency Reporting System (BEERS), survey results show that the number of people who properly inflate their tires by measuring tire pressure at least once per month increased 9% between 2003 and 2005. The program estimates that 50% of these people improved their tire inflation.55
Evaluation question: To what extent has fuel-efficient consumer behaviour increased?
  • The Personal Vehicles Program has undertaken numerous activities to raise awareness of the impact of vehicle choice on fuel efficiency and to provide tools to inform purchase decisions. Behaviour change is not regularly tracked; therefore, the overall impact is unknown.

Although data on indicators is collected and reported bi-annually via BEERS, program-level behaviour change data has not been collected. For example, the data reported in the Personal Vehicles outcome of improved fuel efficient driving behaviour by Canadians is that 3.9 million Canadians were reached through idle-reduction campaigns and 7.7 million were reached through tire maintenance campaigns. While this data provides an indication of reach through the campaigns, it does not provide evidence of behaviour change.

However, in 2009–10, the Program plans to collect some data on purchasing behaviours through surveys of auto show attendees. According to the 2006–07 Report to Parliament under the Energy Efficiency Act, survey results indicated that 71% of Canadians are aware that their vehicle choice has an impact on fuel efficiency and 50% would consider the vehicle's impact on the environment in their next vehicle purchase decision.56

Across all groups of key informants, many believe that, generally, Canadians are becoming more aware of, and concerned with, environmental impacts, which will lead to increased demand for fuel-efficient vehicles. Some provincial key informants suggested that this cycle is just gaining momentum. At this point, a few key informants speculated that because hybrids are expensive, the only Canadians currently purchasing these vehicles may be those who are doing it purely for environmental reasons.

A few provincial key informants postulated that consumers are demanding more fuel-efficient vehicles, but manufacturers are not meeting the demand because it is easier and cheaper for them to manufacture the same vehicle, with the same technology, than it is for them to develop new vehicles and technologies. According to the 2006–07 Report to Parliament under the Energy Efficiency Act, between 2005 and 2006, sales of new light-duty vehicles increased by 2% and sales of fuel-efficient subcompact vehicles increased by 20%.57 Note that none of the selected case studies intended to assess purchasing behaviour.

3.2.2 Fleets

Evaluation question: Have the programs increased fuel-efficient driving behaviour by fleet managers and drivers?
  • Behaviour change performance data has not been collected; therefore, the overall impact is unknown.

The outcomes and indicators that are contained in the approved Results-Management Based Accountability Framework and tracked in the Better Energy Efficiency Reporting System (BEERS) do not support a clear response to this evaluation question, partly because most of the data collected to date for the ecoENERGY for Fleets program and reported in BEERS is largely activity-based yet is used as indicators of outcomes. The outcomes that are tracked include increased awareness of programs and benefits, development and/or implementation of best practices by commercial/institutional transportation industry leading to GHG reductions, and reduced operating costs. Examples of data that are currently collected and available are the number of professional development workshops, the number of campaigns held and the number of drivers trained. These indicators provide a measure of the Program's reach, but they do not provide evidence of behaviour change that supports the projection of GHG emissions outcomes, which is the final outcome of the Program.

Most NRCan representatives commented that it is difficult to measure program performance. Many of the attempts to measure performance, which have yielded promising results, have only been conducted on an ad hoc basis. Examples that NRCan representatives and industry representatives offered to demonstrate program success are:

  • many fleets have attempted to measure the impact of driver training and reportedly have obtained positive results;
  • the trucking sector has provided NRCan with anecdotal evidence of energy efficiency improvements in their fleets; and
  • data from electronic control modules from diesel engine school and transit buses is showing large changes in those sectors.

A few NRCan representatives said another indicator of success is that other countries have contacted the Department to discuss the possibility of using or adapting its material.

Additionally, one case study (Transport Hervé Lemieux) found that driver training yielded an 8% reduction in GHG emissions. The other three case studies showed that project final reports did not include behaviour change-related performance data.

Evaluation question: To what extent have fuel-efficient consumer behaviours by fleets increased?
  • The Program has placed reduced emphasis on technology promotion since the rebate program ended in 2005–06. Data on equipment purchases has not been collected since 2005–06; therefore, the overall impact is unknown.

Performance data relating to the purchase of fuel-efficient equipment for vehicles has not been collected since the rebate program, which was part of the Commercial Transportation Energy Efficiency and Fuels Initiative that ended in 2005–06. Through this Program, rebates were issued for 17,585 units. It was estimated that these rebates would result in annual GHG reductions of 0.25 megatonnes.

One NRCan representative said that the Department's former rebate for anti-idling technology was extremely successful in moving the marketplace. This key informant noted that there are very narrow margins in the trucking industry, so fleets need some encouragement to purchase aftermarket technologies. Through this program, the Department received some vehicle performance data from truck hour-meters.

A few NRCan representatives noted that it is difficult to measure technology uptake because fleets are not required to inform the Department if they purchase fuel-efficient equipment or if they use NRCan tools to inform their purchasing decisions. Nonetheless, a few NRCan representatives reported that some fleets have informally told them about their purchases, and others have read articles in trade magazines about the technologies being used by fleets.

3.2.3 Memorandum of Understanding and Regulatory Action

Evaluation question: To what extent has the fuel efficiency of vehicles improved?
  • The Memorandum of Understanding with the Canadian Automotive Industry's contribution to improving the fuel efficiency of vehicles is unknown.

The Joint GHG Memorandum of Understanding Committee, which was established to monitor the progress of the Memorandum of Understanding with the Canadian Automotive Industry, has not published any information on the extent to which the fuel efficiency of vehicles has been increased. The Committee released its first progress report in 2006; this report included the terms of reference for the Committee, a communications strategy and proposed stakeholder engagement options. The Committee has compiled the data for the 2007 report but has not published the report.58

About half of the key informants from all groups said that Memoranda of Understanding are a good way to initiate working with industry, non-governmental organizations and other governments, and mentioned that some of the successes are improved communication and understanding among stakeholders. Additionally, an NRCan representative reported that a representative of a large auto manufacturer said that the Memorandum of Understanding has influenced the company's technological engineering efforts, and some representatives of other federal departments said that some provinces intend to adopt the California GHG regulations for new vehicles.

One provincial government representative said that the Memorandum of Understanding sends the right signal to manufacturers.

In addition to reporting on the results of the Memorandum of Understanding with the Canadian Automotive Industry, in 2007 and 2008, NRCan worked with Transport Canada on developing Fuel consumption regulations for light-duty vehicles under the Motor Vehicle Fuel Consumption Standards Act. Work on these regulations changed in March 2009, when Environment Canada announced that new light-duty vehicles would be regulated under the Canadian Environmental Protection Act, 1999, to limit greenhouse gas emissions and to align with regulations in the approach in the United States.

3.2.4 GHG emissions reductions

Evaluation question: To what extent have the program activities resulted in a reduction in GHG and air pollution emissions?
  • Some progress has been made in personal vehicles, and limited progress has been made in fleets. The progress made by the Memorandum of Understanding, which represents 86–89% of the overall ecoENERGY target, is between 58% and 64%.

Description of the modelling approach used to estimate GHG emissions reductions

The programs forming the Transportation Sub-Sub Activity aim to reduce GHG emissions from on-road transportation. NRCan uses a two-phase approach to model GHG emissions reductions:

  • The first phase, undertaken in advance of program implementation, involves the use of modelling techniques to produce program targets based on expected program implementation and adoption.
  • The second phase, performance modelling, involves essentially the same models, but uses information from program implementation and adoption to measure progress in reaching targets.

The program targets are a one-time snapshot derived using the most recent information available to NRCan at the time they were produced. The current information on program activity is updated semi-annually (twice per year), and the assumptions within the performance models are updated in rotation, based on priority and cost considerations.

Both phases of the modelling process generate estimates of program impact in terms of reduced GHG emissions for each of the programs forming the Transportation Sub-Sub Activity. By adding the impacts associated with each NRCan program, it is possible to estimate the total GHG emissions reductions expected from programming under the Transportation Sub-Sub Activity.

Yet care is required in this approach since the figures generated through the model are only as valid as the assumptions used to calculate the figures. For example, the percentage of the target reached for personal vehicles is influenced by the Program's estimated reach and assumptions about how that reach translates into sustained behaviour change.

One provincial representative cautioned that a single program will not sustain behaviour for a long period of time, since there are many immediate and intervening variables that impact driver behaviour. This individual said programs can only hope to have a short-term impact and concluded that other motivators beyond the Program are needed to sustain behaviour change. The implication for the Transportation Sub-Sub Activity is that continuous interventions and reminders are needed to influence additional and sustain any previously achieved changes in behaviour.

Estimated GHG emissions reductions achieved

Table 10 shows the GHG emissions reductions achieved through the three components of the ecoENERGY programs.

Table 10: GHG emissions reductions achieved through the ecoENERGY programs
Sub-Sub Activity component Estimated GHG emissions reductions (2007–08 to 2008–09) % progress toward 2010 targets
ecoENERGY for Personal Vehicles 0.0614 Mt of 0.1 Mt by 2010 61%
ecoENERGY for Fleets 0.1844 Mt of 0.5 to 0.7 Mt by 2010 26% to 37%
Memorandum of Understanding with the Canadian Automotive Industry 3.1 Mt to 3.4 Mt of 5.3Mt in 2010 58% to 64%

GHG emissions reductions achievements for pre-ecoENERGY programs are available in an internal summary report prepared in 2006–07 as part of the Climate Change Interim Strategy. This report identifies the following cumulative GHG reduction achievements since the programs began, which includes fiscal years prior to the evaluation period:59

  • The Motor Vehicle Fuel Efficiency Initiative achieved cumulative GHG emissions of 1.7 megatonnes. Its target was a 5.2-megatonne reduction by 2010.
  • The Marketing of Fuel Efficient Vehicles Program achieved cumulative GHG emissions of 0.09 megatonnes. Its target was a 0.34-megatonne reduction by 2010.
  • The Freight Efficiency and Technology Initiative achieved cumulative GHG emissions of 1.45 megatonnes. This report indicates that its target was a 1.5-megatonne reduction by 2010, but the Results-based Management and Accountability Framework stated the target was at least a 2-megatonne reduction.
  • The Commercial Transportation Energy Efficiency and Fuels Initiative achieved cumulative GHG emissions of 0.202 megatonnes. This report indicates that its target was a 1.23-megatonne reduction by 2012, but the Results-based Management and Accountability Framework stated the target was a 2.45-megatonne reduction by 2010.

Note that the percentage of the target reached for personal vehicles is influenced by the Program's estimated reach and assumptions about how that reach translates into sustained behaviour change. An exception to this qualification is the Memorandum of Understanding with the Canadian automotive industry.

In the case of the Memorandum to Understanding, results are tracked through a modelling framework that monitors the difference between the actual performance and the forecasted performance from a reference case. This model is based on a number of factors controlled by the auto industry (e.g., technologies introduced to reduce fuel consumption) as well as factors outside the control of the auto industry (e.g., vehicle sales and distance travelled). The information is tracked annually by the Joint Government-Industry Committee to observe whether the Memorandum of Understanding has had an impact.

3.2.5 Other performance-related results

Evaluation question: Have there been unintended (positive or negative) outcomes? Were any actions taken because of these?
  • Key informants identified a few unintended outcomes but did not identify any actions being taken because of them.

Representatives of NRCan provided the following examples of positive unintended impacts:

  • Fleet managers have reported cost-savings resulting from reductions in at-fault incidents and reduced maintenance costs due to reduced frequency of oil changes, brake replacements and tire replacements.
  • Discussions about fleet fuel efficiency can lead fleets to consider other areas where they can improve energy efficiency (e.g., heating, lighting).
  • Representatives from other countries have contacted NRCan about using its information and materials for their own projects.

One NRCan representative provided the following example of a negative unintended impact:

  • While the Transportation Sub-Sub Activityprograms have encouraged vehicle manufacturers to advertise fuel efficiency information, they may not advertise this information consistently or comprehensively. Manufacturers, for example, may be reporting only highway mileage information, and/or they may be advertising vehicles as the most fuel-efficient in their class without offering class comparisons. This may result in consumers purchasing less fuel-efficient vehicles than they intended.
Evaluation question: How economic are the programs?
  • A lack of immediate and intermediate outcome-level performance data limits the evaluation's ability to accurately assess the economy of the programs.

Although the economy of the programs cannot be determined, key informants commented on some of the approaches the programs are using in an effort to maximize their economy.

Personal Vehicles

Many NRCan representatives spoke of how the strategic use of various program delivery mechanisms has contributed to the cost-effectiveness of the program. They reported that the Department is encouraging programs to deliver their activities through partnerships. Aligning with this directive, about 23 % of the Program's funding goes toward contribution agreements with delivery partners. Many NRCan representatives noted that partnerships enable the Program to get its messages out to drivers without advertising themselves, since a moratorium on advertising, which was referred to in the Better Energy Efficiency Reporting System, prevents programs from advertising. These partnerships also create improved opportunities to reach specific target audiences since delivery partners are often in a better position than NRCan to reach them. For example, provincial government partners, who have the authority to issue driver's licences, work with their network of driving schools, which, in turn, deliver the Auto$mart Program to novice drivers. A few NRCan representatives also reported that posting information such as the Fuel Consumption Guide on NRCan's website has improved the cost-effectiveness of the Program.

Fleets

NRCan representatives all asserted that the Program has done a substantial amount of work with the nine full-time equivalents it has available for the ecoENERGY for Fleets program. Although they acknowledged that the SmartDriver Program was cost-inefficient and unsustainable when NRCan staff were on the road delivering it, they suggested that the shift to using contracted master trainers to deliver the Program has improved its cost-effectiveness.

Most NRCan representatives noted that internal federal government policies and processes have reduced the Program's ability to operate in an efficient and effective manner. Consequently, the ecoENERGY for Fleets Program is inefficient because the process of getting contracts in place is difficult and time-consuming. Further, the Program has not been effective because—as reflected in the ecoENERGY for Fleets financial reports and in data included in the Better Energy Efficiency Reporting System—it has not been able to issue all of its available grants and contributions funding to projects. This means that the Program has not completed all of its intended activities and therefore is in jeopardy of not being able to meet its four-year performance targets.

Evaluation question: To what extent does the design of the programs produce the required behaviour change to achieve outcomes in an economic way?
  • The Program theory is sound; however, a lack of outcome-level performance data limits the evaluation's ability to assess the performance and economy of the programs.

The Program theory is sound. As most NRCan representatives reported and according to the Department's mandate and authorities, for the most part, NRCan delivers programs to influence behaviour change rather than to seek to develop regulations to reduce GHG emissions from on-road transportation. Although the Department has the authority to regulate fuel-efficient equipment for vehicles under the Energy Efficiency Act, it has not exercised this option.

A few key informants from all respondent groups suggested that delivering integrated social marketing campaigns with multiple messages could increase the Program's cost-effectiveness because it will allow the programs to present a variety of complementary messages to a target population at one time.

Evaluation question: What internal and external factors influence/confound the effectiveness, efficiency and economy of the programs?

All groups of key informants identified a number of external factors that may influence or confound the effectiveness, efficiency and economy of the programs.

  • Fuel prices: The most frequently mentioned factor was fuel prices. A few NRCan representatives reported that an increase in fuel prices in 2008 led to increased small car sales and reduced SUV sales. However, they speculated that reduced fuel prices, and the subsequent reduction in the price of large vehicles, combined with the economic downturn in 2009, may have offset these gains.

    Although most provincial representatives agree that fuel prices can increase consumer demand for fuel-efficient vehicles, they suggested that the impacts are short-term in duration. They reported that increases in gas prices tend to lead to a spike in sales of fuel-efficient vehicles, along with a reduction in kilometres travelled. Conversely, if fuel prices fall, fuel efficiency may not be seen as important.
  • Changes in consumer demand: A few representatives of other federal departments noted that some consumers are demanding smaller and more efficient vehicles. However, they cautioned that for new technologies to penetrate the market, consumers need to be comfortable with them (e.g., electric cars and natural gas vehicles drive differently than traditional gas vehicles).
  • U.S. and Canadian regulations: All key informants said regulations can have a significant impact on the fuel efficiency of new vehicles.
  • Privacy considerations: Most NRCan representatives suggested that concerns over privacy issues may prevent some stakeholders from sharing performance data with the Department, thereby limiting its ability to measure the performance of its programs.

Many NRCan representatives mentioned that certain government policies (such as the moratorium on advertising and restrictions on public opinion research) have made it challenging to deliver and measure the impact of the programs. Additionally, they said internal contracting policies and processes have made it difficult to put contracts in place, which has prevented the programs from completing all of their intended activities.

Evaluation question: What changes would improve program delivery and impact?

The following suggestions to improve program delivery and impact were offered across key informant groups:

  • improving collaboration across and cohesiveness among federal government departments;
  • offering industry, stakeholders and the general public one-window access to federal government information and resources;
  • increasing the consistency of information on fuel-efficient driving practices that the federal and provincial governments are providing to drivers;
  • regularly reviewing and updating (such as bimonthly) driver training materials; and
  • increasing the flexibility of contract/contribution agreement deadlines, especially in cases where delays are the result of internal NRCan processes.

4.0 Conclusions

The Transportation Sub-Sub Activity is intended to help Canada meet its commitment under the Kyoto Protocol to reduce GHG emissions by an average of 6% (or 240 megatonnes) from 1990 levels between 2008 and 2012. By encouraging drivers to use more fuel-efficient buying, driving, and vehicle maintenance behaviours and encouraging manufacturers to improve the fuel efficiency of new personal vehicles, the Sub-Sub Activity intends to reduce GHG emissions from on-road transportation by 6.1 megatonnes.

4.1 Relevance

Natural Resources Canada has a role to play in regulating energy efficiency to support the Government of Canada's role in reducing Canadian GHG emissions. The federal government is viewed as a neutral and trustworthy information source, and it has the financial resources needed to fund behaviour change projects. Given its mandate to enhance the responsible use of natural resources, of which fuel is one, it is appropriate for NRCan to deliver fuel efficiency-related behaviour change programming.

The program theory underlying the Transportation Sub-Sub Activity programs is sound. The bulk of the programming uses driver training, information dissemination through publications, websites and trade shows, as well as community-based social marketing, to encourage behaviour change.

However, the Sub-Sub Activity programs have the potential to contribute to GHG emissions reductions in areas where they have a legitimate role on knowledge development and sharing. These programs. along with those of other relevant departments, collectively take action where appropriate. The PAA's component programs, including the Memorandum of Understanding, have the potential to reduce total on-road transportation GHG emissions by approximately 5% and total Canadian GHG emissions by less than 1%. Further, the community-based social marketing (e.g., driver education-targeted campaigns) and outreach portions of the Sub-Sub Activity has the potential to reduce total on-road transportation GHG emissions by 0.1%. To maintain any GHG emissions reductions achieved, individuals must sustain the behaviour changes they implemented. An additional point for consideration is that several factors contribute to changes in behaviour, such as fuel prices and changes in consumer demand.

4.2 Performance

Variations of these programs have been operating since the 1990s. While recognizing that the programs have evolved and expanded over time, an historical record of consistent, time series performance data has not been collected. The programs are not capturing sufficient outcome-level data and instead focus on activity indicators. Many of the indicators included in the performance measurement strategy are activity/output-based, and therefore do not measure behaviour change. Although the programs conduct surveys to collect some indication of behaviour, the surveys are not conducted frequently enough to provide evidence of program impact. Further, strong baseline data to support statements relating to changes in awareness and behaviour is not available. Proper measurement of behaviour change requires the conduct of pre-post surveys, which can be costly and time consuming. However, clearly defining the aspects of and limiting the scope of behaviour change to be measured may help control the cost and time required to obtain the desired measurements.

Nonetheless, all groups of key informants believe that the programs are beginning to influence behaviour change. Fleets managers have told NRCan representatives about some of the fuel efficiency gains they have made. Additionally, some local-level projects have attempted to systematically measure their impact. The results of these projects offer important illustrations of the effectiveness of the interventions conducted through these programs.

Unfortunately, the lack of outcome performance data prevented the evaluation from thoroughly assessing the economy of the programs. However, evidence suggests that some of the programs' delivery approaches are maximizing their available resources. Examples of potentially cost-effective practices are using partnerships to deliver messages, using contracts/contribution agreements to deliver training sessions, and posting information on the Internet.

5.0 Recommendations

  1. NRCan should overhaul the performance measurement strategy in three main ways:
     
    1. establishing detailed and timely data capture for a select number of key behaviour change (outcome) indicators;
    2. ensuring that proponents are accountable for delivering performance data by providing them with technical guidance and/or imposing funding hold-backs where deliverables are insufficient; and
    3. ensuring NRCan's ability to track outcome data by requiring proponents to obtain participants' permission to be contacted within two years following the completion of a project.
       
  2. NRCan should work with other federal departments to ensure better collaboration, communication and coordination of on-road transportation activities.

Annex 1: Logic Model

Logic Model

Annex 2: Summary Information on Transportation Case Studies

Project name Description Objective Results NRCan expenditures ($)
ecoENERGY for Personal Vehicles

Be Car Care Aware

(Car Care Canada)

  • Develop advertising and articles related to vehicle maintenance and the environment
  • Implement a media campaign
  • Customize and translate posters and brochures
  • Develop a website
  • Conduct industry presentations, consumer shows, and media tours
  • Prepare point-of-sale kits
  • Conduct car check clinics and events
  • Offer a young drivers program (ads aimed at youth and Car Care Guides for high schools)
To build awareness and knowledge to prepare consumers to make sensible decisions about vehicles

To help consumers recognize the link between vehicle maintenance and protecting the environment

  • 19,542 visitors to the booth at auto shows
  • 10,500 promotional items distributed
  • 52,000 Car Care Guides distributed
  • 6 industry presentations
  • 27 Car Care clinics for women
  • 7 car check events
  • 338 inspection forms distributed
  • 83,524 visitors to website
  • 200 registrations for e-newsletter
  • 6,000 repair facilities listed on website
  • Program awareness increased from 4% in 2004 to 10% in 2007 (pre-current funding arrangement)
  • Website indicates that 73% of consumers could recognize some of the Program's initiatives
  • No evidence of behaviour change collected/provided

$1,114K

(2007 to 2010)

Be Tire Smart

(Rubber Association of Canada)

  • Deliver a media campaign
  • Attend auto shows
  • Deliver a Leadership Program, which uses "ambassadors" to increase program reach
  • Conduct research (1) to investigate the relationship between tire maintenance and driving behaviour with fuel consumption and environmental and safety impacts, and (2) to measure behaviour changes resulting from the program
  • Continue to produce and distribute existing publications
  • Improve the program website to increase the availability of information and consumer tools
  • Implement a National Be Tire Smart Week
  • Develop and deliver an educational program to automotive technicians

To encourage Canadian drivers to adopt proper tire maintenance practices

To increase fuel efficiency, reduce GHG emissions and improve vehicle safety

  • Media and promotional activities reached an estimated 59 million individuals
  • In 2008, the number of unique website visitors increased by nearly two-thirds, and the number of pages viewed by each visitor increased by more than one-quarter from the previous year
  • The prevalence of tire over-inflation increased slightly
  • In 2009, a survey was conducted to evaluate program impact; however, results were not available at the time of this evaluation

$1,219K

(2007 to 2011)

DriveWiser

(Clean Nova Scotia Foundation, Nova Scotia)

  • Deliver presentations and workshops related to vehicle maintenance in educational institutions
  • Develop a DriveWiser resource kit for driving school instructors and deliver the resource kit to young drivers through driving instructors
  • Collaborate with gas stations, NGOs, other vehicle-related industries and driving schools to increase the reach of DriveWiser materials
  • Create pages targeting 15 to 21-year-olds on the DriveWiser website
  • Use radio advertising to convey program messaging
  • Develop and deliver DriveWiser 101
  • Increase stakeholder involvement with DriveWiser (in-store displays, incentive program for dealerships to promote the most fuel-efficient vehicles in their fleet)
  • Increase program presence in smaller communities
To help Nova Scotians drive, maintain and buy for fuel efficiency to save money, reduce GHG emissions and improve air quality
  • Obtained commitments for specific behaviour changes and followed up one month later. Examples of results achieved:
  • 53% committed to reduce car use through better errand planning, and 55% of these made the change
  • 47% committed to change their air filter according to the instructions in the vehicle manual, and 27% of these made the change
  • 44% committed to try to stop "jackrabbit driving" in town, and 32% of these made the change
  • 53% committed to warm the car up for no more than 30 seconds, and 59% of these made the change
  • 57% committed to drive the speed limit, and 73% of these made the change

$305K

(2008 to 2010)

Changing Driver Behaviour in London (Anti-Idling Tools)

(City of London, Ontario)

  • Conduct surveys of idling incidence and duration at daycares, schools, workplaces and institutions, and commercial plazas. Four sets of surveys were conducted at intervention group locations where Idle-Free Zone signs were posted and control group locations where no signs were posted. Each group included at least three locations. Baseline and post-intervention data was collected.
  • Conduct a multimedia anti-idling advertising campaign (print and radio advertisements, billboards and banners on the London Free Press website) from late January 2009 to the end of February 2009
  • Survey households to determine the ability to recall the anti-idling advertising campaign

To reduce excessive idling of vehicle engines where idling is prone to occur

To measure the effectiveness of a multimedia advertising campaign in reducing the incidence and duration of idling

To measure the seasonal influence of seasonal temperatures on the incidence and duration of idling

  • Examples of the impact of signs alone: idling at one hospital location dropped from about 83% to 5% and at another hospital location from about 65% to below 5%. Idling at one school dropped from about 45% to 4%.
  • Effectiveness of advertising campaign: 31% of households recalled the radio ads, 16% recalled the newspaper ads, and 11% recalled the billboards
  • The advertising campaign appears to have influenced the behaviour of drivers at the control group locations
  • The combination of anti-idling signs and advertising appears to have had a greater influence on behaviour than advertising alone
  • Aside from workplaces and institutions, idling duration increased as temperature dropped. Additionally, idling incidence at schools and commercial plazas increased.
  • Estimated cost-effectiveness for anti-idling signs ($ per one tonne reduction GHG emissions): $25 to $30 at schools, $45 to $55 at daycares, $10 to $15 at workplaces/institutions and $5 to $10 at commercial plazas
  • Estimated cost-effectiveness for a multimedia advertising campaign: $20 per one tonne reduction GHG emissions

$52K

(2008 to 2009)

ecoENERGY for Fleets

Transport Hervé Lemieux

(Quebec)

  • Implement an awareness campaign (informational insert with paycheques, posters at truck yard entrances, lapel pins and driver checklists)
  • Train all drivers using a combination of classroom training and an individual driving session
  • Install on-board computers in trucks to measure performance
  • Survey drivers about their driving habits and satisfaction with the training received

To reduce GHG emissions from highway trucking by 5% (equivalent to 650 tonnes per year)

To reduce average individual truck fuel consumption from 47.4 litres per 100 km to 45 litres per 100 km

  • GHG emissions reduced by 8% (about 1,000 tonnes per year)
  • Contributed to the development of a company culture of environmental protection and a positive company image

$248K

(2007 to 2009)

Smart Taxis Encouraging Environmental Respect

(St. John's, Newfoundland and Labrador)

  • Evaluate workshops from the first phase of the project, which was completed in 2002
  • Expand the education program to reach more taxi drivers
  • Transfer taxi industry workshops to other fleets
  • Expand outreach efforts to the general public and other targeted groups

To provide the taxi industry and other fleet drivers for the City of St. John's with the knowledge and tools needed to engage in sustainable behaviour

To encourage and promote fuel-efficient and environmentally aware behaviours

  • Insufficient data could be collected to evaluate the Phase 1 workshops
  • Conducted fewer than anticipated workshops with taxi drivers, but engaged them in other activities such as a vehicle emissions clinic and a Taxi Driver Appreciation Day
  • Most of the activities related to transferring the taxi industry workshops to other fleets did not occur
  • Updated the program website to appeal to a wider audience
  • Produced a new video for workshops with the taxi industry
  • Co-hosted two high school forums on driving behaviour and littering

$121K

(2004 to 2006)

SmartDRIVER for Transit

(Canadian Urban Transit Association, Quebec, Ontario, Alberta, Yukon)

  • Develop the curriculum for the SmartDRIVER for Transit Program, regularly update its content, and make the information available in both official languages
  • Develop a training video and a pilot site program from training
  • Certify Master Trainers
  • Deliver train-the-trainer workshops
  • Undertake promotional activities (promote the program at conferences, post information on the Internet, place ads in magazines)
  • Develop a web-based performance tracking program
  • Identify barriers to program uptake and identify potential program improvements

To complete a tested training tool that could achieve the projected result of a 10% reduction in fuel consumption and GHG emissions

To improve transit operators' knowledge of defensive driving techniques that incorporate efficient driving

  • Program design is essentially complete and poised for national roll-out
  • Certified three Master Trainers
  • Piloted the Program at five sites (training and evaluation)
  • Collected data on fuel economy parameters at five sites prior to training and six months after training. Results vary across sites; therefore, generalizations about program effectiveness cannot be made. For example, idling time decreased by 27% at one site but increased by 28% at another site.

$1,105K

(2005 to 2010)

FPInnovations, (Feric Division)

(Forest Engineering Research Institute of Canada)

  • Research to assess the ability of various products and technology to reduce GHG emissions (e.g., dual power engines, hydrogen fuel injection and fuel additives)
  • Develop technical information for SmartDRIVER materials
  • Deliver train-the-trainer sessions and refresher courses
  • Provide workshops on equipment selection, maintenance and replacement schedules
  • Represent FleetSmart at trade shows
  • Distribute transportation bulletins
  • Encourage fleets to install on-board computers in trucks to obtain information on vehicle performance and contribute to decision-making processes regarding routes and vehicle downtime
To achieve an overall reduction of 5% in GHG emissions by the forestry transportation industry, which is approximately equivalent to 0.1 megatonnes of GHGs per year
  • Estimated fuel savings are 2.5 million litres. Ongoing annual savings are estimated at 5 million litres.
  • Estimated reduction in CO2 emissions are 7,000 tonnes. Estimated ongoing annual savings are estimated at 13,000 tonnes.
  • Over 400 on-board computers are in use
  • Identified two fuel additives that offer economically viable fuel savings
  • Trained 300 trainers
  • 1,900 truckers used a self-directed SmartDRIVER CD
  • 600 industry members attended 11 seminars

$946K

(2004 to 2007)

1 Under the 2010 Copenhagen Accord, Canada has updated this target to be a reduction of GHG emissions by 17 % by 2020, relative to 2005 levels.

2 Environment Canada. (2009b, May). A Climate Change Plan for the Purposes of the Kyoto Protocol Implementation Act – 2009. Retrieved on March 5, 2010, from http://www.climatechange.gc.ca/default.asp?lang=En&n=4044AEA7-1.

3 Motor Vehicle Fuel Efficiency Initiative. Results-based Management and Accountability Framework. p. 2.

4 Marketing of Fuel Efficient Vehicles. Results-based Management and Accountability Framework. p. 2.

5 ecoENERGY for Personal Vehicles. Results-based Management and Accountability Framework. p. 5.

6 Freight Efficiency and Technology Initiative. Results-based Management and Accountability Framework. p. 4.

7 Ibid.

8 Commercial Transportation Energy Efficiency and Fuels Initiative. Results-based Management and Accountability Framework. p. 3.

9 ecoENERGY for Fleets. Results-based Management and Accountability Framework. p. 8.

10Clean Air Regulatory Agenda. (2008). Results Framework. p. 4.

11 Government of Canada (GoC). (2009, April 4). Notice of intent to develop regulations limiting carbon dioxide emissions from new cars and light-duty trucks. Canada Gazette, Part I, 143(14). Retrieved on March 5, 2010, from http://canadagazette.gc.ca/rp-pr/p1/2009/2009-04-04/html/notice-avis-eng.html#d110.

12NRCan (no date). <cite>Mandate and Vision</cite>. Retrieved on October 20, 2009.

13 Justice Canada. (no date). The Energy Efficiency Act. Retrieved on October 20, 2009, from http://laws-lois.justice.gc.ca/eng/acts/E-6.4/index.html.

14 Transport Canada. (2007). Motor Vehicle Fuel Consumption Standards Act Proclaimed. Retrieved on October 20, 2009, from http://news.gc.ca/web/article-eng.do?crtr.sj1D=16&crtr.mnthndVl=12&mthd=advSrch&crtr.dpt1D=6695&nid=359589&crtr.lc1D=&crtr.tp1D=&crtr.yrStrtVl=2007&crtr.kw=Motor%2BVehicle%2BFuel%2BConsumption%2BStandards%2BAct%2B&crtr.dyStrtVl=2&crtr.aud1D=&crtr.mnthStrtVl=1&crtr.page=1&crtr.yrndVl=2007&crtr.dyndVl=31.

15 A logic model for the Transportation Sub-Sub Activity appears in Annex 1.

16NRCan. (2008b). Improving Energy Performance in Canada. Report to Parliament Under the Energy Efficiency Act for Fiscal Year 2006–2007.

17 Work relating to alternative fuels was beyond the scope of this evaluation.

18 Government of Canada (GoC). (2008, November). Speech From the Throne 2008: Protecting Canada's Future. Retrieved on October 20, 2009, from http://www.sft-ddt.gc.ca/eng/media.asp?id=1383.

19 Under the 2010 Copenhagen Accord, Canada has updated this target to be a reduction of GHG emissions by 17 % by 2020, relative to 2005 levels.

20 Department of Finance Canada. (2008, February). The Budget in Brief 2008. Retrieved on October 20, 2009, from http://www.budget.gc.ca/2008/pdf/brief-bref-eng.pdf.

21NRCan. (2009a). Energy efficiency trends in Canada, 1990 to 2005. Retrieved November 2, 2009, from http://oee.nrcan.gc.ca/Publications/statistics/trends07/index.cfm?attr=0. p. 36.

22 Ibid. p. 39.

23NRCan. (2009b, April 20). Transportation sector – energy use analysis. Website of Natural Resources Canada. Retrieved November 16, 2009, from http://oee.nrcan.gc.ca/corporate/statistics/neud/dpa/tablesanalysis2/tran_00_1_e_4.cfm?attr=0.

24 Environment Canada. (2009b, May). A Climate Change Plan for the Purposes of the Kyoto Protocol Implementation Act – 2009. Retrieved on March 5, 2010, from http://www.climatechange.gc.ca/default.asp?lang=En&n=4044AEA7-1.

25NRCan. (2006).Canada's energy outlook: the reference case 2006. Retrieved May 28, 2009 from /publications/energy-outlook/1165, p. 56.

26 Government of Canada (GoC). (2008, November). Speech From the Throne 2008: Protecting Canada's Future. Retrieved on October 20, 2009, from http://www.sft-ddt.gc.ca/eng/media.asp?id=1383.

27 Department of Finance Canada. (2008, February). The Budget in Brief 2008. Retrieved on October 20, 2009, from http://www.budget.gc.ca/2008/pdf/brief-bref-eng.pdf. p. 13.

28 Treasury Board Secretariat of Canada (2008–09). Clean Air Agenda. Retrieved on April 21, 2009, from http://www.tbs-sct.gc.ca/hidb-bdih/initiative-eng.aspx?Hi=12.

29NRCan. (2008a). 2007–08 Estimates. A Report on Plans and Priorities. p. 7.

30 Ibid. p. 29.

31NRCan (no date). Mandate and Vision. Retrieved on October 20, 2009.

32 Howarth, R. B., & Sanstad, A. H. (1995). Discount Rates and Energy Efficiency. Contemporary Economic Policy, 13, 101-109; Gillingham, K., Newell, R., & Palmer, K. (2009). Energy Efficiency Economics and Policy. Discussion Paper, Resources for the Future. Retrieved August 30, 2009, from http://www.rff.org/RFF/Documents/RFF-DP-09-13.pdf.

33 Decima Research Inc. (2002). 2002 survey of driver attitudes, awareness, and behaviour Final Report. p. 5.

34 Ewing, G., & Sarigöllü, E. (2000). Assessing consumer preferences for clean-fuel vehicles: A discrete choice experiment. Journal of Public Policy & Marketing, 106–118.

35 McKenzie-Mohr, D. (2009). Fostering sustainable behavior: Community-based social marketing. Retrieved August 31, 2009, from http://cbsm.com/public/images/FosteringSustainableBehavior.pdf .

36 UK Energy Research Centre. (2009). What policies are effective at reducing carbon emissions from surface passenger transport? A review of interventions to encourage behavioural and technological change. Retrieved November 16, 2009, from http://www.ukerc.ac.uk/Downloads/PDF/09/0904TPATransport.pdf. p. viii.

37 IEA, International Transport Forum, Intelligent Energy Europe, Ministerie van Verkeer en Waterstaat, & ecodrive.org. (2008, February 29). Workshop on ecodriving: Findings and messages for policy makers. Retrieved January 6, 2010, from http://www.iea.org/work/2007/ecodriving/ecodrivingworkshopfindings_en.pdf. p. 2.

38 Phase 5. (2003). Auto$mart student driving kit retention and levels of understanding study. Final Report.

39 Zarkadoula, M., Zoidis, G., & Tritopoulou, E. (2007). Training urban bus drivers to promote smart driving: A note on a Greek eco-driving pilot program. Transportation Research Part D: Transport and Environment, 12, 449–451.

40 Two things should be noted about these findings. First, the sample consisted of three drivers; the same findings may not have been obtained if a larger sample had been used. Second, as in the Phase 5 study, knowledge retention at two months may not accurately reflect long-term knowledge retention.

41 Barth, M., and Boriboonsomsin, K. (2009). Energy and emissions impacts of a freeway-based dynamic eco-driving system. Transportation Research Part D, 14(6), 400–410.

42 af Wahlberg, A. E. (2007). Long-term effects of training in economical driving: Fuel consumption, accidents, driver acceleration behavior and technical feedback. International Journal of Industrial Ergonomics, 37(4), 333–343.

43 Barkenbus, J. (2010). Eco-driving: An overlooked climate change initiative. Energy Policy, 38, 762–769. p. 765.

44 McKenzie-Mohr, D. (2000). Fostering sustainable behavior through community-based social marketing. American Psychologist, 55(5), 531–537. p. 531.

45 Seethaler, R. (2005). Using the six principles of persuasion to promote travel behavior change: Preliminary findings of a TravelSmart pilot test. Retrieved September 11, 2009, from http://www.transport.vic.gov.au/__data/assets/pdf_file/0013/31234/Seethaler2004.pdf

46 Haldeman, T., and Turner, J. W. (2009). Implementing a Community-Based Social Marketing Program to Increase Recycling. Social Marketing Quarterly, 15(3), 114–127. p. 116.

47 Kassirer, J. (2009a). Turn it Off. Tools of Change Website. Retrieved September 18, 2009, from http://www.toolsofchange.com/en/case-studies/detail/138.

48 Ibid.

49 Cooper, C. (2006). Successfully changing individual travel behavior: Applying community-based social marketing to travel choice. King County Metro Transit. Retrieved September 14, 2009, from http://www.metrokc.gov/kcdot/transit/inmotion/doityourself/images/ResearchPaper.pdf. p. 10.

50 Kassirer, J. (2009a). Turn it Off. Tools of Change Website. Retrieved September 18, 2009, from http://www.toolsofchange.com/en/case-studies/detail/138.

51 Kassirer, J. (2009b). Vancouver's Employee Trip Reduction Program. Tools of Change Website. Retrieved September 16, 2009, from http://www.toolsofchange.com/en/case-studies/detail/28.

52 Ibid.

53 While the program area does periodically conduct studies to measure behaviour change data, such studies were not conducted during the time frame of this evaluation.

54NRCan. (2008b). Improving Energy Performance in Canada. Report to Parliament under the Energy Efficiency Act for Fiscal Year 2006–2007.

55NRCan. (2008b). Improving Energy Performance in Canada. Report to Parliament Under the Energy Efficiency Act for Fiscal Year 2006–2007.

56 Ibid.

57 Ibid.

58 This report was approved by the Minister in May 2010 and is expected to be published on the NRCan website in 2010.

59 Note that much of this ended to reflect changes in Government priorities resulting from a change in Government.