Travel behaviour change evaluation procedures

Technical report

Print version: Travel behaviour change evaluation procedures (PDF, 1.57 MB, 98 pages)

Executive summary

Introduction

Travel Behaviour Change (TBhC) projects are a relatively new category of transport expenditure that employ education, information and marketing based approaches to achieve voluntary changes in the travel behaviour of individuals. Experience to date in New Zealand has found that existing project evaluation procedures are not well suited to TBhC projects. Consequently Transfund and EECA commissioned Maunsell Australia, Pinnacle Research, and Booz Allen Hamilton to review New Zealand and international TBhC procedures and experience, and develop evaluation procedures and guidelines for practitioners in New Zealand.

Evaluation methodology

A key requirement is for the TBhC evaluation procedures developed in this project to be consistent with the Transfund Allocation Process and not inconsistent with other existing Transfund evaluation and funding procedures. Transfund is still refining details of its Allocation Process but the TBhC procedures developed in this project are consistent with the Allocation Process as at November 2004. This involves three categories of information requirements in proposals: seriousness and urgency, effectiveness, and economic efficiency. Each of these is covered in this report and in the accompanying guidance handbook.

A number of factors influenced the selection of the most appropriate evaluation framework for assessing the economic efficiency of TBhC proposals. These included analysis of issues raised by previous evaluations of TBhC proposals in New Zealand and elsewhere, the need for a theoretically sound evaluation framework that was not inconsistent with other Transfund evaluation procedures and that used common benefit parameters as far as possible, and the need for a method which estimated the benefits obtained by travel behaviour changers.

An economic evaluation approach based on changes in perceived costs/benefits, with addition of resource cost corrections and externality effects, was adopted as best satisfying the identified requirements. This is a form of cost benefit analysis that is consistent with other Transfund evaluation procedures, in particular the passenger transport funding procedures and the evaluation procedures for congested network induced traffic effects.

Benefits

The evaluation procedures include three main benefit categories:

• Benefits to travel behaviour changers
• Resource cost corrections
• Externality benefits

Benefits to behaviour changers are derived from the mode share relationships contained in strategic transportation planning models. The mode share of one mode to another is a function of the difference in generalised cost between the two modes. The relationship can be used in reverse to determine the change in generalised cost difference that is required to achieve an observed change in mode share. Because the mode share relationships are calibrated to actual behaviour this generalised cost difference can be equated to the perceived benefit associated with a given change in mode share. The Wellington and Auckland strategic transportation models were used to determine relevant values for New Zealand. The proposed benefit value for travel behaviour changers is $1.00 for a four percentage point change in mode share from car to public transport or cycle/walk.

Travellers’ perceived benefits are not always an accurate reflection of the resource cost changes which are required for a transport project evaluation. Resource cost corrections are required in addition to the travel behaviour changer benefit when there is a divergence between perceived costs and resource costs. Examples include private vehicle operating costs and car parking costs where resource costs tend to be under-perceived, health benefits of cycling and walking which are also considered to be under-perceived, and public transport fares which are perceived as a cost but in fact are a transfer rather than a resource cost. Appropriate resource cost correction values have been derived for New Zealand TBhC evaluations, mostly based on the analysis carried out during development of Transfund’s passenger transport funding policy.

In addition to the internal perceived and unperceived benefits and disbenefits to travel behaviour changers, TBhC projects also result in externality effects on other transport system users and on society. Examples include less congestion for remaining road users, reduced environmental effects, and reduced health system costs from fewer accidents and improved fitness. Values have been determined for these and other relevant externality benefit categories.

Other unquantified potential benefits such as enhanced community cohesion and improved security/safety are also discussed.

Default diversion rates

In order to value the benefits associated with a TBhC project, there needs to be an estimation of the likely impact that the project will have on travel behaviour including changes in mode share. Default diversion rate profiles (mode share changes) were determined from the reported results achieved by TBhC projects in New Zealand, Australia, and the United Kingdom in recent years.

The default diversion rate profiles give the changes in mode share from car-as-driver to other modes including car-as-passenger, public transport, cycling, and walking, expressed as absolute percentage point decreases or increases. These can be used without knowledge of existing mode shares, which simplifies the economic evaluation. This is possible because detailed statistical analysis of the TBhC case study data indicated that the percentage point change in mode share appeared to be independent of the initial mode share.

The recommended diversion rates take account of the fact that many case studies only report the observed behaviour changes for a certain subset of the population, such as those who participated in the programme. Where necessary appropriate adjustments were made for the assumed response of non-participants and the effect of on-going word-of-mouth dissemination of TBhC information.

A number of default diversion rate profiles are estimated for different types of work travel plans, school travel plans and household/ community based projects. It was initially intended to separate each of the TBhC project types into a larger number of sub-groups. Characteristics of a workplace, school or household area would determine the applicable diversion rate from within a sub-group (such characteristics were the size of the company, accessibility to services and amenities, quality of public transport services, etc). After considerable evaluation this method was discarded, as there was insufficient data to determine statistically significant differences between the diversion rates that were achieved with different combinations of characteristics.

Proposed evaluation procedure

An evaluation procedure for determining the economic efficiency of TBhC proposals has been developed that is considered to meet Transfund’s requirements of being appropriate for the scale of individual TBhC initiatives as well as promoting consistency in the evaluation of these projects. A single evaluation procedure for TBhC projects is proposed that is similar to the existing Transfund simplified procedure for walking and cycle projects. However instead of the benefit value being embedded in the procedure, a set of tables is provided giving composite benefit values for the common TBhC types and situations expected to occur in New Zealand.

A MS Excel spreadsheet model was developed to calculate the composite benefit values for the different TBhC project types and situations. This model can also be used to evaluate other types of TBhC project if required.

The evaluation procedure uses a ten year evaluation period and determines a benefit cost ratio. Required inputs are the type of TBhC project, city/location, target population number (workforce, school roll, or residents, etc), and information about the comprehensiveness of the proposed project. Benefits are assumed to remain constant over the evaluation period – different levels of ongoing “maintenance expenditure” are required to achieve this for different TBhC project types.

During development of the evaluation procedure a number of sensitivity tests were carried out to determine the sensitivity of the composite benefit values to alternative values for some benefit parameters. The sensitivity tests show that the approach and benefit values are reasonably robust (and consistent with other Transfund procedures such as passenger transport funding policy) for workplace travel plans and household based projects but that there is somewhat less certainty with school travel plans due to factors such as school mode change being dependent on both student and parent perceptions, and variability in the proportions of change to different modes.

Guidelines are provided on composite project evaluation requirements. Composite evaluations are required if TBhC projects contain non-TBhC components and are estimated to cost more than $200,000. They involve evaluating the non-TBhC components with the appropriate existing Transfund evaluation procedures and then aggregating the results, taking care to avoid double counting of benefits.

Monitoring and review

It is expected that the evaluation procedures proposed in this document would be reviewed and refined after one year to improve their ease of use and applicability, if necessary, following initial experience with their application. It is proposed that a more comprehensive review including default diversion rates should be undertaken in approximately three years when more evidence on effectiveness of different projects has become available from monitoring of initial TBhC projects. The Guidance Handbook prepared in conjunction with this technical report contains advice on the design and timing of appropriate monitoring programmes to measure the effects of TBhC projects.

Conclusions

An economic evaluation procedure has been developed that achieves an appropriate balance between Transfund and EECA’s requirements for procedures that are consistent with the Transfund Allocation Process and are evidence based, yet are simple to understand and apply and involve an analysis effort that is in proportion to the scale of the proposed projects. The evaluation procedure should be regarded as interim and likely to benefit from ongoing review and refinement as further knowledge is gained from evaluation experience and monitoring of actual TBhC projects.

A Guidance Handbook has been prepared that provides advice on types of projects, components, and preconditions that have been found to be successful or unsuccessful in New Zealand and/or overseas, preparation of evaluations and funding applications, design of before and after monitoring programmes, and a listing of relevant sources of further guidance for developing successful TBhC proposals.

Recommendations

It is recommended that:

• the interim evaluation procedure be reviewed after one year, focussing on its practicality, applicability, and success (in generating TBhC proposals);
• a further more comprehensive review be undertaken in approximately three years time to refine the default diversion rates and if possible provide further identification of contributing factors to TBhC project effectiveness;
• further research be undertaken to refine some of the benefit values including:
  • Behaviour changer benefits
  • Congestion (particularly Christchurch)
  • Accidents
  • Health benefits
• further research also be undertaken to confirm/refine the estimates of the extent to which some benefit categories are perceived by behaviour changers or require resource cost corrections.

 

< previous | next >

Page created: 28 October 2008