Travel behaviour change guidance handbook

Print version: Travel behaviour change guidance handbook (PDF, 1.29 MB, 87 pages)

5.   “Immediate future” monitoring requirements

• 5.1   Introduction
• 5.2   Key outcomes to be monitored
• 5.3   Monitoring workplace initiatives
  • 5.3.1   Monitoring focus
  • 5.3.2   Methodology
• 5.4   Monitoring school travel initiatives
  • 5.4.1   Monitoring focus
  • 5.4.2   Methodology
  • 5.4.3   Conducting the surveys
• 5.5   Monitoring community- and household-based TBhC initiatives
  • 5.5.1   Monitoring focus
  • 5.5.2   Methodology
  • 5.5.3   Maximising survey response
  • 5.5.4   External monitoring sources

5.1   Introduction

Because of the relative newness of TBhC programmes to New Zealand and the general lack of good quality evidence to prove their effectiveness, Land Transport NZ has indicated the need for a substantial monitoring programme for this activity class. All TBhC project proposals submitted to Land Transport NZ shall include a detailed outline and costing of their associated monitoring programme. Note that the cost of the monitoring programme is not included in the calculation of the benefit cost ratio for the project, except in the case of workplace and school travel plans and for particular types of household-based initiatives, where the initial survey is an integral part of the project and should be costed as such.

The monitoring requirements stipulated here should be seen as “interim”. Once sufficient information is gathered to confirm the values used in the Interim TBhC Procedures and to justify Land Transport NZ’s ongoing funding commitment by providing evidence of the contribution of TBhC programmes to Land Transport NZ’s objectives, it is intended that these requirements will be reviewed and replaced with the monitoring of “key performance indicators”, much in the same way as passenger transport is monitored.

The purpose of this section is to provide guidance on monitoring methodologies that will assist in the collection of useful data with least effort and cost. Monitoring for workplace and school travel plans is much more straightforward than that for community- or household-based initiatives – there is a clearly identifiable target population and generally the focus is on specific trip types and purposes (e.g. commuting to and from work or the journey to and from school). In the case of community- and household-based initiatives, a wide variety of trip types and purposes by all members of a household are canvassed over varying times of the day and week. This creates a significant challenge to measure such potentially complex changes to travel behaviour with precision.

This section focuses on the methodology for the monitoring programme, as well as identifying the monitoring focus for data for collection. No specific advice is given on the questionnaire design or analysis of the data collected. A comprehensive monitoring guide for travel behaviour change initiatives (including key indicators, sample questions, survey tools / methodologies, statistical analysis and so on) was developed as part of the European Union’s TAPESTRY project (Revised Deliverable 3 - Campaign Assessment Guidance, September 2003).

It is important to understand that questionnaires and survey design must be tailored to the objectives to be met and the information required. In many cases, this will require assistance from specialists in the market research field.

5.2   Key outcomes to be monitored

Based on a review of Land Transport NZ’s objectives as outlined in its Allocation Process and EECA’s objectives in developing the Interim TBhC Procedures, it was determined that the key outcome for measurement in monitoring TBhC initiatives is the overall change in vehicle kilometres travelled (VKT). Based on this indicator, it is possible to derive many other benefits including decongestion, vehicle operating costs (including reductions in fuel use), environmental (such as local air quality, CO₂ emissions and water quality impacts), accident risk reduction, and reduced health costs by using various ‘factors’ that have been established through other rigorous research processes.

In the past, the measure focused on has been the overall change in mode share (usually measured using “trips”) and, if estimated at all, the impact on VKT has been extrapolated using existing transport models or possibly national household travel survey data. However, as transport models are based on average trip lengths and TBhC initiatives are believed to affect shorter-than-average trips, this may generate inaccurate results. Furthermore, most household-based initiatives provide advice on “intelligent” or “eco-friendly” car use (such as trip chaining, using telecommunication to replace trips), which may not result in mode shift, but will affect overall VKT. This would not be captured if only mode shift was monitored. Hence, the monitoring programmes described here focus on change in VKT.

There are various methods available to collect VKT or person-kilometres travelled data from participants in a given TBhC initiative. In the case of school and workplace travel plans, where the trip under observation is well defined (usually home to work or school, sometimes combined with other destinations / stops), it is more reliable to obtain geo-code-able addresses than to rely on respondent estimates of the distances involved. As discussed in Section 5.5.2, for community- and household-based initiatives, where there is a wide range and number of possible trips involved, other methods may be preferable.

In addition to VKT, it is suggested that other data be collected regarding the change in physical activity levels of the target population as this has an effect on the overall health and well being of the community.

If the key outcome sought is a reduction in VKT and therefore a reduction in congestion then this is potentially more applicable for affluent areas as it is parents in these areas taking their children to school that is contributing to the congestion, particularly around schools. However, if school travel plans are implemented in more deprived areas, where more children tend to walk and cycle, safety issues may be a greater concern. In these situations it is reasonable to have safety issues as the main way of prioritising schools for school travel plans and monitoring could include collection of data to identify and quantify the improvement in safety.

Other impacts may be monitored at the discretion of the sponsoring agency and/or the target population, but may not be central to Land Transport NZ’s requirements.

5.3   Monitoring workplace initiatives

As discussed above, it is anticipated that the most common workplace initiative to be undertaken in New Zealand will be the development and implementation of workplace travel plans, as these are intended to be “optimised” packages of measures, selected with the specific needs of the organisation in mind. Hence, the discussion that follows is focused on the monitoring of workplace travel plans. However, with some modifications, this monitoring programme could likely be adapted for use with any other workplace-related TBhC initiative.

It is highly likely that by 2005/06, there will be standardised questionnaire templates developed to facilitate monitoring of workplace initiatives.

5.3.1   Monitoring focus

For any workplace travel initiative, the focus of monitoring efforts should be on collecting evidence of both the mode share (for all modes) and the change in vehicle kilometres travelled as a “single occupancy vehicle” driver by all employees within the organisation. The exception to this, in terms of monitoring, will be the case where an organisation is re-locating and the travel plan is part of the re-location process. In this case, only mode share would be monitored, as vehicle kilometres travelled will be changing due to the relocation.

From the mode share and VKT information collected, a range of impacts can be estimated, as shown in Table 5.1.

Table 5.1: Key outcomes as indicators for other workplace travel plan outcomes

Shifts in mode share (away from car) can indicate:	• Improved health / increased level of productivity, if “new” mode is walking or cycling (may be measured separately)
	• Perception of improved safety in the environment surrounding the workplace
	• Improved PT service viability, if “new” mode is PT
	• Creating in-grained “life long” habits
Reduction in vehicle kilometres travelled by single occupancy vehicle driver can indicate	• Congestion reduction
	• Improved safety in terms of accident reduction
	• Improved local environment / reduced emissions
	• Improved health for remaining transport network users
	• Reduced CO2 emissions
	• Reduced fuel consumption
	• Travel time savings

In addition to the mode shift and reduction of car driver vehicle kilometres, the changes in physical activity levels of staff may be monitored.

It is recognised that organisations and local authorities may have other objectives within their monitoring programmes, such as monitoring the progress in implementing the actions contained within the travel plan or evaluating staff awareness of the plan or its development process. These may be added into the questionnaire(s) as desired.

5.3.2   Methodology

The workplace travel plan monitoring programme establishes the “before” travel patterns of the entire organisation’s staff (including managers) as well as gauging interest in potential travel alternatives. Following the implementation of at least some elements in the travel plan, the “after” survey evaluates the change in these patterns.

Survey timing

The time of year in which the survey is conducted can have a significant influence on the response rate as well as on the utility and relevance of the data collected.

It is important, therefore, to conduct surveys at the “most representative” times of year in terms of both commuting patterns and work attendance by staff. Factors that should be taken into account, include:

• The timing of holidays – including school and tertiary institution holidays
• Seasonal daylight and weather variations
• Any other factors specific to the organisation, such as site “shutdowns” for annual maintenance, etc.

Generally speaking, autumn and spring tend to be the most suitable times for surveying. Ideally, the initial / before and after surveys should take place 12 months apart, to minimise the impact of seasonal variability.

The initial (baseline) survey

The initial survey should target as many staff as possible. Where there are less than 1000 in the organisation, the questionnaire can probably be distributed to all staff members. In organisations of greater than 1000 employees, a representative sample could be used. Response rates over 50% are considered to represent a good level of penetration.

All of the surveys should be conducted in the same one-week period (to eliminate as much external factor bias as possible). In addition, someone should be responsible for recording any special factors that might influence travel pattern results, such as:

• The weather during the morning and afternoon travel periods in the survey week
• Bus service on strike
• Road works near the workplace
• Special events, such as conferences, cocktail / drinks evenings, divisional retreats, etc. That might affect travel decisions

The “Follow-on” survey

After the workplace travel plan has been launched and at least some measures have been implemented, all of the staff or a representative sample (depending on the total number of employees) should be surveyed again.

Ideally, this survey will occur one year later and in the same week of the initial survey. However, if this is not possible or practical, then attempts should be made to match for a week similar to that of the first survey. Again, someone or a group should be responsible for recording the weather for the week as well as any other external factors that might influence travel patterns during the week of the surveys.

5.4   Monitoring school travel initiatives

Auckland Regional Council and EECA, in consultation with Land Transport NZ, commissioned Pinnacle Research in mid-2004 to devise a methodology, including questionnaire templates, to enable consistent monitoring of the impact of school travel plans in the Auckland region. The methodology and questionnaire templates were designed to have general applicability to all of New Zealand. With some modifications, the methodology and “core” questions from the questionnaires could be adapted for use with any other school-related TBhC initiative.

The methodology and questionnaires are currently being piloted in Auckland region and Christchurch (city). Auckland Regional Council is developing a prototype database to facilitate the analysis of the data collected and generate the appropriate reports for schools, councils and Land Transport NZ. Once piloting is complete and the results assessed, the methodology and questionnaire templates will be finalised and available on the EECA website.

5.4.1   Monitoring focus

For any school travel initiatives, the focus of monitoring efforts should be on collecting evidence of both the mode share of the students (away from car) and the change in vehicle kilometres travelled, by either parents (who are driving primary or secondary school students) or secondary school students who drive themselves. From this information a range of impacts can be estimated, as shown in Table 5.2.

Shifts in mode share (away from car driver or car passenger) in students can indicate:	• Improved road safety skills (if accompanied by such programmes in the school)
	• Parents’ perception of improved safety around the school (but only for those whose children have changed mode, may be better canvassing entire parent community)
	• Improved community relationships (if involved in activities such as walking school buses, car pooling or cycle trains) – again, only for those who have changed mode, may be better canvassing whole community
	• Where walking or cycling is involved, improving learning / concentration (more children getting early morning exercise)
	• Creating in-grained “life long” habits
	• Improved health where increased physical activity is involved (but may be better to measure separately)
	• To a limited extent, parents’ awareness of school travel plan programme (or at least some elements of it)
Shift in VKT by parents can indicate:	• Congestion reduction (if parents are not driving their children to school)
	• Improved safety in terms of accident reduction
	• Improved local environment / reduced emissions
	• Reduced CO2 emissions
	• Reduced fuel consumption
	• Travel time savings (this is a potential benefit related to the outcome)

In addition to the mode shift of children and shift in travel patterns of parents, the Auckland Regional Council / EECA questionnaire templates collect information on the changes in:

• Perceptions of the safety of the environment for children’s travel
• Community cohesion
• Children’s physical activity levels.

It is recognised that schools and councils may have other objectives within their monitoring programmes, such as monitoring the progress in implementing the actions contained within the travel plan or evaluating community awareness of the plan or its development process. These may be added into the questionnaire templates as desired. However, it should be noted that, for the sake of consistency and reliability of the data collected, the questions aiming to collect the data discussed above are regarded as “unchangeable” between the before and after surveys.

5.4.2   Methodology

The school travel plan monitoring programme establishes the “before” travel patterns of children and their parents as well as gauging interest in potential travel alternatives. Following the implementation of (at least some) elements in the travel plan, the “after” survey evaluates any change in these patterns. As parent and secondary school student surveys are “self-completion” surveys, it is also important that they be kept as simple as possible, to assist response rates.

Monitoring programme participants

In the case of school travel planning, all students are surveyed as to their current travel patterns to and from school for a one-week period. Staff members are also surveyed with respect to their travel patterns and interest in potential travel alternatives.

For primary schools, parents are surveyed in order to establish their travel patterns as the people who drive their children to school. Parents of primary school children are also asked what travel alternatives they would consider for their children, along with other questions to facilitate the development of the school travel plan.

The initial (baseline) survey

In a primary school, it is essential that children, parents, and staff are surveyed early in the school travel plan development process, as this provides important base information for developing the plan. In a secondary school setting, only students and staff will need to be surveyed. All of the surveys should be conducted in the same one-week period (to eliminate as much external factor bias as possible). In addition, someone (e.g. a senior or junior class, the parent’s committee, etc) should be responsible for recording any special factors that might influence travel results, such as:

• The weather during the morning and afternoon travel periods in the survey week
• Bus service on strike
• Road works near school
• School camp or outings.

The “Follow-on” survey

After the school travel plan has been launched and at least some measures have been implemented, all three groups (parents, children and staff) in a primary school setting should be surveyed again. In secondary schools, the students and staff will need to be re-surveyed.

Ideally, this survey will occur one year later and in the same week and term of the initial survey. However, if this is not possible or practical, then attempts should be made to match for the same week of term. Again, someone or a group should be responsible for recording the weather for the week as well as any other external factors that might influence students, staff or parent’s travel patterns during the week of the surveys.

5.4.3   Conducting the surveys

It is important that the whole student population is included in the initial and follow-on surveys, as different age groups travel to school by different means. This will also provide good, statistically reliable information to supplement the parental surveys if their response rate is lower than expected. In the case of the secondary students’ survey, it is the sole means of gaining reliable results regarding the impact of the school travel plan.

Student surveys are almost always completed in class time at school, while parent surveys are generally distributed with school newsletters, completed at home and then returned to the school. Staff surveys could be completed during a staff meeting or on an individual basis and returned to a designated place.

Table 5.3 illustrates a potential timeline for accomplishing the initial baseline survey in a primary school setting. Essentially the same process would be repeated for the follow-on or “after” survey.

Table 5.3: Example of timeline for surveying a primary school community

	Assuming the school newsletter goes out on Wednesday or Thursday . . .
Week 1 & 2	• Announce (week 1) and, if necessary, remind (week 2) that survey will be happening as part of school travel plans development; explain survey purpose.
	• At regular staff meeting, invite Council travel plan coordinator to explain about travel plan, how the various surveys work, expectations about completion of children’s surveys, and so on.
Week 3	• Distribute parent / family survey with the school newsletter. Ensure that the covering letter explains purpose of survey, what information will be used for, and its due date (Friday the following week).
	• An alternative could be to distribute with homework on Monday and have it due on Friday – but this is up to travel planner & school to determine!
Week 4	• Survey children in all classes.
	• Survey the staff.
	• Ensure that someone / a class is responsible for recording “external factors”.
	• “Spot prizes” for children returning their family’s survey.
	• Reminder in newsletter that completed survey is due in on Friday; spare copies available in the office.
	• Prize awarded for class with highest response rates.
Week 5	• “Stragglers” received.
	• Thank parents for their cooperation and announce class prize winners in the newsletter

The draft School Travel Plan Coordinator’s Guide (EECA, 2004) provides more detail on how the surveys may be conducted for different age groups within the schools. The use of incentives has been suggested to improve overall response rates by parents, particularly for the follow-on survey.

5.5   Monitoring community- and household-based TBhC initiatives

This section provides information on monitoring community- and household-based initiatives such as personalised marketing; travel blending, travel awareness campaigns or personalised journey planning.

Given that there are various methodologies available to monitor community- and household-based initiatives each with their own strengths, weaknesses, and complexities, only basic information can be provided here. Project proponents are advised to seek the advice of credible marketing research firms, particularly those with experience in monitoring these types of initiatives. Documents such as the TAPESTRY (2003) Campaign Assessment Guidance report; Stopher et al. (2004) Evaluating voluntary travel behaviour interventions (ITS-WP-04-17) or other texts will provide useful supplements to the information given here.

5.5.1   Monitoring focus

For any community- or household-based TBhC initiative, the focus of monitoring efforts should be on collecting evidence of both the mode share (for all modes) and the change in vehicle kilometres (VKT) travelled as a “single occupancy vehicle” driver by all household members living in the target population area. From this information a range of impacts can be estimated, as shown in Table 5.4.

Table 5.4: Key outcomes as indicators for other household- or community-based outcomes

Shifts in mode share (away from car driver) can indicate:	• Improved health / increased level of productivity, if “new” mode is walking or cycling (may be measured separately)
	• Perception of improved safety and social cohesion in the community
	• Improved PT service viability, if “new” mode is PT
	• Creating in-grained “life long” habits
	• To a limited extent, community awareness and “uptake” of TBhC initiative (or at least some elements of it)
Reduction in vehicle kilometres travelled by single occupancy vehicle driver can indicate:	• Congestion reduction (if changes occur during peak travel periods)
	• Improved safety in terms of accident reduction
	• Improved local environment / reduced emissions
	• Reduced CO2 emissions
	• Reduced fuel consumption
	• Travel time savings (this is a potential benefit related to the outcome)

Note that this focus on VKT is particularly important for household-based initiatives, as in some cases programmes aim to achieve “smarter” car use, wherein the mode may not change, but the VKT does.

In addition to the mode shift and reduction of car driver vehicle kilometres, the changes in physical activity levels may be monitored.

It is recognised that organisations and local authorities may have other objectives within their monitoring programmes, such as changes in target population attitudes (as opposed to behaviour), evaluating awareness of the TBhC initiative or the level and quality of information received. These may be added into the questionnaire(s) as desired.

5.5.2   Methodology

The community- or household-based TBhC monitoring programme establishes the “before” travel patterns for all members of the respondent household and then evaluates the change in these patterns “after” the implementation of the various components (e.g. education, encouragement / free trials and information) of the TBhC initiative.

Monitoring approaches

The most common approach to date is to survey randomly selected households in the target population area. Surveys typically focus on the use of travel diaries (similar to typical transport study home interview surveys such as the New Zealand Travel Survey). These can provide full details of trips undertaken by a household, and, given sufficient sample size; changes in total travel (trips, person- and vehicle-kilometres travelled) may be estimated for all modes. Usually travel diaries cover between 1 and 7 days, although with anything over 1-2 days there is likely to be a significant loss of response and accuracy. Issues to do with sample size, respondent drop-off, and bias are discussed further below.

Given the need to monitor changes in VKT, odometer-based surveys are an attractive alternative (or complement) to travel diaries. Such surveys could record odometer readings for household vehicles at long intervals (e.g. 3, 6 or 12 months apart) and directly derive VKT and VKT changes by household. Compared with travel diary surveys, odometer surveys are:

• Inexpensive to administer (a few phone calls)
• More accurate in measuring VKT
• Easy for respondents (so response rate should be higher)
• Readily carried out over an extended period of time, thus allowing considerably smaller sample sizes to provide significant results with a given confidence level.

However, using this method on its own will not permit assessment of mode shifts, changes in vehicle occupancy, or changes in trip chaining behaviour. Other problems include: car may be sold; preferred car may change in multi-car household; and/or long trips distort results. Recent (2003) Australian monitoring studies have recorded odometer changes over relatively short periods (one to four weeks), but the approach has not been frequently used to date.

More recently still (2004), there has been experimentation in Australia with the fitting of Global Positioning System (GPS) devices to both people and vehicles to record person-kilometres travelled and vehicle-kilometres travelled respectively. These devices have considerably greater accuracy than travel diaries, which have been found to have significant discrepancies between actual travel and reported travel – as high as 20%. Short trips, the ones most likely affected by TBhC programmes, are the most common type of trip omitted. GPS devices can easily produce information for one week’s travel for a household, including travel times, destinations and trip duration. The ability to collect data for a whole week substantially reduces the sample size requirements for a monitoring programme and may reduce respondent fall-off rates.

Panel survey v independent samples

Regardless of what type of survey is adopted to monitor the project outcomes, there is the need to consider whether a “panel survey” or “independent samples” (also known as cross-sectional surveys) will be used. Panel surveys, wherein the same respondents are used for the before and after surveys, are considerably more statistically efficient than independent samples (where different groups are surveyed in either survey), and hence generally preferred as lower sample sizes can be used for a given degree of confidence. However, panel surveys suffer from progressive drop-off of responses in successive surveys, which is especially important in cases of medium or longer term monitoring and can only be partially overcome through a good survey approach and/or adopting unusually large sample sizes for the before survey (to allow for respondent loss in the after survey).

Survey timing

Travel patterns (especially for environmentally friendly modes) are substantially affected by seasonality: the before and after surveys should be undertaken 12 months apart (in the same month of the year), regardless of the monitoring approach. Hence, typically the initial after survey would be 12 months following the before survey, with program implementation between these two points. If this is not feasible, it is desirable that the after survey be at least nine months following project implementation- to allow for new behaviour patterns to settle down and not record very short-term impacts. If this is the case, extra care will also need to be taken to account for seasonality factors that could be influencing monitoring results. It is probably beneficial to repeat after surveys in order to monitor the stability of changes. Any subsequent after surveys should typically be at 12-month intervals.

It is known that there is considerable variability in travel across the days of the week and by travel mode. For example, an analysis of Auckland New Zealand Travel Survey data (O’Fallon and Sullivan, 2004) found that the number of car driver trips in a household varied by 56% on a weekday while passenger transport trips varied by 196%, assuming that the same households were participating in the before and after surveys (known as a “panel survey”). Other modes varied considerably more: car passenger 270%, walk 185%, and bicycle 1273%. Given this, it is essential that surveys are spread across days of the week and, where the same households are being surveyed before and after, the survey days of the week should be the same in both.

On any given day, travel is affected significantly by weather, special events, or other factors. Some of these variation factors can be avoided (e.g. avoid monitoring just before or after statutory holidays or other local special events). For others it is difficult to do so, it may be necessary to rely on the control group being similarly affected (discussed below).

Monitoring programme participants

The population of all households in the TBhC project area – those that participate in the programme and those that do not (although it is useful to separate these in the analysis) – should be included in the monitoring programme. This allows inclusion of “diffusion” effects, where the project positively affects non-participant’s behaviour, providing a much better basis (than just participating households) for assessing aggregate effects of the project.

It is preferable to involve the whole household, rather than just an individual from the household, in the monitoring programme. TBhC projects generally provide information and incentives on a household basis and, thus the effects are likely to diffuse through the household, meaning that travel behaviour changes could be under-estimated if only one individual from that household was monitored.

Control group

Regardless of which monitoring approach is taken, control groups are essential to allow robust evaluation. Whilst some adjustments for “external factors” may be possible in the absence of a control group (see below) these are most unlikely to be sufficient on their own.

The control group area should be as comparable as possible to the programme area (but unaffected by the programme) in terms of similar socio-demographics, car ownership and use levels, passenger transport levels of service and use, topographical features, and distance from central business district.

It is important that other local changes do not occur in either the project area or control area that may impact on travel behaviour of residents, such as increases in passenger transport services in anticipation of increased demand as a result of the TBhC initiative or other transport system changes (new roads, changes in passenger transport services, other TBhC initiatives introduced). Often it will not be possible or desirable to prevent or delay these – the important thing is to record when they occur and make appropriate effort to distinguish the effects from these changes from those of the TBhC project.

Sample size

The required sample size to assess a defined degree of change, with a given level of statistical confidence, will essentially depend on:

• The underlying variability of the travel behaviour data (e.g. on a day-to-day basis) – which will depend on whether a panel survey or independent sample approach is adopted
• The length of period over which data is collected (e.g. 3-day travel diary, 6 month odometer differences)
• The true extent of any behavioural changes (eg 10% reduction in average VKT per household)
• The accuracy with which this change is estimated and the degree of confidence in the resulting estimate (eg 10%+3%, with 95% confidence)
• The extent of any drop-off in responses from the before survey to the after survey (assuming a panel approach)
• The mode of interest because of different levels of underlying variability (environmentally friendly modes tend to show greater variability than car modes).

Sample sizes are essentially (almost) independent of the size of the population concerned (for large populations). It is recommended that further advice be sought with respect to establishing the appropriate sample size for a particular project.

Systematic survey bias issues

For a random monitoring survey of all households, the typical before and after (successful) response rates are in the order of 50%. Given this, the dangers of non-response (or self-selection) bias are considerable; that is, the change in the behaviour of the responding sample may differ substantially from that of the non-responding households, in a way that is unknown. Hence, there is no reliable basis to extrapolate the sample results to the whole population’s behaviour as, for example, less mobile households have a greater propensity to respond than more mobile households, but more mobile households may make bigger behavioural changes.

Some corrections can and should be made for this problem by comparing statistics (e.g. age, income, car ownership, etc.) for the respondent sample with those for the area population as a whole; and differentially expanding from the sample to the total population. However, it should be recognised that these may not totally correct the entire problem.

5.5.3   Maximising survey response

The main problem relates to after surveys, as before survey data for households that do not complete the after survey cannot be used in panel surveys. It is therefore critical to minimise any drop-off from before to after responses. Suggestions for achieving this include:

• Reference may be made to standard market research texts on how to maximise survey response.
• Surveys need to be designed to be as simple as possible for respondents to complete.
• Personalised contact is likely to help (face-to-face contact is probably ideal).
• Incentives – there is debate about the (cost) effectiveness of incentives in encouraging response, but it is reasonable to expect appropriate incentives (particularly for the after survey) would help.

5.5.4   External monitoring sources

A variety of ‘external’ monitoring sources might be used for before versus after evaluation, to supplement or potentially replace project-specific household surveys. These could use data that is collected in any event, usually for other purposes, or surveys undertaken for this particular purpose.

Examples are:

• PT patronage data (from electronic ticketing or other sources)
• Road traffic counts
• Cycle and pedestrian counts (not commonly undertaken).

Such surveys will establish changes in trips at a point or over a PT route. They will not establish changes in trips by a given set of households.

If a programme is undertaken on a metropolitan-wide scale, changes in trips by metropolitan households in total may be reasonably inferred. If a programme is undertaken over a limited area (as is typically the case), then the impacts of the programme on traffic counts, etc will rapidly diminish further from the area. Even within the area, the impacts are likely to underestimate the changes in travel by area residents, because of the through traffic component. However, in particular circumstances, reasonable inference about the effects on travel by area residents might be made (eg the changes in PT trips boarding in South Perth, as a measure of the change in PT trips generated by South Perth residents).

For larger scale programmes, we consider that such external surveys can provide useful evidence. However, this should normally be regarded as supplementary to, rather than in place of, direct household surveys.

For small-scale programmes, any changes in travel observed in external surveys are likely to be very small. Therefore, external surveys will be of no practical use. However, in such cases, external surveys may be useful as supplementary to control groups, eg: PT patronage changes over a metropolitan area could form part of the controls for comparison with PT changes in the programme group established from surveys. Again, we suggest they would not form a complete substitute for control groups but a supplement to these.

 

< previous | next >

Page created: 21 October 2008