Stakeholder involvement is very important within NISTO. The input stakeholders can provide is absolutely necessary to build the best possible evaluation tool for smart mobility projects. The first step in securing stakeholder engagement is to develop a successful selction process. In all participating regions more than 1000 stakeholders were identified. The first four NISTO regional stakeholder workshops were held in s'Hertogenbosch (The Netherlands), Kassel (Germany), Boulogne sur mer (France) and Leuven (Belgium) between 31/10/2013 and 30/01/2014, a fifth stakeholder workshop took place in April 2014 in Cardiff (UK). All five of the workshops aimed to involve the local and regional stakeholders in the NISTO project to get their feedback on the development of the toolkit. A total of 124 persons participated in the workshops.
In October 2013 an online survey about the evaluation methods was conducted. To analyse the results of it in detail, NISTO-partners organised the stakeholder workshops where they invited multiple stakeholders to collect more information about the commonly used evaluation tools, their experiences with these tools and their expectations towards a new tool. The main goal of each stakeholder workshop was to collect input on the development of the demonstration projects within NISTO and the toolkit. The results of the surveys were presented and a group discussion on different aspects of developing an evaluation tool followed. All stakeholder workshops included a group discussion on evaluation criteria or an exercise of an existing method to use as a basis for discussion.
The survey as well as the stakeholderworkshops proved that evaluating mobility projects is not common. In those cases where evaluation methods are used, the ‘oldest’ ones are most popular. Multi-Criteria Analysis (MCA) and Cost-Benefit Analysis (CBA) are used significantly more than alternatives. Stakeholders expressed the need for more information about evaluation tools. They are not experienced enough with evaluation methods to choose the right method for the right project. In all stakeholder workshops, there was the demand for a flexible evaluation tool. There are numerous types of mobility projects, from very straight forward to complex projects; from local to national, etc. An evaluation tool should be adjustable to the complexity of the project. This is mostly not the case for the existing methods.
Some things are not clear after consulting the survey and stakeholder workshop participants. For example, there was not a distinct conclusion on whether it is necessary to define stakeholder groups within the method or to make a general approach. This is a discussion that needs to be taken into account when developing the toolkit.
The various participants were unable to define the criteria per stakeholder group. Should this be defined before developing the toolkit, or is this a part of the toolkit? They did not achieve consensus on who should define these criteria.
The survey proposed a number of key evaluation criteria. The stakeholder workshops confirmed the top criteria, but also suggested some new ones. The rich discussions lead to a wide variety of suggestions to organise the criteria.
Crucially all four stakeholder workshops concluded that the future evaluation method needs to be adjustable to the complexity of the project it is required to evaluate. Some sort of categorisation should be applied to determine the details of the evaluation tool. The project partnership will organise two further stakeholder workshops in each of the regions in order to continuously involve the stakeholders of the development of the toolkit.