Return to Geddes Center for Planning Research Home
GCPR Home

DUNDEE PARTNERSHIP COMMUNITY REGENERATION STRATEGY

SOCIAL INCLUSION PARTNERSHIP

MONITORING AND EVALUATION

 

SIP Monitoring and Evaluation Manual

SIP Monitoring and Evaluation Unit
Geddes Centre for Planning Research
School of Town and Regional Planning
University of Dundee

June 1997


One of the Monitoring and Evaluation Series

1. Introduction

Priority Partnership Areas (PPAs) represent the most recent urban regeneration policy initiative in Scotland. The designation of PPAs encourage the formation of city-wide partnerships, an associated city-wide urban regeneration strategy and more specific proposals for measures to address the problems of social and economic disadvantage in defined neighbourhoods. The PPA approach has developed from the Scottish Urban Partnerships, which involved the principles of co-operation, integration, and partnership operating within a strategic regeneration context and the involvement of local communities.

Monitoring and evaluation is an integral component of the PPA initiative in Dundee. It is intended that the monitoring and evaluation of the PPA activities and impacts on the local designated communities contribute to the overall initiative through a continuous learning process. Attention will be paid to the progress made on individual projects, within the community areas themselves and within the broader economic and social context of the city and its region. It is intended also that the PPA be monitored to assess its contribution against the evolving policy for community regeneration and economic development in Dundee. The monitoring processes in particular will seek to provide material which will be useful as a source of advice, guidance and ideas for the PPA projects as they continue to evolve in practice. The monitoring and evaluation will seek to capture the changes taking place in the neighbourhoods and communities and in those actively involved in project and policy implementation.

In short, therefore, the purpose of the monitoring is to provide an ongoing review and assessment of change in a number of key indicators; issues and views of those involved in the PPA exercises. It is intended to identify the nature of that change and to provide evidence of that change to the communities and to the Dundee Partnership. This will enable the PPA projects to evolve in a way appropriate to the needs of the communities and the policy context of the initiative. The purpose of the evaluation is to provide a mid term and end of project assessment of the impacts of the PPA initiatives in terms of the quantitative and qualitative changes and the processes involved in community regeneration.

2. Objectives for monitoring and evaluation in the Dundee PPA

The Dundee Partnership has established a strategic vision for community regeneration in the city: "the creation of stable, sustainable and empowered communities throughout Dundee, in which people wish to live and can prosper". This strategic vision for community regeneration is translated into practical effect through four themes: stability, sustainability, empowerment and prosperity. Together, these themes will be realised through the effective and efficient use of infrastructure and resources within Dundee. Statements of strategic intent for the provision or management of measures which directly address the physical, social and economic conditions of Dundee's deprived communities as set out in the designated PPAs provide more detail of the objectives for community regeneration.

The four themes embrace a number of measures against which key indicators can then be placed. Appropriate strategic indicators will be identified to enable monitoring and evaluation of the PPA initiative. These are:

Stability comprise measures

which improve community support

which increase housing tenure mix

which reduce crime and fear of crime

which stabilise population levels and reduce high turnover

which improve physical, economic and social structures

Sustainability comprise measures

which increase sense of belonging

which create work

which enhance educational attainment

which promote community integration

which have positive environmental benefit

which facilitate transportation and safety

which promote youth initiatives

Prosperity comprise measures

which enhance employment prospects

which increase availability of affordable childcare

which enhance the contribution of the private sector and support their involvement

which alleviate poverty

which promote good health

Empowerment comprise measures

which build community capacity

which increase the involvement of citizens in decisions

which facilitate proofing of local authority and agency partners prior to implementation

which deliver training locally

which provide funding for community controlled initiatives

which support volunteers

which build self-confidence and self-esteem.

Monitoring and evaluation is an integral component of the PPA proposal. It is intended that the monitoring and evaluation of the PPA activities and impacts on the local communities contribute to the overall initiative through a continuous learning process of the progress being made on individual projects, within the community areas themselves and within the broader economic and social context of the city and its region. It is intended also that the PPA be monitored to assess its contribution against the evolving policy for community regeneration and economic development in Dundee.

The monitoring processes in particular will seek to provide material which will be useful as a source of advice, guidance and ideas for the PPA projects as they continue to evolve in practice. The monitoring and evaluation will seek to capture the changes taking place in the neighbourhoods and communities and in those actively involved in project and policy implementation.

The purpose of the monitoring is to provide an ongoing review and assessment of change in a number of key indicators, issues and views of those involved in the PPA exercises. It is intended to identify the nature of that change and to provide evidence of that change to the communities and to the Dundee Partnership so as to enable the PPA projects evolve in a way appropriate ton the needs of the communities. The purpose of the evaluation is to provide a mid term and end of project assessment of the impacts of the PPA initiatives in terms of the quantitative and qualitative changes and the processes involved in community regeneration. The monitoring and evaluation arrangements will provide a foundation on which management decisions can be taken in the light of best practical information. Significantly, the arrangements for monitoring and evaluation are being established at the outset of the PPA in order to maximise the benefits from this learning process.

3. Monitoring Indicators

The monitoring and evaluation arrangements are based on a hierarchy of assessment which are complementary in providing a means of identifying change in the PPAs at different scales within the locality, city and region. The levels of assessment comprise different levels of data collection and aggregation but care is taken to ensure that the appropriate indicators are related to the objectives of the Dundee PPA at each level. The hierarchy may be set out as follows:

3.1. Macro level monitoring and evaluation (Regional/strategic)

The macro level of monitoring and evaluation will involve an assessment of change in the regional and strategic levels of the Scottish national economy and the Tay regional economy (defined as the Greater Dundee Housing Market Area) in terms of key indicators which relate to the four themes of the Dundee PPA. The emphasis will be on quantitative data drawn from secondary sources.

3. 2. Meso level monitoring and evaluation (Urban/strategic)

The meso level of monitoring and evaluation will involve an assessment of change in Dundee and the PPA communities in the urban and strategic context to the PPAs. The emphasis here will be primarily quantitative in nature, drawn from secondary sources.

3. 3. Micro level monitoring and evaluation (Community)

The micro level of monitoring and evaluation will involve an assessment of change in the PPA areas. The emphasis here will be primarily qualitative in nature (but will include some quantitative evidence). The purpose of the micro level of monitoring and evaluation is to address the related objectives of outputs and outcomes. Outputs are defined as the progress made in the PPA areas in terms of evidence of community involvement and participation in the projects and initiatives associated with the neighbourhood. Outcome is defined as evidence of increasing or changing community involvement in assuming responsibility for creating and maintaining sustainable community regeneration in the PPA areas.

3.4. Project level monitoring and evaluation (Project)

This involves a quantitative assessment of the inputs and outputs associated with the individual projects in the PPA areas. The principal responsibility for this will rest with Dundee City Council but the results will be integrated with the monitoring and evaluation of the PPAs at the macro, meso and micro levels of analysis.

Each level of monitoring will comprise four inter-related processes of data collection and investigation. Baseline information will be collected at the outset of the PPA programme, the purpose of which is to establish the conditions in the wider regional context at the point at which the PPA commences. It will involve quantitative data from secondary sources, which will be organised according to the PPA themes. Basecase information will be collected, the purpose of which is to assemble an appreciation of the likely way in which the macro indicators would have changed in the absence of the PPA measures. This will be based on projections of the quantitative indicators based on certain assumptions about economic performance in the Scottish and regional areas together with informed judgements from appropriate personnel involved in the PPA exercise. The purpose of the basecase component is to provide an input to estimating the counterfactual of the PPA programme. Monitoring of changes by PPA theme and Evaluation at the mid term and end term stages of the PPA programme.

The four themes of stability, sustainability, empowerment and prosperity will not necessarily apply at the four levels of the monitoring and evaluation hierarchy. Careful consideration of the individual themes and their associated measures suggest that at the macro level the themes of stability and prosperity are more appropriate indicators of contextual change to the PPA communities. As the monitoring and evaluation focuses in more detail on the immediate urban context of the PPAs and the change associated with the designation and funding in the PPAs then the remaining themes of sustainability and empowerment become more appropriate. This can be illustrated in Table 1.

Table 1

Monitoring and Evaluation Themes

Level of the hierarchy

Stability

Prosperity

Sustainability

Empowerment

Macro

*

*

   

Meso

*

*

*

 

Micro

*

*

*

*

Within the individual themes and their associated measures it is important to note that not all the measures will be deployed at each level of the hierarchy. For example, within the theme of stability not all the measures being deployed (including those for community support, increasing housing tenure mix, reducing crime and fear of crime, stabilising population levels and improving physical, economic and social structures) will be appropriate within the macro (or meso) level of monitoring. This is explained in greater detail in separate documentation that sets out the appropriate quantitative indicators to be adopted at the different levels in the monitoring and evaluation hierarchy.

4. Definition, source and frequency of data

The details as to the definitions, sources and frequency of data to be deployed with the monitoring and evaluation framework for the Dundee PPA are given in the Working Paper 05, ‘Priority Partnership Area Indicators and Typology’.

5. Evaluation Plan

The purpose of the evaluation is to provide a mid term and end of project assessment of the impacts of the PPA initiatives in terms of the quantitative and qualitative changes and the processes involved in community regeneration. At this stage, the intention is to build the evaluation into the learning processes of the PPA. Attention thus far has been devoted to devising the appropriate monitoring arrangements on which the evaluation will be based. The inclusion of baseline and basecase data will form an important part of the overall evaluation at the mid term and end term of the PPA.

6. Current Programmes - Baseline Studies

As has been set out above, the monitoring and evaluation arrangements include baseline studies of the four PPA areas in Dundee. Data will be collected to provide an insight into the macro, meso and micro conditions at the outset of the PPA initiative together with information about the projects in place.

7. Administrative arrangements for monitoring and evaluation

Monitoring and evaluation of the Dundee PPA is central to the overall management of the PPA programme. Responsibility for monitoring and evaluation will rest with the PPA Monitoring and Evaluation Group, overseeing its implementation, the Monitoring and Evaluation Research Group, which will be guided by a Monitoring and Evaluation Ethics Committee and which will report to the formal management structures of the Dundee Partnership.

7.1 PPA Monitoring and Evaluation Group

The PPA Monitoring and Evaluation Group has been established to manage and oversee the arrangements for implementing the monitoring and evaluation framework. Its remit includes the production of, and dissemination of monitoring and evaluation material up through the formal management structures of the Dundee Partnership and down to the communities and groups operating at field level. The group comprises key players from both the Council and the University and is central to administrating elements of the framework, bringing together the community, key workers from the different partnership agencies and researchers in the monitoring and evaluation process.

7.2 Monitoring and Evaluation Research Group

The Research Group is based at the Tay Region Research Observatory (TRRO) in the School of Town and Regional Planning, University of Dundee. It is directed by Professor MG Lloyd and Mr WM Edgar. The Group comprises a number of academic researchers based at the Centre for Planning Research, including Ms B Illsley, Ms G Finlayson and Mr J McCarthy. A Research Fellow (Mr K Fernie) has been seconded from Dundee City Council to assume responsibility for the day to day management of the monitoring and evaluation processes. The Monitoring and Evaluation Research Group will be responsible for establishing and maintaining a Management Information System for the monitoring and evaluation research hierarchy. This will comprise the quantitative and qualitative data associated with the different processes of baseline, basecase, monitoring and evaluation of the PPA. The Group will be responsible for the preparation of the monitoring and evaluation reports together with academic papers dealing with policy, concept and methodology of the PPA.

7.3 Monitoring and Evaluation Ethics Committee

A Monitoring and Ethics Committee has been established to ensure that the conduct of the research associated with the PPA communities is appropriate and conforms to ethical research guidelines; to advise on the technical aspects of the monitoring and evaluation processes; and, to provide a forum for discussing the progress of the PPA projects and the interpretation of the monitoring results. The Ethics Committee is chaired by Professor John Mawson and comprises representatives from the University of Dundee and the Dundee Partnership.

8. Statement of Commitment

The Dundee Partnership have, through the establishment of a holistic structure, put in place the necessary arrangements for high quality monitoring and evaluation, which seeks to underpin the PPA in ways which ensure an evolving partnership approach and involvement at every stage of those communities whom the PPA initiative is targeted towards.

 

Contact: Keith Fernie or MG Lloyd at the PPA Monitoring and Evaluation Unit, School of Town and Regional Planning, University of Dundee on 01382 345048 / 433219 and/or e-mail him on either k.fernie@dundee.ac.uk / keith.fernie@dundeecity.gov.uk

Monitoring and Evaluation Loop

 

Organisational Chart

Return to Top