Policy and plan effectiveness monitoring
Monitoring the efficiency and effectiveness of policies, rules or other methods in policy statements or plans (policy and plan monitoring) is an important part of the resource management planning process. It has a direct relationship to sections 32 and 35 of the Act. It is an ongoing activity throughout the planning cycle to assess how well the plan is working. Policy and plan effectiveness monitoring helps determine the need for further action, and possible changes and improvements in policy statements and plans, or in actions taken to implement them.
The Resource Management Act (RMA) requires local authorities to monitor the efficiency and effectiveness of policies, rules, or other methods in its policy statements or plans; to take appropriate action when monitoring indicates that this is necessary and to prepare a report at least every five years on the results of section 35(2)(b) monitoring.
Monitoring closes the loop in the 'plan - do - monitor - review' cycle and informs decision-makers of the consequences of actions and changes in the environment. Policy and plan effectiveness monitoring provides a means for determining how well plans are working in practice. It both builds on and provides information for state of the environment monitoring and can be assisted by monitoring resource consents, compliance and complaints. It is important to have an integrated approach.
Start plan monitoring early
- Think about monitoring at the start of the policy and plan development process. Do not wait until the plan is fully operative or useful information may be missed.
- Develop a monitoring/evaluation programme at the same time as plan provisions are developed but recognise things may change.
- Monitor the implementation of policy and plans. It is essential to know that plan methods are being implemented before checking that anticipated outcomes are being achieved.
- Remember monitoring is a systematic and ongoing process rather than a one-off task. This means there will be a continuous review process for policies and plans.
- Be focused; monitor priority matters first. Many councils have started by monitoring and reviewing the environmental results expected (EREs) in policy and plans. This helps to determine if EREs are being achieved and whether or not they need to be more focused (and rewritten) to be measurable. This is all part of the monitoring process.
- Consider the plan's context and external factors, eg, population growth or decline.
- Establish the links with monitoring the state of the environment and the monitoring of resource consents, compliance and complaints.
Have a clear purpose for policy and plan monitoring
Significant resources are invested into developing policy and planning documents so it is important to have a means of checking that this investment has been worthwhile. Policy and plan monitoring is more than a statutory requirement. It is a useful management tool to evaluate and review the effectiveness of policy provisions and plans.
- Have a clear purpose for policy and plan monitoring. Is it for:
- accountability to the community (to show you have provided a means of managing what you said you would manage and achieved the plan 's environmental, economic, social and cultural goals - such as required by section 32 of the Resource Management Act)? OR
- continuous improvement of your organisation? OR
- both? (which is likely to be the most useful approach).
- Consider summarising the approach in a monitoring strategy.
- Policy and plan effectiveness monitoring is systematic and involves tracking and evaluating whether and how well policy or plan implementation is resolving the issues raised in plans.
- Is the policy or plan achieving its objectives? How do you know?
- Are the implementing agencies delivering on anticipated outcomes?
- Have the environmental outcomes have been achieved? This has strong links to state of the environment monitoring and reporting.
- How effective have policy or plan preparation and implementation processes been?
- Does the policy or plan cover the most important things? Are there emerging issues that are not being addressed? (This is an extension of the section 32 process – see s.32(2)(c)) This also links closely to state of the environment monitoring.
Be systematic and apply a consistent approach
- There are few well-established systems for policy and plan monitoring and reporting, but there is a lot of monitoring and evaluation research to build on. The monitoring process includes:
- being clear about the purpose and goals
- stating what will be monitored and why
- developing indicators
- developing methodologies
- consistent collection of data
- analysing, interpreting and presenting information
- reviewing the plan or delivery of implementation programmes as a result of this process and triggering a continuous review and reporting cycle
- making policy changes and adjustments as necessary ie, taking action.
- There is no one right approach to policy and plan effectiveness monitoring. The approach taken should be flexible.
- Build on other agencies' policy and plan monitoring efforts.
- Ensure the approach used is relevant to your situation.
- Include both quantitative and qualitative assessments (use multiple methods and triangulate the results for greater confidence when making inferences).
- Include questions about WHY, HOW, WHO, WHAT, WHERE, WHEN to monitor. Thresholds are an important consideration in defining and answering these questions.
- Develop a commonly used terminology and language.
- Have a good project management process.
What and how to monitor
- Most councils monitor environmental results expected from policy statements and plans to assess outcomes. Methods are also important as they determine the implementation programme.
- It is important to monitor whether the objectives of the plan or policy statement are being met. If the environmental results expected are not clearly focused on meeting the plan's objectives, then just monitoring EREs will not give you the information, it may be necessary to rewrite the EREs in the plan.
- If your policy/plan structure is simple and clear and your provisions clear and focused, monitoring will be more manageable.
- Ensure key issues, outcomes, processes, impacts and implementation are monitored (this means you need an integrated approach with other monitoring like state of the environment and consents monitoring).
- Think about what information you need to decide whether any change in the environment is due to the effect of the policy statement/plan or to other factors beyond this.
- Think about how to monitor and review permitted activities (eg, links to complaints, state of the environment monitoring).
- Develop a system to trigger reviews of policy statements, plans and implementation programmes in response to the results of policy and plan monitoring.
- Section 79 of the RMA requires councils to undertake a review of provisions in their policy statements and plans at least every 10 years. Given the large number of provisions in each plan, it would be good practice for councils to establish a formal monitoring and recording system to assist the statutory review process.
Have a strategic and integrated approach
- Develop criteria for prioritising monitoring so it is cost-effective.
- Develop indicators to assess the means and ends of the plan, and how effective implementation of policies and methods was/is.
- Make the most of the data you already collect and supplementary information from different agencies. Consider links between:
- other policy objectives within council such as non-Resource Management Act requirements including pest management, reserve, land transport strategies, Long Term Plans and so on
- territorial local authorities, regional councils and others
- state of environment, consents, complaints, compliance monitoring
- different environmental media and other agencies' work.
- Combine your council databases to include all useful existing information (state of environment, compliance and complaints monitoring) and feedback to policy and other plan provisions.
Who to involve
- Corporate commitment is needed to monitor plan effectiveness.
- Involve integrated council teams to ensure policy/plan implementation goals are well understood, for example include planners, scientists, environmental education staff, database managers, engineers, consent and compliance staff.
- Work with other councils in your area on monitoring methodologies for similar issues (eg, amenity values/cross-boundary issues).
- Where possible form or join a monitoring forum to share ideas and information or set up other partnerships and review processes.
- Involve iwi as they are more than another affected party.
- Involve other members of the community as appropriate and find out what community outcomes are wanted and to ensure relevance of your policy and plan provisions. Link this to your requirements under the Local Government Act for Long Term Plans.
- Work in with other agencies such as regional councils, territorial local authorities, Department of Conservation, Ministry for the Environment, Statistics NZ, and build on existing information if possible.
Share ideas and allow for training
- Think about setting up plan effectiveness workshops and focus groups; sharing ideas is helpful and can make monitoring easier.
- Develop partnerships as appropriate and consider the use of a regional monitoring forum to share ideas.
- Train people and develop learning environments for staff.
- Get external guidance where necessary.
- Establish cause-and-effect relationships where possible and illustrate and report on attribution. What has changed, and how does this relate to the policy statement or plan?
- It is important to monitor process, policy and plan implementation, outputs, impacts and outcomes. If all of these things are monitored (rather than just focusing on process) then we will be a step closer to determining causality.
- Be flexible and ask strategic questions like WHY, HOW, WHO, WHAT, WHERE, and WHEN to monitor (known as programme logic). In relation to each issue or group of linked issues:
- does the plan make a difference? Is it effective?
- was it well implemented?
- what worked and didn't work?
- what are the risks of not taking any action, or of taking certain types of action?
- Consider a range of methods to help assess what difference the policies and plan has made on the ground (such as using perception or satisfaction surveys, decision trees, monitoring letters to the editor, sequential visual analysis (eg, by comparing photos over time), comparative risk analysis, and triangulation of approaches).
- The key challenge is to move beyond measuring only processes and outputs, and start focussing on the impacts and outcomes achieved by the policies and plans.
What to do once plan effectiveness is established
- Record the information from your monitoring and evaluation.
- Analyse the results of monitoring and interpret what they mean.
- Report on the results of monitoring and on the effectiveness of the plan/policy statement.
- Assess where changes may be required to your policy provisions and plans because of the results of your monitoring.
- Review your plan provisions as a result of monitoring and evaluation of effectiveness.
- Make recommendations for action (which usually involves reporting on monitoring).
- Take action and trigger reviews as appropriate.
- Provide feedback to the council and any others involved in plan development and the monitoring process.
- Ensure there is a good written record of this process and
- the results of monitoring
- evaluation of effectiveness
- recommendations for action and
- actual actions and reviews undertaken.
Section 28A(1)(b) requires a regional council to supply information about the monitoring of its regional coastal plan to the Minister of Conservation, if requested. Regional councils have 20 working days to supply this information, unless a longer time frame is set by the Minister of Conservation (s28A(3)).
Section 32 requires an evaluation report to be prepared for changes to policies and plans. Among other things, the report must assess the efficiency and effectiveness of proposed provisions in achieving objectives. Environmental, economic (including growth and employment), social, and cultural effects are all relevant considerations.
Section 35 specifies the duty to gather information, monitor and keep records. In particular s35(2)(b) requires every local authority to monitor the efficiency and effectiveness of policies, rules or other methods in its policy statement or plan for its region or district.
Under s35(2A) local authorities are required to prepare a report at least every five years on the results of their monitoring under s35(2)(b) for policy and plan efficiency and effectiveness. This may be some form of integrated policy and plans, consents and compliance, complaints and state of the environment reporting.
Section 35(3) requires every local authority to keep reasonably available at its principal office, information which is relevant to the administration of policy statements and plans, the monitoring of resource consents, and current issues relating to the environment in the area, to enable the public -
- to be better informed of their duties and of the functions, powers and duties of the local authority
- to participate effectively under the Act.
Section 62(1)(j) requires that a regional policy statement must state... 'the procedures used to monitor the efficiency and effectiveness of the policies or methods contained in the statement'.
Section 67(2)(e) notes that a regional plan may state... 'the procedures for monitoring the efficiency and effectiveness of the policies and methods' contained in the plan.
Section 75(2)(e) notes that a district plan may state... 'the procedures for monitoring the efficiency and effectiveness of the policies and methods' contained in the plan.
Section 79 requires councils to undertake a review of provisions in their policy statements and plans at least every 10 years.
Section 360(hk), (hl) and (hm) allow regulations to be made that specify requirements for indicators, standards, methods, thresholds and the timing of reporting.
The Local Government Act 2002
The Local Government Act 2002 requires local authorities to produce a Long Term Plan that outlines community outcomes and provides a basis for accountability from the local authority to the community. A Long Term Plan must cover a period of 10 consecutive financial years (s.93(6)). In the Long Term Plan, councils are required to describe community outcomes, and the Auditor General must report on how that requirement has been met, not less than once every three years.
Section 98(2) also requires local authorities to compare annually their actual activities and performance with their intended activities and level of performance as set out in the Long Term Plan and the annual plan.