Occasional LTC Policy Paper Series © 1997 Duke LTC Resources
Enhancing the Performance of North Carolina's
Area Agencies on Aging: A Best Practice Approach in Brief
Jennifer Hoffmann, MPP
Policy Analyst
Duke Long Term Care Resources
Overview
Performance standards are used by NC's eighteen Area Agencies
of Aging to assess their individual performance, to suggest
areas for improvement and to communicate their effectiveness
to aging network stakeholders. |
Under contract to the North Carolina Association
of Area Agencies on Aging (NC4A), the Duke Long Term Care Resources
Program (Duke LTC) agreed to develop performance standards for
North Carolina's 18 Area Agencies on Aging (AAA's). Beginning
in August 1995, Duke LTC worked with AAA Directors and their
staffs as well as AAA stakeholders to develop performance standards
to enhance what North Carolina AAA's do and to help communicate
effectively the work of AAA's. Emphasis was placed on
standards that could be measured, demonstrated and communicated.
In January 1996, a final report was presented to NC4A with recommended
performance standards, including best practices and performance
measures organized around six broad activities performed by
all of North Carolina's AAA's. The development of these standards
was the first step in an ongoing process that continues today
which includes the refinement of the standards and the monitoring
of AAA's concerning their progress toward and adherence to the
standards. With the support of the NC Division on Aging, AAA
leadership is, in fact, implementing the report recommendations.
Background
Critical Question: "How can AAA's be more effective?"
|
In the summer of 1994, the North Carolina
Association of Area Agencies on Aging began to work on a strategy
to enhance the role of AAA's in the aging network, local governments
and the larger community. The Association leadership
focused on a critical question, "How can AAA's be more effective?"
To help answer that question, the Duke Long Term Care Resources
Program, at the request of NC4A, agreed to work on a project
to develop performance standards for NC's 18 Area Agencies on
Aging.
Duke was not asked to evaluate AAA Directors, staff or their
program activities. Instead, Duke was asked to develop
performance standards that could be used by AAA's to assess
their performance and begin to address the questions of
effectiveness and implementation.
Developing
Performance Standards
Performance measures are the specific indicators that
are used to gauge the achievement of the program goals.
One of the performance standards for AAA's could be to
"successfully resolve complaints received by the ombudsman
program."
AAA stakeholders were asked to describe activities, goals
and assessment procedures. |
Performance standards generally can be defined
as a set of clearly defined goals (targets) designed to describe
a desired outcome. Performance measures are the specific indicators
that are used to gauge the achievement of the program goals.
And always, performance standards and measures are intended
to reflect an agency's mission and effective ways to document
achievement. For example, one of the desired outcomes of the
ombudsman program, an important programmatic activity of North
Carolina AAA's, is to resolve complaints made by, or on behalf
of, residents of long term care facilities. It follows then,
that one of the performance standards for AAA's could be to
"successfully resolve complaints received by the ombudsman program."
The performance measure would be the percentage of complaints
resolved. To track this outcome, one would need to keep
a log of all complaints made, investigated and resolved. The
specific percentage which needs to be achieved in order to declare
the standard as having been met must be established using such
techniques as projections from historical information or negotiations
among stakeholders. For the ombudsman example, where historical
data do exist upon which to base a measure, 90% of complaints
resolved would be a reasonable measure. In this way, a performance
standard that is in keeping with the mission of the ombudsman
program to protect long term care residents can be developed
to help answer the question of "Did this program do a good job?"
To develop performance standards relevant to North Carolina
AAA's, intensive on-site interviews were conducted with 17
AAA Directors (one region's Director position was vacant at
time of interviews) and surveys were mailed to AAA staff, Lead
Regional Organization (LRO) Directors, Division on Aging
(DoA) staff, and members of the Regional Advisory Councils
on Aging (RACA) and Home and Community Care Block Grant
Committees (HCCBG). The purpose of these interviews and
surveys was to answer the following three questions that are
necessary to the development of performance standards:
What do AAA's do, including asking what they
should be doing and what their priorities are?
What are the realistic goals that relate to AAA's
priorities?
What are ways to measure AAA's progress
towards achieving these goals?
Findings
Among AAA stakeholders in NC there is a consensus that AAA's
should be involved in ensuring quality services and collaborating
to identify unmet needs and stimulate solutions.
The responses from the stakeholders' and AAA Directors' respective
surveys follow.
- AAA's role in the "aging network" was perceived by 78 percent
of stakeholders who responded to the survey as either "essential"
or "useful".
- When asked to rank in importance a list of eight "official"
functions of AAA's, "assessment of community service
providers," "development of an elder rights system though the
ombudsman program" and "collaboration in the community to
identify unmet needs and stimulate solutions" were ranked as
either "essential," "very important" or "important" by at least
70% of respondents in each of the five stakeholder groups.
- When asked their opinion about the priority functions of AAA's,
stakeholders across the board listed, "assessment of community
service providers" and "collaboration in the community to
identify unmet needs and stimulate solutions" as top priorities.
Three other areas rated as priorities included: "development of
an elder rights system though the ombudsman program,"
"development of an area aging plan" and "technical assistance to
the home and community care block grant committees."
- AAA Directors (16 of 17) overwhelmingly rated "advocacy" as a
priority activity with which AAA's should be involved. The
other four activities that AAA Directors rated as priorities
included: information brokerage, planning, quality assurance
and program and resource development.
Conclusions
Broad mandates of the Older Americans Act make it difficult
for AAA's to identify and focus on a reasonable number of annual
goals. Administrative tasks demand a large percentage of AAA
Directors' time. Best practices are not an end in themselves
but suggest processes that lead to good outcomes.More useful
planning is indicated and possible.
- Mandates vs. Capacities. AAA Directors are extremely
involved in their communities and knowledgeable about both
Older Americans Act (OAA) programs as well as other publicly-
and privately-funded community-based services. However,
the broad mandates of OAA has led to relatively small AAA
staffs that are spread thinly over a large number of programs
and issues which make it difficult for AAA's to identify
and focus on a reasonable number of annual goals. As a result,
AAA's have expanded in several different directions at once
without a clear sense of what they are trying to achieve
or capable of achieving. AAA Directors are in general agreement
about the types of activities in which they should be involved
(advocacy, program development, planning, information brokerage
and assuring quality services), but the majority of AAA
Directors have considerable difficulty in articulating specifically
what they want to achieve with these activities.
- Advocacy vs. Administration. AAA Directors expressed a strong
desire to be doing advocacy and program development, but
spend most of their time performing administrative tasks. As a
result, they also tend to define "a good job" in terms of
administrative measures -- low error rates, high grant spend rate,
clean audits. Almost all of the AAA Directors mentioned
feedback from stakeholders as important, but only one had
developed recently (within the past year) a formal mechanism
for collecting this feedback information.
- A list of best practices. The lack of consensus on what AAA's
want to achieve (the outcomes of their activities) makes the
development of meaningful performance standards (a common
set of goals) very difficult. However, most of the AAA Directors
were able to suggest one or two measures of success or
performance standards related to what they hoped to achieve
with their activities. From these suggestions, an illustrative short
list of performance standards and measures has been developed.
The performance standards and measures are in a "best
practices" format which are organized to include activity areas
identified by AAA Directors as important. Those areas are
advocacy, program and resource development, planning,
information brokerage and assuring quality services. The best
practices are grouped around six areas:
1) advocacy
2) planning
3) program and resources development
4) information brokerage
5) funds administration (of federal and state
aging funds)
6) management and staffing
- Assuring quality services can be found under the broader headings of
funds administration and advocacy. The sixth set of best practices
was developed for management and staffing of AAA's because activities
in those areas are integral to the achievement of performance standards.
These best practices are not an end in themselves, but suggest
processes that lead to good outcomes and can be used as AAA's continue
to develop and refine performance standards.
- AAA's Important Roles: Quality Assurance and Collaboration.
There was a overwhelming consensus among the stakeholders
(DoA staff, LRO Directors, AAA staff, and RACA and HCCBG
members) surveyed that AAA's play an essential or very
important role in the aging network. There was also a consensus
among AAA stakeholders around two main functions that AAA's
should be involved in: 1) assessment of service providers to
ensure quality services, and 2) collaboration to identify unmet
needs and stimulate solutions. Three other areas also emerged as
important: 1) the development of an elder rights system through
the ombudsman program, 2) development of an area aging plan
and 3) technical assistance to HCCBG committees.
- Useful Planning Needed. Useful planning was identified by
everyone surveyed as an unrealized but important goal of AAA's.
Ironically, 13 of 17 AAA Directors surveyed said they never used
effectively the standardized area aging plans they have to
develop. Comments generally revolved around the perception
that the standardized format for the plan does not accurately
reflect what the AAA's do and consequently becomes "a time
consuming paper process." These comments cannot be
dismissed as simply complaints by the AAA Directors, because 7
out of 9 DoA staff members who responded to the survey made
similar comments. This incongruity between the role that
planning should play and the role it plays now needs to be
addressed. More useful planning is indicated and possible
Recommendations
The development of performance standards is an ongoing process
that needs continuous refinement. Choosing a measure that specifies
when a standard has been met provides AAA's with a clear understanding
of what is expected of them. Pilot implementation.Stakeholder
involvement is essential. Revamp and revitalize the area aging
planning process.
This report, based on both surveys and interaction with all of North
Carolina's AAA's and a broad range of their stakeholders, leads to the
following three recommendations to the North Carolina Association of
Area Agencies on Aging:
- Continue the strategic process
to enhance the performance of AAA's that was begun in 1994.
Developing performance standards for organizations like
AAA's which have broad missions and play interstitial roles
in service provision is particularly challenging. The NC4A
is to be commended for moving in this direction. For all
organizations, the development of performance standards
is an ongoing process that needs continuous refinement.
The important next step for the Association is to choose
reasonable areas in which to begin implementation. Specific
recommendations are:
- Adopt the "best practice" format for
standards which are organized around the basic activity
areas of the day to day work of AAA's.
- Choosing the Right Measure. The
performance indicators need to be refined by further specifying
when an indicator is met. Where the recommended indicators
have left open the designation of the measure, AAA Directors
need to specify a quantification or threshold through reflection
of historical trends, negotiations with stakeholders, or
both.
- Choose one or two of the basic
activity areas to pilot the implementation of performance
standards. Funds administration and advocacy should
be strongly considered because those two areas contain standards
related to the functions identified as important by AAA
stakeholders, particularly assessment of service providers
and collaboration to identify unmet needs and stimulate
solutions. After the initial implementation period (at least
one year), the Association should take the lead in reviewing
the process and recommending further action.
- Responsibilities for Implementation.
At this point, responsibility for implementation of performance
standards for AAA's rests primarily with the AAA Directors.
But if the Association intends to use this process as a
strategy to enhance overall performance and to communicate
that achievement to the broader aging network and older
adult constituents, then it is absolutely essential that
stakeholders, including older adults and service providers,
be involved in the implementation process. It is important
to note, however, that there were strong undercurrents,
in across the board comments, that performance standards
should not contribute to proliferation of bureaucratic activities.
Hence, the Association should carefully consider using existing
committees to the extent possible in the implementation
process.
- Revamp and Revitalize the Area
Aging Planning Process. The area aging plan could be an
excellent vehicle to facilitate the implementation of performance
standards. If the area aging plan can become more
of a working document that specifies the goals and objectives
relevant to each region, then it would better reflect performance
indicators that measure achievement of AAA objectives. The
Association leadership should work with the planning staff
of DoA to clearly define what DoA and the AAA's hope to
achieve with the planning process and refine the format
of the planning document so that it helps achieve those
ends.
Status Report
Recommendations implemented. |
In collaboration with the NC Division of
Aging (DoA), the recommendations listed above have been implemented
by the NC Association of Area Agencies on Aging including the
adoption of the best practices and the revision of the
area aging planing process. A structured performance standards
review process began in 1997 in which all 18 AAA's have been
or will be visited by a review team (comprised of DoA and AAA
staff, local providers and older consumers) to monitor individual
AAA's progress toward achieving the adopted standards.
Useful References
Behn, Robert D. Bottom-Line Government,
The Governors Center at Duke University, 1994.
Behn, Robert D. Leadership Counts: Lessons for Public Managers,
MA: Harvard University Press, 1991 (Chapter 4).
Behn, Robert D. "Performance Measures: To Punish Or to Motivate?,"
Governing, July 1993, p. 84.
Behn, Robert D. "The Wrong Way to Motivate," Governing, December
1994, p. 70.
Behn, Robert D. "Measuring Performance Against The 80-30 Syndrome,"
Governing, June 1993 p. 70.
Estes, Carroll L., Jill C. Feasley and Jo Harris-Wehling Editors.
Real People, Real Problems: An Evaluation of the Long-Term Care
Ombudsman Programs of the Older Americans Act. Washington, DC:
Institute of Medicine, 1995.
Osborne, David and Ted Gaebler. Reinventing Government: How
the Entrepreneurial Spirit is Transforming the Public Sector.
MA: Addison-Wesley Publishing Company, 1992 (Chapter 5 and Appendix
B).
"Managing for Results: The Department of Justice's Initial Efforts
to Implement GPRA," GAO Report, June 1995.
Oregon Benchmarks: Standards for Measuring Statewide Progress
and Government Performance. Oregon Progress Board. Report to
the 1993 Legislature, December 1992.
DRAFT Senior and Disabled Services Division Performance Measures
Senior and Disabled Services Division, Oregon Department of
Human Resources
Acknowledgments
We thank Lousia Cox, Past President of the North Carolina Association
of Area Agency on Aging, who both initiated this project and
served as its contract administrator. Additionally, we thank
the many people and organizations whose cooperation in filling
out and returning the surveys facilitated the preparation of
this report including Lead Regional Organization Directors,
Area Agency on Aging staff, Division of Aging staff, and members
of Home and Community Care Block Grant Committees and Regional
Advisory Councils on Aging.
Finally, we express our appreciation to the members of the steering committee for their
assistance, particularly Kim Dawkins Berry, Steve Freedman and Gina Shell for their
detailed comments about the performance standards themselves.
back to top
|
