100-10 Quality Improvement Organization Manual: Chapter 15 - Performance Evaluation

Quality Improvement Organization Manual
Chapter 15 - Performance Evaluation
Table Of Contents
(Rev. 31, 07-21-17)
Transmittals for Chapter 15
FEDERAL REGISTER NOTICE OF EVALUATION CRITERIA
15100 - Authorities
15110 - Provisions of the Notice
15120 - Uses of Evaluation Criteria
CONTRACT EVALUATION PROCESS
15200 - Timing
15210 - Methods of Evaluation
15220 - Renewal/Non-renewal
15230 - Methods of Evaluation
PERFORMANCE IMPROVEMENT PLANS (PIPs)
15400 - Performance Improvement Plans
15420 - Performance Plan Expectations
TERMINATION PROCEDURES
15500 - Statutory Basis
15510 - Grounds for Termination
15520 - Recommendation to Initiate Termination
15530 - Notice of Intent to Terminate Contract
15540 - Termination Panel
15550 - Termination Decision
RENEWAL/NON-RENEWAL
15600 - Renewal/Non-renewal
CLOSE-DOWN ACTIVITIES
15700 - Introduction
15710 - Boxing of Records - General
15720 - Boxing of Review Records
15730 - Boxing of Data

15740 - Boxing of Miscellaneous Records
15750 - Retention of Financial Records

15100 - Authorities
(Rev.31, Issued: 07-21-17, Effective: 07-21-17, Implementation: 07-21-17)
Section 1153 of the Social Security Act (the Act) generally provides authority for CMS to
enter into contracts with QIOs for the performance of QIO functions described in section
1154. CMS may modify, renew, or discontinue all or part of one or more tasks for one or
more states or territories in the QIO service area or any other part of the contract.
Sections 1153(c)(2) and (4) of the Act provide for CMS to assess the quality and
effectiveness of each QIO in carrying out the functions specified in its contract when
evaluating QIO performance.
Section 1153(h)(1) of the Act requires CMS to publish notices in the Federal Register
whenever it makes significant policy and/or procedural changes that substantially affect
performance of QIO contract obligations 30-days before such change is to take effect.
Section 1153 (h)(2) of the Act requires the Secretary to publish in the Federal Register
the general criteria and standards used for evaluating performance of QIO contract
obligations, and provide an opportunity for public comment.
Section 1153 (h)(3) of the Act requires CMS to regularly provide each QIO an
assessment of its performance in relation to the performance of other QIOs.
15200 - Timing
(Rev.31, Issued: 07-21-17, Effective: 07-21-17, Implementation: 07-21-17)
CMS publishes Federal Register notices for each type of QIO contract that it plans to
award for each QIO contract award cycle. These notices provide the opportunity for
public comment on the general criteria and standards CMS will use to evaluate QIO
performance based on the tasks or subtasks in the statement of work and the evaluation
and performance measures included in QIO contracts as required by Section 1153 (h)(2)
of the Act.
After QIO contracts are awarded, CMS may revise contract performance criteria to
adjust evaluation measures, adjust the expected minimum thresholds for satisfactory
performance, or add or remove criteria from a contractual task for any reason,
including, but not limited to, experience with the amount of improvement achieved during
the contract cycle, information gathered through evaluation of the performance overall,
or any unforeseen circumstances through negotiation of contract modifications or change
orders. CMS negotiations with QIOs of any contract change (modification) or non-
competitive renewal are administered by the cognizant CMS contracting authority in
accordance with authorities of 48 CFR Federal Acquisition Regulations and notices are
not posted in the Federal Register unless required by section 1153(h)(1).

15210 - Methods of Evaluation
(Rev.31, Issued: 07-21-17, Effective: 07-21-17, Implementation: 07-21-17)
Substantial elements of QIO contracts are performance-based, such that a QIO is
assessed relative to specific performance measures established in the contract.
Performance on these measures can be used to objectively make determinations about
non-competitive renewals and/or be used as objective criteria in competitive
procurements. Additionally, CMS may also assess QIO performance based on annual
routine assessment, regularly scheduled teleconferences, data analysis, off-site reviews of
QIO project activities, contract deliverables, reporting, data reports, interviews with QIO
staff, customer satisfaction and/or an examination of other pertinent records.
CMS follows federal acquisition regulations when assessing QIO contractor performance
on the contract. CMS conducts contract performance monitoring activities throughout
the contract term and will act upon findings as necessary.
CMS performs regular formal assessment of each QIO’s overall contract performance
based on the general evaluation criteria and standards identified in each QIO contract.
The assessment is provided to the QIO through the CMS Contracting Officer using the
Contractor Performance Assessment Reporting System (CPARS) criteria used in
performance evaluations: Quality, Schedule/Timeliness, Cost/Price Control, Business
Relations, Management, and Small Business. CMS considers the published general
evaluation criteria and standards within the Quality sub-factor within the CPARS
assessment of overall contractor performance.
CMS performs regular program impact assessments and may contract with an
independent entity to perform assessment of overall QIO Program performance.
Program evaluations are conducted separately from individual QIO contractual
performance evaluations. CMS makes QIO Program performance assessment results
available to each QIO as part of regular QIO program evaluations.
15220 - Renewal/Non-renewal
(Rev.31, Issued: 07-21-17, Effective: 07-21-17, Implementation: 07-21-17)
One of the purposes of the evaluation process is to make determinations on whether QIOs
are eligible for non-competitive contract renewals. Details of the renewal/non-renewal
process can be found in the QIO contract.
15230 - Methods Of Evaluation - (Rev. 1, 05-23-03)
Substantial elements of QIO contracts are performance-based, where a QIO is assessed
relative to specific performance measures established in the contract. Performance on
these measures can be used to objectively make determinations about non-competitive
renewals and/or be used as objective criteria in competitive procurements. Additionally,
ROs assess QIO performance based on on-site visits, regularly scheduled teleconferences,
data analysis, and off-site reviews of QIOs. Specifically, the on-site visit permits the RO
POs and/or Scientific Officers (SOs) to have a face-to-face meeting and allow CMS an

opportunity to understand all of the QIO activities within the context of their state. This
process entails the evaluation of QIO project activities and data reports, interviews with
QIO staff, and an examination of other pertinent records.
Ongoing monitoring will be accomplished by telephone and videoconferencing between
the PO and the QIO, on-site visits and analysis of information reported in project plans,
routine written reports, and review of contract deliverables.
The findings that result from on-site visits are compiled into formal summaries that
describe the QIO’s progress and performance. If the PO identifies major deficiencies, a
Performance Improvement Plan (PIP) will be requested. In the event a QIO is
unsuccessful at correcting the deficiencies contained in the PIP, CMS will consider
terminating the contract. RO POs would continue to perform ongoing monitoring of
other aspects of the contract. These include:
 	QIO proposed information collection activities (e.g., surveys), abstracts or articles
submitted to non-CMS sponsored peer reviewed publications or meetings;
 	Subcontract arrangements;
 	QIOs' internal quality control process;
 	QIOs’ case review requirements;
 	Communication requirements (Web site, physician/provider outreach, annual
reports, etc.);
 	Requests related to quality improvement projects (including ad hoc data requests,
CDAC requests, and approval of locally developed projects);
 	Confidentiality requirements;
 	Board membership and structure; and
 	QIO resource allocation/utilization and management (and other financial
information).
15400 - Performance Improvement Plans
(Rev.31, Issued: 07-21-17, Effective: 07-21-17, Implementation: 07-21-17)
CMS may request a QIO to provide a corrective action (Performance Improvement Plan)
to address noted deficiencies.

15550 - Termination Decision - (Rev. 1, 05-23-03)
The CMS may or may not accept the findings of the panel. After the panel has submitted
its report, CMS may, with concurrence of the QIO, amend the Statement of Work (SOW)
to modify the QIO’s functions or otherwise change the contract. Also, CMS may elect to
terminate the QIO’s contract upon 90 days notice after submission of the panel’s report or
earlier if the QIO agrees. In accordance with §1153(f) of the Act, any determination by
the Secretary to terminate a contract shall not be subject to judicial review.
From the time CMS receives the panel’s report and gives the notice of intent to terminate,
CMS may transfer review responsibilities of the QIO under the contract being terminated
to another Utilization and Quality Control QIO, or to an intermediary or carrier having an
agreement under §1816 of the Act or a contract under §1842 of the Act until CMS enters
into a contract with another QIO.
15600 - Renewal/Non-renewal Procedures - (Rev. 1, 05-23-03)
One of the purposes of the evaluation process is to make determinations on whether QIOs
are eligible for non-competitive contract renewals. Details of the renewal/non-renewal
process can be found in the QIO contract.

Transmittals Issued for this Chapter
Rev # 	Issue Date 	Subject 	Impl Date CR#
R15QIO 	07/21/2017 	QIO Manual Chapter 15 – “Performance
Evaluation”
07/21/17 	N/A
R1QIO 	05/23/2003 	Initial Release of Chapters 2,8,13 and 15 	N/A 	N/A
Back to top of chapter