FTC Comment Before the Federal Energy Regulatory Commission Concerning Appropriate Metrics for Assessing the Performance of Electronic Regional Transmission Organizations and Independent System Operators

UNITED STATES OF AMERICA
FEDERAL ENERGY REGULATORY COMMISSION
RTO/ISO Performance Metrics 	) 	Docket No. AD10-5-000
REPLY COMMENT OF THE FEDERAL TRADE COMMISSION
March 19, 2010
I. 	Introduction
The Federal Trade Commission (FTC) appreciates this opportunity to comment on the
Federal Energy Regulatory Commission’s (FERC’s) Notice Requesting Comments on RTO/ISO
Performance Metrics (Notice).1 FERC lists several potential performance measures that it
developed in consultation with regional transmission organization (RTO) and independent
system operator (ISO) employees in response to a Government Accountability Office report
recommending the development and use of standardized measures of the performance of RTOs
and ISOs. 2
Developing and tracking the performance of RTOs is a laudable objective. We note,
however, that the potential performance measures that FERC has proposed do not measure all
“minimum characteristics and functions” of RTOs that FERC articulated in the founding orders
for those organizations. For example, none of the proposed performance metrics assesses
whether an RTO is constructively engaged in resolving issues that require regional coordination.
Further, the Notice and accompanying list do not recognize the adverse consequences that could
occur unintentionally if RTOs were evaluated by quantitative performance metrics that do not
include all relevant aspects of their performance.
We recommend that FERC select performance metrics that will evaluate accurately the
degree to which RTOs display the required characteristics and perform their required functions.
We further urge FERC to address explicitly the risk of potential distortions in RTO performance
that may result from flawed or incomplete performance metrics. Next, we recommend that
FERC consider adding to the minimum characteristics and functions of RTOs a requirement to
operate efficiently, including being responsive to grid users and the retail customers they serve.
Finally, we respond to the approach that the “Consumer Commenters” group has taken in its
initial comment.
1 75 Fed. Reg. 7581 (Feb. 22, 2010).
2 Except for isolated references to “ISOs,” in this comment the term “RTO” or “RTOs” includes
ISOs.

2
II. 	Interest of the FTC
The FTC is an independent agency of the United States Government responsible for
maintaining competition and safeguarding the interests of consumers, both through enforcement
of the antitrust and consumer protection laws and through competition policy research and
advocacy. The FTC often analyzes regulatory or legislative proposals that may affect
competition or allocative efficiency in the electric power industry. The FTC also reviews
proposed mergers that involve electric and natural gas utility companies, as well as other parts of
the energy industry. In the course of this work, as well as in antitrust and consumer protection
research, investigation, and litigation, the FTC applies established legal and economic principles
and recent developments in economic theory and empirical analysis.
The energy sector, including electric power, has been an important focus of the FTC’s
antitrust enforcement and competition advocacy.3 The FTC’s competition advocacy program has
produced two staff reports on electric power industry restructuring issues at the wholesale and
retail levels.4 The FTC staff also contributed (as did FERC staff) to the work of the Electric
Energy Market Competition Task Force, which issued a Report to Congress in 2007.5 In
addition, the FTC has held public conferences on energy topics, including Energy Markets in the
21st Century (April 10-12, 2007)6 and Carbon Offsets & Renewable Energy Certificates
(January 8, 2008). 7
3 See, e.g. Opening Remarks at the FTC Conference on Energy Markets in the 21st Century:
Competition Policy in Perspective (Apr. 10, 2007), available at
http://www.ftc.gov/speeches/majoras/070410energyconferenceremarks.pdf. 	FTC merger cases
involving electric power markets have included the DTE Energy/MCN Energy (2001) (consent
order), available at http://wwwftc.gov/os/2001/05/dtemcndo.pdf; and PacifiCorp/Peabody
Holding (1998) (consent agreement), available at
http://www.ftc.gov/os/1998/02/9710091.agr.htm. (The FTC subsequently withdrew the
PacifiCorp settlement when the seller accepted an alternative acquisition offer that did not pose a
threat to competition.)
4 FTC Staff Report, Competition and Consumer Protection Perspectives on Electric Power
Regulatory Reform: Focus on Retail Competition (Sept. 2001), available at
http://www.ftc.gov/reports/elec/electricityreport.pdf; FTC Staff Report, Competition and
Consumer Protection Perspective on Electric Power Regulatory Reform (July 2000), available at
http://www.ftc.gov/be/v000009.htm (compiling previous comments from the FTC staff provided
to various state and federal agencies).
5 See http://www.ferc.gov/legal/fed-sta/ene-pol-act/epact-fina-rpt.pdf.
6 Conference materials available at
http://www.ftc.gov/bcp/workshops/energymarkets/index.shtml.

3
The FTC and its staff have filed numerous competition advocacy comments with FERC
and participated in FERC technical conferences on market power issues. For example, in March
2007, the Deputy Director for Antitrust in the FTC’s Bureau of Economics served as a panelist
for a technical conference on FERC’s merger and acquisition review standards under Federal
Power Act (FPA) Section 203 (Docket No. AD07-2-000). Similarly, the FTC submitted
comments in December 2009 in FERC’s proceedings on possible elements of a National Action
Plan on Demand Response (Docket No. AD09-10-000)8 and on transmission planning processes
(Docket No. AD09-8-000).9 The FTC also has commented on FERC’s initiatives to promote
wholesale electricity competition and on various state issues associated with restructuring the
electric power industry.10
III. 	The Risk of Unintended Consequences from Incomplete Performance Metrics
Performance metrics are an important part of any incentive system developed to address
principal-agent problems. Principal-agent problems typically arise whenever one individual –
the principal – contracts with another – the agent – to perform tasks that the former otherwise
would have had to perform personally. The basic cause of the principal-agent problem is that it
is impossible or costly to observe the effort made by an agent.11 In order to reduce principal-
agent problems, principals typically develop both monitoring systems to keep track of agents’
7 Conference materials available at
http://www.ftc.gov/bcp/workshops/carbonoffsets/index.shtml. Other programs have included the
FTC’s public workshop on Market Power and Consumer Protection Issues Involved with
Encouraging Competition in the U.S. Electric Industry, held on September 13-14, 1999
(workshop materials available at http://www/ftc.gov/bcp/elecworks/index.shtm); and the
Department of Justice and FTC workshop on Electricity Policy, held on April 23, 1996.
8 This comment is available at http://www.ftc.gov/os/2009/12/V100002ferc.pdf.
9 This comment is available at http://www.ftc.gov/os/2009/12/V100001ferc.pdf.
10 See, e.g., Federal Trade Commission, Comment before the Federal Energy Regulatory
Commission on Wholesale Competition in Regions with Organized Electric Markets (Apr. 17,
2008), available at http://www.ftc.gov/be/v070014b.pdf. A listing of FTC and FTC staff
competition advocacy comments to federal and state regulatory agencies (in reverse
chronological order) is available at http://www.ftc.gov/opp/advocacy_date.shtm.
11 DENNIS CARLTON AND J EFFREY PERLOFF , M ODERN I NDUSTRIAL ORGANIZATION 414-15 (4th ed.
2005).

4
work efforts and incentive systems based on performance metrics.12 Performance metrics
attempt to align the interests of agents with those of principals.
FERC articulated objectives for RTOs in the orders establishing those organizations.13
FERC Order No. 2000 established “minimum characteristics and functions” that an RTO must
satisfy in the following areas:
Minimum Characteristics:
1. Independence
2. Scope and Regional Configuration
3. Operational Authority
4. Short-term Reliability
Minimum Functions:
1. Tariff Administration and Design
2. Congestion Management
3. Parallel Path Flow
4. Ancillary Services
5. Open Access Same-Time Information System, Total Transmission Capability, and Available
Transmission Capability
6. Market Monitoring
7. Planning and Expansion
8. Interregional Coordination
12 An early framework for the economic analysis of these issues was developed by Oliver
Williamson in M ARKETS AND HIERARCHIES : ANALYSIS AND ANTITRUST I MPLICATIONS (1975) and
THE ECONOMIC I NSTITUTIONS OF CAPITALISM: FIRMS , M ARKETS , RELATIONAL CONTRACTING
(1985).
13 “Independent System Operators grew out of Orders Nos. 888/889 where the Commission
suggested the concept of an Independent System Operator as one way for existing tight power
pools to satisfy the requirement of providing non-discriminatory access to transmission.
Subsequently, in Order No. 2000, the Commission encouraged the voluntary formation of
Regional Transmission Organizations to administer the transmission grid on a regional basis
throughout North America (including Canada). Order No. 2000 delineated twelve characteristics
and functions that an entity must satisfy in order to become a Regional Transmission
Organization.” FERC’s description of the origin of ISOs and RTOs is available at
http://www.ferc.gov/industries/electric/indus-act/rto.asp. The final version of Order No. 2000
(after rehearing) is available at http://www.ferc.gov/legal/maj-ord-reg/land-docs/2000A.pdf.

5
In order to avoid (at a minimum) the worst problems that can arise from using incomplete
performance metrics, there should be metrics concerning these minimum characteristics and
functions, with the relevance of each metric demonstrated.
Each RTO has an independent board of directors responsible for ensuring that the RTO
has the characteristics and functions prescribed in Order No. 2000 and related orders. A
stakeholder process within each RTO connects grid users to the board of directors.14 There are
debates about how much weight RTOs should accord to the views of customer and other
stakeholder organizations. Some observers argue that RTO performance metrics would benefit
from a greater consideration of feedback from market participants (including grid users)
alongside the quantitative performance criteria. With the exception of transmission owners who
can transfer control of their facilities from one RTO to another, RTO customers have no choice
regarding their assigned RTO. Accordingly, some assessment of how well RTOs respond to
customer needs seems necessary.15
If FERC decides to use quantitative performance metrics to evaluate RTOs, it should
ensure that the selected performance metrics effectively gauge all of the important characteristics
and functions of RTOs. FERC should consider the relative importance that it attaches to
different metrics if an aggregate quantitative performance assessment is to be made. FERC also
should bear in mind the costs and time involved in collecting performance information.
The process of linking metrics to objectives may be challenging. Academic articles
regarding optimal incentive contracts (and on principal-agent issues generally) identify many
examples of flawed or insufficient performance metrics that led to detrimental, unintended
consequences for organizations, their suppliers, and their customers. 16 In some situations,
insufficient metrics can lead an organization to focus only on certain of its responsibilities, but to
neglect others that are measured poorly or not at all.17 In other situations, the flaws can lead to
14 Issues considered in the development of the stakeholder process include what groups have a
voice in decision making, the weight given to the views of different groups, and the level of
agreement required to reach a decision.
15 The FTC disfavors “customer satisfaction” surveys because consumers in most markets learn
from their mistakes and are able to purchase their preferred products in the long run. By
contrast, such surveys may be appropriate in the RTO context, where customers generally cannot
choose their RTO.
16 See, e.g., Jonathan Levin, “Relational Incentive Contracts,” 93 Am. Econ. Rev. 835 (2003), and
citations therein (available at http://www.stanford.edu/~jdlevin/Papers/RIC.pdf).
17 George Baker, “Distortions and Risk in Optimal Incentive Contracts,” 37 J. of Human
Resources 727 (2002).

6
excellent short-term results but disastrous long-term results.18 In yet other situations, the focus
on existing performance elements can impede recognition of changes in the organization’s
environment and can hinder optimal adjustments to such changes.19 Some commentators suggest
that inefficiencies in energy markets are at least partially attributable to dubious performance
measurements that blame executives for elements of corporate performance beyond their
control.20 Others suggest that combinations of objective and subjective metrics can mitigate
some incentive distortions caused by imperfect objective metrics.21 We encourage FERC to
examine the proposed performance metrics for RTOs with these concerns in mind.
Finally, we also recommend that if FERC orders RTOs to develop and file performance
metrics, FERC should include in its order a mechanism or schedule for periodically reviewing
the metrics to determine whether more effective and up-to-date metrics should be adopted.
IV. 	Efficient Operations: A Potential Additional Minimum Characteristic of RTOs
As the FTC Bureau of Economics stated in its comment on proposals that culminated in
FERC Order No. 2000, “FERC may wish to establish an additional minimum characteristic
concerning efficient operations of RTOs. With any new independent institution, there is a risk
that independence will devolve into indifference to the quality of service, the pace of innovation,
and changes in customer preferences. RTOs are unlikely to be an exception. To avoid traveling
down such a path, FERC may wish to identify minimum efficiency incentives that will
18 A focus on achieving short-run profitability by reducing quality, for example, can cause a
decline in future demand and expose a firm to litigation from disappointed or endangered
customers.
19 For instance, research indicates that antiquated performance measures contributed to the
myopic management of railroads prior to deregulation. “Detailed regulatory controls have also
constrained railroad management from adapting to changing markets, and, over time, have dulled
initiative and the incentive to innovate within the industry.” T ASK FORCE ON RAILROAD
PRODUCTIVITY , I MPROVING RAILROAD PRODUCTIVITY: REPORT TO THE NATIONAL COMMISSION
ON PRODUCTIVITY AND THE COUNCIL OF ECONOMIC ADVISERS, Ch. VI (1973).
20 Severin Borenstein, Meghan Busse, and Ryan Kellogg, “Principal-agent Incentives, Excess
Caution, and Market Inefficiency: Evidence from Utility Regulation,” Univ. of Cal. Berkeley,
Coleman Fung Risk Mgmt. Research Ctr. (2007), available at
http://escholarship.org/uc/item/3q38g86b.
21 George Baker, Robert Gibbons, and Kevin Murphy, “Subjective Performance Measures in
Optimal Incentive Contracts,” 109 Q.J. Econ. 1125 (1994).

7
characterize RTOs.” 22 The adoption of an RTO minimum characteristic that encompasses
operational efficiency would form a basis for appropriate performance metrics. Although some
of the metrics proposed in the Notice – such as organizational effectiveness – seem to be related
to the efficiency characteristic that we propose, they do not appear to be connected to any of the
currently listed RTO characteristics and functions.
In addition, FERC may wish to consider an RTO’s contribution to the effective
functioning of the broader power system as a grid efficiency performance metric. This metric
could include, for example, economic dispatch efficiency, cooperation with Interconnection-wide
planning efforts, and constructive work to resolve “seams” issues.23 FERC may have an
especially constructive role to play in ensuring that an RTO’s operations do not benefit the
RTO’s stakeholders through operational choices that harm those who are unrepresented or
underrepresented in the RTO’s stakeholder process.
Thus, FERC might consider two efficiency metrics – one concerning grid efficiency, and
the other regarding internal operating efficiency. The goal of the internal operating efficiency
metrics should be to hold RTOs accountable for making cost-effective use of resources such as
staff and software to run the grid efficiently (rather than simply to minimize the cost of operating
the RTO itself). It would be myopic and counterproductive to use efficiency metrics that
criticize RTOs for optional internal spending that yields net benefits for the grid (such as
improvements in dispatch or reductions in market power).
V. 	Measuring Organizational Performance Versus Measuring Policy Performance
The initial comment by the “Consumer Commenters” group proposes that FERC use
“generator costs compared to revenues” as a performance metric for RTOs.24 This proposal
confuses policy performance with organizational performance metrics. FERC’s use of a “single
market-clearing price” approach for compensating generators – rather than a “pay-as-bid”
22 Comment of the Staff of the Bureau of Economics, Federal Trade Commission, on Regional
Transmission Organizations, at 27 (Aug. 16, 1999), available at
http://www.ftc.gov/be/v990011.pdf.
23 Seams issues arise when multiple RTOs are involved in providing a transmission service.
Examples include (1) transmission services in which a supplier is located in one RTO and the
customer is located in another and (2) transmission planning for areas larger than a single RTO.
24 Initial Comments of AARP et al., RTO/ISO Performance Metrics, at 5 (Mar. 5, 2010) (italics
deleted), available at http://elibrary.ferc.gov/idmws/File_list.asp?document_id=13798736.

8
approach combined with plant-specific price controls25 – reflects an energy policy decision. The
results of that policy decision are not under the control of RTOs. Thus, using “generator costs
compared to revenues” as a performance metric for RTOs would hold those organizations
accountable for results that reflect FERC’s policy choices. For this reason, we recommend that
FERC not adopt “generator costs compared to revenues” as a performance metric for RTOs.
An accurate assessment of generator market power, however, could be a useful policy
performance metric regarding FERC’s policy choices. Generators’ costs and revenues – the
metrics identified by the Consumer Commenters – might well be important data to use in
conjunction with appropriate economic theories of market power to create a policy performance
metric for market power and the ability of market forces to discipline prices in FERC-regulated
electricity markets.
25 See Alfred E. Kahn et al., Blue Ribbon Panel Report, Pricing in the California Power
Exchange Electricity Market: Should California Switch from Uniform Pricing to Pay-as-Bid
Pricing? (2001), available at http://www.cramton.umd.edu/papers2000-2004/kahn-cramton-
porter-tabors-blue-ribbon-panel-report-to-calpx.pdf.