The CBO and Scoring Impacts on Legislation
by Dan Perrin
The Epic Errors of the Irreformable Congressional Budget Office and Speaker Pelosi’s Political Protection of the Current Director
The letter below that was sent to the Senate Chairman of the Budget Committee describes the almost incomprehensibly incompetent errors the Congressional Budget Office (CBO) made in scoring Republican health care reform plans and describes a laundry list of desperately needed reforms at CBO.
It seems clear that CBO (and also the Joint Committee on Taxation) will be replaced by some form of Artificial Intelligence that uses actual data and updates their cost estimates in real-time as data is available. In any other environment, the team that made such repeated and public errors of such magnitude would have been fired instantly.
I expect none of the reforms listed below, to be carried out by the current director.
___
February 15, 2019
The Honorable Mike Enzi
Chairman
Committee on the Budget
United States Senate
Washington, D.C. 20510
Dear Mr. Chairman:
There is a crying need for reform and modernization of the Congressional Budget Office (CBO). We are writing you with specific reform suggestions, as well as to urge you to select a completely different type of Director than has been selected in the past, one who has a track record of successfully reforming institutions and who will impose significant and greatly needed reforms on CBO.
Regardless of whether you are a supporter of liberal or conservative policies, having inaccurate data helps no one. If CBO ignores what is actually occurring in the real world, while insisting that their inaccurate projections cannot be changed, then relying on CBO undermines Congress's ability to accomplish its policy objectives.
For example, the CBO’s exchange enrollment projections have been proven highly inaccurate. They consistently and repeatedly overestimate the number of insured by millions. What is most troubling about this is that while actual events were proving them wrong, CBO stubbornly clung to the very models and assumptions that were producing these overestimates.
Graph by Doug Badger, Galen Institute
In a feat never accomplished in the history of scoring, during 2017 health reform debates the CBO scored a provision that spent an additional fifty billion dollars to cover the uninsured as having zero impact on coverage.
Of course, CBO inaccuracy is not limited to health projections. As you are likely aware, the St. Louis Federal Reserve Bank did a study of CBO’s accuracy in projecting debt and deficits. What they found should be alarming to us all. Even at these top levels, CBO estimates experienced approximately a 2% variance each year, and proved no more accurate than a random-walk model.
Yet despite this failed track record, it seems that CBO itself has made little to no effort to address and correct its problems. In fact, much of the scorekeeping rules and procedures CBO still relies on were developed in the late 1960s (prior to CBO’s creation) by Representative George H. Mahon, who was both chairman of the Joint Committee on Reduction of Non-Essential Federal Expenditures and the House Appropriations Committee. While these rules may have been revolutionary in the ‘60s, they are desperately in need of updating. Therefore, we urge you to consider the following reforms as you select the next CBO Director:
A Commitment to and a History of Accurate Financial Projections: A CBO Director who has a career of professional projections that have been accurate and is committed to providing the most accurate projections and a willingness to change them when clearly wrong.
Retroactive Scoring: How about a score based on actual data? CBO should institute a policy of using actual data to compare against their projections and modify their projections regularly to accurately reflect reality. Moreover, the new CBO director should be willing to annually go before the House and Senate Budget Committees to publicly review and discuss scoring accuracy.
Alternate Scores: Develop an independent group of scorers to take a wholly different approach to scoring the most significant legislation, and use an alternative score on the Senate floor to set a precedent. This “Team B” approach worked particularly well during the Cold War with Russia and corrected some significant errors.
Bring in Artificial Intelligence: At the very least there needs to be a hearing on how artificial intelligence could replace or supplement the current CBO scoring process and there should be an effort to create a test bed for new methodologies like using A.I. to produce more accurate scores.
Make the ‘Black Box’ Transparent: CBO should be transparent about its models and assumptions. CBO estimates are black boxes. The public, even members of Congress, do not know very much about the assumptions that lead to CBO’s conclusions. The CBO model should be public domain, and CBO should publish the source code and data files used, as Executive Branch agencies often do when they produce estimates. This would enable academics, think tanks, and the general public to discover errors, dispute assumptions, and run estimates based on alternative assumptions.
Use Scores Other than from CBO: The monopoly on scoring from the CBO must end. Before CBO was established in 1974, Congress did not outsource its responsibility to predict the likely effects of legislative proposals, nor did it grant to any body of unelected experts a “monopoly on the truth.” Instead, it took into account thoughtful estimates and predictions from a variety of sources, including professional committee staff, executive-branch agencies, and private-sector experts, reserving final judgment to itself, based on the weight of available evidence.
As Chairman, you have standing authority to disregard CBO scores, on a case-by-case basis, in favor of other, more accurate projections, such as those by the sophisticated Penn Wharton Budget Model or the Tax Foundation or others. The Budget Chairman’s power to set aside CBO projections, in individual cases, helps to ensure greater accountability and accuracy in the policy-making process. We urge you to make thoughtful use of this power in the public interest.
Do Not Pick an Academic to be CBO Director: Today, virtually every industry utilizes some level of quantitative analysis and projection. Such analysis drives decision making, which means inaccurate projections can cause not only lost revenue, but failed businesses. With the stakes that high, those running analysis divisions know they must be accurate and when they are not, they know changes in models and assumptions must be made. Thus, Congress would be better served by a CBO director who comes from a background where corrections to models are made to reflect actual events, and who has a track record of producing accurate analyses.
Create a Timely Scoring Process: CBO scores have been a regular and repeated impediment to timely consideration of legislation. Time limits for scores, as well as accuracy for scores, need to be imposed on CBO, by reforms and modernization — not by adding staff and budget to an antiquated institution.
We, the undersigned, urge you, Mr. Chairman, to consider these criteria for selecting the next CBO Director because a business-as-usual approach to filling this crucial position will result in business-as-usual at the CBO — and we must do better.
Sincerely,
Phil Kerpen, American Commitment
Dan Perrin, HSA Coalition
Dean Clancy, HSAs for All
Grover Norquist, Americans for Tax Reform
Norm Singleton, Campaign for Liberty
Ryan Ellis, Center for a Free Economy
Jim Martin, 60 Plus Association
George C. Landrith, Frontiers of Freedom
Dick Patten, American Business Defense Council
David Keene, Washington Times
Grace Marie Turner, Galen Institute
5 http://budgetmodel.wharton.upenn.edu/
6 https://taxfoundation.org
Adam Brandon, Freedom Works
Baker Spring, Compact for America Educational Foundation
David Wallace, FAIR Energy Foundation
Dean Riesen, Arizona Free Enterprise Club
Beverly Gossage, HSA Benefits Consulting
Mario H. Lopez, Hispanic Leadership Fund
Bob Carlstrom, The Carlstrom Group
Jim Backlin, Christian Coalition
Doug Badger, Galen Institute
Saulius “Saul” Anuzis, 60 Plus Association
Twila Brase, Citizens’ Council for Health Freedom