Column Editor’s Note: In this Effective Policy Learning piece, Jonathan Auerbach, Assistant Professor in the Department of Statistics at George Mason University, presents the argument that independent monitoring of the U.S. statistical system would help combat politicization of official U.S. government statistics.  He describes a current research effort led by the American Statistical Association and George Mason University to devise a set of measurements by which the health of federal statistical agencies might be measured. The goal of that effort is to produce an annual report by which the several dimensions that constitute the "health" of an agency could be tracked over time. The author describes some politicization issues that have arisen; looks at prior suggestions for maintaining an accurate and objective statistical system  that have only been partially implemented; and discusses how the current research effort might address existing gaps. Auerbach invites readers to help build a community consensus around the monitoring effort by citing several outreach activities the research team is conducting such as surveying data producers and users, convening a series of meetings and workshops, and soliciting academic papers and discussions.

Keywords: official statistics, federal statistics, government statistics, public policy and statistics, trust, independence and autonomy

Safeguarding Facts in an Era of Disinformation

For generations, Americans have invested in their federal statistical system to produce a common set of facts that the federal, state, and local governments—as well as many members of the public—use to make informed decisions. The system is with limited exceptions free of paywalls or profit incentives. A guiding principle has been that society benefits when all Americans have access to accurate and timely information—a principle arguably more relevant today, in the current era of disinformation, than in generations past.

But while significant resources are dedicated to study the health of America’s physical infrastructure, there is relatively little effort to evaluate America’s statistical infrastructure, which is comparably intricate and no less essential.

That is a real problem since a growing number of people in the United States seek to undermine the credibility of government agencies and manipulate publicly available information to further their own agendas. Examples include politicians of both parties who wish unemployment was lower and GDP was higher; special interests that wish to operate without public scrutiny; and media stars with large, public followings who wish to profit from disinformation.

These bad actors do not actually have to change the statistics themselves to achieve their goal. They just have to manufacture doubt—doubt in the accuracy of the data, doubt in the value of making them accessible, or doubt in the value of collecting them in the first place (Groshen, 2021, Section 4.4; Hogan, 2020; Kafadar, 2020; Sullivan, 2020, Section 5). And while attempts to interfere with federal statistics are as old as the system itself, the incentives to do so have never been higher. Increasingly brazen attempts demonstrate that existing safeguards are insufficient.

In this article, we make a case for independently monitoring the U.S. statistical system. We first summarize a new effort led by researchers at the American Statistical Association and George Mason University. We then justify the effort in three parts: 1. we describe the increasing politicization of federal statistics; 2. we review the visions of the experts who have examined the U.S. statistical system over the past century, visions that at best have only been partially implemented; and 3. we discuss how independent monitoring would more fully realize those visions. We conclude that by leveraging the growing volume of publicly available administrative data, independent monitoring can raise the visibility of challenges facing the federal statistical system—not just political interference but other threats such as chronic underfunding—and better safeguard the facts on which modern society depends.

A New Effort to Independently Monitor the Federal Statistical System

Researchers at the American Statistical Association and George Mason University are leading a new effort to independently monitor the health of the U.S. statistical system. The researchers aim to modernize the centuries-old role of the American Statistical Association, from reacting to threats facing the federal statistical system to proactively tracking government operations and assessing progress toward achieving predetermined goals. (For examples of other efforts to monitor government operations, see the USAFacts State of the Union 10K Data Snapshot and the Climate Action Tracker.)

The researchers will monitor the extent to which the U.S. statistical system adheres to long-established standards detailing how governments should collect and disseminate data—such as the U.S. Statistical Policy Directives, the National Academies Principles and Practices for a Federal Statistical Agency, the United Nations Fundamental Principles of Official Statistics, and the Organisation for Economic Co-operation and Development’s (OECD) Recommendation of the Council on Good Statistical Practice. These standards include the need to produce relevant and timely information, maintain professional autonomy, protect the privacy of data providers, and foster innovation (see Auerbach et al., 2023, for details).

The challenge is that many of these standards are abstract guidelines, and adherence is difficult to measure directly. A successful monitoring effort will require a community consensus on the evaluation criteria that most suitably instantiate adherence. The researchers plan to attain community consensus by surveying data producers and users, convening a series of meetings and workshops, and soliciting academic papers and discussions. The purpose of this article is to facilitate consensus by providing a more detailed examination of the current situation, beginning with a description of the growing politicization of U.S. federal statistics. We first note that despite our focus on U.S. federal statistics, the manipulation of official statistics is a worldwide problem, and thus the criteria we seek are relevant to statistical systems worldwide.

Politicization of U.S. Federal Statistics Is on the Rise Despite Decades of Reform

In many countries, professional statisticians grapple with regimes that seek to tamper with statistical production to advantage their policies and leaders. Statisticians that resist have been punished and prosecuted. Examples include when Argentina’s secretary of internal trade directed professional staff at its statistics authority to underreport the official inflation rate following the termination of key personnel (“The Price of Cooking the Books,” 2012), and when Greek prosecutors sued the former president of their statistics authority, Andreas Georgiou, after he refused to underreport official statistics of Greece’s deficit (Tarran, 2023). See Georgiou (2021) for additional examples and discussion.

Historically, the world has viewed the United States as a leader in protecting the integrity of government statistics. The United States played a significant role in international bodies such as the United Nations Statistical Commission and the OECD, developing international standards for official statistics. It also helped develop the statistical systems of other countries (U.S. Department of Commerce, 1978, Chapter 25).

Yet the politicization of federal statistics dates back to the beginning of the United States, when the founders debated whom to count in the decennial census (Anderson, 2020; Anderson & Fienberg, 1999, Chapter 2). Even after federal statistics were professionalized in the early 20th century, political forces have at times interfered with every part of the system. Data collection operations in opposition to the interests of the presiding administration have been curtailed or canceled. Statistical reports have been delayed or withheld, and communications with the press have been blocked. Directors and commissioners have been forced out, and professional staff reassigned. Unqualified political operatives have been placed in professional career-level positions, and political clearance has been required for advisory committee members (Citro et al., 2023; Hauser, 1973; Malkiel, 1978).

A watershed moment occurred in 1971, when professional staff in the U.S. Bureau of Labor Statistics gave an interpretation of the unemployment rate that contradicted political officials overseeing the U.S. Department of Labor. U.S. President Richard Nixon and members of his administration subsequently canceled press conferences held by the bureau, interfered in data collection operations by temporarily discontinuing the Urban Employment Survey, and reorganized staff—resulting in a large number of early retirements (Duncan & Shelton, 1978, p. 167; Norwood, 2016).

Government reforms followed—most notably, in 1971, the U.S. Office of Management and Budget, part of the Executive Office of the President of the United States, issued Statistical Policy Directive 3, which requires the orderly release of key economic indicators and precludes political officials from commenting until an hour after the statistical agency release (see Statistical Policy Directive No. 3, 2019). A year later, the Committee on National Statistics was created at the National Academy of Sciences, and in 1980, the Office of the Chief Statistician was reestablished in the Office of Management and Budget.

But attempts to interfere with the federal statistical system have continued. For example, the George W. Bush administration removed the director of the U.S. Bureau of Justice Statistics after he refused to downplay data on racial profiling at traffic stops (Lichtblau, 2005); the Obama administration rewarded managers of U.S. Department of Veterans Affairs medical centers and clinics known to be falsifying data on patient wait times in order to meet unrealistic policy goals (Brunker, 2014; Hicks, 2014); the Trump administration pressured U.S. Census Bureau employees to manipulate the decennial census for political gain (Percival, 2021; Wines, 2022); and the Trump administration abruptly relocated the U.S. Department of Agriculture Economic Research Service from Washington, DC, to Kansas City, Missouri—a move of over 1,000 miles (1,610 km)—resulting in more than half the employees quitting (Morris, 2021; Wagner, 2023).

While official statistics inevitably reflect the political and social values of their creators (Fienberg, 1989; Prewitt, 1987), these U.S. examples are not genuine policy disagreements over how to best inform the public. In each case, public information was intentionally targeted for immediate political gain.

Reforms Have Only Partially Implemented Recommendations by Experts for a Highly Visible and Comprehensively Audited Federal Statistical System

Many experts have examined the federal statistical system over the past century—Milton Friedman, George Stigler, Frederick Mosteller, John Tukey, William Kruskal, and Stephen Fienberg to name a few—and a variety of recommendations have been made to protect the integrity of federal statistics. These recommendations fall primarily into two categories: 1. comprehensive audits and 2. high visibility.

Yet these recommendations have at best only been partially implemented. For example, the original vision of the President’s Commission on Federal Statistics—constituted by the Nixon administration in 1970 before it retaliated against the Bureau of Labor Statistics (although the commission did not submit its findings until after)—was regular audits of the federal statistical system overseen by a standing committee of the National Academy of Sciences (President's Commission on Federal Statistics, 1971; Rathbun, 1972). While the Committee on National Statistics—established following this recommendation—does review the federal statistical agencies and has made consequential recommendations (Eddy et al., 2010; Fienberg et al., 1995), it does not oversee comprehensive audits as envisioned by the commission. See the comments by Moore (1977) and Office of Management and Budget (1977).

For another example, the original vision of the President's Reorganization Project for the Federal Statistical System—constituted by the Carter administration in 1978—was a separate, highly visible Office of Statistical Policy in the Executive Office of the President, arguing that when the office had been buried within the Office of Management and Budget, the integrity of the federal statistical system was second to the short-term, crisis-driven decisions that dominate budget management (Bonnen, 1996; Bonnen et al., 1981; see also Fienberg, 1983, Section 4; Kaysen et al., 1969, Section 4; Mills & Long, 1949, Recommendation 13). Congress disagreed, however, and coordination functions were returned to the Office of Management and Budget, where the Office of the Chief Statistician currently resides in a subdivision, the Office of Information and Regulatory Affairs. See the comments by Norwood (1995, Chapter 5) and “Infrastructure Series” (2022).

The result of these and other partially implemented reforms is that the institutions primarily charged with protecting the integrity of the federal statistical system operate with one arm tied behind their backs. Both the Committee on National Statistics and the Office of the Chief Statistician publish standards for official statistics: Principles and Practices for a Federal Statistical Agency and the Statistical Policy Directives, respectively. But they lack a mandate to regularly conduct comprehensive audits (for example, the committee requires contracts or grants to carry out its projects) or publicly report on the extent to which those standards are met in practice, and they lack the ability to push back effectively on violations. See Norwood (1995, Chapter 2), Citro (2001), Citro (2020), and National Research Council (2021, Appendix A) for a more detailed discussion of these and related reforms.

Independently Monitoring the Federal Statistical System Would Build on the Recommendations of Experts

An independent monitoring effort would help bridge the gap between expert recommendations and current practice. Independent review is not unprecedented. Reviewers from both inside and outside the U.S. government have at times investigated the integrity of federal statistics. See, for example, the Committee on Government Statistics, and Information Services (1937), American Statistical Association-Federal Statistics Users' Conference Committee on the Integrity of Federal Statistics (1973), and the U.S. General Accounting Office (1995).

Independent monitoring was in fact suggested by the Joint Ad Hoc Committee on Government Statistics (1976, Recommendation 6). Since that report, a growing volume of data on the federal statistical system has become publicly available—most recently government reporting mandated under the Foundations for Evidence-Based Policymaking Act of 2018 (2018; Potok, 2019). Data such as these are already used to monitor the capacity of national statistical systems to support policy goals such as gender equality and climate action (see, e.g., Open Data Watch, Global Data Barometer, and Open Data Barometer). They now offer an unprecedented opportunity to report on the health of the U.S. statistical system itself in a way that more closely achieves a century of expert recommendations for 1. comprehensive audits and 2. high visibility.

That is the rationale behind the new effort to independently monitor the health of the U.S. statistical system. For more than a century, the American Statistical Association has raised awareness of the challenges facing the federal statistical system, evidenced by the numerous works authored by the association and its members and referenced in this article—as well as the many presidential addresses that have brought attention to the system’s organizational structure (Mitchell, 1919), personnel (Bowman, 1964), funding (Anderson, 1984), public perception (Gainsbrugh, 1962; Wallman, 1993), and use of data as evidence (Morton, 2010; Norwood, 1990). (See Martinez, 2021, for a detailed analysis of presidential addresses.) Regular monitoring along with contextual information from experts would modernize this historic role of the association, raise the visibility of challenges facing the federal statistical system—not just from political interference but from other threats such as chronic underfunding—and better safeguard the facts on which modern society depends.

Acknowledgments

The authors thank Nancy Potok, Connie Citro, Steve Pierson, Claire Bowen, May Aydin, Joe Salvo, and two anonymous reviewers for extensive conversations and substantial feedback on the manuscript and independent monitoring effort described in this paper.

Disclosure Statement

The authors thank the Alfred P. Sloan Foundation for their financial support.

References

American Statistical Association-Federal Statistics Users' Conference Committee on the Integrity of Federal Statistics. (1973). Maintaining the professional integrity of federal statistics. The American Statistician, 27(2), 58–67. https://doi.org/10.1080/00031305.1973.10478984

Anderson, M. J. (2020). Historical lessons: How statistics underpin our democracy. Harvard Data Science Review, 2(1). https://doi.org/10.1162/99608f92.490ac65e

Anderson, M., & Fienberg, S. E. (1999). Who counts? The politics of census-taking in contemporary America. Russell Sage Foundation. https://www.russellsage.org/publications/who-counts-1

Anderson, R. L. (1984). Goals: Where are we and where should we be going? Journal of the American Statistical Association, 79(386), 253–258. https://doi.org/10.1080/01621459.1984.10478037

Auerbach, J., Bowen, C. M., Citro, C., Pierson, S., & Potok, N. (2023, April 1). ASA, GMU team on project to assess health of federal statistical agencies. Amstat News. https://magazine.amstat.org/blog/2023/04/01/asa-gmu-fed-stats/

Bonnen, J. T. (1996). The politics of statistical reform: A cautionary tale, 1978–1980. CHANCE, 9(1), 17–26. https://doi.org/10.1080/09332480.1996.10542482

Bonnen, J. T., Clemence, T. G., Fellegi, I. P., Jabine, T. B., Kutscher, R. E., Roberson, L. K., & Waite, C. A. (1981). Improving the federal statistical system: Report of the President's Reorganization Project for the Federal Statistical System. The American Statistician, 35(4), 184–196. https://doi.org/10.1080/00031305.1981.10479352

Bowman, R. T. (1964). The American Statistical Association and federal statistics: Presidential address. Journal of the American Statistical Association, 59(305), 1–17. https://doi.org/10.1080/01621459.1964.10480698

Brunker, M. (2014, June 25). Performance mismanagement: How an unrealistic goal fueled VA scandal. NBC News. https://www.nbcnews.com/storyline/va-hospital-scandal/performance-mismanagement-how-unrealistic-goal-fueled-va-scandal-n139906

Citro, C. F. (2001). Window on Washington: The credibility of federal statistics: Don't take it for granted. CHANCE, 14(2), 39–42. https://doi.org/10.1080/09332480.2001.10542268

Citro, C. F. (2020). Are we up to the challenges of protecting federal statistics? Harvard Data Science Review, 2(1). https://doi.org/10.1162/99608f92.1a3cd97f

Citro, C. F., Auerbach, J., Evans, K. S., Groshen, E. L., Landefeld, J. S., Mulrow, J., Petska, T., Pierson S., Potok N., Rothwell C. J., Thompson J. L., Woodworth J. L., & Wu, E. (2023). What protects the autonomy of the federal statistical agencies? An assessment of the procedures in place to protect the independence and objectivity of official US statistics. Statistics and Public Policy. Advance online publication. https://doi.org/10.1080/2330443X.2023.2188062

Committee on Government Statistics and Information Services. (1937). Government statistics: A report of the Committee on Government Statistics and Information Services. American Statistical Association and Social Science Research Council. https://catalog.hathitrust.org/Record/001677909

Duncan, J. W., & Shelton, W. C. (1978). Revolution in United States government statistics, 1926–1976. U.S. Department of Commerce, Office of Federal Statistical Policy and Standards. https://catalog.hathitrust.org/Record/000257885

Eddy, W. F., Citro, C. F., & Cork, D. L. (2010). Principles and practices for the federal statistical system: The view from the Committee on National Statistics. The Annals of the American Academy of Political and Social Science, 631(1), 210–224. https://doi.org/10.1177/0002716210373864

Fienberg, S. E. (1983). Comment [On the article “Federal statistical coordination today: A disaster or a disgrace?”]. The American Statistician, 37(3), 192–196. https://doi.org/10.1080/00031305.1983.10483099

Fienberg, S. E. (1989). Political pressure and statistical quality: An American perspective on producing relevant national data. Journal of Official Statistics, 5(3), 207–221. https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/political-pressure-and-statistical-quality-an-american-perspective-on-producing-relevant-national-data.pdf

Fienberg, S. E., Martin, M. E., & Straf, M. L. (1995). The Committee on National Statistics: Fostering interactions between statisticians in academia and government. International Statistical Review/Revue Internationale de Statistique, 63(3), 257–269. https://doi.org/10.2307/1403479

Foundations for Evidence-Based Policymaking Act of 2018, Public Law No: 115-435, 132 Stat. 5529 (2018). https://www.congress.gov/bill/115th-congress/house-bill/4174

Gainsbrugh, M. R. (1962). Statistics we live by: Presidential address. Journal of the American Statistical Association, 57(297), 1–9. https://doi.org/10.1080/01621459.1962.10482146

Georgiou, A. V. (2021). The manipulation of official statistics as corruption and ways of understanding it. Statistical Journal of the IAOS, 37(1), 85–105. https://doi.org/10.3233/SJI-200667

Groshen, E. L. (2021). The future of official statistics. Harvard Data Science Review, 3(4). https://doi.org/10.1162/99608f92.591917c6

Hauser, P. M. (1973). Statistics and politics. The American Statistician, 27(2), 68–71. https://doi.org/10.2307/2683959

Hicks, J. (2014, May 20). This memo shows that the VA knew of records manipulations in 2010. The Washington Post. https://www.washingtonpost.com/news/federal-eye/wp/2014/05/20/this-memo-shows-that-the-va-knew-of-records-manipulation-in-2010

Hogan, H. (2020). Distrust in the governments brings risk to the census. Harvard Data Science Review, 2(1). https://doi.org/10.1162/99608f92.11f3e977

Infrastructure Series: Challenges loom for Chief Statistician of the US. (2022, April 1). Amstat News. https://magazine.amstat.org/blog/2022/04/01/infrastructure-series-csotus/

Joint Ad Hoc Committee on Government Statistics. (1976). Improving the federal statistical system. The American Statistician, 30(4), 154–158. https://doi.org/10.2307/2683751

Kafadar, K. (2020). Statisticians’ role in ensuring accuracy and integrity of federal data. Harvard Data Science Review, 2(1). https://doi.org/10.1162/99608f92.aeb6e98b

Kaysen, C., Holt, C. C., Holton, R., Kozmetsky, G., Morrison, H. R., & Ruggles, R. (1969). Report of the Task Force on the Storage of and Access to Government Statistics. The American Statistician, 23(3), 11–19. https://doi.org/10.1080/00031305.1969.10481843

Lichtblau, E. (2005, August 24). Profiling report leads to a demotion. The New York Times. https://www.nytimes.com/2005/08/24/us/front%20page/profiling-report-leads-to-a-clash-and-a-demotion.html

Malkiel, B. G. (1978). Problems with the federal economic statistical system and some alternatives for improvement. The American Statistician, 32(3), 81–88. https://doi.org/10.1080/00031305.1978.10479262

Martinez, W. L. (2021). Back to our future: Text analytics insights. Journal of the American Statistical Association, 116(535), 1039–1047. https://doi.org/10.1080/01621459.2021.1960760

Mills, F. C., & Long, C. D. (1949). Organization of the federal statistical system. National Bureau of Economic Research. https://www.nber.org/books-and-chapters/statistical-agencies-federal-government-report-commission-organization-executive-branch-government

Mitchell, W. C. (1919). Statistics and government. Quarterly Publications of the American Statistical Association, 16(125), 223–235. https://doi.org/10.1080/15225445.1919.10503752

Moore, G. H. (1977). A continuing audit of government economic statistics. Challenge, 20(4), 29–33. https://doi.org/10.1080/05775132.1977.11470345

Morris, F. (2021, February 2). USDA research agencies 'decimated' by forced move. Undoing the damage won't be easy. NPR. https://www.npr.org/2021/02/02/963207129/usda-research-agencies-decimated-by-forced-move-undoing-the-damage-wont-be-easy

Morton, S. C. (2010). Statistics: From evidence to policy. Journal of the American Statistical Association, 105(489), 1–5. https://doi.org/10.1198/jasa.2010.ap09811

National Research Council. (2021). Principles and practices for a federal statistical agency (7th ed.). National Academies. https://www.nationalacademies.org/our-work/7th-edition-of-principles-and-practices-for-a-federal-statistical-agency

Norwood, J. L. (1990). Statistics and public policy: Reflections of a changing world. Journal of the American Statistical Association, 85(409), 1–5. https://doi.org/10.1080/01621459.1990.10475300

Norwood, J. L. (1995). Organizing to count: Change in the federal statistical system. The Urban Institute. https://webarchive.urban.org/publications/205979.html

Norwood, J. L. (2016). Politics and federal statistics. Statistics and Public Policy, 3(1), 1–8. https://doi.org/10.1080/2330443X.2016.1241061

Office of Management and Budget. (1977). Perspectives on fiscal year 1978 budget. U.S. Government Printing Office. https://babel.hathitrust.org/cgi/pt?id=umn.31951p00925223v

Percival, K. (2021, January 25). Documents reveal Trump administration’s ‘unprecedented’ attempts to influence 2020 Census. Brennan Center for Justice. https://www.brennancenter.org/our-work/analysis-opinion/documents-reveal-trump-administrations-unprecedented-attempts-influence

Potok, N. (2019). Deep policy learning: Opportunities and challenges from the Evidence Act. Harvard Data Science Review, 1(2). https://doi.org/10.1162/99608f92.77e63f8f

President's Commission on Federal Statistics. (1971). Federal statistics report of the President's Commission on Federal Statistics. U.S. Government Printing Office. https://books.google.com/books?id=0YqUVJm1aYkC

Prewitt, K. (1987). Public statistics and democratic politics. In W. Alonso & P. Starr (Eds.), The Politics of Numbers (pp. 261–274). Russell Sage Foundation. https://www.russellsage.org/publications/politics-numbers

Rathbun, D. B. (1972). The report of the President's Commission on Federal Statistics. The American Statistician, 26(1), 15–18. https://doi.org/10.1080/00031305.1972.10477317

Statistical Policy Directive No. 3: Compilation, Release, and Evaluation of Principal Federal Economic Indicators. Timing of Public Comments by Employees of the Executive Branch, 84 F.R. 14682. (2019, April 11). Notice. https://www.federalregister.gov/documents/2019/04/11/2019-07172/statistical-policy-directive-no-3-compilation-release-and-evaluation-of-principal-federal-economic

Sullivan, T. A. (2020). Coming to our census: How social statistics underpin our democracy (and republic). Harvard Data Science Review, 2(1). https://doi.org/10.1162/99608f92.c871f9e0

Tarran, B. (2023, January 2). A statistician’s view: New year hopes for Andreas Georgiou as court decisions loom. Amstat News. https://magazine.amstat.org/blog/2023/01/02/a-statisticians-view-georgiou/

The price of cooking the books. (2012, February 25). The Economist. https://www.economist.com/the-americas/2012/02/25/the-price-of-cooking-the-books

U.S. Department of Commerce, Office of Federal Statistical Policy and Standards. (1978). A framework for planning U.S. federal statistics for the 1980's. U.S. Government Printing Office. https://openlibrary.org/works/OL6293299W/

U.S. General Accounting Office. (1995). Statistical agencies: Adherence to guidelines and coordination of budgets. https://www.gao.gov/products/ggd-95-65

Wagner, E. (2023, January 17). Although USDA agencies that relocated to Kansas City have recovered from staff exoduses, their diversity hasn’t. Government Executive. https://www.govexec.com/oversight/2023/01/although-usda-agencies-relocated-kansas-city-have-recovered-staff-exoduses-their-diversity-hasnt/381877/

Wallman, K. K. (1993). Enhancing statistical literacy: Enriching our society. Journal of the American Statistical Association, 88(421), 1–8. https://doi.org/10.1080/01621459.1993.10594283

Wines, M. (2022, January 15). Census memo cites ‘unprecedented’ meddling by Trump administration. The New York Times. https://www.nytimes.com/2022/01/15/us/2020-census-trump.html

©2023 Jonathan Auerbach. This article is licensed under a Creative Commons Attribution (CC BY 4.0) International license, except where otherwise indicated with respect to particular material included in the article.