Statistically Significant: Equity Concerns in Algorithmic Bias, Privacy, and Survey Representation

The U.S. population grew by 16.0% from 281.4 to 326.6 million between 2000 and 2020. During those 20 years, the Asian American (AsA) population had remarkable growth, nearly doubling from 11.9 to 22.2 million. Minorities are often associated with negative outcomes, creating a dimension of disparities in important areas of human life. While data are essential to understand the conditions of life and the disparities that minority groups, such as AsA, are subject to, adequacy of existing data for AsA is unclear. With the Covid-19 pandemic, hate crimes against AsA have surged. In 16 major cities and counties in the U.S., the police reports of hate crimes against AsA increased by 146% between 2019 and 2020. At the same, there is little data available for illustrating dynamics behind such critical issues that the AsA community faces. By leveraging on data from the decennial Census, the American Community Survey and a number of publicly available large-scale sample surveys, this study examines AsA in sample surveys against the AsA population. The focus will be given to the socio-demographics as well as health care access which have important implications for the disparities. 

Researchers and organizations can increase privacy in datasets through methods such as aggregating, suppressing, or substituting random values. But these means of protecting individuals' information do not always equally affect the groups of people represented in the data. A published dataset might ensure the privacy of people who make up the majority of the dataset but fail to ensure the privacy of those in smaller groups. Or, after undergoing alterations, the data may be more useful for learning about some groups more than others.

In this talk, I will introduce a guide that contains a literature review of equity-focused work in statistical data privacy (SDP) and interviews with nine experts on privacy-preserving methods and data sharing. These experts include researchers and practitioners from academia, government, and industry sectors with diverse technical backgrounds, where we sought to understand both how and to what extent they consider the questions of equity in their work. We also created an illustrative example to highlight potential disparities that can result from applying SDP methods without an equitable workflow. 

Automated decision systems are used for decision-making in societally high-stakes settings like healthcare and criminal justice, raising concerns around the suitability and equity of these systems. The discourse on responsible use largely focuses on fairness and ethics, often overlooking first-order questions of validity. In this talk, we explore the important role validity plays in responsible use and consider its implications for fairness.

Drawing on validity theory from the social sciences, we develop a taxonomy of challenges that threaten validity in algorithmic decision-making contexts. We delve into a couple common challenges — selection bias and missing data — in two domains, consumer credit lending and child welfare screening. We illustrate how failure to properly address these issues can invalidate standard fairness assessments and undermine fairness interventions. We present an alternative method for conducting fairness assessments and corrective interventions that addresses common forms of selection bias and missing data using techniques from causal inference. We conclude by considering the broader question of governance of high-stakes decision-making algorithms.