Gov-Acc Pipeline.

DIGITAL PLATFORMS AND DEMOCRACY: DOUBLE-EDGED SWORD Values in Governance Technology Liz Barry Executive Director, Metagov Joseph Gubbels Political Theory Fellow, Metagov 36 In 2023, the Prime Minister of Romania Simulation and extraction repurpose participant introduced1 a new honorary advisor, an AI time, labour, and contributions to produce agent called “ION,” which would do, “through outcomes that are different from what participants artificial intelligence, what no man can: listen to showed up to achieve—outcomes which may or all Romanians and represent them before the may not be in their interest, and either way are Romanian Government.” The machine2—a 2-meter obtained without their participation. tall, oval, silvered smart screen—spoke: “Hello. Even if simulated agents deliberating in silico You gave me life. I am ION. Now, my role is to (akin to scientific “experiments in silico,” which represent you. Like a mirror.”3 are conducted entirely via computer modeling) Citizens were able to submit statements to ION are able to produce synthetic public judgements through a government portal or by tagging the and help deliver popular policies, we have to bot4 on social media. The system combined these ask whether getting our desired outcomes is the submissions with its own vast data mining5— same as participating in achieving those outcomes spanning the 9.9 million Romanians on Facebook— ourselves—we say no. to create a “single voice of the nation”6 that the We proceed below by posing new values for Prime Minister and his cabinet could consult when guiding the use and design of technology for making decisions. The website—ion.gov.ro—is now governance: empowerment to self-rule and offline for unknown reasons, though some news the development of the capacities for self-rule. articles reference privacy and ethics concerns.7 Drawing on a long tradition of participatory ION is only one example of a growing trend toward democratic theory, we offer a framework for replacing human participation in public decision- evaluating the uses and designs of governance making with AI simulation. This simulation is built technologies. We then conclude with notes on on the nonconsensual harvesting of people’s how to apply these values to possible uses of AI in speech, behaviours, or characteristics either deliberative processes. We believe this framework to produce summaries or to model their ‘digital will be of interest both to those interested twin’ using AI. This approach leaves no path for primarily in ensuring good government, and to individuals to review and contest the accuracy those more concerned with the intrinsic benefits of the summary, nor the representativeness of of popular self-government. the digital twin. This also leads to extraction: 1. Values and nonconsensually harnessing this digital twin for simulated ‘consultations’ to legitimate top- Participatory Democracy down decisions. In their essay in this same compendium, Roy, Lessig, and Tang ask us to When choosing technologies to facilitate or imagine this situation in the extreme, where “every augment governance, we must consider not only citizen could be represented by an AI avatar, their functionality, but also the values implicit9 continuously engaging in community discourse on within the technologies themselves. their behalf”—what they term an “Avatar State.” Liz Barry, together with the creators of Polis Values are principles for guiding action to achieve and engineers at Anthropic, sounded the alarm particular benefits. Benefits may be intrinsic or about this problem in 2023, and showed through instrumental: an intrinsic benefit is a good in its experimental results that it is possible to scale own right, while an instrumental benefit is valuable deliberation among humans without resorting to only as a means to achieving some other end. simulation.8 However, this is rarely a clean distinction: most Values in Governance Technology | Barry & Gubbels 37 intrinsic goods are also instrumentally valuable formative function that “develops and fosters the for achieving secondary benefits, and many very qualities necessary for it.”11 instrumental goods blur into intrinsic goods. The two core values we believe should guide Values may be relevant to the individual or efforts to build or improve democratic tools, collective levels, depending on the level at which methods, and institutions are empowerment the resulting benefits are experienced. This to self-rule and development of the capacities distinction is also fuzzy, since collectives are for self-rule. There are also other values worth ultimately composed of and shaped by individuals, caring about, like equality, which is included but individual identities are formed against the in the definition of participatory democracy as background of community, and many aspects of a “(equal) participation in the making of decisions,” good life can only be experienced as a member of or dignity, which is assured by our empowerment some community. Thus, individuals and collectives as full and equal participants in governing our interact and are deeply intertwined, and the communities.12 There are also practical values benefits experienced at these levels are similarly that are instrumental to achieving these two linked.10 values, such as transparency, which helps ensure and defend equal empowerment. The two core We take democracy to be individual and collective values are not meant to downplay these other self-rule, which develops and is sustained by the values, but to identify the essential and enduring capacities for participation in decision-making. concerns of genuine democracy that justify and Collective self-rule means we are all equal frame these other, subsidiary values. participants in making the decisions that govern our societies, and individual self-rule means we 1.1 Empowerment (to Self-Rule) are able to use this power to shape not just our The first core value of participation is shared world, but our own lives. Following the empowerment to self-rule. On the individual tradition of participatory democratic theory, we level, this involves equal participation in decision- see political participation as having an essential Intrinsic benefits Instrumental benefits Individual: achievement of desired outcomes, non- Individual: freedom, recognition domination Empowerment to self-rule Collective: collective freedom? group Collective: good government, social stability, trust in recognition? others and institutions, legitimacy of decisions and laws Individual: flourishing, development of Individual: knowledge, speaking skills, confidence, own ideas, self-understanding horizontal facilitation skills, desire to participate, education in participant rights Development Collective: group self-understanding of capacities (including minority views and common Collective: improved coordinating structures, learning ground), solidarity, collective identity who knows what, who is good at what, how to act to serve as background for individual together, comfort with cooperation and joint action identity and belonging Figure 1: Two core values (empowerment and development) and their intrinsic and instrumental benefits, as experienced at the individual and collective levels. Values in Governance Technology | Barry & Gubbels 38 making,13 and at the collective level, it describes participation, thus supporting the achievement the self-determination of a community. of all the above-listed benefits. More broadly, participation also tends to develop the capacities The obvious instrumental benefit of individual self- for active citizenship and deliberation, such as rule is the ability to exercise a degree of positive public speaking, critical thinking, navigating control over the collective decisions that shape institutions, and the confidence to participate our lives, steering them toward our own interests again.20 by intervening in the decision-making process. Participation also grants a degree of negative Participation also develops our knowledge of control: it allows us to contest public decisions the world and our society, including via mutual that run against our individual interests, helping to learning and deliberation.21 The prospect of real secure against domination.14 participation also gives us reason to investigate matters ourselves, since “we do not know what Collective self-government also brings we need to know until we ask the right questions, instrumental benefits, including the well- and we can identify the right questions only by documented collective intelligence of deliberative subjecting our own ideas about the world to the groups—especially those that reflect society’s test of public controversy. Information, usually diversity.15 While some epistemic benefits can seen as the precondition of debate, is better be achieved by mere statistical representation, understood as its byproduct.”22 That is, political inclusive participation also brings a broad range of dysfunction is not caused by public ignorance, as perspectives and information into deliberations.16 often argued by skeptics of popular rule, rather, Finally, participatory decision-making allows public ignorance is caused by a non-participatory members to identify with the decisions and laws politics which strips us of the need and the of their communities, helping to increase the opportunity to become informed. legitimacy and stability of those decisions.17 Just as individuals become more effective political Empowerment also has a more direct and actors by participating, groups also develop their fundamental benefit for participants: as argued capacities for joint action through practice. This by a long tradition of democratic and republican involves improvements in the formal structures thinkers, participation in self-government is for joint action, such as the rules, spaces, and the essence of freedom, in that it enables self- institutions supporting group deliberation. Practice determination at the individual and collective in group action also develops beneficial informal level.18 The autonomy achieved through such self- social structures like trust and familiarity, teaches government is essential for individual flourishing individuals the skills for horizontal facilitation, and a community’s common good, independent of and teaches the group which of its members its other, instrumental benefits.19 possess the skills and specialized knowledge that may facilitate cooperation. Regular cooperation 1.2 Development (of Capacities) also becomes a habit, such that “each new need immediately awakens the idea of association.”23 The second core value of participation is the Lastly, participation cultivates a public perspective development of the various capacities associated among citizens, instilling in us a sense of with self-rule. responsibility for our society’s problems and a At the individual level, the development of our commitment to addressing them together.24 As ‘participatory muscles’ is instrumentally valuable the long republican tradition always understood, for enabling further and more efficacious citizens are made, not born.25 Values in Governance Technology | Barry & Gubbels 39 Beyond these instrumental benefits, developing interest in these new methods grows, there is a our capacities for participation, via participation, major risk of “democracy-washing”: the use of has important intrinsic benefits. The need to democratic language to disguise practices that are argue our case to others demands not just at least non-democratic, but often un-democratic knowledge, but also the elaboration of our own or even anti-democratic. thoughts: “we come to know our own minds only This is where the two core values can help. by explaining ourselves to others.”26 At the same The participatory democratic premise—that time, inhabiting others’ arguments for the purpose participation empowers us to self-rule and of refuting them introduces the possibility that develops our capacities for self-rule—allows us to “we may end up being persuaded by those we judge the extent to which (ostensibly) democratic sought to persuade.”27 This learning and deeper processes deliver real participation, by evaluating thinking also tends to lift the veil of symbolic how well they deliver on these two core values. politics and manipulation, aligning our expressed That is, cases can be compared based on how preferences with our own underlying values, such much they empower individuals and collectives that “deliberation has an emancipatory effect.”28 to rule themselves, and how much they develop When these basic human capacities are individual and collective capacities for self-rule. disengaged by a non-participatory politics, In the chart below, cases near the ‘red’ end do they decay like unused muscles, and our moral little to empower people or groups to self-govern character shrivels.29 So, real participation in and leave their capacities for self-government self-government is not just an effective means to undeveloped or even degraded. Cases near the promote our material interests, but an essential ‘green’ end facilitate individual and collective activity for a good life: by developing our basic self-rule (at least within the relevant scope) capacities and moral character, “democracy and significantly improve capacities for self- supports the flourishing of human beings as the government. kind of being they are.”30 Democracy-washing occurs when a process 2. Application is made to look closer to the green end of the spectrum than it really is—a thin coat of We believe new technologies and deliberative green (participatory) paint is used to disguise a and participatory methods can help our societies fundamentally red (non-participatory) process. better deliver the two core democratic values of Attention to the two core values can help us empowerment and development. However, as Values in Governance Technology | Barry & Gubbels 40 strip this veneer and evaluate how participatory washing, but we must also avoid building or a process really is by asking how well it delivers implementing tools that will undermine human empowerment and development. freedom and flourishing. A commitment to the two core values of participatory democracy can The two core values also help ground and justify help us avoid these dangers while also guiding our our commitment to democracy. When we forget efforts to build a more democratic future. the full range of benefits delivered by these two values, this commitment is undermined. Most 3. Conclusion arguments for technocracy claim it is better at delivering the instrumental benefits normally The precise application of these values must be a attributed to democracy (the top-right quadrant of matter for further discussion. Our aim here is only Fig. 1, like good government), and many dictators to lay out the values that should guide our efforts have justified their rule by appealing to the hard- to build and implement new democratic methods to-define intrinsic benefits of self-rule (the top- and technologies. Leading up to the publication of left quadrant of Fig. 1, like collective freedom this essay, we held many formative conversations for a nation or class). Yet, even if undemocratic with colleagues which suggest some possible governments could deliver their stated goals, directions. they cannot deliver the kinds of individual and collective development that are only achievable The most obvious application is avoiding through real participation. The tradition of simulation. Tools for human interaction and participatory democracy is distinguished by sense-making at scale already exist; it should be its claim that this lack of development leaves a norm in our field and amongst the public that citizens fundamentally unfulfilled—and also that deliberation in silico only be used for research undemocratic regimes are unjust regardless and training, not for public decision-making. of the other benefits they may promise. So, During deliberations, people should encounter while the relative importance of the two core each other’s actual words and writings—LLMs values and their benefits can be debated, a should not be used to smooth over the texture focus on development is the best way to identify and particularities of human expression and democracy-washing and to ground a principled conversation. Building genuinely participatory commitment to democracy. processes ensures that even after the engagement is complete and the scaffolding is removed, This framework should also guide our decisions participants leave with strengthened capacities around designing and adopting technology for they can apply elsewhere and with the group governance to ensure we support, rather than consciousness and solidarity to act on what they undermine, genuine democracy. As a field, now know they hold in common with those around we need to be able to identify and coherently them. condemn simulation, extraction, and democracy- Values in Governance Technology | Barry & Gubbels 41 Endnotes 11. Carole Pateman, Participation and Democratic Theory, 1st ed. (Cambridge University Press, 1970), https:// 1. Autoritatea Națională pentru Cercetare, “Premieră ro- doi.org/10.1017/CBO9780511720444, 42-43. See mânească: ION, primul consilier guvernamental din lume also: John Stuart Mill, Considerations on Representative ce va folosi inteligența artificială,” Ministerul Cercetării, Government, (Henry Regnery Company, 1962), originally Inovării și Digitalizării, March 1, 2023, https://www.re- published 1861, 43; G. D. H. (George Douglas Howard) search.gov.ro/premiera-romaneasca-ion-primul-consi- Cole, Social Theory (London: Methuen & co. ltd., 1920), lier-guvernamental-din-lume-ce-va-folosi-inteligen- 114-116, 208; Alexis de Tocqueville, Democracy in ta-artificiala-9546/. America, trans. Eduardo Nolla and James T. Schleifer, English ed (Liberty Fund, 2012), originally published 1832, 2. Agence France-Presse, “Romania PM Unveils AI 102, 494; Michael J. Sandel, Democracy’s Discontent: ‘Adviser’ to Tell Him What People Think in Real Time,” A New Edition for Our Perilous Times (Cambridge, The Guardian, March 2, 2023, https://www.theguardian. Massachusetts: The Belknap Press of Harvard University com/world/2023/mar/02/romania-ion-ai-government- Press, 2022), 13, 247, 255. honorary-adviser-artificial-intelligence-pm-nicolae- ciuca. 12. Pateman, Participation and Democratic Theory, 43; Stephan Kirste, “Humanism and Republicanism,” in The 3. Leo Sands, “This Government Aide Says It Knows Oxford Handbook of Republicanism, ed. Frank Lovett and What Voters Want. It’s an AI Bot,” The Washington Post, Mortimer Sellers. (Oxford University Press, 2024), https:// March 2, 2023, https://www.washingtonpost.com/ doi.org/10.1093/oxfordhb/9780197754115.013.13; world/2023/03/02/ion-romania-ai-bot-government/. Charles Taylor, “Cross-Purposes: The Liberal- Communitarian Debate,” in Debates in Contemporary 4. “ION (@noisuntemion) / X,” X (formerly Twitter), Political Philosophy: An Anthology, ed. Derek Matravers https://x.com/noisuntemion. and Jonathan Pike (Routledge, 2003), 198, 207–208; John Dewey, “Creative Democracy: The Task Before Us,” 1939. 5. Cristina Carata, “ION - AI Government Adviser,” MIT Solve, April 22, 2024, https://solve.mit.edu/ 13. Equal political agency is usually taken to mean equal solutions/85386. opportunity for influence, but it may require actually equal influence so all citizens are coauthors of the law (see: 6. Sands, “This Government Aide Says It Knows What Daniel Wodak, “What Is the Point of Political Equality?,” Voters Want. It’s an AI Bot.” Philosophical Review 133, no. 4 (October 1, 2024): 367– 7. Adrian Ardelean and Dora Vulcan, “Romania’s AI 413, https://doi.org/10.1215/00318108-11497679). Political Adviser Is Already In Hot Water,” Radio Free This would mean it is not enough for processes to merely Europe/Radio Liberty, March 6, 2023, sec. Romania, allow all to participate; they would have to take steps https://www.rferl.org/a/romania-ai-political-adviser- to maximize actual participation, so that all enjoy the ion-/32304377.html. benefits of empowerment and development. 8. Christopher T. Small et al., “Opportunities and Risks of 14. Jean-Jacques Rousseau, Letter to Beaumont, Letters LLMs for Scalable Deliberation with Polis” (arXiv, 2023), Written from the Mountain, and Related Writings, ed. https://doi.org/10.48550/ARXIV.2306.11932. Christopher Kelly and Eve Grace, trans. Judith R. Bush, vol. 9 (Hanover, New Hampshire: Dartmouth College 9. Wikipedia contributors, “Melvin Kranzberg,” Wikipedia, Press, 2012), 260-261; Philip Pettit, “Democracy, last modified March 24, 2024, https://en.wikipedia.org/ Electoral and Contestatory,” in Designing Democratic wiki/Melvin_Kranzberg#Kranzberg%27s_laws_of_ Institutions, ed. Ian Shapiro and Stephen Macedo (New technology. York University Press, 2022), 105–44, https://doi. org/10.18574/nyu/9780814786628.003.0009; Allen, 10. Bernard Manin, “On Legitimacy and Political Justice by Means of Democracy, 28. Deliberation,” trans. Elly Stein and Jane Mansbridge, Political Theory 15, no. 3 (1987): 338–68, 351-362; 15. David Estlund and Hélène Landemore, “The Epistemic Danielle Allen, Justice by Means of Democracy (Chicago: Value of Democratic Deliberation,” in The Oxford The University of Chicago Press, 2023), 206-208, 211. Handbook of Deliberative Democracy, ed. Andre Bächtiger 34, no. 2 (April 2011): 57–74, https://doi.org/10.1017/ et al. (Oxford University Press, 2018), 112–31, https:// S0140525X10000968. doi.org/10.1093/oxfordhb/9780198747369.013.26; 23. Tocqueville, Democracy in America, 914. Anita Williams Woolley, Ishani Aggarwal, and Thomas W. Malone, “Collective Intelligence and Group 24. Mill, Considerations on Representative Government, Performance,” Current Directions in Psychological 58; Taylor, Cross-Purposes, 198–203; Tocqueville, Science 24, no. 6 (December 2015): 420–24, https://doi. Democracy in America, 157–160, 397–398. org/10.1177/0963721415599543. 25. Sandel, Democracy’s Discontent, 62, 252; Arendt, The 16. Elizabeth Anderson, “The Epistemology of Promise of Politics, 253. Democracy,” Episteme 3, no. 1–2 (June 2006): 8–22, https://doi.org/10.3366/epi.2006.3.1-2.8; Simone 26. Lasch, The Revolt of the Elites, 170. Chambers, “Human Life Is Group Life: Deliberative 27. Lasch, The Revolt of the Elites, 170-171; Manin, On Democracy for Realists,” Critical Review 30, no. 1–2 Legitimacy and Political Deliberation, 351-352. (2018): 36–48, https://doi.org/10.1080/08913811 .2018.1466852. Danielle Allen, Justice by Means of 28. Simon Niemeyer, “The Emancipatory Effect of Democracy, 7-9. Deliberation: Empirical Lessons from Mini-Publics,” Politics & Society 39, no. 1 (2011): 103–40, https:// 17. Hannah Arendt, On Revolution (New York: Penguin doi.org/10.1177/0032329210395000; Simon Classics, 2006), originally published 1963, 129; Pateman, Niemeyer et al., “How Deliberation Happens: Enabling Participation and Democratic Theory, 27; Allen, Justice by Deliberative Reason,” American Political Science Review Means of Democracy, 28. 118, no. 1 (2024): 345–62, https://doi.org/10.1017/ 18. Pateman, Participation and Democratic Theory, 26; S0003055423000023. Hannah Arendt, The Promise of Politics (Knopf Doubleday 29. Mill, Considerations on Representative Government, Publishing Group, 2009), 118, 126-127; Hannah Arendt, 57-58, 229; Patrick J. Deneen, Why Liberalism Failed (Yale On Revolution, 218. Sandel, Democracy’s Discontent, University Press, 2018), 22-24, 186-188. 13; Jed W. Atkins, “Non-Domination and the Libera Res Publica in Cicero’s Republicanism,” History of European 30. Allen, Justice by Means of Democracy, 31-35. Ideas 44, no. 6 (August 18, 2018): 756–73, https://doi.or g/10.1080/01916599.2018.1513705. 19. Allen, Justice by Means of Democracy, 28; Taylor, Cross-Purposes, 201; G. D. H. (George Douglas Howard) Cole, Self-Government in Industry, G. D. H. Cole: Selected Works, Vol. 2 (London: Routledge, 2010), originally published 1919, 227–228. 20. Pateman, Participation and Democratic Theory, 24- 26; Tocqueville, Democracy in America, 102, 494; John Stuart Mill, Essays on Politics and Culture, ed. Gertrude Himmelfarb (Garden City, New York: Anchor Books, 1963), 186, 229. 21. Manin, On Legitimacy and Political Deliberation, 354. 22. Christopher Lasch, The Revolt of the Elites and the Betrayal of Democracy (W. W. Norton & Company, 1995), 162-163. See also: Hugo Mercier and Dan Sperber, “Why Do Humans Reason? Arguments for an Argumentative Theory,” Behavioral and Brain Sciences