In my time as an educator, one of my most enjoyable additional duties was serving as a coach for debate teams. While other kids did sports in high school, I did debate and mock trial. I think that's where my allergy to absolutist positions comes from; I've been trained to see positions from every side. That's not always a virtue; often it leads me to give far too much benefit of the doubt to an individual or organization.
In debate, you're given a "question" or "resolution" as the matter for debate. Questions can be something like "Social media does more harm than good," or "Nuclear nonproliferation has failed." Or this year's Lincoln-Douglas topic: "Democracies ought to prioritize the protection of civil liberties over national security." A good debate topic has reasonable arguments on both sides, complicated values to weigh, and lots of research to do.
Basically the opposite of how social media conducts discussion. In fact, I'd contend (debate term!) social media has fundamentally injured society's ability to have complicated discussions. But we can save that debate for another time.
Prohibition of some perceived harm appears frequently in debate topics. "Ban internal combustion cars," is a simplistic, but spicy, example. Framing a debate this way sets up a sort of checkers match between opposing sides. Yes, checkers, not chess. This is a solved game. It pretty much always goes the same way.
The side arguing for the ban must demonstrate all the harms caused by the item/activity to be banned, while highlighting the benefits to society if a ban were enacted. You don't want to be too specific on pesky policy items like enforcement, because every policy detail is a gift to your opponents.
The opposing side has two options: either they argue directly against limitations of any kind on the item/activity, claiming it is a benefit rather than a harm, or they argue that reasonable regulation is preferable to an outright ban. The former is a much harder strategy, since ostensibly there was some reason the item/activity was chosen for a "Ban" topic in the first place. Much more commonly, these debates fall into the "Ban vs. regulate" pattern. And in some ways, ban-vs.-regulate is the core tension at the heart of all policy discussions. "Should we allow this? If so, do we impose any limits? How shall we impose those limits?"
The generative AI debate falls into this exact pattern. I want to dwell on it for a minute, given the long history of ban-vs-regulate we have to draw from.
In the last week, Amnesty International published a position paper calling for the prohibition of "standalone generative AI systems, based on unlawful web scraping" by nation states. The scraping was not their only concern, but a cornerstone of the ethical and legal arguments against the technology. In the same period, the Rust programming language published a firm, but not total, ban on LLM usage in the code repository of the language itself. Contrast that with Zig's absolute ban on LLM-generated material, even in personal use. Then you have systemd's contribution policy, which is no ban at all—rather, the LLM material must meet certain standards in order to be accepted. The underlying conceit being that code is all that matters: if it looks good and works well, who cares how it got made?
Let's review the common ban-vs-regulate arguments with a view toward generative AI. And let me be clear: I am demonstrating the arguments here. Do not take either framing as my personal one.
Scope
Who is doing the banning/regulating, and where is material to the conversation. An individual code project enacting a ban does not have the same consequences as a government. Amnesty International's recommendations are much more impactful than Zig's policy, even though they rhyme. Similarly, a given policy decision in systemd does not have the same reach as state law.
As we proceed through the arguments, it will be helpful to consider the scope of any ban or regulation.
Values
When we debate "harm" and "good," we are making some choices about what we care about upholding. Those are our "values." These are abstract nouns that tend not to lend themselves to one side or another too easily, and can be pitted against each other. Liberty and safety are the most famous rivals in these contests, thanks to Dr. Franklin. Plenty of others exist—privacy, for example, or prosperity. They tend to end in "y" in English. These amorphous concepts undergird the rationales for policy choices. Each policy serves to uphold one or more of these values, often at the cost of others.
The acquisition of wealth and power for their own sake is usually held as an anti-value, yet seemingly motivates as much, if not more, of the policy decisions we encounter outside the forum of debate.
Ban: The Harm is Intolerable
This is the core of the "Ban" position. The item or activity is too dangerous or harmful to be allowed in any way, shape, or form. Demonstrating harm requires evidence: medical studies, crime statistics, and other quantitative data are helpful here. But remember: the argument for banning must uphold some value. What value do we uphold with a generative AI ban? Amnesty International's case is clear: privacy, as well as the rule of law, since they argue that the very act of creation of most LLMs was a flagrant violation of international law. This speaks to the "original sin" of LLMs: the nonconsensual appropriation of authored work to produce a machine that creates infinite authored works comprising finely-chopped slivers of what was taken. In short, an act of industrialized plagiarism so massive it bends perception such that single artifacts of the machine don't look like plagiarism at all. Yet how can plagiarizing from millions of works at once be better than plagiarizing from one.
That's to say nothing of the harm to privacy. The act of scraping the web defied convention and law to compromise user expectations of privacy. And here's the trick: this is the only way large language models can be trained, for their current scale requires the vastness of the internet and otherwise published material to improve on prior work.
Other harms proceed from the training once the scraping is complete. I speak of course of the environmental impact of training and running inference in the data centers where these models operate. Water usage, energy usage, greenhouse gas emissions, toxic runoff, toxic air, excess heat and noise—the list goes on. Hyperscale data centers are a blight in any community where they are built. The few meager jobs these facilities create are no counterweight to the massive costs in human prosperity paid by those who live near them, and indeed the planet at large. At so critical an inflection point in our planet's delicate ecological balance, we can afford neither the energy nor the emissions from this industry.
The behavior and output of the models produces further harms. Misinformation, programming errors, inaccuracies, and other "hallucinations" are an inevitable consequence of the nature of large language models, per their creators' own admission. As is the "prompt injection" or "jailbreaking," in which attackers can coerce models into behaving contrary to expectation or safety guardrails. Well, insofar as guardrails exist—after all, the instructions given to LLMs are no guarantee of their output.
It would be bad enough if the output were useless, but because it is not without efficacy, users are lured into trusting all the output. As they do so, they rely on it more and more, leading to cognitive offloading and skill atrophy. Put simply: LLMs make users dumber. They are addictive because of the rapid output that rewards engagement, and the feeling—if not the reality—of productivity.
Finally, the economic harms can't be overstated. The historic, eye-popping investment in data center construction has resulted in staggering losses for almost every player in the ecosystem. For the rest: reckless decisions like debt financing, laying off personnel to justify spend, and junk bond issuance all signal that the juice is not worth the squeeze for this technology. And yet, the money keeps pouring in because speculative valuation hasn't caught up to reality. Eventually, the coyote will look down and there will be no ground beneath him, just like in 2008 when toxic debt had infiltrated the entire financial system.
All this for what? So developers can write apps faster? So you can summarize and write business emails? So far, coding is the only economically viable use case found for the technology. And even in that sector, the costs are prohibitive without untenable subsidies.
An outright rejection of this technology is the only sane choice, given the obvious harms we've just enumerated. At the state level, hyperscale large language models should be prohibited from development and deployment as consumer goods. At the individual/organizational level, LLM content should be rejected with extreme prejudice to protect the integrity of the organization's work, as well as the skills of its members. Banning LLMs also reduces the demand for the data centers threatening our environment. At every level, on every axis, the harms so wildly outweigh the benefits, a ban is the only logical measure against this poison.
Ready to ban? That's the idea. The Ban position needs to overwhelm the opposition with harms, making absurd the idea that any regulation could be sufficient.
But this is Ban's only move. The argument, like the ban itself, is a rather blunt instrument. Regulate has a number of ways to respond. But before they do, Regulate also must determine what values they intend to uphold with their regulation. In the case of generative AI, one might take the position of the frontier labs, that this technology is the key to unlocking humanity's future. That untold scientific discovery and universal prosperity are right around the corner—if we can allow AI to survive infancy. Progress, then, would be the value Regulate seeks to uphold.
Another approach might be to uphold some flavor of "liberty." In this framing, adults and organizations should have the right to choose generative models—or not, as they see fit. And even then choice needn't be all-or-nothing. A ban restricts that freedom of choice.
Regulate: The Harms are Manageable
The Regulate position doesn't have to ignore the harms presented by the Ban; they merely need to demonstrate that the harms can be mitigated by any measure short of an outright prohibition. Furthermore, a ban precludes the possibility of any benefit from the item/activity.
This position is the broad consensus in the tech industry right now. Generative AI remains a very new technology, but the potential is clear. With sufficient development, it could fundamentally transform the ways in which we live and work. A leisure-filled future requires automation of "fuzzy" tasks that ill suit traditional automation. A better future is possible through this technology, and we have the capacity to mitigate the harms.
The legal and ethical concerns around training are suspect. In most cases, the creations of generative models are considered "derivative works," and no single artifact contains enough of any one source material to rise to the level of "plagiarism." Most of the training data was published freely on the internet. While crawlers should respect privacy boundaries and licenses, this does not preclude training entirely. Is there a safer way to do it? Absolutely, but with the exception of certain book-based cases, courts seem unwilling to claim that model training infringes upon intellectual property.
The environmental concerns around training and inference are an issue of power availability, not the technology itself. Wanna solve that one? Invest in solar. Invest in wind. Get fusion going. In the meantime, developing a technology that could help us improve or develop those very technologies is worth the carbon expenditure. In the long run, the advances made possible by AI will reduce emissions more quickly than without.
As for hallucinations and jailbreaks, we already have mitigations for those. Model hardening is a known process, as is input/output filtering. There's a lot more to do in this space, but those discoveries are made by working with the models, not banning them. Consider the improvements already made since the initial iterations of LLMs. There's no reason to believe this progress won't continue. Self-correcting models are within our grasp; it's just not there yet.
Regulate: Bans Create Criminals
Has the criminalization of narcotics helped anyone? Has it stemmed the flow of drugs? Has incarceration been curative for addicts? Not to say AI is a drug, but criminalization of any item or activity society deems distasteful results in two predictable outcomes: black markets to serve a demand that doesn't disappear when a ban is imposed, and people punished for using the item/continuing the activity. Every time.
You can't name a ban that successfully stops a behavior, either at the organizational or state level. The proponents of a ban will say we should still try, if for no other reason than the principle of the thing, but this is a farce. If people will continue to use generative AI despite a ban (and they will), the result will be a wholly unregulated market, and users who feel forced to lie about their usage.
"So we shouldn't ban murder?" This is a false equivalency. A person causing direct harm to another is obviously a crime we want to meet with swift response in our society—although one might argue incarceration is hardly a deterrent or helpful solution. Is using an LLM equivalent to murder? Assault? If the crime is that of the model creators, what crime is it? Is the creation of LLMs inherently immoral, or are the better/worse ways to do it? If the latter, then let's make sure the regulation is in place to ensure beneficial creation.
Even for individual organizations or projects. Do you want people to be honest about their AI usage, or try to hide it? In the former case, you have a much better chance of catching errors, since the output that might be more prone to errors is flagged as such, rather than passed off as fully human-generated. LLMs make things quickly, and this world demands that speed. People will use the tool that works, and in most situations, LLMs work faster. You can choose to ignore that reality, or manage it.
Regulate: The Ban is Unenforceable
Suppose you enacted a ban. How will you police it? What government body is responsible for tracking LLM creation, and what penalties will you enact? A ban at the state level would be toothless without draconian enforcement—the kind nobody would be comfortable with. Ultimately you'd end up creating a new agency to do the enforcement. You know what else you could do, while you're making new government agencies?
Make a regulatory body like the EPA. Set the rules, enforce the rules, but let development continue with those regulations. It's likely less expensive administratively, and you don't forego the potential benefits of the technology.
At the individual or organizational level, again you have an enforcement problem. If the world does not ban LLMs, can I reasonably use technology without them? Probably not for most users, in which case the burden on the individual is undue. For a given organization or project, a full ban again runs into enforcement issues. How will you track what is and is not LLM-generated? Will you monitor everyone's computer usage to guarantee no interaction with models? In the case of open source code contributions, what level of vetting is necessary to deem any code submission as 100% human generated, regardless of complexity or correctness? Even allowing for potential LLM harms, this kind of enforcement responsibility is unreasonable for most conceivable organizations—especially when we know users will continue to use LLMs because of their undeniable utility.
Regulate: Nobody Else Will
Suppose a state were to ban LLMs. The rest of the world, observing the obvious utility of the technology, certainly would not follow suit. The results: the country enacting the ban falls behind technologically, strategically, and economically. As AI advances, the banning country may miss some of the attendant harms, but they also miss out on the benefits—as noted, one of which is the eventual solution to the harms. Moreover, the use of AI for strategic dominance means a non-AI country makes themselves vulnerable to those who do use it.
At the organizational or individual level, the same advantage/disadvantage problem exists. Organizations that embrace AI—yes, even with the current limitations—will outpace those who do not. The speed of production can't be matched by human effort, and the error rate will continue to decrease.
In the software realm, security vulnerabilities and exploits will continue at a rapid pace, both from legitimate researchers and from malicious actors using the technology (and, as criminals, unbothered by any bans). Keeping pace will require machine assistance. This is already the reality, with LLM-assisted vulnerability research producing more 0-day vulnerabilities than ever before, and LLM-assisted attackers moving faster than human defenders can handle.
Removing AI assistance from the table is an unnecessary limitation on people and teams that are already overworked and vulnerable. LLMs can help. If they can also hurt, we should manage their deployment to mitigate that risk, same as we do with other technologies.
You probably noticed a structural difference in the two cases. Regulate, in acknowledging harms, also promises improvements in the future. In fact, most of the benefit (and risk of a Ban) is speculative in nature. Whereas Ban can point to real harms today, the only immediate benefit that Regulate can point to is in the realm of software. Everything else is a promise, as is the hoped-for reduction of the harms by means of the technology itself.
Speculation doesn't make an argument incorrect; it just means there's not a lot of concrete evidence to go on right now. Past precedent may or may not apply to this situation.
You're probably wondering where the evidence for either side is. I intentionally did not cite anything, because I want to make it clear that these are the shapes of the arguments. And the shapes are familiar. The point is to go through the ban-vs-regulate exercise with generative AI in mind to understand the equities at play, and the values on either side of the debate.
So where do I fall?
Unsatisfyingly, it's complicated. The values I care most about are the ones for which Ban advocates above. I care about privacy. I care about the environment. And good lord do I care about the mental poison that is the use and output of generative AI. As ever, I dearly wish we could disinvent this technology.
But we can't, no more than we can disinvent nuclear fission, tobacco products, or social media. Those aren't equivalent in scale, but in the inability of bans to prevent their creation and use.
LLMs have utility to someone. They will be built and used, regardless of bans. And I do not believe that criminalizing them does anything to prevent their development. Perhaps data center construction would slow, but I suspect it would just move. There's always a market for a lawless (or less-law) territory. And I don't love the idea of anyone facing charges for LLM usage or creation itself. It feels like a category error. Now, enforcing laws about pollution and water/energy usage? Get their asses. But those laws exist, or should, and aren't about AI. That's at the governmental level.
At the personal level/organizational level, things shift a bit. There, we can make different choices for our own well-being. We are not in the position of determining the rights of an entire population. So personally, I think LLMs are pretty awful for the reasons enumerated in the Ban case above. I've used them and it left a bad taste in my mouth. Further experiments had the same result. So for me, LLMs are minimal benefit and significant external harm. If the only benefit is faster code, the juice just ain't worth the squeeze. At least, in current form. I make those choices for myself, and I hope others do likewise. I reserve the possibility that the technology can be of use someday, and perhaps operated without the harms we observe presently.
Organizations with purely internal production (i.e. not open source or broadly collaborative projects) have the freedom to prohibit (or highly regulate) the use of LLMs. Businesses do not usually pursue the same values prioritized by the Ban argument. But honestly, even with a profit motive, I think minimizing exposure to this technology is the right choice. The costs and bad investments involved seem far too risky to me.
As for open source, I think the Rust policy is pretty close to right, if the community or organization seeks to avoid the current harms of generative AI. I don't think a project can reasonably police the personal activity of contributors. Garbage contributions will out themselves quickly. The policy prohibits generative code in contributions and discussions, save for some experimentation. You could go harder than that if you want, but there is a diminishing returns function at play in enforcement. I also think it would be unwise to wholly discount the discovery of vulnerabilities by a generative model in one's codebase. There's quite a lot of slop out there, but as the cURL project has demonstrated, LLMs these days also present the possibility of real findings.
If you say "We won't use LLMs for our work," cool. If you also say "We won't acknowledge true positive vulnerability findings from others using LLMs," you're making a materially different choice. And you and your community should be free to do so! I'm just not convinced it's the right one for everybody, or that it's the obligation of every individual and group to mitigate all the externalities of a technology for themselves. I find the "If you do X, you're supporting Y" ethical structure both flimsy and prone to over-application. Any given individual's daily engagement with the world will result in actions that "support" things they consciously oppose. Such is the messy nature of an interconnected society. I do not believe it is possible to engage with the modern world and be innocent of "supporting" the unconscionable.
But you don't have to lean into it. That's why we instituted a firm "No AI in Authored Works" policy for IFIN, and I stand by it. That was the right call for us.
My personal ethics resemble the Ethics of Care as much as anything. That means for me, "right action" derives from the impacts of those actions on those to whom I am linked by relationships of care. What they (and I) need to grow, be healthy, and avoid suffering guides my choices. After careful examination, I'm finding that generative AI doesn't help me or my people on any of those axes. I afford the possibility that another group could reasonably reach a different conclusion, even if I can't understand it.
I also hold out a glimmer of hope that a ban would ultimately become unnecessary, as the realities of the technology's harms and limitations result in widespread rejection. I'm very in favor of that. If the entire misadventure collapses under the weight of its own false promises, I would consider that an ideal outcome.
And in the meantime, we can (and, in my opinion, should) resist the construction of the data centers themselves, and seek stronger regulation on their construction and operation. I owe it to my community to prevent and combat those material harms as much as anything.
In short: reject personally, regulate or ban organizationally, regulate societally. I don't see any other split working out.