AI can bring about incredible benefits for humanity. However, it also has the potential to cause widespread harm to people or society if sufficient steps are not taken to make it safe. Devastating AI-enabled risks include biological and chemical attacks, escalation of nuclear confrontation, widespread political chaos and epistemic collapse, extreme power concentration, economic disempowerment, loss of control to runaway artificial superintelligence, and more. Despite acknowledging theses dangers, either by accident or from misuse, the AI corporations developing and deploying these systems continue to do so at a reckless and increasing pace, often reportedly regarding safety as an afterthought.

It has become clear that regulatory intervention is critical to safeguard our shared future. It is the only mechanism to deliver the transparency, oversight and safeguards necessary to prevent escalating harms, mitigate catastrophic risks, and ensure AI’s benefits are shared by all. At the same time, it is clear that far greater investment in technical safety solutions is critical.

In line with this stated goal, FLI advocates for effective and meaningful governance of AI, including the implementation of robust licensing, auditing, oversight and liability regimes. To achieve this, we work with and advise lawmakers in the United States, European Union, United Nations and beyond. Effective AI governance also requires empowered institutions, and FLI works with key stakeholders to help design, staff, and support them. Alongside these regulatory efforts, FLI funds technical research to improve the quantitative safety of advanced AI systems, and executes educational campaigns designed to drive awareness of risks and necessary solutions.

Ungoverned and accelerating technological development has the potential to concentrate vast amounts of power within a handful of advantaged groups, organizations and individuals. This would have disastrous consequences, and nowhere is this more true than with artificial intelligence.

Governments could weaponize AI’s incredible capabilities to wield Orwellian levels of surveillance and societal control. We have also seen first-hand with social media how AI can be utilized to manipulate public discourse, and access to this kind of information control could be supercharged by advanced AI, leading to paralyzing “truth decay” and the effective collapse of meaningful democratic participation.

Giant AI corporations could create stifling monopolies, granting them vast amounts of economic, social and political power that could surpass elected governments. As a small group of companies hoard power and capital at the expense of jobs and market competition, entire industries and large populations will become increasingly dependent on them – with no satisfactory guarantees that benefits will be shared by all. Both of these scenarios could quickly become irreversible: once AI concentrates sufficient cross-domain power within a specific group, it would lock-in control by a small, ruling party or handful of corporations.

AI powerful enough to command large parts of the political, social, and financial economy is unfortunately also powerful enough to do so on its own. Uncontrolled artificial superintelligences could rapidly take over existing systems and amass increasing amounts of power and resources to achieve their objectives, at the expense of human wellbeing and control, quickly bringing about our near-total human disempowerment or even extinction.

AI-driven extreme-power concentration represents a catastrophic threat, and therefore FLI works actively to bolster democratic institutions and decision-making processes, establish and reinforce competitive markets with a multitude of participants, and empower individuals.

The development of AI systems that surpass human expert capability at general tasks carries enormous risk. At best, such systems would constitute a “second species” of intelligence that would disempower huge swathes of humanity as labor, planning, culture generation, and likely much decision-making is delegated to machines. At worst, these systems could deceive, develop and pursue their own objectives, circumvent safeguards, exponentially self-improve, and self-replicate and displace humanity. Superintelligent systems are likely to be unpredictable, uncontrollable, unalignable and ungovernable by existing institutions. Through misalignment or misuse, this represents an existential threat to humanity, either through terminal disempowerment or literal extinction.

Despite this widely acknowledged threat, frontier AI is currently being developed in an unsafe, unaccountable, and ungoverned manner. Each new training run constitutes an experiment, as the capabilities and risks of the next generation of systems cannot be predicted. This is unacceptable, especially in light of polls repeatedly revealing widespread alarm at the pace and recklessness of advanced AI development. FLI has called for, and continues to support, a pause on such new training runs, until adequate oversight, safety, and regulatory mechanisms are in place.

The goal of these experiments, as stated by many AI corporations, is the development of superhuman AGI. With this in mind, FLI supports a moratorium on developing such superhuman AGI or superintelligence systems, for at least 15 years and until it can be done with satisfactory and provable safety guarantees that are commensurate with their enormous risks.

This development should also be accompanied by a meaningful participatory process of global deliberation and consent. The risks of these systems are borne by everyone, and therefore we must work to ensure a diversity of global voices are included in guiding their development. Our shared future should not be decided by a handful of tech billionaires.

AI can deliver incredible benefits, solve intractable global problems, and realize inspirational futures. However, the vast majority of these benefits do not require artificial superintelligence and the massive risk that come with it.

As we have already seen, existing general purpose AI systems, when empowered by specialized training, have the potential to fundamentally and beneficially transform health, science, productivity and more.

This transformative empowerment can start today. We already have the capability to leverage AI to research cures for many diseases, provide individual tutors for every child on the planet, work toward clean energy solutions, and realize countless innovations. However, to achieve this we must intentionally invest in developing AI to solve specific problems and achieve identifiable goals (the Sustainable Development Goals, for example).

However, AI investment to date has been largely focused on power-seeking, not problem solving: developing very general systems that can replace people across as many sectors as possible, rather than being driven by particular needs. Despite promises of abundance, and vague predictions about utopian futures, there has been relatively little investment by Frontier AI corporations to deliver identifiable benefits or solve specific problems. They continue to make vague promises of future abundance created by AI, but history provides little confidence that said abundance will be shared. Instead, labor market forecasts and diverging living standards are stoking growing alarm and widespread discontent: ever more individuals find themselves economically replaced as a small group of companies hoard wealth at the expense of jobs.

We should be using AIs to help humanity flourish, not to replace and disempower people. Highly capable AI systems have not been created by the ingenuity of a handful of AI corporation leaders, but by a diverse group of researchers (educated with mostly public funds) creating systems that were trained on the collective sum of humanity’s knowledge, creativity, and experience. We believe that the opportunities and benefits of AI should be open to and widely shared by humanity, not consolidated into the hands of a few.