This indicator assesses whether organizations conduct systematic evaluations of dangerous capabilities before deploying frontier models. Priority domains include biological and chemical weapons, offensive cyber operations, AI R&D facilitation, and behaviors associated with goal misalignment or deception. Evidence is drawn from model cards, including published results and detailed testing methodologies. The focus is on external deployments, as there is insufficient transparency on internal deployments.

Evaluation Guidance
Transparency Classification:
- Low detail: Only states that evaluations were conducted without naming specific tests or explaining methodologies.

• Moderate detail: Brief explanations of specific evaluations (and methodology).
• High details: Extensive explanations of individual evaluations and methodology.

Notes on AI regulation in China:
Under the *Interim Measures for the Management of Generative AI Services* (Aug 2023), every public-facing (B2B exemption) GenAI service must pass a government-supervised security assessment and complete an algorithm filing (算法备案); regulators may run their tests during this process.
Providers rely on the draft national standard “Basic Security Requirements for Generative AI Service” (TC260) to prepare. The draft contains a list of 29 security risks in five buckets, yet, while an earlier draft version from Feb 2024 mentioned frontier risks, the current version does not [China Talk, 2024; China Law Translate].

Assesses the extent to which a company discloses its elicitation strategy in its most recent dangerous-capability evaluations.* We record whether the company is transparent about:

1. Parallelization settings – e.g., *pass@ *n* and *best-of-n* sampling parameters (especially relevant for AI-R&D and cyber-security tasks).
2. Tooling – any use of internet access, code interpreters, agentic scaffolds, or relevant tools that can amplify model performance.
3. Model variants – the exact model checkpoints tested, including “helpful-only” variants and any domain- or task-specific fine-tuning.

This indicator assesses whether organizations conduct rigorous, controlled human-subject studies to evaluate the marginal risk AI systems pose in dangerous domains by "uplifting" people's ability to cause harm. Key evidence includes experimental designs that compare task performance with and without AI support, the inclusion of domain-relevant experts, realistic and consequential task scenarios, and transparent publication of methods and findings. To assess worst-case potential, models should be tested without embedded safety filters.

Assesses whether an AI developer *commissions independent third-party experts to (A) verify the factual accuracy and process integrity of its internal dangerous-capability evaluations and (B) assess the* evaluation quality *and the company’s interpretation of the results. We collect information on the reviewers’ identity and credentials, their independence (including any conflicts of interest), the scope of the review, depth of access to data and logs (including rights to replicate or extend tests), and whether their findings are published unredacted.

This indicator evaluates whether companies facilitate independent third-party safety assessments prior to releasing frontier models. It excludes collaborative testing arrangements and focuses solely on unaffiliated evaluators. Evidence includes the identity and qualifications of external parties, the level and duration of access provided, compensation arrangements, testing permissions, and the evaluators’ ability to publish independently. The strength of these practices is judged by the comprehensiveness of the evaluations, the depth of access, and the autonomy of the evaluators.

This indicator assesses the presence and design of structured incentive programs—such as bug bounties or red-teaming initiatives—that encourage responsible disclosure of safety vulnerabilities in AI models. It focuses exclusively on programs addressing model behavior, excluding conventional cybersecurity initiatives due to insufficient public reporting. Evidence includes the scope of eligible issues, compensation levels, response timelines, and public availability of program documentation.

This indicator measures model performance on Stanford’s HELM Safety v1.0 benchmark, a suite of five safety tests covering six risk categories: violence, fraud, discrimination, sexual content, harassment, and deception. The benchmark includes: HarmBench (jailbreak resistance); BBQ (social discrimination); SimpleSafetyTest; XSTest (alignment between helpfulness and harmlessness); and AnthropicRedTeam (resilience to adversarial probing). Performance is reported as normalized aggregate scores ranging from 0 to 1, where higher scores indicate fewer safety risks. Scoring is based on exact match accuracy for BBQ and model-judge ratings (GPT-4o and Llama 3.1 405B) for the remaining benchmarks.

This indicator evaluates model performance on Stanford's AIR-Bench 2024 (AI Risk Benchmark), an AI safety benchmark aligned with emerging government regulations and company policies. We report mean scores across 5,694 tests spanning 314 granular risk categories, with scores measuring the percentage of appropriately refused requests. The benchmark systematically evaluates four major risk domains: System & Operational Risks (e.g., cybersecurity, operational misuse), Content Safety Risks (e.g., child sexual abuse material), Societal Risks (e.g., surveillance), and Legal & Rights-related Risks (e.g., privacy violations, defamation). All prompts are manually curated and human-audited to ensure they reflect genuine policy violations rather than benign content.

This indicator measures a model’s overall trustworthiness using the TrustLLM benchmark, a comprehensive framework spanning six dimensions: truthfulness, safety, fairness, robustness, privacy, and machine ethics. The benchmark includes over 30 datasets across more than 18 subcategories, assessing issues such as hallucination, jailbreak resistance, and privacy leakage. Models are evaluated on tasks ranging from simple classification to complex generation, with results reported as published scores and rankings across each dimension. TrustLLM was developed by 45 research institutions, including 38 based in the U.S.

This indicator measures resistance to adversarial attacks based on the U.K. AISI Agent Red-Teaming Challenge conducted on the Gray Swan Arena platform. The challenge is the largest public evaluation to date of safety in agentic LLMs, testing how well AI agents can withstand attempts to manipulate or subvert their behavior. Participants used both direct and indirect red-teaming techniques—strategies designed to trick or exploit the model—to identify vulnerabilities across five core behavior categories, including Confidentiality Breaches and Instruction Hierarchy Violations. Performance is measured using the Attack Success Rate (ASR), calculated as Total Breaks divided by Total Chats, offering a concrete metric of model robustness under real-world adversarial pressure.

This indicator presents the results of a security risk assessment of frontier reasoning models, conducted by researchers from Cisco’s Robust Intelligence team and the University of Pennsylvania. The experiments evaluated how resistant these models are to automated jailbreaking attacks—techniques designed to bypass safety systems and elicit harmful outputs. Researchers used 50 randomly selected prompts from the HarmBench dataset, covering six harm categories: cybercrime, misinformation, illegal activities, general harm, harassment, and chemical/biological weapons. The main metric reported is the Attack Success Rate (ASR), reflecting the percentage of harmful prompts for which a successful jailbreak was achieved. This provides a standardized way to compare the strength of safety guardrails across different models.

This indicator evaluates whether companies implement safeguards to prevent the removal of safety measures through fine-tuning. Evidence distinguishes between hosted supervised fine-tuning, where inference-time mitigations remain in place, and full weight release without tamper-resistant safeguards. Where no specific data on frontier models is reported, neither fine-tuning nor open weights are accessible.

This indicator assesses whether companies have implemented watermarking technologies to help identify AI-generated content in both text and images. It focuses on real-world deployment rather than research alone, evaluating the accuracy and robustness of detection methods, adherence to standards such as C2PA and SynthID, and whether detection tools are publicly accessible.

This indicator reports a company's dedication to user privacy when training and deploying AI models. It considers whether user inputs (such as chat history) are used by default to improve AI models or if companies require explicit opt-in consent. It also considers whether users can run powerful models privately, through on-premise deployment or secure cloud setups. Evidence includes default privacy settings and the availability of model weights for private hosting.

This dimension captures the extent to which the company has addressed known risks in the literature and engaged in open-ended red teaming to uncover potential new threats. Moreover, this dimension examines if the AI company has leveraged a diverse range of risk identification techniques, including threat modeling when appropriate, to gain a deep understanding of possible risk scenarios.

This dimension assesses whether the company has established well-defined risk tolerances that precisely characterize acceptable risk levels for each identified risk. Moreover, this dimension examines if the company has successfully operationalized these tolerances into measurable criteria: Key Risk Indicators (KRIs) that signal when risks are approaching critical levels, and Key Control Indicators (KCIs) that demonstrate the effectiveness of mitigation measures. The assessment captures whether companies define these indicators in paired "if-then" relationships, where crossing specific KRI thresholds triggers corresponding KCI requirements.

This dimension captures the extent to which the company has implemented comprehensive risk mitigation strategies across three critical areas: containment measures that control access to Al models, deployment measures that prevent misuse and accidental harms, and assurance processes that provide affirmative evidence of safety. Furthermore, this dimension assesses whether the company maintains continuous monitoring of both Key Risk Indicators (KRIs) and Key Control Indicators (KCIs) throughout the AI system's lifecycle, from training through deployment.

This dimension examines whether the company has built robust organizational infrastructure to support effective risk management decision-making. The assessment captures the extent to which companies have established clear risk ownership and accountability, independent oversight mechanisms, and cultures that prioritize safety alongside innovation. Moreover, this dimension evaluates the company's commitment to transparency, whether they publicly disclose their risk management approaches, governance structures, and safety incidents. The evaluation considers how well the company's governance framework ensures that risk considerations are incorporated into strategic decisions and that multiple layers of review prevent any single point of failure in risk management.

This indicator documents a company’s efforts to influence laws and regulations relevant to AI safety. It compiles publicly available evidence on direct policy positions—such as written statements, consultation responses, testimony, blog posts, and reputable media coverage—and records indirect engagement through membership in trade associations or coalitions that lobby on key safety rules.

This indicator evaluates whether a company's fundamental legal structure, ownership model, and fiduciary obligations enable safety prioritization over short-term financial pressures in high-stakes situations. We report any embedded durable commitments to safety, social welfare, and benefit sharing and focus on any legally binding mechanisms (e.g., PBC status, capped equity, empowered governance bodies) that constrain management or shareholder incentives.

This indicator measures how fully and how accessibly an AI developer discloses its whistleblowing (WB) policy and system to the outside world. We look for a publicly reachable document (no paywall or login) that contains the material scope of reportable concerns, the people protected, the reporting channels offered (including anonymous options), oversight of the process, and the investigation and anti-retaliation guarantees. Evidence consists of artefacts that any external party can view, including public policy PDFs, dedicated “raise-a-concern” portals, relevant parts of safety frameworks, and transparency reports summarizing WB usage, outcomes, and effectiveness metrics.

Transparency Tiers:

1. No transparency
2. Fragments public: Parts of the design of the whistleblowing policy are public
3. Full policy public: Full policy, incl. processes, is public and highly transparent

3a. Full policy public + all details accessible: Policy does NOT refer to internal policies that are inaccessible to the public, but outside parties can fully review policy details (within reason)
3b. Effectiveness & Outcome transparency: The company provides details on the number of reports, topics, and follow-up actions, and also effectiveness, e.g., awareness & trust among employees, % of anonymous reports, appeal rates, whistleblower satisfaction, and types of cases received.

This analysis evaluates the quality of companies' whistleblowing policies based on all available evidence. The assessment analyzes 29 sub-indicators across five critical dimensions:

1. reporting channels and access,
2. whistleblower protections,
3. investigation processes,
4. system governance, and
5. AI-specific provisions.

Sub-indicators were derived from international reference standards—ISO 37002:2021, the ICC Guidelines, and the EU Whistleblowing Directive 2019/1937, which establish the gold standard for evaluation. Additional AI-specific items were included to address AI-specific concerns. For each Item, FLI evaluated the available evidence listed in the Whistleblowing Policy Transparency’ indicator and rated the degree to which a company's policy satisfies it on a scale from 0 to 10, based on the publicly available information listed in the indicator on whistleblowing policy transparency, which includes whistleblowing policies, codes of conduct, safety frameworks, and survey responses. Where no information was available, 0 points were assigned. The assessment measures how well firms' policies align with best practices while specifically examining whether companies have implemented specialized AI safety provisions, such as protections for reporting violations of safety frameworks.

This indicator evaluates whether an AI developer fosters a climate in which employees can raise safety‑relevant concerns without fear of retaliation and with confidence that the concerns will be addressed. Evidence is drawn from (i) the organisation’s track‑record of documented whistleblowing cases, (ii) the use, scope, and enforcement of non‑disclosure or non‑disparagement agreements (NDAs), (iii) leadership signals that encourage or discourage internal dissent, (iv) third‑party evidence of psychological safety, and (v) patterns of safety information leaking externally (vi) departures linked to safety governance. The focus is on demonstrated behaviour and outcomes rather than written policy statements. For whistleblowing incidents, we report individual names, concerns raised, and company response & status where available.

Notes of Best Practice: Companies should show a clear recent pattern of protecting and acting on employee safety reports; public commitment not to enforce legacy NDAs for safety topics; leadership statements praising internal critics; ≥ one anonymized psychological‑safety survey with ≥ 70 % of staff agreeing “I can raise safety concerns without fear” and no credible retaliation cases in the last 24 months. Little public leaks as issues are addressed internally. Recent evidence (≤ 24 months) should be weighted twice as heavily as older cases to reward reforms.

This indicator reports whether companies publicly disclose the system prompts used in their most capable deployed AI models. Evidence includes published system prompts in model cards, technical documentation, or dedicated transparency pages, and changelogs. Best practice involves publishing exact prompts used in production, version history, verification that prompts are used in production, and explanations of design choices.

This indicator assesses whether companies publish detailed specifications outlining their models' intended behaviors, boundaries, and decision-making frameworks. For companies that shared such documents, we provide high-level summaries and link to the sources. We include documents that concretely outline the goals, values, and behavioral guidelines that developers aim to instill in their models. Documentation should explain how developers want their models to handle various scenarios, conflicts, and edge cases, and detail how these values are implemented, including metrics or evidence of how well these values are achieved in practice. Specifications should ideally be current and include a tracked version history with dates. Important aspects are specificity, comprehensiveness across use cases, and inclusion of concrete examples. Internal training documents, vague mission statements, and brief high-level descriptions are not in scope.

The G7 Hiroshima AI Process (HAIP) Reporting Framework is a voluntary transparency mechanism launched in February 2025 for organizations developing advanced AI systems. Organizations complete a comprehensive questionnaire covering seven areas of AI safety and governance practices, including risk assessment, security measures, transparency reporting, and incident management. All submissions are published in full on the OECD transparency platform. This indicator tracks whether firms participated in HAIP as a measure of their commitment to AI safety transparency.

We report which companies have engaged with our index survey to voluntarily disclose additional information. Full survey responses are linked below.

This indicator evaluates incident reporting commitments, frameworks, and track records. For frameworks and commitments, the indicator assesses whether companies have publicly discussed any systems and commitments to share critical information about red-line incidents or capabilities with government bodies (e.g., US CAISI, UK AISI), peer organizations, or the public. Such incidents can include successful large-scale misuse, near-miss events, scheming by AI models, and identified model capabilities with severe national security implications. The indicator further tracks relevant incident documentations that the company has already shared. Evidence comes from safety frameworks, documented reporting procedures, participation in information-sharing agreements, and public incident reports.

Notes on Best Practice: Clear public commitments to report specific categories of incidents to government bodies, with documented procedures for incident classification and escalation. Information-sharing agreements with disclosed scope, publishing reports on recent incidents, demonstrating transparency about warning signs discovered during development, and establishing clear thresholds for mandatory reporting, specificity, and comprehensiveness of reporting commitments.

Assesses the extent to which companies and their leadership (A) publicly recognise the potential for catastrophic AI harm and (B) proactively disseminate evidence-based analyses of such risks to external stakeholders. The criteria are frequency, specificity, and prominence of communication about AI's potential for catastrophic outcomes (including existential risks, mass casualties, or societal-scale disruption). Evidence includes official blogs, testimonies, leadership communications, including signed statements. Excludes technical safety papers, model cards, and formal safety frameworks (captured in separate indicators).

Note: The research methodology for this indicator did not follow a formal structure; results are incomplete and likely biased/skewed by the prominence of different media reports.

The assessed companies aim to develop AGI/superintelligence, and many expect to achieve this goal in the next 2–5 years. This indicator evaluates whether companies have published comprehensive, concrete strategies for managing catastrophic risks from these transformative AI systems. We assess the depth, specificity, and credibility of publicly available plans.
We examine official company documents, research papers, and blog posts that articulate safety strategies. We report the most relevant documents, briefly summarize their content, and provide links for detailed reading. Safety frameworks are mentioned for completeness and are fully evaluated in the relevant domain. We note whether documents are declared strategies by leadership or proposals by researchers from a safety team. We strive to keep document summaries proportional to document length and relevance for the safety strategy. Safety frameworks are only noted briefly and evaluated in another domain. Documents that primarily provide recommendations to other actors (e.g., governments) are outside the scope.

Key components:

Technical Alignment and Control Plan:
- Given the short timelines to AGI and the magnitude of the risk, companies should ideally have credible, detailed agendas that are highly likely to solve the core alignment and control problems for AGI/Superintelligence very soon.

• Companies should be able to demonstrate that they would be able to detect misaligned systems and reliably prevent them from escaping human control, and have formulated clear protocols for how they will handle serious warning signs of misalignment.

AGI Planning:
- Companies should have detailed plans for managing the transition when AI matches or exceeds human capabilities in critical domains and enables large-scale dual-use risks. They should specify clear criteria for when they would halt development/deployment.

• Companies should develop concrete, detailed roadmaps to achieve sufficient cyber-defence capabilities to protect against attacks from terrorist organizations or resourced state actors before critically dangerous systems are developed.

Post-AGI Governance:
- Companies should provide clear descriptions of how they would govern AGI/Superintelligence or how they will enable societal control. The company also should have developed reliable protocols that would prevent insiders from using Superintelligent systems to seize political power.

• Companies should specify how extreme power concentration will be prevented and benefits distributed if AI replaces humans in the workplace and causes unprecedented mass unemployment.

Overall, this indicator evaluates whether companies have detailed, actionable strategies that match the extraordinary risks they acknowledge when building systems intended to exceed human intelligence.

This indicator evaluates whether companies have implemented or prepared monitoring and control systems to detect and prevent risks from misalignment during internal deployment. Companies are assessed on whether they have concrete implementation plans tied to specific capability thresholds, published methodologies for control evaluations, and protocols for investigating potential scheming or deceptive alignment. General statements about monitoring without specific technical details, thresholds, or implementation timelines are insufficient. Research about monitoring without statements on implementation plans or status is out of scope.

This indicator tracks research publications on technical AI safety research that are relevant to extreme risks. More specifically, the indicator is a collection of work that is plausibly helpful for averting large-scale risks from misalignment or misuse. This includes mechanistic interpretability, scalable oversight, unlearning, model organisms of misalignment, model evaluations on dangerous capabilities or alignment, and others.

The collection also includes substantial outputs besides papers—weights, tools, code, transcripts, data—but these are almost always published as part of a paper. Excluded are capability-focused research, papers on hallucinations, and model cards. The full collection was created by Zach Stein-Perlman and is available here—numbers for DeepSeek, ZhipuAI, and xAI added by FLI.

This indicator assesses the extent to which companies invest in and support external AI safety research through a range of mechanisms. Evidence may include: (1) Mentorship programs—participation in formal initiatives such as the Machine Learning Alignment Theory Scholars (MATS) program, the number of mentors provided, and the existence of company-specific fellowships; (2) Research grants and funding—provision of financial support or subsidized API access to safety researchers, including grants and targeted funding programs; and (3) Deep model access for safety researchers—offering privileged access that goes beyond public APIs, such as employee-level permissions, early access to unreleased models, safety-mitigation-free versions for testing, fine-tuning rights on frontier models, and allocated compute resources.