How tech companies, journalists, and policymakers can prevent AI decision-making from going wrong.

• Artificial Intelligence
• Governance of AI
• AI Auditing

The ADL AI Index evaluates leading large language models (LLMs) on their ability to detect and counter antisemitic and extremist tropes and narratives.

• Artificial Intelligence
• Evaluate impact
• Governance of AI

The AI Incident Database (AIID) tracks instances of ethical

• AI Auditing

EDIA es un proyecto de la Fundación Vía Libre cuyo objetivo es involucrar a más personas en la evaluación de tecnologías de inteligencia artificial como son los modelos de lenguaje (por ejemplo, ChatGPT).

• Artificial Intelligence
• Watchdogging and transparency
• Watchdogging government

This study examines 51 Freedom of Information requests to reveal how the UK’s Department for Work and Pensions uses opaque data-driven fraud detection systems in welfare, highlighting the limited transparency, the role of journalists and charities in uncovering details, and inconsistencies in the department’s disclosures.

• Artificial Intelligence
• Academia
• Peer-reviewed research

Lighthouse Reports series reveals discrimination in welfare surveillance algorithms

• Govtech
• Access social benefits
• AI Auditing

This paper argues that automated decision-making in UK public administration lacks adequate scrutiny, and proposes regulatory safeguards through mandatory pre-deployment impact assessments and algorithmic auditing to strengthen accountability in Parliament and the courts.

• Govtech
• Evaluate impact
• Academia

AI Evaluation Made Easy

• Artificial Intelligence
• Evaluate impact
• AI Auditing

Worker-led audits in the platform economy

• labor

• Artificial Intelligence
• Research and policy centers
• Crowdsourcing

An open-source Python library designed to assist developers in calculating fairness metrics and assessing bias in machine learning models.

• Artificial Intelligence
• Ethical tech and responsible tech
• AI Auditing

Soon Ranking Digital Rights will release the Generative AI Accountability Scorecard, evaluating major consumer-facing generative AI services’ respect for the human rights to privacy, non-discrimination, freedom of expression, and freedom of information.

• Artificial Intelligence
• Ethical tech and responsible tech
• Evaluate impact

Humane Intelligence is a tech nonprofit building a community of practice around algorithmic evaluations.

• Artificial Intelligence
• Ethical tech and responsible tech
• AI Auditing

In Spain, the level of risk to which a victim of gender violence is subjected is determined by an algorithm. The system to which it belongs is VioGén which, with more than 3 million risk evaluations, is the risk assessment system with the most registered cases in the world.

• Gender-Based Violence (GBV)

• Artificial Intelligence
• AI Auditing

Since 2016, the Laura Robot has analyzed more than 8.6 million visits in 40 clinical and hospital centers in several Brazilian states.

• Good health and well-being

• Artificial Intelligence
• AI Auditing

The success of Koa Health and Eticas serves as a beacon for the broader industry, emphasizing the importance of ethical considerations when developing mental health solutions.

• mental health

• Artificial Intelligence
• AI Auditing

The report explores Uber, Cabify and Bolt’s compliance with competition, labor and consumer protection laws in Spain.

• mobility
• Reduced inequalities

• Artificial Intelligence
• AI Auditing

An In-Depth Audit of Biases in Facial Recognition Technology Impacting Individuals with Disabilities

• Gender Equality

• Artificial Intelligence
• Assistive and Accessible Tech
• AI Auditing

What is the impact of social media on the representation and voice of migrants and refugees in Europe? What are the challenges and opportunities to avoid their invisibilization and promote a fair representation?

• YouTube

• Artificial Intelligence
• Media and social media analysis
• AI Auditing

Eticas audited the algorithmic system that predicts homelessness risk in Allegheny County, Pennsylvania, USA.

• homelessness

• Artificial Intelligence
• AI Auditing

What is the impact of social media on the representation and voice of migrants and refugees in Europe? What are the challenges and opportunities to avoid their invisibilization and promote a fair representation?

• TikTok

• Artificial Intelligence
• Media and social media analysis
• AI Auditing

ACLU's crowd-sourced public tracker of bias audits of automated employment decision tools (AEDTs) released by employers related to NYC's Local Law 144.

• labor

• Artificial Intelligence
• AI Auditing

Community-led AI Audits: Making AI Systems Fair & Accountable

• CTFG partners

• Artificial Intelligence
• AI Auditing

The OASI Register compiles information about algorithms with the aim of increasing public awareness and providing the necessary knowledge for an informed public conversation, and of making possible for experts and the public to search and analyse the use and social impact of algorithmic systems across the world.

• Artificial Intelligence
• AI Auditing
• Algorithmic transparency

To examine and analyze the downside risks associated with the ubiquitous advance of AI & Automation, to engage in risk mitigation and ensure the optimal outcome… ForHumanity.

• Artificial Intelligence
• Governance of AI
• AI Auditing

Eticas Foundation's adversarial audit of an AI-powered criminal justice tool, RisCanvi, that has been used in Catalonia since 2009 finds it "does not meet the required standards of reliability and fairness."

• recidivism
• reports
• criminal justice

• Artificial Intelligence
• Open justice
• AI Auditing

The MLCommons AI Safety working group is composed of a global consortium of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to AI safety.

• Artificial Intelligence
• Ethical tech and responsible tech
• Generative AI

The IAAA is a community of practice that aims to advance and organise the algorithmic auditing profession, promote AI auditing standards, certify best practices and contribute to the emergence of Responsible AI.

• Artificial Intelligence
• Ethical tech and responsible tech
• Peer networks

AI Forensics is a European non-profit that investigates influential and opaque algorithms. We hold major technology platforms accountable by conducting independent and high-profile technical investigations to uncover and expose the harms caused by their algorithms.

• Artificial Intelligence
• Advocacy tech
• Adjacent Fields

We now bring this decade’s worth of insights and our intellectual capital to scale with our automated audits, and proprietary demographic and geographic database used to ensure the validity (and compliance) of the outputs.

• CTFG partners

• Artificial Intelligence
• Adjacent Fields
• Ethical tech and responsible tech

Philadelphia should think twice about its risk-assessment algorithm.

• criminal justice

• The Tech
• Artificial Intelligence
• Govtech

Eticas teams up with organizations to identify black box algorithmic vulnerabilities and retrains AI-powered technology with better source data and content.

• Ethical tech and responsible tech
• Civic and social impact companies
• AI Auditing