Accountability in the context of geodata and AI technologies used for humanitarian action: a spotlight on the newly minted Dr Masinde

As AI technologies become more accessible and various industries find applications for AI, a general question that arises is what would accountability entail? A famous internal memo from IBM in the late '70s reads: “A Computer Can Never Be Held Accountable.” And these words are very prescient in this AI summer. We have long known that only organizations and developers are accountable. But exactly what accountability would entail remains an open question.

For his PhD (which he defended on 18 May), Brian K. Masinde studied accountability in the context of geodata and AI technologies used for humanitarian action, particularly in disaster preparedness where AI is increasingly being used to determine when and whom should be given aid in the event of a natural disaster. In contrast to other applications of AI for the public, in this case, it concerns perhaps the most vulnerable in society who may not have the voice to advocate for their digital rights. 

Two primary concerns demanding accountability emerged. One was privacy and the other biases. Privacy because AI being data-intensive, it becomes the quintessential surveillance apparatus. For biases, well, data and AI are not neutral tools and are not only prone to biases that are present in society, but can also unwittingly create biases through their failures.

By default, when it comes to privacy, the conversation focuses on the individual. But Brian’s PhD recognized the deficiency of this take, particularly for the kinds of data technologies used by humanitarian organizations. For example, satellite images and Unmanned Aerial Vehicles (UAV) images can contain little to no personal information, but this does not mean that there are no privacy implications, especially when AI is involved. Banking on group privacy theory, his dissertation explored various informational harms that can compromise the privacy of groups and not just the individual. 

Informational harms simply mean the kinds of harms that can arise from collecting, processing, and sharing information. Examples of informational harms that were glaring include biases/discrimination, misuse, data mosaicing (that is extra information leaks that are feasible from combining seemingly unrelated data). 

On biases, it is important to note that humanitarians would not want to be biased or unfair on purpose. But rather how biases can trickle into their data technologies. A good example is using AI to map communities from satellite images or UAV images whereby there’s potential for misclassification, misidentification, or lack thereof making communities invisible. 

Both of these concerns show that we should be wary of AI and data technologies in general not only because of their failures but also from the successes that come from their implementation. 

Mechanisms for accountability in the face of group privacy threats.

Consider an emergency post or hospital, you walk in and a health professional assesses you; just how severe is your situation? Are your injuries so bad you need to go to surgery straightaway or an ICU bed, maybe severe but all you need now is a bed and several nights stay for observation or you can go home after a few stitches and painkillers. This is called a triage and it’s an effective system of allocating resources under constraints (e.g., financial and personnel). 

Brian developed a triage system that humanitarians can use under time and personnel constraints to identify group privacy concerns from the AI technologies they use. At the very least, it ensures that due diligence is done, for example, data minimization, secure storage, and sharing only with organizations that share the values of humanitarianism. 

Mechanisms for accountability to mitigate biases. 

Brian’s PhD proposes auditing as a mechanism for accountability when it comes to biases. This would entail auditing the data used as well as how the AI system affects different groups of people. An upside of audits is that it creates a culture of meticulous documentation of the datasets, processing methods, and performance of the AI and this makes it a bit easier in pinpointing the causes of biases.

However, auditing as a mechanism for accountability is limited. One can perhaps only assess in retrospect what could have been done better to mitigate the biases. In his research, Brian also considers causal AI which has better approaches to avoiding biases by design. Causality as opposed to the widely used associational AI has the potential to birth better reasoning AI.

What’s the relationship between Ethics and Accountability?

A particularly important lesson that should be carried from Brian’s research is that aiming for accountability, fairness, or transparency in AI can be misleading if we do not have larger ethical values (such as virtues or deontological values) to anchor on.  For example, accountability can be a half measure if there is no notion of justice ingrained in it. In Brian’s research, accountability was anchored on the humanitarian values of humanity, impartiality, and generally to “do no harm.” 

Brian’s PhD research is indicative of our commitment at EthicEdge where we aid organizations with the development of accountability checks for their AI systems. This could be through threat modeling for various AI-related risks (and data technologies in general) and also through engaging with design processes to proactively mitigate the risks.