Alan M Fox is the Deputy Director of the Independent Evaluation Office of UNDP. Appointed to this position in February 2021, he has been at the office since 2009, serving as a member of the management team and chief of section for corporate – thematic evaluation. Alan’s public service career spans 4 decades, with projects and evaluations managed in over 60 countries. In 1985 he was appointed Executive Assistant to the Mayor of Cleveland Ohio, and in 1989 became the Associate Assistant Administrator for Water at the US Environmental Protection Agency. Prior to joining UNDP, Alan held the position of Vice President and Regional Director, Southern Africa, in the Infrastructure Division of the international engineering and consulting firm Pőyry PLC (Finland). Alan is an honors graduate in Political Science from Case Western Reserve University (USA), with an MBA from the University of Pretoria (South Africa).
The rapid pace of digitalization in development, spurred by the COVID-19 pandemic, is still accelerating. Momentum grows with every new innovation, propelling UN agencies to harness the latest technological powers in response to the formidable challenges of our times. By creating AIDA – or Artificial Intelligence for Development Analytics – the UNDP Independent Evaluation Office has established itself as a frontrunner.
AIDA is a powerful tool. It harnesses artificial intelligence and machine learning to help development practitioners make sense of vast quantities of information that would take months to explore manually. With AIDA, it takes seconds. The platform is being continually improved since its launch in late 2022. The awareness of its potential continues to grow, and AIDA was selected for presentation at this year’s UN World Data Forum in Hangzhou, China.
Some 10,000 data experts congregated online and in person at the UNWDF this year, with the intention of inspiring innovation and forging alliances. AIDA was one of the flagship products presented by UNDP, and one of four sessions on AI at the Forum. AI is becoming part of everyday life, and people everywhere are increasingly curious, and willing to use it. The ChatGPT phenomenon illustrates this perfectly.
The UNWDF brought together a huge number of players to discuss issues, innovations and solutions to common obstacles. This year, AIDA’s ability to extract evidence from unstructured data drew considerable interest from development practitioners, data scientists and private sector companies; all people struggling with manipulating large data sets. Among the AI applications on show, AIDA was the only one diving into qualitative data archives.
"People don’t read anymore"1
Almost a decade ago, the World Bank published a study exposing the rate at which the pdfs in their archives were downloaded. With one third of the documents never having been downloaded, and only a fraction downloaded more than a handful of times, a lot of important policy information was going unread, and unused. The paper acknowledges this from the outset, opening with the words ‘knowledge is central to development’.
The irony is explicit: knowledge is central to development. The IEO has more than 6,000 evaluations in its archives, all in pdf format. Moreover, evaluation reports are dense documents. The evidence policy makers need to make informed decisions can be hidden within them. So how do we make sure those tens of thousands of lessons and recommendations are not lost? AIDA has the power not just to extricate the evidence, but to understand what it means in context.
Does AIDA dream of ChatGPT?
Since ChatGPT exploded onto the scene in November, AI has been thrust into the spotlight. Opinion pieces abound on generative AI, stridently taking sides on the debate. In this polarized maelstrom of opinions, AIDA gently brings us back down to earth.
Let’s be clear: AIDA is not a generative AI tool. ChatGPT is a large language model that uses huge quantities of data from indiscriminate sources, which can include mis- and dis-information on the internet. AIDA is a tailored tool built for the specific purpose of harvesting knowledge to improve the effectiveness of the work done by organizations like UNDP to increase human development. It is a narrow application of supervised machine learning that searches evaluative archives to find the vital nuggets of evidence that underpin the important decisions that are made by development practitioners. The sandbox in which AIDA can search contains only triangulated and verified information. Its searches are not affected by misinformation circulating elsewhere on the internet, because it cannot look outside of this sandbox. It simply isn’t programmed that way.
Unlike ChatGPT, AIDA does not generate language; it searches for evidence and summarizes what it finds, thereby avoiding the associated risk of hallucination. To do this, AIDA was first taught to understand the nuances of UN language, so it could extract the sentiments couched in paragraphs of technical and diplomatic language. As anyone familiar with UN documents will attest, this is not a minor thing.
AIDA is alone in understanding these particular linguistic nuances. That’s not to say there is nothing to be gained by augmenting AIDA with some of the skills of a generative AI model. Complementing AIDA’s smart search capabilities and capacity to understand subtleties with ChatGPT’s ability to summarize is being considered as the platform develops.
Integrating a generative AI model into the AIDA platform inevitably raises some ethical questions. Ethical and responsible uses of data and data equity ran through discussions at the UNWDF, and the core principles of ‘do no harm’ and ‘leave no one behind’ should never fall at the sidelines of digital advances for development. At the IEO, we take these issues very seriously, and take deliberate steps to ensure these advances are ethical and transparent. AIDA’s sandbox comes into its own here. If we do integrate a generative AI model it will have to work inside the sandbox.
1Steve Jobs on Amazon’s then-new product, Kindle, in 2008.
