Public policy wonks and aficionados of bureaucracy swear by evidence-based decision making. However, it is also undeniable that critical public policies seldom achieve intended outcomes.
More often, public policies are associated with either unintended consequences or ambiguous outcomes. Whenever policies succeed or fail, it is unclear for whom and why.
For example, the free primary education policy enacted in all EAC member states has not produced access everywhere. Moreover, learning outcomes are uneven and contested. Various agricultural policy provisions have not improved national food and nutritional security or eliminated the need to import primary food commodities. Moreover, policies and programmes aimed at improving maternal, neonatal and child health have produced mixed results.
Sound evidence is necessary but not sufficient to produce desired, equitable well-being outcomes as the consequence of policy or programme implementation. The abundance of misalignment of policy objectives and policy outcomes suggests existing evidence-based approaches to policymaking do not address critical elements across the policy cycle, which are necessary for effective programme or policy implementation and results or impact.
Moreover, the context of policymaking has become more complex. Public participation and bottom up actions are becoming increasingly important, often required by law. Citizens are more literate, well informed and actively engaged in the policy formulation and implementation process as well as government decisions and actions. More and more, with the ubiquity of social media platforms and with devolved levels of government, citizens have become agents of good governance and not passive beneficiaries.
Big data — the burst in variety, velocity and volume of data, including growing prominence of unstructured, non-sampled data available on social media — is providing new opportunities for innovation in the policy cycle, especially with a huge potential to improve policy effectiveness.
The emergence of big data has made possible the development of tools, including statistical and geospatial analysis, sentiment analysis and visualisation, which aim to support key stakeholders at every stage of the policy cycle, hence, facilitating hindsight, deepening insight and informing foresight.
While traditional policy analysis has been devoted to retrospective (ex post) analysis, the combination of complexity thinking and big data analytics will enable a critical addition to the toolbox, prospective (ex ante) analysis, which essentially informs foresight, hence, defining expected policy impacts, while enabling near real- time policy/programme evaluation and refinement during implementation.
Policy or programme design and implementation are on the cusp of a fundamental leap forward: from decision-based approaches to an exciting new dawn of intelligence-based models that are powered by big data analytics.
Big data therefore provides an invaluable companion for programme development and design, identification of indicators for programme monitoring and evaluation, as well as stakeholder mapping.
In my view, big data analytics presents an opportunity for policymakers and bureaucrats, as well as big donor agencies to think and design policies and programmes as testable change hypothesis.
Moreover, implementation locations provide unique geographic, socioeconomic and institutional arenas for real world experimentation, testing and learning innovation in policy formulation and implementation.
Alex O Awiti is the director of the East Africa Institute at the Aga Khan University