comment 0

Can there be too much data to inform policy and what skills are needed for the policymaking of tomorrow

One of the things I enjoy doing during the summer in Finland is to work in the garden, in the late afternoon, and listen to a LSE public lecture podcast.

Last week I listened to two really interesting ones.

In the first podcast, Tracking the Rise in Global Economic Inequality: new evidence from the world inequality report 2018Lucas Chancel presented the main findings of the first World Inequality Report for 2018.  The report provides the first systemic assessment of globalization in terms of income and wealth inequality since 1980. The discussants were Duncan Green, Paul Segal, and Rebecca Simson.

LSE.png copy 2

In one sentence, the main finding of the report is that ‘income inequality has increased in nearly all world regions in recent decades, but at different speeds. The fact that inequality levels are so different among countries, even when countries share similar levels of development, highlights the important roles that national policies and institutions play in shaping inequality.’

To me,  these findings highlight the importance of developing the state capabilities for monitoring inequality at country, regional, and global level to inform national and  international policy decisions. The report’s webpage makes also available a rich set of data that can be accessed by researchers, practitioners, and civil servants to do some more analysis if they need or want. Will they?

Duncan Green, during his round of comments, made the point that while to have access to the data and the findings of the the report is important, it is also true that today there is a lot of data, analysis, and reports which are produced to inform public policy, maybe even too much.  There is plenty of data produced by development programme to inform government decisions. In other words, there is a lot of supply of evidence.  What is seems to be missing,  is a better understanding of the politics of change and the politics of policy decision making which by nature are very context specific.

To me this does not reduce the importance of producing data and analysis but is more like a call for researchers and practitioners and development agencies to get into the politics and uncertainties of policymaking.



Nick Clegg (Wikipedia)

In the second podcast, Flying the Flag for Openness: why liberalism still matter,  Sir Nick Clegg sets out the case for liberal values at a time of stark social and generational divisions. Towards the end of the Q&A, Clegg was asked by Professor Tony Travers, what skills should a school of public policy (such as the LSE) teach to the policy makers of tomorrow. Clegg said that based on his experience in government as deputy Prime Minister, one of the most critical skills is the understanding  of the trade-offs of any public policy decisions, big or small. Civil servants and policy makers are faced on a daily basis with many choices they have to make. Research, analysis, data, etc. can provide some help. However, the skill that often is most useful (and should be taught more) is about understanding the trade-off of those decisions: less budget for some programmes, the need to increase taxes, policies in some parts of the country but not in other, etc. The skill of understanding and managing trade-offs, according to Nick Clegg, is the one of the most important skills required in the future as the complexity of society and policy decisions continues to increase.

An interesting point, I thought.

Photo by Linus Nylund on Unsplash
comment 0

What is policy impact? Well, it depends from what we mean by policy change

During the last few weeks I have been working on the  design of an adaptive monitoring and learning framework to assess contribution to policy change on climate adaptation in few countries.  Working on the framework made me re-read some articles and papers about policy change which I had read some years ago. Below the key points I used for the development of the framework.

Photo by Linus Nylund on Unsplash

Photo by Linus Nylund on Unsplash

‘Rather than seeing policy as one single, discrete decision, it is important to broaden one’s view, so that policy is understood as a series of documents and decisions that are best described as a set of processes, activities or actions (Neilson, 2001).[1] If we accept this interpretation, it follows that throughout the policy decision process there are different changes that can be identified as the objective of a policy influence strategy or activity.

Jones and Villar (2008)[2] draw on the study that Keck and Sikkink[3] conducted in 1998 on transnational advocacy and the policy process, to describe different types of policy change or policy impact:

  • Framing the debate and getting issues on to the political agenda: this is about producing and communication evidence that draws the attention of policy makers, public opinion, and the media to new issues/solutions. It can result also in policy makers acquiring new knowledge about a specific policy problem. For example, in 2010 the World Bank published a two years study on governance institutions in Vietnam (Vietnam Development Report 2010: Vietnam development report 2010: Modern Institutions) which received considerable attention from media and prompted responses and comments from government officials on the issue of upward/downward accountability and what worked and what did not work well in the policies of devolution pursued by the government.
  • Attitudinal change is about drawing attention to changes in the way key policy stakeholders think about evidence-informed policymaking. For example, the Prime Minister’s Office expresses interest in setting up a policy analysis unit to coordinate the evidence strategy of the different ministries. A Ministry of Planning draws plans to establish a data analytics unit to test ways to use data analytics/dashboards to monitor forest fires.
  • Policy content change is about actual changes in legislation/regulations. For example, in Indonesia after four years of work involving academics, think tanks, the Ministry of Higher Education, the National procurement Agency, the Office of the President, etc., President Jokowi has signed on 15. March 2018 and new Presidential Decree (Perpres – No. 16/2018) that allows Indonesian non-government organisations to be procured by government organisations to conduct studies and policy research. The policy influence team worked and advised on the actual text of the Perpres.
  • Behavioural changes refer to changes in the way of working of government institutions with regard to the demand and use of evidence or the change in behaviour that a new legislation produces. This is the most comprehensive change, which requires more time to achieve and is more difficult to prove. For example, the Prime Minister’s Office in Finland in 2017 allocated 10ml Euro for 42 research topics requested by ministries. The research topics were defined by working groups involving policy makers, academics, and members of civil society organisations. In the Philippines, the Asia Foundation facilitated the process of establishing a coalition involving lawyers, land titling experts, NGOs, and researchers to amend the land titling legislation. President Arroyo signed the new legislation in March 2010. As a result, 50-60.000 land titles are being issued every year (up from ca. 10.000 per year) which has enabled families in urban disadvantaged areas to back up loans requests to banks and to start businesses.[4]

[1] Nielson, S. (2001) ‘Knowledge Utilization and Public Policy Processes: A Literature Review’, IDRC Evaluation Unit. Available at:

[2] Jones, N. with Villar, E. (2008) ‘Situating children in international development policy: challenges involved in successful evidence-informed policy influencing’ in Evidence and Policy, vol4, no.1: p53-73.

[3] Keck, M. and Sikkink, K. (1998) Activists beyond borders: advocacy networks in international politics. Ithaca, NY: Cornell University Press

[4] Watch the video at

comment 0

Reference societies and adaptive development: a conversation with Daniel Suryadarma about Indonesia’s participation in the PISA educational assessment

During a seminar organised by the EduKnow research group at the University of Tampere on comparative education, I came across the interesting concept of reference society. In a nutshell, countries which, like Finland, find themselves at the top of the PISA ranking may become models or references for countries looking for ways to improve their education policies and results.

The PISA ranking has a considerable policy influence internationally but, at the same time, the idea of reference societies seems a bit at odds with the emerging consensus that policy solutions to public policy problems have to be politically-aware, context specific, and that best-practices from elsewhere often lead to failures.

Researching to inform policy … Daniel Suryadarma

I got in touch with Daniel Suryadarma, who is the deputy team leader for the Research on Improving Systems of Education programme in Indonesia and a Research Associate at the SMERU research institute in Jakarta, to discuss about Indonesia’s involvement in the PISA assessment and the reference countries which may influence Indonesia’s education policy.

Daniel, maybe we can start by briefly describing what are the objectives of RISE programme and what kind of activities are implemented by the programme.

RISE is a research programme which focuses on how to make system-level changes that can result in improved learning. The research takes place in six countries: Ethiopia, India, Indonesia, Pakistan, Tanzania, and Vietnam. There are country research teams which focus on system-level questions which are adapted to the different country context. In Indonesia, for example, we focus our research and analysis on teachers by evaluating the impact of different national-level reforms on learning. At the same time, we collaborate with some local governments to design, test, and evaluate different policies and programmes aimed at improving learning. At the local government level, we focus on teachers and system-level questions that are relevant for our local government partners. I think that given Indonesia’s decentralized education service delivery, where local governments have significant freedom to design their policies and programmes, this approach is quite appropriate.

Indonesia has been part of PISA for a number of years now. What do you think is the main motivation for the government to be included in this international ranking?

I do not speak for the government, but I think the motivation differs by agencies, even within the Ministry of Education and Culture. The Assessment Research Centre, for example, is probably interested in having an internationally comparable measure of the quality of education to inform the design of robust assessments instruments to evaluate different learning domains. The Policy Research Centre is perhaps interested in an international assessment which can highlight if national education policies are having a positive effect on learning. The Ministry of Finance, for example, is probably interested in looking whether the increased investment in basic education results in improved learning. I have to admit, though, regardless of the motivation, the fact that Indonesia has continued to participate in PISA since 2003, even with all the criticisms directed at them due to the low ranking of the country in the PISA results, needs to be applauded.

I mentioned to you the concept of reference societies. What are the countries Indonesia is looking at in terms of education policies?

I think there is increasing tendency among policymakers in Indonesia, as well as donor agencies, to stop trying to transplant best practices from other countries to Indonesia. There have been hundreds of such attempts over the past forty years, with little success. In any case, with a country as large as Indonesia, with more than 500 autonomous districts spread across more than 17 thousand islands, it would be quite hard to determine what ‘Indonesian context’ means. This was partly the motivation for the decentralization reform that started in 2001 and which shifted the responsibilities to deliver basic education and health services from the central government to the district governments (although the central government still provides financial transfers to the district to deliver these services). This means that districts have some autonomy in designing their education systems and how to implement national education policies. Ultimately, elected district leaders are responsible for the performance of their education system. Therefore, while five to ten years ago we would still hear stories about the success of Finland or South Korea with their education policies and how Indonesia should adopt practices from those countries, we do not really hear those arguments anymore. 

So, the reference society idea does not apply anymore to Indonesia. What policy changes have occurred in Indonesia as a result of PISA which ranks Indonesia against OECD+ countries? 

I think the PISA results in 2003 provided the motivation for the government and the parliament to revise the law on education and the law on teachers in 2005. In addition, the Indonesian Constitution was amended to incorporate the 20-percent rule. Basically, the government has to invest at least 20 percent of the national budget on education. This is the only sector whose level of public investment is set by the constitution. The PISA ranking has therefore had some influence on education policy development in Indonesia. Overall, I think the government has to be praised for continuing to participate in international tests despite the unflattering results, and for their openness to discuss these challenges with researchers, donor agencies, and other stakeholders. I truly believe that eventually, Indonesia will get effective education policies in place.


Thank you, Daniel Suryadarma.

comment 0

Knowledge, policy, and adaptive development in Indonesia. A discussion at UNICEF’s Office of Research-Innocenti

I was recently in Florence to meet the team at the UNICEF’s Office of Research-Innocenti and present & discuss some of the experiences of the Knowledge Sector Initiative in Indonesia. I was part of the implementing team during phase 1 from 2013-2017 leading the research and learning work of the programme. The Knowledge Sector Initiative is now in its second phase which will run until mid 2022. I remain engaged managing the Overseas Development Institute’s inputs to the programme.

My research background is in education policy in international development and I have followed for some years the research work of the Innocenti Centre. However, this was the first time for me at the Innocenti Center. The sense of history struck me when entering the building.

The Innocenti Centre is hosted in a section of the nearly 600-year-old Ospedale degli Innocenti which was established in 1419 to house and care for the city’s orphans and abandoned children. The building was designed by Filippo Brunelleschi.

Adaptive and sustainable? The Ospedale degli Innocenti opened in 1419.

Adaptive and sustainable? The Ospedale degli Innocenti opened in 1419. (Arnaldo Pellini)

The Ospedale degli Innocenti is arguably the oldest operating children’s care institution in the world. During both World Wars it offered shelter for orphaned children as well as refugee families. Innocenti continues to provide temporary shelter for local women and children at risk. It is a reminder of the good that has been done by Italian institutions amidst the current refugees’ crisis and the battle between political parties around it.

The UNICEF International Child Development Centre was opened in 1988 with a broad mandate and with research quickly becoming the central component of its mission. Within UNICEF, the Innocenti Centre generates evidence and knowledge on a wide range of cutting-edge children’s issues to inform debate, development programming, and policymaking at country level. The current research themes are: child poverty, equity and well-being; child protection; children and the internet; adolescent well-being; children on the move and education policy.


The Centre is de facto UNICEF’s think tank and over the last five years it has invested in strengthening its profile and visibility within UNICEF and with the international community of researchers and development practitioners.

UNICEF country programmes and the Innocenti Centre collaborate to design and develop systems and processes that support an evidence-informed approach to programming and policymaking. Therefore, the team in Florence was interested to hear about the design and implementation of the Knowledge Sector Initiative.

I really enjoyed the discussion we had with the team as well as the preparation for the presentation. It gave me an opportunity to run through the journey of the programme from the initial idea, back in 2009, to the activities of the design phase which lasted until 2012, the first phase of the programme until 2017, the adaptation to a changing context and circumstances. It was great to revisit the knowledge-to-policy system approach of the programme which suggests that to address problems of limited use of evidence in policymaking, may require to experiment simultaneously with knowledge producers, government actors who demand and use evidence, knowledge intermediaries, and the rule and regulations that support the interaction between knowledge producers and policy makers.

The knowledge to policy cycle

The knowledge to policy cycle (KSI)

It is this knowledge system hypothesis that lend itself to building a close collaboration with government and non-government partners and to an adaptive approach to design and implement pilot activities and experiments which generate learning about what works and what does not work and why.

A knowledge system approach, and the contribution that evidence and knowledge can make to inform policy decisions, could provide interesting opportunities for the Innocenti Centre in its collaboration with UNICEF’s country programmes and national level policy initiatives.

I really enjoyed meeting the UNICEF team at Innocenti and learning about their work streams and policy research work and look forward to continuing the conversation.


comment 1

MLE or MEL in adaptive programming?

I was in Manila last week to attend the practitioners’ forum organised by the Asia Foundation on Adaptive Programming and Monitoring, Evaluation, and Learning.

A really good workshop where to share of experiences about tailoring MEL systems to fit the accountability demands from funders and support the adaptation and iteration of programmes. In attendance there were large and small development programmes from the region, funders, implementing organisations, advisors, and researchers.

I gave a presentation at the beginning of day one where I shared my personal experience in this area as well as findings from past and ongoing projects implemented by colleagues at ODI such as David Booth, Alina Rocha Menocal, Tiina Pasanen, Anne Buffardi and others.

One point I made was that the natural sequencing in the day-to-day of a programme is actually Monitoring, Learning and, more intermittently, Evaluation. So, MLE rather than MEL.

Jaime Faustino presented the six simple tools used by TAF’s Coalition for Change  project. These tools show that it is possible to design useful and (at the same time) simple tools for monitoring and learning. An interesting panel on ToCs and their use through an evolving programme. Some of the presentation focused on how programmes design feedback loops to inform the re-design of activities. There was a very interesting and open discussion about what counts as contribution to higher level outcomes for an adaptive and programme.

The Asia Foundation is working on a report of the workshop.

Sharing, discussing, learning .... Asia Foundation

Sharing, discussing, learning …. Asia Foundation

Many interesting points that I am still processing. Here my takeaways:

  • I have participated in few of the adaptive programming and DDD meetings over the last couple of years. I found that this workshop, with its focus on MLE, was a step forward in the discussion. Earlier meetings where more general, which was to be expected as the discussion around adaptive programming was starting. But now we are starting to look at specific elements of adaptive programmes.
  • Large programmes with large budgets, large teams, tackling wicked hard problems of governance capability have specific challenges and opportunities. They cannot change activities or work streams very quickly but, at the same time, they have sufficient budget to develop experimental and adaptive components and MLE systems as part of their design.
  • There was a discussion about the percentage that programmes allocate to MLE. The range was between 3% to 20%. The average at around 10%. This is probably low to support an adaptive implementation. Without insufficient MLE budgets, the risk is that programmes will be asked hard questions about their contribution to outcomes by funders to which they may struggle to answer without adequate MLE resources.
  • Large programme has to set up things quickly and often they develop ToC and M&E frameworks as one of the first deliverables without too much information about problems, possible solutions, and approach. One way to manage this tension, which was shared by some programmes, is to design MLE systems based on Key Evaluation Questions, letting indicators and means of verification emerge as the programme evolved.
  • Some programmes have shared their work with mobile apps and data analytics (TAF Myanmar / Open Data Labs Jakarta) to collect monitoring data. Interesting and innovative solutions, but overall, current MLE systems do not seem to take full advantage of digital technologies. The Excel spreadsheets are still going strong as well as formats in Word with a lot of time spent on the input and output sides by staff.
  • Linked to the previous point, one of the breakout session discussed what MLE will look like 10-15 years from now. We imagined AI that will give us new opportunities to come up with scenarios and ToCs. Monitoring data will be collected through videos and audio recordings. Software will automatically code, analyse, and synthesize the information. Monitoring data will be better at integrating data analysis and AI analysis of changes in social and political context in which a programme operates. Science fiction? Maybe not.

I learned a lot and left with some new questions. It is important to remember that adaptive programming is a mean towards an end. Not an end in itself. In the same ways, a MLE system is mean to support adaptation and iteration of a programme and provide the funder with the information they require. Being adaptive does not mean to try out solutions without a sense of direction.

The question I have is the following: let’s take the Coalition for Change project and some of the policy reforms that the project has contributed to like the property rights reform. The policy change objective has been achieved and is well documented. It is an important policy reform. Is that sufficient or does Coalition of Change (or similar projects) need always to demonstrate a contribution at an higher outcome level, which answer the so what or what does this all add up to question that funders ask?  I am in two minds about this.


comment 0

Serbia’s long journey to joining the EU and what the implications are for evidence informed policy making. A conversation with Nenad Čelarević

Eighteen years have passed since the big demonstrations that took place in the streets of Belgrade and which ultimately led to the overthrow of Slobodan Milošević on 5th October 2000.  Today Serbia is on a new journey which is not without challenges. Ten years ago, the government applied for membership to the EU and in 2012 the accession process kick started a large number of policy reforms required to join the EU. At the same time, former British ambassador to Yugoslavia, Ivor Roberts, has warned in an article published by the Guardian that the west must not just abandon the Balkans to Russia’s embrace.

How is Serbia doing in its journey to joining the EU? What role does the research community play in shaping and informing the reform processes? I have discussed these questions with Nenad Čelarević, Deputy Project Manager & Serbia Programme Officer with the PERFORM: Performing and Responsive Social Sciences project. Nenad is a political activist who is actively engaged in the political transition of the country and who has worked with civil society organizations on issues related to human rights and youth participation in the democratic transition of the country.


Nenad Čelarević…engaged in the conversation about the future of Serbia in the EU 

I travelled to Belgrade last November. It was my first time in the city and in Serbia. I did not know what to expect. My memories, growing up in Italy, were of the Balkan war and of nationalistic rhetoric of president Milošević. What are the main changes that you have seen since 2000.

A lot has changed in Serbia and in the Western Balkans. The main change for me have been elections. The trigger for the big protests against Milošević in October 2000 were allegations of electoral fraud. We now have, for the first time in our history, democratic elections to form our parliament (of course, still not a perfect process and with room for improvements). The governments that have emerged have undertaken policy reforms of almost all areas of social life. As in most countries, not all these reforms have been successful, but they have been designed and implemented through peaceful means and with the overall goal of improving the standard of living in Serbia. This is, in my opinion, the most significant change since the Milošević regime.

In addition to internal reforms, the fall of Milošević has also broken the international isolation of Serbia.

Yes, Serbia has re-engaged with the international community. The application for EU membership, alongside the same request by other Western Balkan countries, was a significant step.

Serbia could join the EU in 2025. It seems to me that the accession process is a massive bureaucratic exercise. There are 35 areas of reforms (or Chapters) that need to be fulfilled in a relatively short period of time. What do you think?

The process of joining the EU is as important as or even more important than the goal of being in the EU. I think that for the Serbian government, the negotiation process is in itself an opportunity to continue strengthening democratic values and the rule of law in our society. To me it is not so important for Serbia to join the EU in 2025. What matters is that we join when we are ready to do so. Being a member of the EU requires high administrative capability at national and local level as well as in the independent bodies that guarantee the check and balances vis-à-vis the state. It takes time to develop these capabilities.

I see two elements that are particularly crucial for the accession process. The first is that Serbia is the largest country in the Western Balkans to join the EU. Sooner or later, Serbia will be asked, as part of the EU, to support other countries in the region with their accession processes. One of these countries is likely to be Kosovo. The 35 accession chapters include a measure that is specific only to Serbia: the normalization of relations with Kosovo. To me, the development of a sustainable and peaceful relation between these two countries is a prerequisite for the enlargement of the EU and for the economic growth and social development in the whole region.

My second point concerns the balance of power in the negotiation process. On one side, there are 28 member states (soon 27 as the UK leaves the EU). On the other, the six countries of the Western Balkans. This division of power will inevitably result in a slow decision-making process. The only way to speed up this process is if the six Western Balkan countries align and coordinate their reform processes. A unity of intents in the region will allow a better negotiation with the EU block.

We both work on processes and systems for evidence-informed policy making. I think we share the belief that good quality and timely research-based evidence can help policy debate and policymaking. Is the research sector included in the 35 Chapters?

In PERFORM we are specifically interested in how the research sector, particularly social sciences, is contributing to the reform processes and the EU accession. While research is included in the 35 Chapters, it is a bit sidelined. The current reforms do not tackle sufficiently the blockages of the research system.  Moreover, the rhetoric against experts’ opinions, which have emerged in some Western democracies, is not helpful for making the case for a stronger evidence-informed policy system in Serbia.

Is the culture of demand and use of evidence in government changing as part of the accession process? If so, how?

If I think about the last 10 years, the way evidence informs specific policy decisions depends on the politics of the policy being debated and the power of line ministries vis-à-vis the overall strategy and direction set by the parliament and the government. It is a spectrum between two extremes: on one end, there are policy decisions where evidence is produced by the government for the government, without much participation by external actors such as researchers. On the other end, there are policy decisions where the government demands evidence and engages with academics, researchers, and activists.

In my opinion, at this point in time, the issue of using research to inform policy (i.e. production, demand, use, etc.) is not a priority. There are a number of important reforms which are highly political and where values and ideology play a stronger role than evidence does. One of these, as I said earlier, is the relation with Kosovo.

Having said this, I think there are areas in the reform process where greater attention is paid to the governance of the demand and use of evidence. For example, the public administration reform and the reform of higher education institutions.

Let’s try to look into the future. If we will have this conversation in 2028 what do you think you will say?

In 2028, I imagine Serbia being a country where young and mid-career researchers from overseas work and conduct their research and where researchers inform policy and debate. I imagine also a Western Balkans region which is more integrated than it is today and where regional disputes and tensions are something of the past, to be found only in history books. I imagine all six countries being members of a reformed EU, which is more democratic and with higher degree of solidarity among member states. I believe that Serbian scientists, researchers, and policy practitioners from different fields can contribute to these changes and shape the future of Serbia, the region, and the EU.



comment 0

What joining the European Social Survey means for social science research and evidence-informed policy making in Serbia. In conversation with Professor Dragan Stanojević.

The Faculty of Philosophy at the University of Belgrade has a long history. It was established in 1838 and today it is located in a modern building in downtown Belgrade. I went there to meet Dragan Stanojević, Assistant Professor at the Department of Sociology. I wanted to meet him because he led the initiative that resulted in Serbia joining the European Social Survey in 2017. The ESS, as it is known, is the most comprehensive cross-national survey that measures the attitudes, beliefs, and behavior patterns of the European population. I wanted to talk to him to understand what the ESS means for the Serbian research community and its significance for evidence informed policy making in the country.


As an academician and a social science researcher, what are the main changes you have observed during the last 15 years in Serbia? 

After a decade of political, economic, and international isolation during the nineties, the early 2000s marked the beginning of a new page in the history of the country with opening towards the European Union and also new opportunities for researchers and universities to be part of international research projects. However, the economic crisis which started in 2008, has somehow slowed down these positive changes and today Serbia, as well as other countries in the Western Balkans and Southern Europe, suffers from a weakening of social welfare systems and a migration of young and educated people overseas which involves also researchers from our universities.

What do you think is the current contribution that social science research is making to policy decisions in Serbia?

Policy impact is not always the main goal for academic researchers. At the same time, academic research does often address important policy issues such as social inequality, social inclusion (or exclusion), and economic growth. Unfortunately, the policy contribution of social sciences research in Serbia is not yet sufficiently recognized among researchers and policy actors.

Currently, there is limited discussion about the contribution that research from universities and social science institutes can make to policy decisions. Most of the policy research used in policy debates comes from think-thanks, with policies mainly designed for and driven by the process of joining the European Union. When universities do get involved, it is mainly because of projects and programmes funded by international development partners.

You have successfully led a team of social science researchers that has worked for 12 months for Serbia to join the European Social Survey? Why is this significant?

The European Social Survey is conducted under the auspices of the European Research Infrastructure Consortium. Methodologically and theoretically, it is the best international comparative and longitudinal social surveys run within the European Union.  For Serbia to join the countries involved in the ESS is significant for three main reasons:

  • Serbia can now bring the methodological rigour and expertise of the ESS into the Serbian research community and contribute to strengthen research and analytical capabilities
  • It will help Serbian researchers to gain a higher profile within the international research community and strengthen the participation of Serbian academic research organisations (rather than simply individual researchers) in international networks. This in turn can inform and influence European policies as well as national policies
  • The access to cross-country comparable datasets and analysis will strengthen Serbian government capability to design and enact public policies required by the process of EU accession.

What do you think were the key factors that helped to bring Serbia into the ESS and for you and your team to succeed?

The Serbian research community has tried for the last 15 years to bring Serbia into the ESS without succeeding. The political interest was simply not there. There main factors that in my opinion explain why we succeeded this time are:

  • The convening and facilitation role of the Performing and Responsive Social Sciences project (PERFORM) has been very important. The project provided the necessary financial support to support the process and, importantly, has taken the role of a broker between the research community and government stakeholders
  • The commitment and enthusiasm by the team of researchers driving the initiative which included young and mid-career researchers, which shows to me that the future is bright for social science research in Serbia
  • The involvement of the Ministry of Education, Science and Technological Development (URL). The ministry recognized the importance of joining the ESS and provided the political buy-in and financial resources that were needed to succeed.

Now that we are part of the ESS, we have an opportunity to generate good quality and longitudinal social development data. Up until now social changes were not tracked systematically over time. This means that on many social issues and problems we cannot say where we stand compared to other European countries. The ESS will give us, for the first time, a chance to generating such data and contribute to designing, costing, implementing and monitoring more evidence-informed policy measures to address social problems the country faces. It is a big step forward for Serbia.


Thank you Professor Dragan Stanojević.

comment 0

Programación adaptativa

"TripleAD": Aprendiendo a Aprender para el Desarrollo

Arnaldo Pellini aporta el pasado abril  un post con el título de una de sus presentaciones: la programación adaptativa a escala: tres desafíos y tres posibles soluciones.

El punto principal de su presentación fue que los programas con un gran presupuesto, grandes equipos de implementación, que operan en un contexto complejo y que abordan problemas duros difíciles, etc. pueden tener más dificultades  que pequeños programas para aplicar los principios de la programación adaptativa. Los desafíos son:

View original post 256 more words

comment 0

Building state capability for evidence-informed policymaking in Albania: In conversation with Abi Dodbiba

Growing up in Italy during the 1970s and 1980s, I remember Albania being geographically very close, just 150 kilometres across the sea from the harbour city of Brindisi. At the same time, closed and isolated by its communist regime, it seemed very far away. Many fleeing Albanians used small boats for the dangerous crossing to Italian shores. Then the Berlin Wall fell bringing with it the Albanian regime and, after the government was put under considerable pressure by students and workers, the first multiparty election took place in 1991.

Last November, carrying these memories with me, I made my first trip to Albania’s capital, Tirana, to work with the PERFORM – Performing and Responsive Social Sciences project and meet Abi Dodbiba, who leads the project team in Albania.

Albania is one of seven countries preparing to join the European Union (EU) after 2020. The process of EU accession requires many policy and regulatory reforms (so-called ‘Chapters’) and I was curious to discuss the transition with Abi, and looking to understand how the country’s research sector is preparing for it.

Evidence-informed policymaking requires good quality knowledge production, knowledge intermediaries, and a regulatory framework that creates incentives for doing (policy) research as well as using research results in policy decisions. Where do you think Albania is right now?

Albania is slowly starting to build these systems, together with the culture of using research and evidence to support policy decisions. The evolution of the knowledge system in the country is closely linked to the history of the country before and after the regime change in the early 1990s.

Prior to 1991, policy research organisations and universities existed under the tutelage of the regime, serving the political propaganda and centrally planned economy. After the fall of the regime in 1991, new and independent research organisations and think tanks emerged and, over time, they gained some international visibility because they were part of international networks and research collaborations.

For universities, however, it has been more challenging. Efforts to reform higher education and scientific research have lacked continuity and have not resulted in a strong and internationally-connected research community. Funding for academic research is very limited, which has an impact on the salaries of academic staff, on access to international literature and participation in international conferences, and on publication in internationally recognised journals. The capacity to apply for and manage external research funding is also limited. As a result, many university researchers seek independent consultancy assignments, which further constrains the potential of academic research institutions.

How is Albania preparing for the necessary knowledge system reforms ahead of its accession to the EU?

Albania’s Law on Higher Education and Scientific Research was amended in 2015; it now stipulates that the ministry responsible for education should draft a yearly ‘Document of Priorities’ listing the government’s policy research priorities. This is intended to guide budget allocation for research funding. It is also envisaged that the document will be created through consultation with policy makers, as well as researchers.

The establishment of a research and policy development unit within the Office of the Prime Minister provides another example of Albania’s preparation. The unit was in charge of the processes and systems necessary for collecting evidence to inform and coordinate inter-sectoral policy decisions. Following a restructuring process in October 2017, its functions have been absorbed by other units but the overall commitment to improving evidence-informed policy-making practices has been preserved.

Policy documents have also been developed to reflect European standards. For example, in anticipation of future negotiations on Science and Research, the Ministry of Education, Sport and Youth adopted the National Strategy for Scientific Research, Technology and Innovation 2017-2022. This embraces all the key principles of the European Research Area, including fostering collaboration between the research, private, and public sectors.

That said, much still needs to be done, especially regarding the strengthening of a culture where demand and consultation of research becomes organic rather than an action required by regulations and legislation. I also think it’s important that academics are more present within the teams negotiating the different chapters of the EU accession, because research will be required to inform policy decisions for each chapter.

Evidence-to-policy initiatives have traditionally focused on strengthening capacity to produce research and evidence, rather than strengthening the demand for and use of knowledge. What do you think about this?

My work with PERFORM in Albania has taught me that that it is crucial to work simultaneously on evidence production and evidence demand capabilities. In other words, helping policy makers to articulate their evidence needs and, at the same time, enabling researchers to produce good quality, relevant, and timely research.

Having said this, a key element of the puzzle is politics. Even if a policy-maker or a government unit has been able to obtain the right type of evidence of the desired quality, it may well happen that politics trumps the analysis. It cannot be assumed that research results will be used by policy makers.

Keeping that in mind helps us to be adaptive and creative in the initiatives we implement. This is what we are trying to do with the government partners with whom we collaborate.

This blog was originally posted on ODI Insight 

Photo credit: Skanderbeg Square in Tirana by vil.sandi

comment 0

Can policy makers catch up with the 4IR?

Lately, I have been reading more about the challenges that the Fourth Industrial Revolution (4IR) is bringing to policy making and governance. The pace of the changes is what is challenging governance institutions. It is like policy makers are catching up with the consequences and impact of the new technologies and as the pace of change accelerates, catching up becomes harder.


Photo: Maurizio Pesce on Flickr

I though about this when reading  and watching some of the commentaries on Mark Zuckerberg’s testimony to Senate and Congress in the US this week.

On NYT Day 2 of Mark Zuckerberg’s Testimony: What to Watch For :

“The Senate hearing made clear that lawmakers aren’t quite sure what Facebook’s business model is or how it works, including what the difference is between selling user data to advertisers and allowing advertisers to target ads to an aggregated slice of Facebook users.

The technological gap between Silicon Valley and Washington was apparent when Senator Roger Wicker, a Republican of Mississippi, asked about internet regulation.

Mr. Zuckerberg explained that when thinking about regulations, government officials need to differentiate between internet companies like his and broadband providers, the companies that build and run the “pipes” that carry internet traffic, like AT&T and Comcast.

The difference is at the heart of net neutrality, a hotly debated regulation that was overturned last year. The rules prevent internet service providers from favoring the flow of all internet content through their pipes.

Mr. Zuckerberg explained that when thinking about regulations, government officials need to differentiate between internet companies like his and broadband providers, the companies that build and run the “pipes” that carry internet traffic, like AT&T and Comcast.

The difference is at the heart of net neutrality, a hotly debated regulation that was overturned last year. The rules prevent internet service providers from favoring the flow of all internet content through their pipes.

“I think in general the expectations that people have of the pipes are somewhat different from the platforms,” Mr. Zuckerberg said.

“When you say pipes, you mean?” Mr. Wicker asked”

On Wired: If Congress Doesn’t Understand Facebook, What Hope Do Its Users Have?

“But the apology rang hollow for members of Congress who said they’d heard it all. “We’ve seen the apology tour before,” senator Richard Blumenthal told Zuckerberg. As proof, he summoned an oversized poster board featuring just a sampling of Zuckerberg’s past apologies in big block lettering. “This was a big mistake on our part, and I’m sorry for it,” read one Zuckerberg quote from 2006. “I’m the first to admit that we’ve made a bunch of mistakes,” read another from 2011.

With that, Blumenthal made it clear that lawmakers didn’t want to hear Zuckerberg apologize. They just wanted to understand how this whole thing works—something Facebook’s users deserve to know as well.”

Did senators questioning Facebook’s Mark Zuckerberg understand the internet? – video

If  lawmakers do not understand the technology, how can they design legislation and  policies that ensure that, as we move fast into the 4IR, economic growth benefits all, natural resources are preserved, and that the political debate is open, transparent, and fair. How to make sure that people come really first.