comment 0

Serbia’s long journey to joining the EU and what the implications are for evidence informed policy making. A conversation with Nenad Čelarević

Eighteen years have passed since the big demonstrations that took place in the streets of Belgrade and which ultimately led to the overthrow of Slobodan Milošević on 5th October 2000.  Today Serbia is on a new journey which is not without challenges. Ten years ago, the government applied for membership to the EU and in 2012 the accession process kick started a large number of policy reforms required to join the EU. At the same time, former British ambassador to Yugoslavia, Ivor Roberts, has warned in an article published by the Guardian that the west must not just abandon the Balkans to Russia’s embrace.

How is Serbia doing in its journey to joining the EU? What role does the research community play in shaping and informing the reform processes? I have discussed these questions with Nenad Čelarević, Deputy Project Manager & Serbia Programme Officer with the PERFORM: Performing and Responsive Social Sciences project. Nenad is a political activist who is actively engaged in the political transition of the country and who has worked with civil society organizations on issues related to human rights and youth participation in the democratic transition of the country.


Nenad Čelarević…engaged in the conversation about the future of Serbia in the EU 

I travelled to Belgrade last November. It was my first time in the city and in Serbia. I did not know what to expect. My memories, growing up in Italy, were of the Balkan war and of nationalistic rhetoric of president Milošević. What are the main changes that you have seen since 2000.

A lot has changed in Serbia and in the Western Balkans. The main change for me have been elections. The trigger for the big protests against Milošević in October 2000 were allegations of electoral fraud. We now have, for the first time in our history, democratic elections to form our parliament (of course, still not a perfect process and with room for improvements). The governments that have emerged have undertaken policy reforms of almost all areas of social life. As in most countries, not all these reforms have been successful, but they have been designed and implemented through peaceful means and with the overall goal of improving the standard of living in Serbia. This is, in my opinion, the most significant change since the Milošević regime.

In addition to internal reforms, the fall of Milošević has also broken the international isolation of Serbia.

Yes, Serbia has re-engaged with the international community. The application for EU membership, alongside the same request by other Western Balkan countries, was a significant step.

Serbia could join the EU in 2025. It seems to me that the accession process is a massive bureaucratic exercise. There are 35 areas of reforms (or Chapters) that need to be fulfilled in a relatively short period of time. What do you think?

The process of joining the EU is as important as or even more important than the goal of being in the EU. I think that for the Serbian government, the negotiation process is in itself an opportunity to continue strengthening democratic values and the rule of law in our society. To me it is not so important for Serbia to join the EU in 2025. What matters is that we join when we are ready to do so. Being a member of the EU requires high administrative capability at national and local level as well as in the independent bodies that guarantee the check and balances vis-à-vis the state. It takes time to develop these capabilities.

I see two elements that are particularly crucial for the accession process. The first is that Serbia is the largest country in the Western Balkans to join the EU. Sooner or later, Serbia will be asked, as part of the EU, to support other countries in the region with their accession processes. One of these countries is likely to be Kosovo. The 35 accession chapters include a measure that is specific only to Serbia: the normalization of relations with Kosovo. To me, the development of a sustainable and peaceful relation between these two countries is a prerequisite for the enlargement of the EU and for the economic growth and social development in the whole region.

My second point concerns the balance of power in the negotiation process. On one side, there are 28 member states (soon 27 as the UK leaves the EU). On the other, the six countries of the Western Balkans. This division of power will inevitably result in a slow decision-making process. The only way to speed up this process is if the six Western Balkan countries align and coordinate their reform processes. A unity of intents in the region will allow a better negotiation with the EU block.

We both work on processes and systems for evidence-informed policy making. I think we share the belief that good quality and timely research-based evidence can help policy debate and policymaking. Is the research sector included in the 35 Chapters?

In PERFORM we are specifically interested in how the research sector, particularly social sciences, is contributing to the reform processes and the EU accession. While research is included in the 35 Chapters, it is a bit sidelined. The current reforms do not tackle sufficiently the blockages of the research system.  Moreover, the rhetoric against experts’ opinions, which have emerged in some Western democracies, is not helpful for making the case for a stronger evidence-informed policy system in Serbia.

Is the culture of demand and use of evidence in government changing as part of the accession process? If so, how?

If I think about the last 10 years, the way evidence informs specific policy decisions depends on the politics of the policy being debated and the power of line ministries vis-à-vis the overall strategy and direction set by the parliament and the government. It is a spectrum between two extremes: on one end, there are policy decisions where evidence is produced by the government for the government, without much participation by external actors such as researchers. On the other end, there are policy decisions where the government demands evidence and engages with academics, researchers, and activists.

In my opinion, at this point in time, the issue of using research to inform policy (i.e. production, demand, use, etc.) is not a priority. There are a number of important reforms which are highly political and where values and ideology play a stronger role than evidence does. One of these, as I said earlier, is the relation with Kosovo.

Having said this, I think there are areas in the reform process where greater attention is paid to the governance of the demand and use of evidence. For example, the public administration reform and the reform of higher education institutions.

Let’s try to look into the future. If we will have this conversation in 2028 what do you think you will say?

In 2028, I imagine Serbia being a country where young and mid-career researchers from overseas work and conduct their research and where researchers inform policy and debate. I imagine also a Western Balkans region which is more integrated than it is today and where regional disputes and tensions are something of the past, to be found only in history books. I imagine all six countries being members of a reformed EU, which is more democratic and with higher degree of solidarity among member states. I believe that Serbian scientists, researchers, and policy practitioners from different fields can contribute to these changes and shape the future of Serbia, the region, and the EU.



comment 0

What joining the European Social Survey means for social science research and evidence-informed policy making in Serbia. In conversation with Professor Dragan Stanojević.

The Faculty of Philosophy at the University of Belgrade has a long history. It was established in 1838 and today it is located in a modern building in downtown Belgrade. I went there to meet Dragan Stanojević, Assistant Professor at the Department of Sociology. I wanted to meet him because he led the initiative that resulted in Serbia joining the European Social Survey in 2017. The ESS, as it is known, is the most comprehensive cross-national survey that measures the attitudes, beliefs, and behavior patterns of the European population. I wanted to talk to him to understand what the ESS means for the Serbian research community and its significance for evidence informed policy making in the country.


As an academician and a social science researcher, what are the main changes you have observed during the last 15 years in Serbia? 

After a decade of political, economic, and international isolation during the nineties, the early 2000s marked the beginning of a new page in the history of the country with opening towards the European Union and also new opportunities for researchers and universities to be part of international research projects. However, the economic crisis which started in 2008, has somehow slowed down these positive changes and today Serbia, as well as other countries in the Western Balkans and Southern Europe, suffers from a weakening of social welfare systems and a migration of young and educated people overseas which involves also researchers from our universities.

What do you think is the current contribution that social science research is making to policy decisions in Serbia?

Policy impact is not always the main goal for academic researchers. At the same time, academic research does often address important policy issues such as social inequality, social inclusion (or exclusion), and economic growth. Unfortunately, the policy contribution of social sciences research in Serbia is not yet sufficiently recognized among researchers and policy actors.

Currently, there is limited discussion about the contribution that research from universities and social science institutes can make to policy decisions. Most of the policy research used in policy debates comes from think-thanks, with policies mainly designed for and driven by the process of joining the European Union. When universities do get involved, it is mainly because of projects and programmes funded by international development partners.

You have successfully led a team of social science researchers that has worked for 12 months for Serbia to join the European Social Survey? Why is this significant?

The European Social Survey is conducted under the auspices of the European Research Infrastructure Consortium. Methodologically and theoretically, it is the best international comparative and longitudinal social surveys run within the European Union.  For Serbia to join the countries involved in the ESS is significant for three main reasons:

  • Serbia can now bring the methodological rigour and expertise of the ESS into the Serbian research community and contribute to strengthen research and analytical capabilities
  • It will help Serbian researchers to gain a higher profile within the international research community and strengthen the participation of Serbian academic research organisations (rather than simply individual researchers) in international networks. This in turn can inform and influence European policies as well as national policies
  • The access to cross-country comparable datasets and analysis will strengthen Serbian government capability to design and enact public policies required by the process of EU accession.

What do you think were the key factors that helped to bring Serbia into the ESS and for you and your team to succeed?

The Serbian research community has tried for the last 15 years to bring Serbia into the ESS without succeeding. The political interest was simply not there. There main factors that in my opinion explain why we succeeded this time are:

  • The convening and facilitation role of the Performing and Responsive Social Sciences project (PERFORM) has been very important. The project provided the necessary financial support to support the process and, importantly, has taken the role of a broker between the research community and government stakeholders
  • The commitment and enthusiasm by the team of researchers driving the initiative which included young and mid-career researchers, which shows to me that the future is bright for social science research in Serbia
  • The involvement of the Ministry of Education, Science and Technological Development (URL). The ministry recognized the importance of joining the ESS and provided the political buy-in and financial resources that were needed to succeed.

Now that we are part of the ESS, we have an opportunity to generate good quality and longitudinal social development data. Up until now social changes were not tracked systematically over time. This means that on many social issues and problems we cannot say where we stand compared to other European countries. The ESS will give us, for the first time, a chance to generating such data and contribute to designing, costing, implementing and monitoring more evidence-informed policy measures to address social problems the country faces. It is a big step forward for Serbia.


Thank you Professor Dragan Stanojević.

comment 0

Programación adaptativa

"TripleAD": Aprendiendo a Aprender para el Desarrollo

Arnaldo Pellini aporta el pasado abril  un post con el título de una de sus presentaciones: la programación adaptativa a escala: tres desafíos y tres posibles soluciones.

El punto principal de su presentación fue que los programas con un gran presupuesto, grandes equipos de implementación, que operan en un contexto complejo y que abordan problemas duros difíciles, etc. pueden tener más dificultades  que pequeños programas para aplicar los principios de la programación adaptativa. Los desafíos son:

View original post 256 more words

comment 0

Building state capability for evidence-informed policymaking in Albania: In conversation with Abi Dodbiba

Growing up in Italy during the 1970s and 1980s, I remember Albania being geographically very close, just 150 kilometres across the sea from the harbour city of Brindisi. At the same time, closed and isolated by its communist regime, it seemed very far away. Many fleeing Albanians used small boats for the dangerous crossing to Italian shores. Then the Berlin Wall fell bringing with it the Albanian regime and, after the government was put under considerable pressure by students and workers, the first multiparty election took place in 1991.

Last November, carrying these memories with me, I made my first trip to Albania’s capital, Tirana, to work with the PERFORM – Performing and Responsive Social Sciences project and meet Abi Dodbiba, who leads the project team in Albania.

Albania is one of seven countries preparing to join the European Union (EU) after 2020. The process of EU accession requires many policy and regulatory reforms (so-called ‘Chapters’) and I was curious to discuss the transition with Abi, and looking to understand how the country’s research sector is preparing for it.

Evidence-informed policymaking requires good quality knowledge production, knowledge intermediaries, and a regulatory framework that creates incentives for doing (policy) research as well as using research results in policy decisions. Where do you think Albania is right now?

Albania is slowly starting to build these systems, together with the culture of using research and evidence to support policy decisions. The evolution of the knowledge system in the country is closely linked to the history of the country before and after the regime change in the early 1990s.

Prior to 1991, policy research organisations and universities existed under the tutelage of the regime, serving the political propaganda and centrally planned economy. After the fall of the regime in 1991, new and independent research organisations and think tanks emerged and, over time, they gained some international visibility because they were part of international networks and research collaborations.

For universities, however, it has been more challenging. Efforts to reform higher education and scientific research have lacked continuity and have not resulted in a strong and internationally-connected research community. Funding for academic research is very limited, which has an impact on the salaries of academic staff, on access to international literature and participation in international conferences, and on publication in internationally recognised journals. The capacity to apply for and manage external research funding is also limited. As a result, many university researchers seek independent consultancy assignments, which further constrains the potential of academic research institutions.

How is Albania preparing for the necessary knowledge system reforms ahead of its accession to the EU?

Albania’s Law on Higher Education and Scientific Research was amended in 2015; it now stipulates that the ministry responsible for education should draft a yearly ‘Document of Priorities’ listing the government’s policy research priorities. This is intended to guide budget allocation for research funding. It is also envisaged that the document will be created through consultation with policy makers, as well as researchers.

The establishment of a research and policy development unit within the Office of the Prime Minister provides another example of Albania’s preparation. The unit was in charge of the processes and systems necessary for collecting evidence to inform and coordinate inter-sectoral policy decisions. Following a restructuring process in October 2017, its functions have been absorbed by other units but the overall commitment to improving evidence-informed policy-making practices has been preserved.

Policy documents have also been developed to reflect European standards. For example, in anticipation of future negotiations on Science and Research, the Ministry of Education, Sport and Youth adopted the National Strategy for Scientific Research, Technology and Innovation 2017-2022. This embraces all the key principles of the European Research Area, including fostering collaboration between the research, private, and public sectors.

That said, much still needs to be done, especially regarding the strengthening of a culture where demand and consultation of research becomes organic rather than an action required by regulations and legislation. I also think it’s important that academics are more present within the teams negotiating the different chapters of the EU accession, because research will be required to inform policy decisions for each chapter.

Evidence-to-policy initiatives have traditionally focused on strengthening capacity to produce research and evidence, rather than strengthening the demand for and use of knowledge. What do you think about this?

My work with PERFORM in Albania has taught me that that it is crucial to work simultaneously on evidence production and evidence demand capabilities. In other words, helping policy makers to articulate their evidence needs and, at the same time, enabling researchers to produce good quality, relevant, and timely research.

Having said this, a key element of the puzzle is politics. Even if a policy-maker or a government unit has been able to obtain the right type of evidence of the desired quality, it may well happen that politics trumps the analysis. It cannot be assumed that research results will be used by policy makers.

Keeping that in mind helps us to be adaptive and creative in the initiatives we implement. This is what we are trying to do with the government partners with whom we collaborate.

This blog was originally posted on ODI Insight 

Photo credit: Skanderbeg Square in Tirana by vil.sandi

comment 0

Can policy makers catch up with the 4IR?

Lately, I have been reading more about the challenges that the Fourth Industrial Revolution (4IR) is bringing to policy making and governance. The pace of the changes is what is challenging governance institutions. It is like policy makers are catching up with the consequences and impact of the new technologies and as the pace of change accelerates, catching up becomes harder.


Photo: Maurizio Pesce on Flickr

I though about this when reading  and watching some of the commentaries on Mark Zuckerberg’s testimony to Senate and Congress in the US this week.

On NYT Day 2 of Mark Zuckerberg’s Testimony: What to Watch For :

“The Senate hearing made clear that lawmakers aren’t quite sure what Facebook’s business model is or how it works, including what the difference is between selling user data to advertisers and allowing advertisers to target ads to an aggregated slice of Facebook users.

The technological gap between Silicon Valley and Washington was apparent when Senator Roger Wicker, a Republican of Mississippi, asked about internet regulation.

Mr. Zuckerberg explained that when thinking about regulations, government officials need to differentiate between internet companies like his and broadband providers, the companies that build and run the “pipes” that carry internet traffic, like AT&T and Comcast.

The difference is at the heart of net neutrality, a hotly debated regulation that was overturned last year. The rules prevent internet service providers from favoring the flow of all internet content through their pipes.

Mr. Zuckerberg explained that when thinking about regulations, government officials need to differentiate between internet companies like his and broadband providers, the companies that build and run the “pipes” that carry internet traffic, like AT&T and Comcast.

The difference is at the heart of net neutrality, a hotly debated regulation that was overturned last year. The rules prevent internet service providers from favoring the flow of all internet content through their pipes.

“I think in general the expectations that people have of the pipes are somewhat different from the platforms,” Mr. Zuckerberg said.

“When you say pipes, you mean?” Mr. Wicker asked”

On Wired: If Congress Doesn’t Understand Facebook, What Hope Do Its Users Have?

“But the apology rang hollow for members of Congress who said they’d heard it all. “We’ve seen the apology tour before,” senator Richard Blumenthal told Zuckerberg. As proof, he summoned an oversized poster board featuring just a sampling of Zuckerberg’s past apologies in big block lettering. “This was a big mistake on our part, and I’m sorry for it,” read one Zuckerberg quote from 2006. “I’m the first to admit that we’ve made a bunch of mistakes,” read another from 2011.

With that, Blumenthal made it clear that lawmakers didn’t want to hear Zuckerberg apologize. They just wanted to understand how this whole thing works—something Facebook’s users deserve to know as well.”

Did senators questioning Facebook’s Mark Zuckerberg understand the internet? – video

If  lawmakers do not understand the technology, how can they design legislation and  policies that ensure that, as we move fast into the 4IR, economic growth benefits all, natural resources are preserved, and that the political debate is open, transparent, and fair. How to make sure that people come really first.

comment 0

Adaptive programming at scale: a discussion at the Ministry for Foreign Affairs of Finland

I was in Helsinki last week for a presentation and discussion at the Ministry for Foreign Affairs. This was part of the yearly Studia Generalia, a series of seminars organised by the Ministry and the Finnish Society for Development Research where researchers have the opportunity to meet staff of the Ministry and discuss various aspects of development cooperation and development policy. The title of my presentation was Adaptive programming at scale: three challenges and three possible solutions.um_logo_2 (1)

Since I live now in Finland, it was a great opportunity to also hear about adaptive programming in the Finnish development cooperation.

The main point of my presentation was that programmes with a large budget, large implementing teams, operating in a complex context, and tackling wicked hard problems, etc. can struggle more than smaller programmes to apply the principles of adaptive programming.

The challenges are:

1) The challenge of governance and management: large programmes plan activities usually over 12 months and submit the plan to a Steering Committee for approval. A lot of work goes into doing that as well as in the approval process. To make changes afterwards it is not easy, and programmes tend to stick to the plan. Possible solutions could be detailed plans covering less than 12 months or the moving budgets across financial years.

2) The challenge of skills within the team: large programmes tend go through an inception period where most of the programme team is hired. This is done before knowing what specific problems the programme will address and how. Some roles are needed by default:  finance management, grants management, communication, etc. Other roles, however, are more linked to the pilots and experimentations that the programme will conduct during its implementation which are not known at the beginning. There are experiences with flexible contracting modalities that show that for specific policy experiments it is possible to get the team on board with the right networks and political skills.

3) The challenge of monitoring and learning: an adaptive approach requires a learning culture within the programme team because without learning and good monitoring it is difficult to generate the knowledge and information that can help to change course, adapt activities, or stop policy experiments that do not work. Investing in developing that learning culture is important.

Helsinki harbour

Photo by Jonathan on Flickr

We had a nice and interesting discussion around these challenges and some possible solutions for about one hour. Here my three main takeaways:

1) Bilateral programmes funded through the Finnish development cooperation have a good degree of flexibility embedded in design and implementation. The programme design document is not considered as set in stone. If the evidence is there, changes to the design and implementation plans are possible. This flexibility is included in the description of the role and function of the Steering Committees established to oversee  programmes at country level. This is a good foundation.

2) Not all activities in large programmes are likely to require an adaptive approach. A mapping of key activities can help to identify which ones require an experimental and iterative approach and which ones do not. This can then be reflected in the annual plans where experimental activities can be only sketched, while other activities can be described in more detail.

3) Finland is known internationally for it experimental approach to public policies. The Universal Basic Income has caught the international media attention. The Prime Minister’s Office has set up the Experimental Finland web page. There is scope, I think, to bring that experimental culture into development policy and programming. This is in addition of having pilots within programmes. It is about designing small experimental  projects which address wicked hard problems, and which can be scaled up at later stage based on the evidence and learning acquired through the experimentation. There is more that can be explored in this area, linking, for example, to the insights from the growth of start-ups in which Finland has also considerable experience.

comment 0

Articles and papers I liked

20180221-AP- Harmela_ice_lake-175547-FINAL_EDIT_BW

Here is an image of the beautiful winter we are having in Tampere to accompany the articles and papers I enjoyed reading last week.

Duncan Green published When does Tech → Innovation? Here’s what 178 projects tell us. The blog is about a study of the the results of 178 small grants to tech projects funded under the Making All Voices Count accountability programme over 4.5 years. Fourteen messages emerged from the study. Here are two messages that I found particularly to the point:

  • Message 6. Technologies can create new spaces for engagement between citizen and state
  • Message 7. Technologies can help to empower citizens and strengthen their agency for engagement

A lot is being written about Facebook, Cambridge Analytica, and the dangers to our democracies. Here is one article, but there are many more: Facebook: is it time we all deleted our accounts?

An article that I found striking earlier in the week, was published by Wired about the incredible power Facebook has as a media organisation. Inside the two years that shook Facebook-and the world shows how FB is a media company that has yet to wake up to the notion that it is one.

I heard an interesting comment on the BBC World Service in one of the programmes about the Cambridge Analytica saga: our democracies were built about 200 years ago. The new digital technologies are incompatible with that model. We need a new forms of politics and probably of democracy


The World Economic Forum has written this interesting article about how the fourth industrial revolution is changing policy making: How can policy keep pace with the Fourth Industrial Revolution?. In sum: policy making has to become much more experimental, agile, adaptive. Sounds familiar, isn’t it?

The Guardian columnist George Monbiot has written a courageous article on cancer: I have prostate cancer. But I am happy.


comment 0

A conversation about policy experiments

The search for ways to design policies that help to solve societal problems is continuously evolving. The traditional evidence-based approach to policy making involves generating research results and using these to inform policy decisions.

This blog post is based on an interview with Mikko Annala, Head of Governance Innovation at Demos Helsinki. It was originally posted on the Think Tank Initiative Blog.

Road smaller

Think tanks around the world play an essential role in the knowledge ecosystem: they generate creative approaches grounded in local realities and use this evidence to inform policy-making. Through my experience working at the Knowledge Sector Initiative in Indonesia I have seen how research and advocacy by think tanks can inform and change policy. For example, we documented how the Institute for Research and Empowerment made a crucial contribution to the passing in 2013 of the new Village Law. In addition, SurveyMETER, a think tank based in Yogyakarta, conducted research which, in the Municipality of Balikpapan, informed plans to invest in better urban infrastructures for the elderly. As a result, I’ve seen the impact that think tanks in the global South are making.

I’m now living in Finland and am interested in the innovative approach to policy experiments they’re adopting here, as this may be of interest to others engaged in similar work. Policy experiments take a different approach: they test solutions and then generate evidence about what works and what does not work, which can inform policies and government programmes. But, what are the key factors that characterise an experimental culture in government? Are policy experiments something new in policymaking? To learn more, I got in touch with Mikko Annala, Head of Governance Innovation at Demos Helsinki, a Finnish independent think tank which is involved, among other things, in policy experiments with the Finnish government as well as overseas.

Arnaldo Pellini: what are policy experiments?

Mikko Annala: “Policy experiments are usually defined as initiatives that help ministries and government departments test new ways to solve policy problems within a limited scale, and within a set timeframe.

I find this definition limiting. It presents experimentation merely as a tool for discovering and testing new solutions. I think that there is more to it.

In our policy experiment work with the Prime Minister’s Office, we have realised that experimentation can also contribute to building trust between citizens and policy-makers. The dialogue and debate around the design of the experiments, the alignment with the government policy goals, the trade-offs between different solutions, and so on, helps to share different perspectives and points of views. In turn, this helps to strengthen trust towards policy institutions.  One example is TAIKA – Taidetta kansalle (MAGIC – Art for the People), a policy experiment that seeks to develop new human-centric ways for connecting arts to social and welfare services in Finland and see this can promote better health and well-being.

My definition of policy experiments includes:

  • Uncertainty of outcomes.
  • The courage to find new, and sometimes even radical, solutions to social problems.
  • Building learning within experiments by setting clear objectives and measurements.
  • A limited timeframe and scale.
  • Open and active inclusion of different stakeholders from the design stage.”

Are policy experiments new?

“Policy experiments are not new, but a lot has been happening in this field lately.

During the 1930s, President Roosevelt led a policy experimentation boom in the United States. But in my opinion, it is only in the last 10 years or so that we have seen a new wave of policy experimentation emerging across the world.

Today the term ‘experimentation’ is not only used by experts, but also by politicians, policy-makers and civil servants.

This is not a coincidence. In my opinion, there has never been a greater need for experimentation in public policy as today. Change in our societies and economies is happening at a fast pace. The complexity of the world is rapidly increasing and will continue to do so.

There is an increasing need to develop faster and more reliable learning processes to solve problems and, at the same time, strengthen the trust between citizen and policy institutions. Policy experiments are a great way to do this.”

Experiments will sometimes fail, failure is not always welcome

“Policy experiments require certain capabilities within government departments and, importantly, a culture that supports policy experimentation. When the culture is not there, the shift can be difficult.

This is why it is important to document and communicate good experimental practices. There are three elements which are a key to experimental culture:

1) Incentives. For example, requiring ministries to commit and use a certain part of their budgets to conduct evaluations of policies and programmes enables, in my experience, a greater openness towards experimental methods within a bureaucracy.

2) Leadership. An experimental culture, requires a high level of leadership and opening up of public discussions about the benefits of experimentation, as well as the value of failing and learning.

3) Results. Policy experiments have to try to show success pretty fast, which is easier if flexibility, adaptation and a limited scale is part of the design.”

You seem to take a problem driven approach in your report to the Prime Minister’s Office, Design for Government: Human-centric governance through experiments. Is that true?

“The approach we suggest in the paper starts from understanding the problem. But let me explain how we suggest going about it.

First, review the research and literature around the problem and map how much is already known.

Second, we research interesting practices that seems to address, and potentially solve, the same challenge or parts of it.

Third, and perhaps most importantly, we build a dialogue with individuals and stakeholders related to the problem.

In our experience, (co)defining the problem among stakeholders increases trust towards the experimentation process and the risks it entails. It also reduces the complexity of problems, because the experts planning the experiments don’t have to do so much guesswork and can actually learn a lot about the problem through interaction with the stakeholders.

Having said this, I also think that one should not fall in love with the problem. After a certain level of elaboration, a team has to come up with potential solutions and start testing whether they work or not!”

What types of evidence do experiments produce?

“Without measurements, we can’t tell what works and what doesn’t. In other words, we can’t learn from what we are doing.

Policy experiments should always try to generate different types of evidence and in my experience, it pays off to remain patient. With sufficient sample sizes, for example, randomised control trials can provide useful evidence through pre- and post-experiment analysis.

At the same time, some policy experiments may have to deliver results and evidence quickly. In those cases, policy experiments need to develop ‘quick and dirty’ learning methods that enable them to learn without major time and resource investment.”

Policy experiments are an approach to test policy solutions and, at the same time, generate evidence that can inform policy decisions around solutions to very policy problems. Being an approach, rather than a model, they can be applied to different contexts and policy issues in a very flexible way. The experiences in Finland, which I discussed with Mikko Annala, can provide ideas and suggestions for other countries, though, as always, the answers and solutions have to be found locally.

comment 0

Has Italy turned right?



The first results of today’s parliamentary election are being broadcasted by media and newspapers’ TV channels. As of today the Movimento 5 Stelle is the first party in Italy . It is followed by the centre-right coalition with Lega Nord and Silvio Berlusconi’s Forza Italia. The centre left party which was leading the government has collapsed to 19%. Italy has turned to the right following other European countries. The campaign centred on immigration and ‘Italy first’ while unemployment in the South is at 29%, the debt to GDP at 130% second only to Greece, economic growth sluggish. We will see in the next few months what this election means for Europe as well.

comment 1

Looking at evidence-informed policymaking through the ‘dissonance’ lens

Last week I attended  the Development Days Conference 2018 in Helsinki. This is a yearly event organised by the Finnish Society for Development Research. This year’s topic was: The Politics of Sustainability: Re-thinking resources, values and justice.


It is a nice and informal event which brings together researchers, activists, and practitioners from all over Finland and overseas.

Sian Sullivan of Bath Spa University gave a very interesting presentation sharing her thoughts on Dissonance Cognitive Theory drawing from the groundbreaking work by Leon Festinger in the mid 1950s . Festinger’s theory challenged Action-opinion Theories which proposed that actions can influence subsequent beliefs and attitudes. His Dissonance Cognitive Theory, turned things around and highlighted the sense of discomfort that people feel when confronting situations involving new knowledge and conflicting beliefs or behaviours.

The fundamental assumptions of Dissonance Cognitive Theory are:

1. Humans are sensitive to inconsistencies between actions and belief.
2. Recognition of this inconsistency causes dissonance, and motivates an individual to resolve the dissonance. Once an individual recognises that one of her/his beliefs are being challenges, the instil is not to just say “oh well, that is fine”. There is usually some sort of mental anguish about this.
3. The degree of dissonance varies with the importance of the beliefs.

How do people resolve dissonances? The theory suggests three main ways:

1. Change beliefs (difficult but not impossible)
2. Make sure that you never do this action again (and do not feel the discomfort)
3. A third and more complex method of resolution is to change the way they view/remember/perceive their action, in other words “rationalise” the actions (eg, I think CO2 is contributing to climate change, yet I take a plane to fly to Helsinki to attend a conference because it is important that I share about my research).

While listing to the presentation I was wondering to what extent we who are working on evidence-informed policy systems are influenced by Action-opinion Theories and the notion of a causal links between new evidence and knowledge produced by researchers, policy analysts, etc. and changes in the beliefs, attitudes, behaviours, decisions by policy makers.

If we look through the Dissonance Theory lens at the relationships between knowledge and policymaking, what we see is resistance, rationalisation of strongly held beliefs, justification of the refusal to accept new facts and knowledge, strengthening of biases, and discomfort. Do we pay enough attention to this? The alternative, of course, is not stop producing knowledge to inform policy. I believe that knowledge is needed and required to inform policy decisions (my bias) but we need also to be aware that one trait of human psychology is to confirm an rationalise biases and not to be open to new knowledge and facts and be ready to change actions.

The researchers’ mindset may be more geared to asking questions and challenging biases and assumptions. But we cannot assume that everybody’s mindset (esp. policy makers) is the one of John Maynard Keynes who famously said, “When the facts change, I change my mind. What do you do, sir?”.

It seems to me that what Dissonance Theory tells us working on evidence-informed policy space, is to be aware of our biases and become more creative in designing ways to bring knowledge into policy, in addition to the research occurring in academia and think tanks. Policy experiments are one way which has gained considerable traction. Other ways include: collaboration between researchers and knowledge intermediaries has great potential; packaging knowledge to communicate through different channels and platforms is important; convene networks and working groups which bring together researchers and policymakers to develop policy research agendas.