Measuring social rights and democratic governance: the Metagora experience

Размещено на http://www.allbest.ru/

Размещено на http://www.allbest.ru/

Measuring social rights and democratic governance: the Metagora experience

Metagora is an based project which focuses on methods, tools and frameworks for measuring human rights, complex socio-economic issues and democratic governance at the national level. It rests on the idea that it is possible, through a variety of tools, to quantitatively measure, assess and monitor specific human rights and socio-economic situations, and that this information can then be used at a local level to feed into the policy-making process and enhance people's living conditions. As such, Metagora is not a technical but a policy-oriented project, which relies on the use of technical tools to impact on policy-making processes.

Metagora was implemented between 2004 and 2008 and consists of seven field activities carried out in different regions of the world. Three pilot surveys were carried out in Mexico City, the Philippines and South Africa; two multi-country surveys were implemented in eight countries of Francophone Africa and in three countries of the Andean Community; and data collection and measurement activities took place in Palestine and in Sri Lanka. Though the thematic focus and characteristics of these seven activities greatly differ, all have a common objective: to provide solid and reliable information to analyse, assess and monitor specific and complex issues of local or national relevance, and to test the capacity of different methodologies to do so.

Metagora addresses and answers a variety of questions revolving around the feasibility of properly measuring human rights and democratic governance, the nature of actors to involve, the methods and conditions upon which such measurements should rely, the purpose of the assessments, etc. Indeed, the debate on the measurability of multi-dimensional rights and governance via statistical and other tools is a long-lasting one. Doubts have been raised in the past regarding the feasibility of measuring such issues properly. The concern that people might not respond to complex and sensitive questions has repeatedly been put forth, along with concerns that the information gathered might not be statistically significant or politically relevant and with doubts as to who can and should be involved in such processes.

As presented in this paper, Metagora provides illustrations of the ways in which governance-related issues and complex human rights, be they civil and political, or social, economic and cultural rights, can be assessed and monitored. Indeed, most aspects of Metagora's approach, methodological considerations and lessons learnt not only apply to the measurement of social rights and personal security, but cover a wide range of human rights and complex socio-economic and governance issues.

Metagora: overall approach and methodological characteristics

In terms of methodology, the originality of Metagora in comparison to other existing international initiatives and projects rests in its bottom-up approach. This approach, opposed to the top-down approach characteristic of most global initiatives, consists of three main elements. First, the identification, in each pilot country and together with stakeholders and experts, of key national issues for which evidence-based assessment is highly-relevant and lacking. Second, the application of statistical methods and tools adapted to that particular context and the assessment of these methods for their capacity to provide policy-relevant results. Third, the production of shared knowledge on the specific policy issues at stake to be used by stakeholders in designing, monitoring and assessing policies.

The bottom-up approach is fundamental in terms of promoting wide multi-disciplinary and inclusive participatory processes, fostering ownership of the assessments by local actors and stakeholders, and ensuring assessments are relevant and scientifically rigorous. These elements are, in turn, essential to make sure that the data and indicators produced are used to inform national policy-making processes, and improve the design, relevance and impact of policies.

Thus, the approach favoured by Metagora is different to the top-down approach upon which most measurement initiatives in the field of human rights and democratic governance tend to rely at the international level. Contrary to most international initiatives, which seek to produce information in different regions of the world, but in a uniform fashion and on the same rights and issues, Metagora is international in nature and scope, but composed of separate and distinct field activities which focus on reflecting national priorities and country specificities. To do so, for each country and specific thematic interest, the most appropriate tool is chosen and implemented. Metagora is more interested in relevance and policy-impact at the national level than in international comparability.

Another element characteristic of Metagora is the belief, shared by its members, that a variety of tools and approaches can usefully be mobilised and combined to refine methodologies and enrich analyses. Faced with the wide variety of existing data-collection methods (i.e. household survey or census data, administrative data, expert or focus group interview data, events-based data, desk study, etc.), as well as with the multi-dimensional nature of many rights and governance-related issues, one of the methodological characteristics of Metagora has been to seek and rely on the complementarity of approaches and data. This paper highlights two ways in which this complementarity can be reached: through the combination of quantitative and qualitative data, and that of objective and subjective data.

Quantitative and qualitative data can usefully interrelate. Metagora has shown that qualitative information is essential to ensure the proper design of survey questionnaires and data to be collected, but also to ensure that statistical analysis focuses on relevant issues and that appropriate contextual frameworks allow for an effective policy-oriented interpretation of quantitative data.

In Mexico, the design of quantitative tools, and in particular of survey questionnaires and databases on the relation between police officers and the population, was based on qualitative information gathered through in-depth narrative interviews with victims of rights infringements, focus group discussions with individuals belonging to the different populations under study, substantive reports of local experts, and consultations with relevant stakeholders. The qualitative data gathered prior to the development and implementation of the survey was crucial in informing its design and content, and ensuring that the dynamics of the violence and lack of trust, characteristic of the relation between law enforcement bodies and the population in Mexico City, were well-captured and understood.

In the Philippines, the combined use of quantitative and qualitative information was also instrumental in the study on the rights of indigenous peoples. Though qualitative data was precious in the Philippines in designing the questionnaire and other quantitative tools, the dual quantitative/qualitative approach was first and foremost critical in the interpretation and validation of the findings. Indeed, multiple qualitative methods (e.g. focus group discussions with target populations) were applied in parallel to the collection of quantitative data and used to complement the analysis emerging from the latter and thus better inform policy-oriented reports and recommendations, ensuring that the research and its results were aligned and responsive to indigenous peoples conceptualisations and focused on salient and meaningful issues. For instance, qualitative data helped clarify noteworthy variations among different tribes of indigenous peoples on specific issues.

In the same way that quantitative and qualitative data can and should interrelate to properly and comprehensively inform monitoring exercises, so-called objective and subjective data should also be used to complement each other. Briefly speaking, objective information deals with observed or experienced facts or situations, whilst subjective information is linked to the perceptions and assessments of the people being surveyed. Though there is a tendency to attribute lesser importance to subjective data, often considered less reliable and seen as «second best», Metagora has shown that both types of data are extremely interesting and relevant to the measurement of rights and governance.

Perceptions are important in understanding local dynamics, concerns and expectations even if, as such, little justification exists for perceptions from an objective point of view. Moreover, when it comes to measuring rights, and enhancing human rights, the complementarity of objective and subjective information can help voice local concerns, thereby furthering the involvement and empowerment of the populations in question. Perceptions are also of real value as they can be early signs of significant problems or issues.

The importance of so-called subjective data is illustrated by the South African survey on land reform. Both objective and subjective data were collected but a particular weight was attached to the opinion and perception of individuals regarding agricultural reforms and, in particular, land reform as developed in the post-apartheid period. The analysis of the data has shown that, as implemented, the reform has failed to live up to popular expectations and, though its main rationale was to redress some of the inequalities dating back to the apartheid, it has not brought about significant improvement in the lives of the black population. Moreover, based on the opinion of a variety of groups of the population (farmers, workers, blacks and whites), the study shows that the reform does not respond to - and is not in line with - the demands and expectations of the population. This negative assessment of the land reform policy is at odds with the conclusion reached by studies which focus on evaluating more objective elements of the reform (i.e. its compliance with the targets and objectives initially set out).

Metagora: main lessons learnt

Based on Metagora's field activities and overall experience, a number of lessons can be drawn. This paper highlights two key lessons that, far from being exclusive, reflect the main value-added of this project. First and foremost, Metagora has shown that measuring rights and governance is both technically feasible and politically relevant.

As illustrated by the study in Mexico City, which focuses on police abuse and personal security, Metagora proves that it is possible to enquire and obtain reliable data about sensitive issues using a household survey but also that, based on that information, it is possible - for the first time in Mexico - to move from discussions as to whether abuse and ill-treatment on behalf of law enforcement authorities is a serious problem or not to an informed dialogue based on the magnitude, characteristics and patterns of such abuse (e.g. where 53% of people with contact with law enforcement authorities suffer some form of abuse, etc.). The survey also provided information on the very low level of reporting of these abuses as well as on the reasons and explanations given by the population for not reporting them. The information collected is extremely valuable in terms of policy-making and serves the purposes of a series of actors; led by a research centre, Fundar, Centro de Anбlisis e Investigaciуn, this study has fostered a dialogue between authorities (i.e. the government and parliamentary groups), civil society actors and human rights commissions, and is being used to inform - and feed into - the police and judicial reforms.

The surveys carried out by DIAL, Dйveloppement, Institutions et Analyses de Long terme, in Africa and in the Andean region, focus on measuring issues related to poverty, governance and democracy and are another excellent illustration of the capacity of surveys to provide solid and relevant data on complex issues. The originality of these surveys is that they rely on specific modules to be attached to standard national household surveys. These «add-on modules» focus, for instance, on people's access to basic needs: access to water, electricity, health services and education are among the elements included in the modules. Questions also contribute to the identification of priorities as perceived by the population; as mentioned previously, these are extremely important in informing and setting priorities for public action.

It is important to stress that, in obtaining solid and relevant data, special attention should be attached to gathering sufficiently disaggregated data (i.e. by income level, age, education, etc.) so that, for example, significant differences between socio-economic levels or vulnerable groups may be identified.

Metagora therefore shows that it is possible to gather statistically relevant information on sensitive and complex situations and rights through various tools. This lesson is a particularly interesting one in light of the critiques usually made to attempts to quantitatively measure and monitor rights. Critiques usually claim that the various parties concerned by these assessment attempts will be reluctant either to take part in the exercise (when referring to the general public) or to authorise the actual measurement and evaluation process (when talking of the authorities). However, experience has shown that, contrary to the belief that people are unwilling or unable to respond to sensitive questions on the respect of their rights and on governance, interviewed people show a great degree of enthusiasm in addressing these issues. Surveys in Africa have shown that people are actually more willing to address such issues than to answer so-called traditional questions regarding their economic situation and income level. It seems that citizens who previously felt their opinion counted for little or nothing appreciate the fact that, for once, there is some interest in capturing their opinion and expectations on issues they consider as important. If anything, Metagora attests of a strong interest and participation on behalf of the population. Much of the same can be said of the authorities, as illustrated in the next point and lesson learnt.

The second lesson to highlight is the fact that national statistical offices (NSOs) can contribute in a positive way to the measurement and assessment of complex rights and issues of democratic governance. An objective of Metagora was to test how far and in what ways NSOs can be involved in the monitoring of human rights and democratic governance. It results that this involvement can take diverse forms and degrees. NSOs can provide advice and technical support to other institutions involved in implementing surveys and measurement activities, develop monitoring tools and mechanisms in collaboration and interaction with others (as shown by the development of a database on the right to education in Palestine), or conduct the surveys and produce relevant information.

The involvement of NSOs as the main actor in the collection and production of data can be illustrated by the activity fostered by DIAL in West Africa and in the Andean region. Indeed, not only does it show, here again, that it is possible, through household surveys, to measure dimensions of governance and democracy and the extent to which they are being respected, but also that NSOs can take part in and lead these assessments. Prior to the launch of these add-on modules, it seemed inconceivable, for instance, that in Togo, a country experiencing strong political upheaval, a statistical survey of the population could be conducted by the NSO to evaluate issues of democracy and governance. However, the functioning of democracy and, in particular, citizen adherence to six principles considered essential for democracy were assessed: `free and transparent elections, ' `freedom of expression and of the press, ' `political freedom (choice of one's own party), ' `equality before the law, ' `freedom of worship, ' and `freedom of movement'.

The question of whether NSOs can be involved in the measurement of rights and governance essentially has to do with the value-added traditionally attached to official statistics. Indeed, a series of benefits are associated to them and include: the quality of the data collected, the robustness of the tools, the application of strict professional standards, a strong analytical potential, the capacity for time series and inter-relational indicators to be developed, etc. The moderate additional costs incurred by adding specific questionnaires to existing surveys (through the approach of piggy-backing add-on modules to standard surveys as was the case in the Andean and West African countries) is a further benefit.

Having said that, two main criteria should be used for deciding, in given situations, whether or not to associate official statistics and NSOs: «legitimacy» and «capacity». These two elements reflect what is fundamentally at stake here. Indeed, Metagora has shown that NSOs can efficiently participate in the measurement of rights and governance but, beyond this capacity, issues of legitimacy should also be taken into account; NSOs with capacities but without legitimacy should not be involved. In such situations, other actors (whether private or public) should be chosen instead. In any case, involving NSOs in the measurement of complex rights or governance issues must not endanger the development of national statistical systems and should remain case-by-case decisions. Before taking part, NSOs need to consider a broad scope of issues and discuss the pros and cons of their participation, bearing in mind country-specific features and needs.

As illustrated throughout this paper, the measurement of complex and multi-dimensional human rights and democratic governance issues is feasible and of considerable potential usefulness at the national level for a great variety of actors. However, in order to enhance evidence-based assessments and their policy-impact as much as possible, a certain number of elements should be considered.

It is crucial to adopt a bottom-up participatory approach to enhance the relevance, ownership, and thereby subsequent policy-impact of the data collected. Great attention should also be attached to disseminating the data, ensuring that the data are sufficiently disaggregated to inform policy as best as possible and identify vulnerable groups, resorting to and combining a wide variety of data, etc. In addition, the importance of long-term analysis and monitoring (i.e. through the construction of time-series) should not be under-estimated but rather built, from the beginning, into strategies to measure, monitor and enhance human rights and democratic governance.

Following the motto: if you can measure it then you can address it, it is crucial to measure the respect and implementation of human rights and governance-related issues. This is the first step to being able to grasp, in given situations, the extent of existing human rights violations and patterns of abuse, and to design policies and programmes to improve the situation.

References

metagora right society

1. Herrera J., M. Razafindrakoto and F. Roubaud (2006), Governance, Democracy and Poverty Reduction: lessons drawn from household surveys in sub-Saharan Africa and Latin America, French Ministry of Foreign Affairs (Directorate for International Co-operation and Development): Paris, 68 p.

2. Naval C. and J. Salgado (2006), Irregularities, abuses of power and ill-treatment in the Federal District: the relation between police officers and Ministerio Pъblico agents, and the population, Fundar: Mexico, 96 p.

Размещено на Allbest.ru