Methodology of Freedom Barometer: Measuring Freedom

I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be. 

 

Lord Kelvin, Electrical Units of Measurement, 1883.

Why measure liberty?

Liberty, in the sense of freedom to live one’s life according to their own preferences as long as it does not infringe freedom of others, is an important normative category, and obviously very important for liberals (the term liberal here is used in its European meaning which is close to its historical roots in classical liberalism, distinguished from its contemporary meaning in the America where it depicts more social democrats). However, beside its intrinsic value, liberty also has an instrumental one: societies with more economic freedom are more affluent providing their citizens with higher standard of living; freedom of speech and media fosters dialogue and social consensus on what are the most important contemporary problems and how to solve them, etc. Of course, as Cicero noted (sub lege libertas), freedom has its meaning only when coupled with the rule of law: fair, independent and impartial courts which deal justice efficiently, and where citizens can protect their rights. All this shows that liberty lies at the core of modern political polities and that more freedom is beneficial for both social and economic progress of a society. Therefore, it is important to measure current status of the level of freedom in different countries, thus allowing for external country-to-country comparison or regional analysis, but also internal scrutiny by identification of areas where improvements have been made or are lagging.

How to measure liberty?

Freedom Barometer is a composite index, which means it consists of data from secondary sources, namely other indices that measure specific aspects of freedom. Freedom Barometer measures freedom in 3 wide areas:

  • Political Freedom (Free and Fair Election; Absence of Unconstitutional Veto Players; Press Freedom)
  • Rule of Law (Independence of the Judiciary; Corruption; Protection of Human Rights)
  • Economic Freedom (Security of Property Rights; Size of Government; Regulation of Credit, Labour and Business; Freedom to Trade Internationally)
  • These 3 broad categories are further divided into 10 subcategories, measuring more specifically different levels of freedom. The subcategories used in creating the Index are weighted in such a manner that each has a maximum of 10 points, and therefore the overall total score is measured on a scale with the maximum of 100 points, where more points indicate higher degree of freedom.

Data sources and period covered

Bearing in mind that Freedom Barometer is a composite index, it uses other freely available indices to calculate its components. These indices have been long established and are considered to have passed the test of time and are widely used both by academics and policy makers. These indices have a built in time lag, which influences the final time lag by Freedom Barometer. The latest publicly available index is used to compute Freedom Barometer, so the existing temporal lag is minimized, but still present. This basically means that time coverage of Freedom Barometer, although published in 2016, mostly represents the state from the previous year. Therefore, texts following the Freedom Barometer data can serve as a useful tool to further elaborate findings of respective countries. The texts cover the first half of the year in which Freedom Barometer is published, and the second half of the previous year. For example, Freedom Barometer 2016 illustrates the situation from July 2015 to June 2016.

Data sources used to compute Freedom Barometer are Freedom in the World, Freedom of the Press (Freedom House), Index of Economic Freedom (Heritage Foundation and Wall Street Journal), Global Competitiveness Report (World Economic Forum), Corruption Perception Index (Transparency International) and Human Rights Index (Friedrich Naumann Stiftung).

What is new in the methodology section of Freedom Barometer 2016?

There was a major methodology revision of the Freedom Barometer 2016 edition, with which two major issues were tackled.

First of all, there was an inconsistency in the time lag - Fraser Institute’s Economic Freedom in the World lagged two years behind the present date while all other data sources were lagging only one year. In order to tackle this issue a solution was found to introduce a different data source that had a closer time frame - in line with other data sources. Therefore, Index of Economic Freedom of the Heritage Foundation and the Wall Street Journal was used for subsections Size of Government; Regulation of Credit, Labour and Business; and Freedom to Trade Internationally.

For the subsection Security of Property Rights, the data from the Index of Economic Freedom were not considered appropriate, since the appropriation of points in this case was done by Heritage Foundation, without any external data sources that could be scrutinized. Therefore, alternative measures were considered. The Rule of Law Index by the World Justice Project was a likely candidate but unfortunately it did not include all countries within the Freedom Barometer focus. Finally, it was chosen to use primary sources from the World Economic Forum Global Competitiveness Report and the World Bank’s Doing Business Report which were also used by the Economic Freedom in the World, but one year in advance of it since the data were allowing that. Therefore, Security of Property Rights was calculated from primary data with the same methodology used by the Economic Freedom in the World, one year ahead of it. With one simple change - since the project team decided to use only the data that were in public access, in order to be able to be hold accountable by wider public which could use the same data, some parts were omitted and therefore it is not possible to completely compare data from Freedom Barometer Security of Property Rights section with their counterparts from Economic Freedom in the World.

Independence of judiciary was calculated in the same way, but with primary data from Global Competitiveness Report in order to eliminate the time lag. However, these data should be completely comparable to the ones in the Economic Freedom of the World.

Human Risk Atlas that was used as a source for calculation on the level of respect of human rights in respective countries could not be used anymore due to the fact that its data and methodology were not in free public use, and therefore could not be contested, supported or dismissed. In order to overcome this obstacle, the project team has compiled its own measure of human rights from a liberal perspective, called the Human Rights Index. This new methodological tool was used to replace the previous Human Risk Atlas data.

Detailed summary of changes in methodology and data sources is as follows:

Economic Freedom

  • 1) Fraser Institute, Economic Freedom in the World Size of Government → Heritage Foundation, Index of Economic Freedom, Government Spending and Fiscal Freedom
  • 2) Fraser Institute, Economic Freedom in the World, Regulation of Credit, Labour and Business → Heritage Foundation, Index of Economic Freedom, Financial Freedom, Business Freedom and Labour Freedom
  • 3) Fraser Institute, Economic Freedom in the World, Freedom to Trade Internationally → Heritage Foundation, Index of Economic Freedom, Trade Freedom

Security of Property Rights

  • 4) Fraser Institute, Economic Freedom in the World, Security of Property Rights → World Economic Forum, Global Competitiveness Reportor World Bank, Doing Business Report
    Global Competitiveness Report, Judicial Independence Question 1.06 → Global Competitiveness Report, Judicial Independence Question 1.06
    Global Competitiveness Report, Impartial Courts Question 1.10 → Global Competitiveness Report, Impartial Courts Question 1.10
    Global Competitiveness Report, Protection of Property Rights Question 1.01 → Global Competitiveness Report, Protection of Property Rights Question 1.01
    Global Competitiveness Report, Business Cost of Crime Question 1.14 → Global Competitiveness Report, Business Cost of Crime Question 1.14
    Global Competitiveness Report, Reliability of Police Question 1.16 → Global Competitiveness Report Reliability of Police Question 1.16
    Doing Business Report, Legal enforcement of contracts → Doing Business Report, Legal enforcement of contracts
    Doing Business Report, Regulatory restrictions on sale of real property → Doing Business Report, Regulatory restrictions on sale of real property
    International Risk Guide, Military Interference with the rule of Law and Politics → omitted
    International Country Risk Guide, Integrity of the Legal System → omitted


Independence of the Judiciary

  • 5) Fraser Institute, Economic Freedom in the World, Judicial Independence and Impartial Courts → Global Competitiveness Report, Judicial Independence and Impartial Courts

Human Rights

  • 6) Maplecroft, Human Risk Atlas → Fridrich Naumann Stiftung, Human Rights Index


Political Freedom Subcategories:

Free and Fair Elections

  • Scores from Freedom House Freedom in the World: (A) Electoral Process and (B) Political Pluralism and Participation
    (A+B / (max A + max B)) * 10
Absence of Unconstitutional Veto Players
  • Scores from Freedom House Freedom in the World: (C) Functioning Government
    (C / 12) *10
Press Freedom
  • Scores from Freedom House Freedom of the Press: final score
    10 – final score / 10

Rule of Law Subcategories:

Independence of the Judiciary

  • Scores from World Economic Forum’s Global Competitiveness Report, Question 1.06 Judicial Independence (A) and 1.10 Impartial Courts (B)
    ((A-1)/6*10 + (B-1)/6*10) / 2
Corruption
  • Final score from Transparency International’s Corruption Perception Index
    Final score / 10
Protection of Human Rights
  • Score from Friedrich Naumann Stiftung’s Human Rights Index

Economic Freedom Subcategories:

Security of Property Rights

  • Data from World Economic Forum Global Competitiveness Report and World Bank Doing Business Report
    Judicial Independence (A): (A-1)/6*10
    Impartial Courts (B): (B-1)/6*10
    Protection of Property Rights (C): (C-1)/6*10
    Business Costs of Crime (D): (D-1)/6*10
    Reliability of Police (E): (E-1)/6*10
    Legal Enforcement of Contracts (F): ((725-score time)/(725-62)*10)+((0,823-score costs)/(0,823-0)*10) /2
    Regulatory Costs on Sale of Real Property (G): ((265-score time)/(265-0))*(10+(0,15- score cost)/(0,15-0)*10) / 2
    Final score: A + B + C + D + E +F + G / 7
Size of Government
  • Heritage Foundation, Index of Economic Freedom, Government Spending (A) and Fiscal Freedom (B)
    (A/10+B/10)/2
Regulation of Credit, Labour and Business
  • Heritage Foundation, ndex of Economic Freedom, Financial Freedom (A),Business Freedom (B) and Labour Freedom (C)
    (A/10)+(B/10)+(C/10)/3
Freedom to Trade Internationally
  • Heritage Foundation, Index of Economic Freedom, Trade Freedom (A)
    A/10

Friedrich Naumann Stiftung’s Human Rights Index

Friedrich Naumann Stiftung’s Human Rights Index is a new composite index created by the Freedom Barometer project team in order to provide data for the subcategory of Human Rights within the category of Rule of Law in the Freedom Barometer, or to serve as independent measure of the level of protection of human rights in various countries of Europe, Caucasus and Central Asia. It is consisted of four sections, with subsections in brackets, as follows:

  • Personal safety (with 2 subsections: Societal safety and security; Human trafficking),
  • Education (with 3 subsections: Primary school enrollment; Gender parity in school enrollment; Inequality in attainment of education),
  • Tolerance and inclusion (with 3 subsections: Group grievances; LGBT rights; Equality of women),
  • Personal rights and choice (with 4 subcategories: Reproductive rights; Freedom of religion; Freedom of speech; Freedom of assembly).

Each of the four sections of the Human Rights Index weighs 25% of the total of (maximum) 10 points awarded for the best performance of countries in the field of protection of human rights.

Each of the three subsections of the section Tolerance and inclusion weighs one third of the total score for that section.

Each of the three subsections of the section Education weighs one third of the total score for that section, except in the case of several countries where the data for the subsection Gender parity in school enrollment was not available and thus the total score for the section Education is consisted: by one half out of the score for subsection Primary school enrollment, while by another half out of the score for subsection Inequality of attainment of education.

The total score for the section Personal safety is consisted by 80% of the score for the subsection Societal safety and security and by 20% of the score for the subsection Human trafficking.

The total score of the section Personal rights and choice is shaped as the average score of its four subsections.

The sources and formulas for determination of the scores for the subsections is as follows:

  • The score for the subsection Societal safety and security is determined according to the Global Peace Index of the Institute for Economics and Peace, by subtracting it from 10.
  • The score for the subsection Human trafficking is determined according to the Trafficking in persons report of the US Department of State, whereby countries in Tier 1 got 10 points, countries in Tier 2 got 6.66 points, countries in Tier 2wl got 3.33 points, while countries in Tier 3 got 0 points.
  • The score for the subsection Primary school enrollment is determined according to the UNESCO database on the topic, whereby the original value (taken that 90, for 90%, is the minimum, and 100, for 100%, is the maximum) is divided by 10.
  • The score for the subsection Gender party in school enrollment is determined according to the UNESCO database on the topic, whereby the original value is multiplied by 10.
  • The score for the subsection Inequality in attainment of education is determined according to the original value in the UNDP database on the topic, by the application of the following formula: [(MaxV–V)/(MaxV-MinV)]x10, whereby MaxV=40, MinV=1.4 and V stands for the original value given by the UNDP for the country.
  • The score for the subsection Group grievances is determined according to the value in the Global Peace Index of the Institute for Economics and Peace, by subtracting it from 10.
  • The score for the subsection LGBT rights is entirely a product of the Freedom Barometer project team. It is based on the qualitative analysis of the legal documents on the topic. It is determined through the formula (V/14)x10, whereby V is a value between 0 and 14. V is composed of: 1 for decriminalization of homosexuality, 1 for adherence to the United Nations documents on protection of discrimination on the basis of sexual orientation, 2 for legal regulation of same-sex partnerships (0 for lack thereof), 2 for legalization of same-sex marriages (0 for lack thereof), 2 for full child adoption rights by same-sex couples (1 for partial rights thereof), 2 for the right of LGBT persons to access to military service and profession (1 for partial rights thereof), 2 for the national anti-discrimination laws protecting LGBTs (1 for a partial legal protection thereof), and 2 for the full legal acceptance of gender identity change (1 for partial, or conditioned, such as by changing sex).
  • The score for the subsection Equality of women is determined according to the original value in the Gender Equality HDI of the UNDP HDI Report, by the application of the following formula: [(MaxV–V)/(MaxV-MinV)]x10, whereby MaxV=0.66, MinV=0.016 and V stands for the original value given by the UNDP for the country.
  • The score for the subsection Reproductive rights is determined according to the original value in the UN Population Division report on contraception, by the application of the following formula: [(MinV–V)/(MinV-MaxV)]x10, whereby MaxV=7.3, MinV=22.1 and V stands for the original value given by the UN Population Division for the country.
  • The score for the subsection Freedom of religion equals the value assigned to the country regarding the Bertelsmann Transformation Index, Question 1.3, for that year. For several countries which are not included into the Bertelsmann Transformation Index, the score was determined by the Freedom Barometer project team, following a methodology as similar to Bertelsmann’s as possible.
  • The score for the subsection Freedom of speech equals the value assigned to the country regarding the Bertelsmann Transformation Index, Question 2.4, for that year. For several countries which are not included into the Bertelsmann Transformation Index, the score was determined by the Freedom Barometer project team, following a methodology as similar to Bertelsmann’s as possible.
  • The score for the subsection Freedom of assembly equals the value assigned to the country regarding the Bertelsmann Transformation Index, Question 2.3, for that year. For several countries which are not included into the Bertelsmann Transformation Index, the score was determined by the Freedom Barometer project team following a methodology as similar to Bertelsmann’s as possible.