I'm normally a fan of the statistical reports produced by the OECD, so was surprised to find myself in disagreement with the methodology of their recently-created Social Institutions and Gender Index (SIGI). The idea behind SIGI is a good one -- to get at the root of gender discrimination by examining traditions and social norms that impede women's empowerment. To this end, SIGI assesses twelve variables in 102 non-OECD countries and then ranks these countries based on their "performance in social institutions." The selection and evaluation problems with their study highlight the difficulty of empirically measuring and ranking intangible phenomena such as social norms.My first concern with the study is that the twelve variables that SIGI has chosen to measure social institutions and their contributions to gender inequality appear to suffer from selection bias. Their model does include variables that impact women's advancement globally, such as access to land and property and inheritance laws. However, given the relatively small number of variables examined, it was surprising to see the inclusion of discriminatory traditions and social norms practiced only in a limited number of regions, such as female genital mutilation, restrictions on freedom of dress and "missing women" (gender-selected abortion or infanticide) as stand-alone variables. To be sure, these forms of hidden discrimination are of concern, but if it's to include traditions and social norms that impede women's progress only in certain regions, the study should be as comprehensive as possible, including a broad range of regionally specific discriminatory norms (such as very limited access to contraception and abortion in much of Latin America). Alternatively, the study could be limited to variables that exist in nearly every country studied, with regionally specific norms as a subset but not a stand-alone variable (examining FGM as one manifestation of violence against women, for example). The study's current approach inappropriately weights these variables and thus leads to odd results in the rankings -- India, for example, is ranked 96 of 102 countries, just below Iran. While there's no disputing that India has its fair share of hidden forms of gender discrimination, a claim that Indian women face more discriminatory norms than Iranian women is difficult to defend.
The study also appears to contain evaluation problems in that it relies on the law on the books to measure social norms without closely examining whether this law is applied in practice. So, for example, I was surprised to see El Salvador ranked number 8, with a mention of violence against women as a "serious problem" but nonetheless receiving an extremely high score for physical integrity. In contrast, the U.S. State Department's 2008 Human Rights Report denotes violence against women as one of the top human rights problems in El Salvador, with over 6000 reports of domestic violence and only 12 prosecutions and 4 convictions last year. Again, it's hard to take seriously an index that ranks highly a country with social norms that widely condone violence against women.
While the authors of the study are correct in their claims that these traditions and social norms often impede progress towards equality for women, it is extremely hard to quantify such intangible phenomena. Social norms and traditions that impede women's empowerment take different forms throughout the world, and do not lend themselves easily to comparative assessment. More importantly, I'm not sure where the value lies in "ranking" hidden forms of gender discrimination. How should these rankings be used? To determine which countries should be prioritized in efforts to ameliorate gender discriminatory norms? As a shaming sanction against those at the bottom of the list? While it's undoubtedly important to examine and elucidate these norms in order to redress them, it seems less worthwhile to measure and sort them, as if eliminating gender inequality were simply a numbers game. And as SIGI's pitfalls illustrate, such a study should be performed by researchers familiar with the societies and cultures in question and should not rely on laws on the books as an adequate proxy for social norms. Moreover, any such study should rigorously select and weight quantitative or qualitative measures of discriminatory norms to avoid culturally biased and unreliable results.
Cross-posted on Concurring Opinions.
