Introduction

Has democracy made life better for the people of the Earth? This project is a continuation of my previous attempt in answering this question by correlating the Human Development Index with the state of democracy in different countries. In my previous analysis, I demonstrated that while on a global scale the world has gotten more democratic and this corresponds with a steady rise in the HDI, there are still cases of countries where this relationship is not as evident. This second phase of the project explores the impact of stability, as opposed to democracy, on a country’s performance on the HDI.

Background

For this project, I am building on top of my previous work on the correlation between changes in democracy and government with the UNDP Human Development Index (HDI). The primary research question I was asking in my initial research was ‘does the state of democracy in a country impact the standard of living for the people?’ I was intrigued to ask this question after getting inspired by this visualization of trends in global democracy by Will Merrow.

I wanted to utilize the Polity5 dataset provided by the Center for Systemic Peace in conjunction with other global indicators in order to synthesize an analysis on how changes in democracy in a country influence the lives of the people of that country. I decided to use the UNDP Human Development Index as a surrogate measure for standard of living as it is widely regarded as a holistic measure of wellbeing. This is reflected by the combination of indicators and dimensions used in calculating the HDI as illustrated below.

My initial research generated a lot of interesting insights but also left space for more questions to be answered. It showed that on a worldwide level the average level of democracy had improved over the last forty years and this corresponds with a steady increase in the HDI. However, a country by country analysis showed that this conclusion did not hold true in many cases. Some interesting cases to consider were:

• Syria – an increase in the Polity5 score results in a corresponding jump in the HDI in the subsequent years
• Pakistan – the Polity5 score is highly volatile while the HDI shows a steady increase throughout
• Qatar – the Polity5 score is stable while the HDI starts at a high point and is slowly increasing throughout

I also developed the chart shown below to highlight the year of the highest Polity5 score for each country and the corresponding growth or decline in the HDI. This showed that for many countries the year of the highest Polity5 score was followed by a growth phase in the HDI. However, there were some factors which could not be fully captured in this chart. It did not show what the Polity5 score had been before and after the highest year point. Consequently, it did not capture how long the country remained at the highest Polity5 score (i.e. how stable it remained at that point).

One of the most surprising learnings was that Highly Autocratic countries performed better on the HDI than Moderately Democratic and Moderately Autocratic countries. This analysis was done by grouping the countries on the basis of their average Polity5 Score.

Considering all of this, I investigate the following questions in this second phase of the study:

1. Is a country’s performance on the HDI more closely linked to how stable that country has been in their form of government as opposed to whether it is a democratic or autocratic country?
2. How have major events like wars, coups and changes in the form of government affected a country’s performance on the HDI?

A big part of this second phase is incorporating events like war, anarchy and coup d’etats into the analysis. I refer to these as ‘Instability Events’ and derive them from the encoding provided by the Polity5 dataset. The Polity5 dataset divided such events into three categories explained in the chart given below.

For a detailed description of how these events are categorized, please refer to section 3 of the manual provided by the Center for Systemic Peace.

One last point of iteration was visualizing all of this information on the world map. I was inspired by this animation of fireworks over New York City to design a world map which would simultaneously show animations of the instability events happening across the globe along with each country’s performance on the HDI.

Process

I used Microsoft Excel to create a joined table from the two datasets. After this I imported the new table in Tableau and used Tableau’s desktop application to build the visualizations. My initial plan was to build an animation over the world map using Carto, however I was not able to geocode the countries on Carto.

Data Preparation

After downloading the datasets, I copied the Human Development Index (HDI) sheet and the ‘p5v2018’ sheet into a single workbook in Microsoft Excel. My goal was to create a table of all countries with their Polity5 score starting from 1970, all Instability Events since 1970 and their HDI indicator since 1990. Each row of the table would represent a single year for each country from 1970 to 2019. The reason for choosing this range is because the UNDP has only determined the HDI from the year 1990 and forward. Since any country’s HDI performance in the 1990s would have been influenced by its state of government over the previous decades, I decided to incorporate Polity5 scores and Instability Events starting from 1970.

Before creating a new table I had to decide how to populate my list of countries. After exploring the two datasets, I concluded that the Polity5 dataset included all of the countries that were in the HDI dataset along with some more which did not exist by 1990. Moreover, the Polity5 dataset also recorded the Polity score for every single year and none of the years were missing. Therefore, I copied the columns of ‘country’ and ‘year’ from the Polity5 dataset into a new sheet.

In order to identify any mismatches in the country names between two datasets, I started by populating the HDI column first. Since the HDI dataset is arranged in the form of a matrix with the years on the top row and the countries in the first column, turning it into a relational form required using a combination of the VLOOKUP and the MATCH functions. The pseudo code for this function is given below:

=VLOOKUP([CountryName],[HDI Table],MATCH([year],[HDI Table Top Row],0),FALSE)   

Once I had populated the HDI column, I used a filter to check all those rows which were returning a not available error code. These represented those countries whose name in the Polity5 dataset either did not exist or did not match with the country name in the HDI dataset. There were a total of 35 such instances. I explored each case one by one and for each case made a decision to either transform the name to a new name in my model or to remove it from the model altogether. These transformations are listed below.

In order to update the list of countries in the model, I used a VLOOKUP function on the transformations table shown above and created a new list of countries in the model. Following this, I created a new column in both the model and the Polity5 dataset with a pointer in the form of <CountryYear> by concatenating the country and year. Then I used the INDEX and MATCH functions in combination to populate the Polity Score column. Lastly, I created a separate column for the Instability Events and used an IF function to isolate the instability scores from the Polity Score. I used a filter to isolate the instability scores in the Polity Score column and removed them. This left me with three columns apart from the country and year, namely the HDI score, the Polity Score (with blanks in years with instability events) and Instability Score (with blanks in years with no instability events). I added another column for a label of each instability event by creating a table of the instability scores and their labels and then using VLOOKUP to populate the labels for the entire list. After this step the model was ready to be imported into Tableau.

After importing the dataset into Tableau, I started building the visualizations by adding the relevant variables to the columns and rows and modifying the charts as per my needs. Most of the visualizations are self-explanatory and can be easily built using Tableau Desktop’s builder. The multi-layered maps are a bit more complicated, for those I implemented the process documented here.

Some Setbacks

There were a couple of setbacks in the process which I highlight here. I ran into trouble with Carto with geocoding the countries because Carto has a limit for geocoding up to 500 rows. My dataset had around 6000 rows. There is a way to get around this which is documented here. However, once I had successfully followed the steps and added the geocoding analysis, Carto was unable to render my styles on the map and only bits and pieces of the map would load at any given point. After some research I decided that this was simply a limitation of Carto because of the large size of the dataset.

Unfortunately, Tableau does not provide animations for maps. I tried building an animation on a simple chart but since I was using too many markers and filters for the instability events, the animation would not load on Tableau Web properly. Similarly, the filters on the maps are slow to function on Tableau Web which also incumbered my UX Research participants.

UX Research

Methods

The UX research was conducted in the form of qualitative observational interviews which included exploration and task completion. The researcher acted as a participant observer, communicating instructions to the participants, guiding them when they got stuck and taking notes. The participants were asked to practice the Think Aloud Technique while navigating the dashboards.

The participants were asked to open the public link of the dashboard on Tableau Web in their browsers in order to mirror the actual usecase of the dashboard. They were then asked to share their screens and video with the researcher before the start of the observational interview.

Each observational interview consisted of two parts. Firstly, the participants were asked to spend five minutes to familiarize themselves with the different dashboards. The participants were encouraged to speak out their thoughts and ask questions from the researcher. Once the researcher determined that the participant was adequately familiar, they were asked to complete the following two tasks:

1. Find out which instability event occurred in Albania in 1991. What was the country’s Polity5 score after this period of instability?
2. Describe the relationship between the deviation in Polity Score and average HDI.

The last two minutes of the interview were spent to get general feedback from the participant about their experience of using the dashboard.

Participants

The participants for the UX research were identified on the basis of the target user persona for this visualization project. I identified the target users as students or academics who are interested in the areas of politics, social sciences, history and international development. One of the UX research participants is a graduate student at the School of International and Public Affairs at Columbia University. The second participant is an MBA student at Oxford University with a special interest in the development sector.

Findings and Improvements

The participants generally found the dashboards easy to use and navigate. Both of the participants felt that the information presented was useful and they would like to refer back to the visualizations again for their work. The participants spent the most time on the ‘Pick a country’ dashboard and were delighted by the ability to see the detailed illustration of HDI, instability events and Polity Score for different countries. Both participants related back to historical events they knew about which impacted the chosen countries.

Some of the issues highlighted by the participants and observed by the researcher included issues with understanding the purpose and context of this analysis, understanding some confusing charts and some usability issues with the filters. These findings and the corresponding improvements made are summarized in the table below.

Design Decisions

The visualizations made for this second phase of the project involve some new design decisions implemented on the basis of the feedback received on the previous dashboard and the UX research conducted for this project.

The UX research showed that the Red-Green scale and the line chart used for average HDI made it difficult for the users to differentiate it from the Polity Score. As a result, I updated this to using solely a Green palette for average HDI and changing the HDI chart to a circle scatter plot instead. Throughout all of the dashboards, I consistently use the color palettes for HDI and Polity Score shown below. The red color represents an autocratic form of government and blue represents democracy.

In addition to this, I also decided to hide all of the component worksheets from the public viewer. This was done to reduce the clutter of sheets. The following design decisions were made for the remaining dashboards and sheets:

Worldwide Dashboard

• Named as such so that users know this is the main dashboard where they should start exploring the visualizations.
• Background Color: A light gray tone chosen so that there is more weight in the background and also provides a sharp contrast with the red, green and blue colors.
• Other specifications: I decided to prioritize the worldwide chart to the top so that the users can use to it to understand the context of the project. As per the UX research findings, I added annotations and clarifications to the charts. Fixed the average HDI axis range from 0 to 1 so that the relative difference between the chosen countries is more evident.

World Map Comparison

• I wanted to provide a side by side comparison of HDI and Polity Scores on the world map. The goal is that the user can use the filter slider to see how these two indicators change across the world with time.
• Background Color: White chosen to match the colors of the oceans on the maps so that the visualization blends smoothly with the background.
• Explanatory headings added on the basis of UX research findings.

Instability Events

• The intensity of the instability events is reflected by both the color and size of the circles.
• Background Color: White chosen to match the colors of the oceans on the map so that the visualization blends smoothly with the background.
• Explanatory and instructional headings added on the basis of UX research findings

Pick a Country

• The name of the dashboard clearly reflects what the user needs to do. This dashboard consolidates all of the information about a single country in one place.
• Background Color: Black chosen to provide a sharper contrast to the red, orange and purple colors of the instability events.
• ‘HDI score in the last 30 years’ and ‘Instability events in the last 30 years’ deliberately placed on top of each other and in close proximity for comparison.

Stability Measures

• Background Color: A slightly darker color of gray to provide a light contrast to all of the colors of the different countries.
• The axes were flipped on the basis of UX research findings – the average HDI was placed consistently on the vertical axis. Secondly, instead of comparing standard deviation in Polity Score with standard deviation in HDI, this was changed to a comparison with average HDI which is easier to understand for the user.

Results

Click here to access the complete visualization dashboards

The results from the project help us build further from our findings in the previous study. While it is true that at a global level democratic countries tend to perform better on the HDI, however, stability in the form of government and incidence of “instability events” might be much better predictors of a country’s HDI. The matrix given below highlights four different case: Autocratic-Stable, Democratic-Stable, Autocratic-Unstable and Democratic-Unstable. The matrix shows how in these four examples the stable countries have performed better on the HDI. The ‘Pick a country’ dashboard enables users to dig further and see how this relationship plays out for other countries.

Similarly, our analysis on the different measures of stability also shows that the countries with the lowest deviation in Polity Score and lowest frequency in Instability Events have the highest average HDI. However, there are some notable exceptions to the first case and there are more countries in the high polity score deviation and high average HDI area than expected.

This is consistent with our observation that there are examples such as Pakistan, Bangladesh and Thailand which have shown a significant fluctuation in their Polity Score yet they have continued to shown a steady increase in the HDI. This points to more factors in the variance in HDI that have not been captured by this study.

Future Direction

There are two key areas for possible future work in this study. The first focuses on building on this analysis and adding more complexity. One particular approach could be to look for and incorporate other indicators of standard of living which can be used in comparison with the HDI. Similarly, the complexity of the analysis could be increased by incorporating other measures of democracy such as voter turnout, public sentiment, fairness of elections etc. Another way to isolate the concerned effects would be to incorporate other factors such the Gross Domestic Product, population, trade, natural resources etc.; normalizing on such factors could prove helpful in determining the sole impact the state of the government has had on the country.

The second area of improvement is incorporating the feedback received from the users during the research and presentations and improving the visualizations. The complexity of the markers and filters used on the maps can be reduced to make it possible to build animations on Tableau. This would make it easier for users to see the sequence of instability events and their corresponding aftereffects. Moreover, other measures such as ‘time since last instability event’ and ‘change in HDI’ can also be incorporated to further the analysis.