The country converter (coco) includes a helper function which provides country classification concordances. The concordances are provided in dict, vector and matrix format. The helper function can be used independent from the country classification included in coco (see 'Small example' below), but can also make use of the different classifications provided by coco ('WIOD example' below). The aggregation function can also be used in Matlab to build concordance matrices (see 'Use in Matlab' at the end).
This example is based on a small country set co:
co = ["c1", "c2", "c3", "c4", "c5"]
Assume, these countries can be classified in two ways (e.g. one could be geographic classification, the other one a political one):
geographic_reg = {"c1": "geo1", "c2": "geo1", "c3": "geo2", "c4": "geo3", "c5": "geo3"}
political_reg = {"c1": "pol1", "c3": "pol1", "c5": "pol2"}
First, we want to aggregated the countries co to political regions, assigning all non classified countries to a Rest of the World region RoW:
import country_converter as coco
vector representation
concordance_vec = coco.agg_conc(
original_countries=co,
aggregates=political_reg,
missing_countries="RoW",
as_dataframe="sparse",
)
This results in a concordance which maps countries co to the regions given in political_reg:
concordance_vec
original | aggregated | |
---|---|---|
0 | c1 | pol1 |
1 | c2 | RoW |
2 | c3 | pol1 |
3 | c4 | RoW |
4 | c5 | pol2 |
Two other output formats are available:
full matrix representation
concordance_matrix = coco.agg_conc(
original_countries=co,
aggregates=political_reg,
missing_countries="RoW",
as_dataframe="full",
)
concordance_matrix
aggregated | RoW | pol1 | pol2 |
---|---|---|---|
original | |||
c1 | 0.0 | 1.0 | 0.0 |
c2 | 1.0 | 0.0 | 0.0 |
c3 | 0.0 | 1.0 | 0.0 |
c4 | 1.0 | 0.0 | 0.0 |
c5 | 0.0 | 0.0 | 1.0 |
The numerical values are available in the pandas.DataFrame attribute values:
concordance_matrix.values
array([[0., 1., 0.], [1., 0., 0.], [0., 1., 0.], [1., 0., 0.], [0., 0., 1.]])
dictionary representation
concordance_dict = coco.agg_conc(
original_countries=co,
aggregates=political_reg,
missing_countries="RoW",
as_dataframe=False,
)
concordance_dict
OrderedDict([('c1', 'pol1'), ('c2', 'RoW'), ('c3', 'pol1'), ('c4', 'RoW'), ('c5', 'pol2')])
The aggregates parameter of coco.agg_conc accepts several classification for the building of the concordance. This allows for a sequential aggregation.
Based on the two classifications above geographic_reg and politic_reg we can build a concordance which first aggregates into political regions and subsequently all countries not included in a political region into geographic regions:
conc_pol_geo = coco.agg_conc(
original_countries=co,
aggregates=[political_reg, geographic_reg],
missing_countries="RoW",
as_dataframe="sparse",
)
conc_pol_geo
original | aggregated | |
---|---|---|
0 | c1 | pol1 |
1 | c2 | geo1 |
2 | c3 | pol1 |
3 | c4 | geo3 |
4 | c5 | pol2 |
The order of the classifications is important here, as subsequent classifications are only applied to countries not already classified:
conc_geo_pol = coco.agg_conc(
original_countries=co,
aggregates=[geographic_reg, political_reg],
missing_countries="RoW",
as_dataframe="sparse",
)
conc_geo_pol
original | aggregated | |
---|---|---|
0 | c1 | geo1 |
1 | c2 | geo1 |
2 | c3 | geo2 |
3 | c4 | geo3 |
4 | c5 | geo3 |
This can be used to classify into different political regions (see example WIOD below) and two exclude certain countries from the classification as for example:
conc_c2_pol_geo = coco.agg_conc(
original_countries=co,
aggregates=[{"c2": "c2"}, political_reg, geographic_reg],
missing_countries="RoW",
as_dataframe="sparse",
)
conc_c2_pol_geo
original | aggregated | |
---|---|---|
0 | c1 | pol1 |
1 | c2 | c2 |
2 | c3 | pol1 |
3 | c4 | geo3 |
4 | c5 | pol2 |
Besides passing custom dictionaries with countries and classification, coco.agg_conc also accepts strings corresponding to any classification available in coco. For example, to aggregated the WIOD countries into EU, OECD and 'OTHER' use:
conc_wiod_3reg = coco.agg_conc(
original_countries="WIOD",
aggregates=["EU28", "OECD"],
missing_countries="OTHER",
as_dataframe="sparse",
)
conc_wiod_3reg.head(10)
original | aggregated | |
---|---|---|
0 | RoW | nan_&_EU28 |
1 | AUS | OECD |
2 | AUT | EU28 |
3 | BEL | EU28 |
4 | BRA | OTHER |
5 | BGR | EU28 |
6 | CAN | OECD |
7 | CHN | OTHER |
8 | CYP | EU28 |
9 | CZE | EU28 |
WIOD already includes a ROW region, which consists of EU and non-EU countries. This leads the to the compound name for the region shown above. To set such many to many matches to None and pass them through until a unique matching is found, set merge_multiple_strings to None:
conc_wiod_3reg = coco.agg_conc(
original_countries="WIOD",
aggregates=["EU28", "OECD"],
missing_countries="OTHER",
merge_multiple_string=None,
as_dataframe="sparse",
)
conc_wiod_3reg.head(10)
original | aggregated | |
---|---|---|
0 | RoW | OTHER |
1 | AUS | OECD |
2 | AUT | EU28 |
3 | BEL | EU28 |
4 | BRA | OTHER |
5 | BGR | EU28 |
6 | CAN | OECD |
7 | CHN | OTHER |
8 | CYP | EU28 |
9 | CZE | EU28 |
Internally, the dicts required for the aggregation are build be calling CountryConverter.get_correspondence_dict(original_class, target_class). If a fine grained control over the aggregation is necessary, call this low level function directly and modify the resulting dict.
As in the small example given above, single countries can be separated by passing a custom dict:
conc_wiod_DE_Aus_OECD = coco.agg_conc(
original_countries="WIOD",
aggregates=[{"Deu": "Deu", "Aus": "Aus"}, "OECD"],
missing_countries="Non_OECD",
merge_multiple_string=None,
as_dataframe="sparse",
)
conc_wiod_DE_Aus_OECD.head(16)
original | aggregated | |
---|---|---|
0 | RoW | Non_OECD |
1 | AUS | OECD |
2 | AUT | OECD |
3 | BEL | OECD |
4 | BRA | Non_OECD |
5 | BGR | Non_OECD |
6 | CAN | OECD |
7 | CHN | Non_OECD |
8 | CYP | Non_OECD |
9 | CZE | OECD |
10 | DNK | OECD |
11 | EST | OECD |
12 | FIN | OECD |
13 | FRA | OECD |
14 | DEU | OECD |
15 | GRC | OECD |
Python function can be called directly in Matlab. However, Matlab follows its trademark and makes simple things complicated. To achieve the above concordance conc_wiod_3reg in Matlab use:
conc_wiod_3reg_dataframe = py.country_converter.agg_conc(...
pyargs('original_countries','WIOD', ...
'aggregates',py.list({'EU', 'OECD'}), ...
'missing_countries', 'OTHER', ...
'merge_multiple_string', py.None, ...
'as_dataframe', 'full'));
This returns the data as pandas DataFrame. Since Matlab does not provide any function for converting pandas DataFrames or Numpy arrays to double, two additional steps are required to get to a valid Matlab array:
temp = cellfun(@cell,cell(conc_wiod_3reg_dataframe.values.tolist),'un',0);
conc_matrix = cell2mat(vertcat(temp {:}));
Since Matlab only works with numerical values, all names provided in the original DataFrame are lost.
Additional dicts for the aggregates list can be build in Matlab with for example:
spec_country_dict = py.dict(pyargs('Aus', 'Aus', 'Deu', 'Deu'))
For more details about agg_conc see the docstring:
help(coco.agg_conc)
Help on function agg_conc in module country_converter.country_converter: agg_conc(original_countries, aggregates, missing_countries='missing', merge_multiple_string='_&_', log_missing_countries=None, log_merge_multiple_strings=None, coco=None, as_dataframe='sparse', original_countries_class=None) Builds an aggregation concordance dict, vec or matrix Parameters ---------- original_countries: list or str List of countries to aggregated, also accepts and valid column name of CountryConverter.data (e.g. name_short, WIOD, ...) aggregates: list of dict or str List of aggregation information. This can either be dict mapping the names of 'original_countries' to aggregates, or a valid column name of CountryConverter.data Aggregation happens in order given in this parameter. Thus, country assigned to an aggregate are not re-assigned by the following aggregation information. missing_countries: str, boolean, None Entry to fill in for countries in 'original_countries' which do not appear in 'aggregates'. str: Use the given name for all missing countries True: Use the name in original_countries for missing countries False: Skip these countries None: Use None for these countries merge_multiple_string: str or None, optional If multiple correspondence entries are given in one of the aggregates join them with the given string (default: '_&_'. To skip these entries, pass None. log_missing_countries: function, optional This function is called with country is country is in 'original_countries' but missing in all 'aggregates'. For example, pass lambda x: logging.error('Country {} missing'.format(x)) to log errors for such countries. Default: do nothing log_merge_multiple_strings: function, optional Function to call for logging multiple strings, see log_missing_countries Default: do nothing coco: instance of CountryConverter, optional CountryConverter instance used for the conversion. Pass a custom one if additional data is needed in addition to the custom country converter file. If None (default), the bare CountryConverter is used as_dataframe: boolean or st, optional If False, output as OrderedDict. If True or str, output as pandas dataframe. If str and 'full', output as a full matrix, otherwise only two columns with the original and aggregated names are returned. original_countries_class: str, optional Valid column name of CountryConverter.data. This parameter is needed if a list of countries is passed to 'original_countries' and strings corresponding to data in CountryConverter.data are used subsequently. Can be omitted otherwise. Returns ------- OrderedDict or DataFrame (defined by 'as_dataframe')