Trying to work out personality disorder prevalence.
Ethnicity facts and figures report: https://www.ethnicity-facts-figures.service.gov.uk/health/mental-health/prevalence-of-personality-disorder-in-adults/latest Data: https://www.ethnicity-facts-figures.service.gov.uk/health/mental-health/prevalence-of-personality-disorder-in-adults/latest/downloads/apms-screen-positive-for-personality-disorder.csv
import pandas as pd
eth_prev_url = "https://www.ethnicity-facts-figures.service.gov.uk/health/mental-health/prevalence-of-personality-disorder-in-adults/latest/downloads/apms-screen-positive-for-personality-disorder.csv"
eth_prev_df = pd.read_csv(eth_prev_url)
eth_prev_df.head()
Measure | Time | Time_type | Geography | Ethnicity | Ethnicity_type | Sex | Sex_type | Value | Value type | Value Note | Lower CI | Upper CI | unweighted sample size | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Screen positive for personality disorder | 2014 | Year | England | White - British | 5 | All | Men/Women/All | 13.9 | % | NaN | 12.80% | 15.10% | 6,023 |
1 | Screen positive for personality disorder | 2014 | Year | England | White - Other | 5 | All | Men/Women/All | 14.2 | % | NaN | 10.70% | 18.70% | 397 |
2 | Screen positive for personality disorder | 2014 | Year | England | Black | 5 | All | Men/Women/All | 17.0 | % | NaN | 10.30% | 26.80% | 181 |
3 | Screen positive for personality disorder | 2014 | Year | England | Asian | 5 | All | Men/Women/All | 17.3 | % | NaN | 10.50% | 27.20% | 325 |
4 | Screen positive for personality disorder | 2014 | Year | England | Mixed/Other | 5 | All | Men/Women/All | 16.7 | % | NaN | 9.00% | 28.80% | 134 |
From prevalance / ethnicity, ethnic groups and sex are:
eth_prev_df["Ethnicity"].unique()
array(['White - British', 'White - Other', 'Black', 'Asian', 'Mixed/Other'], dtype=object)
eth_prev_df["Sex"].unique()
array(['All', 'Men', 'Women'], dtype=object)
For population data (ethnicity) data, the Nomis website lets you pull data from Census 2011.
eth_sex_url = "https://www.nomisweb.co.uk/api/v01/dataset/nm_801_1.data.xlsx?time=latest&geography=1946157281&c_age=0&measures=20100&c_ethpuk11=0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23&c_sex=0,1,2&rows=c_ethpuk11&cols=c_sex&select=noflags"
eth_sex_df = pd.read_excel(eth_sex_url, sheet_name="Sheet 1", skiprows=10).rename(columns={"Sex":"Ethnicity"}).set_index("Ethnicity").dropna(how="all")
eth_sex_df
/usr/local/lib/python3.8/dist-packages/openpyxl/styles/stylesheet.py:221: UserWarning: Workbook contains no default style, apply openpyxl's default warn("Workbook contains no default style, apply openpyxl's default")
All persons | Males | Females | |
---|---|---|---|
Ethnicity | |||
All categories: Ethnic group | 138265.0 | 67424.0 | 70841.0 |
White: Total | 134545.0 | 65568.0 | 68977.0 |
English/Welsh/Scottish/Northern Irish/British | 131099.0 | 63958.0 | 67141.0 |
Irish | 747.0 | 358.0 | 389.0 |
Gypsy or Irish Traveller | 94.0 | 74.0 | 20.0 |
Other White | 2605.0 | 1178.0 | 1427.0 |
Mixed/multiple ethnic group: Total | 1709.0 | 838.0 | 871.0 |
White and Black Caribbean | 507.0 | 266.0 | 241.0 |
White and Black African | 123.0 | 63.0 | 60.0 |
White and Asian | 738.0 | 337.0 | 401.0 |
Other Mixed | 341.0 | 172.0 | 169.0 |
Asian/Asian British: Total | 1514.0 | 664.0 | 850.0 |
Indian | 435.0 | 188.0 | 247.0 |
Pakistani | 80.0 | 50.0 | 30.0 |
Bangladeshi | 131.0 | 83.0 | 48.0 |
Chinese | 219.0 | 91.0 | 128.0 |
Other Asian | 649.0 | 252.0 | 397.0 |
Black/African/Caribbean/Black British: Total | 303.0 | 231.0 | 72.0 |
African | 141.0 | 100.0 | 41.0 |
Caribbean | 115.0 | 92.0 | 23.0 |
Other Black | 47.0 | 39.0 | 8.0 |
Other ethnic group: Total | 194.0 | 123.0 | 71.0 |
Arab | 52.0 | 38.0 | 14.0 |
Any other ethnic group | 142.0 | 85.0 | 57.0 |
It also looks like new 2021 census data is out at the end of the month: The 2021 Census results are starting to come out; ethnicity by end of month? https://census.gov.uk/census-2021-results/phase-one-topic-summaries/ethnic-group-national-identity-language-and-religion
Prevalence stats - pop group mapping:
#Check:
#131099 + (134545-131099) +303+1514+(1709+194)
eth_sex_tot_df = eth_sex_df[eth_sex_df.index.str.contains("Total")].copy()
display(eth_sex_tot_df)
# Check the totals to ensure our sum is correct
eth_sex_tot_df.sum()
All persons | Males | Females | |
---|---|---|---|
Ethnicity | |||
White: Total | 134545.0 | 65568.0 | 68977.0 |
Mixed/multiple ethnic group: Total | 1709.0 | 838.0 | 871.0 |
Asian/Asian British: Total | 1514.0 | 664.0 | 850.0 |
Black/African/Caribbean/Black British: Total | 303.0 | 231.0 | 72.0 |
Other ethnic group: Total | 194.0 | 123.0 | 71.0 |
All persons 138265.0 Males 67424.0 Females 70841.0 dtype: float64
eth_sex_tot_df.loc["White - British"] = eth_sex_df.loc['English/Welsh/Scottish/Northern Irish/British']
eth_sex_tot_df.loc["White - Other"] = eth_sex_df.loc['White: Total'] - eth_sex_df.loc['English/Welsh/Scottish/Northern Irish/British']
eth_sex_tot_df.loc["Mixed/Other"] = eth_sex_df.loc['Mixed/multiple ethnic group: Total'] + eth_sex_df.loc['Other ethnic group: Total']
eth_sex_tot_df = eth_sex_tot_df.drop(index=["White: Total", 'Mixed/multiple ethnic group: Total', 'Other ethnic group: Total'])
eth_sex_tot_df.rename(index={'Black/African/Caribbean/Black British: Total': 'Black',
'Asian/Asian British: Total':'Asian'}, inplace=True)
eth_sex_tot_df.rename(columns={"All persons":"All", "Males":"Men", "Females":"Women"}, inplace=True)
eth_sex_tot_df
All | Men | Women | |
---|---|---|---|
Ethnicity | |||
Asian | 1514.0 | 664.0 | 850.0 |
Black | 303.0 | 231.0 | 72.0 |
White - British | 131099.0 | 63958.0 | 67141.0 |
White - Other | 3446.0 | 1610.0 | 1836.0 |
Mixed/Other | 1903.0 | 961.0 | 942.0 |
#The rates are percentages (Value Type column)
eth_prev_df_wide = eth_prev_df[["Ethnicity", "Sex", "Value"]].pivot(index='Ethnicity', columns='Sex', values='Value')
eth_prev_df_wide = eth_prev_df_wide/100
eth_prev_df_wide
Sex | All | Men | Women |
---|---|---|---|
Ethnicity | |||
Asian | 0.173 | 0.227 | 0.108 |
Black | 0.170 | 0.161 | 0.177 |
Mixed/Other | 0.167 | 0.102 | 0.219 |
White - British | 0.139 | 0.130 | 0.149 |
White - Other | 0.142 | 0.168 | 0.117 |
for c in eth_prev_df_wide.columns:
display(eth_prev_df_wide[c]*eth_sex_tot_df[c])
Ethnicity Asian 261.922 Black 51.510 Mixed/Other 317.801 White - British 18222.761 White - Other 489.332 Name: All, dtype: float64
Ethnicity Asian 150.728 Black 37.191 Mixed/Other 98.022 White - British 8314.540 White - Other 270.480 Name: Men, dtype: float64
Ethnicity Asian 91.800 Black 12.744 Mixed/Other 206.298 White - British 10004.009 White - Other 214.812 Name: Women, dtype: float64
eth_prev_df_wide["All"]*eth_sex_tot_df["All"]
Ethnicity Asian 261.922 Black 51.510 Mixed/Other 317.801 White - British 18222.761 White - Other 489.332 Name: All, dtype: float64