%load_ext autoreload
%autoreload 2
import models
import orca
import pandas as pd
pd.options.mode.chained_assignment = None
orca.run(["rsh_estimate"])
Running model 'rsh_estimate' Filling column non_residential_rent with value 0 (142400 values) Filling column residential_units with value 0 (0 values) Filling column year_built with value 1927.0 (3116 values) Filling column residential_sales_price with value 0 (14196 values) Filling column non_residential_sqft with value 0 (1341 values) Filling column building_type_id with value 2.0 (0 values) OLS Regression Results ============================================================================================= Dep. Variable: np.log1p(residential_sales_price) R-squared: 0.399 Model: OLS Adj. R-squared: 0.399 Method: Least Squares F-statistic: 1.240e+04 Date: Tue, 28 Apr 2015 Prob (F-statistic): 0.00 Time: 11:32:43 Log-Likelihood: -2.5241e+05 No. Observations: 149409 AIC: 5.048e+05 Df Residuals: 149400 BIC: 5.049e+05 Df Model: 8 Covariance Type: nonrobust ================================================================================================ coef std err t P>|t| [95.0% Conf. Int.] ------------------------------------------------------------------------------------------------ Intercept -3.8638 0.170 -22.751 0.000 -4.197 -3.531 I(year_built < 1940)[T.True] 0.0012 0.007 0.160 0.873 -0.013 0.016 I(year_built > 2005)[T.True] -0.0633 0.049 -1.300 0.194 -0.159 0.032 np.log1p(unit_sqft) -1.4830 0.007 -210.948 0.000 -1.497 -1.469 np.log1p(unit_lot_size) -0.1476 0.006 -23.224 0.000 -0.160 -0.135 sum_residential_units 0.0987 0.008 11.987 0.000 0.083 0.115 ave_lot_sqft -0.2177 0.010 -21.273 0.000 -0.238 -0.198 ave_unit_sqft 0.9371 0.020 45.808 0.000 0.897 0.977 ave_income 1.4191 0.015 92.819 0.000 1.389 1.449 ============================================================================== Omnibus: 89684.695 Durbin-Watson: 1.798 Prob(Omnibus): 0.000 Jarque-Bera (JB): 820175.061 Skew: -2.842 Prob(JB): 0.00 Kurtosis: 12.972 Cond. No. 1.01e+03 ============================================================================== Warnings: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. [2] The condition number is large, 1.01e+03. This might indicate that there are strong multicollinearity or other numerical problems. Time to execute model 'rsh_estimate': 1.19s Total time to execute: 1.19s
orca.run(["nrh_estimate"])
Running model 'nrh_estimate' Filling column job_category with value service (331 values) REGRESSION RESULTS FOR SEGMENT Retail OLS Regression Results ========================================================================================== Dep. Variable: np.log1p(non_residential_rent) R-squared: 0.046 Model: OLS Adj. R-squared: 0.045 Method: Least Squares F-statistic: 43.81 Date: Tue, 28 Apr 2015 Prob (F-statistic): 2.83e-44 Time: 11:32:43 Log-Likelihood: -5334.2 No. Observations: 4592 AIC: 1.068e+04 Df Residuals: 4586 BIC: 1.072e+04 Df Model: 5 Covariance Type: nonrobust ================================================================================================ coef std err t P>|t| [95.0% Conf. Int.] ------------------------------------------------------------------------------------------------ Intercept 3.9704 0.331 11.986 0.000 3.321 4.620 I(year_built < 1940)[T.True] -0.3253 0.025 -12.869 0.000 -0.375 -0.276 I(year_built > 2005)[T.True] -0.1107 0.104 -1.067 0.286 -0.314 0.093 np.log1p(stories) 0.2327 0.037 6.311 0.000 0.160 0.305 ave_income -0.0483 0.026 -1.874 0.061 -0.099 0.002 jobs -0.0367 0.014 -2.688 0.007 -0.063 -0.010 ============================================================================== Omnibus: 3246.917 Durbin-Watson: 1.731 Prob(Omnibus): 0.000 Jarque-Bera (JB): 32590.849 Skew: -3.490 Prob(JB): 0.00 Kurtosis: 14.028 Cond. No. 374. ============================================================================== Warnings: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. REGRESSION RESULTS FOR SEGMENT Office OLS Regression Results ========================================================================================== Dep. Variable: np.log1p(non_residential_rent) R-squared: 0.066 Model: OLS Adj. R-squared: 0.064 Method: Least Squares F-statistic: 51.14 Date: Tue, 28 Apr 2015 Prob (F-statistic): 2.16e-51 Time: 11:32:43 Log-Likelihood: -4021.6 No. Observations: 3653 AIC: 8055. Df Residuals: 3647 BIC: 8092. Df Model: 5 Covariance Type: nonrobust ================================================================================================ coef std err t P>|t| [95.0% Conf. Int.] ------------------------------------------------------------------------------------------------ Intercept 4.1593 0.321 12.975 0.000 3.531 4.788 I(year_built < 1940)[T.True] -0.3156 0.027 -11.908 0.000 -0.368 -0.264 I(year_built > 2005)[T.True] -0.0238 0.119 -0.201 0.841 -0.256 0.209 np.log1p(stories) 0.1750 0.024 7.183 0.000 0.127 0.223 ave_income -0.0719 0.025 -2.853 0.004 -0.121 -0.022 jobs -0.0429 0.013 -3.201 0.001 -0.069 -0.017 ============================================================================== Omnibus: 2697.682 Durbin-Watson: 1.827 Prob(Omnibus): 0.000 Jarque-Bera (JB): 29985.208 Skew: -3.650 Prob(JB): 0.00 Kurtosis: 14.988 Cond. No. 348. ============================================================================== Warnings: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. REGRESSION RESULTS FOR SEGMENT Industrial OLS Regression Results ========================================================================================== Dep. Variable: np.log1p(non_residential_rent) R-squared: 0.105 Model: OLS Adj. R-squared: 0.103 Method: Least Squares F-statistic: 59.66 Date: Tue, 28 Apr 2015 Prob (F-statistic): 6.48e-59 Time: 11:32:43 Log-Likelihood: -2035.1 No. Observations: 2558 AIC: 4082. Df Residuals: 2552 BIC: 4117. Df Model: 5 Covariance Type: nonrobust ================================================================================================ coef std err t P>|t| [95.0% Conf. Int.] ------------------------------------------------------------------------------------------------ Intercept 3.7205 0.309 12.025 0.000 3.114 4.327 I(year_built < 1940)[T.True] -0.3186 0.023 -13.995 0.000 -0.363 -0.274 I(year_built > 2005)[T.True] -0.0483 0.101 -0.479 0.632 -0.246 0.150 np.log1p(stories) 0.3851 0.042 9.187 0.000 0.303 0.467 ave_income -0.1826 0.027 -6.852 0.000 -0.235 -0.130 jobs 0.0605 0.012 4.981 0.000 0.037 0.084 ============================================================================== Omnibus: 1781.535 Durbin-Watson: 1.817 Prob(Omnibus): 0.000 Jarque-Bera (JB): 17978.361 Skew: -3.372 Prob(JB): 0.00 Kurtosis: 14.099 Cond. No. 385. ============================================================================== Warnings: [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. Time to execute model 'nrh_estimate': 0.80s Total time to execute: 0.80s
orca.run(["rsh_simulate", "nrh_simulate"])
Running model 'rsh_simulate' count 140780.000000 mean 1156.971200 std 41503.246343 min 0.002067 25% 252.727481 50% 379.993499 75% 554.244653 max 12507033.315516 dtype: float64 Time to execute model 'rsh_simulate': 0.92s Running model 'nrh_simulate' count 10803.000000 mean 20.729909 std 6.833292 min 6.258496 25% 17.431477 50% 20.908725 75% 25.077906 max 47.266957 dtype: float64 Time to execute model 'nrh_simulate': 0.54s Total time to execute: 1.46s
orca.run(["hlcm_estimate"])
Running model 'hlcm_estimate' LCM RESULTS FOR SEGMENT 0 Null Log-liklihood: -13815.511 Log-liklihood at convergence: -11262.625 Log-liklihood Ratio: 0.185 +-----------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +-----------------------------------+-------------+------------+---------+ | np.log1p(residential_sales_price) | -0.000 | 2.513 | -0.000 | | np.log1p(unit_sqft) | -0.352 | 0.023 | -15.020 | | sum_residential_units | 0.402 | 0.108 | 3.723 | | ave_unit_sqft | 0.301 | 0.108 | 2.775 | | ave_lot_sqft | -0.111 | 0.038 | -2.949 | | ave_income | -0.460 | 0.140 | -3.285 | | hhsize | -0.644 | 0.145 | -4.455 | | jobs | 0.032 | 0.032 | 1.012 | | sfdu | -0.045 | 0.019 | -2.341 | | renters | -0.550 | 0.039 | -14.172 | | poor | 0.806 | 0.152 | 5.292 | | population | -0.319 | 0.177 | -1.803 | +-----------------------------------+-------------+------------+---------+ LCM RESULTS FOR SEGMENT 1 Null Log-liklihood: -13815.511 Log-liklihood at convergence: -12669.732 Log-liklihood Ratio: 0.083 +-----------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +-----------------------------------+-------------+------------+---------+ | np.log1p(residential_sales_price) | -0.000 | 3.059 | -0.000 | | np.log1p(unit_sqft) | -0.470 | 0.024 | -19.443 | | sum_residential_units | 0.046 | 0.112 | 0.414 | | ave_unit_sqft | 0.204 | 0.109 | 1.867 | | ave_lot_sqft | -0.233 | 0.041 | -5.716 | | ave_income | 0.420 | 0.190 | 2.213 | | hhsize | -0.331 | 0.140 | -2.362 | | jobs | 0.078 | 0.033 | 2.407 | | sfdu | -0.055 | 0.019 | -2.911 | | renters | -0.684 | 0.045 | -15.063 | | poor | 0.490 | 0.162 | 3.029 | | population | 0.356 | 0.188 | 1.892 | +-----------------------------------+-------------+------------+---------+ LCM RESULTS FOR SEGMENT 2 Null Log-liklihood: -13815.511 Log-liklihood at convergence: -13219.673 Log-liklihood Ratio: 0.043 +-----------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +-----------------------------------+-------------+------------+---------+ | np.log1p(residential_sales_price) | -0.000 | 3.430 | -0.000 | | np.log1p(unit_sqft) | -0.538 | 0.026 | -20.924 | | sum_residential_units | 0.202 | 0.121 | 1.668 | | ave_unit_sqft | 0.206 | 0.127 | 1.631 | | ave_lot_sqft | -0.213 | 0.047 | -4.487 | | ave_income | 0.827 | 0.217 | 3.814 | | hhsize | -0.416 | 0.148 | -2.813 | | jobs | 0.020 | 0.033 | 0.620 | | sfdu | -0.000 | 0.020 | -0.012 | | renters | -0.574 | 0.055 | -10.534 | | poor | 0.165 | 0.167 | 0.988 | | population | 0.544 | 0.206 | 2.639 | +-----------------------------------+-------------+------------+---------+ LCM RESULTS FOR SEGMENT 3 Null Log-liklihood: -13815.511 Log-liklihood at convergence: -13239.529 Log-liklihood Ratio: 0.042 +-----------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +-----------------------------------+-------------+------------+---------+ | np.log1p(residential_sales_price) | -0.000 | 3.888 | -0.000 | | np.log1p(unit_sqft) | -0.587 | 0.024 | -24.442 | | sum_residential_units | -0.076 | 0.123 | -0.621 | | ave_unit_sqft | 0.407 | 0.115 | 3.535 | | ave_lot_sqft | -0.198 | 0.048 | -4.124 | | ave_income | 1.822 | 0.254 | 7.176 | | hhsize | -0.470 | 0.150 | -3.139 | | jobs | 0.115 | 0.032 | 3.607 | | sfdu | -0.026 | 0.020 | -1.309 | | renters | -0.646 | 0.056 | -11.445 | | poor | 0.210 | 0.184 | 1.141 | | population | 0.723 | 0.220 | 3.282 | +-----------------------------------+-------------+------------+---------+ Time to execute model 'hlcm_estimate': 12.44s Total time to execute: 12.44s
orca.run(["elcm_estimate"])
Running model 'elcm_estimate' LCM RESULTS FOR SEGMENT industrial Null Log-liklihood: -17292.414 Log-liklihood at convergence: -14880.463 Log-liklihood Ratio: 0.139 +--------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +--------------------------------+-------------+------------+---------+ | np.log1p(non_residential_rent) | -0.000 | 0.901 | -0.000 | | sum_job_spaces | -0.316 | 0.042 | -7.488 | | sum_residential_units | 0.141 | 0.053 | 2.659 | | ave_unit_sqft | 0.095 | 0.023 | 4.065 | | ave_lot_sqft | 0.473 | 0.030 | 15.634 | | ave_income | -0.221 | 0.058 | -3.799 | | hhsize | -0.009 | 0.118 | -0.075 | | jobs | 0.723 | 0.046 | 15.702 | | poor | -0.393 | 0.033 | -11.862 | +--------------------------------+-------------+------------+---------+ LCM RESULTS FOR SEGMENT agriculture Null Log-liklihood: -990.112 Log-liklihood at convergence: -683.138 Log-liklihood Ratio: 0.310 +--------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +--------------------------------+-------------+------------+---------+ | np.log1p(non_residential_rent) | -0.000 | 3.779 | -0.000 | | sum_job_spaces | -0.932 | 0.294 | -3.172 | | sum_residential_units | 0.426 | 0.297 | 1.434 | | ave_unit_sqft | -0.654 | 0.089 | -7.367 | | ave_lot_sqft | 0.351 | 0.190 | 1.843 | | ave_income | -0.100 | 0.287 | -0.347 | | hhsize | -0.810 | 0.721 | -1.123 | | jobs | 1.676 | 0.315 | 5.312 | | poor | -0.454 | 0.165 | -2.758 | +--------------------------------+-------------+------------+---------+ LCM RESULTS FOR SEGMENT service Null Log-liklihood: -15883.232 Log-liklihood at convergence: -13151.724 Log-liklihood Ratio: 0.172 +--------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +--------------------------------+-------------+------------+---------+ | np.log1p(non_residential_rent) | -0.000 | 0.898 | -0.000 | | sum_job_spaces | -0.692 | 0.046 | -15.135 | | sum_residential_units | -0.008 | 0.052 | -0.146 | | ave_unit_sqft | -0.055 | 0.019 | -2.921 | | ave_lot_sqft | -0.006 | 0.031 | -0.206 | | ave_income | -0.450 | 0.051 | -8.845 | | hhsize | -1.366 | 0.145 | -9.402 | | jobs | 1.362 | 0.048 | 28.130 | | poor | -0.288 | 0.033 | -8.660 | +--------------------------------+-------------+------------+---------+ LCM RESULTS FOR SEGMENT retail Null Log-liklihood: -14750.360 Log-liklihood at convergence: -13907.690 Log-liklihood Ratio: 0.057 +--------------------------------+-------------+------------+---------+ | Component | Coefficient | Std. Error | T-Score | +--------------------------------+-------------+------------+---------+ | np.log1p(non_residential_rent) | -0.000 | 0.915 | -0.000 | | sum_job_spaces | -0.100 | 0.046 | -2.163 | | sum_residential_units | -0.414 | 0.061 | -6.774 | | ave_unit_sqft | -0.128 | 0.031 | -4.136 | | ave_lot_sqft | 0.062 | 0.036 | 1.705 | | ave_income | 0.636 | 0.056 | 11.454 | | hhsize | -1.451 | 0.131 | -11.047 | | jobs | 0.795 | 0.056 | 14.179 | | poor | 0.283 | 0.041 | 6.844 | +--------------------------------+-------------+------------+---------+ Time to execute model 'elcm_estimate': 6.37s Total time to execute: 6.38s