Construction of a rating based on a bankruptcy prediction model Małgorzata Wrzosek, Arkadiusz Ziemba StatConsulting Sp. z o.o.
Agenda Introduction Project description Data preparation Model building Additional analysis Altman s model effectiveness verification (Polish market) Rating construction
Introduction Aim of the project Construction of bankruptcy prediction model concerning companies Motivation Model building for companies reliability assessment basing on data of sufficient volume and adequate quality Significance verification of the previously used ratios Comparison of models effectiveness Comparison of analytical methods effectiveness Data Companies registered in the National Court Register Financial ratios Tools Logistic regression, Random forests
Insolvency in Poland Relatively low insolvency ratio (insolvencies per 10 000 companies) in Europe Reversed insolvency trend since 2009 Actual number of bankruptcies higher than the number of bankruptcies announced Sources: [1] Report Raport nt. upadłości firm w Polsce w 2008r., Coface, 12th January 2009 [2] Report Insolvencies in Europe 2008/09, CreditReform, 10th February 2009 Countries Insolvency ratio Austria 224 Belgium 115 Czech Republic 50 Denmark 149 Estonia 108 Finland 107 France 215 Germany 96 Great Britain 76 Greece 6 Hungary 92 Ireland 82 Italy 18 Latvia 99 Lithuania 115 Luxembourg 233 Netherlands 103 Norway 142 Poland 3 Portugal 40 Slovakia 83 Slovenia 72 Spain 7 Sweden 108 Switzerland 113
DATA PREPARATION
Data description constraints 11 505 companies Legal form including: joint stock company, limited liability company, registered partnership and cooperative Turnover reported Financial statements for accounting years 2003-2006 provided The same level of bankruptcy in subsequent years No repetition of the same company in subsequent years the final data set
The final data set Information about ~5 000 companies Data divided into training and test samples in proportion 7:3 Prior probability of bankruptcy in the population 0.16% Year Count Count % Target Target % 2003 1292 27.0 258 20.0 2004 1584 33.1 264 16.7 2005 1248 26.1 208 16.7 2006 660 13.8 110 16.7
MODEL CONSTRUCTION
Target definition Bankrupt company a company for which bankruptcy procedure was started by the court in accordance with Polish bankruptcy legislation (despite whether the company finally went bankrupt or not) Minimum 6 months forecasting horizon year year year Financial statement Bankruptcy procedure year year year Financial statement Bankruptcy procedure
Prediction variables selection financial statement
The base model ID Variable Name Variable definition Standard coefficient CA Current Assets (logarithm) = logarithm of Current Assets 0.510 POS Profit on sale = Profit (Losses) on Sale 0.366 CL Current Liquidity = Current assets/current Liabilities 0.700 E/A Equity/Assets Ratio = Equity/Total Assets 0.266 ROE Return on Equity = Net Profit (Loss)/Equity 0.138 Gross Return on = Gross Profit (Losses)/Total GROA Assets Assets 0.185 NRFS Net Revenue from = Net Revenue from Sales and Sales Equivalent 0.360
The base model quality assessment Model ROC Area Test 0.868 Train 0.891 ROC curve and ROC area values Lift chart
Random forests Relatively novel approach (Breinman 2001) Characteristic Very robust in aspects of i.a. data preparation Very good prediction performance Incomprehensible structure Method description Training of many (e.g. 100, 1000) classification trees (bootstrap sample) Using random subset of m=sqrt(m) from M variables at each node Making a prediction by each tree to create a score The score as a proportion of votes Data set Sample 1 Sample 2 Sample 3 Tree 1 Tree 2 Tree 3 Score Sample Tree
Random forests quality assessment Model ROC Area Base 0.868 Randforest 0.886 ROC curve and ROC area values Lift chart
Forests Importance Fixed Assets Sales/Total Assets Cash/Total Assets Interest Coverage Assets Productivity Receivables Turnover Fixed Assets Productivity Inventory Turnover Cash/Assets Turnover Liabilities to Banks/Equity Liabilities to Banks/Liabilities Receivables from other Entities/Current Assets Short-term Receivables/Liabilities and Provisions for Liabilities
ADDITIONAL ANALYSIS
The influence of the company size on bankruptcy indicators Tasks Estimation of separate models for every segment with the set of variables from the base model Verification of the base model fitness in subsequent segments ROC curve Identified 4 segments No fixed assets - companies with no current assets Small, Middle, Large - companies with fixed assets not equal zero divided into three quantiles (terciles) according to their Total Assets Segments CA CL E/A NRFS ROE GROA POS No-fix assets 0.67-1.45-0.20-0.37 0.13-0.04-0.30 Small 0.57-0.72-0.40 0.06 0.09-0.18-0.41 Middle 0.34-0.78-0.15-0.25 0.13-0.36-0.30 Large 0.17-0.32-0.33-0.40 0.17-0.41-0.23
The influence of prediction horizon on predictive power of the model Tasks Estimation of additional model basing on the financial statements from 2 years prior to bankruptcy Comparison of this model against the base model (1 year horizon) Model ROC Area Base lag (-1) 0.798 Base 0.868 ROC curve and ROC area values
ALTMAN S MODEL
Altman's Z-score model Variable Name Coefficient WC/TA Working Capital*/Total Assets 0.717 RE/TA Retain Earnings/Total Assets 0.847 EBIT/TA MVE/TL Earnings Before Interest and Taxes/Total Assets Book Value of Equity/Book Value of Total Liabilities 3.10 0.420 S/TA Sales/Total Assets 0.998 *Working Capital = (Current Assets-Current Liabilities)
Tasks * Altman s model effectiveness verification (Polish market) Assessment of the Altman s Z-score discriminant function effectiveness Construction and assessment of the effectiveness of logistic regression model (with 5 variables used in Altman's model) Effectiveness comparison of the models with the base model * The verification didn t consider legal form
Logistic regression model Variable Name Standard Coeff Intercept - - WC/TA Working capital*/ Total assets -0.265 RE/TA Retain Earnings/Total Assets 0.251 EBIT/TA MVE/TL Earnings Before Interest and Taxes/Total Assets Book value of Equity/Book Value of Total Liabilities -0.398-0.461 S/TA Sales/Total Assets -0.197 *Working Capital = (Current Assets-Current Liabilities)
Models quality assessment Model ROC Area Base 0.868 Altman 0.764 Altman reest 0.847 ROC curve and ROC area values Lift chart
Further analysis Conclusions Altman's Z-score model should not be used for companies which are not listed on the stock exchange Further analysis Data set division according to the legal form Identification of 4 categories o Joint stock company, registered partnership, limited liability company and cooperative Models evaluation and assessment o Altman's Z-score model o Logistic regression model (with 5 variables used in Altman's model) o The base Model (re-estimated) o Random forests model (with 60 variables, 10 and 20 most relevant variables)
ROC Area values ROCA Joint stock Partnership Coop LLC ROCA JOIN Total Cases 106 147 136 1103 Bankruptcy 29 9 6 219 Altman s Z-score model 0.85 0.88 0.87 0.74 0.76 Regression model containing Altman's indicators The base model re-estimated for a given category of legal form Random forests (~60 variables) Random forests (best 10 variables) Random forests (best 20 variables) 0.89 0.89 0.88 0.82 0.85 0.88 0.88 0.99 0.86 0.88 0.92 0.92 0.99 0.88 0.90 0.90 0.84 0.98 0.85 0.87 0.91 0.90 0.98 0.87 0.89
RATING
Rating construction (1) DATA model building model testing score _fail_ 0,98 Y 0,97 Y 0,93 Y 0,9 Y 0,89 Y 0,83 N 0,81 Y 0,8 Y 0,79 Y 0,74 Y 0,74 N 0,72 N 0,72 Y 0,66 Y 0,65 Y 0,64 Y 0,63 N 0,58 Y 0,58 N 0,56 N 0,55 Y 0,54 Y 0,54 N 0,54 N 0,53 N 0,52 N 0,52 N 0,52 N 6 5 4 3 2 1
Rating construction (2) 6 5 4 3 2 1 Risk Category 0.000 0.059 0.164 0.373 0.485 0.808 1.000 Risk Rate Risk classes before re-weighting LIFT Cut off Population % 1 0.01% 0.0762 0.00% 50.05% 2 0.08% 0.472 0.05% 30.84% 3 0.32% 1.993 0.15% 13.65% 4 0.87% 5.361 0.57% 2.26% 5 1.80% 11.037 0.96% 2.92% 6 5.98% 36.714 5.61% 0.27% Risk classes after re-weighting to apriori = 0.16 %
Final Conclusions Good predictive power of the models based on financial ratios Surprisingly good results of Altman s indicators of bankruptcy Extending prediction horizon does not drastically constrain model efficiency Company size and legal form influence the set of bankruptcy indicators Choosing different modeling technique can improve predictive power of the model but only in a very limited extend Looking for new sources of information and combining different sources of information as the most promising path to improve model predictive power
Thank you Malgorzata.Wrzosek@statconsulting.com.pl Arkadiusz.Ziemba@statconsulting.com.pl