Building a Logistic Regression Model
Libraries Required
import numpy as np
from datetime import datetime
import pandas as pd
import seaborn as sns
import plotly.express as px
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.subplots as sp
import psycopg2
import warnings
warnings.filterwarnings('ignore')
print('Libraries imported')Data Description
The variables description are as follow:
plcy_id Auto Policy ID
sample Identifies whether the record is available for training or is part of the holdout data
curnt_bi_low Bodily Injury Coverage Individual Limit on the State Farm policy for which they are applying
curnt_bi_upp Bodily Injury Coverage Occurence Limit on the State Farm policy for which they are applying
curnt_pd_lmt Property Damage Coverage Limit on the State Farm policy for which they are applying
curnt_coll_ded Collision Coverage Deductible on the State Farm policy for which they are applying (missing means no coverage)
curnt_comp_ded Comprehensive Coverage Deductible on the State Farm policy for which they are applying (missing means no coverage)
hh_veh_cnt Number of vehicles in the household
hh_cnt_auto Number of automobiles in the household
hh_cnt_mtrcyc Number of motorcycles in the household
hh_veh_w_coll_cnt Number of vehicles in the household with Collision Coverage
hh_veh_w_comp_cnt Number of vehicles in the household with Comprehensive Coverage
hh_veh_lien_cnt Number of vehicles in the household owned but with an outstanding loan
hh_veh_lease_cnt Number of vehicles in the household being leased
hh_veh_own_cnt Number of vehicles in the household owned outright by the named insured
veh_ownership Whether the vehicle on the policy is leased (Lease), owned outright (Own), or has an outstanding loan (Lien)
annual_mileage Annual mileage the vehicle is driven
veh_make Make of the vehicle on the policy
veh_model Model of the vehicle on the policy
veh_age Age of the vehicle on the policy
min_hh_veh_age Minimum age of vehicles in the household
max_hh_veh_age Maximum age of vehicles in the household
avg_hh_veh_age Average age of vehicles in the household
hh_drvr_cnt Number of drivers in the household
hh_min_age Minimum age of drivers in the household
hh_max_age Maximum age of drivers in the household
hh_avg_age Average age of drivers in the household
hh_min_mon_lic Minimum number of months licensed among drivers in the household
hh_max_mon_lic Maximum number of months licensed among drivers in the household
hh_avg_mon_lic Average number of months licensed among drivers in the household
hh_cnt_yth Number of youthful drivers in the household (less than 18 years old)
hh_cnt_female Number of female drivers in the household
hh_cnt_male Number of male drivers in the household
hoh_married Indicator of whether the head of household is married
hh_cnt_majr_viol Total number of major violations for drivers in the household
hh_cnt_minr_viol Total number of minor violations for drivers in the household
hh_cnt_lic_susp Number of drivers with a suspended license in the household
prior_insurer Name of the insurer immediately prior to coming to State Farm
time_w_carr Number of months insured with the prior insurer
inforce_ind Indicator of whether the household was insured immediately prior to coming to State Farm
multiline_ind Indicator of whether the household is also applying for a homeowners or renters policy with State Farm
homeowner_ind Indicator of whether the household owns their home
monthly_pay_ind Indicator of whether the household will pay the policy premium on a monthly payment plan
credit_score Credit score for the named insured driver (values of 0 indicate a credit report was not found)
hh_atf_clm_cnt_py1 Number of prior at-fault claims in the household within the last 1 year
hh_atf_clm_cnt_py2 Number of prior at-fault claims in the household within the last 2 years
hh_atf_clm_cnt_py3 Number of prior at-fault claims in the household within the last 3 years
hh_atf_clm_cnt_py4 Number of prior at-fault claims in the household within the last 4 years
hh_atf_clm_cnt_py5 Number of prior at-fault claims in the household within the last 5 years
hh_naf_clm_cnt_py1 Number of prior not-at-fault claims in the household within the last 1 year
hh_naf_clm_cnt_py2 Number of prior not-at-fault claims in the household within the last 2 years
hh_naf_clm_cnt_py3 Number of prior not-at-fault claims in the household within the last 3 years
hh_naf_clm_cnt_py4 Number of prior not-at-fault claims in the household within the last 4 years
hh_naf_clm_cnt_py5 Number of prior not-at-fault claims in the household within the last 5 years
future_clm_ind Indicator of whether the household had a claim within the first year of being insured with State FarmRead Dataset
path = 'D:/Freelancing/3-11-2023'
df = pd.read_csv(path+'/'+'DS_Work_Sample_Data.csv')
df.head()output:
df.shapeout: (60000, 56)
df[df['future_clm_ind'].isna()].shapeout: (20000, 56)
df_test = df[df['future_clm_ind'].isna()].copy()
df_test.head()output:
df[df['future_clm_ind']==0.0].shapeoutput:
(37532, 56)df[df['future_clm_ind']==1.0].shapeoutput:
(2468, 56)df_train = df[(df['future_clm_ind']==0.0) | (df['future_clm_ind']==1.0)]
df_test.shapeout:
(20000, 56)df_train.shapeout: (40000, 56)
Visulization with Training Data
Model with Feature Engineering
# create fontdicts for formatting figure text
axtitle_dict = {'family': 'serif','color': 'red','weight': 'bold','size': 16}
axlab_dict = {'family': 'serif', 'color': 'black','size': 14}
colours = ['forestgreen','dodgerblue','goldenrod', 'coral' , 'silver' , 'gold' , 'dodgerblue','forestgreen','dodgerblue','goldenrod', 'coral' , 'silver',
'forestgreen','dodgerblue','goldenrod', 'coral' , 'silver' , 'gold' , 'dodgerblue'];Handling with Missing Values
Missing Data - Initial Intuition
Here, we have missing data. In case of missing value, General Thumb Rules:
For features with less missing values- We can use regression to predict the missing values or fill with the mean of the values present, depending on the feature.
For features with very high number of missing values- It is better to drop those columns as they give very less insight on analysis. As there's no thumb rule on what criteria do we delete the columns with high number of missing values, but generally we can delete the columns, if we have more than 30-40% of missing values.
# print number of rows of each attributes for which the value is NULL.
print(df_train.isna().sum().sort_values(ascending = False))
print('Number of Duplicate Values in df : ' ,df_train.duplicated().sum())output:
In this case, more than 30% is considered as a boundary. Since, curnt_comp_ded is more than 30%. So, This feature is removed now. And for remaining feature:
Numerical feature missing values entries are removed.
df_train.drop(columns={'curnt_comp_ded'}, inplace=True)
## Removed useless features
df_train.drop(columns={'Unnamed: 0','plcy_id','sample'}, inplace=True)
# Missing means no coverage
df_train['curnt_coll_ded'].fillna(0, inplace=True)
# Removed the rows where
df_train = df_train.dropna(subset=list(df_train.columns))
df_train=df_train[df_train['credit_score']!=0].copy()
# mode_hh_veh_own_cnt = df_train['hh_veh_own_cnt'].mode().values[0]
# df_train['hh_veh_own_cnt'].fillna(mode_hh_veh_own_cnt, inplace=True)
# mode_hh_veh_lease_cnt = df_train['hh_veh_lease_cnt'].mode().values[0]
# df_train['hh_veh_lease_cnt'].fillna(mode_hh_veh_lease_cnt, inplace=True)
# mode_hh_veh_lien_cnt = df_train['hh_veh_lien_cnt'].mode().values[0]
# df_train['hh_veh_lien_cnt'].fillna(mode_hh_veh_lien_cnt, inplace=True)
# mode_veh_ownership = df_train['veh_ownership'].mode().values[0]
# df_train['veh_ownership'].fillna(mode_veh_ownership, inplace=True)
# mean_annual_mileage = df_train['annual_mileage'].mean()
# df_train['annual_mileage'].fillna(mean_annual_mileage, inplace=True)# print number of rows of each attributes for which the value is NULL.
print(df_train.isna().sum().sort_values(ascending = False))
print('Number of Duplicate Values in df : ' ,df_train.duplicated().sum())output:
curnt_bi_low 0
curnt_bi_upp 0
hh_cnt_female 0
hh_cnt_male 0
hoh_married 0
hh_cnt_majr_viol 0
hh_cnt_minr_viol 0
hh_cnt_lic_susp 0
prior_insurer 0
time_w_carr 0
inforce_ind 0
multiline_ind 0
homeowner_ind 0
monthly_pay_ind 0
credit_score 0
hh_atf_clm_cnt_py1 0
hh_atf_clm_cnt_py2 0
hh_atf_clm_cnt_py3 0
hh_atf_clm_cnt_py4 0
hh_atf_clm_cnt_py5 0
hh_naf_clm_cnt_py1 0
hh_naf_clm_cnt_py2 0
hh_naf_clm_cnt_py3 0
hh_naf_clm_cnt_py4 0
hh_naf_clm_cnt_py5 0
hh_cnt_yth 0
hh_avg_mon_lic 0
hh_max_mon_lic 0
veh_ownership 0
curnt_pd_lmt 0
curnt_coll_ded 0
hh_veh_cnt 0
hh_cnt_auto 0
hh_cnt_mtrcyc 0
hh_veh_w_coll_cnt 0
hh_veh_w_comp_cnt 0
hh_veh_lien_cnt 0
hh_veh_lease_cnt 0
hh_veh_own_cnt 0
annual_mileage 0
hh_min_mon_lic 0
veh_make 0
veh_model 0
veh_age 0
min_hh_veh_age 0
max_hh_veh_age 0
avg_hh_veh_age 0
hh_drvr_cnt 0
hh_min_age 0
hh_max_age 0
hh_avg_age 0
future_clm_ind 0
dtype: int64
Number of Duplicate Values in df : 0df_train.columnsout:
Index(['curnt_bi_low', 'curnt_bi_upp', 'curnt_pd_lmt', 'curnt_coll_ded',
'hh_veh_cnt', 'hh_cnt_auto', 'hh_cnt_mtrcyc', 'hh_veh_w_coll_cnt',
'hh_veh_w_comp_cnt', 'hh_veh_lien_cnt', 'hh_veh_lease_cnt',
'hh_veh_own_cnt', 'veh_ownership', 'annual_mileage', 'veh_make',
'veh_model', 'veh_age', 'min_hh_veh_age', 'max_hh_veh_age',
'avg_hh_veh_age', 'hh_drvr_cnt', 'hh_min_age', 'hh_max_age',
'hh_avg_age', 'hh_min_mon_lic', 'hh_max_mon_lic', 'hh_avg_mon_lic',
'hh_cnt_yth', 'hh_cnt_female', 'hh_cnt_male', 'hoh_married',
'hh_cnt_majr_viol', 'hh_cnt_minr_viol', 'hh_cnt_lic_susp',
'prior_insurer', 'time_w_carr', 'inforce_ind', 'multiline_ind',
'homeowner_ind', 'monthly_pay_ind', 'credit_score',
'hh_atf_clm_cnt_py1', 'hh_atf_clm_cnt_py2', 'hh_atf_clm_cnt_py3',
'hh_atf_clm_cnt_py4', 'hh_atf_clm_cnt_py5', 'hh_naf_clm_cnt_py1',
'hh_naf_clm_cnt_py2', 'hh_naf_clm_cnt_py3', 'hh_naf_clm_cnt_py4',
'hh_naf_clm_cnt_py5', 'future_clm_ind'],
dtype='object')Encoding
df_train['veh_ownership'].value_counts()output:
Own 19480
Lien 9549
Lease 722
Name: veh_ownership, dtype: int64df_train['veh_make'].value_counts()out:
A 2015
D 1952
B 1899
C 1893
F 1694
E 1670
G 1608
H 1490
J 1433
I 1407
K 1359
L 1334
N 1294
M 1259
P 1199
O 1195
Q 1112
R 1105
S 1008
T 579
U 336
V 293
W 233
X 200
Y 109
Z 75
Name: veh_make, dtype: int64# from sklearn.preprocessing import LabelEncoder
# Created a LabelEncoder object
# label_encoder = LabelEncoder()
# # Apply label encoding to the columns
# df_train['veh_make'] = label_encoder.fit_transform(df_train['veh_make'])
# df_train['veh_model'] = label_encoder.fit_transform(df_train['veh_model'])
# df_train['prior_insurer'] = label_encoder.fit_transform(df_train['prior_insurer'])
# df_train['veh_ownership'] = label_encoder.fit_transform(df_train['veh_ownership'])
# # One-Hot Encoding
df_train = pd.get_dummies(df_train, columns=['veh_make','veh_model','prior_insurer','veh_ownership'])### Convert Output Feature Float to int
df_train['future_clm_ind']=df_train['future_clm_ind'].astype(int)df_train['future_clm_ind'].unique()out:
array([0, 1])
df_train['future_clm_ind'].value_counts()out:
0 27891
1 1860
Name: future_clm_ind, dtype: int64Resampling The Data
from sklearn.utils import resample
# Separate the majority and minority classes
majority_class = df_train[df_train['future_clm_ind'] == 0]
minority_class = df_train[df_train['future_clm_ind'] == 1]
# Upsample the minority class to match the size of the majority class
minority_upsampled = resample(minority_class, replace=True, n_samples=len(majority_class), random_state=42)
# Combine the upsampled minority class with the majority class
df_train= pd.concat([majority_class, minority_upsampled])df_train['future_clm_ind'].value_counts()out:
0 27891
1 27891
Name: future_clm_ind, dtype: int64Feature Engineering
df_train.columnsout:
Index(['curnt_bi_low', 'curnt_bi_upp', 'curnt_pd_lmt', 'curnt_coll_ded',
'hh_veh_cnt', 'hh_cnt_auto', 'hh_cnt_mtrcyc', 'hh_veh_w_coll_cnt',
'hh_veh_w_comp_cnt', 'hh_veh_lien_cnt',
...
'prior_insurer_Rho', 'prior_insurer_Sigma', 'prior_insurer_Tau',
'prior_insurer_Theta', 'prior_insurer_Upsilon', 'prior_insurer_Xi',
'prior_insurer_Zeta', 'veh_ownership_Lease', 'veh_ownership_Lien',
'veh_ownership_Own'],
dtype='object', length=334)num_cols = ['curnt_bi_low', 'curnt_bi_upp', 'curnt_pd_lmt', 'curnt_coll_ded',
'annual_mileage', 'veh_age', 'min_hh_veh_age', 'max_hh_veh_age',
'avg_hh_veh_age', 'hh_min_age', 'hh_max_age',
'hh_avg_age', 'hh_min_mon_lic', 'hh_max_mon_lic', 'hh_avg_mon_lic',
'time_w_carr','credit_score']
df_cor = df_train[num_cols].copy()
Box Plot / Outliers Analysis in Numerical Variables
#create figure with 3 x 3 grid of subplots
fig = plt.figure(figsize=[26,22])
fig.suptitle('DISTPLOT OF DATA', fontsize=18, fontweight='bold')
fig.subplots_adjust(top=0.92);
fig.subplots_adjust(hspace=0.5, wspace=0.4);
for i ,col in enumerate(num_cols):
ax = fig.add_subplot(6, 3, i+1)
ax = sns.distplot(df_train[col], color='dodgerblue')
ax.axvline(df_train[col].quantile(q=0.25),color='green',linestyle='--',label='25% Quartile')
ax.axvline(df_train[col].mean(),color='red',linestyle='--',label='Mean')
ax.axvline(df_train[col].median(),color='black',linestyle='--',label='Median')
ax.axvline(df_train[col].quantile(q=0.75),color='blue',linestyle='--',label='75% Quartile')
# ax.text('skewness: {}' .format(str(round(df[col].skew(),3))), ha='right', va='center', size=11)
ax.set_xlabel(f'{col}', fontdict=axlab_dict)
ax.set_title(f'{col.upper()} skewness {round(df[col].skew(),3)}', fontdict=axtitle_dict)
ax.legend(fontsize=10) out:
#create figure with 3 x 3 grid of subplots
fig = plt.figure(figsize=[26,22])
fig.suptitle('BOXPLOT OF DATA', fontsize=18, fontweight='bold')
fig.subplots_adjust(top=0.92);
fig.subplots_adjust(hspace=0.5, wspace=0.4);
for i ,col in enumerate(num_cols):
ax1 = fig.add_subplot(6, 3, i+1);
ax1 = sns.boxplot(data = df_train, x=col , color= colours[i]);
ax1.set_title(f'{col}', fontdict=axtitle_dict)
ax1.set_xlabel(f'{col}', fontdict=axlab_dict)out:
# plot correlation matrix heatmap
fig, ax = plt.subplots(figsize=[13,5])
sns.heatmap(df_cor.corr(), ax=ax, annot=True, linewidths=0.05, fmt= '.2f',cmap='RdBu')
ax.tick_params(axis='both', which='major', labelsize=14)
ax.set_title('Dataset Correlation Matrix', fontdict=axtitle_dict)
fig.show()out:
import pandas as pd
import statsmodels.api as sm
vif_data = pd.DataFrame()
# Loop through each feature and calculate its VIF
for feature in df_cor.columns:
X = df_cor.drop(columns=[feature])
y = df_cor[feature]
model = sm.OLS(y, sm.add_constant(X)).fit()
vif = 1 / (1 - model.rsquared)
vif_data = vif_data.append({'Feature': feature, 'VIF': vif}, ignore_index=True)
# Filter features with VIF less than 15
selected_features = vif_data[vif_data['VIF'] < 25]['Feature'].tolist()
print("Features with VIF < 25:", selected_features)out:
Features with VIF < 25: ['curnt_bi_low', 'curnt_bi_upp', 'curnt_pd_lmt', 'curnt_coll_ded', 'annual_mileage', 'veh_age', 'time_w_carr', 'credit_score']difference = list(set(num_cols) - set(selected_features))
final_features = list(set(list(df_train.columns))-set(difference))
differenceout:
['hh_avg_age',
'hh_min_mon_lic',
'max_hh_veh_age',
'min_hh_veh_age',
'hh_max_age',
'avg_hh_veh_age',
'hh_max_mon_lic',
'hh_avg_mon_lic',
'hh_min_age']df_train['hh_veh_age_new']=(df_train['min_hh_veh_age']+df_train['max_hh_veh_age'])/2
df_train['hh_new_mon_lic']=(df_train['hh_max_mon_lic']+df_train['hh_min_mon_lic'])/2
df_train['hh_veh_age_new1']=(df_train['min_hh_veh_age']+1)*(1+df_train['max_hh_veh_age'])
df_train['hh_veh_age_new2'] = np.log(df_train['hh_veh_age_new1'])
df_train['hh_new_mon_lic1']=(df_train['hh_max_mon_lic']+1)*(1+df_train['hh_min_mon_lic'])
df_train['hh_new_mon_lic2'] = np.log(df_train['hh_new_mon_lic1'])
df_train['hh_new_age']=(df_train['hh_min_age']+1)*(df_train['hh_max_age']+1)
df_train['hh_new_age1'] = np.log(df_train['hh_new_age'])
df_train['annual_mileage1'] = np.log(df_train['annual_mileage'])
df_train['NOD']=df_train['hh_cnt_yth']+df_train['hh_cnt_female']+df_train['hh_cnt_male']
df_train['DV']=df_train['hh_cnt_majr_viol']+df_train['hh_cnt_minr_viol']+df_train['hh_cnt_lic_susp']
df_train['NOV']=df_train['hh_veh_cnt']+df_train['hh_cnt_auto']+df_train['hh_cnt_mtrcyc']+df_train['hh_veh_w_coll_cnt']+df_train['hh_veh_w_comp_cnt']+df_train['hh_veh_lien_cnt']+df_train['hh_veh_lease_cnt']+df_train['hh_veh_own_cnt']
df_train['ATF']=df_train['hh_atf_clm_cnt_py1']+df_train['hh_atf_clm_cnt_py2']+df_train['hh_atf_clm_cnt_py3']+df_train['hh_atf_clm_cnt_py4']+df_train['hh_atf_clm_cnt_py5']
df_train['ATF']=df_train['hh_naf_clm_cnt_py1']+df_train['hh_naf_clm_cnt_py2']+df_train['hh_naf_clm_cnt_py3']+df_train['hh_naf_clm_cnt_py4']+df_train['hh_naf_clm_cnt_py5']
# df_train.drop(columns={'min_hh_veh_age','max_hh_veh_age','hh_max_mon_lic','hh_min_mon_lic','hh_min_age','hh_max_age'},inplace=True)
Modelling
# X=df_train[li]
X = df_train.drop('future_clm_ind' , 1 )
y = df_train['future_clm_ind']
from sklearn.preprocessing import MinMaxScaler
# Initialize the MinMaxScaler
scaler = MinMaxScaler()
# Fit the scaler on the combined data and transform it
X = scaler.fit_transform(X)
from sklearn.model_selection import train_test_split
X_train , X_test , y_train , y_test = train_test_split(X , y ,test_size = 0.33 , random_state = 42)model_list = []
accuracy_list = []
recall_list = []
precision_list = []
f1_score_list= [] def Model_features(X_train , y_train , X_test , y_test , y_pred , classifier , model_name):
# fig ,ax = plt.subplots(figsize = (7,6))
accuracy , precision , recall , f1_s = round(accuracy_score(y_test , y_pred) , 3) , round(precision_score(y_test, y_pred, average="micro") ,3), round(recall_score(y_test , y_pred) ,3), round(f1_score(y_test , y_pred) , 3)
print(f'Accuracy Score is :{accuracy}')
print(f'Precision Score is :{precision}')
print(f'Recall Score is :{recall}')
print(f'f1 Score is :{f1_s}')
model_list.append(model_name)
accuracy_list.append(accuracy)
recall_list.append(recall)
precision_list.append(precision)
f1_score_list.append(f1_s)
# print(f'f1 Score is :{round(specificity_score(y_test , y_pred) , 3)}')
print(metrics.classification_report(y_test, y_pred)) from sklearn.linear_model import LogisticRegression
from sklearn import metrics
from sklearn.metrics import recall_score , classification_report , confusion_matrix ,roc_curve , roc_auc_score , accuracy_score
from sklearn.metrics import precision_recall_curve , auc ,f1_score , precision_score , recall_score
from sklearn.model_selection import cross_val_scorefrom sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
# Create an LDA object
lda = LinearDiscriminantAnalysis()from sklearn.model_selection import GridSearchCV
# Define hyperparameters and their possible values
param_grid = { 'C': [0.001, 0.01, 0.1,0.15], # Regularization parameter 'penalty': ['l1', 'l2'], # Regularization type 'max_iter': [100,180,200,210], # Maximum number of iterations}
# Create a logistic regression model
model_lr = LogisticRegression()
# Perform grid search with 5-fold cross-validation
grid_search = GridSearchCV(model_lr, param_grid, cv=5, scoring='accuracy')
# Fit the grid search to your data
grid_search.fit(X_train, y_train)
# Get the best hyperparameters
best_params = grid_search.best_params_
print("Best Parameters: ",best_params)
# Train the final model with the best hyperparameters
best_model = LogisticRegression(**best_params)
# Create a pipeline combining LDA and Logistic Regression
best_model = Pipeline([('lda', lda), ('logistic', best_model)])
best_model.fit(X_train, y_train)
# Use cross-validation to assess the model's performance
cv_scores = cross_val_score(best_model, X_train, y_train, cv=5, scoring='accuracy')
# Calculate the mean accuracy score from cross-validation
mean_accuracy = cv_scores.mean()
print("Cross-Validation Mean Accuracy:", mean_accuracy)
# Predict on the test data
y_pred = best_model.predict(X_test)
# Evaluate the model on the test data
test_accuracy = accuracy_score(y_test, y_pred)
print("Test Accuracy:", test_accuracy)out:
Best Parameters: {'C': 0.15, 'max_iter': 100, 'penalty': 'l2'}
Cross-Validation Mean Accuracy: 0.6822302940047236
Test Accuracy: 0.682600901732848Without Hyperparameter Tunning
# Create a logistic regression model
model_lr = LogisticRegression(random_state=0)
# Fit the model on the training data
model_lr.fit(X_train, y_train)
# Use cross-validation to assess the model's performance
cv_scores = cross_val_score(model_lr, X_train, y_train, cv=5, scoring='accuracy')
# Calculate the mean accuracy score from cross-validation
mean_accuracy = cv_scores.mean()
print("Cross-Validation Mean Accuracy:", mean_accuracy)
# Predict on the test data
y_pred = model_lr.predict(X_test)
# Evaluate the model on the test data
test_accuracy = accuracy_score(y_test, y_pred)
print("Test Accuracy:", test_accuracy)out:
Cross-Validation Mean Accuracy: 0.6820430495729678
Test Accuracy: 0.6835786843391819Model Evaluation
from sklearn.metrics import roc_auc_score, roc_curve
import matplotlib.pyplot as plt
# Predict probabilities of the positive class
y_prob = best_model.predict_proba(X_test)[:, 1]
# Calculate the ROC curve
fpr, tpr, thresholds = roc_curve(y_test, y_prob)
# Calculate the AUC (Area Under the Curve)
roc_auc = roc_auc_score(y_test, y_prob)
# Plot the ROC curve
plt.figure(figsize=(8, 6))
plt.plot(fpr, tpr, color='darkorange', lw=2, label=f'ROC Curve (AUC = {roc_auc:.2f})')
plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc='lower right')
plt.show()Model_features(X_train , y_train , X_test , y_test , y_pred , model_lr , "Logistic Regression") out:
Accuracy Score is :0.684
Precision Score is :0.684
Recall Score is :0.692
f1 Score is :0.686
precision recall f1-score support
0 0.69 0.68 0.68 9205
1 0.68 0.69 0.69 9204
accuracy 0.68 18409
macro avg 0.68 0.68 0.68 18409
weighted avg 0.68 0.68 0.68 18409from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
# Calculate the confusion matrix
conf_matrix = confusion_matrix(y_test, y_pred)
# Plot the confusion matrix
disp = ConfusionMatrixDisplay(confusion_matrix=conf_matrix, display_labels=best_model.classes_)
disp.plot(cmap='Blues', values_format='d')
plt.title('Confusion Matrix')
plt.show()out:
# Get coefficients and their magnitudes
coefficients = model_lr.coef_
feature_names = df_train.columns
# Calculate the magnitude of coefficients
coeff_magnitudes = abs(coefficients)
# Sort features by magnitude
sorted_features = sorted(zip(feature_names, coeff_magnitudes[0]), key=lambda x: x[1], reverse=True)
# Print the features and their magnitudes
for feature, magnitude in sorted_features:
print(f"Feature: {feature}, Magnitude: {magnitude}")out:
Feature: hh_veh_age_new2, Magnitude: 4.987714406869197
Feature: hh_veh_w_comp_cnt, Magnitude: 4.247272756209435
Feature: hh_new_mon_lic2, Magnitude: 3.3822582934243792
Feature: veh_model_U9, Magnitude: 2.9466963024771693
Feature: hh_new_age1, Magnitude: 2.8501033829418536
Feature: hh_veh_lien_cnt, Magnitude: 2.177153370671304
Feature: hh_cnt_lic_susp, Magnitude: 2.091976999078532
Feature: veh_model_L7, Magnitude: 1.8821370904843262
Feature: hh_atf_clm_cnt_py3, Magnitude: 1.8798106446510203
Feature: veh_model_M7, Magnitude: 1.8390724014369997
Feature: veh_model_Q8, Magnitude: 1.752096811969229
Feature: veh_model_J8, Magnitude: 1.7508455842446162
Feature: veh_model_T2, Magnitude: 1.6533504619092458
Feature: veh_model_U8, Magnitude: 1.6458480789281105
...
...Generate Prediction
df.shapeout:
(60000, 56)# print number of rows of each attributes for which the value is NULL.
print(df.isna().sum().sort_values(ascending = False))
print('Number of Duplicate Values in df : ' ,df.duplicated().sum())out:
future_clm_ind 20000
curnt_comp_ded 18564
curnt_coll_ded 14616
hh_veh_lease_cnt 7746
hh_veh_lien_cnt 7746
hh_veh_own_cnt 7746
veh_ownership 6850
annual_mileage 1928
hh_naf_clm_cnt_py2 0
monthly_pay_ind 0
hoh_married 0
hh_cnt_majr_viol 0
hh_cnt_minr_viol 0
hh_cnt_lic_susp 0
prior_insurer 0
time_w_carr 0
inforce_ind 0
multiline_ind 0
homeowner_ind 0
hh_atf_clm_cnt_py1 0
credit_score 0
hh_naf_clm_cnt_py3 0
hh_cnt_female 0
hh_naf_clm_cnt_py5 0
hh_naf_clm_cnt_py4 0
hh_atf_clm_cnt_py2 0
hh_atf_clm_cnt_py3 0
hh_atf_clm_cnt_py4 0
hh_atf_clm_cnt_py5 0
hh_naf_clm_cnt_py1 0
hh_cnt_male 0
Unnamed: 0 0
hh_cnt_yth 0
hh_avg_mon_lic 0
sample 0
curnt_bi_low 0
curnt_bi_upp 0
curnt_pd_lmt 0
hh_veh_cnt 0
hh_cnt_auto 0
hh_cnt_mtrcyc 0
hh_veh_w_coll_cnt 0
hh_veh_w_comp_cnt 0
veh_make 0
veh_model 0
veh_age 0
min_hh_veh_age 0
max_hh_veh_age 0
avg_hh_veh_age 0
hh_drvr_cnt 0
hh_min_age 0
hh_max_age 0
hh_avg_age 0
plcy_id 0
hh_max_mon_lic 0
hh_min_mon_lic 0
dtype: int64
Number of Duplicate Values in df : 0df.drop(columns={'curnt_comp_ded'}, inplace=True)
## Removed useless features
df.drop(columns={'Unnamed: 0','plcy_id','sample'}, inplace=True)
# Missing means no coverage
df['curnt_coll_ded'].fillna(0, inplace=True)
# Removed the rows where
# df_test = df_test.dropna(subset=list(df_test.columns))
# df_test=df_test[df_test['credit_score']!=0].copy()
mode_hh_veh_own_cnt = df['hh_veh_own_cnt'].mode().values[0]
df['hh_veh_own_cnt'].fillna(mode_hh_veh_own_cnt, inplace=True)
mode_hh_veh_lease_cnt = df['hh_veh_lease_cnt'].mode().values[0]
df['hh_veh_lease_cnt'].fillna(mode_hh_veh_lease_cnt, inplace=True)
mode_hh_veh_lien_cnt = df['hh_veh_lien_cnt'].mode().values[0]
df['hh_veh_lien_cnt'].fillna(mode_hh_veh_lien_cnt, inplace=True)
mode_veh_ownership = df['veh_ownership'].mode().values[0]
df['veh_ownership'].fillna(mode_veh_ownership, inplace=True)
mean_annual_mileage = df['annual_mileage'].mean()
df['annual_mileage'].fillna(mean_annual_mileage, inplace=True)# df_test.drop(columns={'curnt_comp_ded'}, inplace=True)
# ## Removed useless features
# df_test.drop(columns={'Unnamed: 0','plcy_id','sample'}, inplace=True)
# # Missing means no coverage
# df_test['curnt_coll_ded'].fillna(0, inplace=True)
# # Removed the rows where
# df_test = df_test.dropna(subset=list(df_test.columns))
# df_test=df_test[df_test['credit_score']!=0].copy()
df = pd.get_dummies(df, columns=['veh_make','veh_model','prior_insurer','veh_ownership'])
df['hh_veh_age_new']=(df['min_hh_veh_age']+df['max_hh_veh_age'])/2
df['hh_new_mon_lic']=(df['hh_max_mon_lic']+df['hh_min_mon_lic'])/2
df['hh_veh_age_new1']=(df['min_hh_veh_age']+1)*(1+df['max_hh_veh_age'])
df['hh_veh_age_new2'] = np.log(df['hh_veh_age_new1'])
df['hh_new_mon_lic1']=(df['hh_max_mon_lic']+1)*(1+df['hh_min_mon_lic'])
df['hh_new_mon_lic2'] = np.log(df['hh_new_mon_lic1'])
df['hh_new_age']=(df['hh_min_age']+1)*(df['hh_max_age']+1)
df['hh_new_age1'] = np.log(df['hh_new_age'])
df['annual_mileage1'] = np.log(df['annual_mileage'])
df['NOD']=df['hh_cnt_yth']+df['hh_cnt_female']+df['hh_cnt_male']
df['DV']=df['hh_cnt_majr_viol']+df['hh_cnt_minr_viol']+df['hh_cnt_lic_susp']
df['NOV']=df['hh_veh_cnt']+df['hh_cnt_auto']+df['hh_cnt_mtrcyc']+df['hh_veh_w_coll_cnt']+df['hh_veh_w_comp_cnt']+df['hh_veh_lien_cnt']+df['hh_veh_lease_cnt']+df['hh_veh_own_cnt']
df['ATF']=df['hh_atf_clm_cnt_py1']+df['hh_atf_clm_cnt_py2']+df['hh_atf_clm_cnt_py3']+df['hh_atf_clm_cnt_py4']+df['hh_atf_clm_cnt_py5']
df['ATF']=df['hh_naf_clm_cnt_py1']+df['hh_naf_clm_cnt_py2']+df['hh_naf_clm_cnt_py3']+df['hh_naf_clm_cnt_py4']+df['hh_naf_clm_cnt_py5']
# df_train.drop(columns={'min_hh_veh_age','max_hh_veh_age','hh_max_mon_lic','hh_min_mon_lic','hh_min_age','hh_max_age'},inplace=True)df.columnsout:
Index(['curnt_bi_low', 'curnt_bi_upp', 'curnt_pd_lmt', 'curnt_coll_ded',
'hh_veh_cnt', 'hh_cnt_auto', 'hh_cnt_mtrcyc', 'hh_veh_w_coll_cnt',
'hh_veh_w_comp_cnt', 'hh_veh_lien_cnt',
...
'hh_veh_age_new2', 'hh_new_mon_lic1', 'hh_new_mon_lic2', 'hh_new_age',
'hh_new_age1', 'annual_mileage1', 'NOD', 'DV', 'NOV', 'ATF'],
dtype='object', length=347)df.head()out:
# X=df_train[li]
X = df.drop('future_clm_ind' , 1 )
from sklearn.preprocessing import MinMaxScaler
# Initialize the MinMaxScaler
scaler = MinMaxScaler()
# Fit the scaler on the combined data and transform it
X = scaler.fit_transform(X)
y_test_pred = best_model.predict(X)df_predictions = pd.DataFrame({'glm_pred': y_test_pred})df_predictions['plcy_id'] = df_predictions.reset_index().indexdf_predictions['plcy_id']=df_predictions['plcy_id']+1df_predictions.sample(10)out:
sel_cols = ['plcy_id','glm_pred']
df_pred = df_predictions[sel_cols].copy()
df_pred=df_pred[df_pred['glm_pred']==1].copy()
df_pred.to_csv(path+'/'+'DS_Worked_Sample_Scored.csv')