Description
About The Data Our goal for this lab is to construct a model that can take a certain set of features related to the Titanic and predict whether a person survived or not (0 or 1). Since we’re trying to predict a binary categorical variable (1 or 0), logistic regression seems like a good place to start from. The dataset that we’ll be using for this task comes from kaggle.com and contains the following attributes: PassengerId Survived (0 or 1) Pclass: Ticket class (1, 2, or 3 where 3 is the lowest class) Name Sex Age: Age in years SibSp: # of siblings / spouses aboard the Titanic Parch: # of parents / children aboard the Titanic Ticket: Ticket number Fare: Passenger fare Cabin: Cabin number Embarked: Port of Embarkation (C = Cherbourg, Q = Queenstown, S = Southampton) Note before starting: Please refer back to the matplotlib lab if you’re having trouble creating any graphs up to this point. You’re free to use any library to create your graphs, so don’t feel like you need to match this code 100%. Exploratory Data Analysis Let’s begin by importing some necessary libraries that we’ll be using to explore the data. Our first step is to load the data into a pandas DataFrame PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 1 0 3 Braund, Mr. Owen Harris male 22.0 1 0 A/5 21171 7.2500 NaN S 1 2 1 1 Cumings, Mrs. John Bradley (Florence Briggs Th… female 38.0 1 0 PC 17599 71.2833 C85 C 2 3 1 3 Heikkinen, Miss. Laina female 26.0 0 0 STON/O2. 3101282 7.9250 NaN S 3 4 1 1 Futrelle, Mrs. Jacques Heath (Lily May Peel) female 35.0 1 0 113803 53.1000 C123 S 4 5 0 3 Allen, Mr. William Henry male 35.0 0 0 373450 8.0500 NaN S From here, it’s always a good step to use describe() and info() to get a better sense of the data and see if we have any missing values. PassengerId Survived Pclass Age SibSp Parch Fare count 891.000000 891.000000 891.000000 714.000000 891.000000 891.000000 891.000000 mean 446.000000 0.383838 2.308642 29.699118 0.523008 0.381594 32.204208 std 257.353842 0.486592 0.836071 14.526497 1.102743 0.806057 49.693429 min 1.000000 0.000000 1.000000 0.420000 0.000000 0.000000 0.000000 25% 223.500000 0.000000 2.000000 20.125000 0.000000 0.000000 7.910400 50% 446.000000 0.000000 3.000000 28.000000 0.000000 0.000000 14.454200 75% 668.500000 1.000000 3.000000 38.000000 1.000000 0.000000 31.000000 max 891.000000 1.000000 3.000000 80.000000 8.000000 6.000000 512.329200 We can see that Age, Cabin, and Embarked contain missing values since this dataset contains 891 entries in total, and Age, Cabin, and Embarked only contain 714 non‑null entries, 204 non‑null entries, and 889 non‑null entries respectively. Thus, we will have to take care of these missing values. <class ’pandas.core.frame.DataFrame’> RangeIndex: 891 entries, 0 to 890 Data columns (total 12 columns): # Column NonNull Count Dtype 0 PassengerId 891 nonnull int64 1 Survived 891 nonnull int64 2 Pclass 891 nonnull int64 3 Name 891 nonnull object 4 Sex 891 nonnull object 5 Age 714 nonnull float64 6 SibSp 891 nonnull int64 7 Parch 891 nonnull int64 8 Ticket 891 nonnull object 9 Fare 891 nonnull float64 10 Cabin 204 nonnull object 11 Embarked 889 nonnull object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB Note, we can also make a plot of our missing data if we’d prefer to visualize it. Here we use seaborn’s barplot sns.barplot(x, y) and pass our DataFrame’s columns as the x axis and the sum of all missing values in each column in the y axis. since Embarked only has 2 missing values, it’s very hard to see, but there’s a slight raise in the y axis under Embarked. Tip: If you’re ever confused how a chained line of code works in this course, just break it down into multiple steps. For example, say you didn’t know how the piece of code above ‘y=titanic_data.isnull().sum().values’ gives us all of the missing values. Well, let’s break it down. titanic_data.isnull() gives us back the original DataFrame (titanic_data), but with True and False values placed where there is a missing value. PassengerId Survived Pclass Name Sex Age SibSp Parch Ticket Fare Cabin Embarked 0 False False False False False False False False False False True False 1 False False False False False False False False False False False False 2 False False False False False False False False False False True False 3 False False False False False False False False False False False False 4 False False False False False False False False False False True False … … … … … … … … … … … … … 886 False False False False False False False False False False True False 887 False False False False False False False False False False False False 888 False False False False False True False False False False True False 889 False False False False False False False False False False False False 890 False False False False False False False False False False True False 891 rows × 12 columns Then calling .sum() off of this gives us back a Series telling us how many true (missing values) were in each column. Recall that True is an alias for 1, which is why we can take the sum of True False columns. PassengerId 0 Survived 0 Pclass 0 Name 0 Sex 0 Age 177 SibSp 0 Parch 0 Ticket 0 Fare 0 Cabin 687 Embarked 2 dtype: int64 Finally, if you remember from lab 3, calling .index on this will give us the index labels (left side), and .values will give us the missing value counts for each column (right side), which is the array that we passed in as y. array([ 0, 0, 0, 0, 0, 177, 0, 0, 0, 0, 687, 2]) Keep this tip in mind when exploring other people’s notebooks on github or kaggle, since you’ll soon find out that it’s very common on kaggle for people to chain functions together, which can sometimes be hard to understand at first, but much easier to understand once you break it down into smaller chunks. Let’s continue on with our data exploration by next seeing how many people survived (1) and did not survive (0) in our dataset. To accomplish this, we can pass any column in our DataFrame into sns.countplot(x), which will list all of the unique values in that column along the x‑axis, and plot the total counts for each unique value along the y‑axis. So here we can see that majority of the people in our dataset did not survive (0). Did more men or females survive? Recall that hue parameter seaborn gives us access too. This will let us expand on the previous graph by also telling us how many from each value (0 or 1) were male and female. Interpretation: We can see that from those who did not survive (0), majority of them were male. How about from ticket class? Was the lower class less likely to survive? Interpretation: We can see that from those who did not survive (0), majority of them were from the lower class, 3. What did the Titanic age distribution look like? count 714.000000 mean 29.699118 std 14.526497 min 0.420000 25% 20.125000 50% 28.000000 75% 38.000000 max 80.000000 Name: Age, dtype: float64 Interpretation: The average age on the Titanic seems to be ~30, with 75% of people onboard being 38 years of age or younger. What’s the most common number of siblings one had with them on the Titanic? Interpretation: Majority of those onboard had 0 siblings/spouses also onboard, with the 2nd most popular being having 1 sibling/spouse onboard (most likely that 1 person onboard was a spouse). What was the Fare distribution on the Titanic? How much did the average person pay? count 891.000000 mean 32.204208 std 49.693429 min 0.000000 25% 7.910400 50% 14.454200 75% 31.000000 max 512.329200 Name: Fare, dtype: float64 Interpretation: The average person paid 32.204208, with 75% of people paying 31.000000 or less. One interesting note is that the min is 0. This could mean that there were people unaccounted for who managed to sneak in for free. Or someone who won a free ride or something. Data Preprocessing Let’s first take care of our missing values. Recall how much data was missing: For Age, our best bet would be to impute any missing values with the mean age. We can do this very quickly with pandas .apply(func). This will apply any function to every value along a column. If you’re not familiar with lambda functions, you can create a normal python function that accepts the age and mean_age, and returns the mean age if age is null, or the age itself if it’s not null. Then you can supply that function to .apply(func). So here we’re reassigning the titanic_data[‘Age’] column to titanic_data[‘Age’] after our function has been applied on it, which will essentially fill any missing age values with the mean age calculated. If we recreate our missing data plot, we can see that there are no longer any missing Age values. For Cabin, we have so much data missing (more missing than non‑null data) that performing any type of imputation seems like a bad idea since we don’t have much original data to work with. For this reason, we will just drop this column. I will go ahead and also drop the 2 missing Embarked rows while we’re at it, but you can choose to keep them if you’d like and impute them. Recalling .info(), we can see that there are no more missing values in this dataset. <class ’pandas.core.frame.DataFrame’> Int64Index: 889 entries, 0 to 890 Data columns (total 11 columns): # Column NonNull Count Dtype 0 PassengerId 889 nonnull int64 1 Survived 889 nonnull int64 2 Pclass 889 nonnull int64 3 Name 889 nonnull object 4 Sex 889 nonnull object 5 Age 889 nonnull float64 6 SibSp 889 nonnull int64 7 Parch 889 nonnull int64 8 Ticket 889 nonnull object 9 Fare 889 nonnull float64 10 Embarked 889 nonnull object dtypes: float64(2), int64(5), object(4) memory usage: 83.3+ KB Our next step is to handle categorical variables since machine learning algorithms can only understand numbers. The variables to consider are Name, Sex, Ticket, and Embarked. We’ll use dummy variables for Sex and Embarked and drop Name and Ticket. You can choose to do some type of feature engineering on Name and Ticket and compare it with our model without these features if you wish. Recall that a dummy variable is a variable that takes the value 0 or 1 to indicate the absence or presence of some category. Pandas has a convenient function pd.get_dummies(data, columns) that will automatically assign dummy variables for us. For example, if we include Sex in columns, it will create 2 new columns (sex_male, sex_female) and place a 1 for the one that’s true, and 0 in the other. So if a specific observation is female, we will place a 1 in sex_female and 0 in sex_male. One important note is that you should always add an additional drop_first=True parameter when using get_dummies. This will drop one of the columns created in the dummy process, since keeping all of them will result in multicollinearity. Now that our data is in the correct form, we’re ready to begin building our model. PassengerId Survived Pclass Age SibSp Parch Fare Sex_male Embarked_Q Embarked_S 0 1 0 3 22.0 1 0 7.2500 1 0 1 1 2 1 1 38.0 1 0 71.2833 0 0 0 2 3 1 3 26.0 0 0 7.9250 0 0 1 3 4 1 1 35.0 1 0 53.1000 0 0 1 4 5 0 3 35.0 0 0 8.0500 1 0 1 Creating our Logistic Regression Model We’re now ready to begin creating and training our model. We first need to split our data into training and testing sets. This can be done using sklearn’s train_test_split(X, y, test_size) function. This function takes in your features (X), the target variable (y), and the test_size you’d like (Generally a test size of around 0.3 is good enough). It will then return a tuple of X_train, X_test, y_train, y_test sets for us. We will train our model on the training set and then use the test set to evaluate the model. We’ll now import sklearn’s LogisticRegression model and begin training it using the fit(train_data, train_data_labels) method. In a nutshell, fitting is equal to training. Then, after it is trained, the model can be used to make predictions, usually with a predict(test_data) method call. LogisticRegression(max_iter=1000) Model Evaluation Now that we’ve finished training, we can make predictions off of the test data and evaluate our model’s performance using the corresponding test data labels (y_test). Since we’re now dealing with classification, we’ll import sklearn’s classification_report and confusion_matrix to evaluate our model. Both of these take the true values and predictions as parameters. precision recall f1score support 0 0.85 0.87 0.86 167 1 0.77 0.75 0.76 100 accuracy 0.82 267 macro avg 0.81 0.81 0.81 267 weighted avg 0.82 0.82 0.82 267 [[145 22] [ 25 75]] Not bad ! We could certainly do better, but we’ll leave it up to you to mess around with the data some more and see what you can imporove on. You can also check out the actual kaggle competition with the full Titanic dataset and compete on there with your classmates. Kaggle competitions are great for testing your new datascience skills. In [1]: import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns In [2]: from matplotlib import rcParams rcParams[‘figure.figsize’] = 15, 5 sns.set_style(‘darkgrid’) In [3]: titanic_data = pd.read_csv(‘titanic.csv’) titanic_data.head() Out[3]: In [4]: titanic_data.describe() Out[4]: In [5]: titanic_data.info() In [6]: sns.barplot(x=titanic_data.columns, y=titanic_data.isnull().sum().values) plt.xticks(rotation=45) plt.show() In [7]: titanic_data.isnull() Out[7]: In [8]: titanic_data.isnull().sum() Out[8]: In [9]: titanic_data.isnull().sum().values Out[9]: In [10]: sns.countplot(x=titanic_data[‘Survived’]) plt.show() In [11]: sns.countplot(x=titanic_data[‘Survived’], hue=’Sex’, data=titanic_data) plt.show() In [12]: sns.countplot(x=titanic_data[‘Survived’], hue=’Pclass’, data=titanic_data) plt.show() In [13]: sns.histplot(x=titanic_data[‘Age’].dropna()) plt.show() titanic_data[‘Age’].describe() Out[13]: In [14]: sns.countplot(x=titanic_data[‘SibSp’]) plt.show() In [15]: sns.histplot(x=titanic_data[‘Fare’]) plt.show() titanic_data[‘Fare’].describe() Out[15]: In [16]: sns.barplot(x=titanic_data.columns, y=titanic_data.isnull().sum().values) plt.xticks(rotation=45) plt.show() In [17]: mean_age = int(titanic_data[‘Age’].mean()) titanic_data[‘Age’] = titanic_data[‘Age’].apply(lambda age : mean_age if pd.isnull(age) else age) In [18]: sns.barplot(x=titanic_data.columns, y=titanic_data.isnull().sum().values) plt.xticks(rotation=45) plt.show() In [19]: titanic_data.drop(labels=[‘Cabin’], axis=1, inplace=True) titanic_data.dropna(inplace=True) In [20]: titanic_data.info() In [21]: titanic_data = pd.get_dummies(data=titanic_data, columns=[‘Sex’, ‘Embarked’], drop_first=True) titanic_data.drop(labels=[‘Name’,’Ticket’], axis=1, inplace=True) In [22]: titanic_data.head() Out[22]: In [23]: from sklearn.model_selection import train_test_split X = titanic_data[[‘PassengerId’, ‘Pclass’, ‘Age’, ‘SibSp’, ‘Parch’, ‘Fare’,’Sex_male’, ‘Embarked_Q’, ’Embarked_S’]] y = titanic_data[‘Survived’] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3) In [24]: from sklearn.linear_model import LogisticRegression logmodel = LogisticRegression(max_iter=1000) logmodel.fit(X_train,y_train) Out[24]: In [25]: predictions = logmodel.predict(X_test) In [27]: from sklearn.metrics import classification_report from sklearn.metrics import confusion_matrix print(classification_report(y_test,predictions)) print(confusion_matrix(y_test,predictions))