Crisp overview of the dataset
- Accelerometer and Gyroscope readings are taken from 30 volunteers(referred as subjects) while performing the following 6 Activities. 1) Walking 2) WalkingUpstairs 3) WalkingDownstairs 4) Standing 5) Sitting 6) Lying.
- Readings are divided into a window of 2.56 seconds with 50% overlapping.
- Accelerometer readings are divided into gravity acceleration and body acceleration readings, which has x,y and z components each.
- Gyroscope readings are the measure of angular velocities which has x,y and z components.
- Jerk signals are calculated for BodyAcceleration readings.
- Fourier Transforms are made on the above time readings to obtain frequency readings.
- Now, on all the base signal readings., mean, max, mad, sma, arcoefficient, engerybands,entropy etc., are calculated for each window.
- We get a feature vector of 561 features and these features are given in the dataset.
- Each window of readings is a datapoint of 561 features.
Problem Framework
- 30 subjects(volunteers) data is randomly split to 70%(21) test and 30%(7) train data.
- Each datapoint corresponds one of the 6 Activities.
Problem Statement
- Given a new datapoint we have to predict the Activity it belongs to.
In [240]:
# All the necessary imports
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import numpy as np
In [241]:
# features are loaded from the features file given
features = list()
with open('/home/codename_sai/Desktop/1_HumanActivityRecognition/UCI HAR Dataset/features.txt') as f:
features = [line.split()[1] for line in f.readlines()]
In [242]:
print("No of features : {}".format(features.__len__()))
print(features[:10])
In [243]:
datapath = '/home/codename_sai/Desktop/1_HumanActivityRecognition/UCI HAR Dataset/'
In [244]:
# Train data is loaded from the X_train and y_train files given
X_train = pd.read_csv(datapath + 'train/X_train.txt', sep = '\s+',names=features)
# add subject column to the dataframe
X_train['Subject'] = pd.read_csv(datapath + 'train/subject_train.txt')
y_train = pd.read_csv(datapath + 'train/y_train.txt', sep = '\s+',names=['Activity'])
In [245]:
# Test data is loaded from the X_test and y_test files given
X_test = pd.read_csv(datapath + 'test/X_test.txt', sep = '\s+',names=features)
# add subject column to the dataframe
X_test['Subject'] = pd.read_csv(datapath + 'test/subject_test.txt')
y_test = pd.read_csv(datapath + 'test/y_test.txt', sep = '\s+',names=['Activity'])
In [246]:
# train data has 7352 datapoints and 562 features
X_train.shape
Out[246]:
In [247]:
# 1 - Walking
# 2 - WalkingUpstairs
# 3 - WalkingDownstairs
# 4 - Standing
# 5 - Sitting
# 6 - Lying
y_train['Activity'].value_counts()
Out[247]:
- y_train has 6 Activities. It is a multiclass classification
- It is a reasonably balanced dataset
In [248]:
type(X_train), X_train.shape, type(y_train), y_train.shape
Out[248]:
In [249]:
# Add class label to the X_train
X_train['Activity'] = y_train['Activity']
In [250]:
X_train.head()
Out[250]:
In [251]:
y_train.head()
Out[251]:
Data Cleaning - Dirtying hands
In [252]:
# Checking for duplicated
sum(X_train.duplicated()), sum(X_test.duplicated())
Out[252]:
- There are no duplicates as we see from above
In [253]:
# checking for NaN/null values
X_train.isnull().values.any(), X_test.isnull().values.any()
Out[253]:
- There are NaN's, let's locate them
In [254]:
# Taking the rows which has NaN
nan_rows_train = X_train[X_train.isnull().T.any().T]
nan_rows_test = X_test[X_test.isnull().T.any().T]
print('NaN rows in train')
nan_rows_train
Out[254]:
In [255]:
print('NaN rows in test')
nan_rows_test
Out[255]:
- Subject of 7351 indexed datapoint is missing. As it is 1/7352 we discard the complete row indexed 7351
In [256]:
# Here we get an extra feature for train as we have added class label to X_train
X_train.shape, X_test.shape
Out[256]:
In [257]:
# dropping the NaN/Null rows
X_train = X_train.dropna(how = 'any')
X_test = X_test.dropna(how = 'any')
In [258]:
# Drop the corresponding y_train and y_test rows
y_train = y_train.drop([7351])
y_test = y_test.drop([2946])
In [259]:
# Checking the shape after dropping Nan's
X_train.shape, X_test.shape
Out[259]:
In [260]:
# Cross Checking for the succesful drop of NaN
X_train.isnull().values.any(), X_test.isnull().values.any()
Out[260]:
In [261]:
X_train.columns
Out[261]:
From above we can see that,
- There are '()', '-' in column names, making the data less readable. Let's drop them.
- We can also see 'BodyBody' occuring multiple times. Let's remove of the Body.
In [262]:
# Removing '()','-' from column names for readability
X_train.columns = X_train.columns.str.replace('[()]','')
X_train.columns = X_train.columns.str.replace('[-]','')
X_test.columns = X_test.columns.str.replace('[()]','')
X_test.columns = X_test.columns.str.replace('[-]','')
In [263]:
# Replacing BodyBody from column name to Body which is a typo error
X_train = X_train.rename(columns = lambda x : str(x).replace('BodyBody','Body'))
X_test = X_test.rename(columns = lambda x : str(x).replace('BodyBody','Body'))
In [264]:
# Adding 'ActivityNames' column for interpretability.
X_train['ActivityNames'] = X_train['Activity']
In [265]:
# Replacing a Activity numbers with Activity Names for interpretability
X_train.ActivityNames = X_train.ActivityNames.replace(1,'Walking')
X_train.ActivityNames = X_train.ActivityNames.replace(2,'WalingUpstairs')
X_train.ActivityNames = X_train.ActivityNames.replace(3,'WalkingDownstairs')
X_train.ActivityNames = X_train.ActivityNames.replace(4,'Sitting')
X_train.ActivityNames = X_train.ActivityNames.replace(5,'Standing')
X_train.ActivityNames = X_train.ActivityNames.replace(6,'Laying')
In [266]:
# Copying X_train to df just to avoid all the labour done above
# incase of any change in df later
df = X_train
Featuring Engineering from Domain Knowledge - Dirtying mind
As we tend to learn about Accelerometer and Gyroscope and the features carefully we get following
- 1) In static activities (sit, stand, lie down) motion information will not be very useful.
- 2) In the dynamic activities (Walking, WalkingUpstairs,WalkingDownstairs) motion info will be significant.
- 3) Angle variables will be useful both in differentiating 'lie vs stand' and 'walk up vs walk down'.
- 4) Acceleration and Jerk variables are important in distinguishing various kinds of motion.
- 5) Magnitude and angle variables contain the same info as (strongly correlated with) XYZ variables, therefore we remove all the x,y,z component variables and retain Magnitude and angle variables.
- 6) We ignore the band variables as we have no simple way to interpret the meaning and relate them to physical activities.
- 7) Mean and std are important, skewness and kurtosis may also be hence we include all these.
- 8) We ignore the band variables as we have no simple way to interpret the meaning and relate them to physical activities.
After all the feature engineering and analysis we get following 38 features.
In [267]:
# final list of features
f = ['tBodyAccMagmean','tBodyAccMagstd','tBodyAccJerkMagmean','tBodyAccJerkMagstd','tBodyGyroMagmean',
'tBodyGyroMagstd','tBodyGyroJerkMagmean','tBodyGyroJerkMagstd','fBodyAccMagmean','fBodyAccMagstd',
'fBodyAccJerkMagmean','fBodyAccJerkMagstd','fBodyGyroMagmean','fBodyGyroMagstd','fBodyGyroJerkMagmean',
'fBodyGyroJerkMagstd','fBodyGyroMagmeanFreq','fBodyGyroJerkMagmeanFreq','fBodyAccMagmeanFreq',
'fBodyAccJerkMagmeanFreq','fBodyAccMagskewness','fBodyAccMagkurtosis','fBodyAccJerkMagskewness',
'fBodyAccJerkMagkurtosis','fBodyGyroMagskewness','fBodyGyroMagkurtosis','fBodyGyroJerkMagskewness',
'fBodyGyroJerkMagkurtosis','angletBodyAccJerkMean,gravityMean','angletBodyAccMean,gravity',
'angletBodyGyroJerkMean,gravityMean','angletBodyGyroMean,gravityMean','angleX,gravityMean',
'angleY,gravityMean','angleZ,gravityMean']
In [268]:
# we add 'Subject', Activity and 'ActivityNames' to the features
f1 = f + ['Activity','ActivityNames','Subject']
In [269]:
dfR = df[f]
In [272]:
dfR_test = X_test[f]
In [206]:
# Taking a new dataframe with reduced features
df = df[f1]
In [207]:
df.shape
Out[207]:
In [208]:
# Just for readability of feature names we remove Body and Magnitude
# and replace mean with 'Mean' and std with 'SD'
# tAccMean refers to tBodyAccMagMean
# fAccMean refers to fBodyAccMagMean
df = df.rename(columns = lambda x : str(x).replace('Body',''))
df = df.rename(columns = lambda x : str(x).replace('Mag',''))
df = df.rename(columns = lambda x : str(x).replace('mean','Mean'))
df = df.rename(columns = lambda x : str(x).replace('std','SD'))
In [209]:
# for plotting purposes taking datapoints of each activity to a different dataframe
df1 = df[df['Activity']==1]
df2 = df[df['Activity']==2]
df3 = df[df['Activity']==3]
df4 = df[df['Activity']==4]
df5 = df[df['Activity']==5]
df6 = df[df['Activity']==6]
In [210]:
sns.distplot(df1['tAccMean'],color = 'red',hist = True, kde = False,label = 'Walking')
sns.distplot(df2['tAccMean'],color = 'blue',hist = True, kde = False,label = 'Walking Up')
sns.distplot(df3['tAccMean'],color = 'green',hist = True, kde = False,label = 'Walking down')
sns.distplot(df4['tAccMean'],color = 'yellow',hist = True, kde = False,label = 'Sitting')
sns.distplot(df5['tAccMean'],color = 'm',hist = True, kde = False,label = 'Standing')
sns.distplot(df6['tAccMean'],color = 'orange',hist = True, kde = False,label = 'Laying')
sns.set_style("whitegrid")
plt.legend()
plt.tight_layout()
plt.show()
- Histogram of Body Acceleration Magnitude Mean above is a predictor of static vs dynamic activities.
- This is an example of data exploration in support of our heuristic variable selection using domain knowledge.
In [211]:
plt.subplot(2,2,1)
sns.distplot(df1['angletAccMean,gravity'],color = 'red',hist = True, kde = True,label = 'Walking')
sns.distplot(df2['angletAccMean,gravity'],color = 'blue',hist = True, kde = True,label = 'Walking Up')
sns.distplot(df3['angletAccMean,gravity'],color = 'green',hist = True, kde = True,label = 'Walking down')
plt.legend()
plt.subplot(2,2,2)
sns.distplot(df4['angletAccMean,gravity'],color = 'yellow',hist = True, kde = True,label = 'Sitting')
sns.distplot(df5['angletAccMean,gravity'],color = 'm',hist = True, kde = True,label = 'Standing')
sns.distplot(df6['angletAccMean,gravity'],color = 'orange',hist = True, kde = True,label = 'Laying')
sns.set_style("whitegrid")
plt.legend()
plt.tight_layout()
plt.show()
- Variance of the Angle between the gravity and static activities is very less.
- Variance of the Angle between the gravity and dynamic activities is more.
- This can also be used to distinguish the static and dynamic activities
In [212]:
sns.boxplot(x='ActivityNames', y='tAccMean',data=df, showfliers=False, saturation=1)
# plt.savefig('pair.png')
plt.xticks(rotation=90)
plt.show()
- If tAccMean is < -0.8 then the Activities are either Standing or Sitting or Laying.
- If tAccMean is > -0.6 then the Activities are either Walking or WalkingDownstairs or WalkingUpstairs.
- If tAccMean > 0.0 then the Activity is WalkingDownstairs.
- We can classify 75% the Acitivity labels with some errors.
In [213]:
sns.boxplot(x='ActivityNames', y='angleX,gravityMean', data=df, showfliers=False)
plt.xticks(rotation = 40)
plt.show()
- If angleX,gravityMean > 0 then Activity is Laying.
- We can classify all datapoints belonging to Laying activity with just a single if else statement.
In [215]:
sns.boxplot(x='ActivityNames', y='angleY,gravityMean', data = df, showfliers=False)
plt.xticks(rotation = 40)
plt.show()
- If angleY_gravityMean > 0.25 then the Activity is Walking Upstairs.
- We can classify 75% of the datapoints belonging to WalkingUpsatirs Activity with some errors.
Let's see the whether the features we hand picked heuristically from the domain knowledge makes sense
- Let's build a simple model with the reduced features and check it's preformance
Logistic Regression on reduced features
In [237]:
# import
from sklearn.linear_model import LogisticRegression
# instantiate LogisticRegression model
lgr = LogisticRegression()
In [270]:
lgr.fit(dfR,y_train)
Out[270]:
In [273]:
y_pred_lgr = lgr.predict(dfR_test)
In [275]:
from sklearn.metrics import accuracy_score
In [276]:
print(accuracy_score(y_test,y_pred_lgr))
- Using Logictic Regression we get an accuracy of 0.88 with the reduced feature set which we have taken with the help of domain knowledge.
- Therefore our feature engineering done using domain knowledge proved to be correct.
Random forests on reduced features
In [277]:
from sklearn.ensemble import RandomForestClassifier
In [279]:
clf = RandomForestClassifier(n_estimators=200)
clf.fit(dfR,y_train)
pred = clf.predict(dfR_test)
In [280]:
print(accuracy_score(y_test,pred))
- Using Random Forests we get an accuracy of 0.89 with the reduced feature set which we have taken with the help of domain knowledge.
- Therefore our feature engineering done using domain knowledge proved to be correct.
Tsne on the original dataset
- Let's do Tsne on the original dataset to visualze our data in lowe dimensions
In [216]:
X_train.shape
Out[216]:
In [ ]:
dft = X_train
In [218]:
del dft['Subject']
del dft['Activity']
del dft['ActivityNames']
In [219]:
dft.shape
Out[219]:
In [282]:
# import
from sklearn.manifold import TSNE
In [283]:
# Taking [2,5,10,30,50] as perplexities to see which one gives converges better
l = [2,5,10,30,50]
In [233]:
# loop to calculate and display tsne for different perplexities
X_tsne_p = []
for i in l:
tsne1 = TSNE( perplexity = i)
X1_tsne = tsne1.fit_transform(dft)
a1 = X1_tsne
a1 = a1.reshape((-1,2))
d_1 = pd.DataFrame({'x':a1[:,0],'y':a1[:,1]})
d_1['ActivityNames'] = df['ActivityNames']
X_tsne_p.append(d_1)
sns.FacetGrid(d_1,hue = 'ActivityNames',size = 8).map(plt.scatter,'x','y').add_legend()
print("Tsne for perplexity = ", i)
plt.show()
- It seems that for perplexity 50 data converges.
- Let's invidually plot the tsne plot for perplexity = 50
In [236]:
# Tsne plot for perplexity = 50
sns.FacetGrid(X_tsne_p[4],hue = 'ActivityNames',size = 8).map(plt.scatter,'x','y').add_legend()
print("Tsne for perplexity = 50", )
plt.show()
Conclusions
- Laying Activity is clustered together, it can be classified by an hyperplane
- Sitting and Standing are completely overlapping but, it seem that in higher dimensions it has an seperating hyperplane.
- Walking Downstairs are almost clustered at one place except a very few.
- Walking is also almost clustered together.
- Walking upstairs is distributed randomly.
- It gives us an insight that linear decision boundary does not clasify the datapoints effectively.
- Non Linear decision boundaries might work well as the classification boundary between the classes looks both linear and non linear.
- We can try Logistic Regression, despite of it being good at binary classification, we can give try as baseline model, as it is super fast.
- We can also try Linear SVM.
- Random forests and GBDT will do a fair amount of good job in classifying overlapping data and multiclass situation, let's try them aswell
In [ ]: