Classification and Regression, Part 2 - Exercise¶
import numpy as np
Exercise 1. Univariate Classification¶
About the data¶
The dataset you will use for today exercise is a modified version of Computers https://www.timeseriesclassification.com/description.php?Dataset=Computers
These problems were taken from data recorded as part of government sponsored study called Powering the Nation. The intention was to collect behavioural data about how consumers use electricity within the home to help reduce the UK's carbon footprint. The data contains readings from 251 households, sampled in two-minute intervals over a month. Each series is length 720 (24 hours of readings taken every 2 minutes). Classes are Desktop and Laptop.
Read the data¶
Load the numpy files X_train_computers.npy, X_test_computers.npy, y_train_computers.npy, y_test_computers.npy
X_train = np.load('data/X_train_computers.npy')
X_test = np.load('data/X_test_computers.npy')
y_train = np.load('data/y_train_computers.npy')
y_test = np.load('data/y_test_computers.npy')
X_train.shape, X_test.shape, y_train.shape, y_test.shape
((251, 1, 720), (250, 1, 720), (251,), (250,))
from sktime.transformations.series.impute import Imputer
X_train_no_out = X_train.copy()
X_test_no_out = X_test.copy()
# treat outliers as missing values
X_train_no_out[X_train > 8000] = np.nan
X_test_no_out[X_test > 8000] = np.nan
# replace missing values
imputer = Imputer(method="drift")
# the imputer is instance-wise, so it is ok to fit_transform also the test set (there is no data leakage)
X_train_imputed = imputer.fit_transform(X_train)
X_test_imputed = imputer.fit_transform(X_test)
Classification¶
After you removed the main problems in the data, you can start the classification process. Does the dataset need further preprocessing (normalization maybe)? If so, do it. Use the LabelEncoder to transform the target variable in a numerical format.
Goal¶
Your goal here is to find the best possible classification pipeline to predict the class of the computers dataset. You can use any classifier you saw in the course up to now.
Hyperparameter tuning¶
Try to find the best hyperparameters for your model. You can a validation set, cross validation or a GridSearchCV or RandomizedSearchCV for this task.
Evaluation¶
Don't overfit the test set! Only once you think you found the best model, test performance on the test set. Find the model that has the highest ROC AUC score.
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import roc_auc_score
Label Encoding¶
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
y_train_encoded = le.fit_transform(y_train)
y_test_encoded = le.transform(y_test)
How does a dummy classifier perform?¶
from sktime.classification.dummy import DummyClassifier
dummy = DummyClassifier()
cross_val_score(dummy, X_train_imputed, y_train_encoded, cv=5, scoring="roc_auc").mean()
0.5
Exercise 2. Multivariate Regression¶
About the data¶
The goal of this dataset is to predict total energy usage in kWh of a house. This dataset contains 138 time series obtained from the Appliances Energy Prediction dataset from the UCI repository. The time series has 24 dimensions. This includes temperature and humidity measurements of 9 rooms in a house, monitored with a ZigBee wireless sensor network. It also includes weather and climate data such as temperature, pressure, humidity, wind speed, visibility and dewpoint measured from Chievres airport. The data set is averaged for 10 minutes period and spanning 4.5 months.
Load the data¶
Load the numpy files X_train_AppliancesEnergy.npy, X_test_AppliancesEnergy.npy, y_train_AppliancesEnergy.npy, y_test_AppliancesEnergy.npy
X_train = np.load('data/X_train_AppliancesEnergy.npy')
X_test = np.load('data/X_test_AppliancesEnergy.npy')
y_train = np.load('data/y_train_AppliancesEnergy.npy')
y_test = np.load('data/y_test_AppliancesEnergy.npy')
X_train.shape, X_test.shape, y_train.shape, y_test.shape
((95, 24, 144), (42, 24, 144), (95,), (42,))
Regression¶
Similar to the previous exercise, you can use any regressor you saw in the course up to now. Find the best model in terms of MAE. As metric use neg_mean_absolute_error in the cross_val_score function (higher is better).
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import mean_absolute_error
Dummy Regressor¶
from sktime.regression.dummy import DummyRegressor
dummy = DummyRegressor()
cross_val_score(dummy, X_train, y_train, cv=5, scoring="neg_mean_absolute_error").mean()
-3.537271468144044