Regression using Neural Network

Keras a warpper API that runs on top of Tensorflow or theano is very popular and easy to use. Scikit learn also very popular libraries for machine learning.In this post I will show how to use keras and scikit learn to build neural network architecture in python and develop a regression linear model.

Jump to code and spare reading time:

Define a base model to be used to build a model for regression using scikitlearn API KerasRegressor.

Now we write a method for training the model we created above:

Define a method to visualize loss — we are using MSE loss for regression.

Main method to perform data preprocessing such as replace null values, standardize data and split into train and test.

Data processing from train and test data files:

Create predictions and submission file for kaggle like submission.

Grid Search Deep Learning Model Parameters

The previous example showed how easy it is to wrap your deep learning model from Keras and use it in functions from the scikit-learn library.

In this example, we use a grid search to evaluate different configurations for our neural network model and report on the combination that provides the best-estimated performance.

After creating our model, we define arrays of values for the parameter we wish to search, specifically:

Optimizers for searching different weight values. Initializers for preparing the network weights using different schemes. Epochs for training the model for a different number of exposures to the training dataset. Batches for varying the number of samples before a weight update. The options are specified into a dictionary and passed to the configuration of the GridSearchCV scikit-learn class. This class will evaluate a version of our neural network model for each combination of parameters (2 x 3 x 3 x 3 for the combinations of optimizers, initializations, epochs and batches). Each combination is then evaluated using the default of 3-fold stratified cross validation.

That is a lot of models and a lot of computation. This is not a scheme that you want to use lightly because of the time it will take. It may be useful for you to design small experiments with a smaller subset of your data that will complete in a reasonable time. This is reasonable in this case because of the small network and the small dataset (less than 1000 instances and 9 attributes).

Finally, the performance and combination of configurations for the best model are displayed, followed by the performance of all combinations of parameters.

This might take about 5 minutes to complete on your workstation executed on the CPU (rather than CPU). running the example shows the results below.

We can see that the grid search discovered that using a uniform initialization scheme, rmsprop optimizer, 150 epochs and a batch size of 5 achieved the best cross-validation score of approximately 75% on this problem.

In this post, you discovered how you can wrap your Keras deep learning models and use them in the scikit-learn general machine learning library.

You can see that using scikit-learn for standard machine learning operations such as model evaluation and model hyperparameter optimization can save a lot of time over implementing these schemes yourself.

Wrapping your model allowed you to leverage powerful tools from scikit-learn to fit your deep learning models into your general machine learning process

Related posts:

My definition of a PROgrammer is a coder that aims to produce professional quality code. It is a state of mind that unites coders beyond the language barriers.

The door is ajar and I push through slowly, unsure for a second if I should have knocked. He looks up from his desk. He has been expecting me of course, sending me a text saying he could see me…