Concept of Overfitting

[0.208575632499395, 0.20178321640091842, 0.15247400351622362, 0.10418786631408623, 0.09688701939648986, 0.09263963531131172, 0.08283677775295668, 0.06327629715761585, 0.06147112825631159] [0.7076444970946971, 0.7023260466949512, 1.7102279649595118, 8.775219946391115, 26.828407071463392, 117.45559764444442, 2039.7780917210393, 68181.7212153289, 714896.6962217717]
  1. Increasing Dataset
  2. Regularisation
# divide the new dataset into train & testX_train_new, y_train_new, X_test_new, y_test_new = train_test_split(X_new, y_new, test_size=0.5)

Now draw test and train error according to lamda = 1, 1/10, 1/100, 1/1000, 1/10000, 1/100000

Best Model (According to test performance)

as from the Train error using L2 & Test error using L2 graph, for each lambda values, our train error is almost the same but test error differs. we can see for lambda = 1, there is some test error whereas for lambda = 1/100000, test error is huge. so according to this graph lambda = 1/100 is a best model.

My Contribution

I went thorugh various tutorials, understood code & implemented this on my own. added data points & experimented with multiple degrees as well as captured train & test error. Also plotted the graphs.

Challanges

The first challange was to fit model with many degrees, used pipeline module from sklearn to fix this.

Experiments & Finding

Experiment tried with many (1/1000000, 1/10000000) lambdas values to see wheather train & test error increase or decrease.

What’s Next

Ensemble Technique

References

https://medium.datadriveninvestor.com/regression-in-machine-learning-296caae933ec

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jay Prakash Thakur

Jay Prakash Thakur

A Deep Learning Researcher & Developer