Machine Learning in Python From scratch

A Vectorized Python 🐍 implementation using only NumPy, SciPy, and Matplotlib resembling as closely as possible to both provided and personally-completed code in the octave/matlab as part of the excellent Stanford University's Machine Learning Course on Coursera. The course is taught by Andrew Ng a genius and an excellent popularizer, which is a rare combination.

This course helped me write a blog answering the following question What is Machine Learning?

Supervised Learning

Given a set of labeled observations, find a function f which can be used to assign a class or value to unseen observations. Predictions should be similar to real labels.

Regression

In a regression problem, we are trying to predict results within a continuous output, meaning that we are trying to map input variables to some continuous function.

1. Linear regression with one variable to predict profits for a food truck

• 🐍 Demo | Linear Regression Notebook
• ▶️ Demo | Linear Regression Matlab

2. Regularized Linear regression with multiple variables to predict the prices of houses

• 🐍 Demo | Linear Regression with multiple variables Notebook
• ▶️ Demo | Linear Regression with multiple variables Matlab

Classification

In a classification problem, we instead are trying to predict results in a discrete output. In other words, we are trying to map input variables into discrete categories.

3. Regularized logistic regression to predict whether microchips passes quality assurance (QA)

• 🐍 Demo | Regularized Logistic Regression Notebook
• ▶️ Demo | Regularized Logistic Regression Matlab

4. Multi-class Logistic regression to recognize handwritten digits

• 🐍 Demo | Multi-class Logistic regression Notebook
• ▶️ Demo | Multi-class Logistic regression Matlab

5. Neural Networks (MLP) to recognize handwritten digits

• 🐍 Demo | Neural Networks Notebook Part I, Demo | Neural Networks Notebook Part II
• ▶️ Demo | Neural Networks Matlab Part I, Demo | Neural NetworksMatlab Part II

6. Support Vector Machines SVM ( with and without Gaussian Kernels)

• 🐍 Demo | Support Vector Machines Notebook
• ▶️ Demo | Support Vector Machines Matlab

• 🐍 Demo | SVM for Spam Notebook
• ▶️ Demo | SVM for Spam Matlab

Metrics to evaluate ML algorithms

Tackling Overfitting and Underfitting problems.

7. High Bias vs High Variance

• 🐍 Demo | High Bias vs High Variance Notebook
• ▶️ Demo | High Bias vs High Variance Matlab

Unsupervised Learning

Labeling can be tedious (too long, too slow), often done by humans and no real labels to compare. Unsupervised learning allows us to approach problems with little or no idea what our results should look like. We can derive structure from data where we don't necessarily know the effect of the variables. We can derive this structure by clustering the data based on relationships among the variables in the data. With unsupervised learning there is no feedback based on the prediction results.

Clustering

Group objects in clusters, similar within cluster, dissimilar between clusters.

8. K-means clustering algorithm for image compression

• 🐍 Demo | K-means Notebook
• ▶️ Demo | K-means Matlab

Dimensionality reduction

Reduce data set dimensions. Used for ata compression or big data visualization.

9. Principal Component Analysis (PCA) to perform dimensionality reduction

• 🐍 Demo | Principal Component Analysis Notebook
• ▶️ Demo | Principal Component Analysis Matlab

Anomaly detection

Identifies rare items (outliers) which raise suspicions by differing significantly from the majority of the data.

10. Anomaly detection algorithm to detect anomalous behavior in server computers of a data center

• 🐍 Demo | Anomaly detection algorithm Notebook
• ▶️ Demo | Anomaly detection algorithm Matlab

Recommender System

Predicts the rating or preference a user would give to an item.

11. Collaborative filtering recommender system applied to a dataset of movie ratings

• 🐍 Demo | Collaborative filtering recommender system Notebook
• ▶️ Demo | Collaborative filtering recommender system Matlab