Note: Contents in bold are included in Coursera Machine Learning lectures.
A few topics are not identified: regularized regression, neural networks, and anomaly detection.
 Feature extraction and transformation
 Basic statistics: summary statistics, correlations, hypothesis testing
 Anomaly detection: kNN (kNearest Neighbors)
 Neural networks: perceptron, convolutional neural network
 Optimization: stochastic gradient descent, limitedmemory BFGS (LBFGS, Broyden–Fletcher–Goldfarb–Shanno)
Figure: scikitlearn machine learning algorithm map.
dlib has an alternative map.
Problems
Learning problems can be roughly categorized as either supervised or unsupervised.
Supervised learning builds a statistical model to predict or estimate an output (label) based on some inputs:
classification if label is categorical, regression if label is quantitative.
Unsupervised learning describes the relationships and structure among a set of inputs:
dimensionality reduction, clustering.
Other areas of machine learning:
Reinforcement learning is concerned with maximizing the reward of a given agent (person, business, etc).
linear regression
 Linear classifiers:
 Generative model: linear discriminant analysis (LDA), naive Bayes classifier;
 Discriminative model: Logistic regression (logit), support vector machines (SVM), perceptron;
 Isotonic regression;
 kmeans clustering;
 hierarchical clustering (dendrogram);
 Gaussian mixture;
 power iteration clustering (PIC);
 latent Dirichlet allocation (LDA);
Standardization is required in case of different units.
 singular value decomposition (SVD);
 principal component analysis (PCA): find the direction (or orthogonal directions) in a Euclidean space that explain the most sample variance (minimize the residual sum of squares);
Programming Tools
Machine Learning:
 R:
glmnet
, randomForest
, gbm
, e1071
(interface to libsvm), caret
, and more.
 Python: scikitlearn
sklearn

H2O: GLM (Generalized linear models), GBM (Gradient boosting machine; also supports random forest), GLRM (generalized lower rank models), deep neural network.

xgboost: Gradient boosting machine.
 Vowpal Wabbit
 Spark: MLlib
H2O scales the best (fastest without lesser accuracy) for the algorithms it supports on data over ~10M records and as long as it fits in memory of a single machine.
(Benchmark for GLM, RF, GBM)
Deep Learning:
 Python:
Pylearn2
, Theano
 Java: Deeplearning4j
 C++/CUDA: Caffe, cudaconvnet2
 TensorFlow
🏷 Category=Computation Category=Machine Learning