CATEGORY / Learning

Regression Analysis

Definition: discovering correlations between the outcome and the set of regressors (features)

: real random variable
: vector or random variables

Example
For wages, suppose
: hourly wage
: regressors (experience, gender, education)

: target regressor
: controls of components

Prediction: how can we use to predict well?
Inference: how does the predicted value of change if we change a component of holding the rest of the components of fixed?

Cost Function

Linear regression: solve a minimisation problem

minimise for

cost function:

Model Representation

: number of training examples
: input variables / features
: output variables / target variables
: single training example
: training example

Training set to learning algorithm to hypothesis (based on size of house) to estimates price

: parameters

Linear regression in one variable = univariate linear regression

Modularity Clustering

Method: define modularity score that we aim to maximise

Characteristic idea: compare communities to a random baseline graph that shares some properties with the actual graph, such as the number of edges and the degree of the nodes

1. Compute the number of edges within a community, then subtract the expected number of edges as per the baseline model ()
2. : if edge is between
: if there is no edge

Modularity score:
If score is large, then community is dense

Process:
1. Start with some partitioning, then move nodes between the groups to see if it improves the score
2. Use eigenvectors

Spectral Clustering

1. Create neighbourhood graph
2. Compute laplacian
3. Compute eigenvectors of laplacian, the ones with the smallest eigenvalues give new features
4. Run k-means clustering on new features

Eigenvectors

To capture global connectivity structure, eigenvectors are really useful. Results will be spectral clustering.

Spectrum of matrix: set of eigenvalues
Matrix: Laplacian of graph

Labelled graph:

Adjacency matrix:

Laplacian matrix:

Clusters

Cluster: points that are well-connected with each other (lots of edges)
Number of edges = volume
Volume per node = density
Cut value: separation of clusters
Cut(C) = 1 (number of edges between clusters)

1st criteria: density must be large
2nd criteria: there should not be too many edges between different clusters

Normalised Cut

Conductance

Good clusters are not too small, internally well-connected and separated from the rest of the nodes

Principal Component Analysis

Patterns = principal component = vector
Finds major axis of variation in data
Each data point expressed as a linear combination of patterns

Eigenvectors capture major direction that are inherent in the matrix
The larger the eigenvalue, the more important is the vector
Covariance matrix contains terms for all positive pairs of features

Data Analysis Application: Human-Generated Text Data

Examples: presenting news articles (based on relevancy), search engine presenting results (based on topic than popularity)

Mixed membership model: using a model exhibit multiple topics

Using LDA (latent dirichlet allocation) to analyse what MIT EECS are working on in their research
By Julia Lack

Steps:
1. Assemble abstracts from each professor’s published papers (over 900 abstracts)
2. Pre-processing: remove most common words / least common words
3. Choose for number of topics and run stochastic variational inference

k-Means Clustering Application: Understanding The Genetic Code

By Alexander Gorban and Andrei Zinoyvey

Question: Is it true that DNA breaks down into meaningful words of the same length? If so, how long are the words?

Steps:
1. Gather some real DNA of ACGT (fragment of full DNA sequence of particular bacteria: about 300,000 letters)
2. Break full sequence into strings of 300 letters each and make sure they do not overlap
3. Each string = data set (therefore, having 1017 data points)
4. Let
5. Divide the DNA string into substrings of
6. Count how many times the same substring occur
For : possible words: AA, AC, AG,…, TT
For : possible words
For : possible words
7. Run PCA (principal component analysis)
8. Pick out top 2 principals components
9. Plot 2 principal components for each data set

Observation: shows clear structure and symmetry

10. Run k-means (after normalisation)

Conclusion:
1. DNA is composed of words of 3 letters each as well as non-coding bits
2. The 3 letter words are known as codons
3. They encode amino acids

Previous Page

- PAGE 3 OF 4 -

Next Page

×