Why learn linear algebra here?

Machine Learning workloads

Convenience of linear algebra

Notation

This and the following sections are drawn from Stanford’s cs229.

• vector vs. matrix vs. tensor
• row, col notation
• indexing

  transpose

• what it means
• notation

pytorch examples

generation

• from a list torch.tensor()
• from numpy torch.from_numpy()
• torch.ones, torch.zeros
• torch.diag, torch.eye
• torch.rand, .randn, .randint

indexing, selection

• A[0][1]
• A[0,:]
• A.diagonal()

  masking

• mask = A == 1, mask = A.eq(1)
• A[mask]

messing with a tensor

• transpose A.T
• flatten A.flatten(), A.view(-1)
• reshape A.view(3,2)

simple operations

• A - 1
• A * 2
• A.mean(), A.mean(dim=0)
• A.sum(), A.sum(dim=0)

diagonal matrices are square identity as a special case commutative – AI = A = IA symmetry: square matrix is symmetric if A = A^T trace: sum of diagonal elements in a square matrix \(trA = \sum_{i=1}^N{A_{ii}}\)

Matrix Multiplication

cosine similarity

• diagram and examples
• usefulness for comparing vectors
• definition

  inner / dot product

• unbounded
• normalize by euclidean norm product, and it’s cos similariy

  norms

• euclidean distance (l2 norm)
• manhattan distance (l1 norm)
• general form of $l_p$ norms
• $l_\infty$ case

  outer product

  general 2d multiplication