Skip to main content

Others

Concepts​

  • Matrix factorization
  • Singular value decomposition
  • Moore-Penrose Pseudoinverse
  • Hadamard product
  • Entropy
  • Kullback-Leibler Divergence
  • Gradient Descent

https://www.datacamp.com/community/tutorials/demystifying-mathematics-concepts-deep-learning

Norm​

In mathematics, anormis a function from a real or complex vector space to the nonnegative real numbers that behaves in certain ways like the distance from the origin: it commutes with scaling, obeys a form of the triangle inequality, and is zero only at the origin. In particular, the Euclidean distance of a vector from the origin is a norm, called the Euclidean norm, or 2-norm, which may also be defined as the square root of the inner product of a vector with itself.

A pseudo norm or seminorm satisfies the first two properties of a norm, but may be zero for other vectors than the origin.A vector space with a specified norm is called a normed vector space. In a similar manner, a vector space with a seminorm is called aseminormed vector space.

https://en.wikipedia.org/wiki/Norm_(mathematics)

Vector Norm​

Calculating the size or length of a vector is often required either directly or as part of a broader vector or vector-matrix operation.

The length of the vector is referred to as the vector norm or the vector's magnitude.

The length of a vector is a nonnegative number that describes the extent of the vector in space, and is sometimes referred to as the vector's magnitude or the norm.- The L1 norm that is calculated as the sum of the absolute values of the vector.

  • The L2 norm that is calculated as the square root of the sum of the squared vector values.
  • The max norm that is calculated as the maximum vector values.

https://machinelearningmastery.com/vector-norms-machine-learning