Commonly Used Distance Metrics
How to measure distance is a very fundamental problem in many domains. In this post, I give an overview of commonly used distance metrics. Importantly, I will dig into some metrics if they are related. I will also try to describe how each metric is used in actual problems.
Spatial Distance
Spatial distance is the most intuitive concept that measure how far two points are. Such a simple concept can be generalized to many different scenarios.
Euclidean Distance
Euclidean distance is the most basic concept that we have learned. It answers the smallest distance from point $A$ to point $B$.
Given point $A: (x_1, y_1, z_1, \cdots)$, and point $B: (x_2, y_2, z_2, \cdots)$ in a high-dimensional space, we have
[ d_{Enc} = \sqrt{(x_1-x_2)^2 + (y_1-y_2)^2 + (z_1-z_2)^2 + \cdots } ]
Such a simple concept has been adapted in many different forms. For instance, its variant, also commonly known as mean-square error (MSE), has been widely used in measure how good solution $A$ is, compared to baseline $B$. By using the Euclidean distance as an error metric, it is also applied to many optimization problems such as linear regression, low-rank approximation.