[Summary] Support Vector Learning for Ordinal Regression

Topic: Support Vector Learning for Ordinal Regression

Author: R. Herbrich

Summary:

Problems of ordinal regression arises in many fields. Two main scenario were considerd:

• Classification: Y is a finite unordered set.
• Regression estimation: Y is a metric space.

In ordinal regression, the problem shares the properties of both classification (finite set) and metric regression (an ordering among the elements of Y).

The Risk Formulation for Ordinal Regression
Given a set S={(x,y)}, x: feature vector; y: rank.
It want to train a function h(.) that can map the objects to the corresponding ranks.
Comparing the elements in S in a pairwise-way, and uses ERM principle to minimize the empirical risk.
Theorem 1 states that the empirical risk of a certain mapping h on a sample S is equivalent to the empirical risk based on the l0-1 loss of the related mapping p on the sample S' up to a constant factor t/(l^2) which depends neither on h nor on p. Thus, the problem of ordinal regression can be reduced to a classification problem on pairs of objects (the problem of preference learning).

Support Vector Machines for Ordinal Regression

From the figure, the algorithm degrades the dimensions of the generated samples by a function U. The rank conditions should be hold after the process.
The process is just like a classification. By using the method of SVM, the algorithm can find theta(r) (a hyperplane in the higher dimension) which can separate the samples with different rank.

Comments:
The paper proposed a algoithm to aggregate the method of classification and regression that can have a better performance. The paper has lots of terms of machine learning, so it's a little hard to totally understand.