May 14, 2009

Frobenius norm with Hama

Recently, The Frobenius norm was nicely implemented as below by Samuel, which is the one of Hama committers. The Frobenius norm is submultiplicative and is very useful for linear algebra. This norm is often easier to compute than induced norms.


/** Frobenius Norm Mapper */
public static class MatrixFrobeniusNormMapper extends MapReduceBase implements
MatrixNormMapper {
@Override
public void map(IntWritable key, MapWritable value,
OutputCollector output, Reporter reporter)
throws IOException {
double rowSqrtSum = 0;
for (Map.Entry e : value.entrySet()) {
double cellValue = ((DoubleEntry) e.getValue()).getValue();
rowSqrtSum += (cellValue * cellValue);
}

nValue.set(rowSqrtSum);
output.collect(nKey, nValue);
}
}

/** Frobenius Norm Combiner */
public static class MatrixFrobeniusNormCombiner extends MapReduceBase
implements MatrixNormReducer {
private double sqrtSum = 0;

@Override
public void reduce(IntWritable key, Iterator values,
OutputCollector output, Reporter reporter)
throws IOException {
while (values.hasNext()) {
sqrtSum += values.next().get();
}
// Note: Tricky here. As we known, we collect each row's sum with key(-1).
// the reduce will just iterate through one key (-1)
// so we collect the max sum-value here
nValue.set(sqrtSum);
output.collect(nKey, nValue);
}
}


You can see this performance on this page

No comments:

Post a Comment