On Soft Maximum and Numerical Stability

When doing mathematical modeling, it is desirable that your function are continuos and have defined gradients. This is because there are numerous numerical methods that use a function’s gradient information in order to converge faster. However not mathematical functions have gradients. We shall see how to deal with these functions.

For example, max(a,b) doesn’t have a defined gradient. However, we can use log(exp(a) + exp(b)) to get a nice approximation for the max function. There is one problem though with this definition – it is numerically unstable in cases of a large a or b. We can resolve this instability using the following fact:

log( exp(a – k) + exp(b – k) ) = log( exp(a) + exp(b) ) – k.

Which means:

log( exp(x) + exp(y) ) = log( exp(xk) + exp(yk) ) + k

Which is guaranteed to be numerically stable. If you are wondering how can we model the min function, as in min(a,b), we just use the above soft maximum implementation with the arguments: -max(-a,-b).

There are are numerical stability issues as well, please see Eli Bendersky’s blog for softmax in the context of neural network, or more specifically, when diving an exponent by a some of exponents.

For more numerical stability issues see What Every Computer Scientist should know about Floating Point Arithmetic .

Advertisements

Setting up Blender to link Obj and Mtl files

Hello avid readers! I spent a part of my morning trying to link a Obj to it’s Mtl file, but it didn’t work. After some google-ing, I came across this blender stack exchange article.

To make sure that the Obj file ‘can talk’ to the Mtl file, you need to have both the Obj and Mtl files configured:

# OBJ FILE
mtllib [external .mtl file name]
...
f 3103/18700/18700 3081/18701/18700 3119/18702/18700
f 279/18703/18703 3118/18704/18703 274/18705/18703
f 265/18706/18706 3101/18707/18706 278/18708/18706

# usemtl material name (reference to the .mtl file)
usemtl MyMaterialName

Where MyMaterialName is going to be defined in the referenced Mtl file:

# MTL FILE
newmtl MyMaterialName
Ka 1.000 1.000 1.000     # white
Kd 1.000 1.000 1.000     # white
Ks 0.000 0.000 0.000     # black (off)
map_Ka myMap.jpg         # ambient texture map
...

For more reading on Mtl files, see here, and here