Friday, February 19, 2010

Dimensionality redux



Costa sunshine. Finally, a reson to go outside. Ultrahigh-dimensional variable selection as extreme case of Curse of Dimensionality. Machine Learning for common people. Haystack of simple algorithms easily embeddable to every web page. Scalable machine learning api ! Too much learning should skew the data. Approaching computational bounds and inseparability. Freedom. Get ready for Mahout weekend.

The mean integrated squared error of fits increases faster than linearly in p. That's the curse for you. Yellow buses marchning down the boulevard.

Fantastic! Efficient frontier is actually derived by constrained maximization of the objective (risk/return) function, under unit sum of asset ratios constraint. Lagrangian multiplier derivation in no time.

Production code releases. Is there a worse thing ? Boredom mixed with fear from failure. Ok, occasionally there is a twofold pleasure - both because something is accomplished and that something is finally over. However, weekly releases prevent pleasure from lasting long. Damn Agile. Ok, they might make the post-release pressure smaller because anything that's broken can be fixed next week, not year. Ah, the tradeoffs !

Reflecting on variability of Java/NIO performance across various linux environments. No free lunch, apparently. Linux distributed storage solutions. Benchmarking on IOPS. Mahout ways. Getting the simple dev environment up. Nice maven hierarchy. Pollution. Politics and dancing.