This is a guest post by Russell Jurney, a technologist and serial entrepreneur. His new startup, Cloud Stenography, will launch later this year. The article is an extension of a simple question on Twitter asking the importance of Map Reduce. Some subjects take much more than 140 characters.
The Technical Situation in Brief
The advent of the personal computer and the Visicalc spreadsheet were the foundation for a revolution in computing, business and life whereby normal people could carry out sophisticated accounting, analysis and forecasting to inform their decisions to arrive at more positive outcomes. As Moore’s law has progressed and processors have become faster, and computers inter-networked, large volumes of highly granular data have been collected. Analysis of terabyte datasets on the same level as a spreadsheet has been limited by the disparity of acceleration between processor speed and computer I/O (input/output) operations. Intel has produced ever faster processor clock speeds without accompanying disk, RAM or bus speeds. Put simply: We have cheap and numerous computing resources and abundant data, but bringing those resources to bear on that data to generate real value from it has proven exceedingly difficult.
I did they guest post on Lance Weatherby's blog to explain mapreduce to the people.
