Andrew Pavlo, Erik Paulson, Alexander Rasin, Daniel Abadi, David DeWitt, Sam Madden, and Michael Stonebraker:

There is currently considerable enthusiasm around the MapReduce (MR) paradigm for large-scale data analysis. Although the basic control flow of this framework has existed in parallel SQL database management systems (DBMS) for over 20 years, some have called MR a dramatically new computing model. In this paper, we describe and compare both paradigms. Furthermore, we evaluate both kinds of systems in terms of performance and development complexity. To this end, we define a benchmark consisting of a collection of tasks that we have run on an open source version of MR as well as on two parallel DBMSs. For each task, we measure each system's performance for various degrees of parallelism on a cluster of 100 nodes. Our results reveal some interesting trade-offs. Although the process to load data into and tune the execution of parallel DBMSs took much longer than the MR system, the observed performance of these DBMSs was strikingly better. We speculate about the causes of the dramatic performance difference and consider implementation concepts that future systems should take from both kinds of architectures.

Previously, from Stonebraker:

Database management systems are 20 years out of date and should be completely rewritten to reflect modern use of computers.

Recently:

This is a guest post by Russell Jurney, a technologist and serial entrepreneur. His new startup, Cloud Stenography, will launch later this year. The article is an extension of a simple question on Twitter asking the importance of Map Reduce.

A Comparison of Approaches to Large-Scale Data Analysis

Peggy Chi et al:

Although modern ease of access to technology enables many of us to obsessively document our lives, much of the captured digital content is often disregarded and forgotten on storage devices, with no concerns of cost or decay.

Can we design technology that helps people better appreciate captured memories? What would people do if they only had one more chance to relive past memories?

In this paper, we present a prototype design, PY-ROM, a matchstick-like video recording and storage device that burns itself away after being used. This encourages designers to consider lifecycles and human-computer relationships by integrating physical properties into digitally augmenting everyday objects.

From last year's best-of:

So many things these days are made to look at later. Why not just have the experience and remember it?

Burn Your Memory Away

Kacie Kinzer:

In New York, we are very occupied with getting from one place to another. I wondered: could a human-like object traverse sidewalks and streets along with us, and in so doing, create a narrative about our relationship to space and our willingness to interact with what we find in it? More importantly, how could our actions be seen within a larger context of human connection that emerges from the complexity of the city itself? To answer these questions, I built robots.

From the archive:

Is possibly noteworthy possibly a bot?

I always assumed he was a grad student.

Either the most prolific grad student ever, or possibly the single greatest purveyor of procrastination known to man.

We are all going to die. And for some of us ... it's the damned robots that are going to do it.

Oh how silly. The obvious application for this technology is robots that eat people. Duh.

Collected here are a handful of images of our recent robotic past, and perhaps a glimpse into the near future.

Feedback is the sound of musicians desperately trying to embody the superior self they glimpsed in the mirror and, potentially, turning themselves into robots in the process.

It is a short step from discovering that the world we know is a fake or a cheat to discovering that human beings are themselves factitious: that we are robots, ‘simulacra’ (the title of one of Dick’s novels), ‘just reflex machines’, ‘repeating doomed patterns, a single pattern, over and over’ in accordance with biological or economic ukases. Where other SF asks whether made-up entities (aliens, androids, emoting computers etc) deserve the respect we give real human beings, Dick more often asks whether we ought to view ourselves as fakes or machines.

Tweenbots

New work by Arvind Narayanan:

Operators of online social networks are increasingly sharing potentially sensitive information about users and their relationships with advertisers, application developers, and data-mining researchers. Privacy is typically protected by anonymization, i.e., removing names, addresses, etc.

We present a framework for analyzing privacy and anonymity in social networks and develop a new re-identification algorithm targeting anonymized social-network graphs. To demonstrate its effectiveness on real-world networks, we show that a third of the users who can be verified to have accounts on both Twitter, a popular microblogging service, and Flickr, an online photo-sharing site, can be re-identified in the anonymous Twitter graph with only a 12% error rate.

Our de-anonymization algorithm is based purely on the network topology, does not require creation of a large number of dummy “sybil” nodes, is robust to noise and all existing defenses, and works even when the overlap between the target network and the adversary’s auxiliary information is small.

See also, earlier work by Arvind Narayanan and Vitaly Shmatikov, from 2006-2007:

We present a new class of statistical de-anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on. Our techniques are robust to perturbation in the data and tolerate some mistakes in the adversary's background knowledge.

De-anonymizing Social Networks

Command-Line-Fu is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on and discussed - digg-esque voting is also encouraged so the best float to the top.

Command-line Fu: The best UNIX commands on the web

We have come across a botnet worm spreading around called "psyb0t". It is notable because, according to my knowledge, it:

* is the first botnet worm to target routers and DSL modems
* contains shellcode for many mipsel devices
* is not targeting PCs or servers
* uses multiple strategies for exploitation, including bruteforce username and password combinations
* harvests usernames and passwords through deep packet inspection
* can scan for exploitable phpMyAdmin and MySQL servers

Network Bluepill

Watcher is a runtime passive-analysis tool for HTTP-based Web applications. Watcher provides pen-testers hot-spot detection for vulnerabilities, developers quick sanity checks, and auditors PCI compliance auditing. It looks for issues related to mashups, user-controlled payloads, cookies, comments, HTTP headers, SSL, Flash, Silverlight, referrer leaks, information disclosure, Unicode, and more.

Major Features:

1. passive detection of security, privacy, and PCI compliance issues in HTTP, HTML, Javascript, and CSS
2. Works seamlessly with complex Web 2.0 applications while you drive the Web browser
3. non-intrusive, will not raise alarms or damage production sites
4. Real-time analysis and reporting - findings are reported as they’re found, exportable to XML
5. configurable domains with wildcard support
6. extensible framework for adding new checks

Casaba Security's Watcher tool for Web Security Auditing and Testing

Kellan Elliott-McCrea:

Don't let your design make promises you can't keep.

From the Economist:

He has to start deciding whom to disappoint.

From the archive, Siva Vaidhyanathan:

"It's the collapse of inconvenience. It turns out inconvenience was a really important part of our lives, and we didn't realize it."

Streams, affordances, Facebook, and rounding errors

Richard Hamming:

If you do not work on an important problem, it's unlikely you'll do important work.

From the archive:

Marge: I'd really like to give it a try!
Homer: I don't know, Marge, trying is the first step towards failure.

Also:

One passionate person is worth a thousand people who are just plodding along ...

Finally:

There are 260 million people in America, and you are one of them.

Get Excited and Make Things

Newssift is a unique search tool for business professionals offering access to a comprehensive database, indexing millions of articles from thousands of global business news sources. A next generation vertical search tool, searches are based on meaning and relationships, moving beyond traditional keyword search.

Qualitative news is a powerful determinant affecting stock prices and corporate reputations. However, this type of news has been difficult to search and nearly impossible to analyze through keyword searches alone. This tool offers business professionals deeply qualitative business news and more relevant results that cut out the commercial clutter found with typical keyword search. Newssift.com offers a better way to view the qualitative news, trends, and opinions that shape business.

Newssift streamlines the process of search and search refinement to help users become more informed. Moving beyond simple keyword search, Newssift offers search based on meaning, relationships and business themes. Content is aggregated and annotated by editors to ensure relevant and deep analysis of global business news. Newssift allows you to know before you go.

Clay Shirky:

If the old model is broken, what will work in its place?

The answer is: Nothing. Nothing will work.

Ted Nelson:

The trick is to make people think that a certain paradigm is inevitable, and they had better give in.

Ira Glass:

Not enough gets said about the importance of abandoning crap.

Newssift