RAID-Z

The original promise of RAID (Redundant Arrays of Inexpensive Disks) was that it would provide fast, reliable storage using cheap disks. The key point was cheap; yet somehow we ended up here. Why?

RAID-5 (and other data/parity schemes such as RAID-4, RAID-6, even-odd, and Row Diagonal Parity) never quite delivered on the RAID promise -- and can't -- due to a fatal flaw known as the RAID-5 write hole. Whenever you update the data in a RAID stripe you must also update the parity, so that all disks XOR to zero -- it's that equation that allows you to reconstruct data when a disk fails. The problem is that there's no way to update two or more disks atomically, so RAID stripes can become damaged during a crash or power outage.

To see this, suppose you lose power after writing a data block but before writing the corresponding parity block. Now the data and parity for that stripe are inconsistent, and they'll remain inconsistent forever (unless you happen to overwrite the old data with a full-stripe write at some point). Therefore, if a disk fails, the RAID reconstruction process will generate garbage the next time you read any block on that stripe. What's worse, it will do so silently -- it has no idea that it's giving you corrupt data.

There are software-only workarounds for this, but they're so slow that software RAID has died in the marketplace. Current RAID products all do the RAID logic in hardware, where they can use NVRAM to survive power loss. This works, but it's expensive.

There's also a nasty performance problem with existing RAID schemes. When you do a partial-stripe write -- that is, when you update less data than a single RAID stripe contains -- the RAID system must read the old data and parity in order to compute the new parity. That's a huge performance hit. Where a full-stripe write can simply issue all the writes asynchronously, a partial-stripe write must do synchronous reads before it can even start the writes.

Once again, expensive hardware offers a solution: a RAID array can buffer partial-stripe writes in NVRAM while it's waiting for the disk reads to complete, so the read latency is hidden from the user. Of course, this only works until the NVRAM buffer fills up. No problem, your storage vendor says! Just shell out even more cash for more NVRAM. There's no problem your wallet can't solve.

Partial-stripe writes pose an additional problem for a transactional filesystem like ZFS. A partial-stripe write necessarily modifies live data, which violates one of the rules that ensures transactional semantics. (It doesn't matter if you lose power during a full-stripe write for the same reason that it doesn't matter if you lose power during any other write in ZFS: none of the blocks you're writing to are live yet.)

If only we didn't have to do those evil partial-stripe writes...

Enter RAID-Z.

We deploy RAID Z with our solutions. I know there are some storage gurus on memestreams. Thoughts?

RAID Z : Jeff Bonwick's Weblog

Does Cinderella have a TV?

Curious Walt Disney World fans such as Leny Sou of Chicago want to know.

Of all the prizes Disney is giving away in its big, 15-month "Year of a Million Dreams" campaign, perhaps none is more tantalizing than a night in Cinderella Castle, the signature icon of the Magic Kingdom.

It's also the one prize that seems to have most taxed Disney officials' imaginations.

Details fall into place at Disney - Orlando Sentinel : Business

Colbert on O'Reilly:

O'Reilly on Colbert:

Colbert meets O'Reilly

This is your country on Medicare and Social Security.

The ratio of federal debt held by the public to GDP would climb from 37 percent currently to roughly 100 percent in 2030 and would continue to grow exponentially after that. The only time in U.S. history that the debt-to-GDP ratio has been in the neighborhood of 100 percent was during World War II. People at that time understood the situation to be temporary and expected deficits and the debt-to-GDP ratio to fall rapidly after the war, as in fact they did. In contrast, under the scenario I have been discussing, the debt-to-GDP ratio would rise far into the future at an accelerating rate. Ultimately, this expansion of debt would spark a fiscal crisis, which could be addressed only by very sharp spending cuts or tax increases, or both.

There is some very sound advice in here. The following statement seems so logical and obvious that one wonders why the Fed Chief has to say it.

Members of the Congress who put special emphasis on keeping tax rates low must accept that low tax rates can be sustained only if outlays, including those on entitlements, are kept low as well. Likewise, members who favor a more expansive role of the government, including relatively more-generous benefits payments, must recognize the burden imposed by the additional taxes needed to pay for the higher spending, a burden that includes not only the resources transferred from the private sector but also any adverse economic incentives associated with higher tax rates.

Unfortunately, there are a whole lot of people who are living in total denial about this.

Fed chief Bernanke's prepared testimony before Senate - Jan. 18, 2007

Stratfor focusing on the Iranian perspective...

By George Friedman

The Iraq war has turned into a duel between the United States and Iran. For the United States, the goal has been the creation of a generally pro-American coalition government in Baghdad -- representing Iraq's three major ethnic communities. For Iran, the goal has been the creation of either a pro-Iranian government in Baghdad or, alternatively, the division of Iraq into three regions, with Iran dominating the Shiite south.

The United States has encountered serious problems in creating the coalition government. The Iranians have been primarily responsible for that. With the death of Abu Musab al-Zarqawi in June, when it appeared that the Sunnis would enter the political process fully, the Iranians used their influence with various Iraqi Shiite factions to disrupt that process by launching attacks on Sunnis and generally destabilizing the situation. Certainly, Sunnis contributed to this, but for much of the past year, it has been the Shia, supported by Iran, that have been the primary destabilizing force.

So long as the Iranians continue to follow this policy, the U.S. strategy cannot succeed. The difficulty of the American plan is that it requires the political participation of three main ethnic groups that are themselves politically fragmented. Virtually any substantial group can block the success of the strategy by undermining the political process. The Iranians, however, appear to be in a more powerful position than the Americans. So long as they continue to support Shiite groups within Iraq, they will be able to block the U.S. plan. Over time, the theory goes, the Americans will recognize the hopelessness of the undertaking and withdraw, leaving Iran to pick up the pieces. In the meantime, the Iranians will increasingly be able to dominate the Shiite community and consolidate their hold over southern Iraq. The game appears to go to Iran.

Americans are extremely sensitive to the difficulties the United States faces in Iraq. Every nation-state has a defining characteristic, and that of the United States is manic-depression, cycling between insanely optimistic plans and total despair. This national characteristic tends to blind Americans to the situation on the other side of the hill. Certainly, the Bush administration vastly underestimated the difficulties of occupying Iraq -- that was the manic phase. But at this point, it could be argued that the administration again is not looking over the other side of the hill at the difficulties the Iranians might be having. And it is useful to consider the world from the Iranian point of view.

The Foundation of Foreign Policy

It is important to distinguish between the rhetoric and the reality of Iranian foreign policy. As a general principle, this should be done with all countries. As in business, rhetoric is used to shape perceptions and atte... [ Read More (1.3k in body) ]

Stratfor | Rhetoric and Reality: The View from Iran

Stratfor: Geopolitical Intelligence Report - January 4, 2007

The 'Surge Strategy': Political Arguments and Military Realities

By George Friedman

U.S. President George W. Bush is preparing a new strategy for Iraq.
According to reports being leaked to the media, the primary option
being considered is a "surge strategy," in which U.S. troop levels
in Iraq would be increased, particularly in the Baghdad region. The
numbers of additional troops that would deploy -- or that would not
be rotated home -- are unclear, but appear to be in the low tens of
thousands. This "surge" strategy is interesting in that it runs
counter to general expectations after the midterm elections in
November, when it appeared that the president was tied to a phased
withdrawal plan. Instead, Bush seems to have decided to attempt to
break out of the military gridlock in which the United States finds
itself. Therefore, the questions now are why the president is
considering this strategy and whether it will work.

As we have discussed previously, the United States appears to have
four strategic options in Iraq:

1. Massively increase the number of troops in Iraq, attempting to
break the back of both the Sunni insurgents and the Shiite militias
and create room for a political settlement.

2. Begin a withdrawal process that allows the Iraqis to shape the
politics of the country as they will -- and that leaves a huge
opportunity for Iran to fill the vacuum.

3. Abandon attempts to provide security for Iraq but retain forces
there, in a redeployed posture, with the goal of blocking any
potential Iranian moves toward the Arabian Peninsula.

4. Attempt to reach a political accommodation with Tehran that
concedes Iraq to the Iranian sphere of influence, in order to
provide guarantees against Iranian expansion southward. This
diplomatic option is compatible with all others.

Each of these options has strengths and weakness. The first option,
the surge, rests on the assumption that the United States has
enough troops available to make a difference on the ground in Iraq;
it also would decrease the strategic reserve for dealing with other
crises around the world. The phased withdrawal option eliminates
the need for Iraqi Shia and Iran to engage in political discussion
-- since, given time, U.S. forces would depart from Iraq and the
Shia would be the dominant force. The blocking strategy puts the
United States in the position of protecting Saudi Arabia (a Sunni
kingdom that doesn't want to appear to be seeking such protection)
against Iran -- a Shiite state that could, in that situation,
choose the time and place for initiating conflict. In other words,
this option would put U.S. forces on a strategic defensive in
hostile areas. The fourth option, diplomacy, assumes so... [ Read More (1.6k in body) ]

Stratfor on the Surge: A Crap Shoot

Conclusion

It seems like Rails is roughly 62% faster than Catalyst at this time. Keep in mind this benchmark does not take into account the ORM performance. This benchmark tests how quick the frameworks themselves dispatch methods and render views.

Also take into consideration when choosing a framework you need to look at the problem at hand. Catalyst can feed off Perl's vast CPAN resource library. Catalyst has features that Rails does not have. Catalyst's DBIC ORM supports multi-column primary keys and can do relationship mapping just by reading the schema! You don't even have to bother writing any has_many belongs_to definitions!

I am going to have to take a look into Django see how well it fairs in this benchmark. Perhaps an update on this?

Catalyst is still kind of slow. It allows you to harness the power of the CPAN, and you get ORM, MVC, etc. But its kinda slow. This doesn't whatsoever matter for my application, where lots of numbers are crunched, and so the DB is the limiting factor, but it is noteworthy.

I'm sure mst and crew are working on it.

Catalyst V Rails :: Speed

How to Change the IPv4 Address and Other Network Configuration Parameters

How to change the static IP of Solaris 10 boxen.

docs.sun.com: System Administration Guide: IP Services

PAR is a Cross-Platform Packaging and Deployment tool, dubbed as a cross between Java's JAR and Perl2EXE/PerlApp. It was developed by Audrey Tang. Currently, it is maintained via an active mailing list [par@perl.org] with help from numerous UsersAndContributors.

Update: The PAR distribution was split into two distributions with release 0.970: PAR and PAR-Packer. PAR-Packer contains all development tools such as pp, parl, etc. and requires a C compiler to install. PAR itself is now pure-Perl and can be installed even without a C development environment.

* PAR offers a wide set of Features.
* Current version of PAR: 0.970, released on approximately 2006-12-4 Download | ChangeLog | Requirements | CPAN page
* Current version of PAR-Packer: 0.970, released on approximately 2006-12-4 Download | ChangeLog | Requirements | CPAN page
* Repository - Accessible with SVK, Subversion, WebDAV, or any HTTP clients | Instructions for getting the development version
* CPAN Request Tracker: Bug and feature request queue for PAR. Get involved!

Note: Contrary to popular belief, you do not need a C compiler on Windows to install PAR – just follow the "Download" link, extract it to some temporary directory, and install it like any other perl module by following README's instructions. An incomplete compatibility list between PAR binary releases and Win32 Perl versions can be found here.

Main Page - Perl Archiving Toolkit - A Wikia wiki

Introducing Fiddler

What is Fiddler?

Fiddler is a HTTP Debugging Proxy which logs all HTTP traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP Traffic, set breakpoints, and "fiddle" with incoming or outgoing data. Fiddler is designed to be much simpler than using NetMon or Achilles, and includes a simple but powerful JScript.NET event-based scripting subsystem.

Internet explorer is a terrible thing. This can help. A little.

Fiddler HTTP Debugger - Fiddler