I LOL'd when Matt told me about this... Those people that complained about the limit... This is for you. I recall someone on the webappsec mailing list saying when they tried to scan their site it stopped due to "the limit". Wow they must have a big site ;)

Start at bottom for maximum effect...

_____________________________________________
From: Hoffman, Billy
Sent: Thursday, June 26, 2008 5:27 PM
To: Wood, Matt (); Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

This is too great. I'm posting this to Memestreams.

Billy Hoffman
--
Manager, HP Web Security Research Group
HP Software – Application Security Center
Direct: 770-343-7069

_____________________________________________
From: Wood, Matt ()
Sent: Thursday, June 26, 2008 5:27 PM
To: Wood, Matt (); Hoffman, Billy; Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

Stivo! you crazy! Change-set 27173. 6/21 @ 6:37pm in SimpleUrlCrawler.cs 

I guess the build-box is building with the debug symbols in it?

So the crawl limit is 2.1 billion right now  2^31-1

_____________________________________________
From: Wood, Matt ()
Sent: Thursday, June 26, 2008 5:19 PM
To: Hoffman, Billy; Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

Whoops! Here:

private void buildCrawlLimit()
{
crawlLimit = 1500;
#if DEBUG
crawlLimit = int.MaxValue;
#endif
}

Pretty sure the Labs build box is pumping out debug builds...

_____________________________________________
From: Hoffman, Billy
Sent: Thursday, June 26, 2008 5:19 PM
To: Wood, Matt (); Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

... ... STFU! Are you telling me the limit most people are bitching about doesn’t even exist? Haha, Should we even patch that?

Billy Hoffman
--
Manager, HP Web Security Research Group
HP Software – Application Security Center
Direct: 770-343-7069

_____________________________________________
From: Wood, Matt ()
Sent: Thursday, June 26, 2008 5:15 PM
To: Hoffman, Billy; Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

Haha… scrawlr may not have a limit…

I just set a break point in the function that checks it and it never gets called… apparently it got lost somehow…

_____________________________________________
From: Hoffman, Billy
Sent: Thursday, June 26, 2008 5:10 PM
To: Wood, Matt (); Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

Then explain this:
[Screen shot removed]

Billy Hoffman
--
Manager, HP Web Security Research Group
HP Software – Application Security Center
Direct: 770-343-7069

-----Original Message-----
From: Wood, Matt ()
Sent: Thursday, June 26, 2008 5:07 PM
To: Hoffman, Billy; Millar, Steve A
Subject: RE: uhhhh does Scrawlr really have a limit?

Nah, just a lot of parameters. We will only crawl 1500 pages, but we will audit more.

-----Original Message-----
From: Hoffman, Billy
Sent: Thursday, June 26, 2008 5:09 PM
To: Wood, Matt (); Millar, Steve A
Subject: uhhhh does Scrawlr really have a limit?

Guys,

I noticed a Chinese site offer Scrawlr for download. Its classic ASP so I decide to scan it with Scrawlr.

Site is: [Site Removed]

The only thing is, Scrawlr is saying it has visited 3879 pages so far and is still going. Perhaps a bug in our limiting?

Billy Hoffman
--
Manager, HP Web Security Research Group
HP Software – Application Security Center
Direct: 770-343-7069

I'm glad I got a copy. Is it going to be changed?