Web site "crawl"

Reply Subscribe

Thread Tools

Jan 1, 2007 | 07:57 PM

vital49

Thread Starter

Junior Member

Joined: Mar 2003

Posts: 1

Likes: 5

From: Purgatory

Web site "crawl"

I'm trying to find the "hidden" pages behind a webpage, that appears to be blank, but resolves to a blank page. I have a suspicion the site is being used, but not the main index page. Does anyone know of a way to "crawl" the site? A true site crawl only reveals links to other pages within the domain name. I don't think there are links on this site...only standalone HTML pages from the URL. Anyone know how to find these pages?

Jan 2, 2007 | 08:35 PM

vital49

Thread Starter

Junior Member

Joined: Mar 2003

Posts: 1

Likes: 5

From: Purgatory

Anyone? I really want to accomplish this!

Jan 2, 2007 | 09:24 PM

bandit

Senior Member

True Car Nut

Joined: Jan 2006

Posts: 8,029

Likes: 1

From: NEBF:06,07 | NYBF:06,07 | ONBF:06,07 | CNBF:06 & more............

This is one link i found

http://monkey.org/~provos/crawl/

looking for more...

Jan 2, 2007 | 09:29 PM

wjcollier07

Junior Member

Joined: Aug 2005

Posts: 0

Likes: 0

i do know frontpage has a feature where you can download an entire website..but umm, i'm not sure if thats kinda what you mean...it downloads every page from the root.