Create browsable copy of a website on my hard drive (Tool)

[retroborg]

Hello,
I would like to make a browsable copy of a website on my hard drive in HTML, HTM, including all its subdirectory structure, files (text, graphics, sound), links (As well as links pointing to other servers) and most importantly, its search engine. (So that when I type something in the “search index” box, it will find the appropriate page from that website, stored in my hard drive)


What would be the best tool/program for this task?

Thanks in advance

[kidzer]

File -> Save As in your browser would save the current page to your HD.

You're going to struggle to get it all on your HD though - the site says it has over 25,000 articles (so 1 page each right there I would expect), and the search function will probably run server-side code to search, so I dont expect you'll be able to get that.

EDit: Ooh yes, what saracen said also

[Saracen]

Retroborg, please be careful with this. Depending on what the site is, you're treading close to copyright issues with this. Even with a wiki site operating under a GFDL, there may (probably are) elements of most websites protected by copyright, including perhaps site design but certainly including some third-party elements, like advertising.

I'm assuming that the site you wish to copy is one to which you either own the copyright, or have permission from the owner to do this. If not, we'd have to close this thread.

[burble]

Assuming no copyright issues, grab a copy of wget and then do 'wget -r www.mysite.com' from a command prompt.

[ikonia]

that won't take the search engine or follow external links

[nvening]

surely you would not want it to follow external links because you would end up with half the internet being stored on your hdd?? lol

Also as said before i doubt very much you would be able to save the search feature as it would probably be running off some sort of database which would not be accessible without being the site admin

[retroborg]

Well, if I can't get the search engine it will be no good, as that’s what makes that site so cool and manageable.
Will I be able to get the search engine with any of the following tools?

HTTrack (Free)
http://www.httrack.com/

Offline Explorer Pro
http://www.metaproducts.com/mp/Offline_Explorer_Pro.htm

Offline commander
http://www.zylox.com/

wget (free)
http://www.gnu.org/software/wget/#downloading

Teleport pro / ultra (commercial)
http://www.tenmax.com/teleport/pro/

Wysigot Light
http://www.snapfiles.com/reviews/Wys...t/wysigot.html

Thanks in advance.

[directhex]

how do you intend to get a server-side search engine?

look at it this way, click "view source" on this page - you think there's a file stored on disk on the hexus servers called "showthread.php?t=117034" with this exact thread in it? no. the page is generated on request, which is how search engines work

[charleski]

Making the web content available in an unrestricted fashion grants you an implicit licence to download and view said content, copyright only comes into it if you were to redistribute the content.

If the site is dynamically generated using server-side scripting then nothing will allow you to download an exact copy. The tools mentioned, though, will generate a folder structure that you could search using Google Desktop or a similar search tool.

[Saracen]

Making the web content available in an unrestricted fashion grants you an implicit licence to download and view said content, copyright only comes into it if you were to redistribute the content.

Putting material on the web may be viewed as granting an implicit licence, but it will only be an implicit licence to do the things people would expect in the normal course of web usage. I would suggest that that includes browsing a website, together with the implicit caching issues, but does not imply wholesale download, storage or archive. Furthermore, it ONLY grants an implicit licence if the explicit terms of that licence don't preclude it.

retroborg didn't say what site it was. It may well be that he has permission to download this site. It might even be his own site. Or it may be that the T&Cs explicitly exclude such usage.

[mycarsavw]

Backstreet Browser

Crap name, not so crap program.

EDIT: Holy thread revival, sorry

[retroborg]

Thanks!
Will I be able to get the search engine as well with that app?

[mycarsavw]

I used it a few years ago on a small site, and once downloaded I used Windows Explorer/Search to search for stuff.

[mroz]

Thanks!
Will I be able to get the search engine as well with that app?

Strictly speaking that depends on how the searching is implemented, but in reality NO - as has already been explained multiple times.

Such sites utilise dynamic content. This means there are programs running on the server which perform the searches on their internal data & which most likely generate the pages themselves. Even if you ran the software on your own machine to support such programs, there's no way you can download the site's program code itself - it isn't visible to anyone other than the site maintainers.