FlashGot - a beginners guide to easy downloading with Firefox

FlashGot is the best Firefox download manager integration around. That is their words and they may very well be true. FlashGot ( http://flashgot.net/ ) is a tool to integrate the website with a download manager of your choice.

In regard to download managers then we are in business with tools like DownThemAll ( http://www.downthemall.net/ ), which is free and the commercial tool ReGet ( http://www.reget.com/en/ ). I will demonstrate both here in this micro tutorial. We will also have a quick look into some of the configuration possibilities of FlashGot.

Quote: "Spend less time downloading and more time on organizing". 

The image screenshot below shows the Preferences for FlashGot. From the red marked circles you can see that I have a possibility to choose the default download manager as either DTA (Down Them All), ReGet or the build in browser download capability. I am most familiar with ReGet as it gives me the most information and control, but the choice matters little as both tools manage to download properly.

FlashGot supports a wide range of download managers whereas the most popular probably is DownThemAll, FlashGet, GetRight, jDownloader and ReGet.

FlashGot basically has two modes, either you choose to download 1 link or you choose to download the entire page, and everything that is linked to from the page, one level deep. Let's see how things looks using DownThemAll and choosing 'FlashGot All'.

If we switch the default download manager to ReGet then we get a slightly different picture although they are quite similar. With DTA some files are checked as which to download. With ReGet all files are picked as to be downloaded. Both tools require you to hand pick which, because in this case you may want only the default size video and the HD largest quality video.

With both DTA and ReGet the next step is to click OK and let the downloading begin. If you right click a link and choose 'FlashGot Link' which is the single file download mode then you get thrown directly into the single file download mode of the download manager you have set the preference too. Again it is the OK button and your download starts.

The force of download managers like DTA and ReGet are their ability to stop and resume huge downloads with multiple threads. Downloading a 2.18Gb file may take a little while and the best way is to use a tool like DTA or ReGet. Give it a try.

Offline Explorer - site ripping for beginners

Here is a small tutorial with pictures for you to better understand the steps in creating a useable Offline Explorer project which downloads what you want and nothing more.

First off, you will have to get the tool. Which you buy/get from http://www.metaproducts.com. The cheapest version is about 50 USD and the most expensive is about 500 USD. The price for the product may seem quite high, but believe me, even the most expensive version is worth the cost if you excell in siteripping. It will save you an emmense amount of time. The current version is 5.6, my own version is an old one I bought a long time ago and will be using v4.8 for demonstration use.

The target site for this demonstration is http://www.betanews.com. Betanews is a site which holds some downloadable files (programs), which mainly are the newest versions of various software. We will have a wack at the WWW & Internet section of tools. Have a look below for a peek of the site.

I'd suggest you visit the site and pan the links, so that you have some feeling of what we're about to download. Click the circled item and see what happends.

Make note of the URL in the address bar of the browser and start Offline Explorer and create a New Project. Type the URL and it will look something like the picture below.


One of the things you should make not of is what you tell OE to download. Leave it as it is for now and take not that you are downloading from the site and 1 level down in the sites hierarchy.

Click OK, and start the project. Quickly switch the top tab's to Queue. There you will be able to see what OE intends to download. It will look something like the picture below.


You can pause the downloading by pressing F9, pressing F9 again will resume the download. For more information on how to make cool use of this feature then look for HAM download here on this site.

Scroll down in the list and notice the links with the /download/ path. These are the ones of interest, the rest is "junk" and without value to us. This can be compared to make note of the links for the .jpg and the .avi / .wmv files on a site (IF that is what you want - hehe).


So we have to filter the /download/ links. Go back into your project and find 'URL Filters -> Directory' and switch to 'View included directories keyword'. Type '/download/' without the ' and add it to the list of keywords are included. Also go back up in the project and find 'File Filters -> Archive'. Add the two items 'rar' and 'exe' to the list of what to download and uncheck everything else under the File Filters. Be sure that the File Filters -> Archive is set to 'Load using URL filters settings'. Take a close look at the screenshots below. Delete the files downloaded and start the project again.


One thing you will notice is that you don't actually get the files (.exe and .rar) files because you still have a setting to download only 1 level deep. So increase the level to 2. Look below.


Delete your downloaded files and start the project again. You will see that you lack the files because they use another link structure than '/download/'. The keyword you need to include is '/sendfile/'. Add this to the directories keywords like shown below.


This time you will be able to see the list below, and that is what you will download. Let the project finish and explore the files you downloaded. If you do not know where the files are located then click the 'Options' button of OE and look under 'File Location'.


What you hopefully have learned is the use of OEEE's Filter settings and how to limit what you download. The list of keywords can greatly improve the quality of you download.

Fixing the web? - downloading by forums

This summary is not available. Please click here to view the post.

Use the net! - usenet downloading

You've been briefly introduced to torrents, rapidshare and web ways of getting what you are looking for.

It is important to say that none of the sources we have covered so far are valid as sources for official csvs. Only direct source site can be a valid source. Yet you can be lucky and cut down your download times alot if you utilize torrents or the web.

But still!, there are faster ways and if you have a fast inet line and alot og free GB (woops, let's say TB) on your hdd then you can dig into the usenet alias news groups. You will have to buy a usenet account. They cost anything from 9.95$ USD to 25$ USD per month. It is pretty cheap and you get excellent service for that amount of money.

There are several usenet providers out there which are considered as good, but put you money on one of these three

• Giga News ( http://www.giganews.com/ )
  
• Easy News ( http://www.easynews.com/ )
  
• Usenet Server ( http://www.usenetserver.com/en/index.php )

Once you have an usenet account then you need a download client. Much like the μTorrent client for use with torrents you need special software for usenet. The choice of usenet clients is based on religion, but a few is worth mentioning.

• Forte Agent ( http://www.forteinc.com/ )
• NewsLeecher ( http://www.newsleecher.com/ )
• NewsBin Pro ( http://www.newsbin.com/ )

You can use all 3 as trial for a limited periode of time, although Forte Agent is crippleware in the free version. All three can import NZB files (they are much like the .torrent file) and places like NewzBin they are all categorized and listed.

Groups
Usenet consists of groups. Each group can has a specific purpose, which normally is written in a FAQ or similar and each group has rules. There are well over 80.000+ groups. By far most groups contain discussions and help on normal subject and thus they contain text messages.

Some groups are binary groups. They contain little text, but on the other hand they contain alot of binaries. These binaries can be anything, a MP3, ZIP, RAR, AVI, JPG, MPG, VOB, ISO etc. These binary groups normally have the word .binaries. as part of the group name. Such a group could be

• alt.binaries.test

The name of the group is alt (alternate) binaries (binary content) test (for test purpose). Most binary groups are named alt.binaries..

Reasons!
The reason for using usenet is that it holds alot of data. A binary group can easily hold a terabyte of data and you can go back 240 days and see what someone put into the group. On EasyNews and UsenetServer you can utilize Global Search and search through an insane amount of data to find something specific for your taste. The usenet servers also have alot of bandwidth. My bet is that you can max out your inet line 100% while downloading from usenet. So!, TOP SPEED.

Contents
The contents of each usenet group is not organized in any specific way, nor can you trust the names of the articles to contain what they claim to contains. There is quite alot of spam on usenet. If you want to utilize usenet best then you should get to know WinRAR (compression / extraction utility) and QuickPar (PAR2 utility for verifying/fixing data). Some of the contents can be downloaded and viewed directly. Some has to be downloaded and extracted/fixed before viewing is possible.

Where
So how to choose among the 80.000+ groups. Basically you can go for the groups with the most interesting names and the biggest post count and it indicates an active group. This will allow you to hit wide and get stuff. But it is not always the case for a person with collectors syndrome. If you suffer from CS then you should look towards:



The lingo for groups are abpn, for the 1st group and so forth. The abecr group is special. The titles are not searchable and nor is the contents. You will need QuickPAR to be able to handle contents in this group and you will need to rename some files before you become a player in abecr. The group community wants to keep each group spam free, and if you spam post then you can be pretty sure that you will get an abuse mail sent to your Usenet provider and they will possible close down your account. Advice: lay low, observe, lurk and download. If you decide to get into the game then start off by saying thanks to the posters. If you decide to become a player and contribute, then be sure what you do, know the tools, know the ways, read the rules.

Getting the daily porn fix - bittorrent

Sometimes you get stuck with nothing to download and yet the need is there. A true addiction or collectors syndrome.

Yet you know that it is out there you just need to get it. And help is on the way. See there are quite a few places which is open to your disposal if you do good.

There's pr0n on the web, usenet, irc, ftp and on torrents. So if you are up for it then you should get yourself a torrents client. And the one which puts the least pressure on your hardware is by far μTorrent. You can find μTorrent on http://www.utorrent.com. There are quite a bunch of others available and one of the popular ones is Azurerus/Vuze which is a opensource project in Java. You can find it here: http://azureus.sourceforge.net/.

So you need to setup your torrent client and the most important thing is A) use encryption and B) make sure that your torrent client can be contacted from the outside. There are excellent articles about how to proper set up your torrent client, but basically they are all about port forwarding and allowing traffic through your firewall.

Next up is a place with torrents. And the ones you should be looking for are one of the 3 greats. And they are (no specific order).

• Cheggit
• Pure T'n'A
• Empornium

There you can get pretty much everything. Google them to find the url's. You will have to create an account, you will have to let others download from you and you will have to let your box be online just a little bit more than normal. But putten that aside then you only have to learn to browse torrents and to stay online.

And just a heads up on A) DO READ THE SITE RULES and B) Do not be a complete leech.

Hard labour of maintaining - auto schedule download and organizing

Well, maintaining can be a bitch sometimes, and most often demand quite some of your time. And you have to look out for pitfalls and every other nasty thing a webmaster can pull on you.

But sometimes it can also be easy. Take a look at the screenshot.

Click to enlarge

What you see is 2 scripted update maintainig scripts, which are hooked into Windows' Task Schedule program (Start -> Accessories -> System Tools). One of the scripts even unzip's the contents after download (if you look closely).

This is just for inspiration and not a production run.

One pickup line for Bang Bros

This summary is not available. Please click here to view the post.

Practical TXT2WJR

Here you have a video tutorial with a practical example of how to use the tool. The video takes advantage of the newest version of TXT2WJR, which is v0.03. You can download it here.

Be advised that it is better if you download the video (XViD format) and view it as the image may seem blurred here on the blog.




Download video file for better view: http://www.mediafire.com/?zjzwsymflwe

Generating ReGet Queues

There is a new version available thanks to Flanker37 input. The new version is v0.02.

The new feature is a switch called /UP which is used like this:

TXT2WJR urls.txt reget.wjr /UP my@emailaddy.com mypassword

The reason is that if you specify a username with a @ (like an email address) in the username field with a regular HTACCESS based url then parsing it will go wrong in most cases. This is useful with sites like KarupsPC and others. If your target site does not require a username with a @ then feel free to stick it into the url list, otherwise put it togetther with the /UP switch.

Download TXT2WJR v0.02

Understanding the URL / links

URL's or links as they are often called can be quite complex. This article will dig into urls and try to explain them. Let's look at 4 url examples below.

1)hxxp://aaabbbcccd.n1.brazzers.com/members/?action=scenes&site_id=5&order_by=date&order_order=desc
2)hxxp://aaabbbcccd.n1.brazzers.com/members/?action=scene&tab=home&scene_id=2776
3)hxxp://members.bangbros.com/milfsoup/updates-1.html
4)hxxp://members.bangbros.com/membercheck?path=ms4276/streaming&fname=ms4276500k.wmv


There are several parts to an url. The easiest is no. 3 which is a plain a simple url. The url consists of 4 parts.

hxxp://www.sitename.com/path_on_server/web_page.html

Part 1
The first part is the protocol, in our case the protocol is http (hxxp). You may have experienced protocols as mms:, https:, ftp:, rtsp: or pnm:. But let us concentrate on http.

Part 2
The second part is the name on the server. A servername consists of a server and a domainname. As example: members.bangbros.com, members is the server, bangbros.com is the domain. Together they define a unique webserver.

Part 3
The second part is the path on the server, which boils down to the directory on the server. Now the directory is a virtual directory. The real physical path could very well be something like /user/bangbros/www/milfsoup/ (as example, if any Unix like server was used) or c:\inetput\www\www.bangbros.com\milfsoup\ (if Windows is used as web server). But you should not care about the real physical path.

Part 4
This is the last part of the simple url. It tells you the name of the html file on the harddisk on the server. A plain .html file is often user created, which basically means that some guy or girl has actually taken time in DreamWeaver or FrontPage (God forbid it) or some other tool and actually written and composed the page. Then he or she has saved the page to the proper directory on the server.

When the humans leave
Well, a plain html url is where it all began. But human edited urls whould require billions of people to constantly sit and re-write webpages everytime you visit the TV Guide or Google or Wired or somewhere else. The contents is simply too alive (dynamic) to have people edit html files and put them on servers. So what to do?
Well instead we get a programmer, preferabilly a webprogrammer to write a program which can generate/create a webpage based on certain details we specify for his program. The details we specify for his program are the ones which makes up for the dynamic contents.
The programmer then writes a program which can read small details and then, based on the details, do some lookup's in database tables and the take that contents from there and put on the webpage with a specific pre-defined style. Ie. CGI which means common gateway interface. A webserver can use CGI to run programs instead of just loading .html pages to the user.


PHP, ASP, PL, JSP ? = and & etc
URL 1), 2) and 4) are all CGI based URL's. And those are the most common way to do things. CGI based urls can use a horde of programs. In the beginning we saw real PC based .EXE files which were run by the webserver. Today it is more common to see .ASP if the webserver is Windows based or PHP if it is Unix based. If you run into JSP then the webserver is hooked up on a Java based engine to execute the programs (Windows or Unix). PL (Perl) is used on both Windows and Unix, but most commonly we see it on the Apache based webserver opposite to ASP which works only on the Windows webserver (IIS Internet Information Server).

CGI based URL's
Have a look url 4). and take notice of the clear red part of the url. Ie. the part which spells membercheck, this is the name of the program written in PHP, ASP, PL, JSP or something else. There are several reasons why the webmasters tend to hide the language used inside the webserver. Because it makes harder for an exploiter to find his way in. Look at the url below and I bet you can answer which language is used and which part is the program

hxxp://nikkisplaymates.com/sd3.php?show=recent_gal_updates

Correct, PHP is the language and sd3 is the name of the program which generates the webpage.

If we look at the url in case 4) we have a part after the domain name. It looks like this:

/membercheck?path=ms4276/streaming&fname=ms4276500k.wmv

We already agreed that membercheck is the program name written in some language. Now this program has to have some details to perform what ever it does to find the proper information for our webpage. Those details are called parameters and in our case we have:

2 parameters:

path
fname

The parameters are seperated by the & sign. Both parameters have values. The value for path is ms4276/streaming and the value for the fname parameter is ms4267500k.wmv.

2 values for the 2 parameters

ms4276/streaming
ms4267500k.wmv

Now, we can only guess that the program (membercheck) uses the path parameter to find the proper directory on the webserver and further more the program finds the video file by the information in the fname parameter. Last there is a fat chance that the program checks that the user is authenticated, as the program name is membercheck.

It is VALUES of the PARAMETERS which you have to figure out, when you make a download list for ReGet Deluxe or OEEE, or whatever program you choose.

Invisible program ?
Well, take a look at url 1) and 2). As we know that the program name is located behind the / for the domain name and before the first ? then we realise that the program name is not shown. It is simply invisible.

This is surely not the case, because there is a program there. Except you do not know the name of it. The reason is that any webserver has some default pages it loads if the user does NOT specify which specific page to load. The most common default pages are:

index.html
index.htm
default.asp
index.jsp
index.php

The web administrator can configure this as he sees fit. So if you find and url as 1) or 2) then the case is just that the webserver is using the configured default file as the program which generates the real webpages.

How to siterip - fooling around with a purpose

Now this is a subject which will be divided into several articles, mainly due to the fact that there is no 100% correct way of doing it and the diversity of sites is huge. So look at this as a learning curve where you pick what ever advice and tips you find useful for your needs.

Define your goal
First of all you need to define your goals for your siter!p (later: SR). What do you want to accomplish ?

• Getting all the pictures from a site ?
• Getting all pictures and videos ?
• Getting everything ?
• Getting just a partial rip ?

If you're not doing an update, as they most often can be done quite easily and often manually then pick your poisen. Personally I aim form completeness.

This means get all media which is of value to you - pictures, videos and other things which does NOT change over time.

Size your opponent
Given you've choosen completeness the you must size your opponent. Is the site a solo model site or is it a larger multimodel site where the contents could spand 10+ dvd's ?. When sizing your opponent there is two things which seems to be of defacto rules.

• Multi model sites = cleaner html/cgi and structure = Autoleechable
• Solo model site = can be dirty = Expect manual tasks

Collect intel
You need to collect intelligence about the site to find out what the best SR way is. Here is a list of things you should make note of while you click around and view the html page sources (because that will be required from you).

1. Is the site using plain HTML and files ?
2. Is the site using CGI/PHP/ASP/JSP/Perl or similar to generate pages ?
3. Is the site using redirection from the pages to the content you want ?
4. Is the site a front cover for a larger contents server on another address ?
5. Does individual picture/video pages link to other pages or is it a "dead end" ?
6. Is the site using Adobe Flash to present contents ?
7. Is the site using JavaScript and pop-up's to present contents ?
8. Is the site providing zip files for contents ?
9. Is contents available in multiple resolutions and qualities/formats ?
10. Is the site based on FORM login or basic HTACCESS ?
11. Does the site require you to re-login after a periode of time ?
12. What seems to be the limitations of simultanious downloads ?
    

1) If you can answer YES to bullet no. 1 then go further and copy/paste random links to
pictures and videos. Put them into NotePad, UltraEdit or EditPad Pro. Copy 2-5 links to pictures from the most recent sets and 2-5 from sets which seems to have been produced at the beginning of the site and 2-5 from the time when the sites was half-old. Go to your editor and look at the url's and detemine: Is there a common structure ? Was the name of the set the same as the directory part of the url ? What tasks is needed to go from set name on the page to files on the site ? The reason is to determine wether or not it is sufficient to just download the media or if you need the HTML also. Downloading the HTML will require you to do more cleaning up after the download, it will also require you to do more structure work when the download has finished.

2) If the answer to this is YES then expect some decent structure. Because the computer generated pages comes partly from a database on the server which holds set names, links and other pieces of information. Do the same as required in 1) but make not of what parts of the url's change from media to media. To get the url's you must be looking at the pictures/videos and rightlickhing choosing Properties and copy/paste the url of the media, possible also copy/paste the url in the address linie of your browser. Copy/paste the url (copy link) which you clicked to arrive at the media. Look at them and determine what steps was needed to go from the frontpage to the media.

3) If redirection is used then primarely make a note (copy/paste) the path of the media and compare them to the link the media came from. Especially make a note of the content is on a deeper level than the page the link came from or a higher page. Deeper and higher is determined by the number of / in the url of the media/page. Higher amount of / means deeper. Also, find out what part of the link url was used to determine the specific redirection - if any special part seems to be the cause.


4) Some sites hosts all their real contents on other servers or on download servers. This has been the case for sites like Only Tease, Brazzers etc. This tends to change as the cost of hosting fluctuates. The download servers seldom require login. Instead they either require a proper cookie or a proper session url. You will have to find out which of the two it is, further you will need to find out how long time the session/cookie lasts before it needs to be refreshed. You will also need to find out if the download server require a proper referer. The easiest way of finding this is to try to use ReGet Deluxe on a link. Look at the HTTP properties in ReGet to look at the cookie/session. And copy the link to a new browser and see if it allows you to download. Remove the refere in ReGet and see if it allows you to download. Most often you just need a proper cookie. Some times you just need the proper name of the encoded url for the contents server. Most often the cookie/session lasts 1-4 hours.


5) If every page with a picture/video has multiple links to the main page, set page and next picture/video etc. then find the urls which links further and copy/paste them so that you can filter them away during your download.


6) If Flash is used to present the contents then you're in for a 100% manual download - more or less. None of the current tools handle Flash sites very good. So get your patience and long hours ready.


7) If JavaScript and pop-up's are used the examine the code and the urls from link to media to see if you can avoid having to execute the JavaScript to download the media. So far I have only seen 1 site that was un-circumvental. If you cannot avoid the JavaScript then you're in for a manual download.


8) If the site provides zip files for the contents then make sure that there ALWAYS is a zip file. Also make sure that it is the proper zip file. Most often the zip files are pre-produced (except for 1 site - to my knowledge) so you will have to probe a few zip files to see if the zip everything or if they are sloppy. Brazzers sometimes zips thumbnails alongside the contents as example. If the zip's match the individuals then go for the zip's otherwise omit the zips and go for the individual downloads.


9) Always aim for the higest quality. So if the pictures come in 2-3 resolution then choose the higest resolution. If the videos comes in same quality and bitrate but multiple formats then pick the one with the lowest MB. That is most often WMV. MPEG/MOV tends to be larger. Download both the AVI and WMV version and watch the video to determine the better of the two.

10) FORM login is the login type you see at Hotmail, YouTube etc. These days the FORM login often comes with some OCR to accomplish the username and password. If FORM is used then be certain that a cookie/sessions is the key to accessing the site from your download tools. If HTACCESS is used then you should be able to go directly to the members page with an url like http://username:password@www.sitename.com/members/ or what ever the URL is. Now make a note that Internet Explorer does not allow that method for logging in (unless you tweaked it), so try it with FireFox or your download program.


11) Re-logins is used with FORM sites and it refreshes/renews the cookie/session. So if you queue up a huge download which will take several hours then find the time limits and do not queue up more than you can manage to download with the time limitations. Why?, because otherwise the site admin will see you hammering the site all night and the chances for a ban is severly increased which means you will be cut off.


12) Some sites has limitations to the number of simultanious downloads. For Ann Angel as example there is a limit of 2. This means that you should not exceed this number or you will be cut off by the site. Choose a number which fits a hardcore download guy using FireFox or some other tool. That means go no higher than 8 simultanious downloads. Go down to 2-3 when you start your download manager and watch it progress to see if it could be increased. Do NOT roll over a site with 32 simultanious like a steam roller. The site admin or system will notice and you will get a ban. Nobody likes their site to be 'raped'.

Probing
Now it is time to test wether your intel is sufficient. Unless it is a video site then try with the intel you have on the pictures first. This means that you will need to make a new project in Offline Explorer Enterprise Edition and put your urls, cookie=, referer=, username and passwords into OEEE and see wether it will download outside your Internet browser. Start with 1 set at the beginnig. Set your URL filters as tight as possible according to your intel, both for server/domain, path and filename (to avoid downloading thumbnails). Limit your file types to pictures only. Hit Download and switch to the queue tab to see if things are resolved properly and that you infact do download pictures. If NOT, then relax your restrictions until you get an acceptable result. Next try with 2 sets and see if it works. If it does then queue up the lot and let it run. Still taking into condiration any time limitits and simultanious limitations. Use F9 to pause you download when needed to watch the url resolving. And use # as a comment to play around with your urls.

cookie=user=myself&pass=secrets&sessionid=abch48d1gaup39Hl
referer=http://www.site.com/members/image?setname1
http://www.site.com/members/image?setname1

#http://www.site.com/members/image?setname2
#http://www.site.com/members/image?setname3
#http://www.site.com/members/image?setname4

It is a good idea to configure OEEE to either Download All Files or Do not download existing files because while you are probing you may get some "trash" along the download. Try to limit the amount of excessive stuff you download. Clean you download folder after probing has ended and before you begin the real deal.

Finale
To get good download lists then you may need to save some html pages and possibly use some regex or some column copy/paste. Macro downloading may also be of benefit to you. See articles ablut MACRO download and regex.

ReGet Deluxe can import a text file with URL's so make one for the videos and import it. Mark all the url's in ReGet and set the properties for the download path, cookies, possible username/passwords etc, before you begin. Set max simultanious downloads in ReGet to 8 (on huge videos).




The article serie will continue with more specific information and tool practice alongside regex and download list creation. Until then - take care.

HAM downloads

HAM downloading is a lazy way to obtain your download links, but it is rather effective. The basis of a HAM download is to grab the urls the site produces while in pause. Let's take Ann Angel as an example. Visit her site and login, go to the pictures page (if that is what you are on the look for) and rightclick anywhere and choose "Download with ReGet". I assume you have installed ReGet Deluxe and configured it to expert mode.

Once ReGet pops up you can switch tab and go to the HTTP properties tab. See it below.

Click to enlarge

Since the site is a FORM site, then we cannot use direct username and password in OEEE. We have to obtain the cookie which hold the information needed to login to the site. Mark the line and copy the cookie to your clipboard.

Now start OEEE and create a new project. Paste the cookie from your clipboard into the project. Make it look something like below.

Click to enlarge

Now change the maximum simultanious downloads in OEEE into 1. Do that by changing the Options in OEEE, see below.

Click to enlarge

With that done you hit the Suspend button so that all downloads are set to pause. After that click the Queue tab in OEEE. Now you hit Download button in OEEE (everything is in pause).

Click to enlarge

Here comes the nice thing hit F9 twice. This will turn off Suspend for a brief second (allowing the link to be downloade) and then put OEEE into Suspend mode right after that. This will make OEEE resolve the 1st download into all subsequent links in level 1. Now all you have to do is to mark all the lines you wish to download anc choose Copy URL.

Click to enlarge

You can now do paste the list back into OEEE (hit the Stop button first) or paste the url's into Notepad for some GREP processing or whatever you wish for, example deduct url filters.

MACRO downloading

Using macros when downloading is a powerful thing. Offline Explorer Enterprise Edition can help you quite a deal. If you have 10 pictures on a http server like this:

http://www.server.com/pictures/image01.jpg
http://www.server.com/pictures/image02.jpg
http://www.server.com/pictures/image03.jpg
http://www.server.com/pictures/image04.jpg
http://www.server.com/pictures/image05.jpg
http://www.server.com/pictures/image06.jpg
http://www.server.com/pictures/image07.jpg
http://www.server.com/pictures/image08.jpg
http://www.server.com/pictures/image09.jpg
http://www.server.com/pictures/image10.jpg

Then you can download all 10 directly by using a numbering macro in OEEE. Write this in your download box.

http://www.server.com/pictures/image{:01..10}.jpg

That will do the job and generate 10 urls when you hit the run/play button in OEEE. Now you can spiff up the numbering macro a bit by using a enumerater step. Let's say that the pictures are numbered 01,03,05,07,09,11 instead of a direct squence. Then you can create your macro like this:

http://www.server.com/pictures/image{:01..11|2}.jpg

That will generate the urls:

http://www.server.com/pictures/image01.jpg
http://www.server.com/pictures/image03.jpg
http://www.server.com/pictures/image05.jpg
http://www.server.com/pictures/image07.jpg
http://www.server.com/pictures/image09.jpg
http://www.server.com/pictures/image11.jpg

Now this is all handy. The big advantage is being able to use multiple macros in the same command. Very often you need set names to your download list. If the webmaster of the site of your choice is drug free (which is almost none) then the sets will be numbered 001, 002 and so forth. But this is almost NEVER the case, although Ann-Angel's webmaster does that - thank you for that.

The easiest way of obtaining set names is to visit whatever gallery page that holds the episodes/sets you wish to download. Let's try with an example. Reality Kings has a site called Mikes In Brazil. Visit the url hxxp://www.mikesinbrazil.com and save the intro page to your hdd. It should be the page which is is the commercial page which lists a bunch of episodes.

Each of the episodes has a episode name. This is hidden a little. But if you let your mouse hover over the Watch Trailer button then you can spot the episode name as the first part of the wmv file. Example sol.wmv. The episode name is: sol.

With your saved file run this command on the command line:

grep -Po "(?<=trailer=)[a-z0-9]*" mikesinbrazil.html >set_names_mib.txt

If you do not redirect the output to the text file set_names_mib.txt then your result could look very much like the picture below.

Click to enlarge

With a file of episode names we can enhance our OEEE macro download a bit. We can now construct it to use the set names and a numbering macro.

http://www.server.com/{:file=c:\set_names_mib.txt}/pictures/image{:001..321}.jpg

The macro download will pr0duce 321*8 = 2568 direct urls for downloading images. The urls will look something like:

http://www.server.com/sol/pictures/image001.jpg
http://www.server.com/sol/pictures/image002.jpg
...
http://www.server.com/sol/pictures/image321.jpg
http://www.server.com/natashia2/pictures/image001.jpg
http://www.server.com/natashia2/pictures/image002.jpg
... and so forth

I hope this will help you in your downloading.

Downloading tools

There's as many tools for downloading as there are religions. But to cut it down to a few good ones I suggest you really take a close look at:

Tool: ReGet Deluxe
Author: ReGet
URL: http://www.reget.com/

Tool: Offline Explorer Enterprise Edition
Author: Meta Products
URL: http://www.metaproducts.com/

Now, there is a reason for two tools, aswell as there is a reason for not just doing 'Save As' or 'Save Target As' with your browser.

First off, you browser does a poor job saving correctly, especially with huge files. Firefox does a better job than Internet Explorer, but it is still not good enough. And when it comes to streaming contents. Then simply forget your browser.


ReGet Deluxe is a versatile tool, much like GetRight and others. There are 3 reasons to choose ReGet. A) it downloads correctly, B) it can use multiple connections to the same huge file and C) it handles cookies and referer's correctly.

You have to set ReGet Deluxe to Expert mode, this is done during installation. In Expert Mode you can see the proper referer and the cookie involved. These can be copied to other tools, Offline Explorer as example to handle multiple file downloads.

ReGet Deluxe is good for downloading huge files, especially videos og huge zip files. In the newer versions it integrates nicely with both Internet Explorer and Firefox, so you can rightclick the object of your desire and choose: "Download with ReGet" or you can choose: "Download the page with ReGet".

Another nice thing with ReGet Deluxe is that ReGet Deluxe is able to import text files with URL's. Use tools like GREP and SED to make a list of URL's in a text file. And make them have the format like:

hxxp://u:p@members.bangbrosnetwork.com/membercheck?path=ap4044/streaming&fname=ap4044.wmv
hxxp://u:p@members.bangbrosnetwork.com/membercheck?path=ap2340/streaming&fname=ap2340.wmv
hxxp://u:p@members.bangbrosnetwork.com/membercheck?path=ap4029/streaming&fname=ap4029.wmv
hxxp://u:p@members.bangbrosnetwork.com/membercheck?path=ap4000/streaming&fname=ap4000.wmv
... etc

And choose to import the file into ReGet Deluxe. That will give you a nice list of alot of all the videos you want to download.



Offline Explorer Enterprise Edition is a cool tool for downloading alot of files. OEEE has some features which are essential if you want to make good downloads.

First off you can filter your downloads on a file basis, saying that you either only want to include certain files or you want exclude certain files. A good example is to exclude all files named tn*.jpg, as they most often are thumbnail files.

Furthermore OEEE lets you include/exclude files based on directory basis. So you can put in names like thumbs in the list of folders to exclude. You can exclude/include multiple file types and directories. And you can use wildcards when doing so. You can even copy/paste a list of filetypes/directories into OEEE's filter settings.

Always configure your downloads to only download from the starting server or mayby, starting domain, if nessary. Or put the two servers in question into the server filter list. This is important with OnlyTease and RealityKings among others.

You can put multiple url's into OEEE and you can add the proper referer and cookie to the urls. So that the sites behave as they should. This is done by writing:

referer=hxxp://ann-angel.com/Membersite/photos/497/pages/index.html
cookie=sbuser=someuser; sbpasswd=somepass; savepassword=1; sbsession=sbx9dyh1wda&sbuser=someuser
hxxp://ann-angel.com/Membersite/photos/497/pages/index.html

as example. Get the cookie and referer from ReGet Deluxe. Put /497/ into included folders and put th*.jpg into excluded files.

NOTE: All usernames and passwords are invalid. hxxp should be changed into the apropiate.