HOWTO: Download all files from a remote FTP directory and save them to a local disk

UPDATE: This blogpost from year 2009 uses old API (GetFiles method), in the new versions of Rebex components there are the newer multiple file operations Upload and Download. See the multiple-file operations feature page for code samples in C# and VB.NET.

---

Quick & dirty (yet a bit naive) directory download code

To download all files in a specified remote folder seems to be a trivial task at first. Calling Ftp.GetList(), iterating through the returned collection and caling Ftp.GetFile() on each item looks like a no brainer. It would work in simple cases. However, there are some caveats in this approach. Consider the following situations:

• Remote directory has both files and folder with other files. Should we download the content of subdirectories too?
• What if some of the remote files already exist on the local computer? Should they be overwritten? Skipped? Should we overwrite older files only?
• What if the local file is not writable? Should the whole transfer fail? Should we skip the file and continue to the next?
• How to handle files on a remote disk which are unreadable because we don’t have sufficient access rights?
• How are the symlinks, hard links and junction points handled? Links can easily be used to create an infinite recursive directory tree structure. Consider folder A with subfolder B which in fact is not the real folder but the *nix hard link pointing back to folder A. The naive approach will end in an application which never ends (at least if nobody manage to pull the plug).

Download the directory – resolve the conflicts automatically

public static void DownloadFolderSimple()
{
    using (Ftp client = new Ftp())
    {
        // connect and login to the FTP site
        client.Connect("mirror.aarnet.edu.au");
        client.Login("anonymous", "my@password");

        // download all files
        client.GetFiles(
            "/pub/fedora/linux/development/i386/os/EFI/*",
            "c:\\temp\\download",
            FtpBatchTransferOptions.Recursive,
            FtpActionOnExistingFiles.OverwriteAll
            );

            client.Disconnect();
    }
}

The above code solves all the issues mentioned above. The programmer decides how to resolve most conflicts. In this case existing files are overwritten, links are followed and downloaded correctly, infinite link loops are detected and handled by the component code by throwing an exception. Such code is best for unattended processing.

What if the user needs to choose which local files should be overwritten and which not? What if we want to skip files with certain type of problem? Let’s check another scenario and give this power to the hands of the user.

Download the directory – let the user resolve the conflicts

public static void DownloadFolderWithUserInteraction()
{
    using (Ftp client = new Ftp())
    {
        // connect and login to the FTP site
        client.Connect("mirror.aarnet.edu.au");
        client.Login("anonymous", "my@password");

        // subscribe to the event which occurs when transfer conflict
        // or problem occures
        client.BatchTransferProblemDetected += client_BatchTransferProblemDetected;

        // download all files
        client.GetFiles(
            "/pub/fedora/linux/development/i386/os/EFI/*",
            "c:\\temp\\download",
            FtpBatchTransferOptions.Recursive
        );

        client.Disconnect();
    }
}

static void client_BatchTransferProblemDetected(object sender, FtpBatchTransferProblemDetectedEventArgs e)
{
    // Problem type can be found in e.ProblemType
    //
    // Actions usable for solving the specific problem
    // can be retrieved by checking flags in e.PossibleActions or
    // by calling e.IsActionPossible method.
    //
    // e.Action = MyTransferrProblemDetecttedDialog.ShowModal(e);
    //
    // See http://www.rebex.net/ftp.net/sample-batch-transfer.aspx
    // for sample implementation.
}

Each time the component reaches the point in which a decision has to be made the user is asked. A dialog pops up and the user is given an option to resolve the issue for the specific file or directory. He is also able to resolve all similar cases with one choice - e. g. he is given an option to either  ‘Overwrite a specific file’ or ‘Overwrite all files if older’ as shown on the screeshot.

To see this approach implemented in a fully functional application download the component and check the FTP Batch Transfer Sample. Both C # and VB.NET code available.