org.apache.commons.pipeline.stage
Class HttpFileDownloadStage

java.lang.Object
  org.apache.commons.pipeline.stage.BaseStage
      org.apache.commons.pipeline.stage.HttpFileDownloadStage

All Implemented Interfaces:

Stage

public class HttpFileDownloadStage
extends BaseStage

This Stage provides the functionality needed to retrieve data from an HTTP URL. Multipart responses are not yet supported.

Field Summary

Fields inherited from class org.apache.commons.pipeline.stage.BaseStage

context

Constructor Summary

HttpFileDownloadStage()

HttpFileDownloadStage(java.lang.String workDir)
Creates a new HttpFileDownloadStage which will download files to the specified work directory.

Method Summary

java.lang.String	getWorkDir() Returns the name of the file download directory.
java.net.URL	handleRedirects(java.net.URL url) Follows redirects from the specified URL and recursively returns the destination URL.
void	process(java.lang.Object obj) Removes a java.net.URL (an HTTP URL) or string representing a URL from the input queue, and then retrieves the data at that URL and stores it in a temporary file.
void	setWorkDir(java.lang.String workDir) Sets the working directory for the file download.

Methods inherited from class org.apache.commons.pipeline.stage.BaseStage

emit, emit, init, postprocess, preprocess, release

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HttpFileDownloadStage

public HttpFileDownloadStage()

HttpFileDownloadStage

public HttpFileDownloadStage(java.lang.String workDir)

Creates a new HttpFileDownloadStage which will download files to the specified work directory.

Parameters:

workDir - the path to which files will be downloaded.

Method Detail

process

public void process(java.lang.Object obj)
             throws StageException

Removes a java.net.URL (an HTTP URL) or string representing a URL from the input queue, and then retrieves the data at that URL and stores it in a temporary file. The file is stored in the directory specified by setWorkDir(), or to the system default temporary directory if no work directory is set.

Specified by:

process in interface Stage

Overrides:

process in class BaseStage

Parameters:

obj - The URL from which to download data.

Throws:

java.lang.IllegalArgumentException - if the parameter obj is not a string or an instance of URL.

StageException - if there is an error retrieving data from the URL specified.

setWorkDir

public void setWorkDir(java.lang.String workDir)

Sets the working directory for the file download. If the directory does not already exist, it will be created during the preprocess() step. If you do not set this directory, the work directory will be the default temporary directory for your machine type.

getWorkDir

public java.lang.String getWorkDir()

Returns the name of the file download directory.

handleRedirects

public java.net.URL handleRedirects(java.net.URL url)
                             throws java.io.IOException,
                                    java.net.MalformedURLException

Follows redirects from the specified URL and recursively returns the destination URL. This method does not check for circular redirects, so it is possible that a malicious site could force this method into infinite recursion. TODO: Add a max_hops parameterized version

Throws:

java.io.IOException

java.net.MalformedURLException