java.lang.Object
org.apache.commons.imaging.formats.tiff.datareaders.ImageDataReader
Direct Known Subclasses:
DataReaderStrips, DataReaderTiled

public abstract class ImageDataReader extends Object
Defines the base class for the TIFF file reader classes. The TIFF format defines two broad organizations for image pixel storage: strips and tiles. This class defines common elements for both representations.

The TIFF Floating-Point Formats

In addition to providing images, TIFF files can supply data in the form of numerical values. As of March 2020 the Commons Imaging library was extended to support some floating-point data formats.

Unfortunately, the TIFF floating-point format allows for a lot of different variations. At this time, only the most widely used of these are supported. When this code was written, only a small set of test data products were available. Thus it is likely that developers will wish to extend the range of floating-point data that can be processed as additional test data become available. When implementing extensions to this logic, developers are reminded that image processing requires the handling of literally millions of pixels, so attention to performance is essential to a successful implementation (please see the notes in DataReaderStrips for more information).

The TIFF floating-point specification is poorly documented. So these notes are included to provide clarification on at least some aspects of the format. Some documentation and C-code examples are available in "TIFF Technical Note 3, April 8, 2005)".

The Predictor==3 Case

TIFF specifies an extension for a predictor that is intended to improve data compression ratios for floating-point values. This predictor is specified using the TIFF predictor TAG with a value of 3 (see TIFF Technical Note 3). Consider a 4-byte floating point value given in IEEE-754 format. Let f3 be the high-order byte, with f2 the next highest, followed by f1, and f0 for the low-order byte. This designation should not be confused with the in-memory layout of the bytes (little-endian versus big-endian), but rather their numerical values. The sign bit and upper 7 bits of the exponent are given in the high-order byte, followed by the one remaining exponent bit and the mantissa in the lower-order bytes.

In many real-valued raster data sets, the sign and magnitude (exponent) of the values change slowly. But the bits in the mantissa vary rapidly in a semi-random manner. The information entropy in the mantissa tends to increase in the lowest ordered bytes. Thus, the high-order bytes have more redundancy than the low-order bytes and can compress more efficiently. To exploit this, the TIFF format splits the bytes into groups based on their order-of-magnitude. This splitting process takes place on a ROW-BY-ROW basis (note the emphasis, this point is not clearly documented in the spec). For example, for a row of length 3 pixels -- A, B, and C -- the data for two rows would be given as shown below (again, ignoring endian issues):

   Original:
      A3 A2 A1 A0   B3 B2 B1 B0   C3 C2 C1 C0
      D3 D3 D1 D0   E3 E2 E2 E0   F3 F2 F1 F0

   Bytes split into groups by order-of-magnitude:
      A3 B3 C3   A2 B2 C2   A1 B1 C1   A0 B0 C0
      D3 E3 F3   D2 E2 F2   D1 E1 F1   D0 E0 F0
 
To further improve the compression, the predictor takes the difference of each subsequent bytes. Again, the differences (deltas) are computed on a row-byte-row basis. For the most part, the differences combine bytes associated with the same order-of-magnitude, though there is a special transition at the end of each order-of-magnitude set (shown in parentheses):
      A3, B3-A3, C3-B3, (A2-C3), B2-A2, C2-B2, (A1-C2), etc.
      D3, E3-D3, F3-D3, (D2-F3), E3-D2, etc.
 
Once the predictor transform is complete, the data is stored using conventional data compression techniques such as Deflate or LZW. In practice, floating point data does not compress especially well, but using the above technique, the TIFF process typically reduces the overall storage size by 20 to 30 percent (depending on the data). The TIFF Technical Note 3 specifies 3 data size formats for storing floating point values:
     32 bits    IEEE-754 single-precision standard
     16 bits    IEEE-754 half-precision standard
     24 bits    A non-standard representation
 
At this time, we have not obtained data samples for the smaller representations used in combination with a predictor.

Interleaved formats

TIFF Technical Note 3 also provides example code for cases where each pixel (or raster cell) in the image is associated with more than one floating-point samples. Data in this format might be used for real-valued vector data, complex-valued pairs, or other numerical applications).

At this time, we have encountered only a limited selection of the possible configurations for multi-variable data. The code below only supports those configurations for which we had actual images that could be used to verify our implementation. The implementation supports the following formats:

  • 32-bit floating-point data
  • Uncompressed, Deflate, or LZW compression
  • Optional horizontal predictors used with compression
  • PlanarConfiguration interleaved (CHUNKY) or non-interleaved (PLANAR)

Note that integer formats are not supported at this time.

Often, the TIFF files store multi-variable data in so that samples are interleaved. For example, a configuration that gave two samples per pixel (or cell) would give the two values for the first pixel in order followed by the two values for the second pixel, etc. If a differencing approach were used for data compression, the byte-stream would begin with the high-order byte for each of the two samples for the first pixel, followed by the high-order byte for each of the next two samples, and so forth for the remainder of the row of pixels. It would then follow with the second-highest-order bytes for the first two samples, etc.

This implementation also supports the non-interleaved (PLANAR) configuration. One consideration in implementing this feature was that TIFF Technical Note 3 does not address the case where a TIFF image uses the alternate planar configuration. For conventional images, the TIFF specification (Revision 6.0) recommends that the planar configuration should be avoided (see pg. 38). But for numerical data products, the planar configuration may yield better data compression in the case where different sample sets have different statistical properties. Because separated groups often have more uniformity and predictability than interleaved data sets, they sometimes lead to a small improvement in storage-size reduction when data compression is used.

  • Field Details

  • Constructor Details

  • Method Details

    • readImageData

      public abstract ImageBuilder readImageData(Rectangle subImageSpecification, boolean hasAlpha, boolean isAlphaPremultiplied) throws ImageReadException, IOException
      Read the image data from the IFD associated with this instance of ImageDataReader using the optional sub-image specification if desired.
      Parameters:
      subImageSpecification - a rectangle describing a sub-region of the image for reading, or a null if the whole image is to be read.
      hasAlpha - indicates that the image has an alpha (transparency) channel (RGB color model only).
      isAlphaPremultiplied - indicates that the image uses the associated alpha channel format (pre-multiplied alpha).
      Returns:
      a valid instance containing the pixel data from the image.
      Throws:
      ImageReadException - in the event of a data format error or other TIFF-specific failure.
      IOException - in the event of an unrecoverable I/O error.
    • isHomogenous

      protected boolean isHomogenous(int size)
      Checks if all the bits per sample entries are the same size
      Parameters:
      size - the size to check
      Returns:
      true if all the bits per sample entries are the same
    • resetPredictor

      protected void resetPredictor()
    • applyPredictor

      protected int[] applyPredictor(int[] samples)
    • applyPredictorToBlock

      protected void applyPredictorToBlock(int width, int height, int nSamplesPerPixel, byte[] p)
    • decompress

      protected byte[] decompress(byte[] compressedInput, int compression, int expectedSize, int tileWidth, int tileHeight) throws ImageReadException, IOException
      Throws:
      ImageReadException
      IOException
    • unpackFloatingPointSamples

      protected int[] unpackFloatingPointSamples(int width, int height, int scanSize, byte[] bytes, int bitsPerPixel, ByteOrder byteOrder) throws ImageReadException
      Given a source file that specifies the floating-point data format, unpack the raw bytes obtained from the source file and organize them into an array of integers containing the bit-equivalent of IEEE-754 32-bit floats. Source files containing 64 bit doubles are downcast to floats.

      This method supports either the tile format or the strip format of TIFF source files. The scan size indicates the number of columns to be extracted. For strips, the width and the scan size are always the full width of the image. For tiles, the scan size is the full width of the tile, but the width may be smaller in the cases where the tiles do not evenly divide the width (for example, a 256 pixel wide tile in a 257 pixel wide image would result in two columns of tiles, the second column having only one column of pixels that were worth extracting.

      Parameters:
      width - the width of the data block to be extracted
      height - the height of the data block to be extracted
      scanSize - the number of pixels in a single row of the block
      bytes - the raw bytes
      bitsPerPixel - the number of bits per sample, 32 or 64.
      byteOrder - the byte order for the source data
      Returns:
      a valid array of integers in row major order, dimensions scan-size wide and height height.
      Throws:
      ImageReadException - in the event of an invalid format.
    • unpackIntSamples

      protected int[] unpackIntSamples(int width, int height, int scanSize, byte[] bytes, int predictor, int bitsPerSample, ByteOrder byteOrder)
      Given a source file that specifies numerical data as short integers, unpack the raw bytes obtained from the source file and organize them into an array of integers.

      This method supports either the tile format or the strip format of TIFF source files. The scan size indicates the number of columns to be extracted. For strips, the width and the scan size are always the full width of the image. For tiles, the scan size is the full width of the tile, but the "width" parameter may be smaller in the cases where the tiles do not evenly divide the width (for example, a 256 pixel wide tile in a 257 pixel wide image would result in two columns of tiles, the second column having only one column of pixels that were worth extracting.

      Parameters:
      width - the width of the data block to be extracted
      height - the height of the data block to be extracted
      scanSize - the number of pixels in a single row of the block
      bytes - the raw bytes
      predictor - the predictor specified by the source, only predictor 3 is supported.
      bitsPerSample - the number of bits per sample, 32 or 64.
      byteOrder - the byte order for the source data
      Returns:
      a valid array of integers in row major order, dimensions scan-size wide and height height.
    • readRasterData

      Defines a method for accessing the floating-point raster data in a TIFF image. These implementations of this method in DataReaderStrips and DataReaderTiled assume that this instance is of a compatible data type (floating-point) and that all access checks have already been performed.
      Parameters:
      subImage - if non-null, instructs the access method to retrieve only a sub-section of the image data.
      Returns:
      a valid instance
      Throws:
      ImageReadException - in the event of an incompatible data form.
      IOException - in the event of I/O error.