streaming-commons-0.1.14.2: Common lower-level functions needed by various streaming data libraries

Safe Haskell | None
Language | Haskell98

Data.Streaming.Zlib

Contents

- Inflate
- Deflate
- Data types

Description

This is a middle-level wrapper around the zlib C API. It allows you to work fully with bytestrings and not touch the FFI at all, but is still low-level enough to allow you to implement high-level abstractions such as enumerators. Significantly, it does not use lazy IO.

You'll probably need to reference the docs a bit to understand the WindowBits parameters below, but a basic rule of thumb is 15 is for zlib compression, and 31 for gzip compression.

A simple streaming compressor in pseudo-code would look like:

    def <- initDeflate ...
    popper <- feedDeflate def rawContent
    pullPopper popper
    ...
    finishDeflate def sendCompressedData

You can see a more complete example is available in the included file-test.hs.

Synopsis

- data Inflate
- initInflate :: WindowBits -> IO Inflate
- initInflateWithDictionary :: WindowBits -> ByteString -> IO Inflate
- feedInflate :: Inflate -> ByteString -> IO Popper
- finishInflate :: Inflate -> IO ByteString
- flushInflate :: Inflate -> IO ByteString
- getUnusedInflate :: Inflate -> IO ByteString
- data Deflate
- initDeflate :: Int -> WindowBits -> IO Deflate
- initDeflateWithDictionary :: Int -> ByteString -> WindowBits -> IO Deflate
- feedDeflate :: Deflate -> ByteString -> IO Popper
- finishDeflate :: Deflate -> Popper
- flushDeflate :: Deflate -> Popper
- fullFlushDeflate :: Deflate -> Popper
- data WindowBits :: * = WindowBits Int
- defaultWindowBits :: WindowBits
- data ZlibException = ZlibException Int
- type Popper = IO PopperRes
- data PopperRes
  - = PRDone
  - | PRNext !ByteString
  - | PRError !ZlibException

# Inflate

data Inflate

The state of an inflation (eg, decompression) process. All allocated memory is automatically reclaimed by the garbage collector. Also can contain the inflation dictionary that is used for decompression.

initInflate :: WindowBits -> IO Inflate

Initialize an inflation process with the given `WindowBits`. You will need to call `feedInflate` to feed compressed data to this and `finishInflate` to extract the final chunk of decompressed data.

initInflateWithDictionary :: WindowBits -> ByteString -> IO Inflate

Initialize an inflation process with the given `WindowBits`. Unlike initInflate a dictionary for inflation is set which must match the one set during compression.

feedInflate :: Inflate -> ByteString -> IO Popper

Feed the given `ByteString` to the inflater. Return a `Popper`, an IO action that returns the decompressed data a chunk at a time. The `Popper` must be called to exhaustion before using the `Inflate` object again.

Note that this function automatically buffers the output to `defaultChunkSize`, and therefore you won't get any data from the popper until that much decompressed data is available. After you have fed all of the compressed data to this function, you can extract your final chunk of decompressed data using `finishInflate`.

finishInflate :: Inflate -> IO ByteString

As explained in `feedInflate`, inflation buffers your decompressed data. After you call `feedInflate` with your last chunk of compressed data, you will likely have some data still sitting in the buffer. This function will return it to you.

flushInflate :: Inflate -> IO ByteString

Flush the inflation buffer. Useful for interactive application.

This is actually a synonym for `finishInflate`. It is provided for its more semantic name.

Since 0.0.3

getUnusedInflate :: Inflate -> IO ByteString

Retrieve any data remaining after inflating. For more information on motivation, see:

https://github.com/fpco/streaming-commons/issues/20

Since 0.1.11

# Deflate

data Deflate

The state of a deflation (eg, compression) process. All allocated memory is automatically reclaimed by the garbage collector.

initDeflate

Arguments

:: Int | Compression level
-> WindowBits |
-> IO Deflate |

Initialize a deflation process with the given compression level and `WindowBits`. You will need to call `feedDeflate` to feed uncompressed data to this and `finishDeflate` to extract the final chunks of compressed data.

initDeflateWithDictionary

Arguments

:: Int | Compression level
-> ByteString | Deflate dictionary
-> WindowBits |
-> IO Deflate |

Initialize an deflation process with the given compression level and `WindowBits`. Unlike initDeflate a dictionary for deflation is set.

feedDeflate :: Deflate -> ByteString -> IO Popper

Feed the given `ByteString` to the deflater. Return a `Popper`, an IO action that returns the compressed data a chunk at a time. The `Popper` must be called to exhaustion before using the `Deflate` object again.

Note that this function automatically buffers the output to `defaultChunkSize`, and therefore you won't get any data from the popper until that much compressed data is available. After you have fed all of the decompressed data to this function, you can extract your final chunks of compressed data using `finishDeflate`.

finishDeflate :: Deflate -> Popper

As explained in `feedDeflate`, deflation buffers your compressed data. After you call `feedDeflate` with your last chunk of uncompressed data, use this to flush the rest of the data and signal end of input.

flushDeflate :: Deflate -> Popper

Flush the deflation buffer. Useful for interactive application. Internally this passes Z_SYNC_FLUSH to the zlib library.

Unlike `finishDeflate`, `flushDeflate` does not signal end of input, meaning you can feed more uncompressed data afterward.

Since 0.0.3

fullFlushDeflate :: Deflate -> Popper

Full flush the deflation buffer. Useful for interactive applications where previously streamed data may not be available. Using `fullFlushDeflate` too often can seriously degrade compression. Internally this passes Z_FULL_FLUSH to the zlib library.

Like `flushDeflate`, `fullFlushDeflate` does not signal end of input, meaning you can feed more uncompressed data afterward.

Since 0.1.5

# Data types

data WindowBits :: *

This specifies the size of the compression window. Larger values of this parameter result in better compression at the expense of higher memory usage.

The compression window size is the value of the the window bits raised to the power 2. The window bits must be in the range `8..15` which corresponds to compression window sizes of 256b to 32Kb. The default is 15 which is also the maximum size.

The total amount of memory used depends on the window bits and the `MemoryLevel`. See the `MemoryLevel` for the details.

Constructors

WindowBits Int |

Instances

Eq WindowBits |
Ord WindowBits |
Show WindowBits |
Generic WindowBits |
type Rep WindowBits = D1 D1WindowBits ((:+:) (C1 C1_0WindowBits (S1 NoSelector (Rec0 Int))) (C1 C1_1WindowBits U1)) |

defaultWindowBits :: WindowBits

The default `WindowBits` is 15 which is also the maximum size.

data ZlibException

Exception that can be thrown from the FFI code. The parameter is the numerical error code from the zlib library. Quoting the zlib.h file directly:

- #define Z_OK 0
- #define Z_STREAM_END 1
- #define Z_NEED_DICT 2
- #define Z_ERRNO (-1)
- #define Z_STREAM_ERROR (-2)
- #define Z_DATA_ERROR (-3)
- #define Z_MEM_ERROR (-4)
- #define Z_BUF_ERROR (-5)
- #define Z_VERSION_ERROR (-6)

Constructors

ZlibException Int |

Instances

Show ZlibException |
Exception ZlibException |

type Popper = IO PopperRes

An IO action that returns the next chunk of data, returning `Nothing` when there is no more data to be popped.

data PopperRes

Constructors

PRDone |
PRNext !ByteString |
PRError !ZlibException |

Instances

Show PopperRes |