mmap-0.5.9: Memory mapped files for POSIX and Windows

Copyright | (c) Gracjan Polak 2009
License | BSD-style
Stability | experimental
Portability | portable
Safe Haskell | None
Language | Haskell98

System.IO.MMap

Contents

- Mapping mode
- Memory mapped files strict interface
- Memory mapped files lazy interface

Description

This library provides a wrapper to mmap(2) or MapViewOfFile, allowing files or devices to be lazily loaded into memory as strict or lazy ByteStrings, ForeignPtrs or plain Ptrs, using the virtual memory subsystem to do on-demand loading. Modifications are also supported.

Synopsis

- data Mode
  - = ReadOnly
  - | ReadWrite
  - | WriteCopy
  - | ReadWriteEx
- mmapFilePtr :: FilePath -> Mode -> Maybe (Int64, Int) -> IO (Ptr a, Int, Int, Int)
- mmapWithFilePtr :: FilePath -> Mode -> Maybe (Int64, Int) -> ((Ptr (), Int) -> IO a) -> IO a
- mmapFileForeignPtr :: FilePath -> Mode -> Maybe (Int64, Int) -> IO (ForeignPtr a, Int, Int)
- mmapFileByteString :: FilePath -> Maybe (Int64, Int) -> IO ByteString
- munmapFilePtr :: Ptr a -> Int -> IO ()
- mmapFileForeignPtrLazy :: FilePath -> Mode -> Maybe (Int64, Int64) -> IO [(ForeignPtr a, Int, Int)]
- mmapFileByteStringLazy :: FilePath -> Maybe (Int64, Int64) -> IO ByteString

# Documentation

This module is an interface to `mmap(2)` system call under POSIX (Unix, Linux, Mac OS X) and `CreateFileMapping`, `MapViewOfFile` under Windows.

We can consider mmap as lazy IO pushed into the virtual memory subsystem.

It is only safe to mmap a file if you know you are the sole user. Otherwise referential transparency may be or may be not compromised. Sadly semantics differ much between operating systems.

In case of IO errors all function use `throwErrno` or `throwErrnoPath`.

In case of `ForeignPtr` or `ByteString` functions the storage manager is used to free the mapped memory. When the garbage collector notices there are no further references to the mapped memory, a call to `munmap` is made. It is not necessary to do this yourself. In tight memory situations it may be profitable to use `performGC` or `finalizeForeignPtr` to force an unmap action. You can also use `mmapWithFilePtr` that uses scope based resource allocation.

To free resources returned as Ptr use `munmapFilePtr`.

For modes `ReadOnly`, `ReadWrite` and `WriteCopy` file must exist before mapping it into memory. It also needs to have correct permissions for reading and/or writing (depending on mode). In `ReadWriteEx` the file will be created with default permissions if it does not exist.

If mode is `ReadWrite`, `ReadWriteEx` or `WriteCopy` the returned memory region may be written to with `poke` and friends. In `WriteCopy` mode changes will not be written to disk. It is an error to modify mapped memory in `ReadOnly` mode. If is undefined if and how changes from external changes affect your mmapped regions, they may reflect in your memory or may not and this note applies equally to all modes.

Range specified may be `Nothing`, in this case whole file will be mapped. Otherwise range should be 'Just (offset,size)' where offsets is the beginning byte of file region to map and size tells mapping length. There are no alignment requirements. Returned Ptr or ForeignPtr will be aligned to page size boundary and you'll be given offset to your data. Both `offset` and `size` must be nonnegative. Sum `offset + size` should not be greater than file length, except in `ReadWriteEx` mode when file will be extended to cover whole range. We do allow `size` to be 0 and we do mmap files of 0 length. If your offset is 0 you are guaranteed to receive page aligned pointer back. You are required to give explicit range in case of `ReadWriteEx` even if the file exists.

File extension in `ReadWriteEx` mode seems to use sparse files whenever supported by oprating system and therefore returns immediatelly as postpones real block allocation for later.

For more details about mmap and its consequences see:

- http://opengroup.org/onlinepubs/009695399/functions/mmap.html
- http://www.gnu.org/software/libc/manual/html_node/Memory_002dmapped-I_002fO.html
- http://msdn2.microsoft.com/en-us/library/aa366781(VS.85).aspx

Questions and Answers

- Q: What happens if somebody writes to my mmapped file? A: Undefined. System is free to not synchronize write system call and mmap so nothing is sure. So this might be reflected in your memory or not. This applies even in `WriteCopy` mode.
- Q: What happens if I map `ReadWrite` and change memory? A: After some time in will be written to disk. It is unspecified when this happens.
- Q: What if somebody removes my file? A: Undefined. File with mmapped region is treated by system as open file. Removing such file works the same way as removing open file and different systems have different ideas what to do in such case.
- Q: Why can't I open my file for writting after mmaping it? A: File needs to be unmapped first. Either make sure you don't reference memory mapped regions and force garbage collection (this is hard to do) or better yet use mmaping with explicit memory management.
- Q: Can I map region after end of file? A: You need to use `ReadWriteEx` mode.

# Mapping mode

data Mode

Mode of mapping. Four cases are supported.

Constructors

ReadOnly | file is mapped read-only, file must exist
ReadWrite | file is mapped read-write, file must exist
WriteCopy | file is mapped read-write, but changes aren't propagated to disk, file must exist
ReadWriteEx | file is mapped read-write, if file does not exist it will be created with default permissions, region parameter specifies size, if file size is lower it will be extended with zeros

Instances

Enum Mode |
Eq Mode |
Ord Mode |
Read Mode |
Show Mode |

# Memory mapped files strict interface

mmapFilePtr

Arguments

:: FilePath | name of file to mmap
-> Mode | access mode
-> Maybe (Int64, Int) | range to map, maps whole file if Nothing
-> IO (Ptr a, Int, Int, Int) | (ptr,rawsize,offset,size)

The `mmapFilePtr` function maps a file or device into memory, returning a tuple `(ptr,rawsize,offset,size)` where:

- `ptr` is pointer to mmapped region
- `rawsize` is length (in bytes) of mapped data, rawsize might be greater than size because of alignment
- `offset` tell where your data lives: `plusPtr ptr offset`
- `size` your data length (in bytes)

If `mmapFilePtr` fails for some reason, a `throwErrno` is used.

Use `munmapFilePtr ptr rawsize` to unmap memory.

Memory mapped files will behave as if they were read lazily pages from the file will be loaded into memory on demand.

mmapWithFilePtr

Arguments

:: FilePath | name of file to mmap
-> Mode | access mode
-> Maybe (Int64, Int) | range to map, maps whole file if Nothing
-> ((Ptr (), Int) -> IO a) | action to run
-> IO a | result of action

Memory map region of file using autounmap semantics. See `mmapFilePtr` for description of parameters. The `action` will be executed with tuple `(ptr,size)` as single argument. This is the pointer to mapped data already adjusted and size of requested region. Return value is that of action.

mmapFileForeignPtr

Arguments

:: FilePath | name of file to map
-> Mode | access mode
-> Maybe (Int64, Int) | range to map, maps whole file if Nothing
-> IO (ForeignPtr a, Int, Int) | foreign pointer to beginning of raw region, offset to your data and size of your data

Maps region of file and returns it as `ForeignPtr`. See `mmapFilePtr` for details.

mmapFileByteString

Arguments

:: FilePath | name of file to map
-> Maybe (Int64, Int) | range to map, maps whole file if Nothing
-> IO ByteString | bytestring with file contents

Maps region of file and returns it as `ByteString`. File is mapped in in `ReadOnly` mode. See `mmapFilePtr` for details.

munmapFilePtr

Arguments

:: Ptr a | pointer
-> Int | rawsize
-> IO () |

Unmaps memory region. As parameters use values marked as ptr and rawsize in description of `mmapFilePtr`.

# Memory mapped files lazy interface

mmapFileForeignPtrLazy

Arguments

:: FilePath | name of file to mmap
-> Mode | access mode
-> Maybe (Int64, Int64) | range to map, maps whole file if Nothing
-> IO [(ForeignPtr a, Int, Int)] | (ptr,offset,size)

The `mmapFileForeignPtrLazy` function maps a file or device into memory, returning a list of tuples with the same meaning as in function `mmapFileForeignPtr`.

Chunks are really mapped into memory at the first inspection of a chunk. They are kept in memory while they are referenced, garbage collector takes care of the later.

mmapFileByteStringLazy

Arguments

:: FilePath | name of file to map
-> Maybe (Int64, Int64) | range to map, maps whole file if Nothing
-> IO ByteString | bytestring with file content

Maps region of file and returns it as `ByteString`. File is mapped in in `ReadOnly` mode. See `mmapFileForeignPtrLazy` for details.