exceptionfree-readfile

exceptionfree-readfile provides a readFile that reads files into memory without throwing any exceptions (internally).

The code for exceptionfree-readfile mirrors base's implementation of readFile wherever possible, but avoids the exception-throwing style of base in favor of recursion with an accumulator.

Why?

exceptionfree-readfile was created after the realization that running a binary with +RTS -xc -RTS in prodution logs an error on every successfully read file, due to base's implementation of readFile's throwing of IOErrors internally.

This particular bad idea was spawned by a reddit comment.

Getting started

Stack

Here's how to incorporate exceptionfree-readfile into a stack-powered project:

1. Add the library to your stack.yaml's extra-deps

# Dependency packages to be pulled from upstream that are not in the resolver
# (e.g., acme-missiles-0.3)
extra-deps:
- ... other deps ...
- exceptionfree-readfile

2. Add the library to your project's project.yaml/<project>.cabal

build-depends: base >= 4.7 && < 5
             , ... lots of other deps ..
             , exceptionfree-readfile
             , ... more deps ..

3. Use the library in your project

Here's an example of ExceptionFree used in function that reads some SQL out of a file!

module YourModule where

import qualified System.IO.ExceptionFree as ExceptionFree

-- | Read a given file without logging any exceptions
readFileWithoutXCTraces :: FilePath -> IO (Either IOError String)
readFileWithoutXCTraces = ExceptionFree.readFile

Known Issues / Gotchas

There are a few things to keep in mind when evaluating whether you should use exceptionfree-readfile:

• exceptionfree-readfile performs only non-blocking reads, and is thus not a good choice for reading pseudo-files like stdin.
• exceptionfree-readfile can be ~1-10x slower than regular readFile depending on the size of the file (see benchmark below).
• While exceptionfree-readfile makes a concerted effort to be at as robust as System.IO.readFile it is a work in progress, and does not have a test suite as extensive as base.

exceptionfree-readfile does worse than original for almost every case (2x-10x) except very small files. The Oxidized version does better, but not as good as the code in base (and requires dynamic linking of the rust library at runtime).

Benchmark Results

You can find the most recent results in bench/most-recent-results.txt, generated by make bench-gen-results. Here is a set of recent results:

exceptionfree-readfile> benchmarks
Running 1 benchmarks...
Benchmark bench: RUNNING...
benchmarking readFile/Original-KB1
time                 25.71 μs   (24.04 μs .. 27.52 μs)
                     0.973 R²   (0.956 R² .. 0.988 R²)
mean                 27.11 μs   (26.19 μs .. 28.13 μs)
std dev              3.471 μs   (2.658 μs .. 4.366 μs)
variance introduced by outliers: 90% (severely inflated)

benchmarking readFile/ExceptionFree-KB1
time                 38.19 μs   (37.00 μs .. 39.64 μs)
                     0.985 R²   (0.977 R² .. 0.991 R²)
mean                 39.59 μs   (38.39 μs .. 41.02 μs)
std dev              4.608 μs   (3.948 μs .. 5.511 μs)
variance introduced by outliers: 88% (severely inflated)

benchmarking readFile/Oxidized-KB1
time                 21.65 μs   (20.49 μs .. 22.96 μs)
                     0.983 R²   (0.978 R² .. 0.991 R²)
mean                 21.11 μs   (20.39 μs .. 22.00 μs)
std dev              2.924 μs   (2.294 μs .. 3.925 μs)
variance introduced by outliers: 91% (severely inflated)

benchmarking readFile/Original-KB10
time                 197.0 μs   (185.8 μs .. 210.0 μs)
                     0.977 R²   (0.962 R² .. 0.990 R²)
mean                 193.0 μs   (185.9 μs .. 206.8 μs)
std dev              31.11 μs   (22.64 μs .. 46.27 μs)
variance introduced by outliers: 92% (severely inflated)

benchmarking readFile/ExceptionFree-KB10
time                 444.1 μs   (429.3 μs .. 463.7 μs)
                     0.969 R²   (0.951 R² .. 0.983 R²)
mean                 498.1 μs   (477.1 μs .. 525.0 μs)
std dev              90.58 μs   (74.15 μs .. 115.0 μs)
variance introduced by outliers: 92% (severely inflated)

benchmarking readFile/Oxidized-KB10
time                 200.0 μs   (192.7 μs .. 208.2 μs)
                     0.984 R²   (0.976 R² .. 0.990 R²)
mean                 217.8 μs   (206.3 μs .. 235.5 μs)
std dev              45.13 μs   (27.80 μs .. 72.23 μs)
variance introduced by outliers: 95% (severely inflated)

benchmarking readFile/Original-KB100
time                 1.600 ms   (1.448 ms .. 1.746 ms)
                     0.955 R²   (0.940 R² .. 0.980 R²)
mean                 1.641 ms   (1.547 ms .. 1.807 ms)
std dev              425.7 μs   (309.3 μs .. 595.0 μs)
variance introduced by outliers: 95% (severely inflated)

benchmarking readFile/ExceptionFree-KB100
time                 16.16 ms   (13.78 ms .. 18.39 ms)
                     0.945 R²   (0.919 R² .. 0.996 R²)
mean                 14.23 ms   (13.74 ms .. 15.54 ms)
std dev              1.898 ms   (1.235 ms .. 2.798 ms)
variance introduced by outliers: 63% (severely inflated)

benchmarking readFile/Oxidized-KB100
time                 9.387 ms   (8.010 ms .. 11.36 ms)
                     0.866 R²   (0.787 R² .. 0.970 R²)
mean                 9.796 ms   (9.046 ms .. 10.87 ms)
std dev              2.187 ms   (1.214 ms .. 3.476 ms)
variance introduced by outliers: 86% (severely inflated)

benchmarking readFile/Original-MB1
time                 18.93 ms   (14.96 ms .. 24.41 ms)
                     0.796 R²   (0.693 R² .. 0.974 R²)
mean                 16.11 ms   (15.08 ms .. 18.72 ms)
std dev              3.853 ms   (1.724 ms .. 7.110 ms)
variance introduced by outliers: 86% (severely inflated)

benchmarking readFile/ExceptionFree-MB1
time                 173.6 ms   (111.2 ms .. 219.5 ms)
                     0.955 R²   (0.926 R² .. 1.000 R²)
mean                 220.9 ms   (202.2 ms .. 245.1 ms)
std dev              29.51 ms   (20.92 ms .. 38.04 ms)
variance introduced by outliers: 32% (moderately inflated)

benchmarking readFile/Oxidized-MB1
time                 130.4 ms   (116.8 ms .. 143.0 ms)
                     0.990 R²   (0.981 R² .. 1.000 R²)
mean                 148.7 ms   (138.4 ms .. 174.8 ms)
std dev              23.15 ms   (4.719 ms .. 35.59 ms)
variance introduced by outliers: 41% (moderately inflated)

benchmarking readFile/Original-MB10
time                 140.3 ms   (129.8 ms .. 153.5 ms)
                     0.990 R²   (0.964 R² .. 1.000 R²)
mean                 155.6 ms   (145.7 ms .. 171.3 ms)
std dev              18.40 ms   (9.635 ms .. 27.28 ms)
variance introduced by outliers: 27% (moderately inflated)

benchmarking readFile/ExceptionFree-MB10
time                 1.952 s    (1.365 s .. 2.302 s)
                     0.990 R²   (NaN R² .. 1.000 R²)
mean                 2.012 s    (1.894 s .. 2.189 s)
std dev              185.2 ms   (81.54 ms .. 258.6 ms)
variance introduced by outliers: 22% (moderately inflated)

benchmarking readFile/Oxidized-MB10
time                 1.433 s    (1.278 s .. 1.584 s)
                     0.998 R²   (0.994 R² .. 1.000 R²)
mean                 1.435 s    (1.409 s .. 1.451 s)
std dev              25.96 ms   (7.586 ms .. 35.23 ms)
variance introduced by outliers: 19% (moderately inflated)

Benchmark bench: FINISH
Completed 2 action(s).

Running it yourself

The tests, benchmark, and profile runs. The easiest way to run the tests and benchmarks is to use the Makefile:

• make test (which runs test-unit and test-e2e targets)
• make bench
• make profile

Building the Oxidized (Rust-powered) Version

NOTE You must build this library yourself to use the oxidized version.

To build the Rust oxidized version of the library use the cabal flag "oxidized". You can run any of the common targets (build, bench, etc) with the OXIDIZED=true Make variable to build the oxidized version:

$ make rust                # build the rust lib
$ make build OXIDIZED=true # build the exceptionfree-readfile lib & binary using rust

Note that you must have the rust toolchain (rustc, cargo, etc) installed before that command is run. To test out that it's working on your machine, build and use the binary like so:

$ make build OXIDIZED=true
$ ./target/exceptionfree-readfile -m oxidized test/fixtures/example-long-file.txt
<file output>

Using the oxidized version in your code

If you'd like to use the oxidized version in your code -- i.e. passing the flag in when you're building your project so cabal knows to enable the "oxidized" flag for exceptionfree-readfile, you'll need to:

1. Build the rust code (make rust)
2. Ensure the dynamic shared library files (rust/target/release/*.so) are available for your project at link & compile time via the usual methods (LD_LIBRARY_PATH, etc)
3. Build your project (building exceptionfree-readfile along the way)
4. Ensure when you run your project's binary that the dynamic shared libraries (*.so) files are available the usual ways (ex. in /lib//usr/lib)

Contributing

Contributions and pull requests are welcome! If you'd like to contribute:

0. Fork this repository
1. Make changes
2. make
3. make test
4. make profile bench (ensure the benchmark results are indeed better)
5. Submit a PR