persistent-postgresql-streaming

This library allows for memory-constant streaming of persistent entities from PostgreSQL databases.

This code makes use of the PostgreSQL-only cursors, which allow for batched access to the result set of a query at speeds comparable to loading all the results into memory at once.

See the main project README for more.

Streaming Persistent queries

The main function of this library is selectStream, which can be used in place of selectSource. selectSource runs in a ConduitT - consuming the conduit will pull results from the database in constant memory.

FAQ

Why is selectStream so slow?

Have you configured PostgreSQL correctly? See the section in the README about it.