shibuya-kafka-adapter

Kafka adapter for the Shibuya queue-processing framework.

Integrates with Apache Kafka via kafka-effectful for the consumer effect (polling, offset store, partition pause) and hw-kafka-streamly for error classification (skipNonFatal), on top of hw-kafka-client. Provides polling, offset commit semantics, partition awareness, and graceful shutdown.

Packages

• shibuya-kafka-adapter — the adapter library (Shibuya.Adapter.Kafka, .Config, .Convert).
• shibuya-kafka-adapter-bench — micro-benchmarks for the conversion hot path (ConsumerRecord → Envelope, W3C header extraction, timestamps).
• shibuya-kafka-adapter-jitsurei — runnable examples: BasicConsumer, MultiTopic, MultiPartition, OffsetManagement.

Building

The repo ships a Nix flake and direnv config for a reproducible toolchain.

direnv allow        # or: nix develop
cabal build all
cabal test shibuya-kafka-adapter

Benchmarks and examples:

cabal bench shibuya-kafka-adapter-bench
cabal run BasicConsumer

Layout

shibuya-kafka-adapter/            library sources and tests
shibuya-kafka-adapter-bench/      tasty-bench micro-benchmarks
shibuya-kafka-adapter-jitsurei/   runnable usage examples
docs/plans/                       execution plans
mori.dhall                        project manifest (mori registry)

License

MIT. See package cabal files for details.