Broadway producer for PgQue, a high-throughput, zero-bloat Postgres queue engine.

Example Usage

Add BroadwayPgQue.Producer to your Broadway pipeline:

defmodule MyPipeline do
  use Broadway

  alias BroadwayPgQue.Producer, as: PgQueProducer

  def start_link(_opts) do
    Broadway.start_link(__MODULE__,
      name: __MODULE__,
      producer: [
        module: {
          PgQueProducer,
          repo: MyApp.Repo,
          queue: "notifications",
          concurrency: 1
        }
      ],
      processors: [
        default: [concurrency: 10]
      ],
      batchers: [
        default: [batch_size: 50, batch_timeout: 1000]
      ]
    )
  end

  @impl true
  def handle_message(_processor, message, _context) do
    # message.data contains the payload
    # message.metadata contains :id, :batch_id, :type, :retry_count, etc.
    message
  end
end

Instant Wake Up

When enqueuing events in PostgreSQL, you can immediately wake up the Broadway producer (bypassing any idle poll interval):

BroadwayPgQue.wake_up(MyPipeline)