Skip to content

DO NOT MERGE Per-queue delivers for old classic queues#3253

Closed
lhoguin wants to merge 2 commits into
masterfrom
old-index-per-queue-delivers
Closed

DO NOT MERGE Per-queue delivers for old classic queues#3253
lhoguin wants to merge 2 commits into
masterfrom
old-index-per-queue-delivers

Conversation

@lhoguin

@lhoguin lhoguin commented Jul 30, 2021

Copy link
Copy Markdown
Contributor

This is a change that comes from the modern index work and proves beneficial for the old index and could provide a boost in a 3.9 patch release. Context: #3029 (comment)

It's very WIP at the time of opening and the PR is only opened for the sake of tracking the work. Because August will be slow there might not be too much progress at first. Lots of code removal/simplification remains to be done and so the performance boost might increase further.

Types of Changes

Non-breaking performance improvement.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating
the PR. If you're unsure about any of them, don't hesitate to ask on the
mailing list. We're here to help! This is simply a reminder of what we are
going to look for before merging your code.

  • I have read the CONTRIBUTING.md document
  • I have signed the CA (see https://cla.pivotal.io/sign/rabbitmq)
  • All tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in related repositories

@lhoguin

This comment has been minimized.

Comment thread deps/rabbit/src/rabbit_queue_index.erl
@lhoguin lhoguin force-pushed the old-index-per-queue-delivers branch from 5a10378 to f6947c1 Compare August 16, 2021 10:05
@lhoguin

lhoguin commented Aug 20, 2021

Copy link
Copy Markdown
Contributor Author

The old index segment format is unfortunate. While the journal has 3 types of entries (message, deliver, ack), the segment file only has two (message, deliver+ack), where a deliver is indicated by having the second entry type written once, and an ack is indicated by having the second entry type written twice. The code then changes from undelivered to delivered to acked as it encounters these entries in the file when reading.

This makes any changes to deliver handling difficult. Even if the queue was tracking delivers, the index still needs to track them because of that unfortunate design, so I do not think there's much to be gained. At best we can drop the deliver writes in the journal. But the risk of doing that kind of change is important as the previous index change mess shows, and the gains while significant are not earth shattering.

I am dropping this PR as a result.

This change remains valid for the new index however.

@lhoguin lhoguin closed this Aug 20, 2021
@lhoguin lhoguin deleted the old-index-per-queue-delivers branch August 20, 2021 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants