nsqd: allowing clients to integrate with message ID?

[Dieterbe]

I'm tracing messages through my nsq flow and i note:

• when consuming a message and publishing the same message to another topic, it'll get a new id.
• there's no way to set the message ID at publish time, or get it as a return value from the Publish function

because I need a consistent ID of messages as they are published, consumed (and potentially re-published and re-consumed) so i can trace them throughout the pipeline, I have to basically put my own ID in the first 16 bytes of all messages.
but this seems awkward, especially since the message api does expose the ID property publically, but i can't do what i want with it, so i still have to write my own logic for writing and reading ID's, and now all messages essentially have 2 ID's.

would it be acceptable for producers to provide ID's to use when publishing messages?
is there anything harmfull about re-using the existing ID when republishing a given existing (just-consumed) message to a new topic?

thanks

[mreiferson]

Originally we thought it might be useful for NSQ's message IDs to be globally unique and transparent to the user. You're basically seeing the side-effects of these admittedly half-baked ideas.

As it has played out, message IDs are effectively an internal implementation detail. The best path forward is to more explicitly enforce that, by making all of those exposed touch points private - potentially as part of #367?

In short, it is highly recommended (required?) that you use your own message IDs and pretend that NSQ's aren't actually exposed, because at some point in the future that will be true. This addresses all of the issues you've highlighted except for the "inefficiency" of technically having two IDs, which I don't think has any practical impact.

Gonna close, because this doesn't feel like an actionable issue - we can continue the discussion in IRC or on the mailing list.

[Dieterbe]

fair enough, thanks.