Send IDONTWANT on publish to avoid downloading data we already have

[jimmygchen]

Issue Addressed

With the introduction of getBlobsV1 optimisation, nodes may be publishing blobs if they retrieved it from the EL but haven't received it from gossip. In these scenarios, the node already have the blobs and can save download bandwidth by sending out IDONTWANT to peers on the blobs we publish, so that it avoids downloading them from gossip.

This would apply to all messages > 1000 bytes (idontwant_message_size_threshold) though, and in most cases we won't be publishing the same messages as other, so perhaps we could also consider adding a send_idontwant bool flag to the behavour.publish function, and only set it to true when publishing blobs?

[jxs]

Thanks for starting this JImmy, left a comment.
Meanwhile the spec is clear in stating that IDONTWANT's should be sent for received messages:

In more specific terms, a new control message is introduced: IDONTWANT. It's primarily intended to notify mesh peers that the node already received a message and there is no need to send its duplicate.

and:

When the peer receives the first message instance it immediately broadcasts (not queue for later piggybacking) IDONTWANT with the messageId to all its mesh peers. This could be performed prior to the message validation to further increase the effectiveness of the approach.

can we forward this conversation to the spec and open an issue there proposing to also update the spec?

[jxs]

Re-reading on your comment Jimmy,

though, and in most cases we won't be publishing the same messages as other, so perhaps we could also consider adding a send_idontwant bool flag to the behavour.publish function, and only set it to true when publishing blobs

I think this makes sense, we probably should offer this option to users

[jimmygchen]

I think this makes sense, we probably should offer this option to users

I had a chat with Age regarding this and we decided not to add this flag.

The idontwant_message_size_threshold should limit the amount of IDONTWANT being sent to peers - if a message exceeds idontwant_message_size_threshold, it's probably worth sending an IDONTWANT.

Let me know what you think!

[jxs]

LGTM thanks Jimmy!

I had a chat with Age regarding this and we decided not to add this flag.

The idontwant_message_size_threshold should limit the amount of IDONTWANT being sent to peers - if a message exceeds idontwant_message_size_threshold, it's probably worth sending an IDONTWANT.

Let me know what you think!

Yeah makes sense

[jimmygchen]

@mergify queue

[mergify]

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 6a8d13e