Enable concatenation of search queries (split by " / ")

[MaxGyver83]

Implements #1536.

This PR enables the concatenation of two or more search queries. They will be split at ' / ' (a slash surrounded by spaces).

Example:

:search tag:flagged / tag:inbox NOT tag:flagged

This gives you first all flagged e-mails, then all e-mails tagged inbox (not repeating the flagged ones).

Possible problem: I don't know how you would search for a string containing ' / '.

[MaxGyver83]

Hi @pazz , I don't think that Travis CI failed because of my PR. The job log says:

./configure: 1558: ./configure: WITH_PYTHON_DOCS: parameter not set

[pazz]

This was an issue with notmuch master. It is fixed now. Could you squash the two commits? Quoting Max (2020-07-16 19:40:19)

…

Hi @pazz , I don't think that Travis CI failed because of my PR. The job log says: ./configure: 1558: ./configure: WITH_PYTHON_DOCS: parameter not set — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.*

[MaxGyver83]

Could you squash the two commits?

Sure, they are squashed now.

[pazz]

@lucc: opinions?

[mmartin]

This is really wonky. It breaks my hooks (search returns double messages or no messages at all) and the result count is wrong.

[lucc]

Is it possible to add some test for the new query syntax? I can imagine there are some corner cases that are worth testing (see inline comment).

[lucc]

What happens if this is called after somebody started iterating over the IterableWalker already? Is that a case we have to think about? Even if this method is trivial it might be worth it to add a docstring to mention such constrains (if there are some).

[MaxGyver83]

Good question. I'll try to test that case somehow.

[MaxGyver83]

This is really wonky. It breaks my hooks (search returns double messages or no messages at all) and the result count is wrong.

@mmartin: Thanks for testing this PR. I have fixed the wrong result count. Could you give an example for a search that returns double messages?

[mmartin]

@MaxGyver83 It's this one: https://github.com/pazz/alot/wiki/Contrib-Hooks#auto-refresh-search-buffer

[MaxGyver83]

@MaxGyver83 It's this one: https://github.com/pazz/alot/wiki/Contrib-Hooks#auto-refresh-search-buffer

@mmartin: I could reproduce that behavior. Fixed in 5cf9bdf.

[lucc]

@pazz LGTM. I would be happy if we can have some tests for this. But if you want to push this PR you can "overrule" this request for tests :)

[MaxGyver83]

I couldn't find any search related tests. Probably because you need real e-mails for that.

[MaxGyver83]

I don't know if slash is a good query divider. Maybe the word THEN is even better because it matches better the notmuch keywords. For example: search tag:flagged THEN tag:inbox AND NOT tag:flagged.

[pazz]

I agree that adding tests should be good.

Conceptually, what bugs me about this series is that it diverges from notmuch's query syntax, at least it looks like it does, which is something that I've so far tried to avoid.

One way to get around that would be not to have a single "meta query" but something like the 'refine' command, which allows to stepwise append result lists for individual, valid notmuch queries.
One could then concatenate such commands to have the same effect as proposed here..

[MaxGyver83]

One way to get around that would be not to have a single "meta query" but something like the 'refine' command, which allows to stepwise append result lists for individual, valid notmuch queries.
One could then concatenate such commands to have the same effect as proposed here..

Sure, this sounds good to me. If I understand you right, you prefer something like :search tag:flagged; append tag:inbox AND NOT tag:flagged. Or maybe append-search. I'll update my PR. Can you already think of a command name that you like?

[MaxGyver83]

I started working on the requested change and it looks like it's a big one!

Adding a new command to commands/search.py is easy. But the next logical step would be to replace self.querystring in buffers/search.py:SearchBuffer with a list of querystrings. So the new append command could add a new query to that list and then rebuild the current buffer. But unfortunately the querystring is used in many places, p.e. when checking if there is already an open buffer with the same query or when using the refine command. We need to define a new behavior for these, too. For example, we check instead if there exists already a buffer with the same list of queries and the refine command will only be applied to the last query in that list.

For me it's okay to finish this task but first I want to check if you are okay with such extensive changes to the code. @pazz: What do you think about this?