[Bug] High CPU use when connecting

[Kabouik]

nmail seems to be heavy on CPU when connecting to an account. In most cases it is a non-issue because the connection is quick, but it can be more annoying if nmail is launched when actually offline. I suspect this high CPU demand is a bug and not really a necessity?

[d99kris]

Hello @Kabouik - thanks for reporting! A few questions to facilitate the troubleshooting: 1. Did this high CPU usage occur only recently? Or it's always been like this? 2. How many messages are there in the inbox (you can press A in message list view to get a number, I don't need exact - rounded to two value figures is sufficient)? 3. Are your encryption settings default? I.e. does your ~/.nmail/main.conf have the following (default) values, or any deviations? addressbook_encrypt=0 cache_encrypt=1 cache_index_encrypt=0 queue_encrypt=1 auth_encrypt=1 In general I agree we shouldn't need high CPU usage on startup, and I don't see any high CPU usage on my systems after the initial 200 ms load - but I'm using the "inbox zero" strategy, which may be very different from your and others use-cases.

…

On 2021-04-08 23:31 Kabouik ***@***.***> wrote: nmail seems to be heavy on CPU when connecting to an account. In most cases it is a non-issue because the connection is quick, but it can be more annoying when nmail is launched when actually offline. I suspect this high CPU demand is a bug and not really a necessity?  -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #74

[Kabouik]

1. I've been using nmail since a long time and rarely launch it when offline so I can't tell if it was happening at the beginning, but it's definitely months at least.
2. A lot, we're in the 7500 and 1700 range for the two accounts in the above screenshot, respectively. Both of them also have thousands of emails in other folders that are in the local cache, but those folders are not set as inbox in main.conf so I don't know if they may have an influence. I'll try tomorrow on the same machine with some less cluttered accounts or just with changing the inbox folders in main.conf to see if it's related.
3. I use the default values, except for addressbook_encrypt set to 1.

While nmail is quick to load compared to any other email client I tried, it's not exactly 200 ms in my case. This is something that I assume is related to the size of my inbox, since it seems to be quicker with accounts with small inbox. I feel that the initial fetching speed is affected even if the last fetch was fairly recent and just a few new emails came in (subsequent fetchings when nmail is already running are quick).

[d99kris]

Thanks for the quick reply. I suspect it's nmail not handling a large number of messages in the inbox very well, essentially the time it takes to read all the cached email headers. Number of messages in other folders should not be a factor (until one navigates to those folders). I have some ideas on what can be done. Unfortunately not a quick fix, but I'll slowly try to work on it.

[d99kris]

Hi @Kabouik - some performance improvements have been implemented now, in the latest couple commits (up to v3.32 / b5479f9). There is still some room for further improvements, but the last step will require major rework, so it wont happen too soon. Please let me know how you find the latest version w.r.t. performance.

Do note that the recent code changes implements a new type of offline cache (based on sqlite rather than simple file-based). So any old cached messages will be cleared, for nmail to build a new cache.

[Kabouik]

Thanks @d99kris for the hard work! As discussed in a chat, I observed (much) longer quit times after the change when encryption is enabled, which I assume are related to the fact that my cache/ folder and its individual subfolders are rather big.

Since this is probably associated with the time it takes to re-encrypt the folder and delete temp/, maybe an additional status for "Encrypting" in the top right would be welcome, so that users have a feedback when hitting the quit keybind and seeing nmail becoming unresponsive for some time. What do you think?

Regarding performance itself, with encryption enabled and large folders, there might be a regression since nmail is less responsive than it was with plain .eml. But CPU load may be lower, and I know that nmail is now much faster with small folders and/or encryption disabled, which is the recommended setting. There are also other advantages like exporting encrypted folders is now possible.

[Kabouik]

Also, related:

Not sure if related to the above or to my large full-sync being partly incomplete, but I observe "Fetching" much more frequently than before

Hm maybe we can change the status message for reading cache, so at least it's more clear whether it's fetching from server or reading from cache.. there shouldn't be more time spent doing either now though..

[d99kris]

Let's track the performance improvements for encrypted cache under #88 and close this issue.