outlook-oauth2 issues

[fallwest]

Problems authenticating with outlook-oauth2:

Environment:

• Version: nmail v4.14
• OS / distro: Ubuntu 22.04.2 LTS

Summary
This is fantastic and desperately needed software but I had to hack a bit to get Outlook oauth2 to work.

Issues

• --setup outlook-oauth2 not documented in readme.md
• Running --setup outlook-oauth2 failed. After successfully logging into the Microsoft website that opens during the setup process, the setup exited with message "Setup failed". I had to comment validation as follows to get the setup to work:

diff --git a/src/auth.cpp b/src/auth.cpp
index 37fa540..c179957 100644
--- a/src/auth.cpp
+++ b/src/auth.cpp
@@ -102,7 +102,9 @@ bool Auth::GenerateToken(const std::string& p_Auth)

   int status = PerformAction(Generate);

-  return (WIFEXITED(status) && (WEXITSTATUS(status) == 0));
+  // return (WIFEXITED(status) && (WEXITSTATUS(status) == 0));
+  LOG_DEBUG("status code from GenerateToken(): %d", status);
+  return true;
 }

 std::string Auth::GetName()
diff --git a/src/main.cpp b/src/main.cpp
index 0a3304f..a4902cf 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -744,7 +744,7 @@ bool ValidateConfig(const std::string& p_User, const std::string& p_Imaphost,
                     const uint16_t p_Imapport, const std::string& p_Smtphost,
                     const uint16_t p_Smtpport)
 {
-  if (p_User.empty()) return ReportConfigError("user");
+  // if (p_User.empty()) return ReportConfigError("user");
   if (p_Imaphost.empty()) return ReportConfigError("imaphost");
   if (p_Imapport == 0) return ReportConfigError("imapport");
   if (p_Smtphost.empty()) return ReportConfigError("smtphost");

• Smtp with oauth2 (Modern Auth) fails. Get message that client is trying to use SmtpClientAuthentication:

2023-03-08 20:56:52.557 | DEBUG | smtp->auth = 0x9  (smtp.cpp:166)
2023-03-08 20:56:52.557 | TRACE | smtp 1: >>>>>>> send >>>>>>  (smtp.cpp:778)
2023-03-08 20:56:52.557 | TRACE | smtp 1:   (smtp.cpp:778)
2023-03-08 20:56:52.557 | TRACE | smtp 1: >>>>>>> end send >>>>>>  (smtp.cpp:778)
2023-03-08 20:56:52.558 | TRACE | smtp 1: >>>>>>> send >>>>>>  (smtp.cpp:778)
2023-03-08 20:56:52.558 | TRACE | smtp 1:   (smtp.cpp:778)
2023-03-08 20:56:52.558 | TRACE | smtp 1: >>>>>>> end send >>>>>>  (smtp.cpp:778)
2023-03-08 20:56:52.558 | TRACE | smtp 1: >>>>>>> send >>>>>>  (smtp.cpp:778)
2023-03-08 20:56:52.558 | TRACE | smtp 1:   (smtp.cpp:778)
2023-03-08 20:56:52.558 | TRACE | smtp 1: >>>>>>> end send >>>>>>  (smtp.cpp:778)
2023-03-08 20:57:00.027 | TRACE | smtp 0: <<<<<<< read <<<<<<  (smtp.cpp:778)
2023-03-08 20:57:00.028 | TRACE | smtp 5: 535 5.7.139 Authentication unsuccessful, SmtpClientAuthentication is disabled for the Tenant. Visit https://aka.ms/smtp_auth_disabled for more information. [OS6P279CA0168.NORP279.PROD.OUTLOOK.COM 2023-03-08T19:56:59.901Z 08DB1FA36E0D28FE]  (smtp.cpp:778)
2023-03-08 20:57:00.028 | TRACE | smtp 0:   (smtp.cpp:778)
2023-03-08 20:57:00.028 | TRACE | smtp 0: <<<<<<< end read <<<<<<  (smtp.cpp:778)
2023-03-08 20:57:00.028 | ERROR | mailsmtp_oauth2_authenticate(smtp, m_User.c_str(), token.c_str()) = MAILSMTP_ERROR_UNEXPECTED_CODE  (smtp.cpp:171)
2023-03-08 20:57:00.028 | DEBUG | ~Smtp()  (smtp.cpp:45)
2023-03-08 20:57:00.028 | DEBUG | new status: Idle  (status.cpp:119)

See for more info: https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/authenticated-client-smtp-submission

[fallwest]

Quick update: I got the mail admin to enable SmtpClientAuthentication by following instructions here: https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/authenticated-client-smtp-submission#enable-smtp-auth-for-specific-mailboxes
But it would still be great if nmail could use oauth2 (modern auth) for smtp authentication

[d99kris]

Thanks for reporting and providing a detailed bug report. 👍
I'll look into this.

[d99kris]

To summarize I believe there are three issues:

1. Documentation - the README does not mention --setup outlook-oauth2. Thanks for highlighting, I will fix this.
2. Outlook OAuth2 Setup Failing - this does not happen for my personal/free hotmail address, but perhaps happens for certain corporate/organization email accounts. Based on mentioned workaround it seems for those corps/orgs nmail is not able to fetch user email address. Possibly nmail needs to prompt user to enter the address during outlook-oauth2 setup wizard, in case it's unable to determine from Outlook server. It would be neat to have a corp/org outlook account for reproducing the issue and testing a fix, but I'm not sure if there's any provider offering them free/cheap. Will try do some research.
3. Outlook OAuth2 Smtp / Send Failing - email send used to work (last tested on Jan 14, 2023) so it appears the Outlook service has changed. Googling I found some other recent posts online - like https://learn.microsoft.com/en-us/answers/questions/1168272/oauth2-for-smtp-send-granting-accesstoken-but-retu - which hints there may be other applications facing the same recent issue with Outlook. Will need to do some more research on this as well.

[fallwest]

Thanks for investigating. I would suggest prompting for the user address when using the outlook-oauth2 setup wizard. With my monkey-patching that removes user validation, combined with manually updating the user in config, I find the client works great. I use it continually. As stated above, I could not get oauth2 to work for smtp validation. But my company's mail admin activated the permission specified here and now sending works fine: https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/authenticated-client-smtp-submission#enable-smtp-auth-for-specific-mailboxes . So maybe smtp2 oauth2 can be a less critical new feature development. I can't offer you a company email but am happy to test a new feature branch :-)

[fallwest]

I have set up debugging to understand better what happens in my environment. Here are my findings:

• The value I get from int status = PerformAction(Generate); in auth.cpp is 1280
• The contents of my outlook-oauth2.tokensare:

token_type=Bearer
scope=https://outlook.office.com/IMAP.AccessAsUser.All https://outlook.office.com/SMTP.Send https://outlook.office.com/User.Read
expires_in=5210
ext_expires_in=5210
access_token=1ZDVlZmYverylonghash1ZDVlZmY
refresh_token=1ZDVlZmYanotherlonghash1ZDVlZmY

No email address here.

[fallwest]

Found this in nmails log.txt

72023-03-16 16:13:07.873 | INFO  | starting nmail v4.14  (main.cpp:147)
2023-03-16 16:13:26.069 | WARN  | oauth2 generate failed (5): /home/user/dev/other/nmail/build/oauth2nmail -g > /home/user/dev/other/nmail/conf/temp/tmpfile.VOyUbg.txt 2>&1  (auth.cpp:318)
email address request failed <Response [400]>
{'error': {'code': 'Request_BadRequest', 'message': "Unrecognized query argument specified: 'access_token'.", 'innerError': {'date': '2023-03-16T15:13:26', 'request-id': '9232c061-d5ca-4d5b-981b-d6a39916a760', 'client-request-id': '9232c061-d5ca-4d5b-981b-d6a39916a760'}}}

[d99kris]

Thanks a lot for the PR! Looks fine, will probably merge tomorrow.

So maybe smtp2 oauth2 can be a less critical new feature development.

Yes, I will probably wait and observe how other email clients will handle this.

I can't offer you a company email but am happy to test a new feature branch :-)

Fully understandable 🙂 Actually I found that MS offers 30-days trial of Microsoft 365 so I was able to reproduce setup/authentication issue you were facing.