Have any other O365 admins noticed this behavior?
If I change a password, it can take hours for the mail client to ask for the new password. This means that mail is still being delivered with the old credentials.
This came up recently with an employee that was terminated. I also noticed it yesterday when I changed my personal O365 password online. My mail continued to sync on my mobile and computer for around 5hrs before I was prompted for the new password.