[wix-devs] FilesInUse in Burn

[Sean Hall]

Testing out https://github.com/wixtoolset/issues/issues/5790 made me learn
about how we're handling INSTALLMESSAGE_RMFILESINUSE
and INSTALLMESSAGE_FILESINUSE in Burn. As it turns out, neither of the
hypotheses from
http://windows-installer-xml-wix-toolset.687559.n2.nabble.com/Feature-4382-Add-support-for-quot-files-in-use-quot-in-WixStdBA-td7594302.html
is
right about how IDRETRY works. Georg was right that when you return IDRETRY
from INSTALLMESSAGE_RMFILESINUSE, you're asking Windows Installer to
fallback to INSTALLMESSAGE_FILESINUSE (
https://docs.microsoft.com/en-us/windows/win32/msi/using-restart-manager-with-an-external-ui-).
If you return IDRETRY from INSTALLMESSAGE_FILESINUSE, then you're asking
Windows Installer to go back and do costing again. This means that the
FilesInUse table is cleared and the process starts over (
https://docs.microsoft.com/en-us/windows/win32/msi/installvalidate-action).
So if you only look at the messages coming to the external UI handler then
it looks like it just switches back and forth, but Windows Installer is
actually doing real work after the IDRETRY from INSTALLMESSAGE_FILESINUSE.
The detection logic doesn't change, it's either using Restart Manager or it
isn't (this is based on my interpretation of the documentation of
MSIRESTARTMANAGERCONTROL
https://docs.microsoft.com/en-us/windows/win32/msi/msirestartmanagercontrol -
value "DisableShutdown" - The installer still uses the Restart Manager to
detect files in use by applications).

It isn't clear in the documentation how Windows Installer handles results
from INSTALLMESSAGE_FILESINUSE when Restart Manager is being used. I don't
know how to test whether it's actually using Restart Manager. But since it
uses Restart Manager to detect files and populate the FilesInUse table, I
would expect it to also use Restart Manager to handle the files from the
FilesInUse table during install.

So for Burn if the BA returns IDRETRY from INSTALLMESSAGE_RMFILESINUSE,
then I think that Burn should automatically return IDRETRY to the follow up
INSTALLMESSAGE_FILESINUSE. I don't think there's anything actionable on
https://github.com/wixtoolset/issues/issues/5208.

I think we should let the BA know whether the message was
INSTALLMESSAGE_RMFILESINUSE or INSTALLMESSAGE_FILESINUSE, just in case the
BA needs to know that.

For both messages, there are always two fields per file (
https://docs.microsoft.com/en-us/windows/win32/msi/sending-messages-to-windows-installer-using-msiprocessmessage).
Burn or wiutil should probably send more structured data than just the
array of fields.