[wix-devs] Running WiX on Linux using Mono

[Rob Mensching]

1. file versions - I'm less worried about "weird cases" and more interested in grabbing a bunch of .wxs code (off GitHub?) to ensure that the output of WiX using FileVersionInfo and ::MsiGetFileVersion() always return the same thing. If you're willing to run down a bunch of test cases, I'm willing to switch WiX to the x-plat option. Any one know if FileVersionInfo is supported in .NET Core 2.0 (wouldn't surprise me if it's something they left out)?

3. winterop.exe - I would be willing to create and maintain a documented interface to winterop.exe (command-line input and stdout output) if you'd be willing to share the Linux implementation. Additionally, I think the only code that will remain native is the dutil.lib based smart-cabbing work. The `HashPublicKeyInfo()` can go away since the deprecated 
Authenticode validation in Burn will go away. I expect the `ResetAcls()` can be implemented in managed code now (although I haven't looked). 

PS: "winterop.exe" may not be its final name but you know what I mean.

4. merge modules - I'm fine punting on Merge Module because. .wixlibs are a better solution anyway.

5. "supporting project" - so let's explore options to execute on this. I'd be willing to give you a repo in the "wixtoolset" to place "winterop.exe" for Linux. We haven't touched winterop.dll in many, many years so I expect that once we get "winterop.exe" working, it is code that should be able to just keep working without serious maintenance. Plus, I'm not a Linux dev but if the fixes are straightforward (and we can get a Linux CI machine - TravisCI?) I'll definitely help maintain.

Sound good? Interested? Thoughts?

6. .cab extraction - WiX v4's intermediate representation doesn't use CAB files any more so this will not be an issue. Or rather, "Job's done". <smile/>


So, I'm heads down for the next month but starting in August I expect to make some serious progress on .NET Core 2.0 (to aid with http://wixtoolset.org/development/wips/5489-repository-reorganization/). winterop.exe can get extracted as part of that and open up the opportunity to address #5.

With your and Iain's help, I expect it is a very real possibility we could get to a reasonably supported x-plat core WiX Toolset.


-----Original Message-----
From: Simon Tatham [mailto:anakin at pobox.com] 
Sent: Saturday, July 8, 2017 1:19 AM
To: Rob Mensching <rob at firegiant.com>
Cc: wix-devs <wix-devs at lists.wixtoolset.org>
Subject: Re: [wix-devs] Running WiX on Linux using Mono

<rob at firegiant.com (Rob Mensching)> wrote:
> This isn't something we'll consider for WiX v3.x (WiX v3.x line is
> done) but there are big enough changes in WiX v4 that we can try to
> make life easier.

Thanks for the point-by-point breakdown.

Sorry for the delay in replying - I've just finished actually getting
the PuTTY 0.70 release out of the door. (Which has its MSI files built
by my Linux-only method, incidentally, if you or anyone else has any
interest in looking at them in detail.)

> 1. ::MsiGetFileVersion() - [...] if it can be shown that the C#
> FileVersionInfo class is a perfect replacement for
> ::MsiGetFileVersion() then I'd consider that like for like swap. I'm
> assuming that Mono (or whatever) has an implementation for
> FileVersionInfo.

I hadn't even found C#'s FileVersionInfo class, so thanks for the
pointer there.

A quick check in Mono confirms that it does indeed have an
implementation of that, and it's at the very least competent to retrieve
the same numeric PE VERSIONINFO fields that my own current Linux support
code is fetching. So to start with I wish I'd known about that facility
before I spent 200 lines of C writing my own custom code to do the same
job :-)

I don't know exactly what you'd consider proof that the replacement is
perfect, but I can certainly look into preparing some weird kinds of PE
file (with multiple, malformatted or missing version resources, that
sort of thing) and do some cross-testing on Windows between
MsiGetFileVersion and (the proper Windows implementation of) C#
FileVersionInfo.

> 3. Cab compression in C# - I'm not at all interested at recreating
> "smart cabbing" in C#. However, I am very much thinking about getting
> rid of the multi-threading and winterop.dll and instead launching a
> process to do cab compression. Essentially, much of winterop.dll turns
> into one (or more) executable(s). If you've recreated winterop.dll
> already, not sure this helps significantly... but maybe.

It may not help 'significantly' in the sense of reducing the amount of
code I have to maintain in addition to the main WiX tree, but it would
still help somewhat, so if you were planning to do it anyway then that's
great.

Having a process boundary in place of a DLL API between WiX and my
winterop reimplementation would make testing easier on my side. Also, if
the semantics of the subprocess's command line / standard input are
documented as stable - unlike the current interface to winterop.dll,
which as I understand it is internal to the WiX codebase and subject in
principle to change without notice - then that cuts down my risk of
maintenance churn just to keep the same code working against future
releases.

> 4. Stopping at .idt files - unfortunately, with Merge Modules there
> are phases of processing that happen after .idt file generation (WiX
> actually uses .idt files to create the MSI today). You are probably
> avoiding use of Merge Modules today.

*nods* You're right that I hadn't spotted that feature bceause I'm not
personally using it.

OK, so between that and point #3 it sounds as if I'm not going to be
able to offload _all_ my work into the upstream WiX C# code, and will
have to continue maintaining some sort of separate supporting project
for people wanting to cross-run WiX in my way. That's fair enough (it
always seemed to me like a strong possibility anyway). But if the .cab
and .msi building functionality were to move from .dll into .exe, then
that would seem like a pretty good outcome for me, and provide a
generally sensible and well-defined place at which to draw a line
between the two projects.

> So how much of the above would actually help you? Are there are other
> tasks needed?

There is one other thing, which is that my WiX shim code also had to
implement .cab _extraction_ as well as .cab creation, and the only thing
I've actually seen that used for is extracting things from
.cab-formatted files in the WiX distribution archive itself. For
example, in the PuTTY MSI build, I find that WiX asks my winterop.dll to
extract a file called "cab_1_WixUtilExtension.cab", which as far as I
can see comes from data embeddded in WixUtilExtension.dll in the WiX
binary distribution.

.cab extraction is actually a much more painful job than .cab creation.
(For the usual reason when a file format has many options - a producer
can choose whatever subset of the options it finds easiest, and commit
to only generating those, but a consumer has to be prepared to handle
them all. In the case of .cab, the relevant options are three different
compression methods, of which one is the completely standard and readily
available Deflate and the other two are both proprietary and more
complicated.)

So if I'm right that WiX's only need for .cab extraction arises from the
files in its own distribution, then one helpful thing would be to
convert those files into a format that's less dependent on helper code
provided by a particular host platform. Would that be a reasonable
request?

Cheers,
Simon

-- 
import hashlib; print (lambda p,q,g,y,r,s,m: m if (lambda w:(pow(g,int(hashlib.
 sha1(m).hexdigest(),16)*w%q,p)*pow(y,r*w%q,p)%p)%q)(pow(s,q-2,q))==r else "!"
 )(0xb80b5dacabab6145, 0xf70027d345023, 0x7643bc4018957897, 0x11c2e5d9951130c9,
 0xa54d9cbe4e8ab, 0x746c50eaa1910, "Simon Tatham <anakin at pobox.com>")