[wix-users] Merge changes for Config files and IIS settings

Mon Mar 4 11:41:04 PST 2019

That is 100% completely true but you'll also run into the same problem with
a flat key-value-pair configuration file.

Suppose the base file had settings like so:

key1 = value1
key2 = value2
key3 = value3

Then suppose one of your users modified the file as:

key1 = customvalue1
key3 = value3
key2 = customvalue2
customkey4 = value4

And in your upgrade you need to upgrade to a  new settings file that looks
like this:

key1 = newvalue1
key2 = value2
key3 = value3
key4 = differentvalue4
key5 = value5

where key4 may actually  map to customkey4 but the keys are actually
different (perhaps they are alternatives for the same setting and the user
chose one approach and the developer chose a different approach).

How exactly do you implement a general merge algorithm to merge that flat
dataset? How do you ensure it can handle any changes a user could possibly
make? How do you handle conflicts? How upset are your users going to be if
their settings disappear (even if they were "wrong")? Which base version of
the configuration file to use as base in your merge algorithm?

Adding structure to the data complicates the merge process regardless of
whether we're doing it with XML, JSON, YAML, or even just namespacing
settings in a "flat" key-value-pair format! In any case, you still have all
the problems you had previously and you still don't know how to solve
conflicts in a general fashion.

If you know where you are starting and you know where you are going and you
know all the changes the user could possibly have made, then you could
write specific code to handle such a migration but that is not scalable in
any definition of the word.

The truth of the matter is that configuration is always a little database.
You have schema (the keys and structural organization of said keys and the
possible values for those keys) and you have data (the actual values). Both
schema and data can change between versions and decisions need to be made
to ensure an old schema can be upgraded successfully to the new schema as
well as ensuring existing data is migrated accordingly. This is hard and
usually a very custom operation version to version.

Separating configuration into "installer-owned" and "user-owned" sources
(files) simplifies this by allowing the installer to blindly change the
installer-owned file while keeping the user's custom settings around in
another file that is never touched by the installer. If there ever is a
schema change or an incompatible data change, then it becomes the user's
responsibility to update their user-owned file. They would have to do that
part manually anyway (resolving conflicts).

This is a problem that would bite you regardless of installation
technology. That is why other systems (rpm, for example) will at most let
you tag files as config files and they are automatically backed up prior to
getting updated. That provides the user a copy of their data as well as a
copy of the new file so they can manually transfer data and handle any
conflicts.

I suppose I am violently agreeing here. I'm hoping this information can
help arm those folks who may be forced to attempt to try this by management
that won't listen.

--
Edwin G. Castro

On Mon, Mar 4, 2019 at 10:44 AM Christopher Painter <chrpai at iswix.com>
wrote:

> Part of my 20 minute talk on this is that MSI was created in the 90s
> before XML was a thing.  An XML file is essentially an entire registry hive
> in one file.   The component rules / default versioning rules don't support
> that.
>
> You can do basic settings migration using  the remember property pattern
> but that doesn't scale very well.   At the end of the day the best thing to
> do is keep them separate and make it the applications responsibility to
> merge the layers together to get the setting.
>
> ------------------------------
> *From:* Edwin Castro <egcastr at gmail.com>
> *Sent:* Monday, March 4, 2019 11:23 AM
> *To:* WiX Toolset Users Mailing List
> *Cc:* Christopher Painter
> *Subject:* Re: [wix-users] Merge changes for Config files and IIS settings
>
> I'd recommend Christopher's suggestions as well.
>
> Merging any kind of data is hard but merging structured data is harder.
> It, of course, can be done but you'll probably need to implement it
> yourself. You can find tools built-in into editors but those won't help you
> in an installer. Checkout "A Three-way Merge for XML Documents", below, if
> you want to get a sense for what you'd need to do and the challenges
> involved.
>
>
> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.59.2179&rep=rep1&type=pdf
>
> If you want to read the entire thesis, then you can find it at
>
> https://www.cs.hut.fi/~ctl/3dm/thesis.pdf
>
> There's a java implementation available there too:
>
> https://www.cs.hut.fi/~ctl/3dm/
>
> To do any meaningful merging of data you'll need 3 documents, the base
> document (initial document), the current document (their changes), and the
> upgrade document (the new changes the installer wants to apply). The
> algorithm described in x will help create a fourth document (the actual
> document left on disk at the end of the process) that will contain the
> merged data. You will still need to make some arbitrary decision for
> handling conflicts if you cannot provide a mechanism to allow the user to
> select a resolution for a conflict.
>
> Handling conflicts is probably we're you'd get stuck. I know of literally
> no installer technology anywhere that actually merges data because you
> cannot do it successfully 100% of the time without user intervention. It is
> possible you can succeed a lot of the time but it is impossible to predict
> because you don't know what changes have been made by users!
>
> --
> Edwin G. Castro
>
>
> On Mon, Mar 4, 2019 at 7:46 AM Christopher Painter via wix-users <
> wix-users at lists.wixtoolset.org> wrote:
>
> If you must meet the "requirements" I don't have any suggestions.  If the
> requirements are negotiable I can tell you how this would handle this with
> a prospective customer.
>
> 1+2) Keep "stock" / "default" data owned by the installer and "custom"
> data owned by the user in separate files.    For web.config/app.config this
> can typically be handled by leveraging the  appSettings file attribute to
> point to an override file.   Simply put  MSI wasn't designed for meging two
> sources of truth in an XML file.   This is typically a 20 minute discussion
> so hit me up if you need more information.
>
> 3)  I tell customers to tell their customers that the installer owns the
> website/application that it creates.  Now basic override / variation points
> can be supported by leveraging the iis:WebSite ConfigureIfExists=No  but at
> some point if they need to do something very different then the installer
> then they should just create their own website pointed to the same document
> directory and manage it themselves.
>
> I have many customers that ship shrinkwrapped  web applications like this
> and it always works out for them  keeping it simple this way.
>
> ________________________________
> From: wix-users <wix-users-bounces at lists.wixtoolset.org> on behalf of Ven
> H via wix-users <wix-users at lists.wixtoolset.org>
> Sent: Monday, March 4, 2019 4:28 AM
> To: WiX Toolset Users Mailing List
> Cc: Ven H
> Subject: Re: [wix-users] Merge changes for Config files and IIS settings
>
> Does anyone have any inputs? Please help.
>
> On Fri, Feb 22, 2019 at 11:26 AM Ven H <venh.123 at gmail.com> wrote:
>
> > I have a business requirement. I know many eyebrows will be raised, but
> > unfortunately, I need to achieve this requirement.
> >
> > There is an application folder that already exists (which was probably
> > installed by some other MSI). Now, I have to install the same application
> > folder using my MSI. The application folder has a lot of config files.
> > Since the end customer would have made some changes to this config, my
> MSI
> > should not overwrite these files. But, the config file changes from my
> MSI
> > should be merged from my MSI. For this, I think I may have to use a
> Custom
> > Action. However, I have the following challenges / questions.
> >
> > 1. If a file already exists and my MSI also has the same file to be
> > installed, will the file from MSI overwrite the existing file? If so, can
> > this be prevented selectively ( I don't want some selected files like
> > config files to be overwritten) by using NeverOverwrite attribute (Will
> > NeverOverwrite work on files outside of MSI, i.e., the files already
> > present and not installed by MSI or it will work 2nd time when the same
> MSI
> > is being installed or upgraded)?
> >
> > 2. Let's say this file is not overwritten and merged using Custom Action,
> > what will happen when I run an Upgrade again using my MSI? Will it
> > overwrite the other files (config file components will have
> NeverOverwrite
> > attribute and hence they will be spared, I believe)?
> >
> > 3. I have similar requirement for IIS (web sites and app pools). They
> > might have been created manually or using another MSI. First time, when I
> > run my MSI, they should not be recreated since the end customer could
> have
> > updated some settings manually. So, I should be able to merge the
> changes.
> >
> > Please help / advise.
> >
>
> ____________________________________________________________________
> WiX Toolset Users Mailing List provided by FireGiant
> http://www.firegiant.com/
>
> ____________________________________________________________________
> WiX Toolset Users Mailing List provided by FireGiant
> http://www.firegiant.com/
>
>