[wix-users] Database creation issue

[Habib Salim]

Edwin,
You really are an excellent resource and a great asset to this forum.
Thank you for all that you do.
HS

-----Original Message-----
From: wix-users [mailto:wix-users-bounces at lists.wixtoolset.org] On Behalf
Of Edwin Castro via wix-users
Sent: Friday, May 11, 2018 12:58 PM
To: Ven H <venh.123 at gmail.com>
Cc: Edwin Castro <egcastr at gmail.com>; WiX Toolset Users Mailing List
<wix-users at lists.wixtoolset.org>
Subject: Re: [wix-users] Database creation issue

In previous emails I have explained about how immediate, deferred, and
rollback custom actions work. Please review those emails. Also review

https://blogs.msdn.microsoft.com/rflaming/2006/09/21/uac-in-msi-notes-the-
saw-tooth-diagram/

as that blog post does a really good job explaining when various actions
are actually executed. Specifically pay attention to the "My Saw Tooth
Diagram" and 'Framing the "Saw Tooth" Diagram' sections.

Please do take the time to understand the blog post above. I've noticed
that many of the problems you are running into stem from not understanding
the execution model of the Windows Installer engine. Understanding how the
Windows Installer engine works is very important because it decides what
we can do. WiX cannot do anything to change how the Windows Installer
engine works.

You can use immediate actions in the InstallUISequence for the "UI Client"
line in the Violet Region. Immediate actions run right away and cannot
modify the system.

The "Generate" line in the Orange Region represents immediate actions in
the InstallExecuteSequence scheduled between InstallInitialize and
InstallFinalize. These immediate actions run right away and cannot modify
the system. Their purpose is to make decisions about what needs to happen.
They generate an "install script" (the Teal Region) that will be executed
later on by deferred actions. Those InstallSqlData log lines come from an
immediate action running in the Generate phase. That immediate action gets
the SQL scripts to execute writes them to something called
"CustomActionData" that will be read later by an appropriate deferred
action.

The "Commit Server" line in the Orange Region is when deferred actions can
be executed. These deferred actions can change the system but they do
*NOT* have access to almost anything in the MSI database. They need to be
told what to do through "CustomActionData".

In the context of the WiX custom actions for running SQL scripts, all this
means that all the SQL scripts must be written to "CustomActionData"
before they can be executed later. Remember that the actions are not
"called" once per row in the table. Rather the action is called once and
the action processes everything in one or more tables (for immediate
actions) or processes everything in CustomActionData (for deferred or
rollaback actions).

Note this is not a limitation of the WiX implementation of its custom
actions but rather a restriction imposed by the Windows Installer engine.
If you want different behavior, you'll need to implement it yourself
within the same restrictions. Specifically, you could write your own
custom action to execute a large SQL script that you have installed to the
file system.
You can then read that script *only* when you need it rather than
depending on all scripts being read into memory. The downside is you are
installing the script to the file system. That may or may not be a bad
thing in your case.

--
Edwin G. Castro


On Fri, May 11, 2018 at 12:52 AM, Ven H <venh.123 at gmail.com> wrote:

> Thanks a lot Edwin. For the rollback part, as you mentioned, I 
> scheduled them before execute. So, now, it seems to be getting 
> executed during rollback. For the other scripts, I can see in the log 
> that for every SQL script file, it is logging the following message
>
> *InstallSqlData:  Scheduling SQL string: <SQL statement>*
>
> It is logging the above for all the scripts, but upon refreshing the 
> databases in the SQL Management Studio, I don't see any objects 
> created yet. So, looking at the log, it seems it is only scheduling 
> all the scripts first, before it even starts to execute them. Is there 
> a way, we can control this behavior, i.e., execute each and every 
> script as per the sequence, before just scheduling all at once?
>
> Regards,
> Venkatesh
>
> On Wed, May 9, 2018 at 7:00 PM, Edwin Castro <egcastr at gmail.com> wrote:
>
>> Since you have not provided a verbose log for us to see what is 
>> actually is going on (always get a verbose log, always) the best I 
>> can do is make a guess.
>>
>> In the past I sent email about how immediate custom actions decide 
>> what to do and schedule rollback and "execute" deferred custom 
>> actions to run later by the server process when the install script is
executed.
>>
>> It might be possible that the immediate custom action is failing with 
>> the "out of memory" error you are seeing. In that case none of the 
>> actions would actually run.
>>
>> But your question implies that at least some of the "execute" 
>> deferred custom actions are running so one would expect some of the 
>> rollback custom actions to run. Of course those rollback actions will 
>> only run if they are sequenced before their "execute" counterparts. 
>> Perhaps the rollback you expect is not happening because it is not
scheduled correctly.
>>
>> Then again, you are running out of memory. Perhaps it *IS* trying to 
>> run the rollback but it can't because it ran out of memory!
>>
>> Really, the only thing we can do here is make guesses without the 
>> verbose log. What does the verbose log say?
>>
>> --
>> Edwin G. Castro
>>
>>
>>
>>
>> On Wed, May 9, 2018, 05:52 Ven H via wix-users < 
>> wix-users at lists.wixtoolset.org> wrote:
>>
>>> I have an MSI which has lots of sql script files (around 1500) to 
>>> create around 8 databases and tables, stored procs, views, indexes, 
>>> functions and so on. These sql script files are executed in a 
>>> sequence. Almost towards the end, there is a very big file in the 
>>> 8th DB (around 138 MB). This file basically has insert statements 
>>> with binary data.
>>>
>>> When I install the MSI, I expect that it would create all the 8 DBs 
>>> but will fail only when it reaches this script. But it gets stuck 
>>> for a very long time, before it finally throws the error saying 
>>> "Failed to allocate memory to CustomActionData string.". But when I 
>>> check the log, I can see that it is logging all the sql statements 
>>> from the previous scripts before this file.
>>>
>>> Although, there is a rollback script to drop everything in case of 
>>> error, I thought, when I refresh the SQL Management Studio, I would 
>>> see the databases during installation, but I don't. So, does it try 
>>> to load everything into memory first before starting to execute? If 
>>> not, why does it not create the databases during installation and 
>>> then rollback after error? Can anyone please provide any inputs?
>>>
>>> ____________________________________________________________________
>>> WiX Toolset Users Mailing List provided by FireGiant 
>>> http://www.firegiant.com/
>>>
>>
>

____________________________________________________________________
WiX Toolset Users Mailing List provided by FireGiant
http://www.firegiant.com/