[wix-users] Error starting custom action

Thu Sep 6 04:14:19 PDT 2018

I figured as much. 22 years or writing installers and antivirus/end point protection agents continue to be a thorn.

I don't know that there is much WiX can do to make this a better error.  RunDLL is used to get the managed code to run in a different process so that the original msiexec sandbox process isn't tattoo'd with the CLR version of the CA.    The error seems to be in RunDLL and all DTF can know if that it returned a -1.

Still, feel free to file an issue against DTF and maybe someone more familiar with the code can come up with a better answer.

________________________________
From: Kirti Bauer <kirti.bauer at ch.abb.com>
Sent: Thursday, September 6, 2018 1:38 AM
To: Christopher Painter; WiX Toolset Users Mailing List
Subject: RE: Error starting custom action

The problem has been resolved.

An “advanced endpoint protection” program called Traps (from PaloAlto) prevented the installation from loading a DLL that was copied to the temp directory.

After deactivating this restriction, the installation succeed without errors.

The error message from the setup log was not helpful in finding this problem. (SFXCA: RUNDLL32 returned error code: -1).

Would it be possible to output a more verbose error message?

Thanks for your help. I appreciate the time you took to reply to my messages.

From: Christopher Painter [mailto:chrpai at iswix.com]
Sent: Mittwoch, 5. September 2018 16:29
To: Kirti Bauer <kirti.bauer at ch.abb.com>; WiX Toolset Users Mailing List <wix-users at lists.wixtoolset.org>
Subject: Re: Error starting custom action

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Did you run dependency walker on the bad machine also?

Are you able to repro this in house or just the one server?

________________________________

From: Kirti Bauer <kirti.bauer at ch.abb.com<mailto:kirti.bauer at ch.abb.com>>
Sent: Wednesday, September 5, 2018 6:32 AM
To: Christopher Painter; WiX Toolset Users Mailing List
Subject: RE: Error starting custom action

The .CA.dll depends only on standard windows references.

I used dependency walker to on the .CA.dll and it found some missing dependencies (see attached text file). I don’t know if this could be a problem.

The only difference that I can see between the working Windows 2016 Server and the this one is that the current machine (where it is not working) is a German installation.

This hasn’t been a problem so far on other servers.

We have put in a request to the customer to disable anti-virus and group policy settings.

The user is an administrator and has all admin rights.

I have not tried a deferred custom action yet. I will have to look at an example to see how this is done.

Just as a note, I am using Visual Studio 2017  Wix Toolset Extension to create the package.

From: Christopher Painter [mailto:chrpai at iswix.com]
Sent: Mittwoch, 5. September 2018 13:11
To: Kirti Bauer <kirti.bauer at ch.abb.com<mailto:kirti.bauer at ch.abb.com>>; WiX Toolset Users Mailing List <wix-users at lists.wixtoolset.org<mailto:wix-users at lists.wixtoolset.org>>
Subject: Re: Error starting custom action

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

I didn't catch that you were saying it worked on other machines.

I'm not sure what this is then.  Perhaps you could look at the .CA.dll in depends for any missing dependencies (shouldn't be but just to rule it out) .  I don't think you need to verify the .net framework installation because it doesn't seem to have gotten that far.

Anything strange about this machine?  Perhaps some antivirus/security end point software?  Any special group policy settings?    Does it work for another user account on the machine?   Do deferred customactions with no impersonation work on this machine?

________________________________

From: Kirti Bauer <kirti.bauer at ch.abb.com<mailto:kirti.bauer at ch.abb.com>>
Sent: Wednesday, September 5, 2018 1:23 AM
To: WiX Toolset Users Mailing List
Cc: Christopher Painter
Subject: RE: Error starting custom action

I am attaching my entire solution for your reference.

I have checked the points you indicated and they seem to be correct.

As stated before, the setup runs without errors on other machines.

Thanks in advance for your help.

Here is the requested wxs code:

<?xml version="1.0" encoding="UTF-8"?>

<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fschemas.microsoft.com%2Fwix%2F2006%2Fwi&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209164799&sdata=ISoZMEba7WxPEHPWxgC7I0Z0ypCeiGbz0Msi4XHhOo4%3D&reserved=0>">

<Product Id="*" Name="MyWixSetup" Language="1033" Version="1.0.0.0" Manufacturer="Microsoft" UpgradeCode="9c8cc017-8812-4265-91f6-8a66df16ee81">

<Package InstallerVersion="200" Compressed="yes" InstallScope="perMachine" />

<MajorUpgrade DowngradeErrorMessage="A newer version of [ProductName] is already installed." />

<MediaTemplate />

<Feature Id="ProductFeature" Title="MyWixSetup" Level="1">

<ComponentGroupRef Id="ProductComponents" />

</Feature>

</Product>

<Fragment>

<Directory Id="TARGETDIR" Name="SourceDir">

<Directory Id="ProgramFilesFolder">

<Directory Id="INSTALLFOLDER" Name="MyWixSetup" />

</Directory>

</Directory>

</Fragment>

<Fragment>

<Binary Id="MyCustomActions" SourceFile="$(var.MyCA.TargetDir)$(var.MyCA.TargetName).CA.dll" />

<CustomAction Id="Test" BinaryKey="MyCustomActions" DllEntry="Test" />

<InstallUISequence>

<Custom Action="Test" After="CostFinalize"/>

</InstallUISequence>

<ComponentGroup Id="ProductComponents" Directory="INSTALLFOLDER">

</ComponentGroup>

</Fragment>

</Wix>

From: Christopher Painter [mailto:chrpai at iswix.com]
Sent: Dienstag, 4. September 2018 17:26
To: WiX Toolset Users Mailing List <wix-users at lists.wixtoolset.org<mailto:wix-users at lists.wixtoolset.org>>
Cc: Kirti Bauer <kirti.bauer at ch.abb.com<mailto:kirti.bauer at ch.abb.com>>
Subject: Re: Error starting custom action

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.

Can you show your wxs  code that consumes said dll?

This is my list of usual suspects:  (Your provided code rules out #'s  4-9 but not 1-3 or 10.)

https://stackoverflow.com/questions/3560370/custom-action-in-c-sharp-used-via-wix-fails-with-error-1154/3566822#3566822<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F3560370%2Fcustom-action-in-c-sharp-used-via-wix-fails-with-error-1154%2F3566822%233566822&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209164799&sdata=KmdwXQzvcBif7YMh%2FfW2NlaLmkdUrmmqi5CW4yDddUI%3D&reserved=0>

[https://cdn.sstatic.net/Sites/stackoverflow/img/apple-touch-icon@2.png?v=73d79a89bded]<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F3560370%2Fcustom-action-in-c-sharp-used-via-wix-fails-with-error-1154%2F3566822%233566822&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209321050&sdata=lVDwmeZVrO5exeu0c3K6kSGcNOxKUDjsQ9OUsKfUvCg%3D&reserved=0>

Custom Action in C# used via WiX fails with error 1154 ...<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F3560370%2Fcustom-action-in-c-sharp-used-via-wix-fails-with-error-1154%2F3566822%233566822&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209321050&sdata=lVDwmeZVrO5exeu0c3K6kSGcNOxKUDjsQ9OUsKfUvCg%3D&reserved=0>

stackoverflow.com

I am using WiX 3.5.1930 in Visual Studio 2010, targeting the .NET Framework 3.5. (Later weekly builds of WiX seem to be very broken with respect to their custom action template, at least for now. ...

________________________________

From: wix-users <wix-users-bounces at lists.wixtoolset.org<mailto:wix-users-bounces at lists.wixtoolset.org>> on behalf of Kirti Bauer via wix-users <wix-users at lists.wixtoolset.org<mailto:wix-users at lists.wixtoolset.org>>
Sent: Tuesday, September 4, 2018 4:08 AM
To: wix-users at lists.wixtoolset.org<mailto:wix-users at lists.wixtoolset.org>
Cc: Kirti Bauer
Subject: [wix-users] Error starting custom action

I had an installation created by using WIX 3.11 in Visual Studio 2017. Until now I have had no major issues with the installation and the custom actions library.
My current installation site is a Windows 2016 Server running .NET Framework v4.0.30319.

My MSI works until the first access to a custom action.

I created a simple custom action function that presents the user with a dialog box and returns the success result.
[CustomAction]
public static ActionResult Test(Session session)
{
session.Log("Test started");
MessageBox.Show("Test");
session.Log("Test ended");
return ActionResult.Success;
}

Using verbose logging I get the following error:

MSI (c) (50:B4) [16:33:41:039]: Doing action: Test
Action 16:33:41: Test.
Action start 16:33:41: Test.
MSI (c) (50:38) [16:33:41:039]: Invoking remote custom action. DLL: C:\Users\SVC_AB~1\AppData\Local\Temp\2\MSI1E2A.tmp, Entrypoint: Test
MSI (c) (50:E4) [16:33:41:055]: Cloaking enabled.
MSI (c) (50:E4) [16:33:41:055]: Attempting to enable all disabled privileges before calling Install on Server
MSI (c) (50:E4) [16:33:41:055]: Connected to service for CA interface.
SFXCA: RUNDLL32 returned error code: -1
CustomAction Test returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)
Action ended 16:33:44: Test. Return value 3.
MSI (c) (50:B4) [16:33:44:717]: Doing action: FatalError
Action 16:33:44: FatalError.
Action start 16:33:44: FatalError.
Action 16:33:44: FatalError. Dialog created
MSI (c) (50:64) [16:33:44:748]: Note: 1: 2731 2: 0
Action ended 16:33:46: FatalError. Return value 2.
Action ended 16:33:46: INSTALL. Return value 3.

What does the error code -1 mean from the rundll32?
I have searched for this error but have found no results.

Any help would be appreciated.

BR
kibau

____________________________________________________________________
WiX Toolset Users Mailing List provided by FireGiant http://www.firegiant.com/<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.firegiant.com%2F&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209321050&sdata=nKr4PQRDoM2dIjqZ8YQtYck%2FL8NcNXxSYQ2qROzTGsM%3D&reserved=0>

WiX Support | WiX Experts and Resources from FireGiant<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.firegiant.com%2F&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209321050&sdata=nKr4PQRDoM2dIjqZ8YQtYck%2FL8NcNXxSYQ2qROzTGsM%3D&reserved=0>

www.firegiant.com<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.firegiant.com&data=02%7C01%7Ckirti.bauer%40ch.abb.com%7C927edb86f83d49d11d5408d6133bdeed%7C372ee9e09ce04033a64ac07073a91ecd%7C0%7C0%7C636717545209321050&sdata=4er0hqSZLYMIVo6NMUKG3f0Mp2Cvo54tD5inSGLB084%3D&reserved=0>

WiX Support | Installation, Development, Deployment | WiX Experts and Resources from FireGiant