Fails to document PowerShell 6 (PowerShell Core) binary modules

[jherby2k]

I am developing a powershell library using PowerShell core (via the PowerShellStandard.Library nuget package currently at https://powershell.myget.org/F/powershell-core). The library targets .NET standard 2.0, which will be typical for PowerShell modules going forward as it allows your module to run on PowerShell 3+ as well as PowerShell core cross-platform.

Anyway, xmldoc2cmdletdoc fails with the following trace:
Loader exception: System.IO.FileNotFoundException: Could not load file or assembly '<project path>\bin\Debug\netstandard2.0\netstandard.dll' or one of its dependencies. The system cannot find the file specified.

This is on Windows 10 1703. I actually think the issue might be solved by .NET 4.7.1 as its supposed to resolve some binding issues with netstandard libraries. I'll update this case later today or tomorrow as Windows 10 1709 (including .NET 4.7.1) comes out today.

If its fixed, it might just be a good idea to note the dependency on .NET 4.7.1 since this will be a common scenario once PowerShell 6.0 is released.

[jherby2k]

OK so the good news is that it works now with .NET 4.7.1, but if you want this tool to work for PowerShell core modules on dev machines with older frameworks, I believe it can be fixed by having xmldoc2cmdletdoc reference the netstandard20 library.

Long term I'd recommend porting this tool to .net core. Not only is it the standard for PowerShell 6, it should allow your utility to work when building on Linux and Mac.

[johncrim]

I believe this is also the same issue when targeting newer .NET Core runtimes (.NET Core 3.0 in my case) - loaded dependencies fail because the dependencies can't be resolved when the assembly is loaded. I'm getting (with fuslogvw on to view assembly loading issues):

λ  dotnet build
Microsoft (R) Build Engine version 16.0.462+g62fb89029d for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 21.53 ms for C:\some-project\src\SomeProject\SomeProject.csproj.
C:\Program Files\dotnet\sdk\3.0.100-preview5-011568\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(157,5): message NETSDK1057: You are using a preview version of .NET Core. See: https://aka.ms/dotnet-core-preview [C:\some-project\src\SomeProject\SomeProject.csproj]
  SomeProject -> C:\some-project\src\SomeProject\bin\Debug\netcoreapp3.0\Ease.Operations.SomeProject.dll
  AssemblyPath: C:\some-project\src\SomeProject\bin\Debug\netcoreapp3.0\Ease.Operations.SomeProject.dll, OutputHelpFilePath: C:\some-project\src\SomeProject\bin\Debug\netcoreapp3.0\Ease.Operations.SomeProject.dll-Help.xml, TreatWarningsAsErrors False
  System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.
     at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
     at System.Reflection.Assembly.GetTypes()
     at XmlDoc2CmdletDoc.Core.Engine.GetCommands(Assembly assembly)
     at XmlDoc2CmdletDoc.Core.Engine.GenerateHelp(Options options)
  Loader exception: System.IO.FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
  File name: 'System.Runtime, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' ---> System.IO.FileNotFoundException: Could not load file or assembly 'file:///C:\some-project\src\SomeProject\bin\Debug\netcoreapp3.0\System.Runtime.dll' or one of its dependencies. The system cannot find the file specified.
  File name: 'file:///C:\some-project\src\SomeProject\bin\Debug\netcoreapp3.0\System.Runtime.dll'
     at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
     at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
     at System.Reflection.RuntimeAssembly.InternalLoadFrom(String assemblyFile, Evidence securityEvidence, Byte[] hashValue, AssemblyHashAlgorithm hashAlgorithm, Boolean forIntrospection, Boolean suppressSecurityChecks, StackCrawlMark& stackMark)
     at System.Reflection.Assembly.LoadFrom(String assemblyFile)
     at XmlDoc2CmdletDoc.Core.Engine.<>c__DisplayClass7_0.<LoadAssembly>b__0(Object sender, ResolveEventArgs args)
     at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

  === Pre-bind state information ===
  LOG: Where-ref bind. Location = C:\some-project\src\SomeProject\bin\Debug\netcoreapp3.0\System.Runtime.dll
  LOG: Appbase = file:///C:/Users/SomeUser/.nuget/packages/xmldoc2cmdletdoc/0.2.12/tools/
  LOG: Initial PrivatePath = NULL
  Calling assembly : (Unknown).
  ===
  LOG: This bind starts in LoadFrom load context.
  WRN: Native image will not be probed in LoadFrom context. Native image will only be probed in default load context, like with Assembly.Load().
  LOG: No application configuration file found.
  LOG: Using host configuration file:
  LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config.
  LOG: Attempting download of new URL file:///C:/src/ease/Operations/src/SomeProject/bin/Debug/netcoreapp3.0/System.Runtime.dll.

[ChrisLambrou]

We're in the process of supporting PowerShell 6 Core for a whole bunch of Redgate PowerShell products, so this is likely to get worked on shortly. Since we also need to maintain compatibility with PowerShell 5 and earlier, for existing products that use this tool, I'm considering multi-targeting this tool. Probably get the core to target net4xx and netstandard2.x, and provide executables to support net4xx and netcoreapp2.x, as a minimum. Any thoughts?

[johncrim]

Multi-targeting sounds great to me.

I think being able to use it with new versions of PS Core is the key. Developers can write PS Core modules targetting older versions of .NET Core if they want, while building using newer versions of .NET Core. Obviously this is a build time dependency.

If you have an early versions that need testing, I'm happy to try it out.

[CraigChamberlain]

Trying to use with a .net Standard PowerShell module and it causes build to fail. Using nuget package: 0.2.13. Is this likely to be supported in a forthcoming update? Any timeline?

~/.nuget/packages/xmldoc2cmdletdoc/0.2.13/build/XmlDoc2CmdletDoc.targets(28,9): error MSB3073: 
The command ""~/.nuget/packages/xmldoc2cmdletdoc/0.2.13/build/../tools/XmlDoc2CmdletDoc.exe" -excludeParameterSets "" "~/Proj/bin/Debug/netstandard2.0/Proj.dll"" exited with code 4. [~/src/Proj/Proj.csproj]

[arunswarnam]

@CraigChamberlain Is there any work around that you found? I have a netStandard2.0 Powershell Core module and uses 0.2.13 of XmlDoc2Cmdlet Doc. I get the same error code 4 error MSB3073:

[CraigChamberlain]

I haven't tried in a long time. I'll be doing so soon though. Will more than likely just do in c# comments as per original spec and forget the xml.

…

On Thu, Jun 25, 2020, 1:16 AM Arun Swarnam ***@***.***> wrote: @CraigChamberlain <https://github.com/CraigChamberlain> Is there any work around that you found? I have a netStandard2.0 Powershell Core module and uses 0.2.13 of XmlDoc2Cmdlet Doc. I get the same error code 4 error MSB3073: — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALRDEQY7SQUUGW7Q5FS3XJLRYKJMFANCNFSM4D7Q6PGQ> .

[gigi81]

I have ported the project to dotnet standard in my fork here: https://github.com/gigi81/XmlDoc2CmdletDoc/tree/feature/dotnetcore

Would appreciate some feedback before having a PR.
Feel free to have a PR to my fork if you find any issue.

Example of how to use/test:
nuget.config (place file in the same folder as your solution file)

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
    <add key="jolt" value="https://luigigrilli.pkgs.visualstudio.com/Jolt/_packaging/jolt/nuget/v3/index.json" />
  </packageSources>
</configuration>

Example csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <GenerateDocumentationFile>true</GenerateDocumentationFile>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="PowerShellStandard.Library" Version="5.1.0" PrivateAssets="All" />
    <PackageReference Include="XmlDoc2CmdletDoc" Version="1.0.0.4" />
  </ItemGroup>

</Project>

[ChrisLambrou]

I have ported the project to dotnet standard in my fork here:
https://github.com/gigi81/XmlDoc2CmdletDoc/tree/feature/dotnetcore

Would appreciate some feedback before having a PR.
Feel free to have a PR to my fork if you find any issue.

Ah, I'm so sorry I didn't spot this earlier. I've been working on an off to port this tool to .NET Standard and .NET Core. I've just published a preview package for this, too (https://www.nuget.org/packages/XmlDoc2CmdletDoc/0.4.0-dotnetcore0000). I'd be interested to see what approach you've taken to supporting .NET Standard and .NET Core. It was easy to get the core project to target .NET Standard, but I needed to generate framework-specific executables in order for it to be useful to plug into msbuild. Also, we ran into a bunch of compatibility issues, so ended up releasing specific suport for a whole bunch of target frameworks. We're testing it internally at Redgate, but if you want to give it a try, we'd love to know how you get on with it.

[gigi81]

Hi @ChrisLambrou
I briefly tested your preview package and it works as well as mine I guess. I don't have a library of projects to test it as you might have. I think the best way to find out problems is to put it out there as a preview as you already did to get some feedback.

I had a quick look at the internals of your nuget package and I'm a not sure as to why you need so many versions of the same tool.
For the msbuild part, I followed this very well structured and detailed guide here. You can read all the details there but in short, there are only 2 versions of the tool needed, one for the net framework, and one for net core.

The main problem I faced in porting this was to actually port "Jolt", which as you know is the key component that reads the XML document from the c# project. I would have really liked to get rid of it, but I couldn't find a modern alternative to it.
You can find my fork of Jolt here https://github.com/gigi81/JoltNet-core

The other change I made was to actually refactor the csproj of the tool to target net45 and netstandard 2 which was quite straightforward as you noted and then the refactoring of the msbuild targets files an nuget package as detailed above.

I think both our packages would use some reduction in size, in particular I don't think the inclusion of the "runtime" folder is needed for the dotnet version, as we are running on a dev box where the sdk is already installed.

[ChrisLambrou]

Yes, I'd definitely like to get the size of the package down. I've targeted different versions of .NET Core for the tooling because I know we have build servers at Redgate with different versions of .NET Core, and the documentation about compatibility between netcoreapp2.1 and netcoreapp3.1, and exactly what is required when you publish self-contained vs non-self-contained projects, is at best merely confusing! As it gets tested internally, I'm hoping I'll be in a position to reign back the number of bundled frameworks and runtimes. For now, I just aimed for something that I hoped would just work for all our internal and external consumers, and then I'll start to clean it up a bit.

[ChrisLambrou]

Also, I've quietly dropped support for net45. It was kinda accidental - as I was porting our Jolt.NET fork, I lost track of which frameworks I should support. Are people still writing binary PowerShell modules that target net45?

[gigi81]

In my fork, I actually built only the net45 version. Any net framework including and after 4.5 will work with it (if you have 4.6 4.7 or 4.8 it will work, no need to install 4.5). I agree nobody uses net45 anymore but it is more to make it as compatible as possible and also I don't think there is any advantage in using a more recent version.

[ChrisLambrou]

Yeah, I'm also looking into just supporting net45 and netcoreapp2.1, mainly as a way to keep the package size down. I'd like to do some compatibility testing first, though, across multiple versions of some of the referenced packages. To be honest, if it's a trade off between reliability, compatibility and minimal package size, package size loses out every time. The initial download might be annoying for consumers, but once it's cached, the file size is essentially irrelevant compared to the number and size of files that are copied around by dotnet build operations.