aspnetcore icon indicating copy to clipboard operation
aspnetcore copied to clipboard

Packaging of static assets of razor libraries is different in Githhub Actions from Local Computer

Open rbuergi opened this issue 11 months ago • 10 comments

I had already asked the following issue: https://github.com/dotnet/aspnetcore/issues/60680. You have kindly explained to me how to configure the razor libs to create proper nuget packages. This setup works perfectly when I build the packages locally. I can also use them in a Blazor project, and all the js files get added to the staticwebassets folder.

However, when I try to build the razor library with the exact same commands on github actions under linux, I get warnings for the javascript files that

Warning: /usr/share/dotnet/sdk/9.0.202/Sdks/NuGet.Build.Tasks.Pack/build/NuGet.Build.Tasks.Pack.targets(221,5): warning NU5118: File '/home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Blazor/wwwroot/arc--U5YtJTU.js' is not added because the package already contains file 'contentFiles/any/net9.0/wwwroot/arc--U5YtJTU.js' [/home/runner/work/MeshWeaver/MeshWeaver/src/MeshWeaver.Blazor/MeshWeaver.Blazor.csproj]

this pattern repeats for all javascript files.

In the published package, the js files get correctly added under contentFiles/any/net9.0/wwwroot, but they are missing from staticwebassets.

I have executed the exact same commands from the github action on my local machine, and the packages build fine. What's the difference between running the command on my local machine versus github actions? I really don't know how to advance on this; I wouldn't like to publish packages from my local PC....

rbuergi avatar Mar 24 '25 17:03 rbuergi

@rbuergi thanks for contacting us.

Capture an msbuild binlog and check the casing on the paths in the binlog.

javiercn avatar Mar 25 '25 13:03 javiercn

Hi @javiercn. Thanks for resonding so quickly. This is the entire log from github actions. At least judging on my windows setup, the casings seem to be correct... However, I wonder why I get all the duplicate messages which I don't get on Windows.

job-logs.txt

rbuergi avatar Mar 25 '25 18:03 rbuergi

@rbuergi thanks for the additional details.

Unfortunately, the logs we need are the MSBuild binlogs generated with dotnet build /bl.

Alternatively, we would suggest you either create a new class library and start adding files until you are able to trigger the issue and provide a minimal repro.

javiercn avatar Mar 26 '25 09:03 javiercn

@javiercn I am sorry I didn't understand. I have now produced the binlog for the pack process. It took me a while, I didn't even know this exists.

I guess this is what I need to do? If not, I am sorry. I know now how to produce this on github, so can also add the one for build.

Thanks a lot for taking care.

msbuild-binlog.zip

rbuergi avatar Mar 27 '25 14:03 rbuergi

I am sorry to ask again, but I am kind of stuck releasing my packages. For me it is all a lot of voodoo, where the js files are supposed to be, what I need to put in csproj, etc. I have googled and asked many LLMs but there seems to be no documentation whatsoever about how to package js files in nuget packages for Razor libs with the new net9.0 static assets feature. Can you please at least give me a hint at how to debug?

rbuergi avatar Mar 31 '25 08:03 rbuergi

Thank you very much @MackinnonBuck for assigning this to yourself. Anything you have discovered? Reverting to my question, is there any form of documentation available how settings in the project affect the nuget package and which kind of layout is needed in nuget for static assets to work?

rbuergi avatar Apr 05 '25 11:04 rbuergi

Hi @rbuergi,

Thanks for providing the binlog! It seems this line is causing each asset to be included twice:

<None Include="wwwroot\**\*" Pack="true" PackagePath="contentFiles/any/net9.0/wwwroot/" />

(once as a Content item, and once as a None item)

I see this was added in response to a comment here. Perhaps you could elaborate on the problem you were having in https://github.com/dotnet/aspnetcore/issues/60680?

If you're able to provide us with a minimal repro project hosted on a GitHub repository, it would help us narrow down the root cause of the issue. Thanks!

MackinnonBuck avatar Apr 09 '25 23:04 MackinnonBuck

Thank you so much for responding, @MackinnonBuck. I have the following setup, which leads to this result.

  1. I have a vite build residing in the StaticAssets folder
  2. I use the following StaticAssets.target file:
<Project>
  <Target Name="NpmInstall"
          Inputs="$(MSBuildThisFileDirectory)package.json"
          Outputs="$(BaseIntermediateOutputPath)package-lock.json"
  >
    <Message Importance="high" Text="Running npm install..." />
    <Exec Command="npm install"
          WorkingDirectory="$(MSBuildThisFileDirectory)"
    />
    <Copy SourceFiles="$(MSBuildThisFileDirectory)package-lock.json"
          DestinationFolder="$(BaseIntermediateOutputPath)"
    />
  </Target>

  <ItemGroup>
    <ViteBuildInputs Include="$(MSBuildThisFileDirectory)**"
                     Exclude="$(MSBuildThisFileDirectory)node_modules\**"
    />
    <UpToDateCheckInput Include="@(ViteBuildInputs)"
    />
  </ItemGroup>

  <Target Name="ViteBuild"
          DependsOnTargets="NpmInstall"
          BeforeTargets="CoreBuild"
          Inputs="@(ViteBuildInputs)"
          Outputs="$(MSBuildThisFileDirectory)..\wwwroot\index.mjs"
  >
    <Message Importance="high" Text="Running vite build..." />
    <Exec Command="npm run build:$(Configuration)"
          WorkingDirectory="$(MSBuildThisFileDirectory)"
    />
    <!-- Validate output -->
    <Error Text="Vite build failed to generate wwwroot content"
           Condition="!Exists('$(MSBuildThisFileDirectory)..\wwwroot')" />
  </Target>
  
</Project>
  1. In the vite config, I copy the outputs:
import { defineConfig } from 'vite';
import { viteStaticCopy } from 'vite-plugin-static-copy';
import commonjs from '@rollup/plugin-commonjs';

export default defineConfig({
    plugins: [
        viteStaticCopy({
            targets: [
                {
                    src: 'node_modules/highlight.js/styles/*.min.css',
                    dest: './css/',
                    rename: (name, extension) => `${name}.${extension}`  // Preserve the full filename
                },
                {
                    src: 'node_modules/@primer/css/dist/markdown.css',
                    dest: './css/',
                    rename: (name, extension) => `${name}.${extension}`  // Preserve the full filename
                },
                {
                    src: 'node_modules/mermaid/dist/mermaid.min.js',
                    dest: './'
                },
                {
                    src: 'node_modules/mathjax-full/es5/**/*',
                    dest: './mathjax/'
                }
            ],
            structured: false  // This will prevent maintaining folder structure
        }),
        commonjs({
            include: [/mathjax-full/],
            transformMixedEsModules: true,
            requireReturnsDefault: 'auto'
        })
    ],
    build: {
        emptyOutDir: true,
        minify: false,
        lib: {
            entry: [
                'highlight.ts',
                'mermaid.ts',
                'mathjax.ts'
            ],
            formats: ['es']
        },
        outDir: '../wwwroot',
        rollupOptions: {
            external: [/^mathjax-full\/es5\//]
        }
    },
    resolve: {
        alias: {
            'mathjax-full': 'mathjax-full/es5'
        }
    }
})
  1. In the csproj, I have put
        <Content Update="StaticAssets\**" Pack="false" CopyToPublishDirectory="false" />
        <None Include="wwwroot\**\*" Pack="true" PackagePath="contentFiles/any/net9.0/wwwroot/" />

The first line was required to be able to publish to ACA, as it also complained I have duplicates. Then I did not get any static assets in the nuget, and hence I created the referenced ticket which told me to add the second line, which creates the static assets at least when building locally.

I am really sorry I have such a mess, I am not used to javascript builds, especially in conjunction with this new blazor feature. In Blazor 8, this setup had worked. Please let me know what I should change. Again, thank you very much for your support.

rbuergi avatar Apr 10 '25 12:04 rbuergi

@MackinnonBuck any update? I am still stuck building packages from github actions. Please note that the local package process is fine, it is related to github actions / linux. If it helps: it is an open source project [MeshWeaver](https://github.com/Systemorph/MeshWeaver. All the sources can be seen there.

rbuergi avatar Apr 30 '25 07:04 rbuergi

Hi @rbuergi, so sorry for the delayed response.

It's a little hard to gather what's happening from the repo due to its size, but I would suggest defining static web assets using the DefineStaticWebAssets and DefineStaticWebAssetEndpoints tasks, as this will allow the static web assets SDK to handle the details. An example of this can be seen in our CustomElements package, which includes dynamically-built JavaScript files as static web assets.

Hope this helps - let me know if there's anything else I can help clarify.

MackinnonBuck avatar Jun 13 '25 13:06 MackinnonBuck

Hi @MackinnonBuck

Thank you for getting back to me. I have now tried a more-or-less vanilla setup, where I include only files which should be packaged by default. I have a number of MyComponent.razor.css and MyComponent.razor.js files.

when i build packages locally on Windows, they are perfect. But building them on github actions using linux, the static assets are missing from the razor libraries.

I am really not sure why github actions should behave differently from my local builds. Any ideas?

Thanks, Roland.

rbuergi avatar Jun 25 '25 09:06 rbuergi

Thanks for the reply, @rbuergi.

I am really not sure why github actions should behave differently from my local builds. Any ideas?

I'm not sure why it would behave differently unless there's some kind of path casing inconsistency, but I didn't see anything like that from looking at your binlog.

I appreciate you providing all the information you have so far - large projects like this can be difficult for us to investigate since it's not always clear whether it's an application or framework issue. If you find time to create a minimal repro project (without external tools like Vite) that shows the same problem, that will allow us to investigate the problem further, and if we conclude it's a problem with the app's build configuration, we may be able to provide a potential solution. Otherwise, I'm afraid we may not be able to investigate this problem further. If this issue auto-closes at some point due to the lack of a repro, feel free to open a new one if you're able to construct a minimal repro. Thanks again.

MackinnonBuck avatar Jun 25 '25 18:06 MackinnonBuck

Hi @MackinnonBuck. I have cleaned out all my csproj files that they are basically empty. See e.g. https://github.com/Systemorph/MeshWeaver/blob/main/src/MeshWeaver.Blazor/MeshWeaver.Blazor.csproj. I am using mostly the pre-packaged setup

MyComponent.razor.css
MyComponent.razor.js

These components should, as far as I understand the razor lib packaging, be included automatically in static assets. This works

  • If I include the project as project reference
  • If I build the pack (using vanilla dotnet pack) on my local Windows and include nuget

It does not work when I build this on github actions with the exact same dotnet pack command. Of course we can delete all the components there except one; I don't see what it would change, though, there is literally no complexity left as I have even removed all the vite builds.

rbuergi avatar Jun 27 '25 18:06 rbuergi

Hi @rbuergi,

Could you please clarify what exact repro steps are needed to reproduce the problem? Which project should be run against dotnet pack? Does this only repro on .NET SDK 9.0.202? Does the issue only reproduce on GitHub Actions, or does it occur on other Linux environments as well?

I tried running dotnet pack on the MeshWeaver.Blazor.csproj project in Ubuntu 24.04 and .NET SDK 9.0.108 and didn't observe the issue. Please let me know what steps I'm missing. A repro that doesn't rely on GitHub Actions would be ideal. Thanks!

MackinnonBuck avatar Jul 10 '25 18:07 MackinnonBuck

Hi @MackinnonBuck

When you say that you couldn't reproduce in Ubuntu, I guess you are referring to using WSL. This uses still a FAT or an NTFS file system, which is case insensitive. As already @javiercn suspected, I suspect as well that it is a casing issue but one buried within the packing. When working with several IDEs in parallel, i got exceptions to have duplicate files, once packaged with C:\somepath and once with c:\somepath My guess is that when working with a file system which is essentially case insensitive, you don't notice this. Github Actions will use a true ext file system which is case sensitive and thus the packaging stops working. So in order to repro locally, you'd probably have to have an ext file system. I haven't tried it, though, I have to admit, but have a strong feeling that this is the point.

Thanks for taking care!

rbuergi avatar Jul 13 '25 11:07 rbuergi