docfx icon indicating copy to clipboard operation
docfx copied to clipboard
verified publisher icon dotnet

[Bug] Solutions with code generator projects fail to build

Open AptiviCEO opened this issue 1 year ago • 14 comments

Describe the bug The projects in the solution with the code generators cause DocFX to fail to generate documentation. After updating Microsoft.CodeAnalysis.CSharp from 4.8.0 to 4.9.2, DocFX began to fail to generate documentation if there were projects that referenced a namespace generated by the code generator.

To Reproduce Steps to reproduce the behavior:

  1. Clone the Terminaux project
  2. Install DocFX
  3. Execute the docfx command, pointing it to the DocGen/docfx.json file: docfx DocGen/docfx.json

Expected behavior DocFX successfully recognizes all generated code and builds the documentation.

Context (please complete the following information):

  • OS: Ubuntu in GitHub Actions
  • Docfx version: 2.75.2

Additional context The failed build log is here and the docfx.json is here. It worked fine before upgrading Microsoft.CodeAnalysis.CSharp from 4.8.0 to 4.9.2 with all the generated code, such as this.

It looks like that the code generators – Terminaux.ColorDataGen, Terminaux.SequenceTypesGen, and Terminaux.TermInfoGen – didn't generate anything during DocFX's build stage, hence the error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?) errors.

Previously, all code generators worked perfectly fine as in this log.

AptiviCEO avatar Mar 09 '24 06:03 AptiviCEO

I built it by adding the CompilerGeneratedFilesOutputPath property, and the instructions https://github.com/dotnet/docfx/issues/9619

  <PropertyGroup>
    <AssemblyTitle>Terminaux</AssemblyTitle>
    <AssemblyDescription>Console Manipulation Library</AssemblyDescription>
    <Description>A C# library that allows you to manipulate with your console, like colors, VT sequences, and more.</Description>
    <Version>3.1.5</Version>
    <Authors>Aptivi</Authors>
    <TargetFramework>netstandard2.0</TargetFramework>
    <DebugType>portable</DebugType>
    <AssemblyName>Terminaux</AssemblyName>
    <PackageId>Terminaux</PackageId>
    <PackageTags>console;shell;readline;gnu;cui;tui;cli;manipulation;console;vt;sequences;color;vt100;truecolor;library;csharp;reader;input;writer;styles;formatter</PackageTags>
    <PackageProjectUrl>https://github.com/Aptivi/Terminaux</PackageProjectUrl>
    <RepositoryType>git</RepositoryType>
    <RepositoryUrl>https://github.com/Aptivi/Terminaux</RepositoryUrl>
    <GenerateDocumentationFile>True</GenerateDocumentationFile>
    <GeneratePackageOnBuild>True</GeneratePackageOnBuild>
    <IncludeSymbols>True</IncludeSymbols>
    <SymbolPackageFormat>snupkg</SymbolPackageFormat>
    <Company>Aptivi</Company>
    <Copyright>Copyright (c) 2022-2023 Aptivi.</Copyright>
    <PackageReadmeFile>README.md</PackageReadmeFile>
    <PackageLicenseExpression>GPL-3.0-or-later</PackageLicenseExpression>
    <PackageRequireLicenseAcceptance>True</PackageRequireLicenseAcceptance>
    <PackageIcon>OfficialAppIcon-Terminaux-512.png</PackageIcon>
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
    <EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>
    <SignAssembly>True</SignAssembly>
    <AssemblyOriginatorKeyFile>..\aptivi_snk.snk</AssemblyOriginatorKeyFile>
    <CompilerGeneratedFilesOutputPath>Generated</CompilerGeneratedFilesOutputPath>
  </PropertyGroup>

image

NOTE! Also, consider moving your documentation to the new format and use the current Github workflow.

paulushub avatar Mar 09 '24 07:03 paulushub

Thanks! Your suggestions worked! I also had to start installing DocFX manually in the workflow instead of using Docker to do the same thing. Changes here.

AptiviCEO avatar Mar 09 '24 08:03 AptiviCEO

I think you will also need

    <ItemGroup>
        <Compile Remove="$(CompilerGeneratedFilesOutputPath)/**/*.cs" />
    </ItemGroup>

and add the Generated folder to gitignore if used as above to prevent generated codes being added to the repository.

Reference (work that prompted the https://github.com/dotnet/docfx/issues/9619): Project File: https://github.com/paulushub/SVG/blob/master/Source/Svg.csproj Workflow: https://github.com/paulushub/SVG/blob/master/.github/workflows/builddocs.yml Output: https://paulushub.github.io/SVG/

paulushub avatar Mar 09 '24 09:03 paulushub

I've added the above ItemGroup to the project file and added the Generated folder to .gitignore. Thanks!

However, the documentation now fails to build again with this:

2024-03-09T09:43:27.2279941Z ##[group]Run docfx DocGen/docfx.json
docfx DocGen/docfx.json
shell: /usr/bin/bash -e {0}
env:
  DOTNET_ROOT: /usr/share/dotnet
Using .NET Core SDK 8.0.201
Loading project /home/runner/work/Terminaux/Terminaux/Terminaux/Terminaux.csproj
  Determining projects to restore...
  All projects are up-to-date for restore.
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Base/Buffered/ScreenPart.cs(24,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Base/Extensions/ConsoleClearing.cs(23,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Base/Extensions/ConsolePositioning.cs(32,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Interactive/InteractiveTuiTools.cs(34,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Presentation/PresentationTools.cs(28,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/ColorSelector.cs(32,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxButtonsColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxColor.cs(34,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxInputColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxInputPasswordColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxProgressColor.cs(32,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxSelectionColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Infobox/InfoBoxSelectionMultipleColor.cs(33,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Selection/SelectionMultipleStyle.cs(35,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Inputs/Styles/Selection/SelectionStyle.cs(35,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/ConsoleWriters/TextWriterWhereColor.cs(28,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/BoxColor.cs(25,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/BoxFrameColor.cs(28,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/ProgressBarColor.cs(29,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
error: /home/runner/work/Terminaux/Terminaux/Terminaux/Writer/FancyWriters/ProgressBarVerticalColor.cs(29,35): error CS0234: The type or namespace name 'Types' does not exist in the namespace 'Terminaux.Sequences.Builder' (are you missing an assembly reference?)
Creating output...
warning: No .NET API detected for .
Searching custom plugins in directory /home/runner/.dotnet/tools/.store/docfx/2.75.2/docfx/2.75.2/tools/net8.0/any/...
Post processor ExtractSearchIndex loaded.
Post processor SitemapGenerator loaded.
No files are found with glob pattern images/**, excluding <none>, under directory "/home/runner/work/Terminaux/Terminaux/DocGen"
7 plug-in(s) loaded.
Building 1 file(s) in TocDocumentProcessor(BuildTocDocument)...
Building 1 file(s) in ConceptualDocumentProcessor(BuildConceptualDocument=>ValidateConceptualDocumentMetadata)...
/home/runner/work/Terminaux/Terminaux/DocGen/toc.yml: warning: Unable to find either toc.yml or toc.md inside api/. Make sure the file is included in config file docfx.json!
Applying templates to 2 model(s)...
XRef map exported.
Extracting index data from 1 html files
Sitemap file is successfully exported to /home/runner/work/Terminaux/Terminaux/docs/sitemap.xml


Build failed.

    2 warning(s)
    20 error(s)

Process completed with exit code 255.

AptiviCEO avatar Mar 09 '24 09:03 AptiviCEO

Sorry, please reopen the issue. The ItemGroup removes the previous generated sources and makes Visual Studio happy, but it causes the Docfx to fail. Still investigating the reason. It seems to work well in my case.

paulushub avatar Mar 09 '24 13:03 paulushub

It seems successfully generate API metadata with latest main branch code with Release build.
(When running Debug build. It failed with #9777 problem)

Because #9753 update Roslyn version to 4.9.2 and MSBuild code analysis is executed with separated process.

filzrev avatar Mar 09 '24 14:03 filzrev

OK, I've re-opened the issue. That's most likely the Roslyn version issue because Terminaux uses 4.9.2 and the latest public release of docfx uses 4.8.0.

AptiviCEO avatar Mar 09 '24 15:03 AptiviCEO

Latest version of docfx (2.76.0) use roslyn version 4.9.2. And code analysis is executed on separated BuildHost.

So roslyn version relating problems will not expected occurred after docfx updates.

filzrev avatar Apr 26 '24 05:04 filzrev

It might help to include a test sample so that future changes could be monitored. Code generation is a widely used .NET feature.

paulushub avatar Apr 26 '24 05:04 paulushub

I think ive run into the same problem, i use an analyzer to generate a class with strings containing filenames. The analyzer is in the same solution. When running docfx (2.76.0) the build fails because the class wasnt found. Are there any workarounds for that?

StefanPoelloth avatar Jun 11 '24 09:06 StefanPoelloth

@StefanPoelloth

When running docfx (2.76.0) the build fails because the class wasnt found.

What's warning/error messages are shown when running docfx metadata command? Ans is it able to provide reproduceable source codes?

filzrev avatar Jun 11 '24 09:06 filzrev

@filzrev the errors just tell me that the generated class wasnt found:

error: C:\SourceCode\myprojectname\someclass1.cs(5,14): error CS0234: The type or namespace name 'generatedTypeName' does not exist in the namespace 'generatedTypeNamespace' (are you missing an assembly reference?)
error: C:\SourceCode\myprojectname\someclass2.cs(11,14): error CS0234: The type or namespace name 'generatedTypeName' does not exist in the namespace 'generatedTypeNamespace' (are you missing an assembly reference?)

Im having problems creating a repro, the error doesnt appear there. I think i found a workaround: run dotnet build -c Release before docfx.

StefanPoelloth avatar Jun 11 '24 11:06 StefanPoelloth

@StefanPoelloth

I think i found a workaround: run dotnet build -c Release before docfx.

Do you have any of the following warning messages?

There is .NET Analyzer that can't be resolved. If this analyzer is .NET Source Generator project. Try build with dotnet build -c Release command before running docfx. Path: {path}

If above warning message is not shown. There is an unknown condition that cause Source Generator failure.

filzrev avatar Jun 11 '24 11:06 filzrev

@filzrev no, i dont get that message.

however I see the message in my repro project and tried it on my original solution which seems to work.

StefanPoelloth avatar Jun 11 '24 11:06 StefanPoelloth

Semi-related, since we ran into this issue:

Docfx seems to required the same MSBuild "environment" which was applied during build.

We have modified our dotnet build -c Release call with /property:BaseOutputPath=c:\somepath\, which in turn triggered the docfx warning mentioned above.

Only after modifying the docfx metadata call with --property BaseOutputPath=c:\somepath\ the issues went away. I also found that it needs to be exactly the same. Quoting, trailing slashes, etc. do matter.

jwfx avatar Nov 04 '24 10:11 jwfx

CompilerGeneratedFilesOutputPath workaround doesn't work well with multi-target projects when generated code depends on current TFM as generated code used by all TFMs on build.

docfx.zip

>build

>dotnet build -c Release -p CompilerGeneratedFilesOutputPath=Generated solution\docfx-with-codegen\docfx-with-codegen.csproj
Restore complete (1.4s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  generator netstandard2.0 succeeded (0.2s) → solution\generator\bin\Release\netstandard2.0\generator.dll
  docfx-with-codegen net48 succeeded with 1 warning(s) (0.3s) → solution\docfx-with-codegen\bin\Release\net48\docfx-with-codegen.dll
    c:\...\solution\docfx-with-codegen\Class1.cs(14,16): warning CS8618: Non-nullable property 'Prop' must contain a non-null value when exiting constructor. Consider adding the 'required' modifier or declaring the property as nullable.
  docfx-with-codegen netstandard2.0 succeeded with 1 warning(s) (0.9s) → solution\docfx-with-codegen\bin\Release\netstandard2.0\docfx-with-codegen.dll
    c:\...\solution\docfx-with-codegen\Class1.cs(14,16): warning CS8618: Non-nullable property 'Prop' must contain a non-null value when exiting constructor. Consider adding the 'required' modifier or declaring the property as nullable.
  docfx-with-codegen net9.0 failed with 3 error(s) (1.0s)
    c:\...\solution\docfx-with-codegen\Generated\PolySharp.SourceGenerators\PolySharp.SourceGenerators.PolyfillsGenerator\System.Runtime.CompilerServices.IsExternalInit.g.cs(4,12): error CS0729: Type 'IsExternalInit' is defined in this assembly, but a type forwarder is specified for it
    c:\...\solution\docfx-with-codegen\Generated\PolySharp.SourceGenerators\PolySharp.SourceGenerators.PolyfillsGenerator\System.Runtime.CompilerServices.RequiresLocationAttribute.g.cs(4,12): error CS0729: Type 'RequiresLocationAttribute' is defined in this assembly, but a type forwarder is specified for it
    c:\...\solution\docfx-with-codegen\Generated\generator\MyCodeGenerator\ExpressionBuilder.g.cs(6,24): error CS0757: A partial method may not have multiple implementing declarations

Build failed with 3 error(s) and 2 warning(s) in 3.9s

>docfx
Loading project c:/.../solution/docfx-with-codegen/docfx-with-codegen.csproj
Restore complete (1.0s)

Build succeeded in 1.4s
error: c:\...\solution\docfx-with-codegen\Generated\PolySharp.SourceGenerators\PolySharp.SourceGenerators.PolyfillsGenerator\System.Runtime.CompilerServices.IsExternalInit.g.cs(4,12): error CS0729: Type 'IsExternalInit' is defined in this assembly, but a type forwarder is specified for it
error: c:\...\solution\docfx-with-codegen\Generated\PolySharp.SourceGenerators\PolySharp.SourceGenerators.PolyfillsGenerator\System.Runtime.CompilerServices.RequiresLocationAttribute.g.cs(4,12): error CS0729: Type 'RequiresLocationAttribute' is defined in this assembly, but a type forwarder is specified for it
Creating output...
warning: No .NET API detected for .
Searching built-in plugins in directory C:\Users\user\.dotnet\tools\.store\docfx\2.78.2\docfx\2.78.2\tools\net9.0\any\...
Post processor ExtractSearchIndex loaded.
No files are found with glob pattern api/results/*.yml, excluding <none>, under directory "c:\..."
7 plug-in(s) loaded.
EmptyInputFiles: No file found, nothing will be generated. Please make sure docfx.json is correctly configured.
EmptyInputContents: No content file found. Please make sure the content section of docfx.json is correctly configured.


Build failed.

    1 warning(s)
    2 error(s)

MaceWindu avatar Jan 22 '25 15:01 MaceWindu

Tried to build code from master and looks like it works now. Probably due to 4.12 roslyn tooling. Still need to prebuild generator using dotnet build, but that's a minor issue.

Would be nice to have a release soon.

MaceWindu avatar Jan 22 '25 15:01 MaceWindu