Incorrect display of raw / endraw tags as part of the `codeAndOutput.md` boilerplate

[lhw-1]

Please confirm that you have searched existing issues in the repo

Yes, I have searched the existing issues

Any related issues?

#2118, #2120

Tell us about your environment

Ubuntu 20.04

MarkBind version

4.0.2

What did you do? Describe the bug

This issue was initially brought up in PR #2118.

While trying to add the following snippet using the codeAndOutput.md boilerplate to the documentation for raw / endraw tags, the following unexpected output occurred.

Code:

<include src="codeAndOutput.md" boilerplate >
<variable name="highlightStyle">markdown</variable>
<variable name="code" id="list-example">
{{ '{% raw %}' }} content {{ '{% endraw %}' }}
</variable>
</include>

Output:

~~The {% raw %} should not show up as part of the output.~~ The {% endraw %} should show up as part of the output.

Including raw / endraw tags to enable the rendering of the tags inside the variable to be bypassed like this:

{% raw %}

<include src="codeAndOutput.md" boilerplate >
<variable name="highlightStyle">markdown</variable>
<variable name="code" id="list-example">
{% raw %} content {% endraw %}
</variable>
</include>

{% endraw %}

This produces the exact same result as above.

Steps to reproduce

Include the above code snippet (ensure that the codeAndOutput.md boilerplate is also included) into a pre-existing MarkBind site, and observe the displayed result.

Expected behavior

The output section should display {% raw %} content {% endraw %}.

Actual behavior

The {% endraw %} did not show up as part of the output.

Anything else?

No response

[lhw-1]

To add to the original issue, here are some updates.

When the boilerplate of codeAndOutput.md in the MarkBind documentation is used, this should intuitively work, as the variable with name="code" is expected to display the raw text above and the output at the bottom.

<include src="codeAndOutput.md" boilerplate >
<variable name="highlightStyle">markdown</variable>
<variable name="code" id="list-example" v-pre>
{% raw %} content {% endraw %}
</variable>
</include>

However, this is the resulting display of the above code:

The display is not "raw enough", because it is processed at least once before being displayed, which necessitates the use of either raw / endraw tags or encasing the raw / endraw tags in the code as a variable (both of which methods result in the same error highlighted in this issue). Hence, it makes sense to tackle the core issue of being able to display the code inside the variable as-is and not process it.

This is the relevant part of the codeAndOutput.md that handles the display of the code given:

%%CODE:%%
<div class="indented">

```{{ highlightStyle | safe }}{ {% if heading %}heading="{{heading}}"{% endif %}}
{{ code | safe | trim }}
```

</div>

But inserting raw / endraw tags in any part of this snippet will expectedly cause the code variable inside the fenced code to not be processed:

%%CODE:%%
<div class="indented">

```{{ highlightStyle | safe }}{ {% if heading %}heading="{{heading}}"{% endif %}}
{% raw %} {{ code | safe | trim }} {% endraw %}
```

</div>

At this point, the code variable is processed and then passed into the fenced code block. Hence, the issue likely needs to be addressed somewhere before this happens.

[ang-zeyu]

For the OP, not sure I understand this,

<include src="codeAndOutput.md" boilerplate >
<variable name="highlightStyle">markdown</variable>
<variable name="code" id="list-example">
{{ '{% raw %}' }} content {{ '{% endraw %}' }}
</variable>
</include>

The {% raw %} should not show up as part of the output.

Shouldn't it show up because the variable you are passing in is {% raw %} content {% endraw %}? Rather the problem seems to be to me that {% endraw %} isn't showing up. (likely due to markdown-it-attrs, maybe we can consider changing the delimiter..)

I.e. in codeAndOutput.md, after nunjucks rendering, we have:

<div class="indented">

<box border-left-color="grey" background-color="white">

{% raw %} content {% endraw %}
</box>
</div>

({% endraw %} is erased due to markdown-it-attrs)

[ang-zeyu]

Replying to the update,

The display is not "raw enough", because it is processed at least once before being displayed, which necessitates the use of either raw / endraw tags or encasing the raw / endraw tags in the code as a variable (both of which methods result in the same error highlighted in this issue). Hence, it makes sense to tackle the core issue of being able to display the code inside the variable as-is and not process it.

This is intended, as nunjucks (the templating engine) is always processed first. You'd indeed need to double wrap it but not in the way in your example:

<include src="codeAndOutput.md" boilerplate >
<variable name="highlightStyle">markdown</variable>
<variable name="code" id="list-example" v-pre>
{% raw %}{% raw %}  content {% endraw %}{% endraw %}
</variable>
</include>

If you wrap it at {% raw %} {{ code | safe | trim }} {% endraw %}, Nunjucks correctly treats {{ code | safe | trim }} as raw text.

Labelled this byDesign for now but let me know if you have any suggestions on this behaviour. (e.g. improve the documentation around this)

[lhw-1]

Shouldn't it show up because the variable you are passing in is {% raw %} content {% endraw %}? Rather the problem seems to be to me that {% endraw %} isn't showing up. (likely due to markdown-it-attrs, maybe we can consider changing the delimiter..)

I.e. in codeAndOutput.md, after nunjucks rendering, we have:

<div class="indented">

<box border-left-color="grey" background-color="white">

{% raw %} content {% endraw %}
</box>
</div>

({% endraw %} is erased due to markdown-it-attrs)

You are right! I believe what I initially wanted to convey was that in the ideal use case, the {% raw %} tag should not show up since we want to tell users that {% raw %} content {% endraw %} will be processed as content.

The use case I posted should display an {% endraw %} tag since it should be processed as a variable, and the issue is indeed that the {% endraw %} tag does not show up. I'll update the issue to reflect this.

[lhw-1]

Replying to the update,

The display is not "raw enough", because it is processed at least once before being displayed, which necessitates the use of either raw / endraw tags or encasing the raw / endraw tags in the code as a variable (both of which methods result in the same error highlighted in this issue). Hence, it makes sense to tackle the core issue of being able to display the code inside the variable as-is and not process it.

This is intended, as nunjucks (the templating engine) is always processed first. You'd indeed need to double wrap it but not in the way in your example:

<include src="codeAndOutput.md" boilerplate >
<variable name="highlightStyle">markdown</variable>
<variable name="code" id="list-example" v-pre>
{% raw %}{% raw %}  content {% endraw %}{% endraw %}
</variable>
</include>

If you wrap it at {% raw %} {{ code | safe | trim }} {% endraw %}, Nunjucks correctly treats {{ code | safe | trim }} as raw text.

That makes sense 👍

Labelled this byDesign for now but let me know if you have any suggestions on this behaviour. (e.g. improve the documentation around this)

I believe the missing {% endraw %} tag in the use case posted originally can still be tackled (e.g. by modifying delimiters as you pointed out), so I think we can remove the byDesign label?

Additionally, yes we should update the documentation for this behavior (ideally in the Tips and Tricks section of the UG) with the proper use cases!

[ang-zeyu]

I believe the missing {% endraw %} tag in the use case posted originally can still be tackled (e.g. by modifying delimiters as you pointed out), so I think we can remove the byDesign label?

Just a fleeting suggestion but not sure if we should actually do it. This is a very rare use case (displaying {% endraw %} in documentation), unless there are other common instances where {...} appears at the end of markdown paragraphs, this seems like a drastic move.

Should we consider modifying the markdown-it-attrs algorithm to instead prevent false positives? e.g. % is probably not a valid HTML attribute name, so ignore it in that case.

Ideally this lands upstream but I'm not sure if the maintainer would still take contributions like these according to its readme policy. We can also consider forking otherwise.

Still I think it would be very low priority as this case is very rare.

Additionally, yes we should update the documentation for this behavior (ideally in the Tips and Tricks section of the UG) with the proper use cases!

Agree a minor note in the documentation is a good start still. We have a related section for this https://markbind.org/userGuide/formattingContents.html#classes-attributes-and-amp-identifiers.