[Bug] Foreign Language Broken

[VaclavElias]

Describe the bug
The most recent update v2.70.2 broke something with the encoding 😭😭. Is this intended and we need to include any new settings?

Generated HTML

To Reproduce

Broken
Check here, built by 2.70.2 https://stride-doc-staging.azurewebsites.net/latest/jp/index.html
Check here, built by 2.70.2 https://stride-doc-staging.azurewebsites.net/latest/jp/manual/index.html

Ok
Check here, built by 2.70.0 https://stride-docs-test.azurewebsites.net/latest/jp/
Check here, built by 2.70.0 https://stride-docs-test.azurewebsites.net/latest/jp/manual/index.html

If needed:
https://github.com/stride3d/stride-docs/blob/staging/en/docfx.json

Expected behaviour
The Japanese language should be rendered correctly.

[filzrev]

As far as I've tested on my environment. (Windows+ Ubunts(on WSL))
Non-ascii docs is rendered correctly with docfx v2.70.2.

It might be caused by post-processing with powershell.exe (not pwsh).
https://github.com/stride3d/stride-docs/blob/master/jp-build.ps1#L19

Because powershell.exe's Get-Content use OS culture when following condition met.

• BOM is not exists.
• -Encoding parmeter is not explicitly specified.

Note: I'm not against to reverting HTML file encoding for backward compatibility.

[VaclavElias]

@filzrev I will check our scripts, retest again with v2.70.2 and report back later today.

[KalleOlaviNiemitalo]

@VaclavElias, you'd better try v2.70.3, which includes 3968fc5 reverting #9131.

[VaclavElias]

@KalleOlaviNiemitalo, I will test that one for sure as well 🤣.

Just wanted to test v2.70.2 before our power shell postprocessing just in case @filzrev is right that something might not be right with our scripts.

[VaclavElias]

@yufeih, @filzrev, @KalleOlaviNiemitalo unfortunately, we had indeed missing one encoding option in our post-processing script. With that corrected now, I tested again v2.70.2 and there were no issues.

Note: I am aware that there is now 2.70.3, I specifically re-tested on v2.70.2

I didn't assume the issue could be on our side, as our deployment automatically updated to the latest docfx version, at that time v2.70.2 and the language broke so I wasn't checking our post-processing in the first place.

You can revert the reverted in the future 🤣.

Thank you everyone for your recent contributions 🙂