Skip to content

Conversation

@bep
Copy link
Member

@bep bep commented Nov 13, 2024

This commit also adds validation to prevent the "Billion Laughs" attack (see goccy/go-yaml#461). The limit of non-scalar aliases to the same node is set to 10,000. See benchmarks below.

UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                 125.2µ ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10               655.8µ ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10              9.223µ ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10            9.443µ ± ∞ ¹
geomean                                                                51.71µ
¹ need >= 6 samples for confidence interval at level 0.95

                                                            │ fix-goyaml-8822.bench │
                                                            │         B/op          │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                177.0Ki ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10              177.0Ki ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10             11.67Ki ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10           11.67Ki ± ∞ ¹
geomean                                                               45.45Ki
¹ need >= 6 samples for confidence interval at level 0.95

                                                            │ fix-goyaml-8822.bench │
                                                            │       allocs/op       │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                 3.302k ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10               3.305k ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10               253.0 ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10             253.0 ± ∞ ¹

Fixes #8822
Fixes #13043
Fixes #14053

@bep bep force-pushed the fix/goyaml-8822 branch 4 times, most recently from ecb0fa6 to 6a658a6 Compare November 14, 2024 19:12
@bep bep marked this pull request as ready for review November 14, 2024 19:12
@bep
Copy link
Member Author

bep commented Nov 15, 2024

@gohugoio gohugoio deleted a comment from Accuweaty24 Nov 30, 2024
@bep
Copy link
Member Author

bep commented Dec 15, 2024

With goccy/go-yaml#461 closed, I must also close this. For now, at least. I may revisit this if once I understand it better / find a workaround on Hugo's side. But re. Hugo's security policy, the templates are trusted, the content files are not. It's fairly common to print/marshal (e.g. jsonify) front matter values. So, even if the Go maps themselves isn't big, one could easily imagine the rendered output blow up with front matter constructs like the below:

---
title: Biiiig Front Matter
a: &a [_,_,_,_,_,_,_,_,_,_,_,_,_,_,_]
b: &b [*a,*a,*a,*a,*a,*a,*a,*a,*a,*a]
c: &c [*b,*b,*b,*b,*b,*b,*b,*b,*b,*b]
d: &d [*c,*c,*c,*c,*c,*c,*c,*c,*c,*c]
e: &e [*d,*d,*d,*d,*d,*d,*d,*d,*d,*d]
f: &f [*e,*e,*e,*e,*e,*e,*e,*e,*e,*e]
g: &g [*f,*f,*f,*f,*f,*f,*f,*f,*f,*f]
h: &h [*g,*g,*g,*g,*g,*g,*g,*g,*g,*g]
i: &i [*h,*h,*h,*h,*h,*h,*h,*h,*h,*h]
---
@bep bep closed this Dec 15, 2024
@bep
Copy link
Member Author

bep commented Dec 19, 2024

I'll reopen; I have worked with this library in another project, and having those anchors/aliases is really useful. I will try to figure out a way to detect these anomalies in the Go map.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR replaces the gopkg.in/yaml library with github.com/goccy/go-yaml to improve security and stability. The primary motivation is to add protection against the "Billion Laughs" attack by validating YAML structures during unmarshaling, with a limit of 10,000 non-scalar aliases.

Key changes:

  • Implements YAML validation logic to prevent resource exhaustion from malicious YAML files
  • Updates all YAML marshaling/unmarshaling to use new wrapper functions
  • Adjusts test expectations to match new library's type handling (integers become uint64)

Reviewed Changes

Copilot reviewed 20 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
parser/metadecoders/encoder.go New file with YAML marshaling configuration including attack prevention
parser/metadecoders/decoder.go Adds UnmarshalYaml with validation and removes old map key stringification logic
parser/metadecoders/decoder_test.go Updates tests for uint64 types and adds billion laughs benchmarks
tpl/transform/remarshal_test.go Adds test for billion laughs attack protection
tpl/transform/transform_integration_test.go Adds integration test for excessive YAML structure validation
langs/i18n/translationProvider.go Updates to use new UnmarshalYaml function
tpl/openapi/openapi3/openapi3.go Removes old yaml import and uses new metadecoders function
parser/frontmatter.go Updates YAML marshaling to use new wrapper function
hugolib/page__content.go Improves error handling for front matter parsing
common/herrors/file_error.go Refactors line number extraction logic
common/herrors/line_number_extractors.go Adds support for YAML error format [line:col]
go.mod Replaces yaml dependencies
Multiple test files Updates expectations for uint64 types and error messages

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@bep bep force-pushed the fix/goyaml-8822 branch 3 times, most recently from b220107 to 96d25a0 Compare October 17, 2025 13:08
@bep
Copy link
Member Author

bep commented Oct 17, 2025

I have created a new issue upstream to improve the handling of the "Billion laughs attack" (goccy/go-yaml#803), but I think we can go ahead with this PR for now.

@bep bep added this to the v0.152.0 milestone Oct 17, 2025
@bep bep requested a review from jmooring October 17, 2025 14:01
@bep
Copy link
Member Author

bep commented Oct 17, 2025

@jmooring I think this PR is good to go now, I would appreciate if you could have a look/take it for a stab. I now notice that the old go-yaml library is in "archive mode", so I guess it's time to move away from it. I don't think this should create too many surprises in the wild, but I will push it out into its own release so it (hopefully) gets noticed.

I have also tested this with the docs site and the code-toggle shortcode, and YAML aliases works great there, too, assuming you start out with YAML.

@jmooring
Copy link
Member

jmooring commented Oct 17, 2025

I think this is great, but we need to note that this is a breaking change given that we now recognize Norway as a country instead of negating its existence.

Values Old meaning New meaning
yes, Yes, YES, y, Y, on, On, ON true (bool) yes, Yes, YES, y, Y, on, On, ON (string)
no, No, NO, n, N, off, Off, OFF false (bool) no, No, NO, n, N, off, Off, OFF (string)
@bep
Copy link
Member Author

bep commented Oct 17, 2025

I think this is great, but we need to note that this is a breaking change given that we now recognize Norway as a country instead of negating its existence.

As a Norwegian I consider that a bug fix, but yes, I will add some comments in the release notes.

@jmooring
Copy link
Member

Note that this also addresses #8427.

@bep bep force-pushed the fix/goyaml-8822 branch 2 times, most recently from d965d9f to 26c8606 Compare October 18, 2025 10:28
@bep bep requested a review from Copilot October 18, 2025 10:46
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

This commit also adds validation to prevent the "Billion Laughs" attack (see goccy/go-yaml#461). The limit of non-scalar aliases to the same node is set to 10,000. See benchmarks below.

```                                                            │        sec/op         │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                 125.2µ ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10               655.8µ ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10              9.223µ ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10            9.443µ ± ∞ ¹
geomean                                                                51.71µ
¹ need >= 6 samples for confidence interval at level 0.95

                                                            │ fix-goyaml-8822.bench │
                                                            │         B/op          │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                177.0Ki ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10              177.0Ki ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10             11.67Ki ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10           11.67Ki ± ∞ ¹
geomean                                                               45.45Ki
¹ need >= 6 samples for confidence interval at level 0.95

                                                            │ fix-goyaml-8822.bench │
                                                            │       allocs/op       │
UnmarshalBillionLaughs/Billion_Laughs_no_validation-10                 3.302k ± ∞ ¹
UnmarshalBillionLaughs/Billion_Laughs_with_validation-10               3.305k ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10               253.0 ± ∞ ¹
UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10             253.0 ± ∞ ¹
````

Fixes gohugoio#8822
Fixes gohugoio#13043
Fixes gohugoio#14053
Fixes #gohugoio#8427
@bep bep force-pushed the fix/goyaml-8822 branch from 60b42c3 to cb9ad41 Compare October 18, 2025 10:53
@bep bep merged commit a3d9548 into gohugoio:master Oct 18, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants