-
-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Replace to gopkg.in/yaml with github.com/goccy/go-yaml (note) #13033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
ecb0fa6 to
6a658a6
Compare
|
|
With goccy/go-yaml#461 closed, I must also close this. For now, at least. I may revisit this if once I understand it better / find a workaround on Hugo's side. But re. Hugo's security policy, the templates are trusted, the content files are not. It's fairly common to print/marshal (e.g. |
|
I'll reopen; I have worked with this library in another project, and having those anchors/aliases is really useful. I will try to figure out a way to detect these anomalies in the Go map. |
f326453 to
1114cb7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR replaces the gopkg.in/yaml library with github.com/goccy/go-yaml to improve security and stability. The primary motivation is to add protection against the "Billion Laughs" attack by validating YAML structures during unmarshaling, with a limit of 10,000 non-scalar aliases.
Key changes:
- Implements YAML validation logic to prevent resource exhaustion from malicious YAML files
- Updates all YAML marshaling/unmarshaling to use new wrapper functions
- Adjusts test expectations to match new library's type handling (integers become uint64)
Reviewed Changes
Copilot reviewed 20 out of 21 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| parser/metadecoders/encoder.go | New file with YAML marshaling configuration including attack prevention |
| parser/metadecoders/decoder.go | Adds UnmarshalYaml with validation and removes old map key stringification logic |
| parser/metadecoders/decoder_test.go | Updates tests for uint64 types and adds billion laughs benchmarks |
| tpl/transform/remarshal_test.go | Adds test for billion laughs attack protection |
| tpl/transform/transform_integration_test.go | Adds integration test for excessive YAML structure validation |
| langs/i18n/translationProvider.go | Updates to use new UnmarshalYaml function |
| tpl/openapi/openapi3/openapi3.go | Removes old yaml import and uses new metadecoders function |
| parser/frontmatter.go | Updates YAML marshaling to use new wrapper function |
| hugolib/page__content.go | Improves error handling for front matter parsing |
| common/herrors/file_error.go | Refactors line number extraction logic |
| common/herrors/line_number_extractors.go | Adds support for YAML error format [line:col] |
| go.mod | Replaces yaml dependencies |
| Multiple test files | Updates expectations for uint64 types and error messages |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
b220107 to
96d25a0
Compare
|
I have created a new issue upstream to improve the handling of the "Billion laughs attack" (goccy/go-yaml#803), but I think we can go ahead with this PR for now. |
|
@jmooring I think this PR is good to go now, I would appreciate if you could have a look/take it for a stab. I now notice that the old I have also tested this with the docs site and the |
|
I think this is great, but we need to note that this is a breaking change given that we now recognize Norway as a country instead of negating its existence.
|
As a Norwegian I consider that a bug fix, but yes, I will add some comments in the release notes. |
|
Note that this also addresses #8427. |
d965d9f to
26c8606
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 20 out of 21 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
This commit also adds validation to prevent the "Billion Laughs" attack (see goccy/go-yaml#461). The limit of non-scalar aliases to the same node is set to 10,000. See benchmarks below. ``` │ sec/op │ UnmarshalBillionLaughs/Billion_Laughs_no_validation-10 125.2µ ± ∞ ¹ UnmarshalBillionLaughs/Billion_Laughs_with_validation-10 655.8µ ± ∞ ¹ UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10 9.223µ ± ∞ ¹ UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10 9.443µ ± ∞ ¹ geomean 51.71µ ¹ need >= 6 samples for confidence interval at level 0.95 │ fix-goyaml-8822.bench │ │ B/op │ UnmarshalBillionLaughs/Billion_Laughs_no_validation-10 177.0Ki ± ∞ ¹ UnmarshalBillionLaughs/Billion_Laughs_with_validation-10 177.0Ki ± ∞ ¹ UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10 11.67Ki ± ∞ ¹ UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10 11.67Ki ± ∞ ¹ geomean 45.45Ki ¹ need >= 6 samples for confidence interval at level 0.95 │ fix-goyaml-8822.bench │ │ allocs/op │ UnmarshalBillionLaughs/Billion_Laughs_no_validation-10 3.302k ± ∞ ¹ UnmarshalBillionLaughs/Billion_Laughs_with_validation-10 3.305k ± ∞ ¹ UnmarshalBillionLaughs/YAML_Front_Matter_no_validation-10 253.0 ± ∞ ¹ UnmarshalBillionLaughs/YAML_Front_Matter_with_validation-10 253.0 ± ∞ ¹ ```` Fixes gohugoio#8822 Fixes gohugoio#13043 Fixes gohugoio#14053 Fixes #gohugoio#8427
This commit also adds validation to prevent the "Billion Laughs" attack (see goccy/go-yaml#461). The limit of non-scalar aliases to the same node is set to 10,000. See benchmarks below.
Fixes #8822
Fixes #13043
Fixes #14053