Skip to content

[winlogbeat] Add missing functionality to experimental API#41525

Merged
marc-gr merged 29 commits intoelastic:mainfrom
marc-gr:feat/experimental-api-userdata
Nov 25, 2024
Merged

[winlogbeat] Add missing functionality to experimental API#41525
marc-gr merged 29 commits intoelastic:mainfrom
marc-gr:feat/experimental-api-userdata

Conversation

@marc-gr
Copy link
Contributor

@marc-gr marc-gr commented Nov 5, 2024 •
edited
Loading

Proposed commit message

Makes experimental api on par with the default one:

  • For events that have UserData instead of EventData, populate it also for experimental api
  • Include XML is respected
  • Forwarded events use renderedtext info
  • Language setting is respected
  • Language setting also added to decode xml wineventlog processor
  • Format embedded messages in the experimental api

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Also added include xml as part of the benchmark matrix, to basically check we perform as the default API when having it enabled:

$ go test -run TestBenchmarkRead -benchmem -benchtime 10s -benchtest -v .
=== RUN   TestBenchmarkRead
=== RUN   TestBenchmarkRead/api=wineventlog
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=10
    bench_test.go:119: 3710.24 events/sec        21030 B/event   210302 B/batch  282 allocs/event        2825 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=100
    bench_test.go:119: 4600.02 events/sec        20658 B/event   2065829 B/batch         281 allocs/event        28127 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=500
    bench_test.go:119: 4636.17 events/sec        20301 B/event   10150670 B/batch        281 allocs/event        140554 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=1000
    bench_test.go:119: 4620.39 events/sec        20930 B/event   20930935 B/batch        281 allocs/event        281082 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=10
    bench_test.go:119: 4326.34 events/sec        19713 B/event   197133 B/batch  281 allocs/event        2815 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=100
    bench_test.go:119: 4659.42 events/sec        19391 B/event   1939168 B/batch         280 allocs/event        28026 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=500
    bench_test.go:119: 4647.01 events/sec        19011 B/event   9505945 B/batch         280 allocs/event        140055 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=1000
    bench_test.go:119: 4578.59 events/sec        19644 B/event   19644712 B/batch        280 allocs/event        280085 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=10
    bench_test.go:119: 4191.40 events/sec        21941 B/event   219410 B/batch  281 allocs/event        2817 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=100
    bench_test.go:119: 4410.18 events/sec        21652 B/event   2165202 B/batch         281 allocs/event        28123 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=500
    bench_test.go:119: 4509.38 events/sec        21278 B/event   10639211 B/batch        281 allocs/event        140572 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=1000
    bench_test.go:119: 4480.46 events/sec        21941 B/event   21941332 B/batch        281 allocs/event        281122 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=10
    bench_test.go:119: 32061.20 events/sec       4324 B/event    43242 B/batch   28 allocs/event         283 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=100
    bench_test.go:119: 46401.75 events/sec       3917 B/event    391734 B/batch  27 allocs/event         2767 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=500
    bench_test.go:119: 48948.90 events/sec       3554 B/event    1777464 B/batch         27 allocs/event         13792 allocs/batch
=== RUN   TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=1000
    bench_test.go:119: 48948.13 events/sec       4246 B/event    4246701 B/batch         27 allocs/event         27580 allocs/batch
--- PASS: TestBenchmarkRead (262.38s)
    --- PASS: TestBenchmarkRead/api=wineventlog (98.03s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=10 (12.52s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=100 (14.23s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=500 (10.89s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=true/batch_size=1000 (12.12s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=10 (12.10s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=100 (13.94s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=500 (10.87s)
        --- PASS: TestBenchmarkRead/api=wineventlog/include_xml=false/batch_size=1000 (11.37s)
    --- PASS: TestBenchmarkRead/api=wineventlog-experimental (135.84s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=10 (11.73s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=100 (14.35s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=500 (11.10s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=true/batch_size=1000 (12.49s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=10 (24.58s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=100 (17.02s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=500 (22.27s)
        --- PASS: TestBenchmarkRead/api=wineventlog-experimental/include_xml=false/batch_size=1000 (22.29s)
PASS
ok      github.com/elastic/beats/v7/winlogbeat/eventlog 262.417s
@marc-gr marc-gr added enhancement Winlogbeat Team:Security-Windows Platform Windows Platform Team in Security Solution backport-8.x Automated backport to the 8.x branch with mergify labels Nov 5, 2024
@marc-gr marc-gr requested a review from andrewkroh November 5, 2024 14:56
@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Nov 5, 2024
@mergify mergify bot assigned marc-gr Nov 5, 2024
@marc-gr marc-gr marked this pull request as ready for review November 5, 2024 15:38
@marc-gr marc-gr requested a review from a team as a code owner November 5, 2024 15:38
@elasticmachine
Copy link
Contributor

elasticmachine commented Nov 5, 2024

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)
@marc-gr marc-gr changed the title [winlogbeat] Put data under UserData also in experimental api Nov 11, 2024
@marc-gr marc-gr mentioned this pull request Sep 20, 2024
Closed
7 tasks
@mergify
Copy link
Contributor

mergify bot commented Nov 18, 2024

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b feat/experimental-api-userdata upstream/feat/experimental-api-userdata
git merge upstream/main
git push upstream feat/experimental-api-userdata
intxgo
intxgo approved these changes Nov 21, 2024
View reviewed changes
@marc-gr marc-gr merged commit 4278366 into elastic:main Nov 25, 2024
@marc-gr marc-gr deleted the feat/experimental-api-userdata branch November 25, 2024 10:35
mergify bot pushed a commit that referenced this pull request Nov 25, 2024
@marc-gr @mergify
[winlogbeat] Add missing functionality to experimental API (#41525)
7706feb 
* Put data under UserData also in experimental api

* Change docs and changelog

* check evt meta

* Propagate locale config appropiately

* Extract metadata cache

* Add render config

* Simplify render functions

* Add xml rendering to experimental api

* Add benchmarks

* Update docs

* Fix multi os build

* Format embedded messages in the experimental api

* Safer assert

* Test exp api include xml with same test suite

* Check for nil metadata

* Revert "Safer assert"

This reverts commit db5a57d.

* Use single buffer to render xml

(cherry picked from commit 4278366)
@mergify mergify bot mentioned this pull request Nov 25, 2024
Merged
6 tasks
marc-gr added a commit that referenced this pull request Nov 25, 2024
@mergify @marc-gr
[winlogbeat] Add missing functionality to experimental API (#41525) (#…
7e5b5f5 
…41769)

* Put data under UserData also in experimental api

* Change docs and changelog

* check evt meta

* Propagate locale config appropiately

* Extract metadata cache

* Add render config

* Simplify render functions

* Add xml rendering to experimental api

* Add benchmarks

* Update docs

* Fix multi os build

* Format embedded messages in the experimental api

* Safer assert

* Test exp api include xml with same test suite

* Check for nil metadata

* Revert "Safer assert"

This reverts commit db5a57d.

* Use single buffer to render xml

(cherry picked from commit 4278366)

Co-authored-by: Marc Guasch <marc-gr@users.noreply.github.com>
Kavindu-Dodan pushed a commit to Kavindu-Dodan/beats that referenced this pull request Nov 27, 2024
@marc-gr @Kavindu-Dodan
[winlogbeat] Add missing functionality to experimental API (elastic#4…
0e4df9b 
…1525)

* Put data under UserData also in experimental api

* Change docs and changelog

* check evt meta

* Propagate locale config appropiately

* Extract metadata cache

* Add render config

* Simplify render functions

* Add xml rendering to experimental api

* Add benchmarks

* Update docs

* Fix multi os build

* Format embedded messages in the experimental api

* Safer assert

* Test exp api include xml with same test suite

* Check for nil metadata

* Revert "Safer assert"

This reverts commit db5a57d.

* Use single buffer to render xml
@khushijain21 khushijain21 mentioned this pull request Jun 23, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-8.x Automated backport to the 8.x branch with mergify enhancement Team:Security-Windows Platform Windows Platform Team in Security Solution Winlogbeat

3 participants

@marc-gr @elasticmachine @intxgo