Skip to content

Checkpoint | support unix and unix_ms ts for time field - #4781

Merged
kcreddy merged 3 commits into
elastic:mainfrom
kcreddy:sdh_2801_checkpoint
Dec 14, 2022
Merged

Checkpoint | support unix and unix_ms ts for time field#4781
kcreddy merged 3 commits into
elastic:mainfrom
kcreddy:sdh_2801_checkpoint

Conversation

@kcreddy

@kcreddy kcreddy commented Dec 7, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds support for having checkpoint.time field as UNIX_MS.

Just adding UNIX_MS format to existing formats in date processor here doesn't work as there is no check done on DateProcessor whether the value is seconds or milliseconds. This makes the current date processor continues to match UNIX format even for millisecond epoch values.

Alternate approach to check the number of digits of checkpoint.time is taken to resolve this issue.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
@kcreddy kcreddy self-assigned this Dec 7, 2022
@kcreddy kcreddy added bug Something isn't working, use only for issues Integration:checkpoint Check Point labels Dec 7, 2022
@kcreddy kcreddy changed the title support unix and unix_ms ts for time field Dec 7, 2022
@elasticmachine

elasticmachine commented Dec 7, 2022 •
edited
Loading

Copy link
Copy Markdown

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-12-08T17:56:55.604+0000

  • Duration: 16 min 2 sec

Test stats 🧪

Test Results
Failed 0
Passed 12
Skipped 0
Total 12

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.
@elasticmachine

elasticmachine commented Dec 7, 2022 •
edited
Loading

Copy link
Copy Markdown

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (1/1) 💚
Classes 100.0% (1/1) 💚
Methods 100.0% (15/15) 💚 13.333
Lines 83.534% (695/832) 👍 29.397
Conditionals 100.0% (0/0) 💚
@kcreddy
kcreddy marked this pull request as ready for review December 8, 2022 15:30
@kcreddy
kcreddy requested a review from a team as a code owner December 8, 2022 15:31
@elasticmachine

elasticmachine commented Dec 8, 2022

Copy link
Copy Markdown

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)
P1llus
P1llus approved these changes Dec 8, 2022
View reviewed changes

@P1llus P1llus left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the question is answered or fixed then we can merge
Comment thread packages/checkpoint/data_stream/firewall/elasticsearch/ingest_pipeline/default.yml Outdated
- convert:
field: checkpoint.time
type: long
ignore_failure: true

@P1llus P1llus Dec 8, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want ignore failure here? Is there no other if condition we can use?

@kcreddy kcreddy Dec 8, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, let me change 👍🏼

@kcreddy kcreddy Dec 8, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the convert and date processors and written some painless to parse all 3 date formats. Let me know if that looks good.
@kcreddy
kcreddy merged commit 64c4e2d into elastic:main Dec 14, 2022
@elasticmachine

elasticmachine commented Dec 14, 2022

Copy link
Copy Markdown

Package checkpoint - 1.9.1 containing this change is available at https://epr.elastic.co/search?package=checkpoint
@kcreddy
kcreddy deleted the sdh_2801_checkpoint branch February 7, 2025 08:37
orestisfl pushed a commit to orestisfl/integrations that referenced this pull request May 15, 2026
@kcreddy
Checkpoint | support unix and unix_ms ts for time field (elastic#4781)
6ebc68a 
* support unix and unix_ms ts for time field

* zoned dt

* remove convert and date processor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working, use only for issues Integration:checkpoint Check Point

3 participants

@kcreddy @elasticmachine @P1llus