Skip to content

[Discover] Added context aware logic for logs view in discover to show Load More…#211176

Merged
achyutjhunjhunwala merged 55 commits intoelastic:mainfrom
achyutjhunjhunwala:add-logic-to-load-more-in-logs-view-in-discover
Mar 12, 2025
Merged

[Discover] Added context aware logic for logs view in discover to show Load More…#211176
achyutjhunjhunwala merged 55 commits intoelastic:mainfrom
achyutjhunjhunwala:add-logic-to-load-more-in-logs-view-in-discover

Conversation

@achyutjhunjhunwala
Copy link
Contributor

@achyutjhunjhunwala achyutjhunjhunwala commented Feb 14, 2025

Summary

Closes #166679

What's included ?

  • The PR adds a feature in Logs View of Observability (to start with) to hide the regular pagination toolbar from the footer and show Load More only when the user has scrolled to the bottom of the page.
  • The table would always load the items in batches of default set 500
  • This PR also add 2 helper functions useThrottleFn and useDebounceFn. Current React help library which KIbana uses called -react-use does not have these and we cannot use Lodash variant of these. We need such hooks which are React safe. Hence added these 2

What's pending ?

  • Unit tests for the 2 new helper React hooks
  • Unit tests for data table footer component
  • Unit tests for Profile Resolution
  • Functional Serverless Tests
  • Functional Stateful Tests

Feb-14-2025 15-25-18

… for pagination rather than regular page number based pagination
@achyutjhunjhunwala achyutjhunjhunwala added release_note:feature Makes this part of the condensed release notes Team:obs-onboarding Observability Onboarding Team labels Feb 14, 2025
@achyutjhunjhunwala achyutjhunjhunwala self-assigned this Feb 14, 2025
@achyutjhunjhunwala achyutjhunjhunwala requested review from a team as code owners February 14, 2025 09:45
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

@achyutjhunjhunwala achyutjhunjhunwala marked this pull request as draft February 14, 2025 09:46
@achyutjhunjhunwala achyutjhunjhunwala added the backport:skip This PR does not require backporting label Feb 14, 2025
@kertal kertal added Feature:Discover Discover Application Project:OneDiscover Enrich Discover with contextual awareness labels Feb 17, 2025
Copy link
Contributor

@davismcphee davismcphee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave this a spin locally, and it's working well overall, but I am noticing that UI flickering I was referring to in the issue when scrolling to the point where the footer is toggled. It's a bit harder to hit using debounce vs throttle, but still there:
flicker

I'm also not really a fan of the debounce effect in general. It makes things feel laggy and delayed while scrolling IMO. Is there a particular reason to use it over throttle?

achyutjhunjhunwala and others added 3 commits March 10, 2025 10:18
…ents/data_table_footer.test.tsx

Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
…ents/data_table_footer.test.tsx

Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
@davismcphee
Copy link
Contributor

@achyutjhunjhunwala Thanks for looking into it more! Unfortunately I noticed an issue when testing where the footer visibility gets reset and remains visible on loading complete if scrolling up and then down again while loading more:
scroll

I played around with it a bit and opened a tiny PR against this one with an alternate approach that doesn't use local state and should avoid the reset issue: achyutjhunjhunwala#1. Let me know if you think it makes sense to try something like this instead!

achyutjhunjhunwala and others added 2 commits March 11, 2025 14:43
Alternate approach to hiding footer on "load more" complete
@achyutjhunjhunwala
Copy link
Contributor Author

@achyutjhunjhunwala Thanks for looking into it more! Unfortunately I noticed an issue when testing where the footer visibility gets reset and remains visible on loading complete if scrolling up and then down again while loading more: ![scroll]

I played around with it a bit and opened a tiny PR against this one with an alternate approach that doesn't use local state and should avoid the reset issue: achyutjhunjhunwala#1. Let me know if you think it makes sense to try something like this instead!

@davismcphee This approach indeed fixes the bug which you found. I have accepted the PR.
Though i do have one thought - hasScrollToBottom is no longer reliable if the footer component is not the only consumer as it now relies on loading state and returns the previous value.

Eg -

  1. Scroll to the bottom
  2. Click on Load More
  3. Now immediately Scroll Up and then again to the bottom before the Loading is completed
  4. hasScrollToBottom still returns false and now one has to rely on the loadingState
Copy link
Member

@weltenwort weltenwort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you. I defer to @davismcphee for the ongoing discussion 😇.

Copy link
Contributor

@davismcphee davismcphee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything seems to be working great now! Thanks for all the work on this and LGTM 👍 I left a couple test related comments that would be great to address if possible, but I don't want to block merging this more than I already have.

Regarding hasScrollToBottom, that's true, although I think it's ok for now since the footer is currently the only component which depends on it. I think we can cross that bridge if/when we get there 🙂

Also quick side note, I updated the labels to backport to 8.19 too since it's pre-FF and we're trying to keep our work in sync between 8.19 and 9.1 when possible.

@davismcphee davismcphee added backport:version Backport to applied version labels v9.1.0 v8.19.0 and removed backport:skip This PR does not require backporting labels Mar 11, 2025
@achyutjhunjhunwala achyutjhunjhunwala enabled auto-merge (squash) March 12, 2025 09:50
@achyutjhunjhunwala achyutjhunjhunwala merged commit 591c5b7 into elastic:main Mar 12, 2025
9 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/13811573108

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
apm 1880 1885 +5
cloudSecurityPosture 663 680 +17
controls 346 353 +7
discover 1203 1208 +5
esqlDataGrid 399 416 +17
infra 1445 1451 +6
inventory 228 234 +6
investigateApp 256 263 +7
lens 1513 1527 +14
maps 1246 1253 +7
presentationPanel 111 117 +6
securitySolution 7070 7077 +7
slo 1110 1126 +16
streamsApp 371 378 +7
visualizations 466 471 +5
total +132

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/unified-data-table 107 109 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
cloudSecurityPosture 487.3KB 488.8KB +1.5KB
discover 959.1KB 960.4KB +1.4KB
esqlDataGrid 157.6KB 159.1KB +1.5KB
securitySolution 8.9MB 8.9MB +1.5KB
slo 920.3KB 921.8KB +1.5KB
total +7.3KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
@kbn/react-hooks 0 2 +2

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
discover 20.2KB 20.2KB -1.0B
Unknown metric groups

API count

id before after diff
@kbn/react-hooks 20 26 +6
@kbn/unified-data-table 185 188 +3
total +9

History

cc @achyutjhunjhunwala

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [Streams] Replay loghub data with synthtrace (#212120)
- [Discover] Enable consistent-type-imports eslint rule (#212293)
- [Discover] Replace DiscoverInternalStateContainer with Redux based InternalStateStore (#208784)
- [Unified search] Change codeowners to presentation team (#212855)
- SKA: Update and breakdown x-pack/.gitignore (#212341)
- [Discover] Fix formatting and sorting for custom ES|QL vars (#209360)

Manual backport

To create the backport manually run:

node scripts/backport --pr 211176

Questions ?

Please refer to the Backport tool documentation

@achyutjhunjhunwala
Copy link
Contributor Author

@davismcphee I am seeing this sort of Backport failed message for the 1st time.

Does this mean i need to backport all those other PRs to get this one also done

@achyutjhunjhunwala
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@davismcphee
Copy link
Contributor

@achyutjhunjhunwala Looks like you got this figured out anyway since I see the backport now, but generally the answer is no. I think it's just a suggestion based on where it noticed conflicts occurred.

@achyutjhunjhunwala achyutjhunjhunwala deleted the add-logic-to-load-more-in-logs-view-in-discover branch March 13, 2025 13:19
achyutjhunjhunwala added a commit that referenced this pull request Mar 13, 2025
…to show Load More… (#211176) (#214229)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Discover] Added context aware logic for logs view in discover to
show Load More…
(#211176)](#211176)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Achyut
Jhunjhunwala","email":"achyut.jhunjhunwala@elastic.co"},"sourceCommit":{"committedDate":"2025-03-12T12:39:27Z","message":"[Discover]
Added context aware logic for logs view in discover to show Load More…
(#211176)\n\n## Summary\n\nCloses -
https://github.com/elastic/kibana/issues/166679\n\n## What's included
?\n\n- The PR adds a feature in Logs View of Observability (to start
with) to\nhide the regular pagination toolbar from the footer and show
Load More\nonly when the user has scrolled to the bottom of the page.\n-
The table would always load the items in batches of default set 500 \n-
This PR also add 2 helper functions `useThrottleFn`
and\n`useDebounceFn`. Current React help library which KIbana uses
called\n-`react-use` does not have these and we cannot use Lodash
variant of\nthese. We need such hooks which are React safe. Hence added
these 2\n\n\n## What's pending ?\n\n- [x] Unit tests for the 2 new
helper React hooks\n- [x] Unit tests for data table footer component\n-
[x] Unit tests for Profile Resolution\n- [x] Functional Serverless
Tests\n- [x] Functional Stateful
Tests\n\n\n![Feb-14-2025\n15-25-18](https://github.com/user-attachments/assets/fa66de6e-b3bd-46b4-a0ed-e30c4209a695)\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Davis
McPhee <davismcphee@hotmail.com>\nCo-authored-by: Felix Stürmer
<weltenwort@users.noreply.github.com>\nCo-authored-by: Davis McPhee
<davis.mcphee@elastic.co>","sha":"591c5b73c00f8a4e5316aed9c1d32c7316e6dd34","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Feature:Discover","release_note:feature","Team:obs-ux-logs","Project:OneDiscover","backport:version","v9.1.0","v8.19.0"],"title":"[Discover]
Added context aware logic for logs view in discover to show Load
More…","number":211176,"url":"https://github.com/elastic/kibana/pull/211176","mergeCommit":{"message":"[Discover]
Added context aware logic for logs view in discover to show Load More…
(#211176)\n\n## Summary\n\nCloses -
https://github.com/elastic/kibana/issues/166679\n\n## What's included
?\n\n- The PR adds a feature in Logs View of Observability (to start
with) to\nhide the regular pagination toolbar from the footer and show
Load More\nonly when the user has scrolled to the bottom of the page.\n-
The table would always load the items in batches of default set 500 \n-
This PR also add 2 helper functions `useThrottleFn`
and\n`useDebounceFn`. Current React help library which KIbana uses
called\n-`react-use` does not have these and we cannot use Lodash
variant of\nthese. We need such hooks which are React safe. Hence added
these 2\n\n\n## What's pending ?\n\n- [x] Unit tests for the 2 new
helper React hooks\n- [x] Unit tests for data table footer component\n-
[x] Unit tests for Profile Resolution\n- [x] Functional Serverless
Tests\n- [x] Functional Stateful
Tests\n\n\n![Feb-14-2025\n15-25-18](https://github.com/user-attachments/assets/fa66de6e-b3bd-46b4-a0ed-e30c4209a695)\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Davis
McPhee <davismcphee@hotmail.com>\nCo-authored-by: Felix Stürmer
<weltenwort@users.noreply.github.com>\nCo-authored-by: Davis McPhee
<davis.mcphee@elastic.co>","sha":"591c5b73c00f8a4e5316aed9c1d32c7316e6dd34"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211176","number":211176,"mergeCommit":{"message":"[Discover]
Added context aware logic for logs view in discover to show Load More…
(#211176)\n\n## Summary\n\nCloses -
https://github.com/elastic/kibana/issues/166679\n\n## What's included
?\n\n- The PR adds a feature in Logs View of Observability (to start
with) to\nhide the regular pagination toolbar from the footer and show
Load More\nonly when the user has scrolled to the bottom of the page.\n-
The table would always load the items in batches of default set 500 \n-
This PR also add 2 helper functions `useThrottleFn`
and\n`useDebounceFn`. Current React help library which KIbana uses
called\n-`react-use` does not have these and we cannot use Lodash
variant of\nthese. We need such hooks which are React safe. Hence added
these 2\n\n\n## What's pending ?\n\n- [x] Unit tests for the 2 new
helper React hooks\n- [x] Unit tests for data table footer component\n-
[x] Unit tests for Profile Resolution\n- [x] Functional Serverless
Tests\n- [x] Functional Stateful
Tests\n\n\n![Feb-14-2025\n15-25-18](https://github.com/user-attachments/assets/fa66de6e-b3bd-46b4-a0ed-e30c4209a695)\n\n---------\n\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by: Davis
McPhee <davismcphee@hotmail.com>\nCo-authored-by: Felix Stürmer
<weltenwort@users.noreply.github.com>\nCo-authored-by: Davis McPhee
<davis.mcphee@elastic.co>","sha":"591c5b73c00f8a4e5316aed9c1d32c7316e6dd34"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Mar 22, 2025
…w Load More… (elastic#211176)

## Summary

Closes - elastic#166679

## What's included ?

- The PR adds a feature in Logs View of Observability (to start with) to
hide the regular pagination toolbar from the footer and show Load More
only when the user has scrolled to the bottom of the page.
- The table would always load the items in batches of default set 500 
- This PR also add 2 helper functions `useThrottleFn` and
`useDebounceFn`. Current React help library which KIbana uses called
-`react-use` does not have these and we cannot use Lodash variant of
these. We need such hooks which are React safe. Hence added these 2


## What's pending ?

- [x] Unit tests for the 2 new helper React hooks
- [x] Unit tests for data table footer component
- [x] Unit tests for Profile Resolution
- [x] Functional Serverless Tests
- [x] Functional Stateful Tests


![Feb-14-2025
15-25-18](https://github.com/user-attachments/assets/fa66de6e-b3bd-46b4-a0ed-e30c4209a695)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels Feature:Discover Discover Application Project:OneDiscover Enrich Discover with contextual awareness release_note:feature Makes this part of the condensed release notes Team:obs-onboarding Observability Onboarding Team v8.19.0 v9.1.0

10 participants