(Work in progress) Add Filebeat input documentation to agent

[dedemorton]

NOTE: This PR is not ready for review

Previews: https://observability-docs_424.docs-preview.app.elstc.co/guide/en/fleet/master/elastic-agent-input-configuration.html

What I need to do to get this ready for review:

• Fix the layout/format.
• Compare content to latest version of Filebeat (the content is very old).
• Make sure all lists are alphabetized.
• Make a pass at fixing the examples.
• Make sure content is correctly flagged as beta.
• Make sure changes/additions from Update documentation of filestream input with the new improvements beats#25303 get into these docs.
• Add harvester_buffer_size to topics where it is missing
• Add "settings" to each section title that contains settings reference info
• Add shared settings if we agree we want to repeat them
• Add topic about AWS credentials
• Add multiline settings.
• Add info about Kerberos
• Remove commented out links when the dependency is live

Dev input required:

• Identify which settings are valid for Elastic Agent
• Provide correct syntax and examples (current ones are Beats)
• Review the content (many descriptions have been edited)
• Verify types shown for settings and provide missing info
• Identify all required settings
• Identify missing defaults

TODO after merging:

• Document basic config file structure and contents - describe what an input is, what a stream is,

[ghost]

A documentation preview will be available soon:

• 📚 HTML diff
• 📘 Fleet and Elastic Agent Guide
• 📙 Observability Guide

[ph]

@ph's notes:

• verify the examples, provide example to dede.
• PH to commit directly on the PR.

[ruflin]

@ph I remember that this is not working and it has to go under streams currently?

[ph]

Yes exactly.

[dedemorton]

All of these examples are simply copied from the Beats documentation and need to be modified to follow the new agent syntax. I need help with that, though, because the examples should be reviewed and validated, too. Rather than adding comments, it's probably better for you to pull down the branch and then push the example changes to my fork. I think it's OK if we don't test everything for the first iteration, but we should have a plan to test all of the documented configs before GA. If there's something repetitive that needs to be changed, I can ask other folks on the doc team to help out.

(I had no idea we had SO MANY input settings in Filebeat. Some of the descriptions needed a lot of love, too, so this has been a lot more work than I anticipated.)

[dedemorton]

I've changed the chunk level to make it easier to navigate the settings reference. I think there are too many options for one long, scrolling page (especially in the current doc system where the navigation disappears). We can change this structure when we migrate to the new doc system later on. Here's what it looks like now:

[dedemorton]

This PR will need to be updated with new settings added in elastic/beats#25303.

[dedemorton]

@elasticmachine run elasticsearch-ci/docs

[mergify]

This pull request is now in conflict. Could you fix it @dedemorton? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b issue#23741 upstream/issue#23741
git merge upstream/main
git push upstream issue#23741

[mergify]

This pull request does not have a backport label. Could you fix it @dedemorton? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-/d./d is the label to automatically backport to the /d./d branch. /d is the digit
  NOTE: backport-skip has been added to this pull request.

[dedemorton]

@ph Let's revive this PR. Needs a review.

[dedemorton]

@bmorelli25 Remember this old thing? It feels like we're reviving Frankenstein's monster. I followed Kibana's lead and formatted this info in tables, but I see that starting with kibana issue 104482, they are using definition lists again.

I am super frustrated with how poorly tables are rendered in our current docs (especially when you have long qualified setting names), so I am very tempted to kill the tables. It'll take a little time to cleanup, but I think it might be worth it to avoid having to enter manual breaks to avoid display problems with text in the right-hand column. WDYT?

[ph]

@jlind23 @cmacknz Would be good to have someone from data plane to look into this, for some reason I was not able to add it to the data plane board, it might a permission issue?

[jlind23]

I've pinged @belimawr on it.

[bmorelli25]

I am super frustrated with how poorly tables are rendered in our current docs (especially when you have long qualified setting names), so I am very tempted to kill the tables. It'll take a little time to cleanup, but I think it might be worth it to avoid having to enter manual breaks to avoid display problems with text in the right-hand column. WDYT?

I'm looking through the Kibana updates and the definition lists greatly improve readability. At the same time, there's the potential for improvement in table functionality and customization in the new doc system. If the time to update this PR isn't too ridiculous, I say go for it. Otherwise we can wait to see if tables will be functional again in the future.

[dedemorton]

@belimawr Hold off on reviews for a day or so because I want to change the format of these docs. There are rendering issues in tables in our doc system that make the content difficult to read in places. I want to fix that. Also this content hasn't been update for a year, so I want to see if there is additional content that needs to be added.

[belimawr]

@belimawr Hold off on reviews for a day or so because I want to change the format of these docs. There are rendering issues in tables in our doc system that make the content difficult to read in places. I want to fix that. Also this content hasn't been update for a year, so I want to see if there is additional content that needs to be added.

No problem, just ping me here once it's ready for review.

[mergify]

This pull request is now in conflict. Could you fix it @dedemorton? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b issue#23741 upstream/issue#23741
git merge upstream/main
git push upstream issue#23741

[dedemorton]

Closing this issue because the content is quite out-of-date and out-of-sync with Beats. Whoever works on https://github.com/elastic/ingest-docs/issues/131 might want to look at this PR because I did edit a lot of the content.