Extend kibana-system permissions to manage security entities

[kubasobon]

As part of the entity store feature we need the Kibana system user to be able to access .entities.* indices and manage .entities.*.history* indices.

What is the entity store?

The entity store is a new security feature which extracts entities (hosts & users) from logs and metrics data.

The documents in the .entities.v1.latest.security* index each represent an entity extracted from event data, properties are added as we see them over time, for example for a host we store ip, mac, and OS information, for a user we store things like email, name, roles. Documents in .entities.v1.history.* indices represent historical snapshots of entities at certain points in time.

[elasticsearchmachine]

@kubasobon please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[elasticsearchmachine]

Hi @kubasobon, I've created a changelog YAML for you.

[kc13greiner]

Heya @kubasobon !

Thank you for the detailed description 🚀

2 questions:

• Does the kibana_system user need read/write to all .entities.* or could it be limited to .entities.*security*?
• Does the kibana_system users need the create/manage for .entities.*history*?

Just looking for clarification for these additional privileges

[kubasobon]

Hi @kc13greiner, and thank you for taking a look here.

• We were looking to widen the permissions to .entities.*, but I can't see the harm in setting it to .entities.*security*. cc: @hop-dev, @romulets WDYT?
• Yes it does, it is used by a Task Manager task that creates snapshot indices on a schedule. You can refer to [Security Solution][Entity Store] Add history snapshots to Entity Store kibana#233970 or ping me if you need more detail on how we aim to use this permission setting.

[kc13greiner]

@kubasobon

We were looking to widen the permissions to .entities.*, but I can't see the harm in setting it to .entities.security. cc: @hop-dev, @romulets WDYT?

This would be better, if possible. Let me know what you and the team decide

[kubasobon]

@kc13greiner I went back and checked, unfortunately some of our indices (already in use) include .entities.v1.latest.noop, so .entities.*.security* pattern would not work for us.

[jeramysoucy]

@kubasobon Just chiming in as @kc13greiner is on PTO

I went back and checked, unfortunately some of our indices (already in use) include .entities.v1.latest.noop, so .entities..security pattern would not work for us.

Would it be a pain to create patterns for the applicable security index patterns? e.g. .entities.v1.latest*, .entities.*.security*, etc.
Or am I misunderstanding the scope here, and is everything in .entities.* applicable?

[kubasobon]

@jeramysoucy Thanks for stepping in! I have widened the scope from .entities.v1.latest.security* to .entities.* as this allows us to cover all the current Entity Store needs:

• .entities.v1.latest.*
• .entities.v1.history.* ([Security Solution][Entity Store] Add history snapshots to Entity Store kibana#233970)
• .entities.v1.reset.* ([Security Solution][Entity Store] Add history snapshots to Entity Store kibana#233970)
• .entities.v1.updates.* (in development: [Entity Store] Add Upsert Entity API kibana#234454)
• .entities.v1.latest.noop
  ...and possibly others under .entities prefix in the coming weeks. If you would prefer me adding those permissions to kibana-system as explicitly as possible, I'm happy to oblige. I did not see the harm in covering read/write access with a single clause.

.entities.*history* is a bit of an outlier and needs extra permissions, since it's entirely managed by Task Manager tasks.

[jeramysoucy]

@kubasobon Thanks for the explanation. We're ok with this, given the needs you've stated above. Thanks for your patience!

[kubasobon]

@jeramysoucy & @kc13greiner Sorry to bother you again gentlemen, but it seems I missed adding the new index pattern to viewer and editor roles (it is meant to have exact same permissions as latest pattern). Also added kibana-system permissions for reset index, which I mentioned in the list, but forgot to push.

[kubasobon]

Extended dot-index exemption for .entities indices based on #116266