[cherry-pick][1.44]Reconfigure Linseed and linseed user when migrating from multi-index …

[asincu]

…to single-index (#5132)

• (bootstrap): Add knob to reconfigure Linseed indices for a single tenant cluster

This will be set operator bootstrap config map and enabled when migrating from multi-index format to single-index format. Linseed will reconfigure its environment variables to set the correct backend for the indices it is using and also set the base index name used for Cloud.

• (logstorage-users): Provision Elasticsearch users from the operator while migrating to single-index storage

Single-tenant clusters migrating to single-index storage need Linseed to hold RBAC for the new calico_* indices. es-kube-controllers cannot grant that, so during the migration the operator's log-storage users controller takes over user provisioning and es-kube-controllers stops running its elasticsearch configuration controller.

• Run the users controller in single-tenant mode when IndexMigration is set, building the tenant configuration from the cloud config ConfigMap since single-tenant clusters have no Tenant resource.
• Name the single-tenant Linseed and Dashboards users the way es-kube-controllers named them (--secure), and repoint existing credential secrets at those users while keeping their passwords, so credentials provisioned before the migration keep resolving.
• Declare the standard single-index names on the Tenant that CloudConfig.ToTenant builds, gated on the caller opting in, so that clusters which are not migrating keep falling back to their existing index names. Sort the declared indices, as they are generated from a map.
• Report the users TigeraStatus in the log-storage conditions aggregate while migrating.

• (logstorage-conditions): Stop the log-storage conditions controller from writing on every reconcile

updateConditions built its result by ranging over the desiredConditions map, so the order of LogStorage.Status.Conditions was randomized on every reconcile. Conditions is an atomic list, so a reorder is a real change to the stored object: each reconcile bumped the resourceVersion, and since this controller also watches LogStorage, that re-enqueued itself. The write loop ran continuously, and reconciles fired faster than the informer cache could converge - so reconciles read a stale tigera-secure and their status updates were rejected with "the object has been modified".

Sort the conditions by type so the stored list is stable - an unchanged reconcile then computes a list identical to the stored one, which the API server discards without bumping the resourceVersion - and requeue instead of erroring when an update does hit a conflict.

• (fix): Use CalicoEnterprise instead of TigeraEnterprise

• (logstorage-users): Provision Elasticsearch users for all single tenant clusters

Calico Cloud single-tenant clusters had their Elasticsearch users provisioned by es-kube-controllers, and the operator only took over while migrating to single-index storage. Take over for all of them, and grant Linseed access to the indices its cluster actually stores data in rather than to both name formats.

• Run the log-storage users controller for every Calico Cloud single-tenant cluster, and stop es-kube-controllers running its elasticsearch configuration controller there so that it does not overwrite the users we own. Wait for the cloud config ConfigMap rather than erroring when it is not there yet.
• Derive the Linseed role's index privileges from the tenant: the declared base index names when the cluster stores data in single-index format, the multi-index names otherwise - dropping the tenant qualifier for clusters on their own Elasticsearch, whose indices do not carry it.
• Report the users TigeraStatus in the log-storage conditions aggregate for Calico Cloud rather than only while migrating.
• Rename the index migration knob to USE_SINGLE_INDEX / UseSingleIndex, matching the naming already used by the linseed controller and render code.
• Move CloudStandardIndices out of the API module and unexport it, as it is only consumed when building the single-tenant Tenant from the cloud config.

• (logstorage-users): Address review feedback on Linseed index privileges

• Skip indices with an empty base index name when building the Linseed role's index privileges. A Tenant declaring an index without a base name - whether misconfigured, or carrying a DataType added later without a mapping - would otherwise be wildcarded into "*", granting Linseed access to every index in Elasticsearch. Fall back to Linseed's default calico_ names when no usable base index name remains, which is what a Tenant declaring no indices at all already got.
• Fix a comment on the single-tenant Linseed backend, which described the branch as migrating to multi-tenant style indices when it is gated by UseSingleIndex and migrating to single-index storage.

• (fix): grant Linseed access to the policy activity index

Policy activity is only ever stored in single-index format, so it is named outside the tigera_secure_ee_ pattern even on clusters that have not moved to single-index storage. The multi-index role we provision granted only that pattern, and since the role name matches the user name, putting it replaced the definition es-kube-controllers wrote - which did include the index. Linseed was then denied indices:admin/aliases/get when ingesting policy activity logs.

Grant calico_policy_activity* alongside the multi-index pattern. The wildcard covers both Linseed's default index name and the calico_policy_activity_standard name pinned for clusters sharing an external Elasticsearch.

• (fix): mark operator-provisioned Elasticsearch users as system users

es-kube-controllers' authorization controller sweeps Elasticsearch on every resync and deletes any user that is neither marked with the "system:serviceaccount" full_name nor present in its OIDC user cache. The operator's User struct had no full_name field at all, so the Linseed and Dashboards installer users it provisions were created with an empty one and swept within a resync period, then recreated on the next reconcile - leaving the user flapping while both components are enabled.

Add FullName to User, set it on both users, and send full_name in the create request. It is only sent when non-empty: the request replaces the user document, so sending it empty would strip the marker off a user that es-kube-controllers had already created with it.

• (test): Make WithStandardIndices idempotent and cover it

WithStandardIndices appended an entry for every data type without checking what the Tenant already declared, so applying it more than once - or alongside an option that declares its own indices - duplicated entries. Skip data types that are already present, letting an explicitly declared base index name win over the standard one rather than being duplicated or overridden.

No current caller hits this: TenantFromCloudConfig always starts from a tenant with no indices, and every call site passes at most one option. This is hardening of a shared helper, not a behaviour change for any caller.

Cover both cases - applying the option twice, and applying it after an option that already declared an index. Both specs fail against the previous implementation.

---

Description

Release Note

TBD

For PR author

• Tests for change.
• If changing pkg/apis/, run make gen-files
• If changing versions, run make gen-versions

For PR reviewers

A note for code reviewers - all pull requests must have the following:

• Milestone set according to targeted release.
• Appropriate labels:
  • kind/bug if this is a bugfix.
  • kind/enhancement if this is a a new feature.
  • enterprise if this PR applies to Calico Enterprise only.

[Copilot]

Pull request overview

This PR updates the operator’s Calico Cloud log-storage flow to better support single-tenant clusters during (and after) migration from multi-index to single-index storage, including Linseed backend/index configuration, Elasticsearch user provisioning, and status-condition stability.

Changes:

• Introduces a UseSingleIndex bootstrap/controller option and wires it through Linseed rendering to select the single-index backend and configure index base names.
• Moves Elasticsearch user provisioning for Calico Cloud single-tenant clusters to the operator (and disables es-kube-controllers’ elasticsearch configuration controller there), including backward-compatible user naming and credential-secret repointing.
• Stabilizes LogStorage status conditions by sorting and avoids hot-looping on status update conflicts.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cmd/main.go	Reads USE_SINGLE_INDEX from bootstrap config and passes it into controller options.
pkg/common/discovery/discovery.go	Adds bootstrap discovery helper UseSingleIndex.
pkg/controller/options/options.go	Adds UseSingleIndex to controller options for wiring through controllers/rendering.
pkg/controller/logstorage/linseed/linseed_controller.go	Threads UseSingleIndex into cloud single-tenant tenant-building and Linseed render config.
pkg/controller/logstorage/linseed/linseed_controller_test.go	Updates controller shim setup to use the consolidated opts field.
pkg/render/logstorage/linseed/linseed.go	Adds UseSingleIndex rendering behavior to select backend + index env vars for single-tenant migrations.
pkg/controller/logstorage/users/users_controller.go	Runs users controller in cloud single-tenant mode; derives tenant config from ConfigMaps; provisions/repoints ES users and secrets.
pkg/controller/logstorage/users/users_controller_test.go	Adds coverage for single-tenant cloud user provisioning, secret repointing, and index privilege selection.
pkg/controller/logstorage/initializer/conditions_controller.go	Sorts status conditions and requeues on conflict to prevent reconcile hot loops; includes users status for Cloud single-tenant.
pkg/controller/logstorage/esutils/elasticsearch.go	Adjusts Linseed/Dashboards user naming, roles/index privileges, and marks operator users as system users (full_name).
pkg/controller/logstorage/esutils/elasticsearch_test.go	Adds tests for full_name handling and single-/multi-tenant role/index behavior.
pkg/enterprise/utils/cloudconfig.go	Adds GetClusterID helper and ClusterInfoConfigMap constant for cluster-id lookup.
pkg/enterprise/utils/tenant.go	Adds Tenant options (WithStandardIndices*) and deterministic index ordering when building a single-tenant Tenant from CloudConfig.
pkg/enterprise/utils/tenant_test.go	Adds tests for TenantFromCloudConfig options and idempotent standard-index declaration.
pkg/enterprise/kubecontrollers/kubecontrollers.go	Disables elasticsearchconfiguration controller in Calico Cloud to avoid user overwrites.
pkg/enterprise/kubecontrollers/es_kube_controllers_test.go	Adds test asserting elasticsearchconfiguration is disabled in Cloud.
pkg/controller/logstorage/dashboards/dashboards_controller.go	Updates comment to reflect cloud credentials provisioning source.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.