Two Identical AKS Clusters, Two Different Elastic Metadata Bugs

https://hackernoon.imgix.net/images/ATuRYnOn0QNfPpDbuAtt2kf63vE2-a283cj2.png

Two AKS clusters used the equivalent Fleet processor configuration and produced different cluster metadata. One cluster produced an empty orchestrator.cluster.name. The other retained the configured name and looked correct in Kibana. I observed this behaviour with Elastic Agent 9.5.3.

The difference came from Azure identity. The cluster that looked correct had two user-assigned identities on its node pool. Elastic Agent could not choose one, so its Azure token request failed, and the cloud metadata processor skipped the update. The cluster with one identity obtained a token, completed an Azure Resource Manager request, found no matching cluster, and wrote an empty string. The failed credential path hid the metadata defect.

The Fleet processor configuration did not explain the difference

I began with the rendered agent configuration. Running elastic-agent inspect inside an agent pod shows the policy that the agent actually received, including the processors attached to each stream. This avoids concluding...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE