Name prefix "aws-sigv4-proxy" to the prometheus server pod causing failure when remotewrite to AMP

[jagadishbb]

Setup:
kube-prometheus-stack, prometheus-operator, aws-sigv4-proxy for remoteWrite to AMP.

When using the kube-prometheus-stack and deploying the prometheus server with the below overrides

kube-prometheus-stack:
enabled: true
prometheus:
serviceAccount:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam:::role/EKS-AMP-ServiceAccount-Role
prometheusSpec:
containers:
- name: aws-sigv4-proxy-sidecar
image: .dkr.ecr.us-east-1.amazonaws.com/aws-observability/aws-sigv4-proxy:1.0
args:
- --name
- aps
- --region
- us-east-1
- --role-arn
- arn:aws:iam:::role/EKS-AMP-Central-Role
- --host
- aps-workspaces.us-east-1.amazonaws.com
- --port
- :8005
ports:
- name: aws-sigv4-proxy
containerPort: 8005
remoteWrite:
- url: http://localhost:8005/workspaces//api/v1/remote_write

Hitting an issue where the prefix "aws-sigv4-proxy" is adding to the already existing calculated length of kube-prometheus-stack and causing the roleArn to go over 64 chars and failing.

ts=2022-02-22T10:59:04.797Z caller=dedupe.go:112 component=remote level=warn remote_name=ec6d05 url=http://localhost:8005/workspaces//api/v1/remote_write msg="Failed to send batch, retrying" err="server returned HTTP status 502 Bad Gateway: unable to proxy request - ValidationError: 1 validation error detected: Value 'aws-sigv4-proxy-prometheus-AAA-monitoring-kube-promet-prometheus-0' at 'roleSessionName' failed to satisfy constraint: Member must have length less than or equal to 64"

Here the roleSessionName - aws-sigv4-proxy-prometheus-AAA-monitoring-kube-promet-prometheus-0

"AAA-monitoring" is the release name.
"prometheus-AAA-monitoring-kube-promet-prometheus-0" - server pod name

If I shorten the release name, the template code in kube-prometheus-stack adds characters to pod name based on the logic it has, so that doesnt help.

Is there a significant reason to have the hardcoded prefix (aws-sigv4-proxy) ?
Anyway to configure or override the prefix?

[alvinlin123]

I think the reason was simply to make it clear that the role was from the aws-sigv4-proxy. We can definitely make that behaviour configurable. I am thinking a flag called "session-name", if not set the current behaviour is use, else, use the configured value. How does that sound? @jagadishbb

I also welcome PRs from anyone :)