usage of add-mask still echoes the value to the log

[ericsampson]

Describe the bug
According to #159, the issue where the add-mask workflow command echoes/leaks the secret was supposed to be fixed, but we still observe it.
This was also mentioned on the GitHub forum by a Partner

To Reproduce
Steps to reproduce the behavior:
echo "::add-mask::${{ steps.mystep.outputs.myvalue }}"

Expected behavior
raw output is not echoed to the log

Runner Version and Platform
Hosted
Ubuntu

[Nyholm]

I can confirm this issue.

[Briggsdf]

Have there been any updates(Or workarounds) on this? It seems like they recently pushed a fix for this, but it doesn't seem to have resolved the issue.

[ericsciple]

The mask needs to be added before the output is registered.

Otherwise at the time when the inputs are evaluated for the next step, the value is not yet registered as a secret. It doesn't get registered until the next step executes. When the step inputs are logged, it hasn't yet been registered as a secret.

[ericsciple]

We may need to detect this specific case and error early, rather than evaluate the expression.

[Briggsdf]

OK, at what point in the workflow would that be?

[ericsciple]

In the example from the description, the step mystep sets the output. It should register the mask value before setting the output.

The output is streaming, so too late if value is already printed.

[ZebraFlesh]

The mask needs to be added before the output is registered

Given the following example using AWS credentials and the setup-terraform action:

    - name: Terraform Output - Access Key ID
      id: terraform-output-1
      run: terraform output access_key_id
    - name: Terraform Output - Secret Key
      id: terraform-output-2
      run: terraform output secret_access_key
    - run: |
        echo "::add-mask::${{ steps.terraform-output-1.outputs.stdout }}"
        echo "::add-mask::${{ steps.terraform-output-2.outputs.stdout }}"
        echo "::set-env name=test_key_id::${{ steps.terraform-output-1.outputs.stdout }}"
        echo "::set-env name=test_secret::${{ steps.terraform-output-2.outputs.stdout }}"

I tried changing the masking to the following:

    - run: |
        echo "::add-mask::${{ steps.terraform-output-1.outputs.stdout }}"
        echo "::add-mask::${{ steps.terraform-output-2.outputs.stdout }}"
    - name: Capture Terraform Outputs
      run: |
        echo "::set-env name=test_key_id::${{ steps.terraform-output-1.outputs.stdout }}"
        echo "::set-env name=test_secret::${{ steps.terraform-output-2.outputs.stdout }}"

But my AWS secrets are still showing up in the logs. Those values are dynamic and not known until terraform outputs them, so I'm not sure what else I could do.

[ericsampson]

It doesn't get registered until the next step executes. When the step inputs are logged, it hasn't yet been registered as a secret.

Regardless of anything else, this should be added to the documentation for add-mask. There's no mention that it's only effective for the step after it's registered.

[ZebraFlesh]

I guess I’m back to my original post then: what is the point of add-mask if it inherently exposes secrets?

[ericsciple]

@ericsampson sorry bad phrasing on my part. I was referring to the specific example when I said that. The statement on it's own is not true.

The add-mask command takes effect from the moment it is processed. The step writes the command over stdout to the runner. From the moment the runner processes the line, all future lines will be scrubbed.

@ZebraFlesh @ericsampson here is an example how add-mask should be used with secret outputs:

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          the_secret=$((RANDOM))
          echo "::add-mask::$the_secret"
          echo "::set-output name=secret-number::$the_secret"
      - run: |
          echo "the secret number is ${{ steps.sets-a-secret.outputs.secret-number }}"

If you set ACTIONS_STEP_DEBUG you will see lots of additional output.

The secret value will be masked everywhere. If you remove the add-mask command, you will see the secret value printed in many places - even when the set-output command is processed. So it is critical the add-mask command comes first.

Hope that explanation helps. Let me know.

[ericsampson]

Thanks @ericsciple!

I was able to do a little playing around, and have a modified version of your example that shows what I'm experiencing. Pretend that the the_secret output below in init is set by something like an Terraform action as in the more realistic examples above. With the following code, the local_secret line does print the secret to the log:

on:
  pull_request:
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: init
        run: |
          echo "::set-output name=the_secret::$((RANDOM))"
      - id: sets-a-secret
        run: |
          local_secret="${{ steps.init.outputs.the_secret }}"
          echo "::add-mask::$local_secret"
          echo "::set-output name=secret-number::$local_secret"
      - run: |
          echo "the secret number is ${{ steps.sets-a-secret.outputs.secret-number }}"     

So it seems to me like the issue is in the context expansion? It seems like it's being expanded first and then printed to the log, whereas something like $MYVAR or $(EXPRESSION) is printed to the log as is and expand later (which is desirable in the case of secrets).
Is there a way to write the context expression to avoid this early expansion?
I also tried it without the local, and just inline in the add-mask, but that didn't help:

... 
        run: |
          echo "::add-mask::${{ steps.init.outputs.the_secret }}"
          echo "::set-output name=secret-number::$local_secret"

[imjohnbo]

Hi @ericsampson! Just taking a look from the Terraform perspective and wanting to confirm this behavior isn't only after setup-terraform – from your example here, it looks like it's general?

[ericsampson]

Hi @imjohnbo, I'm still coming up to speed on GH Actions, but AFAICT this is a platform thing and not specific to Terraform. It just tends to get exposed (and mentioned) by people because this scenario naturally happens often when using Terraform (or other IaC). My latest example uses no 3rd-party Actions, just the built in platform stuff, and still demonstrates it. And also the first time I came across this, I wasn't using setup-terraform. Unless I'm just out to lunch. Thanks for caring!

[ericsampson]

Yes, that latest example/repo steps is fully self-contained and runnable, I didn't run it with a step using setup-terraform or other 3rd-party actions that could have messed up some internal Action runner state/settings, and then trim that out of the code shown here. That code sample demonstrates what I'm seeing.

[ericsciple]

@imjohnbo what do you think about the terraform wrapper supporting a -secret argument?

Puts the burden on the caller to indicate whether they expect the output to be a secret.

For example:

  steps:
    - run: terraform -secret output secret_access_key

The terraform wrapper would:

• Intercept the -secret flag and not pass it to the underlying command (terraform output secret_accesskey)
• Echo ::add-mask::the secret value before echoing ::set-output name=stdout::the secret value

I skimmed the terraform CLI docs. Doesn't look like any commands accept a -secret flag today.

[ericsampson]

this isn't limited to Terraform though, that's adressing one symptom rather than the issue which could come from a multitude of sources. It seems like it's not possible to mask any arbitrary context expression? Unless I'm mistaken and there is a way for the user to achieve this (without requiring every Action author everywhere to support inbuilt masking, which seems unrealistic).

[Briggsdf]

this isn't limited to Terraform though, that's adressing one symptom rather than the issue which could come from a multitude of sources. It seems like it's not possible to mask any arbitrary context expression? Unless I'm mistaken and there is a way for the user to achieve this (without requiring every Action author everywhere to support inbuilt masking, which seems unrealistic).

Agreed, the only sense I can make of this is that they want you to lock in with Github Secrets. But trying to mask anything in the GitHub context seems undoable. In my case I'm trying to mask a value sent in a Webhooks payload ${{github.event.client_payload.some_value}}

[ericsciple]

Thanks all for the feedback!

There are two ways to register a secret today:

1. When you use a repository secret, it gets masked automatically
2. Otherwise need to echo add-mask before outputting a secret. For example, if using set-output, need to echo add-mask first.

I would advise against adding secrets into the github.event.client_payload. That is a good feature request though.

/cc @chrispat fyi regarding ^ feature request

[ericsampson]

Thanks Eric, but how do you do point 2 for arbitrary context expressions? I'd love to understand, but either I'm missing something or we're talking by each other.

FWIW, regarding the Terraform Action, there is already a way in TF code to mark fields as containing sensitive/secret information, it just needs to get supported int the setup-terraform Action - no extra output argument required. @ZebraFlesh has already opened up this PR for that.

[ZebraFlesh]

@ZebraFlesh has already opened up this PR for that.

Issue, not PR 😛
edit: I also think it unlikely that it will be solved in that action. It seems to invalidate the entire design of the setup-terraform action (thin wrapper that just forwards things as outputs; now it needs to be smart and not so thin).

[ericsciple]

@ericsampson thank you for your patience

how do you do point 2 for arbitrary context expressions?

Sorry but it is not possible today. If the value is already in the context, it is too late.

Here are a few examples to illustrate. I'll summarize at the bottom.

Example 1: Debug output from set-output

This example illustrates why add-mask after set-output is too late. First, enable step debug logging. Then run:

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          echo "::set-output name=my-secret::$((RANDOM))"

Because debug logging is on, and because add-mask was not echoed first, the log contains:

::set-output name=my-secret::31487

Example 2: Action input with expression

Here is another example (debug logging not required):

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          echo "::set-output name=my-secret::$((RANDOM))"
      - uses: actions/checkout@v2
        with:
          not-a-real-input: ${{ steps.sets-a-secret.outputs.my-secret }}

Near the top of the logs for the checkout step, expand the folded line Run actions/checkout@v2 to see the inputs for the step. The inputs are printed to the log before the step executes. Because add-mask was not echoed, the log contains the secret in plain text.

Run actions/checkout@v2
  with:
    not-a-real-input: 32467
    repository: ericsciple/testing
    token: ***
    ssh-strict: true
    persist-credentials: true
    clean: true
    fetch-depth: 1
    lfs: false
    submodules: false

Example 3: run script with expression

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          echo "::set-output name=my-secret::$((RANDOM))"
      - name: My script only contains bash comments
        run: |
          # comment 1
          # comment 2 ${{ steps.sets-a-secret.outputs.my-secret }}

Near the top of the logs for the second step, expand the folded line Run # comment 1 to see the inputs for the step. The inputs are printed to the log before the script executes. Because add-mask was not echoed, the log contains the secret in plain text.

Run # comment 1
  # comment 1
  # comment 2 26441
  shell: /bin/bash -e {0}

Example 4: Default display name for a run step

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          echo "::set-output name=my-secret::$((RANDOM))"
      - run: |
          echo default step display name will the secret ${{ steps.sets-a-secret.outputs.my-secret }}

In the web UI you will see Run echo default step display name will the secret 31510. The default display name is calculated before the script executes. Because add-mask was not echoed, the default display contains the secret in plain text.

Example 5: Debug output from expression evaluation

First, enable step debug logging. Then run:

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          echo "::set-output name=my-secret::$((RANDOM))"
      - run: |
          echo hello
        env:
          MY_SECRET: ${{ steps.sets-a-secret.outputs.my-secret }}

Because debug logging is on, and because add-mask was not echoed first, the log contains debug output from the expression evaluation:

##[debug]Evaluating: steps.sets-a-secret.outputs.my-secret
[...]
##[debug]Result: '16195'

Summary

The key factors why add-mask must be called first:

• Logs are streaming to the web console (the runner scrubs the lines before sending)
• Context values can be printed to the log for a variety of reasons (critical add-mask echoed prior):
  • When step debug logging is enabled, the set-output command is logged (example 1 above)
  • Action inputs (i.e. uses) are printed to the log (example 2 above)
  • Inline scripts (i.e. run) are printed to the log (example 3 above)
  • Default display name for inline scripts (i.e. run) is calculated before the script executes (example 4 above)
  • Debug output from expression evaluation (example 5 above)

Hope that helps. Feedback is welcome.

I need to look over the docs, and see whether the guidance can be improved. Open to ideas and I can pass along.

I also wonder whether we should add a secret parameter for the set-output command. It may guide folks down the correct path. For example:

echo "::set-output name=my-secret, secret=true::the value"

[ericsciple]

(edited the above comment, added 2 more examples)

[ericsampson]

Thanks very much for the detailed information @ericsciple.

1. I'm going to open up a new feature request issue to add a new add-mask bool to set-output, to match the syntax of the standalone add-mask command:
   echo "::set-output name=my-secret, add-mask=true::the value"
   This will help make it so that every Action author (like the TF action) isn't forced to reimplement this. They can if it makes sense for their application and they choose to, but if not there is always the fallback for the user.
   1b) this will still not cover every case where a person might want to mask content in the wider non-output action context, like @Briggsdf's use case of webhooks. There are probably others.

2. I think it would be very valuable to add more information to the documentation somewhere to describe what gets executed before the script gets run, along the lines of your bullet points. Because this isn't obvious at all, without laboriously playing with scenarios : )

Along sort of similar lines, is there any mention in the documentation on the ability to run sub-shells in inline scripts? I only stumbled upon that in a Forum posting when trying to figure out how to do something.

[Constantin07]

I'm also trying to mask AWS credential variables but it fails authentication in the next step, despite the values are properly hidden in the workflow output:

      - name: Assume the 2nd role and overwrite the AWS env. vars
        run: |
          OUT=$(aws sts assume-role --role-arn arn:aws:iam::${{ env.AccountId }}:role/**** --role-session-name *** --external-id ***)
          AWS_ACCESS_KEY_ID=$(echo $OUT | jq -r '.Credentials''.AccessKeyId')
          echo "::add-mask::$AWS_ACCESS_KEY_ID"
          echo "AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID" >> $GITHUB_ENV
          AWS_SECRET_ACCESS_KEY=$(echo $OUT | jq -r '.Credentials''.SecretAccessKey')
          echo "::add-mask::$AWS_SECRET_ACCESS_KEY"
          echo "AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY" >> $GITHUB_ENV
          AWS_SESSION_TOKEN=$(echo $OUT | jq -r '.Credentials''.SessionToken')
          echo "::add-mask::$AWS_SESSION_TOKEN"
          echo "AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN" >> $GITHUB_ENV

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
        with:
          registries: ${{ env.registry }}

if I omit the masking, ECR login step works just fine. Does the add-mask somehow modifies the actual values ?

[jsoref]

@gr0vity-dev Here's an update to #475 (comment), it is definitely possible to do this with $GITHUB_OUTPUT as you can see in my new run of the same workflow: https://github.com/jsoref/actions-runner-issues-475/actions/runs/4020880990/jobs/6909255016

workflow

on: push
jobs:
  my-job:
    runs-on: ubuntu-latest
    steps:
      - id: sets-a-secret
        run: |
          the_secret=$((RANDOM))
          echo "::add-mask::$the_secret"
          echo "secret-number=$the_secret" >> "$GITHUB_OUTPUT"
      - run: |
          echo "the secret number is ${{ steps.sets-a-secret.outputs.secret-number }}"

raw logs

2023-01-27T02:05:02.2138028Z Requested labels: ubuntu-latest
2023-01-27T02:05:02.2138056Z Job defined at: jsoref/actions-runner-issues-475/.github/workflows/leak-secret-1.yml@refs/heads/test
2023-01-27T02:05:02.2138081Z Waiting for a runner to pick up this job...
2023-01-27T02:05:02.3518563Z Job is waiting for a hosted runner to come online.
2023-01-27T02:05:07.3882099Z Job is about to start running on the hosted runner: Hosted Agent (hosted)
2023-01-27T02:05:09.9067922Z Current runner version: '2.301.1'
2023-01-27T02:05:09.9094093Z ##[group]Operating System
2023-01-27T02:05:09.9094596Z Ubuntu
2023-01-27T02:05:09.9094968Z 22.04.1
2023-01-27T02:05:09.9095271Z LTS
2023-01-27T02:05:09.9095519Z ##[endgroup]
2023-01-27T02:05:09.9095844Z ##[group]Runner Image
2023-01-27T02:05:09.9096198Z Image: ubuntu-22.04
2023-01-27T02:05:09.9096521Z Version: 20230122.1
2023-01-27T02:05:09.9097047Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230122.1/images/linux/Ubuntu2204-Readme.md
2023-01-27T02:05:09.9097698Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230122.1
2023-01-27T02:05:09.9098100Z ##[endgroup]
2023-01-27T02:05:09.9098441Z ##[group]Runner Image Provisioner
2023-01-27T02:05:09.9098818Z 2.0.98.1
2023-01-27T02:05:09.9099077Z ##[endgroup]
2023-01-27T02:05:09.9100045Z ##[group]GITHUB_TOKEN Permissions
2023-01-27T02:05:09.9100702Z Actions: write
2023-01-27T02:05:09.9101047Z Checks: write
2023-01-27T02:05:09.9101511Z Contents: write
2023-01-27T02:05:09.9101849Z Deployments: write
2023-01-27T02:05:09.9102226Z Discussions: write
2023-01-27T02:05:09.9102498Z Issues: write
2023-01-27T02:05:09.9102800Z Metadata: read
2023-01-27T02:05:09.9103116Z Packages: write
2023-01-27T02:05:09.9103396Z Pages: write
2023-01-27T02:05:09.9103750Z PullRequests: write
2023-01-27T02:05:09.9104108Z RepositoryProjects: write
2023-01-27T02:05:09.9104432Z SecurityEvents: write
2023-01-27T02:05:09.9104762Z Statuses: write
2023-01-27T02:05:09.9105086Z ##[endgroup]
2023-01-27T02:05:09.9108597Z Secret source: Actions
2023-01-27T02:05:09.9109109Z Prepare workflow directory
2023-01-27T02:05:09.9931621Z Prepare all required actions
2023-01-27T02:05:10.0391318Z Complete job name: my-job
2023-01-27T02:05:10.1407707Z ##[group]Run the_secret=$((RANDOM))
2023-01-27T02:05:10.1408312Z �[36;1mthe_secret=$((RANDOM))�[0m
2023-01-27T02:05:10.1408961Z �[36;1mecho "::add-mask::$the_secret"�[0m
2023-01-27T02:05:10.1409419Z �[36;1mecho "secret-number=$the_secret" >> "$GITHUB_OUTPUT"�[0m
2023-01-27T02:05:10.1920593Z shell: /usr/bin/bash -e {0}
2023-01-27T02:05:10.1921012Z ##[endgroup]
2023-01-27T02:05:10.2773305Z ##[group]Run echo "the secret number is ***"
2023-01-27T02:05:10.2773827Z �[36;1mecho "the secret number is ***"�[0m
2023-01-27T02:05:10.2827404Z shell: /usr/bin/bash -e {0}
2023-01-27T02:05:10.2827788Z ##[endgroup]
2023-01-27T02:05:10.3053495Z the secret number is ***
2023-01-27T02:05:10.3173144Z Cleaning up orphan processes

[jsoref]

@Constantin07 you can test this theory by using something like (actual markup left as an exercise):

steps:
  - x='something'; echo $x; echo $x | shasum ; echo "x=$x" >> "$GITHUB_OUTPUT"
  - x="${{steps.previous.outputs.x}}"; echo $x; echo $x | shasum

[Constantin07]

Thanks @jsoref, already sorted out by using $GITHUB_OUTPUT and outputs with add-mask.

[douniwan5788]

Accidentally discovered the following undocumented feature that can be used as a workaround for masking sensitive data. GitHub Actions appears to automatically mask inputs / environment variables following certain naming conventions. For instance, a plaintext variable named WEBHOOK_TOKEN holding a JWT is masked same way as encrypted secrets would. It would be great to officially document this behavior along with the supported keywords to make it safe to rely upon.

GitHub Action configuration:

name: Test
on:
  workflow_dispatch:
    inputs:
      WEBHOOK_URL:
        description: 'Webhook URL'
        required: true
      WEBHOOK_METHOD:
        description: 'Webhook method'
        required: true
        default: 'GET'
      WEBHOOK_TOKEN:
        description: 'Webhook token'
        required: true
jobs:
  test:
    name: Test sensitive data masking
    runs-on: ubuntu-latest
    env:
      WEBHOOK_URL: ${{ github.event.inputs.WEBHOOK_URL }}
      WEBHOOK_METHOD: ${{ github.event.inputs.WEBHOOK_METHOD }}
      WEBHOOK_TOKEN: ${{ github.event.inputs.WEBHOOK_TOKEN }}
    steps:
      - name: Notify job start
        run: |
          curl -s -o /dev/null -w "%{http_code}\n" \
            -X "$WEBHOOK_METHOD" "$WEBHOOK_URL" \
            -H "Authorization: Bearer $WEBHOOK_TOKEN"

GitHub Action log:

I found out that this is done by MaskHint

runner/src/Runner.Worker/Worker.cs

Lines 167 to 183 in 5421fe3

	// Add mask hints
	foreach (MaskHint maskHint in (message.MaskHints ?? new List<MaskHint>()))
	{
	if (maskHint.Type == MaskType.Regex)
	{
	HostContext.SecretMasker.AddRegex(maskHint.Value);
	// We need this because the worker will print out the job message JSON to diag log
	// and SecretMasker has JsonEscapeEncoder hook up
	HostContext.SecretMasker.AddValue(maskHint.Value);
	}
	else
	{
	// TODO: Should we fail instead? Do any additional pains need to be taken here? Should the job message not be traced?
	Trace.Warning($"Unsupported mask type '{maskHint.Type}'.");
	}
	}

and currently github send these MaskHints with the job
\bv1\.[0-9A-Fa-f]{40}\b
\bgh[pousr]{1}_[A-Za-z0-9]{36}\b
\bgithub_pat_[0-9][A-Za-z0-9]{21}_[A-Za-z0-9]{59}\b
\b(?:eyJ0eXAiOi|eyJhbGciOi|eyJ4NXQiOi|eyJraWQiOi)[^\s'";]+
\bBearer\s+[^\s'";]+
\b(?i:Password|Pwd)=(?:[^\s'";]+|"[^"]+")
-(?i:Password|Pwd)\s+(?:[^\s'";]+|"[^"]+")
(?:[a-zA-Z][a-zA-Z\d+-.]*):\/\/([a-zA-Z\d\-._~\!$&'()*+,;=%]+):([a-zA-Z\d\-._~\!$&'()*+,;=:%]*)@

[c3-yuhsuan]

I'm also trying to mask AWS credential variables but it fails authentication in the next step, despite the values are properly hidden in the workflow output:

      - name: Assume the 2nd role and overwrite the AWS env. vars
        run: |
          OUT=$(aws sts assume-role --role-arn arn:aws:iam::${{ env.AccountId }}:role/**** --role-session-name *** --external-id ***)
          AWS_ACCESS_KEY_ID=$(echo $OUT | jq -r '.Credentials''.AccessKeyId')
          echo "::add-mask::$AWS_ACCESS_KEY_ID"
          echo "AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID" >> $GITHUB_ENV
          AWS_SECRET_ACCESS_KEY=$(echo $OUT | jq -r '.Credentials''.SecretAccessKey')
          echo "::add-mask::$AWS_SECRET_ACCESS_KEY"
          echo "AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY" >> $GITHUB_ENV
          AWS_SESSION_TOKEN=$(echo $OUT | jq -r '.Credentials''.SessionToken')
          echo "::add-mask::$AWS_SESSION_TOKEN"
          echo "AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN" >> $GITHUB_ENV

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
        with:
          registries: ${{ env.registry }}

if I omit the masking, ECR login step works just fine. Does the add-mask somehow modifies the actual values ?

I faced the same issue. Any solution for this?

[ericsampson]

@Rahulkhinchi03 please do not use GitHub Issues to advertise your product.

[pascalgulikers]

"It doesn't work on my machine"

    - name: Login to Amazon ECR
       id: login-ecr
       uses: aws-actions/amazon-ecr-login@v1
     
     - name: Fetch Docker credentials
       id: fetch-docker-creds
       shell: bash
       run: |
         docker_creds="${{ toJSON(steps.login-ecr.outputs) }}"
         echo "::add-mask::$docker_creds"
         echo $docker_creds

outputs:

I looks like it's only tested against a runtime variable generating case, i.e. $((RANDOM)).

Solution would be that the output of the echo "::add-mask" command itself should be hidden as well.

[jsoref]

docker_creds is a multiline string. Masks are line oriented things. You'll want to do something like:

docker_password=$(echo "$docker_creds" | jq -r .docker_password_SOMETHING )
echo "::add-mask::$docker_password"

[pascalgulikers]

docker_creds is a multiline string. Masks are line oriented things. You'll want to do something like:

docker_password=$(echo "$docker_creds" | jq -r .docker_password_SOMETHING )
echo "::add-mask::$docker_password"

Thank you for your quick reply @jsoref , the problem is that the first line of your example is already in the logs, including the output of the jq command

EDIT: actually my unmasked line is
echo "docker_username=`jq -n '${{ toJSON(steps.login-ecr.outputs) }}' | jq '. | with_entries(select(.key | startswith(\"docker_username\")))[]'`" >> $GITHUB_OUTPUT

But how to mask it before sending it to $GITHUB_OUTPUT?

[jsoref]

don't do it like that.

use:

env:
  login_ecr_outputs: ${{ steps.login-ecr.outputs }}

or something ... you may need to cast a toJSON on it, or you may be able to actually dig into it (dunno, but if you can manage to dig into it and get just the thing you want, you'll be happier)

[ericsampson]

you can always base64 it, to get a single-line maskable string out of a multiline blob.
if it's good enough for Kubernetes... :D

[pascalgulikers]

@ericsampson I've put in a FR: aws-actions/amazon-ecr-login#483

[pascalgulikers]

@jsoref I've tried with env, but unfortunately

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v1
    
      - name: Fetch Docker credentials
        id: fetch-docker-creds
        shell: bash
        run: |
          echo "login_ecr_outputs=${{ steps.login-ecr.outputs }}" >> $GITHUB_ENV

  job-2:
    environment: ${{ needs.Initialize.outputs.environment }}
    runs-on: ubuntu-latest
    needs: [Initialize, Setup, login-to-amazon-ecr]
    permissions:
      contents: write
      packages: write
      pull-requests: write
      # This is used to complete the identity challenge
      # with sigstore/fulcio when running outside of PRs.
      id-token: write
    
    steps:  
      - name: Fetch Docker credentials
        id: fetch-docker-creds
        shell: bash
        run: |
        
          echo ${{ env.login_ecr_outputs }}

It's empty because I'm writing to env at the steps level. Can't assign it at job or workflow level since the variable is not existing at that moment, i.e.

env:
  login_ecr_outputs: ${{ steps.login-ecr-outputs }}

I get the error that "steps" is not recognised

[jsoref]

No No No.

• Don't use steps[*].run to set an env your thing.
• Do use steps[*].env to define them.

And note that you can't reach env from a different job. You could perhaps use GITHUB_OUTPUT.

[pascalgulikers]

I used GITHUB_OUTPUT but the value isn't masked

[jsoref]

But you put the string into the command instead of the env: block.

[pascalgulikers]

@jsoref nevermind, the problem is in the used upstream action:
aws-actions/amazon-ecr-login#485

[sourceful-karlson]

Just want to share some code here for anyone that got stuck from this, taken from here google-github-actions/get-secretmanager-secrets#288

	  - id: fetch-secret
	    uses: google-github-actions/get-secretmanager-secrets@v2
	    with:
	      secrets: |-
	        secrets:projects/${{ inputs.projectId }}/secrets/${{ inputs.dbSecretName }}/versions/latest
	
	  - id: pg-password
	    name: mask the password
	    env:
	      SECRET: '${{ steps.fetch-secret.outputs.secrets }}'
	    run: |
	      VALUE1="$(jq .key1 <<< "${SECRET}")"
	      VALUE2="$(jq .key2 <<< "${SECRET}")"
	      echo "::add-mask::${VALUE1}"
	      echo "::add-mask::${VALUE2}"

[tuxillo]

@sourceful-karlson I'd say the secret value is going to be shown in the details of the run, in the env section, unless that google action does something I don't know

[Croydon]

I don't understand how this issue gains no traction. The existence of the function add-mask, which does not fulfill its purpose at all, gives people a false sense of security.

The function should be fixed ASAP with high priority or be deleted.

[ericsampson]

@Croydon I don't know how to get more attention from someone at GitHub. Get a popular YTer like @ThePrimeagen to upload a video blasting this security issue that hasn't been addressed for four years?

[p-pal]

New to GH Actions, but having the same issue.

Just trying to re-iterate and simply present it again for attention.

jobs:
  printing-secret:
    runs-on: ubuntu-latest
    steps:
    - name: generate
      id: generate
      run: | 
        secret=`echo $RANDOM`
        echo "::add-mask::$secret"
        echo "secret=$secret" >> $GITHUB_OUTPUT
    - name: print
      run: |
        echo "generated secret is : ${SECRET}"
      env:
        SECRET: ${{ steps.generate.outputs.secret }}
    outputs:
     secret: ${{ steps.generate.outputs.secret }} 
  fetch-secret:
    runs-on: ubuntu-latest
    needs: [printing-secret]
    steps:
    - name: fetch
      id: fetch
      run: |
        echo "fetched secret is : ${FETCHED_SECRET}"
      env:
        FETCHED_SECRET: ${{ needs.printing-secret.outputs.secret }}

It performs masking for steps under a job (and won't show-up when debugging is enabled)
printing-secret -> print

generated secret is : ***

But doesn't fetch any value when passed between the jobs with add-mask.
fetch-secret:

fetched secret is :

[jsoref]

@p-pal, the first job triggers:

Warning: Skip output 'secret' since it may contain secret.

https://github.com/actions/runner/blame/43d67e46db3fedfd8247d8d6fb968a4e11765643/src/Runner.Worker/JobExtension.cs#L565

Oddly, I can't find any documentation that warns about this behavior. But it is clearly a design feature.

Defining outputs for jobs

Job outputs containing expressions are evaluated on the runner at the end of each job. Outputs containing secrets are redacted on the runner and not sent to GitHub Actions.

GitHub's documentation does include information about how to pass secrets between jobs:

https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#example-masking-and-passing-a-secret-between-jobs-or-workflows

(I wrote that documentation a while ago...) It includes some hand waving, so you'll have to fill in the details using your preferred magic.