Remediating a leaked secret in your repository

Learn how to respond effectively to a leaked secret in your GitHub repository.

Introduction

Secrets, such as API keys, tokens and credentials, can pose significant security risks to your team and organization if inadvertently exposed in your codebase or stored improperly.

You should consider any leaked secret to be immediately compromised and it is essential that you undertake proper remediation steps, such as revoking the secret. Simply removing the secret from the codebase, pushing a new commit, or deleting and recreating the repository do not prevent the secret from being exploited.

This tutorial walks you through what to do if you've accidentally committed a secret to your repository, or if you've been alerted to a secret leak in your repository.

Prerequisites

• You have at least write access to the repository.
• Optional: Secret scanning is enabled for the repository.

  Note

  Secret scanning is free for public repositories. It is available as part of GitHub Secret Protection for private repositories on GitHub Team and GitHub Enterprise Cloud plans.

Step 1. Identify the secret and gather context

Gather as much information as you can about the leaked secret. This will help you assess the risk and determine the best course of action for remediation.

1. Determine the secret type and its provider.
   • For example, is the secret a GitHub personal access token (PAT), an OpenAI API key, an SSH private key?
2. Locate the repository, file and line that contains the leaked secret.
3. Identify the secret owner. This is the person or team who created, or is responsible for, the secret.
   • Check the CODEOWNERS file of the repository to determine the responsible team.
   • Use git log -S to help search the commit history of your repository to identify who committed the secret.

Step 2. Assess risk

How you approach remediation will be determined by the risk factors associated with the secret leak.

Secret scanning can help you assess the risk associated with an alert, but if you don't yet have secret scanning enabled, you can still perform a risk assessment based on the information available to you.

Option 1. Secret scanning is enabled

Review the secret scanning alert associated with the leak, checking the alert labels and any available metadata:

1. Check the secret's validity status to determine if the secret is still active. The alert will include a status that describes whether the secret is active, inactive, or if its validity is unknown.

   Note

   • Validity checks are only available for certain secret types. To check if your secret type is supported, see Supported secret scanning patterns.
   • The secret provider is always the most reliable source of truth for determining the validity of a secret.
2. Check for the public exposure label to determine if the secret was leaked in a public repository.
3. Check for the multiple leaks label to determine if the secret is exposed in multiple locations.
4. If the secret is a GitHub PAT, check the alert metadata for any information on when the secret was last used and its access scope.
5. Assess which services or applications depend on the secret, and consider the potential for downtime or disruption if you were to immediately revoke the secret.

Option 2. Secret scanning is not enabled

If you don't yet have secret scanning enabled for the repository, perform a risk assessment based on the following:

1. Check the repository's visibility. Is the repository public?
2. Look for indications that the secret has been used recently. Are there any recent commits or pull requests that reference the secret? Are there any logs or audit trails that show the secret being used?
3. Assess the file containing the secret and the surrounding context. Is the secret used in a production deployment script (higher risk) or a test file (lower risk)? Is the secret associated with a database credential or admin key (higher risk)?
4. Assess which services or applications depend on the secret, and consider the potential for downtime or disruption if you were to immediately revoke the secret.

Step 3. Strategize remediation

The next step depends on the risk assessment you performed in the previous step.

Act quickly for high-risk secrets

Automated scanners can locate publicly exposed secrets in minutes. They can be exploited by malicious actors within hours. The longer that an active secret is left exposed, the greater the potential for serious breaches.

If the secret is high risk (that is, the secret is still active, is exposed in a public repository, or is a production credential), we would recommend that you:

1. Prioritize revoking the secret immediately. See Step 4.

   If you are concerned about downtime to services, you may want to first generate a new secret with the same permissions, make the application start using the new token, and *then* revoke the old secret.

2. Communicate with the secret owner (identified in Step 1), repository administrators, and security leads during or after revocation.

Plan for medium to low-risk secrets

If the secret is medium to low risk (that is, the secret is no longer active, is exposed in a private repository, or is a test or development credential), you can plan the remediation strategy accordingly:

1. Using the information gathered in Step 1, locate the responsible team for the secret and alert them to the secret leak.
2. Explain what was leaked and when. Explain that you'll need to revoke the secret, generate a new secret and that affected services will require updating.
3. Inform the repository administrators and security leads about the leak, explaining any remediation actions needed or already taken.
4. Plan a time for revocation and rotation together with the appropriate team to coordinate a smooth transition.

It is important to remediate even medium- to low-risk secrets, as they can still pose a risk to both security and compliance if left exposed.

Step 4. Revoke the secret

It is not sufficient to simply remove the secret from your codebase. The most important remediation step is revoking the secret with the secret's provider. By revoking the secret, you drastically reduce the potential for the secret to be exploited.

1. Using the information gathered in Step 1, locate the secret provider's website or documentation.

2. Follow the provider's instructions for revoking the secret. This typically involves logging into the provider's portal and navigating to the section where the secret is managed.

   If you lack access to the provider portal, contact the secret owner or relevant repository administrator to get help with revoking the secret.

3. Generate a new secret, if necessary, to replace the revoked secret. This is often required to restore functionality for services that relied on the original secret.

Step 5: Identify and update affected services

Next, you need to coordinate updates to all affected services using the leaked secret and update them with the new secret.

Identify

1. Use GitHub's code search to check all code, issues, and pull requests for the secret.
   • Search across your organization using org:YOUR-ORG "SECRET-STRING".
   • Search your repository using repo:YOUR-REPO "SECRET-STRING".
2. Check the repository's stored deploy keys and secrets and variables.
   • Click "Settings," then under "Security," click Secrets and variables or Deploy keys.
3. Check for any installed GitHub Apps and integrations that may use the secret.

Coordinate

1. Instruct Copilot to create issues (and sub-issues) for each task involved in updating an affected service. See Using GitHub Copilot to create or update issues.
2. If multiple stakeholders are involved, create a project board for the issues to track progress and facilitate communication.

Update and verify

1. Update your application with the new secret, ensuring that your application properly uses the new credential.

   A safe way of providing sensitive credentials to your application is through a vault. For example, you can make sensitive credentials available to GitHub actions and workflows through the "Secrets and variables" store under your repository's settings page.

2. Test affected services to ensure they are functioning correctly with the new secret.

Step 6. Check for unauthorized access

Once services are back up and running, it's important to check for any unauthorized access that may have occurred while the secret was exposed.

1. Review GitHub's audit logs for events relating to the secret and its usage.

   • Security log for your personal account. See Reviewing your security log.
   • Audit log for your organization. See Reviewing the audit log for your organization.
   • Audit log for your enterprise. See Audit log events for your enterprise.

   For the organization- and enterprise-level audit logs, you can specifically search for events related to an access token. See Identifying audit log events performed by an access token (organizations) and Identifying audit log events performed by an access token (enterprises).

   Access to GitHub's audit logs depends on your role, so you may need to contact an organization owner or enterprise administrator if you don't have the necessary permissions.

2. Review the secret provider's audit logs.

   • For example, for Amazon Web Services (AWS) secrets, you can check the CloudTrail logs for any unauthorized access attempts using the leaked secret. See What Is AWS CloudTrail? in the AWS CloudTrail documentation.

Step 7. Clean the repository

Although you've now revoked and updated the secret in your codebase, the secret may still exist in your repository's commit history. Ideally, you should search for and remove all instances of the secret from your repository.

However, cleaning Git history can be a destructive and disruptive process, as it may involve force pushing changes to the repository.

Together with your repository's security leads, consider carefully the effects of cleaning the repository's history against your compliance or security obligations. See Removing sensitive data from a repository.

Step 8. Resolve the alert

1. Close the secret scanning alert in the repository by selecting Close as and marking the alert as "Revoked."
2. Document the incident in your team's knowledge base or incident management system, including the steps taken to remediate the leak and any lessons learned.

Step 9. Prevent further leaks

Dealing with secret leaks is often disruptive, complicated, and time-consuming. The focus for secret handling should always be on preventing leaks at all costs:

1. Ensure that push protection (part of GitHub Secret Protection) is enabled for the repository, if it's not already. Explore implementing strict bypass controls, so that only trusted users can bypass push protection. See Push protection.
2. Ensure you have "Push protection for users" enabled on your personal account, which protects you from accidentally pushing supported secrets to any public repository.
3. Advocate for, or implement, best practices for secret management within your team or organization:
   • Use environment variables to store secrets instead of hardcoding them in the codebase.
   • Use secret management tools like GitHub's "Secrets and variables" store under your repository's settings page to securely store and manage secrets.
   • Regularly rotate secrets to minimize the impact of any potential leaks.
4. Document incidents and remediation steps to help your team learn from past mistakes and improve future practices.
5. Advocate for, and undertake, regular learning and security training. See, for example, GitHub Advanced Security course from Microsoft Learn.