Version 02. What’s new?

Link Invoke-WebRequest

For secrets stored in source code

For secrets stored in configuration files

Storing sensitive data directly in source code (secrets hardcoding) poses a significant security vulnerability. If the source code is accidentally exposed or if attackers gain access to files on the server, these secrets can be compromised, potentially allowing unauthorized access to related systems.

Therefore, this practice must be strictly avoided during software development. Below, we’ll show you how to safely detect and remove secrets from your projects using Locker Secrets Manager.

Installing Locker CLI

For Windows

Start-Process powershell -Verb runAs
New-Item -Path "$HOME\.locker" -ItemType Directory -Force
Invoke-WebRequest -Uri "https://locker.io/secrets/download/cli-windows " -OutFile "$HOME\.locker\locker.exe"
$oldPath = [System.Environment]::GetEnvironmentVariable("Path", [System.EnvironmentVariableTarget]::Machine)
$newPath = "$oldPath;$HOME\.locker"
[System.Environment]::SetEnvironmentVariable("Path", $newPath, [System.EnvironmentVariableTarget]::Machine)

For Linux

mkdir -p ~/.locker
wget https://locker.io/secrets/download/cli-linux -O ~/.locker/locker
chmod +x ~/.locker/locker
sudo ln -sf ~/.locker/locker /usr/local/bin/locker

For macOS

arm

mkdir -p ~/.locker
wget https://locker.io/secrets/download/cli-mac-arm64 -O ~/.locker/locker
chmod +x ~/.locker/locker
sudo ln -sf ~/.locker/locker /usr/local/bin/locker

x64

mkdir -p ~/.locker
wget https://locker.io/secrets/download/cli-mac-x64 -O ~/.locker/locker
chmod +x ~/.locker/locker
sudo ln -sf ~/.locker/locker /usr/local/bin/locker

Detecting secrets in the project

Use the scan command below:

locker scan --source <path-to-source-code>

You will obtain results as shown below, including:

Path to the file containing secrets

Found secret value

Line number containing the secret

Type of secret

Commit hash

Implementing secure secret storage

Following the scan results, it is essential to adopt a more secure method of storing secrets, rather than hardcoding them.

Setting up your project with Locker Secrets

Create a Locker Secrets account here if you don't have one.

Create a new project corresponding to the one containing your secrets.

Generate an Access Key within the project; this key pair will be used with the Locker SDK in subsequent steps.

For secrets stored in source code

For a small number of hardcoded secrets, you can manually update them using the following steps

Add the hardcoded secrets to your newly created Locker Secrets project via the web interface.

Install the Secrets SDK for your programming language here.

Initialize the SDK Config object using the Access Key and Secret Access Key generated earlier.

Manually locate the hardcoded secrets in your source code and replace them with calls to the SDK to retrieve the securely stored secrets.

💡

Note: The automatic secrets update feature is currently being developed and will be available in future versions.

Example with Python

Suppose you have a file settings.py with hardcoded secrets as follows:

# ...
DEFAULT_CONFIG = {
  "database": {
    "host": "mysql.myserver.com",
    "port": 3306,
    "username": "my_database",
    "password": "Ol1PPgsG4htlBb0q7THJP7nWyBJXbAJg"
  },
  "request": {
    "access_key_id": "ak29NASAs",
    "secret_token": "qHg34VAGQZwCkUy4UjZQAsCVeH6TvsJg"
  }
}

# ...

The Locker CLI has detected hardcoded secrets in the lines mentioned

"password": "Ol1PPgsG4htlBb0q7THJP7nWyBJXbAJg"
...
"secret_token": "qHg34VAGQZwCkUy4UjZQAsCVeH6TvsJg"

Create the corresponding secrets in your Locker Project via the web interface.

DATABASE_PASSWORD: Ol1PPgsG4htlBb0q7THJP7nWyBJXbAJg
REQUEST_SECRET_TOKEN: qHg34VAGQZwCkUy4UjZQAsCVeH6TvsJg

Install the Locker Secret SDK and replace the hardcoded secrets with SDK usage.

requirements.txt

...
lockerpm
...

Store your previously generated Locker Access Key pair as environment variables.

Linux/MacOS

export ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
export SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>

Windows

Powershell

$Env:ACCESS_KEY_ID = <YOUR_ACCESS_KEY_ID>
$Env:SECRET_ACCESS_KEY = <SECRET_ACCESS_KEY>

Command Prompt

set ACCESS_KEY_ID= '<YOUR_ACCESS_KEY_ID>'
set SECRET_ACCESS_KEY= '<YOUR_SECRET_ACCESS_KEY>'

Proceed to declare and initialize the Locker Secret Manager object.

secret_management.py

import os
from locker import Locker

client = Locker(
    access_key_id=os.get_env(ACCESS_KEY_ID),
    secret_access_key=os.get_env(SECRET_ACCESS_KEY),
)

Replace the hardcoded values by using the SDK to call variables at the locations where your secrets are exposed.

settings.py

from secret_management import client
# ...
DEFAULT_CONFIG = {
  "database": {
    "host": "mysql.myserver.com",
    "port": 3306,
    "username": "my_database",
    "password": client.get("DATABASE_PASSWORD"),
  },
  "request": {
    "access_key_id": "ak29NASAs",
    "secret_token": client.get("REQUEST_SECRET_TOKEN")
  }
}
# ...

For a large number of secrets, you can use the scan fix option.

This option will upload all your secrets to the corresponding Locker project and automatically replace the hardcoded secrets in your source code by using the SDK to call the corresponding variables from the Locker project:

Store the Locker Access Key pair you created earlier as environment variables.

Use the scan fix command.

locker scan fix --source <path-to-source-code> --language <program-language>

Review your source code to ensure that all secrets are called and used correctly.

💡

Currently, the scan fix option is supported only for Python. Support for additional languages will be added in the future."

Here’s an example in Python:

Suppose you have a file settings.py with hardcoded secrets like this:

# ...
DEFAULT_CONFIG = {
  "database": {
    "host": "mysql.myserver.com",
    "port": 3306,
    "username": "my_database",
    "password": "Ol1PPgsG4htlBb0q7THJP7nWyBJXbAJg"
  },
  "request": {
    "access_key_id": "ak29NASAs",
    "secret_token": "qHg34VAGQZwCkUy4UjZQAsCVeH6TvsJg"
  }
}

# ...

Locker CLI has detected hardcoding in the following lines:

"password": "Ol1PPgsG4htlBb0q7THJP7nWyBJXbAJg"
...
"secret_token": "qHg34VAGQZwCkUy4UjZQAsCVeH6TvsJg"

Store the Locker Access Key pair you created earlier as environment variables, similar to the manual update process described above.

Use the scan fix command as follows:

locker scan fix --source . --language python

After running this command, the secrets will be automatically uploaded to your project with incrementing key names like SECRET1, SECRET2. The Locker SDK instance will also be initialized and the secrets will be replaced in your source code.

secret_manager.py

import os
from locker import Locker
locker_client = Locker(
	access_key_id=os.getenv("LOCKER_ACCESS_KEY_ID"),
	secret_access_key=os.getenv("LOCKER_SECRET_ACCESS_KEY")
)

settings.py

from secret_management import locker_client
# ...
DEFAULT_CONFIG = {
  "database": {
    "host": "mysql.myserver.com",
    "port": 3306,
    "username": "my_database",
    "password": locker_client.get("SECRET1"),
  },
  "request": {
    "access_key_id": "ak29NASAs",
    "secret_token": locker_client.get("SECRET2")
  }
}
# ...

For secrets stored in environment variable files

Assess the necessity of the environment variable file for your project. If unnecessary, proceed to delete this file.

If your project requires secrets from the environment variable file, configure Locker with the generated Access Key pair using the command:

locker configuration --access-key-id {access key id} --secret-access-key {secret access key}

Save the secret to the Locker Secret Vault using the command locker secret create:

locker secret create --key {secret key} --value {secret value} --description {secret description (optional)} --environment {secret environment (if the secret is associated with a specific environment)}

Use the command locker secret run to execute the project with the saved secrets:

locker secret run --environment dev --command your_command

The locker run command will inject the secrets from a specified environment (previously stored on Locker Secret) as environment variables into your program execution command.

Remove the secrets stored in your environment variable file.

For secrets stored in configuration files

Configure the Locker CLI with the Access Key and Secret Key you created earlier.

locker configuration --access-key-id {YOUR_ACCESS_KEY_ID} --secret-access-key {YOUR_SECRET_ACCESS_KEY}

Based on the scan results, identify the configuration file locations that contain your secrets. Use the Locker CLI to import these exposed secrets into your project.

locker secret import --source {CONFIG_FILE_LOCATION}

If your program uses secrets stored in configuration files, replace the code that accesses these configuration files with the Locker SDK.

Finally, remove the secrets from the configuration file.

Here’s an example in Python:

Suppose you have a configuration file containing exposed secrets as follows: config.ini

[DEFAULT]
Timeout = 30
SecretKey = this.is.secret
Forward = 1
MaxRetries = 10

[mysql1]
Port = 3306
Host = mysql1.example
Username = root
Password = root

[mysql2]
Port = 3307
Host = mysql2.example
Username = root
Password = root

Assume your source code that loads the above configuration file looks like this:

import configparser


class Config:
    def __init__(self):
        self.config_parser = configparser.ConfigParser()
        self.config_parser.read('config.ini')
        self.set_attr()

    def set_attr(self):
        for sec in self.config_parser.sections():
            for key, value in self.config_parser.items(sec):
                setattr(self, f"{sec}_{key}", value)

You need to import these secrets into your Locker project using the command:

locker secret import --source config.ini

Replace the code that loads the configuration file with the Locker Python SDK.

from locker import Locker

LOCKER_ACCESS_KEY_ID = os.getenv('LOCKER_ACCESS_KEY_ID')
LOCKER_SECRET_ACCESS_KEY = os.getenv('LOCKER_SECRET_ACCESS_KEY')

class Config:
    def __init__(self):
        self.locker = Locker(access_key_id=LOCKER_ACCESS_KEY_ID, secret_access_key=LOCKER_SECRET_ACCESS_KEY)
        self.set_attr()

    def set_attr(self):
        secrets = self.locker.list()
        for arg in secrets:
            setattr(self, arg['key'], arg['value'])

Finally, remove the configuration file from the directory.

Removing secrets from the Git repository

Unlike typical file organization on computers, Git is a software with a version-based file storage and complex data organization mechanism. Therefore, if you delete or change the content of a file, it simply removes and changes it in the current version, while the content still exists in previous versions and can be accessed through git commit.

For sensitive and confidential data, complete removal requires deleting all related git commit history of the file containing that data. There are 2 main methods for this:

git filter-branch is a tool built into the git command.

BFG Repo-Cleaner (short for BFG) is a tool used to rewrite commit history with similar functionality to git filter-branch but much faster, up to 10-720 times.

The following guide will perform with BFG

Installation

To run BFG, you need to have Java installed on your computer. Please download and install Java corresponding to your operating system, if Java is not installed already.

Download BFG from the following link.

Using BFG to remove secrets

Before running BFG, create a backup for your project to prevent data loss in case of errors.

Clone the repository to your machine using the --mirror flag. This will give us the full git index and metadata of the project without needing actual data.

git clone --mirror https://github.com/owner/repo.git

Remove the file containing secrets throughout the commit history.

java -jar bfg.jar <file_name> some-big-repo.git

Clear the repository's trash.

cd some-big-repo.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive

Push the rewritten git history to the server.

git push -f

Contact

Email: [email protected]

Website: https://locker.io/secrets

Results