SoftHSM2 Development Setup

The keystore needs AES wrap and bind keys and EC provider signing keys in an HSM. During development, SoftHSM2 stands in for it. SoftHSM2 is a library, not a server: the keystore process loads libsofthsm2 and reads the token files from a directory. The scripts live in scripts/softhsm2 and are Python 3.9+ (standard library only). They run standard tools (softhsm2-util, pkcs11-tool, OpenSSL 3 and pkcs11-tools). Details, the list of negative tests and troubleshooting are in the scripts README.

Requirements

• SoftHSM2 2.7.0 on the development machine; 2.6.1 (from apt) in CI on Ubuntu 24.04. setup.py needs only --show-slots and --init-token. Version 2.6.1 lacks --show-config, so the scripts then search the standard install directories.
• OpenSC pkcs11-tool.
• Python 3.9+ (standard library only) and a POSIX sh (used by test_setup.py only).
• OpenSSL 3.x first on PATH (macOS: the system openssl is LibreSSL, Homebrew’s openssl@3 is keg-only).
• For provisioning: p11keygen, p11od, p11ls, p11req and p11importcert from pkcs11-tools (selftest.py also p11setattr).

brew install softhsm opensc openssl@3        # macOS, library: /opt/homebrew/lib/softhsm/libsofthsm2.so
export PATH="$(brew --prefix openssl@3)/bin:$PATH"
sudo apt install softhsm2 opensc openssl     # Ubuntu, library: /usr/lib/softhsm/libsofthsm2.so

Tokens and Objects

setup.py creates three empty tokens:

• sk-custody: the custody token, PIN KEYSTORE_HSM_PIN;
• sk-shared: provider keys and certificates only, PIN KEYSTORE_HSM_SHARED_PIN;
• sk-recovery-test: a throw-away token for the recovery tests.

A key set is one generation <n>: a wrap-<n> and a bind-<n> key. provision.py then creates the objects:

Every key is born in the HSM, private, sensitive, not extractable and not modifiable. The check also requires CKA_LOCAL, CKA_ALWAYS_SENSITIVE and CKA_NEVER_EXTRACTABLE, which rejects imported keys and keys that were once extractable. The unwrap template forces the attributes of every key unwrapped under wrap-<n>:

class=private_key, key_type=ec, token=false, private=true, sensitive=true,
extractable=false, sign=true, derive=false, modifiable=false

Setting Up the Tokens

Set the PINs and the library path in .env (copy of .env.example; replace the <...> placeholders with your own throw-away values; never commit it), then run the setup script:

set -a; source .env; set +a
scripts/softhsm2/setup.py            # --dir <path> changes the directory (default .softhsm/), --reset recreates everything

Environment variables:

The script is idempotent. It creates the empty tokens and writes a throw-away test PKI (EC P-256, made with OpenSSL) to <dir>/pki/:

• root.pem: the test root, valid 3650 days. Its key is deleted right after it issues the intermediate.
• provider-ca.pem: the intermediate CA, valid 1825 days, so the 365-day provider certificates fit inside.
• provider-ca.key (mode 0600): used by provision.py to issue the provider certificates, then deleted.
• provider-ca.sha256: the SHA-256 of the DER of provider-ca.pem; set it as KEYSTORE_HSM_PROVIDER_CA_SHA256.

Every subject contains TEST ONLY.

The local run is source .env, provisioning of the key sets (below), then mvn spring-boot:run -Dspring-boot.run.profiles=dev.

Provisioning the Custody Key Sets

Once the tokens exist, scripts/softhsm2/provision.py creates the key sets with p11keygen and reads them back with p11od, both from the standard pkcs11-tools toolkit (Mastercard; brew install pkcs11-tools or build from source). There is no project-specific PKCS#11 program. Each key set is two AES-256 keys on sk-custody: wrap-<n>, a key-wrapping key whose CKA_UNWRAP_TEMPLATE forces the attributes of every key unwrapped under it, and bind-<n>, a data key for AES-GCM. Both are born in the HSM, private, sensitive, not extractable and not modifiable. OpenSC pkcs11-tool cannot create them: it sets neither CKA_UNWRAP_TEMPLATE nor CKA_MODIFIABLE=false.

When run against the default token sk-custody, the script also creates the provider objects on sk-shared: for each of ka-signing and wia-signing, an EC P-256 sign-only key and a 365-day TEST ONLY certificate, plus the provider-ca certificate. The x5c chain is the leaf and the intermediate. The provider prerequisites are checked before any custody key is created, but provisioning of the provider objects is not atomic: a failure can leave a partial set that the next run refuses, and cleanup removes provider-ca.key. Recover with setup.py --reset, which recreates all three tokens, and provision again. --check also proves that each private key belongs to its certificate and verifies each leaf against the provider-ca read back from the HSM. --token sk-recovery-test creates a key set only.

set -a; source .env; set +a
scripts/softhsm2/provision.py 1 2                          # one or more generations; --dir <path> as for setup.py, --token <label> (default sk-custody)
scripts/softhsm2/provision.py --check 1 2                  # creates nothing, only reads back and compares with the profile
scripts/softhsm2/provision.py --token sk-recovery-test 1   # key set only

The script refuses a label that already exists (PKCS#11 does not enforce unique labels) before it creates anything, reads every key back after creating it and exits non-zero when one differs from the profile. It uses KEYSTORE_HSM_PIN, KEYSTORE_HSM_SHARED_PIN and KEYSTORE_HSM_LIBRARY (and the softhsm2.conf of --dir, unless SOFTHSM2_CONF is set); the custody PIN reaches the pkcs11-tools programs through their :::exec: PIN source (see pkcs11-tools), so it is on no command line. A startup check of the same attributes in the keystore (refusing keys that differ, fail closed) is planned but not implemented yet; until then, verify the objects with provision.py --check. Run scripts/softhsm2/provision.py --help for the rest.

Checking the Available Mechanisms

SoftHSM2 must offer the mechanisms the keystore uses. List them with p11slotinfo -l "$KEYSTORE_HSM_LIBRARY" -t sk-custody or pkcs11-tool --module "$KEYSTORE_HSM_LIBRARY" --list-mechanisms and compare with the table. A startup check of flags and key sizes in the keystore (fail closed) is planned but not implemented yet; until then, check the mechanisms by hand as described here. A mechanism counts as available only when C_GetMechanismList lists it and C_GetMechanismInfo reports the required flags.

Measured results. The 2.6.1 column was measured on Ubuntu 22.04; CI runs the same SoftHSM2 release (2.6.1) on Ubuntu 24.04. softhsm2-util reports no build options.

Tests

• scripts/softhsm2/test_setup.py tests the setup script. It runs in a temporary directory and does not touch .softhsm/; exit code 2 means a prerequisite is missing. It ignores KEYSTORE_HSM_LIBRARY and uses only a libsofthsm2.* file reported by softhsm2-util --show-config (2.7+) or found in the standard install directories, so it never loads another PKCS#11 module.
• scripts/softhsm2/selftest.py tests the provisioning script (temporary directory, same rules; needs p11keygen and p11od as well; exit code 2 means a prerequisite is missing).

Neither is part of mvn test; the Test with SoftHSM2 workflow runs them.

CI

The Test with SoftHSM2 workflow (.github/workflows/softhsm-test.yml) runs on Ubuntu 24.04. It installs softhsm2 (2.6.1) and opensc from apt and the pkcs11-tools 3.1.0 .deb from the official release, verified against a pinned SHA-256 before installation. It then provisions the tokens and key sets, runs selftest.py, test_setup.py and mvn -B verify.

• To bump pkcs11-tools, change the version, the file name and the hash together in the workflow: take the hash from the digest of the release asset and recompute it with sha256sum.
• OpenSSL 3.0 of Ubuntu 24.04 exits 0 from openssl req -verify even when the signature does not verify, so the scripts require the printed verify OK.
• The PINs are random throw-away values generated per run, masked in the log and passed only through the environment; the workflow uses no secrets, and the tokens are neither cached nor uploaded.

Related Documentation

• HSM Setup
• Key Naming Conventions
• Build and Test