Index ¦ Archives ¦ Atom

Using Cosign (and Vault and Fulcio and Rekor) to sign binaries

Code signing, what is it good for?

Supply chain is security's new hotness, and the Sigstore set of code signing related projects is getting a lot of attention in this part of the Twittersphere / industry.

The project has 3 main parts:

  • Cosign, a tool for signing container images (and other blobs, such as arbitrary binaries)

  • Fulcio, a root CA that issues short-lived code signing certificates based on OIDC-authenticated identity

  • Rekor, a transparency log for recording supply chain events.

There are various blog posts & examples of using the tools, but it seems most Cosign content is container-related - probably because that's what the tool was originally focused on.

More generally, say I have some software I've written, compiled, and want to distribute:

$ cat hello.go
package main

import "fmt"

func main() {
    fmt.Println("hello y'all")
}
$ go build hello.go
$ ./hello
hello y'all

I want to sign the hello binary, so my users can verify that it is an official and legitimate release.

If I wanted to go the platform-specific route, I'd need to figure out macOS notarization, Windows Authenticode, then for Linux perhaps distro-specific packaging and signing. Those methods have their benefits, especially for end-user UX, but I'm going to stay platform-agnostic for now.

I'm going to try three signing methods:

  • A basic signing use case, using Cosign with locally-generated key material.

  • A more advanced signing use case, with Cosign with remote key material stored in a KMS provider (HashiCorp Vault).

  • The full (experimental) Sigstore experience, using Cosign, Fulcio, a GitHub OIDC identity, and Rekor.

Basic signing use case with local key material

First, I download and install a recent Cosign release. (Ironically, I have to jump through the macOS hoops to run an unsigned executable... Cosign signatures are published, so I could use Cosign to verify Cosign using something like cosign verify-blob --cert cosign-darwin-arm64-keyless.pem --signature cosign-darwin-arm64-keyless.sig cosign-darwin-arm64.)

With Cosign installed, I generate a key pair, sign the binary, and do a test verification:

$ cosign generate-key-pair
Enter password for private key:
Enter password for private key again:
Private key written to cosign.key
Public key written to cosign.pub
$ cosign sign-blob --key cosign.key --output-signature hello.sig hello
Using payload from: hello
Enter password for private key:
Signature wrote in the file hello.sig
$ cosign verify-blob --key cosign.pub --signature hello.sig hello
Verified OK

This is all pretty straightforward. I can publish the binary, public key, and the signature, and hello users could install Cosign and do that same verify-blob step.

But let's say that I'm concerned about exposure of that private key.

More advanced signing use case with remote key material

I've read that Cosign integrates with external KMS providers including HashiCorp Vault. That's good for storing private keys, right?

I download and install a recent Vault release. (This one is notarized, so no macOS unsigned executable hoops.) Then I bring up a local Vault (vault server -dev), export the VAULT_ADDR and VAULT_TOKEN variables, and enable the transit secrets engine (vault secrets enable transit). This could just as well be an existing Vault cluster, either self-managed or in the cloud. (If I was doing this "in production", I'd also want to setup Vault tokens with least-privilege policy, but I'm just using the root token here for simplicity.)

Similarly to earlier, I generate a key pair, sign the binary, and do a test verification:

$ cosign generate-key-pair --kms hashivault://hello-code-signing
Public key written to cosign.pub
$ cosign sign-blob --key hashivault://hello-code-signing --output-signature hello.sig hello
Using payload from: hello
Signature wrote in the file hello.sig
$ cosign verify-blob --key hashivault://hello-code-signing --signature hello.sig hello
Verified OK

I can also verify against the public key, which was written out at generation time:

$ cosign verify-blob --key cosign.pub --signature hello.sig hello
Verified OK

As in the previous use case, I can publish that binary, public key, and the signature, and my users could do that same verify-blob step. From a security perspective, signing material is less exposed than in the local key material example, as the private key never leaves Vault and access to it is captured by the Vault audit log.

The hello-code-signing key shows up in the Vault transit configuration:

Vault Transit Screenshot

This is all well and good (and a pattern that could be useful, due to its security properties), but the centralized and transparent workflow that Sigstore is offering is interesting.

Getting the full (experimental) Sigstore experience

There are various "experimental" disclaimers on this, but I want to try the full Sigstore suite. In the sign-blob output here, you'll see the Fulcio and Rekor interactions.

$ COSIGN_EXPERIMENTAL=1 cosign sign-blob --output-signature hello.sig --output-certificate hello.crt hello
Using payload from: hello
Generating ephemeral keys...
Retrieving signed certificate...
Your browser will now be opened to:
https://oauth2.sigstore.dev/auth/auth?access_type=online&client_id=sigstore&code_challenge=cXc8b-NKTo[snip]7b4&code_challenge_method=S256&nonce=29Pd9[snip]yH1&redirect_uri=http%3A%2F%2Flocalhost%3A49950%2Fauth%2Fcallback&response_type=code&scope=openid+email&state=29Pd9[snip]2eX7oZI
Successfully verified SCT...
using ephemeral certificate:
-----BEGIN CERTIFICATE-----
MIICFjCCAZ2gAwIBAgITKdweJIniizraM1XXe7dWts1e4TAKBggqhkjOPQQDAzAq
[snip]
RIXNSFwCvzAlFg==
-----END CERTIFICATE-----

tlog entry created with index: 2406341
Signature wrote in the file hello.sig
Certificate wrote in the file hello.crt

As noted in the cosign output, there's a browser-based Fulcio OIDC authentication flow in the middle:

Fulcio OIDC Flow Screenshot

The ephemeral certificate used for the signing is interesting... it is issued by sigstore.dev, has a 10 minute lifetime, is marked for code signing usage, and has a SAN containing the email for the GitHub profile which I used to do the OIDC authentication step.

$ openssl x509 -in ephemeral.crt -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            29:dc:1e:24:89:e2:8b:3a:da:33:55:d7:7b:b7:56:b6:cd:5e:e1
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: O = sigstore.dev, CN = sigstore
        Validity
            Not Before: May 20 03:48:24 2022 GMT
            Not After : May 20 03:58:23 2022 GMT
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:0e:64:6e:e4:a8:53:19:6d:be:a6:8b:02:f8:4b:
                    [snip]
                    7f:b8:5b:f9:45
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage:
                Code Signing
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                C3:22:19:D4:D0:AB:B7:9E:EF:16:9A:6B:6A:EA:FA:5A:41:C0:DC:CB
            X509v3 Authority Key Identifier:
                58:C0:1E:5F:91:45:A5:66:A9:7A:CC:90:A1:93:22:D0:2A:C5:C5:FA
            X509v3 Subject Alternative Name: critical
                email:redacted@redacted.com
            1.3.6.1.4.1.57264.1.1:
                https://github.com/login/oauth
    Signature Algorithm: ecdsa-with-SHA384
    Signature Value:
        30:64:02:30:1d:7a:b6:d7:5c:3e:b5:e0:c5:2f:22:a7:a0:a6:
        [snip]
        81:fc:44:85:cd:48:5c:02:bf:30:25:16

After completing the signing process, my hello users (or anyone) could use Cosign to verify that signature, and also check Rekor for the log record:

$ COSIGN_EXPERIMENTAL=1 cosign verify-blob --signature hello.sig hello
tlog entry verified with uuid: 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f index: 2406341
Verified OK

I'd note that Rekor supports other signing types, and doesn't require usage of Cosign... that's just what I'm focusing on for now. The Rekor CLI tool can be used for other Rekor interactions.

Interested hello users (or anyone) can lookup an artifact in the log:

$ rekor-cli search --artifact hello
Found matching entries (listed by UUID):
1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f

.. and retrieve the signing record:

$ rekor-cli get --uuid 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f
LogID: c0d23d6ad406973f9559f3ba2d1ca01f84147d8ffc5b8445c224f98b9591801d
Index: 2406341
IntegratedTime: 2022-05-20T03:48:27Z
UUID: 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f
Body: {
  "HashedRekordObj": {
    "data": {
      "hash": {
        "algorithm": "sha256",
        "value": "e114febb6619f5b07c57a0a672dc97c0dadfc5250ad49b65c1fa12711853a999"
      }
    },
    "signature": {
      "content": "MEYCIQCGKhKXxH5GU9GTEDc3ccU2z0lNxWM4PFgik/ARREcDvgIhAJnjTNhCn5DCZAFeM8+hVbD3UW6hOJHcqJ+SVG7wWyha",
      "publicKey": {
        "content": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNGakNDQVoyZ0F3SUJBZ0lUS2R3ZUpJbmlpenJhTTFYWGU3ZFd0czFlNFRBS0JnZ3Foa2pPUFFRREF6QXEKTVJVd0V3WURWUVFLRXd4emFXZHpkRzl5WlM1a1pYWXhFVEFQQmdOVkJBTVRDSE5wWjNOMGIzSmxNQjRYRFRJeQpNRFV5TURBek5EZ3lORm9YRFRJeU1EVXlNREF6TlRneU0xb3dBREJaTUJNR0J5cUdTTTQ5QWdFR0NDcUdTTTQ5CkF3RUhBMElBQkE1a2J1U29VeGx0dnFhTEF2aExMYkQ0QVFqRlRKU092aXFRcjFyNnRqeVhTOTBlTnBTS01WS0sKWjRTNU8xMlU4S3I2WDhtdXdJTHlvcE5tZjdoYitVV2pnY3N3Z2Nnd0RnWURWUjBQQVFIL0JBUURBZ2VBTUJNRwpBMVVkSlFRTU1Bb0dDQ3NHQVFVRkJ3TURNQXdHQTFVZEV3RUIvd1FDTUFBd0hRWURWUjBPQkJZRUZNTWlHZFRRCnE3ZWU3eGFhYTJycStscEJ3TnpMTUI4R0ExVWRJd1FZTUJhQUZGakFIbCtSUmFWbXFYck1rS0dUSXRBcXhjWDYKTUNVR0ExVWRFUUVCL3dRYk1CbUJGMnBoYldsbFptbHVibWxuWVc1QVoyMWhhV3d1WTI5dE1Dd0dDaXNHQVFRQgpnNzh3QVFFRUhtaDBkSEJ6T2k4dloybDBhSFZpTG1OdmJTOXNiMmRwYmk5dllYVjBhREFLQmdncWhrak9QUVFECkF3Tm5BREJrQWpBZGVyYlhYRDYxNE1VdklxZWdwai9ENXpRcmVJdHFXdzE0ZHRlQ2s4d0IycEhrTVFIZFBDa1kKSmZMeHdNdSswdWNDTUZNYlhQY3ViYTExWHNESW1KZkkzalRkWDI3MEt6eDMzZ0RVbnFjNVFhZzBTSHd5LzRIOApSSVhOU0Z3Q3Z6QWxGZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"
      }
    }
  }
}

They can pull out the signature & compare to the local copy:

$ rekor-cli get --uuid 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f --format json | jq -r .Body.HashedRekordObj.signature.content
MEYCIQCGKhKXxH5GU9GTEDc3ccU2z0lNxWM4PFgik/ARREcDvgIhAJnjTNhCn5DCZAFeM8+hVbD3UW6hOJHcqJ+SVG7wWyha
$ cat hello.sig
MEYCIQCGKhKXxH5GU9GTEDc3ccU2z0lNxWM4PFgik/ARREcDvgIhAJnjTNhCn5DCZAFeM8+hVbD3UW6hOJHcqJ+SVG7wWyha

... and also compare the SHA256 values:

$ rekor-cli get --uuid 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f --format json | jq -r .Body.HashedRekordObj.data.hash.value
e114febb6619f5b07c57a0a672dc97c0dadfc5250ad49b65c1fa12711853a999
$ shasum -a 256 hello
e114febb6619f5b07c57a0a672dc97c0dadfc5250ad49b65c1fa12711853a999  hello

They can look at the cert associated with the identity that was used for signing:

$ rekor-cli get --uuid 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f --format json | jq -r .Body.HashedRekordObj.signature.publicKey.content | base64 -d | openssl x509 -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            29:dc:1e:24:89:e2:8b:3a:da:33:55:d7:7b:b7:56:b6:cd:5e:e1
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: O = sigstore.dev, CN = sigstore
        Validity
            Not Before: May 20 03:48:24 2022 GMT
            Not After : May 20 03:58:23 2022 GMT
[snip, because this is the same as the ephemeral cert from earlier]
                X509v3 Subject Alternative Name: critical
                email:redacted@redacted.com

Lastly, for verification, they can extract the public key from the signing cert, then verify the signature of the message digest:

$ rekor-cli get --uuid 1f6c3ca08b53bfe21c4fe35aea98f939a287fb91a678bb197a9a7c6289b0718f --format json | jq -r .Body.HashedRekordObj.signature.publicKey.content | base64 -d > hello.crt
$ openssl x509 -pubkey -noout -in hello.crt > hello_public.key
$ base64 -d hello.sig > hello.sig.bin
$ openssl dgst -sha256 -verify hello_public.key -signature hello.sig.bin hello
Verified OK

... and to take it all the way up the chain, using the Fulcio root CA:

$ openssl verify -verbose -no_check_time -CAfile fulcio-root.pem hello.crt
hello.crt: OK

(We use -no_check_time because the ephemeral certificate used for signing has expired, by design.)

What's next?

The first two use cases felt reasonably straightforward, and I like the ease of use for the KMS provider integration in particular.

Overall, the full Sigstore experience is described as and feels like work in progress. Some interesting ideas, and nice evolution of the underlying model from other established signing mechanisms.

I'm curious to see how development continues and - more importantly? - how integration into existing tools / platforms / workflows goes. Also curious to see how decisions are made regarding usage of public Fulcio and Rekor instances vs. project, community, or enterprise-specific private instances.

Watch this space, I suppose...

© Jamie Finnigan; opinions my own and not my employers. Built using Pelican. Modified from theme by Giulio Fidente on github.