Spelunk is a Golang library for extracting secrets from various sources (Kubernetes, Vault, env vars, files) using a unified URI-based string we are calling Secret Coordinates. Here are some example of coordinates:
# Secret from the namespace `ns`,
# stored inside the secret `my-team-secret`
# at data key `the-key`
k8s://ns/my-team-secret/the-key
# Secret provided in the form
# of base64-encoded string
base64://bXktYmlnLXNlY3JldAo=
# Secret stored in a JSON
# file at a specific field
file://kafka-credentials.json?jp=$.kafka.passwordSpelunk simplifies the access to secrets by just providing the coordinates for "digging up" configuration values in cloud-native CLI tools and applications.
Its primary application is command line tools, but... you do you! Users point at a secret from any source, providing the right coordinates: your tool/service/software can use Spelunk to adapt dynamically and fetch the secret.
With a single library, the source of secrets is flexible and adapts to your environment, situation and/or needs.
Spelunk can be configured to support more Sources, and users can apply Modifiers to "prepare" the secret in the exact way they need it.
And, if you want to use Spelunk in your scripts, it also comes with its own spelunk CLI.
Starting with version v2.x, Spelunk implements a highly efficient Go Multi-Module Workspace architecture.
Previously, importing Spelunk pulled down every single heavy SDK dependency (including the AWS, GCP, Azure, HashiCorp Vault, Kubernetes, and 1Password SDKs) regardless of whether you used them or not.
From v2.0.0 onwards:
- Ultra-Lean Core: The root core module github.com/detro/spelunk/v2is completely bare and carries virtually zero production dependencies.
- Pay Only For What You Use: Sibling plugins are completely decoupled into isolated submodules. Heavyweight dependencies are only pulled down by Go if you explicitly choose to import and register their corresponding plugin.
Add the core library to your project:
# Pull the ultra-lean core library
go get github.com/detro/spelunk/v2
# Pull only the specific plugins you want to use
go get github.com/detro/spelunk/plugin/source/kubernetes/v2Setup a new Spelunker and start digging up secrets:
package main
import (
"context"
"github.com/detro/spelunk/v2"
"github.com/detro/spelunk/v2/types"
"github.com/detro/spelunk/plugin/source/kubernetes/v2"
v1 "k8s.io/client-go/kubernetes/typed/core/v1"
)
// Initialize the Kubernetes client...
k8sClient, err := v1.NewForConfig(restConfig)
// Create a Spelunker
spelunker := spelunk.NewSpelunker(
kubernetes.WithKubernetes(k8sClient),
)
// Get coordinates to a secret from one of many supported sources:
// from Kubernetes...
coord, err := types.NewSecretCoord("k8s://my-namespace/my-secret/my-data-key")
// ... or from plain text (please don't!)
coord, err := types.NewSecretCoord("plain://MY_PLAINTEXT_SECRET")
// ... or from a local file
coord, err := types.NewSecretCoord("file://secrets.json?jp=$.kafka.password")
// ... or from environment variable
coord, err := types.NewSecretCoord("env://GITHUB_PRIVATE_TOKEN")
// Dig-up secrets!
secret, _ := spelunker.DigUp(ctx, coord)Find some useful /examples directory for how to use spelunk with various
libraries for configuration or command line arguments parsing.
Spelunk comes with a bunch of features: Sources
and Modifiers, the role of which is explained below.
Some are built-in, and a new spelunk.Spelunker instance comes with those enabled;
others are plug-in, and you will have to enable them as SpelunkerOption
provided at construction time:
package main
import (
"github.com/detro/spelunk/v2"
"github.com/detro/spelunk/plugin/source/kubernetes/v2"
"github.com/detro/spelunk/plugin/source/vault/v2"
)
_ = spelunk.NewSpelunker(
kubernetes.WithKubernetes(k8sClient),
vault.WithVault(vaultClient),
)Spelunk includes an official standalone command-line interface located in cmd/spelunk.
It bundles all built-in and plugin backends into a single binary, supporting secret retrieval (dig),
existence checks (exists), credential verification (creds),
and shell auto-completion directly from terminal or CI/CD pipelines.
See Spelunk CLI Documentation for details and installation instructions.
spelunk.Spelunker is the entry point type, and it does its job using
the following types.
This is the starting point: take a string containing Secret Coordinates as documented
above, and use types.NewSecretCoord to turn it into a SecretCoord.
This is a generic, secret-type-agnostic representation of how to find a secret. And
it's all that Spelunker needs to dig-up the secret.
SecretCoord implements encoding.TextUnmarshaler, so it can be created through the unmarshalling
of command-line user input, through json.Unmarshal and any other type-aware process.
For example, when using the awesome Kong library:
package main
import "github.com/detro/spelunk/v2"
type CLI struct {
Password spelunk.SecretCoord `name:"password" short:"p" help:"your password"`
// ...
}Sources are places out of which a secret can be "dug-up".
Some are built-in to spelunk.Spelunker, others are plug-in and need to be enabled.
Modifiers are optional behaviour applied to a secret after it has been dug-up by Spelunk. It can be seen as a function in the mathematical sense:
Each modifier is applied in the same order provided in the secret coordinates:
<type>://<location>?mod1=A&mod2=B&mod1=C
will result in this sequence:
- Spelunkerdigs-up the secret- <value>of type- <type>from the- <location>
- mod1takes the- <value>and applies- mod1(<value>, A) = <value_A>
- mod2takes the- <value_A>and applies- mod2(<value_A>, B) = <value_A_B>
- mod1takes the- <value_A_B>and applies- mod1(<value_A_B>, C) = <value_A_B_C>
- client code is returned the final <value_A_B_C>
If you are interested in contributing (for example, you have a brilliant idea for a plug-in), we have some contribution guidelines.
This project is shared under the MIT license.
- Spelunk CLI documentation: standalone CLI tool and reference
- Architecture documentation: understand how Spelunk works internally
- Contribution guidelines: setting some ground rules
- Agents documentation: helps LLM-agent augmented developers in their contribution journey
- Changelog: track the evolution of the project
Footnotes
-
Untested: Looking for contributors with access to a test account/vault! ↩ ↩2
-
Library only: The Bitwarden SDK requires CGO and a platform-specific native library, so this source is not bundled in the spelunkCLI. It remains fully usable from Go code. ↩
-
Not Implemented: Not implemented due to the lack of maintained Go SDK, no suitable REST API, and no local Testcontainers for simulation. ↩ ↩2