Using auth-source in a Real Emacs Package

Introduction

When an Emacs package talks to an external service, it often needs some kind of credential.

In the case of shpaste, the package talks to paste.sr.ht through the SourceHut GraphQL API. To create, list, open, and delete pastes, it needs a SourceHut OAuth2 personal access token.

The boring solution would be to ask users to put a token in a variable.

Something like:

(setq some-token "...")

I don’t like that approach.

It works, but it moves secret management into the package configuration. It also encourages users to store sensitive values directly in their Emacs config, sometimes in plain text, sometimes in a repository, and sometimes in multiple places.

For shpaste, I wanted something simpler and more standard.

So the package uses auth-source.

Not as an optional extra.

Not as a nice-to-have.

As the only place where the token is looked up.

The Setup From the User Point of View

The README keeps the setup intentionally small.

First, generate an OAuth2 personal access token on SourceHut with the PASTES grant in read-write mode.

Then store it in auth-source:

machine paste.sr.ht password <YOUR_TOKEN>

That’s the important part.

There is no shpaste-token variable.

There is no token stored in the package configuration.

There is no custom secret storage layer.

The machine field is also not arbitrary. It must match the configured SourceHut instance.

The default instance is defined in shpaste-api.el:

(defcustom shpaste-instance "paste.sr.ht"
  "Host of the sourcehut paste instance.
Defaults to the official instance; set to a self-hosted host if needed."
  :type 'string
  :group 'shpaste)

This small choice matters more than it may seem.

The host is used for two different things:

building the API endpoint
looking up the token in auth-source

So when a user changes shpaste-instance to target a self-hosted instance, the same value is used consistently everywhere.

Looking Up the Token

The whole token lookup lives in shpaste--token.

Here is the actual function:

(defun shpaste--token ()
  "Return the sourcehut personal access token for `shpaste-instance'.
The token is looked up in auth-source with the host set to the instance.
Signal a `user-error' when no token is configured."
  (let ((entry (car (auth-source-search :host shpaste-instance
                                         :require '(:secret)
                                         :max 1))))
    (unless entry
      (user-error "No sourcehut token for host %s; add one to auth-source (OAuth2 personal access token, scope PASTES, read-write)"
                  shpaste-instance))
    (let ((secret (plist-get entry :secret)))
      (if (functionp secret) (funcall secret) secret))))

There are a few interesting details here.

First, the lookup uses shpaste-instance as the host:

(auth-source-search :host shpaste-instance
                    :require '(:secret)
                    :max 1)

That means the package does not need a separate configuration value for credentials.

If the instance is paste.sr.ht, the token is looked up for paste.sr.ht.

If the instance is changed to a self-hosted host, the token lookup follows that host automatically.

This keeps the mental model simple:

configure the instance once, then store the matching token in auth-source.

That’s usually the kind of configuration I prefer in Emacs packages. One concept, one value, no duplicated settings.

Requiring a Secret

The lookup also uses:

:require '(:secret)

This is small but important.

The package is not interested in any random auth-source entry matching the host.

It needs a secret.

If no usable entry exists, auth-source-search won’t return something that pretends to be good enough.

This keeps the rest of the code simpler because shpaste--token has a clear contract:

it returns a token
or it signals a user-facing error

There is no half-configured state leaking into the HTTP layer.

Failing Early

Missing credentials are not handled later by the API client.

They are handled immediately:

(unless entry
  (user-error "No sourcehut token for host %s; add one to auth-source (OAuth2 personal access token, scope PASTES, read-write)"
              shpaste-instance))

I like this for two reasons.

First, the error is explicit.

The user is not left with a random 401, a GraphQL error, or a failed HTTP request. The package tells exactly what is missing and which host was used for the lookup.

Second, it fails before building the request.

This makes debugging much easier. If authentication is not configured, the problem is reported at the boundary where credentials are read, not five functions later.

That may sound obvious, but it is the kind of detail that makes a package feel nicer to use.

Secrets Are Not Always Strings

One easy mistake with auth-source is assuming that :secret is always the final string.

shpaste does not make that assumption:

(let ((secret (plist-get entry :secret)))
  (if (functionp secret) (funcall secret) secret))

This is one of those small pieces of code that is easy to miss if you only test with one local setup.

Depending on the backend, auth-source may return the secret directly, or it may return a function that has to be called to obtain the secret.

By handling both cases, the package avoids coupling itself too tightly to one specific auth-source backend.

The important part here is not that the code is complicated.

It is the opposite.

The implementation is tiny, but it accounts for a real behavior of the API.

That’s usually a good sign.

Building the Authorization Header

Once the token retrieval is isolated, the Authorization header becomes a small helper:

(defun shpaste--auth-header ()
  "Return the Authorization header entry carrying the bearer token."
  (cons "Authorization" (concat "Bearer " (shpaste--token))))

This keeps the rest of the transport code readable.

For normal GraphQL queries, shpaste--query uses it here:

(raw (plz 'post (shpaste--endpoint)
       :headers (list (shpaste--auth-header)
                      '("Content-Type" . "application/json"))
       :body (json-encode payload)
       :as 'string))

For blob fetching, the same helper is reused:

(plz 'get url
  :headers (list (shpaste--auth-header))
  :as 'string)

There is no repeated token lookup logic scattered around the codebase.

The package has one way to build authenticated requests.

That’s boring, but it is exactly what I want in this kind of code.

The Endpoint Uses the Same Instance

The token lookup is not the only place where shpaste-instance is used.

The GraphQL endpoint is also derived from it:

(defun shpaste--endpoint ()
  "Return the GraphQL endpoint URL for `shpaste-instance'."
  (format "https://%s/query" shpaste-instance))

This gives shpaste-instance a clear role:

it identifies the SourceHut paste host
it selects the matching auth-source entry
it builds the GraphQL endpoint

Again, the interesting part is not the code itself.

The interesting part is that the package avoids inventing three different configuration knobs for what is really one concept.

Testing the Token Lookup

Credential code is easy to ignore in tests because it often depends on the developer’s local machine.

shpaste avoids that by stubbing auth-source-search.

The first test checks that a function secret is called:

(ert-deftest shpaste-test-token-found ()
  "Token is read from auth-source; a function secret is funcalled."
  (cl-letf (((symbol-function 'auth-source-search)
             (lambda (&rest _) (list (list :secret (lambda () "tok123"))))))
    (should (equal (shpaste--token) "tok123"))))

This test is useful because it verifies the non-obvious case.

If :secret is a function, shpaste--token must call it.

A simpler implementation returning (plist-get entry :secret) would pass some manual tests, but it would fail this one.

The second test checks the missing-token path:

(ert-deftest shpaste-test-token-missing ()
  "A missing token raises a user-error."
  (cl-letf (((symbol-function 'auth-source-search) (lambda (&rest _) nil)))
    (should-error (shpaste--token) :type 'user-error)))

This is just as important.

The behavior is not only documented in the function docstring. It is tested.

No token means user-error.

Not nil.

Not a later HTTP failure.

Not a cryptic exception from another layer.

A clear user-facing error.

Testing the Header

The tests also verify that the token actually ends up in the HTTP request.

In shpaste-test-query, the token function is stubbed:

(cl-letf (((symbol-function 'shpaste--token) (lambda () "tok"))
          ((symbol-function 'plz)
           (lambda (&rest args)
             (setq captured-args args)
             "{"data":{"ping":"pong"}}")))

Then the test checks the Authorization header:

(let ((auth (alist-get "Authorization" (plist-get (cddr captured-args) :headers)
                       nil nil #'equal)))
  (should (equal auth "Bearer tok")))

This may look like a small assertion, but it protects the integration point.

The test does not only check that shpaste--token works in isolation.

It also checks that shpaste--query builds authenticated requests properly.

That’s the useful level of testing for a small package:

test the unit
test the boundary where the unit is consumed
avoid hitting the real network

Testing Blob Fetching Too

The same idea appears in the blob-fetching test.

shpaste--fetch-blob performs an authenticated GET:

(defun shpaste--fetch-blob (url)
  "Fetch and return the raw text content at URL (authenticated)."
  (plz 'get url
    :headers (list (shpaste--auth-header))
    :as 'string))

The corresponding test checks that the request also carries the bearer token:

(let ((auth (alist-get "Authorization" (plist-get (cddr captured) :headers)
                       nil nil #'equal)))
  (should (equal auth "Bearer tok")))

This matters because fetching the paste metadata and fetching the paste content are not the same operation.

Both need authentication.

Both should use the same mechanism.

Both are tested.

What I Like About This Design

The whole authentication layer is small.

There is no framework.

There is no custom credential store.

There is no complicated abstraction.

The flow is simply:

read the token from auth-source
normalize the secret if needed
build a bearer header
reuse that header everywhere HTTP authentication is needed
test both success and failure cases

That is enough.

And honestly, that’s what I like about it.

The package does not try to be clever. It uses an Emacs mechanism that already exists and keeps the project-specific code focused on SourceHut.

Things Worth Keeping in Mind

This design is not magic.

Users still need to configure their token correctly.

For the default instance, the README shows:

machine paste.sr.ht password <YOUR_TOKEN>

If shpaste-instance is changed, the machine field must match the new host.

That coupling is intentional, but it is also something users need to know.

The package helps by making the missing-token error explicit:

(user-error "No sourcehut token for host %s; add one to auth-source (OAuth2 personal access token, scope PASTES, read-write)"
            shpaste-instance)

Could this message be shorter?

Maybe.

But for a configuration error, I prefer being slightly verbose and useful.

A short error like "No token" would be less helpful.

My Take

For small Emacs packages, I think the best configuration is often the one that does not introduce a new concept.

shpaste already needs to know the SourceHut paste instance.

So it reuses that value:

as the API host
as the auth-source lookup key

That keeps the user-facing setup compact.

It also keeps the implementation easy to follow.

The full credential retrieval logic fits in one function, and the tests cover the important paths.

That’s a good trade-off.

Not because it is fancy.

Because it is boring in the right way.

Wrapping Up

If your Emacs package needs an API token, storing it in a custom variable is tempting.

It is easy to implement.

It is also usually not the best user experience.

With auth-source, Emacs already gives us a standard place to retrieve credentials.

In shpaste, that leads to a small and clear design:

shpaste-instance identifies the host
auth-source-search retrieves the token
function secrets are handled properly
missing credentials raise a user-error
HTTP requests reuse a single Authorization header helper
tests verify both lookup and request integration

There is nothing spectacular here.

And that’s the point.

Good authentication plumbing should be boring, explicit, and easy to test.

Share on

Twitter Facebook LinkedIn

Have comments or want to discuss this topic?

Send an email to ~bounga/public-inbox@lists.sr.ht