Full Stack Web Developer
LinkedIn Mastodon GitHub Feed Twitter

Currently working at Yummy Publishing (previously at valantic, Sulu and MASSIVE ART)
and lecturing at the Vorarlberg University of Applied Sciences

Making the shell history more useful by using shell variables

cli, linux, fish

I already described that I like using curl for testing HTTP APIs. I have also used that approach a lot lately, but with an API that required some kind of HTTP authentication. This is also no problem for curl, which can send the necessary Authorization header using its -H flag:

curl -X GET -H "Authorization: Bearer some-lengthy-but-not-infinitely-valid-token" localhost/some-uuid-that-might-be-valid-only-once

While this basically works, it is still problematic from a shell history perspective. I already hinted at the problems in the placeholder values in the above curl command. Re-executing this command only works easily as long as the passed values in headers and URL are valid. In case the token or the passed value changes frequently it becomes very tedious to reuse the command from history, since after pressing up until the desired command appears you must fiddle around (moving the cursor, deleting some characters, inserting new ones, …) to replace values which became invalid with valid ones. In my case this mostly was about the bearer token, which I had to retrieve again after it became invalid, giving me a hard time reusing existing commands.

However, I just realized recently that there is quite an easy fix for that. By placing those changing values in shell variables the command in the history is much easier to reuse. So instead of copying tokens, UUIDs, etc. directly into the command I set them as shell variables and use these shell variables in the command. The following examples work with the fish shell:

set TOKEN some-lengthy-but-not-infinitely-valid-token
set UUID some-uuid-that-might-be-valid-only-once

curl -X GET -H "Authorization: Bearer $TOKEN" localhost/$UUID

This might look like more work for the first command (who am I kidding, it also is more work for the first command), but the huge advantage is that the last command can be easily reused. Instead of pressing up and manually moving the cursor and replacing parameters you can set new values to the shell variables and execute the command from history as is. So the workflow after retrieving a new token and UUID is as follows:

set TOKEN another-lengthy-but-not-infinitely-valid-token
set UUID another-uuid-that-might-be-valid-only-once

# Press up to get the curl command in history and execute it right away

This might not look like a big deal now, but especially if the command to execute is more complex this gets really useful. To give an even better example, I recently worked on an HTTP API using authentication, which had a use case involving a two-step process, i.e. receiving a token from the API that has to be sent in another request. For testing I executed two curl commands at once by using command substitution, whereby both are using the same shell variables for the token and the inner one uses jq as described in my previous blog post to only retrieve the value I need for the second curl command (the used -r flag for jq returns the raw value, not a beautified one, which would probably make problems when used with other commands). In fish command substitution uses parentheses instead of backticks as in e.g. bash, but the idea is pretty much the same in any shell. This is how such a command can look like:

set TOKEN some-lengthy-but-not-infinitely-valid-token

curl -X GET -H "Authorization: Bearer $TOKEN" localhost/item/(curl -X POST -H "Authorization: Bearer $TOKEN" localhost/item | jq -r .id)

In this case, using the environment variable makes even more sense since the token has to be used twice in the same command, which would make replacing it even more cumbersome.

This small change in writing shell commands can make a huge difference when re-executing commands multiple times, for which reason I wanted to share it with you!