Day 17 - Stupid SSH tricks

Written by: Corey Quinn (@kb1jwq)
Edited by: Ben Cotton (@funnelfiasco)

Every year or two, I like to look back over my client's SSH configuration file and assess what I've changed.

This year's emphasis has been on a few options that center around session persistence. I’ve been spending a lot of time on the road this year, using SSH to log into remote servers over terrible hotel wireless networks. As a result, I’ve found myself plagued by SSH session resets. This can be somewhat distracting when I’m in the midst of a task that requires deep concentration— or in the middle of editing a configuration file without the use of screen or tmux.

ServerAliveInterval 60

This triggers a message from the client to the server every sixty seconds requesting a response, in the event that data haven’t been received from the server in that time. This message is sent via SSH’s encrypted channel.

ServerAliveCountMax 10

This sets the number of server alive messages that will be sent. Combined with ServerAliveInterval, this means that the route to the server can vanish for 11 minutes before the client will forcibly disconnect. Note that in many environments, the system’s TCP timeout will be reached before this.

TCPKeepAlive no

Counterintuitively, setting this results in fewer disconnections from your host, as transient TCP problems can self-repair in ways that fly below SSH's radar. You may not want to apply this to scripts that work via SSH, as "parts of the SSH tunnel going non-responsive" may work in ways you neither want nor expect!

ControlMaster auto
ControlPath ~/.ssh/%r@%h:%p
ControlPersist 4h

These three are a bit interesting. ControlMaster auto permits multiple SSH sessions to opportunistically reuse an existing connection, the socket for which lives at ControlPath (in this case, a socket file that lives at ~/.ssh/$REMOTE_LOGIN_USENAME@$HOST:$SSH_PORT). Should that socket not exist, it will be created— and thanks to ControlPersist, it will continue to exist for four hours. Taken as a whole, this has the effect of causing subsequent SSH connections (including scp, rsync (provided you’re using SSH as a transport), and sftp) to be able to skip the SSH session establishment.

As a quick test, my initial connection with these settings takes a bit over 2 seconds to complete. Subsequent connections to that same host complete in 0.3 seconds -- almost an order of magnitude faster. This is particularly useful when using a configuration management that’s establishing repeated SSH connections to the same host, such as ansible, or salt-ssh. It’s worth mentioning that ControlMaster was introduced in OpenSSH 4.0, whereas ControlPersist didn’t arrive until OpenSSH 5.6.

The last trick is a bit off the topic of SSH, as it’s not (strictly speaking) SSH based. Mosh (from “mobile shell”) is a project that uses SSH for its initial authentication, but then falls over to a UDP-based transport. It offers intelligent local echoing over latent links (text that the server hasn’t acknowledged shows up as underlined locally), and persists through connection changes. Effectively, I can start up a mosh session, close my laptop, and go to another location. When I connect to a new wireless network, the session resumes seamlessly. This has the effect of making latent links far more comfortable to work with; I’m typing this post in vim on a server that’s currently 6000 miles and 150ms away from my laptop, for instance.

As an added benefit, mosh prioritizes Ctrl-C; if you’ve ever accidentally catted a 3GB log file, you’ll appreciate this little nicety! Ctrl-C stops the flood virtually instantly.

I will say that mosh is relatively new, and implements a different cryptography scheme than SSH does. As a result, you may not be comfortable running this across the open internet. Personally, I run it over OpenVPN only; while I have no reason to doubt its cryptography implementation, I tend to lean more toward a paranoid stance when it comes to new cryptographic systems.

Ideally this has been enlightening; SSH has a lot of strange options that allow for somewhat nifty behavior in the face of network issues, and mosh is a bit of a game-changer around this space as well.