How to transfer binary files using only bash built-in commands
The Problem
You were experimenting with something on a remotely-accessed server as root, and suddenly you typed the wrong command and nuked the system, or broke it in such a horrible way that normal programs like cp, mv, ls, etc., don't work anymore. However, most of your filesystem is still intact, and you figure that if you could only transfer a rescue utility to the machine (such as a statically-linked version of busybox, which would still run even if you hosed your system libraries or broke your dynamic linker), you could still salvage the system without having to reinstall it from scratch.
However, since useful commands like cp don't work anymore, you can't transfer any files directly to the system from, say, a network drive. Needless to say, ftp, wget, and a bunch of other useful utilities don't work, either. The only thing that's still working is that one instance of the bash shell over that one last remote connection you still have open to the system.
What can you do?
A Glimmer of Hope
Fortunately, the bash shell is rather versatile, and in the place of programs like ls, you could use commands such as:
echo *
or type the name of the directory and then use tab-completion to see its contents.
Furthermore, even though you can't use a text editor, you can still use the bash shell's built-in echo command with output redirection to create text files piece-by-piece. You could even use a loop to achieve cat-like functionality:
(IFS=$'\n';while read line;do echo "$line";done) < inputfile
will allow you to see the contents of a text file, and, by adding a few more things to the loop, you can achieve rudimentary versions of head and tail. Inverting the redirection lets you create files instead:
(IFS=$'\n';while read line;do echo "$line";done) > outputfile
Simply type in the contents of the file, and end with a single control-D on the last line to end the loop.
However, this still doesn't address the ultimate solution to the situation: how do you transfer binary files across a text terminal? Trying to copy-n-paste an executable or a library file into the above hack version of cat does not correctly preserve the binary data of the file, since the bytes get interpreted by the shell and get mangled. A single mangled byte in an executable may very well completely break it. You may even run the risk of getting logged out of your very last working session to the remote machine, if your binary file happens to contain the wrong byte sequences.
Enter echo
Well, it is possible to write binary data using echo, by using escape sequences of the form \xhh, where hh is a 2-digit hexadecimal number. For example, the command:
echo -ne $'\x41\x42\x43'
prints ABC; the bytes \x41, \x42, and \x43 being the ASCII codes for the three respective letters. Of course, you can also output non-printable bytes using this escape sequence.
There are a few gotchas, however. Attempting to write a binary zero as \x00 does not work, because it causes the echo command to terminate prematurely. Since executables and library files tend to contain a lot of binary zeroes, it's rather important that we be able to write them. This can be achieved by using an octal sequence instead:
echo -ne $'\\0000'
This is the correct way to output a binary zero. Furthermore, the backslash character is prone to getting interpreted by the shell once more than you'd expect; thus the following command:
echo -ne $'\\nancy'
does not output \nancy, but rather a linebreak followed by ancy, because \\ gets translated to \, and then it joins with the next character to form \n, which gets translated into a linebreak. Hiding the backslash as \x5c still isn't immune to this effect. The correct way of representing a literal backslash in echo is four backslashes:
echo -ne $'\\\\'
Now, armed with this knowledge, we're ready to write binary data to the remote system. On your local system, write a script to translate a binary file into a sequence of echo commands, encoded as described above, that recreates the binary file on the remote system. In order not to overflow the maximum input line length on the remote shell, of course, the binary file will have to be encoded in small chunks of about 64 or 128 bytes each.
Copy-n-paste, 1000 times?
Now, any binary file that is capable of rescuing your remote system is probably so large that you'll need at least several thousand echo lines to recreate it. Especially if you hosed your system libraries or broke your dynamic linker, in which case the executable has to be statically-linked, which makes them a whole lot bigger than they'd normally be. This could easily translate to tens of thousands of echo commands.
You can't possibly type that many echo commands within any reasonable amount of time; so you'd have to write a script to generate them, and then copy-n-paste them to the remote shell. Unfortunately, copying and pasting has its own gotchas: the shell normally alters some terminal settings after every command, or otherwise does some stuff (perhaps some flow control operations) that takes a bit of time to complete. If you try pasting 10000 lines into the shell prompt, you'll discover that some of the lines get truncated or otherwise garbled, due to the shell not being able to catch up with that many input commands at a time. All it takes is for one byte to go wrong among those 10000 lines, and your executable is corrupted and won't run.
So it seems that you must only paste a small number of lines each time, say about 10-12 or so. But to recreate a binary file that requires 10000 echo lines by pasting 10 lines at a time means you have to copy-n-paste 1000 times. That's 1000 opportunities for just one paste to be wrong, and your executable is again screwed up. Besides, cutting and pasting 1000 times isn't exactly a very practical solution to a probably urgent problem, since the remote server being down is likely to be causing financial loss. You don't have all day to copy-n-paste 1000 times.
Copy-n-paste, Once
Well, fortunately, it is possible to copy-n-paste just once to transmit a binary file to the remote system using only echo commands. The insight is that the reason the shell truncates or garbles some lines is because it's fiddling with the terminal after every command. What we need to do is to bypass this fiddling by using a trick we saw earlier:
(IFS=$'\n';while read line;do echo "$line";done) > outputfile
Remember that we said we couldn't transmit a binary file using this command? That's only true if you try directly pasting binary data into the terminal. This little trick will readily accept textual data, including our echo commands, and accept them at a very high rate that lets us copy-n-paste 10000 commands without any mangling, writing them to a file on the remote system. Once those echo commands are written to the file, we just tell bash to run them:
(source outputfile)
And this will recreate our binary file!
... right?
Well, not quite. If you try to copy-n-paste our echo commands directly into that read line loop, you'll discover that the contents of the remote file aren't quite what you might expect. The trouble is, that loop writes out the data using another echo command, which means that metacharacters in the data gets yet another level of interpretation by the remote shell, causing the resulting file to contain echo commands with prematurely-expanded binary data. Running such a file will not recreate our binary file at all, just a highly-mangled version of the hexadecimal escape sequences.
In order for our echo commands to withstand another level of shell interpretation, we need to double-escape them. Meaning that instead of \xhh, we have to write \\xhh, and instead of \\0000, we have to write \\\\0000, and instead of \\\\, we have to write \\\\\\\\. (Leaning Toothpick Syndrome, anyone?) Once our echo commands with these doubly-escaped sequences get pasted into the read line loop above, the remote shell will translate them into the correct form that can recreate the binary file.
Use the source, Luke!
Good news! You don't have to pull through all those shenanigans in order to get those echo commands into the right form for transmitting via a terminal. I've written a little utility that does it automatically, and even spells out that read line loop that you have to type in your remote shell prompt.
Compile the program with a C compiler, and use it to convert any binary file into a sequence of echo commands:
echoize /bin/busybox test > busybox.echo
This writes echo commands to recreate /bin/busybox as a file named test. (If the second parameter is omitted, the resulting commands will write to the original filename: be careful with that on your local system in case something goes wrong and you overwrite something you didn't intend to.) You can then run the script and check that the file test is identical to /bin/busybox, byte-for-byte:
(source busybox.echo) ; diff /bin/busybox test
If you examine busybox.echo, you'll see that this is the singly-escaped version of the echo commands. It is not safe for pasting into the remote shell! For that, you'll need to double-escape the echo commands, which fortunately the program will do for you if you specify the -d option:
echoize -d /bin/busybox > busybox.paste
Now, on the remote shell, start the read line loop:
(IFS=$'\n';while read line;do echo "$line";done) > busybox.echo
(If you ever forget exactly how this loop is spelled out, just run echoize with the -h option and it will spell it out for you, as well as list a number of other options you may find helpful.)
Now copy the entire contents of the file busybox.paste into your clipboard. If you're using X Windows, it's best to do this with the handy xclip utility, which can read directly from a file and thus minimize the chances of botching the copy-n-paste operation:
xclip -i < busybox.echo
Now paste everything into the remote shell. This may take a while, depending on the speed of your connection. After it's all done, type control-D to terminate the read line loop, and run the resulting script on the remote prompt:
(source busybox.echo)
If all goes well, this will recreate the busybox binary on the remote system.
Run, boy, run!
There's one final problem, though. A file written using echo is not executable, and the shell's umask command doesn't do it either, since it only sets the maximum permissions, not the default permissions. (Remember, the remote system is hosed so chmod doesn't run.) So now that you've finally transferred your executable file over, how do you make it executable?
Well, observe that even though a file created by echo is not executable, a preexisting executable file overwritten by echo is still executable. That means that all you need to do is to find an existing executable, and replace it. (That's why my little utility has that second parameter that lets you specify what name the file should be recreated under. You may have to use a non-conventional name, such as feedback.py, for a utility like cp, if you've completely hosed all your /bin directories. Don't laugh, it happens!) After the file is overwritten, you now have an executable version of your binary file!
Hopefully, your new-found ability to transfer binary files using only bash built-in commands will help you recover your system before your boss fires you. ☺
Words of Advice
Throughout, I've used busybox as an example binary file that can be very useful in recovering your broken system. Since busybox can be compiled as a statically-linked executable, it is highly advisable to keep a static copy of it somewhere in all your systems, in case you ever need it for emergencies. It is also wise to keep this copy outside the usual places for binaries, such as /bin, /sbin or /usr/bin. You never know if you accidentally remove everything in these directories. (It has actually happened to someone when a packaging utility gone wrong decided to uninstall everything in the system, and proceeded to do just that until it broke itself: all the system directories were emptied by then.)
Of course, the more relevant advice is that the root account is very dangerous. You have the power to completely shoot yourself in the foot and then wipe all evidence of it in a single command. Especially when you are logged in as root, think before you act! Commands like rm and wildcards like * really don't mix. All it takes is for you to accidentally insert a space in rm ./* to turn it into rm . /*, and your day is ruined. This is just one of many examples.
Credits
I owe the echo trick in large part to David Fendrich's blog, which I stumbled across when faced with a system whose dynamic linker had been hosed, causing all dynamically-linked executables (which is pretty much all executables) to be unexecutable. Unlike him, my only remaining connection to the remote machine was not under Konsole, and new ssh sessions would not start due to it requiring the dynamic linker at some point in the authentication process. So I did not have the luxury of spooling those echo commands to the remote shell via DBUS at a rate that won't get things garbled, and had to make do with a bare copy-n-paste to transfer the rescue binary.
