Shell escaping in Perl

Shell Escaping:

There are several methods for escaping special characters in the Linux shell:

  • Double quotes: Double-quoted strings require escaping of only a few characters.  Example:
    >echo "Some String: &>|\"\$'\`\s\\"
    Some String: &>|"$'`\s\

    This method is useful as long as the string contains few or no characters that are special to the shell.  It also allows embedding all types of quotes.
  • Single quotes: Single-quoted strings require no escaping at all.  Example:
    >echo 'Some String: &>|"$'\''`\s\'
    Some String: &>|"$'`\s\

    This method is even shorter, but has a major drawback: Strings may not contain single quotes.  To get around this, “glue” is used, where the shell automatically concatenates adjacent strings.  This requires multiple strings, and any single-quote must then be escaped using one of the other methods.
  • Dollar quotes: Dollar-quoting requires minimal escaping and allows all quotes inside.  Example:
    >echo $'Some String: &>|"$\'`\s\\'
    Some String: &>|"$'`s\

    With this method, only single-quotes and backslashes must be escaped.  This method also tolerates unnecessary escaping.
  • Unquoted: Unquoted strings may be used by escaping all special characters.  Example:
    >echo Some\ String:\ \&\>\|\"\$\'\`\s\\
    Some String: &>|$"'`s\

    The main advantages of this method are that it has a very simple rule-set (escape everything!), and it also tolerates unnecessary escaping.

Perl System Calls:

Perl interacts with the shell in several different ways: system() and exec() calls, back-ticks (“) and the qx// operator, open() pipe’d filehandles, etc.  Often commands include some interpolation of variables, and these variables may contain special characters.  Forgetting to escape such characters can be a security risk, as it’s very easy for special characters (such as semi-colon (;) the command separator, pipe (|) command chaining, greater-than (>) redirection, etc) to sneak their way into a command and wreak havoc.

system(), exec(), and open() calls that use comma-separated arguments (list form) escape automatically, so this is the preferred method of dealing with escaping:

my $var = '&>|"$\'`\s\\';
system ( 'echo', "Some String: $var" );
open ( my $fh, '-|', 'echo', "Some String: $var" );

However, this method does not use the shell (it executes directly using the internal system() function), so shell interpolation is not supported.  This means no globbing (*,?,[],{}), piping (|), redirects (>), background (&), etc – so if you want any of those, then do not use the list form.

Interpolated (single-string, backticks, and qx//) calls require escaping.  It’s tempting to use one of the various quoting methods listed above and manually escape special characters as needed, but Perl provides functionality that best supports the unquoted escaping method.  The built-in function quotemeta(), and its equivalent interpolation \Q…\E are perfect for this job:

my $var = '&>|"$\'`\s\\';
system ( "echo \QSome String: $var\E" );
open ( my $fh, "echo \QSome String: $var\E|" );
my $result = `echo \QSome String: $var\E`;

Note that shell quoting escaping rules differ from Perl quoting escaping rules – specifically, single quotes in Perl allow embedding single-quotes (as long as they are escaped), interpolation (double-quoting) requires escaping of a different set of characters considered special, and Perl tolerates unnecessary quoting where the shell does not.

Nesting:

Nesting strings that require escaping can be daunting some times…  Consider this command:
>sh -c "echo \"Some String: &>|\\\"\$'\\\`\s\\\\\""
Some String: &>|"$'`\s\

This double-escaping mess can be reduced somewhat by using different quotes, but applying the right rules will give you a headache:
>sh -c 'echo "Some String: &>|\"\$'\''\`\s\\"'
Some String: &>|"$'`\s\

Fortunately, Perl’s quotemeta() takes care of all this, so nesting is straightforward:
my $var = '&>|"$\'`\s\\';
system ( "sh -c \Qecho \QSome String: $var\E\E" );

5 thoughts on “Shell escaping in Perl

  1. Thank you very much for this instructive page. This situation has been plaguing me for a few weeks. I’m trying to run wput from a perl script, and the quoting was driving me crazy. I finally got it to u/l 1 file, but now trying to u/l several sequential files using square brackets as in: [1..8].mp3 with no joy. Any ideas on that please?

  2. Yes, I understand. Here is the code I’m trying in a Perl script from a shell window:
    system ('/usr/local/bin/wput', "[123].mp3", "ftp:// ...
    That errors out with Error: File `[123].mp3' does not exist. Don't know what to do about this URL.
    Nothing done. Try `/usr/local/bin/wput --help'.

    I’ve also tried various quotes but without understanding it completely. Very frustrating.

    1. As mentioned in my post: “system(), exec(), and open() calls that use comma-separated arguments (list form) escape automatically”.
      In this case, you do not want to escape, so do not use the list form.
      system (‘/usr/local/bin/wput [123].mp3 ftp:// …’);

  3. OMG. I was making this too hard. Thank you so much.
    system ('/usr/local/bin/wput [1357].mp3 ftp:// wth NO inner quotes. Worked like a charm – only those file were u/l. What a great way to start the new year. Thank you again.

Leave a Reply to Amer Neely Cancel reply

Your email address will not be published. Required fields are marked *