Skip to main content.
home | support | download

Back to List Archive

Re: [SWISH-E:375] RE: Using Parentheses

From: Marjolein Katsma <webmaster(at)>
Date: Sat Jul 18 1998 - 13:51:40 GMT
Roy, and anyone on this list,

At 15:47 1998-07-16 -0700, Roy Tennant wrote:
>In working with Leslie, we determined that the problem was a lack of
>quotes around the search string. I need to check the documentation, which
>I don't think is clear about the need for quotes around the query string
>to make sure it is parsed appropriately.
>Roy Tennant

To which I replied:

>I'm not so sure...
>    [snip]
>What I'm doing is *not* passing the argument within quotes, but I'm
>"escaping" the whole command string.
>The reason, of course is that on Unix '(' and ')' are special characters.
>Escaping the command string makes sure ALL Unix special characters are
>escaped so the complete command is interpreted by Swish. the (PHP)
>statements I'm using:
>         $command = "$swish -w $pattern -m $maxresults -f $index";
>          exec (EscapeShellCmd($command), $result, $rc);
>where $pattern is the exact string typed into the entry field.

Since then,  a private email I received as a reaction to my post has made
me realize that the true significance of my post may not have registered in
every reader's mind, so it is entirely possible it hasn't registered in
your mind either. (If it has, please bear with me - this is also for the
benefit of other members of the list! But please, do, at least, read the
This is in fact extremely important. So let me explain.

1. To have SWISH-E execute a search, one needs to form a command line. That
command line may be typed directly in a shell (for instance in a telnet
session) or it may be built with the user's search string by whaterever
interface you're using. 
2. This command line is then sent to the shell for execution. (By pressing
Enter on the command line, or by executing something like exec() or
system() from the interface.)

3. The shell will interpret the command line, execute what it can, and
return the result (or an error code). 
4. ==> There are many different shells. Different shells have different
special characters with a special meaning in the command line. Even the
same special characters may be interpreted differently or in a different
order by different shells.

When some kind of user interface allows the user to type in a string which
becomes (or becomes part of) a command line, the user interface should take
appropriate action to prevent any special characters from being interpreted
by the shell rather than by the intended program. In particular, a SWISH-E
interface should prevent the shell from interpreting ANY special character
which is typed in as part of a query string.

Actually, the Perl program for a search as generated by AutoSwish quotes
the query string:

    open(SWISH, "$swish -w \"$query\" -m $results -f $index|"); 

just as suggested by you should be done.
However, depending on what shell you're running, this may in fact 1) not
work and 2) leave your sytem wide open to malicious attack!

My literature suggests that Bourne-type shells are most usual on System V
type Unix systems, while C-type shells are usually found on BSD type Unix
systems - but there are no hard and fast rules. But it is entirely possible
that you've only tested your scripts running on a Bourne-type shell, and
never (at least not thoroughly) tested on a C-type shell.

Well, I *have* tested this (I'm running on BSD with a C shell). That was
well over a month ago (right after I translated the script generated by
AutoSwish into PHP) - and I tested with some help from Unix-knowledgeable
and known-to-be-friendly people. One thing became very clear very soon:
enclosing a search string in quotes when that contains any of the special
characters for the C shell DOES NOT WORK. Any of those special characters
will be interpreted *before* the ending quote is. Which means that
enclosing a search string in quotes will not only *not work* when running
on a C shell, it will also leave your system wide open to an attack -
someone may be creative enough to have the system mail them your passwords

This is of course the reason I am NOT enclosing any search string in
quotes: I'm escaping the command line (escaping the query string will have
the same effect) instead. This is of course *also* the reason why PHP
actually has an EscapeShellCmd() function which escapes *every* special
character in its argument. To quote from the PHP documentation:

"EscapeShellCmd escapes any characters in a string that might be used to
trick a shell command into executing arbitrary commands. This function
should be used to make sure that any data coming from user input is escaped
before this data is passed to the Exec() or System() functions."

1. Enclosing a query string in quotes *may* work. Then again, it may not
work  -  and leave your system open to attack.
2. The documentation should not make clear "the need for quotes around the
query string to make sure it is parsed appropriately." as you suggest. What
it *should* do is make clear that an interface to SWISH-E should prevent
*any* special character in a query string from being interpreted by *any*
shell; and that the only secure way to do this is to *escape* every special
3. AutoSwish should be updated to generate a searching command line in the
search script which does not make assumptions about the shell it's running
on, but which is *secure on any shell*. In other words, it should make sure
the query string (or the command line) has every special character escaped.
I'm no Perl expert, but if Perl does not already have a function which does
the equivalent of PHP's EscapeShellCmd() function, I'm sure a Perl expert
would have no problem writing one of those nice Perl-one-liners using
regular expressions to do exactly the same thing.

(Actually, I think it's bordering on the irresponsible to release AutoSwish
which creates scripts which could leave the user's system open to attack.
Of course, you may not have realized this before. But surely now, you do
... So I'm fully expecting the next version of SWISH-E to be corrected as
indicated above!)

Final note: special characters (which may be typed in by a user) which
should be escaped are (separated by spaces for readability):

	& ; ` ' " | * ? ~ < > ^ ( ) [ ] { } $ \

And oh yes, you do of course realize that even people who can install and
compile SWISH-E may *not* have the option of using a different shell?


Marjolein Katsma
Java Woman -
Received on Sat Jul 18 07:01:19 1998