Skip to main content.
home | support | download

Back to List Archive

Re: [SWISH-E:381] RE: Using Parentheses

From: Marjolein Katsma <webmaster(at)>
Date: Sat Jul 18 1998 - 16:40:33 GMT

At 08:01 1998-07-18 -0700, Eli's List Clearing House wrote:
>Marjolein Katsma wrote:
>> 1. To have SWISH-E execute a search, one needs to form a command line. That
>One needs to fomr an argument vector.

OK. (terminolgy...)

>> 2. This command line is then sent to the shell for execution. (By pressing
>This need not be true. One can exec() the program directly, but this is
>more complicated.

I'd like to learn sometime. (Not now.)

>> 3. The shell will interpret the command line, execute what it can, and
>> return the result (or an error code). 
>When the shell is involved, yes.
>> 4. ==> There are many different shells. Different shells have different
>> special characters with a special meaning in the command line. Even the
>> same special characters may be interpreted differently or in a different
>> order by different shells.
>True. BUT I am fairly confident that perl will use /bin/sh (Bourne shell)
>for system() and open(FOO,"...|"). Run these to check:
>perl -e 'system(q(ps|grep $$))'
>perl -e 'open(FOO, q(ps|grep $$|));print <FOO>'

OK. Like I said, I'm no Perl expert. I'm not even *using* Perl. But I did try.
The first command results in:

10598  p0  S+     0:00.01 sh -c ps|grep $$
10600  p0  S+     0:00.01 grep 10598

The second results in:

10700  p0  S+     0:00.01 sh -c ps|grep $$

I'll admit I don't know what those responses mean. If the 'sh' in there
means a Bourne shell is being used by Perl, fine.
But... 1) I'm not using Perl, and 2) Roy Tennant's reply did not suggest
anyone should or would, either. All he said was that a query string should
be enclosed in quotes. (But I'm saying documentation for SWISH-E should not
*assume* everyone is using Perl!)

>> When some kind of user interface allows the user to type in a string which
>> becomes (or becomes part of) a command line, the user interface should take
>> appropriate action to prevent any special characters from being interpreted
>> by the shell rather than by the intended program. In particular, a SWISH-E
>> interface should prevent the shell from interpreting ANY special character
>> which is typed in as part of a query string.
>You should be runing with perl in "taint" mode since that will make perl
>complain loudly if you do insecure things. Use "#!/path/to/perl -T" as
>the first line of the perl script. And read the perlsec(1) man page (or
>"perldoc perlsec".)

Are you saying whatever is generated by AutoSwish should be running in
"taint" mode (whatever that is)?
(I'm not using Perl, and AutoSwish seems to be developed for people who
don't write their own scripts: How would they know what to do or change?)

>> Actually, the Perl program for a search as generated by AutoSwish quotes
>> the query string:
>>     open(SWISH, "$swish -w \"$query\" -m $results -f $index|"); 
>Double quotes are bad there.
>      open(SWISH, "$swish -w '$query' -m $results -f $index|"); 
>Double quotes allow things to be interpolated, single quotes do not.

So (even if for different reasons) you seem to agree with me that code
generated by AutoSwish is not,in fact, what should be generated, - let
alone what should be used. But as I understand it, AutoSwish is meant for
those people who don't write their own scripts (in whatever language) so it
should not assume such knowledge of the user.

>> However, depending on what shell you're running, this may in fact 1) not
>> work and 2) leave your sytem wide open to malicious attack!
>Quoting conventions of "" vs '' are the same in sh, csh, ksh, bash, zsh,
>tcsh, and probably most other shells. (Perl, too). Note that quotes within
>quotes are different. In the open I wrote above perl will not see the
>single quotes as special, since they are in double quotes. When the
>shell gets it, there will be no outer quotes and the inner ones will
>be active.

Like I said, quoting the query string with just "" does NOT work in the
plain C shell I'm using. I haven't tried single quotes yet, though. (But
would single quotes, on their own, be any different than double quote on
their own? By "on their own" I mean not enclosed/embedded in quotes of the
other type.)

>> My literature suggests that Bourne-type shells are most usual on System V
>> type Unix systems, while C-type shells are usually found on BSD type Unix
>Virtually all modern Unices have both sh and csh. (Unless they have 
>bash and tcsh, which are free.)

Do I "have" it? Maybe. Even probably, if the results from your test Perl
commands mean what they suggest. BUT not everyone is in fact using Perl
(I'm not). So not everyone is using sh!

>> Well, I *have* tested this (I'm running on BSD with a C shell). That was
>> well over a month ago (right after I translated the script generated by
>> AutoSwish into PHP) - and I tested with some help from Unix-knowledgeable
>What is PHP?

"PHP Hypertext Preprocessor". A server-side embedded scripting language. To
steal a competing product's slogan:  "The document *is* the application."
Depending on what you need to do, either PHP or Perl could be the best
suited. But if you *can* use PHP, you often need fewer lines of code than
you would in Perl for the same functionality. If you need to interface to a
database, PHP has built-in support for many databases, from freeware mSQL
to industry-standard Oracle. See  for more.

(Please don't misunderstand me: I'm not against Perl in any way. But I do
believe in using the right (or most efficient!) tool for the job, and I
don't like learning too many things at once. Learning PHP and re-learning C
while at the same time learning some Unix and web server configuration is
about all I can handle in my own time right now. Perl's on my list for
tackling later... And I also have a day-time job ;-) )

>> and known-to-be-friendly people. One thing became very clear very soon:
>> enclosing a search string in quotes when that contains any of the special

>> characters for the C shell DOES NOT WORK. Any of those special characters
>There are some bugs in that, because I have not kept it in sync with

>the cgi I use, but those bugs are not signficant security holes --
>since I do not escape "-"s in the input, the user can pass extra
>options to swish-e. 

Hmm. Interesting idea! (Create a special form for "power users" to exploit

>The other bugs are cosmetic. It is necessary
>to run that with perl version 5.004 or higher, or else the delete()
>statement might fail. Since there are known security holes in all
>previous versions of perl, you should be running 5.004 anyway for
>external services.

Thanks for pointing that out. I do know I have "a" Perl 5 available but I
don't know which subversion. I'm sure you have a special command that
enables me to find out? I might want to use Perl in the future so it would
be nice to know!

>One major problem RE this discussion is that that cgi will not /let/
>the user put parenthesis into the query.
>Maybe I will try to fix it this weekend.

But mine (using PHP) does... <grin> - see my suggested examples! Better
than "see": try!

>> I'm no Perl expert, but if Perl does not already have a function which does
>> the equivalent of PHP's EscapeShellCmd() function, I'm sure a Perl expert
>> would have no problem writing one of those nice Perl-one-liners using
>> regular expressions to do exactly the same thing.
>	$query =~ s/([^A-Za-z0-9])/\\$1/g;

<grin> in return. I might even be able to translate that into PHP. Thanks!

>Or use quotemeta:
>	$query = quotemeta($query);
>> (Actually, I think it's bordering on the irresponsible to release AutoSwish
>> which creates scripts which could leave the user's system open to attack.
>I would agree. 

Interpreting what you've said so far, it seems you do agree with me that:
1) the documentation of SWISH-E needs to be updated, and
2) the scripts generated by AutoSwish should be changed to make sure they
are secure. 

That, of course is the very reason I'm taking the effort of writing these

So I'll amend my suggestion (CONCLUSIONS point 3.): That AutoSwish use
either your nice one-liner to escape all non-alpha-numeric characters in a
query string, or use quotemeta() for the same purpose. Do you agree?

>> Final note: special characters (which may be typed in by a user) which
>> should be escaped are (separated by spaces for readability):
>> 	& ; ` ' " | * ? ~ < > ^ ( ) [ ] { } $ \
>You should not rely upon such a list, but should escape all non-
>alphanumerics so you don't have to worry about new evironments.

Good point! That list is in fact what I grabbed from PHP. But see above -
I'll see if I can translate your one-liner into PHP - quite likely. I
*like* not worrying about future environments. (Isn't that what Y2K is all
about? :-))

>> And oh yes, you do of course realize that even people who can install and
>> compile SWISH-E may *not* have the option of using a different shell?

>If they don't have the option of using perl with the latest security
>fixes then they will never be able to write a secure perl cgi.

True - but like I said, not everyone is using Perl - let alone *writing*
Perl - which is the whole reason for the existence of AutoSwish!
It's all too often that I see "CGI" equated with "Perl", and it isn't of
course. (I didn't mean you were - just that it's very common to assume the

You can use any server-side language that can handle a CGI interface
(interpreted or compiled) to write a web interface to the SWISH-E engine.
PHP, ASP, MIVA, Cold Fusion, etc. and yes, of course Perl, too.
Instructions for what to do with a user-entered query string should *not*
assume any particular language or any particular shell being used, but
should point out (in generic terms) what needs to be done to make sure 1)
the intended (entered) query string gets passed on the SWISH-E search
engine completely and 2) what needs to be done (again, in generic terms) to
make sure the system is secure from query strings entered by malicious
users or by users simply making a typo or even typing something in good
faith but not quite knowing what harm they *could* be doing.

I think you agree that:
1) the code generated by AutoSwish is not (always) secure, and
2) the SWISH-E documentation should *not* suggest that enclosing a query
string in (double) quotes is all you need to do to make it work.

Do you?

>now working on an Apache/mod_perl version of his search.cgi

(Now working on a new parsing algorithm for SWISH(-E or whatever) that
correctly replaces entities and correctly parses HTML comments <grin>)


Marjolein Katsma
Java Woman -
Received on Sat Jul 18 09:50:10 1998