Skip to main content.
home | support | download

Back to List Archive

Running swish with mod_perl

From: Bill Moseley <moseley(at)not-real.hank.org>
Date: Thu May 02 2002 - 00:49:12 GMT
I wanted to revisit running swish-e under mod_perl.

I had done some bench marking about a year ago on some real "production"
code that uses swish-e.  The program ran under mod_perl and I wanted to
compare forking swish-e vs. using the swish-e library (embedding swish-e
into Apache).

At that time I saw very little difference in number of requests per second.
 That was probably due to the overhead of the application that was
formatting and displaying the results.  More "real world" type of results,
you know.

Today I created some very trimmed down perl code to run swish-e.  One is a
very simple mod_perl handler, another is a CGI script that forks swish, and
the third is a CGI script that uses the swish-e library.

Each of these runs below does a query that returns 1160 "hits" from an
index of 23,000 files.  It's just displaying a "page" of results -- the
first 20 (i.e. -m 20).  It's not returning any long properties, just rank,
title, path and file size.

Request per second is what to look at.

Here's the results for a perl *CGI* script that forks and exec swish-e:
(That's a fork for Apache to run the CGI script, and then a fork in the CGI
script to run swish.).

Concurrency Level:      5
Time taken for tests:   49.187 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      1011000 bytes
HTML transferred:       937000 bytes
Requests per second:    10.17
Transfer rate:          20.55 kb/s received


Here's a CGI script that uses the perl library:

Concurrency Level:      5
Time taken for tests:   60.069 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      1003968 bytes
HTML transferred:       929376 bytes
Requests per second:    8.32
Transfer rate:          16.71 kb/s received

That shows you that forking just isn't that expensive.  It looks like the
overhead of Perl loading the swish-e library is more significant than forking.

I'm sure that would be a win if the CGI script was written in C, though.

Now for a mod_perl handler:

Concurrency Level:      5
Time taken for tests:   3.159 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      1003968 bytes
HTML transferred:       929376 bytes
Requests per second:    158.28
Transfer rate:          317.81 kb/s received

Ya baby.  Now that's a bit more like it.

YMMV, as they say.

Now, if I do a query that returns almost 12,000 "hits" (but still only
display the first 20):

Concurrency Level:      5
Time taken for tests:   25.045 seconds
Complete requests:      500
Failed requests:        0
Total transferred:      997449 bytes
HTML transferred:       923005 bytes
Requests per second:    19.96
Transfer rate:          39.83 kb/s received

Ok, it's not Google, but I doubt they are running on $800 machines either.

And in this case, the CGI script that forks doesn't do that "much" worse:

Concurrency Level:      5
Time taken for tests:   14.261 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      201100 bytes
HTML transferred:       186300 bytes
Requests per second:    7.01
Transfer rate:          14.10 kb/s received

And to be complete, the perl CGI using the swish-e library:

Concurrency Level:      5
Time taken for tests:   16.850 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      202266 bytes
HTML transferred:       187170 bytes
Requests per second:    5.93
Transfer rate:          12.00 kb/s received




-- 
Bill Moseley
mailto:moseley@hank.org
Received on Thu May 2 00:49:25 2002