Re: [hylafax-users] Document conversion failed

["Darren Nickerson" <darren.nickerson@xxxxxxxx>]

"Lee Howard" <faxguy@xxxxxxxxxxxxxxxx> wrote:

> Darren Nickerson wrote:
>
> > Whereas HylaFAX used to bog down with faxq consuming 99% CPU and jobs 
> > submitting _VERY_ slowly, it should now be pretty zippy.
>
>
> I can't say that I recall the last time I saw faxq consume 99% CPU (or 
> anything close to even half of it)... and I routinely stuff between 500 or 
> 1000 jobs into the outbound queue... this without the major refactoring.

I guess maybe 500 or 1000 jobs isn't sufficiently large to uncover the 
problem these days.  10,000 jobs is not a contrived scenario - we work with 
a lot of people who submit that many jobs to their servers in one go. I 
first introduced the problem in 2002 because it was even worse back then:

   http://bugs.hylafax.org/show_bug.cgi?id=344

although there was some improvement from the work in bug 667, we really just 
moved the goalposts a bit. There was still a fundamental problem queueing 
jobs with long queues and a more comprehensive solution was needed. It 
really was an algorithmic problem ... we were limited by the way faxq was 
written and unless things were restructured we'd only be able to acheive 
incremental improvements. We finally took the plunge in 4.4.x ;-)

> I did look through all of the code changes in that work, and I can't see 
> where any significant number of CPU cycles would be getting saved.  I'd 
> enjoy seeing some real statistical comparisons of faxq performance between 
> 4.3.4 and 4.4.0... and then 5.1.8.  ;-)

Here's some rough numbers for the time taken to queue 10,000 jobs in one go:

HylaFAX+-5.1.9: 07:03:58
HylaFAX-4.3.5: 03:06:18
HylaFAX-4.4 CVS: 00:02:52

That's 7 hours for HylaFAX+, 3 hours for HylaFAX-4.3.x, and about 3 minutes 
for HylaFAX-4.4 CVS commit 2d0b444c473ce4d7b0e5bd8da14725e2923f16e0).

The command we ran was:

   sendfax -h test@localhost -n -t 1 -T 1 -z diallist-10000.txt 
somefile.pdf

and the diallist file contained 10,000 unique numbers.

Some settings from faxq config:

LogFacility:  local1
CountryCode:  1
AreaCode:  415
LongDistancePrefix: 1
InternationalPrefix: 011
DialStringRules: etc/dialrules
SendFaxCmd:  /usr/sbin/faxsend
Use2D:   yes
DestControls:  etc/destcontrols
PollLockWait:  1
MaxConcurrentCalls: 100
MaxBatchJobs:  2
MaxConcurrentCalls: 1
NotifyCmd:  /bin/true
ServerTracing:  0

I know that with benchmarks the devil is in the details, and so I'm the 
first to consider them suspect. These may be wrong. This was on a dual AMD 
Athlon box with a Raid5 array (which was running in degraded mode due to a 
failed disk so reduced I/O performance). We can repeat the test and gather 
more details if anyone cares. I find the above numbers a little suspect 
myself, but we repeated the tests twice. I'm particularly concerned by the 
slowness of HylaFAX+ versus 4.3.5 - that would seem unlikely to me, and 
should probably be reconfirmed. But we definitely did expect 4.4.x to sing 
compared to 4.3.x.

How about you run similar tests, show us the results you get, and if they're 
dramatically different we can dig into what we may have done wrong here on 
our side?

-Darren 


____________________ HylaFAX(tm) Users Mailing List _______________________
 To subscribe/unsubscribe, click http://lists.hylafax.org/cgi-bin/lsg2.cgi
On UNIX: mail -s unsubscribe hylafax-users-request@xxxxxxxxxxx < /dev/null
 *To learn about commercial HylaFAX(tm) support, mail sales@xxxxxxxxx*