HylaFAX The world's most advanced open source fax server
[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [hylafax-users] Performance Tuning and sendfax -S

Hi again,

thanks for the hints in the right direction! Due to your ideas, our setup is pretty fast now :) [sure: I am always listing if anyone has ideas to make it even faster]. For those interested, our results/conclusions follow below.
----

First: the initial tests, 2 days ago, have been performed using HylaFAX 4.2.1 (I thought it was 4.2.5 - I was wrong).

Now we have upgraded and I am referring to HylaFAX 4.2.5.

We also reinstalled one Diva modem (ttyds01) using faxaddmodem and declared it there as Class 2 modem (as Darren suggested).

After first tests we understood the core performance problem: V.34. If the modem (Diva Server PRI, TTY setup [ttyds01... ttyds30]) is using V.34 then it's fast.


After some more tests (see logs below) we saw that we also made a mistake with the sendfax commandline (we prevented the use of V.34).

Like this, using Class 2, no usage of V.34, transmission is SLOW:
sendfax -3 -m -F "Tagline" -d "+41..." input.tif

Like this, using Class 2, usage of V.34, transmission is FAST:
sendfax -m -F "Tagline" -d "+41..." input.tif


Now the reproducable performance results of this 1-page-pdf (converted to a TIFF, see below), sent out using fine resolution, to a V.34 recipient(!), do look like this:

Class 2, with "sendfax -m ...":       27 seconds processing time
Class 1, with "sendfax -3 -m ...":    45 seconds processing time
Class 2, with "sendfax -3 -m ...":    79 seconds processing time

In theory we would prefer to send using Class2/V.34 by default and as a fallback (recipient does not support V.34) we would like to use Class1. But we did not have the time yet to figure out how this works... :)


Additionally we changed the production of the TIFF output file. We wanted to increase the quality (input is always a PDF) of the output file and we wanted to reduce the size of the output file. We do this now like this [ImageMagick needed] (works for single page and multi page pdf's):

gs -q -dNOPAUSE -dBATCH -sPAPERSIZE=a4 -dFIXEDMEDIA -dUseCropBox -r209x196 -sDEVICE=pnggray -sOutputFile=in.pdf.out_%03d.png in.pdf

convert +antialias +dither in.pdf.out_*.png -modulate 90 -contrast -dither -monochrome in.pdf.out.tif

sendfax -m [...] in.pdf.out.tif


It is obvious that this procedure costs extra CPU time. Nevertheless we decided to use it. Reasons:

a) the "Plain Vanilla" sendfax -some-opts some.pdf did not work for us using more complex pdf's. Especially combination of images, tables and (small font) text gave problems. Either the output was not as close as possible to the original doc or the faxed document was shifted to the upper right corner (and therefore useless). We did not investigate further, why the Plain Vanilla sendfax / pdf2fax / ps2fax was not working with these "complex" input PDF docs.

b) -modulate 90 and -contrast. These two options do change the image slightly. Simplified: it "increases the whitespace a bit". This leads to shorter transmission times. We believe that the recipient will NOT notify the change of the contrast. But we save 2 seconds transmission time per page of our "typical" PDF's (mix of some logo, small image and mostly text) and up to 10 seconds per page on a more complex document (images - text - tables). It costs us about 13 seconds additional CPU time per page (only once - during tif creation). Since we send out one doc to more than 50 recipients... it saves us quite a bit transmission time.


Config and log files extracts below:
------

-- The "fast" config.ttyds01 --
[...]
ModemSoftResetCmdDelay:	0
ModemResetDelay:	0
#

ModemType:		Class2
ModemRate:		57600	# on boards supporting V.34-fax,
				# this may need to be 57600
ModemFlowControl:	rtscts
ModemNoFlowCmd:		AT&K0
ModemSoftFlowCmd:	AT&K4
ModemHardFlowCmd:	AT&K3

Class2APQueryCmd:	none
Class2SPLCmd:		none
Class2TBCCmd:		none
Class2ECMType:		2.0	# follows Class 2.0 spec, not Class 2


Logs / 1. Diva Call History, Extract, always to the same fax number
-------------------------------------------------------------------
[...]
Sat Mar 25 17:14:55 2006 Speed: 33600/33600 [Fine][ECM][T6][V.34] 0:0:27
Sat Mar 25 17:16:55 2006 Speed: 33600/33600 [Fine][ECM][T6][V.34] 0:0:27
Sat Mar 25 17:20:48 2006 Speed: 9600/14400  [Fine][ECM][T6]       0:1:19
[...]

Logs / 2. HylaFAX xferfaxlog, Extract
---------------------------------------
[...]
03/25/06 17:14 SEND ttyds01 "+41..." 2170921 1 0:27 0:09 ... "00 00 00"
03/25/06 17:16 SEND ttyds01 "+41..." 2170921 1 0:27 0:09 ... "00 00 00"
03/25/06 17:20 SEND ttyds01 "+41..." 2170921 1 1:18 0:39 ... "00 00 00"
[...]


Logs / 3a. HylaFAX, slow session
--------------------------------
Mar 25 17:20:48.07: [16944]: SESSION BEGIN 000000101 41...
Mar 25 17:20:48.07: [16944]: HylaFAX (tm) Version 4.2.5
Mar 25 17:20:48.07: [16944]: SEND FAX: JOB 106 DEST +41...
  COMMID 000000101 DEVICE '/dev/ttyds01' FROM ...
Mar 25 17:20:48.07: [16944]: <-- [10:AT+FBOR=0\r]
Mar 25 17:20:48.07: [16944]: --> [2:OK]
Mar 25 17:20:48.07: [16944]: <-- [13:AT+FPHCTO=30\r]
Mar 25 17:20:48.07: [16944]: --> [5:ERROR]
Mar 25 17:20:48.07: [16944]: MODEM Command error
Mar 25 17:20:48.07: [16944]: <-- [24:AT+FDCC=1,5,2,2,0,2,0,0\r]
Mar 25 17:20:48.07: [16944]: --> [2:OK]
Mar 25 17:20:48.07: [16944]: DIAL ...
Mar 25 17:20:48.07: [16944]: <-- [15:ATDT...\r]
Mar 25 17:21:27.68: [16944]: --> [5:+FCON]
Mar 25 17:21:27.68: [16944]: --> [28:+FCSI:"    +41 ..."]
Mar 25 17:21:27.68: [16944]: REMOTE CSI "+41 ..."
Mar 25 17:21:27.68: [16944]: --> [22:+FDIS: 1,5,0,2,0,2,0,0]
Mar 25 17:21:27.68: [16944]: --> [2:OK]
Mar 25 17:21:27.68: [16944]: REMOTE best rate 14400 bit/s
Mar 25 17:21:27.68: [16944]: REMOTE max A4 page width (215 mm)
Mar 25 17:21:27.68: [16944]: REMOTE max unlimited page length
Mar 25 17:21:27.68: [16944]: REMOTE best vres 7.7 line/mm
Mar 25 17:21:27.68: [16944]: REMOTE format support: MH
Mar 25 17:21:27.68: [16944]: REMOTE supports T.30 Annex C, half duplex
  ECM
Mar 25 17:21:27.68: [16944]: REMOTE best 0 ms/scanline
Mar 25 17:21:27.68: [16944]: USE 14400 bit/s
Mar 25 17:21:27.68: [16944]: USE error correction mode
Mar 25 17:21:27.68: [16944]: SEND file "docq/doc106.tif;01"
Mar 25 17:21:27.68: [16944]: USE A4 page width (215 mm)
Mar 25 17:21:27.68: [16944]: USE unlimited page length
Mar 25 17:21:27.68: [16944]: USE 7.7 line/mm
Mar 25 17:21:27.68: [16944]: USE 1-D MH
Mar 25 17:21:27.68: [16944]: USE 0 ms/scanline
Mar 25 17:21:27.68: [16944]: <-- [24:AT+FDIS=1,5,0,2,0,0,0,0\r]
Mar 25 17:21:27.68: [16944]: --> [2:OK]
Mar 25 17:21:27.68: [16944]: <-- [7:AT+FDT\r]
Mar 25 17:21:39.70: [16944]: --> [22:+FDCS: 1,3,0,2,0,2,0,0]
Mar 25 17:21:39.91: [16944]: --> [7:CONNECT]
Mar 25 17:21:39.91: [16944]: SEND wait for XON
Mar 25 17:21:39.93: [16944]: --> [1:]
Mar 25 17:21:39.93: [16944]: SEND begin page
Mar 25 17:21:39.93: [16944]: <-- data [1026]
Mar 25 17:21:39.93: [16944]: <-- data [1027]
[...]
Mar 25 17:21:39.94: [16944]: <-- data [1024]
Mar 25 17:21:39.94: [16944]: <-- data [952]
Mar 25 17:21:39.94: [16944]: SENT 55224 bytes of data
Mar 25 17:21:39.94: [16944]: <-- data [2]
Mar 25 17:21:39.94: [16944]: SEND end page
Mar 25 17:22:00.97: [16944]: --> [2:OK]
Mar 25 17:22:00.97: [16944]: SEND send EOP (no more pages or documents)
Mar 25 17:22:00.97: [16944]: <-- [9:AT+FET=2\r]
Mar 25 17:22:05.60: [16944]: --> [8:+FPTS: 1]
Mar 25 17:22:05.60: [16944]: --> [8:+FHNG: 0]
Mar 25 17:22:05.60: [16944]: REMOTE HANGUP: Normal and proper end of
  connection (code 0)
Mar 25 17:22:05.60: [16944]: SEND recv MCF (message confirmation)
Mar 25 17:22:05.60: [16944]: SEND FAX (000000101): FROM ... TO
  +41... (docq/doc106.tif;01 sent in 0:38)
Mar 25 17:22:05.60: [16944]: SEND FAX (000000101): FROM ... TO
  +41... (page 1 of 1 sent in 0:38)
Mar 25 17:22:06.61: [16944]: <-- [5:ATH0\r]
Mar 25 17:22:06.61: [16944]: --> [2:OK]
Mar 25 17:22:06.61: [16944]: SESSION END


Logs / 3a. HylaFAX, fast session
--------------------------------
Mar 25 17:55:29.29: [20939]: SESSION BEGIN 000000106 41...
Mar 25 17:55:29.29: [20939]: HylaFAX (tm) Version 4.2.5
Mar 25 17:55:29.29: [20939]: SEND FAX: JOB 111 DEST +41...
  COMMID 000000106 DEVICE '/dev/ttyds01' FROM ...
Mar 25 17:55:29.29: [20939]: <-- [10:AT+FBOR=0\r]
Mar 25 17:55:29.29: [20939]: --> [2:OK]
Mar 25 17:55:29.29: [20939]: <-- [13:AT+FPHCTO=30\r]
Mar 25 17:55:29.29: [20939]: --> [5:ERROR]
Mar 25 17:55:29.29: [20939]: MODEM Command error
Mar 25 17:55:29.29: [20939]: <-- [24:AT+FDCC=1,5,2,2,0,2,0,0\r]
Mar 25 17:55:29.29: [20939]: --> [2:OK]
Mar 25 17:55:29.29: [20939]: DIAL ...
Mar 25 17:55:29.29: [20939]: <-- [15:ATDT...\r]
Mar 25 17:55:47.48: [20939]: --> [5:+FCON]
Mar 25 17:55:47.48: [20939]: --> [28:+FCSI:"    +41 ..."]
Mar 25 17:55:47.48: [20939]: REMOTE CSI "+41 ..."
Mar 25 17:55:47.48: [20939]: --> [22:+FDIS: 1,5,0,2,0,2,0,0]
Mar 25 17:55:47.48: [20939]: --> [2:OK]
Mar 25 17:55:47.48: [20939]: REMOTE best rate 14400 bit/s
Mar 25 17:55:47.48: [20939]: REMOTE max A4 page width (215 mm)
Mar 25 17:55:47.48: [20939]: REMOTE max unlimited page length
Mar 25 17:55:47.48: [20939]: REMOTE best vres 7.7 line/mm
Mar 25 17:55:47.48: [20939]: REMOTE format support: MH
Mar 25 17:55:47.48: [20939]: REMOTE supports T.30 Annex C, half duplex
  ECM
Mar 25 17:55:47.48: [20939]: REMOTE best 0 ms/scanline
Mar 25 17:55:47.48: [20939]: USE 14400 bit/s
Mar 25 17:55:47.48: [20939]: USE error correction mode
Mar 25 17:55:47.48: [20939]: SEND file "docq/doc111.tif;01"
Mar 25 17:55:47.48: [20939]: USE A4 page width (215 mm)
Mar 25 17:55:47.48: [20939]: USE unlimited page length
Mar 25 17:55:47.48: [20939]: USE 7.7 line/mm
Mar 25 17:55:47.48: [20939]: USE 1-D MH
Mar 25 17:55:47.48: [20939]: USE 0 ms/scanline
Mar 25 17:55:47.48: [20939]: <-- [24:AT+FDIS=1,5,0,2,0,0,0,0\r]
Mar 25 17:55:47.48: [20939]: --> [2:OK]
Mar 25 17:55:47.48: [20939]: <-- [7:AT+FDT\r]
Mar 25 17:55:47.98: [20939]: --> [22:+FDCS: 1,5,0,2,0,2,0,0]
Mar 25 17:55:48.20: [20939]: --> [7:CONNECT]
Mar 25 17:55:48.20: [20939]: SEND wait for XON
Mar 25 17:55:48.22: [20939]: --> [1:]
Mar 25 17:55:48.22: [20939]: SEND begin page
Mar 25 17:55:48.22: [20939]: <-- data [1027]
[...]
Mar 25 17:55:48.23: [20939]: <-- data [1024]
Mar 25 17:55:48.23: [20939]: <-- data [949]
Mar 25 17:55:48.23: [20939]: SENT 55221 bytes of data
Mar 25 17:55:48.23: [20939]: <-- data [2]
Mar 25 17:55:48.23: [20939]: SEND end page
Mar 25 17:55:54.51: [20939]: --> [2:OK]
Mar 25 17:55:54.51: [20939]: SEND send EOP (no more pages or documents)
Mar 25 17:55:54.51: [20939]: <-- [9:AT+FET=2\r]
Mar 25 17:55:55.85: [20939]: --> [8:+FPTS: 1]
Mar 25 17:55:55.85: [20939]: --> [8:+FHNG: 0]
Mar 25 17:55:55.85: [20939]: REMOTE HANGUP: Normal and proper end of
  connection (code 0)
Mar 25 17:55:55.85: [20939]: SEND recv MCF (message confirmation)
Mar 25 17:55:55.85: [20939]: SEND FAX (000000106): FROM ... TO +41...
  (page 1 of 1 sent in 0:08)
Mar 25 17:55:55.85: [20939]: SEND FAX (000000106): FROM ... TO +41...
  (docq/doc111.tif;01 sent in 0:08)
Mar 25 17:55:56.85: [20939]: <-- [5:ATH0\r]
Mar 25 17:55:56.85: [20939]: --> [2:OK]
Mar 25 17:55:56.85: [20939]: SESSION END



Lee Howard schrieb:
Konrad Baechler wrote:

We have been testing two different modem config settings:

ModemType: Class1

and

#ModemType: Class1

Whereas the second setup intends to let HylaFax/Diva choose the class themselves.

For the first setting, ModemType: Class1, we get always transmission times of 31 or 32 seconds. This is highly reliable.

For the second setting, #ModemType: Class1, we get transmission times of 10 seconds or alternativly of 61 seconds.

We did those tests always to the same fax target number and about 20 times.


As Darren said, you really can't get a good picture of exactly what's going on from the tty responses when using a Diva Server. You'll have to look into other sources for more trustworthy information. So as to why there's a variation when using Class 1 and Class 2 I could only speculate due to that reason.

Analyzing the logs has shown that the transfer of the DATA is done very quickly.


This is because the Diva Server buffers the entire page (and then does stuff with that buffer, I assume) before it actually begins transmitting it. So you have to configure HylaFAX to basically sit around and wait forever until the Diva Server responds.

BUT there is one main difference: either the modem hangs up VERY quickly OR the modem waits for another 40 seconds to hang up and close the connection. And this is the difference between those 10 seconds transmission time and the 61 seconds transmission time.


The 61 seconds transmission time is probably an enforced timeout by HylaFAX. I'd need to see the logs to say.

When ECM protocol is in use Phase C (image data transfer) could take, in theory, an indefinitely long period of time (retransmitting frames and blocks until all of it's good). So with Class 2 modems that perform ECM and with the Diva Server in any Class mode after the image data is transferred to the modem HylaFAX just has to sit and wait ... potentially for several minutes. The question is whether or not the modem can be trusted to handle that while HylaFAX waits patiently until it gets an OK or NO CARRIER or whatever response back from the modem. I think that HylaFAX has had a 60-second timeout on that Phase C response since a long time, and I think that it was put there before anyone really considered this possibility. I think that short of a timeout is still appropriate for Class 1, but in Class 2 it may need to be made much longer.

Has anybody an idea how to solve this? And we are VERY happy for any additional performance hint :) .


I think that we'd need to see more information (like a log or something) before saying exactly how to improve things.

Question 2 (sendfax -S):
man pages say:
<<
-S tsi
Pass tsi to the server as the suggested sender identification to be used, for example, in tagline imaging and fax protocol.
>>

But the -S option is not available anymore. Is there any replacement for it?


It's still there:

       *case* *'S'*:            /// set TSI
/            proto.setTSI(optarg);


*break*;

http://cvs.sourceforge.net/viewcvs.py/hylafax/hylafax/sendfax/sendfax.c%2B%2B?view=markup


Lee.



____________________ HylaFAX(tm) Users Mailing List _______________________
 To subscribe/unsubscribe, click http://lists.hylafax.org/cgi-bin/lsg2.cgi
On UNIX: mail -s unsubscribe hylafax-users-request@xxxxxxxxxxx < /dev/null
 *To learn about commercial HylaFAX(tm) support, mail sales@xxxxxxxxx*

Project hosted by iFAX Solutions