Re: (Fwd) Fw: Problems with outgoing faxes from a ZyXEL Omni

["Harald Pollack" <Harald.Pollack@omv.co.at>]

On Tue, 31 Mar 1998 01:01:48 +0530, Shuvam Misra wrote:

Hello Shuvam!

>I'd like to discuss some of the points raised. And I'm posting this to
>the Hylafax mailing list hoping to get some of the fax experts there to
>respond to the Hylafax related points you've raised.

Fine!

>I've gone through your mail, and I can see your strong reservations
>about Hylafax.

My 'strong reservations' do not target any specific fax software. I
have the experience, that almost each (comercial or noncomercial)
faxsoftware contains more or less 'misbehaviours'! Mainly this
(misbehaviour) is based on the fact, that producers of such software
want to make money (which is OK) primarly and 'satisfaction' of user is
only a (wellcomed) sideeffect :-)

>I make no claim that Hylafax is the world's best fax
>software. :) However, I believe that faxing is a technically challenging
>function, and any fax software will be less than perfect.

Yes, I agree! But I think that faxsoftware should primarly follow the
standards (and NOT misdesigned faxmodems).

>That said, I must add that I find it hard to believe that Hylafax will
>be less competent than any other good fax software.

I agree also. Hylafax is in 'good society' :-)

>It has two major plusses to its credit:
>
>1.	It is very widely used. It is probably the most popular fax
>	software on Unix.

agree

>2.	It is written by Sam Leffler.

What shall I say ? 

>I presume Sam Leffler needs no introduction to the Unix world. He was a
>member of the core CSRG that created BSD Unix. He is one of the best
>engineers that the software community has today, I feel. His track
>record for good engineering makes it unlikely that there will be any
>slipshod work either in the software that prepares TIFF G3 data out of
>the input file, or in the fax transmission software itself. Sam has
>written perhaps the definitive TIFF library available on the Internet,
>and this library handles the TIFF file creation for Hylafax.

I know this 'TIFF library' very well :-) ... I DO NOT use it!

>In this context, it is unlikely that Sam Leffler would not know about
>not retransmitting the same page after a negative RTN, as you've pointed
>out.

Maybe, he has some reasons, why he is doing so (retransmit after RTN),
but this retransmission - in the discussed case - is in fact wrong! I
agree, that fax standard (ITU-T.30) is not so easy to understand and
very difficult to 'interpret'.

As I know from other standards, 'right interpretation' is not a problem
of 'well understoodnes' (this might be subjectiv) it is more the
'knowledge' of what 'creators of standard' have had indent!

let's have a short historical review: T.30 was 'founded' by CCITT,
where all the (former) national telecoms were members.

SO, all I had to do, was ASK persons which were (or are still) members
in CCITT (or now ITU) what exactly they have meaned with RTN.

National telecoms have 'test suits' to aprove a faxmachine (or even a
faxmodem :-) in accordance to T.4 and T.30.

And finaly, one such test is the reaction on CCITT "ERRORCHART 1" and
"ERRORCHART 2".

ERRORCHART 1 has to be received with response MCF (MessageConFirmation)
or RTP (ReTrainPositiv).

ERRORCHART 2 has to be REJECTED with response RTN (ReTrainNegativ).

This will mean, that if 'source of sent faxdata' is not changed, this
MUST NOT be retransmitted any more.

Now, what means 'source of sent faxdata' ? This is (in case of Hylafax
and almost all faxsoftware in world) a CONSTANT set of bits (or bytes)
as it is obtained be PRE-compression of some imagedata.

There is only ONE possible case, where a retransmit after RTN MIGHT
occur: faxsender (in our case Hylafax) REcompresses the imagedata with
some OTHER (alternativ) algorithm. In real world, this could be, if
first attempt (which yields RTN) was a two dimensional
coding/compression scheme than second attempt will be made with an
alternativ coding/compression scheme (one dimensional eg).

But, as you have mentioned, Hylafax does not 'recompress' on the fly,
so it never should retransmit the same data set, which yielded in RTN.
Clear ? :-)

My investigations at some national telecoms resulted in a second
'experience':

A fax equipment should response RTP, if obviously LINE was source of
errors and should response RTN if obviously T.4 coding was source of
errors.

It is not so easy for a receiving fax equipment, to decide if either
line or T.4 coding was possible source of error, but with some
'pragmatism', it is possible (but not with the functions in TIFF
library :-)!

>If Hylafax retransmits the same page, and it conflicts with the fax
>protocol, I guess Sam had some reasons for it.

Yes, he must have VERY GOOD reasons ... :-)

>I'm hoping to hear from Sam or others on the Hylafax mailing list about this.

Normal answer from fax programmers is: "Heh! What do you mean ?" :-)

>My point is not "Sam is the new God I follow" but that the designers of
>Hylafax must have their reasons, which I hope to discover now, thanks
>to your inputs.

I am very interested in ...

>The last point I'd like to add is that we Unix users running Linux and
>operating on shoestring budgets, don't have a choice. Hylafax is the
>most comprehensive fax software that's freely redistributable in source

I have downloaded source yesterday and I will 'investigate' it ....

>for the Unix platform, and mgetty+sendfax is the only other well-known
>alternative.

Yes, I know mgetty+sendfax very well, but all I could say is: well, it
works ...

But keep in mind, sendfax is nothing more than a simple 'copier' of
prebuild fax rawdata from a harddisk file via seriell port to modem.

In my opinion, this is not, what I understand as 'fax programm' ....

>I haven't tried mgettty+sendfax, but I could, I guess.
>Maybe this too could be something others on the Hylafax mailing list
>could advise me on. Do you think there are problems in the fax protocol
>implementation of Hylafax, and is mgetty+sendfax better?

I miss any 'protocol related' things in sendfax... 

>| Only 'NO CARRIER' is the response of interrest !
>
>This is not true. There seems to be a big advantage to be gained by
>being able to differentiate between NO ANSWER and NO CARRIER.
>If ZyXEL could have returned the correct response to distinguish between them,
>the fax software could have figured out whether there's a fax machine at
>the other end (NO CARRIER after handshake attempt) or just an ordinary
>phone (NO ANSWER).
>I got this info from the Hylafax documentation. Check
>http://www.vix.com/hylafax/

i will do so ...

>| >I am getting an enormous number of reports of failed faxes, all due to
>| >a few repeated reasons....
>| 
>| Not so good. As a rule of thumb: appr. 40 % of outgoing faxes from a
>| (public faxserver) will fail...
>
>This figure was very useful. I didn't know that the typical observed
>figures for failed attempts is so high even in your parts of the world
>in spite of your better phone networks.

The reason, why calls do fail is much more protocol problems (fax
equipment just a little step beside the standard) than line. 

Exactly said, I have NEVER watched a broken fax call due to line faults
(will yield FTT = FailureToTrain at 2400 bps)

>| >   1 Error: Document was encoded with 2DMR, but client does not support thi
s
>| >data format
>| 
>| Faxsoftware MUST DO a 'fallback' to 1-D Huffman !!!!
>
>Hylafax does. It disconnects the line, reconverts the data, and retries
>the call.

This is NOT a fallback! This is a simple RETRY with altered session
parameters! Is Hylafax also a 'copier' ???

Let me explain:

A caller (which normally has something to SEND) dials a number, emits
Fax CNG and waits for an answering remot fax equipment.

Remote (called) fax goes off-hook, hears the CNG and sends
'Answering-tone' and some HDLC frames (CSI+DIS) to inform caller about
its capabilities.

Local (caller) fax 'knows' its own capabilities and due to some
algorithm it set the appropriate send mode (resolution, compression
scheme, wait between scanlines and much more).

If local fax was 'influenced' by operator to use primarly two
demensional coding, but called (remote) fax is not able to handle this,
local fax must 'fallback' from two dimensional to one dimensional
coding and signal this by HDLC frames (TSI+DCS+TCF).

Oh! You say, Hylafax cannot do such a 'fallback' (between receiving DIS
and transmitting DCS)? Sorry, than it is far away from being a
'FAXPROGRAM' :-)

>| >   4 Error: Failure to train at 2400 bps or +FMINSP value
>| A sign for bad telco line ...

Oh! I have forgotten to 'comment' this '+FMINSP' ....

This command (from Rockwells propriatory fax convention) is an absolut
nonesense, because in T.30 there exists no way to signal such a
'minimal phase c transmission speed' (you can only signal a MAXIMAL
speed) !

>Very possible that my line is noisy. This is usual in India, and specially in 
my exchange.

Not the NOISE ! Nonlineare distortions or phase shifts or spikes are
source of such fault.

>| >  29 Error: No response to MPS repeated 3 times
>| 
>| This is 'system immanent' to all faxmodems, I believe :-) We have to
>| live whit this, regardles which brand of faxmodem, there ALWAYS exist
>| (remote) certain fax equipment, which will not fit our site ...
>
>Can you give me some more details on this? What causes this?

This is a typical procedural error:

Sender finishes data transmission in phase C and sends the MPS frame to
receiver. Receiver gets the MPS and (let's assume) responses with MCF.
Sender will not (errorfree) received this response and retransmits its
MPS. Now we must tell between two kinds of receiver (one follows the
standard, the other not :-)

1) standard: receiver gets (second) MPS and responses again with MCF.
Sender will not (errorfree) receive this response again.

2) nonstandard: receiver had already sent its MCF and leaves the phase
B and enters phase C and waits for fax data ... will never come :-)

Sender retransmits (third attempt) MPS and waits for a valid response.
Nonstandard equipment will react in two ways:

1) receiver recognizes, that no phase C carrier is there, quits by
sending DCN to caller (a lot of faxmodems/programs will do so)

2) receiver waits for phase C carrier and waits for phase C carrier and
waits for phase C carrier ....

Sender receives NO response after third MPS and quits.

So, what is the reason, why the sender will not (errorfree) receive the
MCF in above example?

Normally (in case fax MODEM is sender) this is a (internal) problem in
switching from HDLC sending (the MPS frame) to receiving HDLC frames.
Called station switches FASTER from receive (the MPS) to transmitt (the
MCF) than sender. So the initial HDLC flag(s) are never recognized by
receiver and therefore the frame is discarded.

There is also a well known situation (of this fault) if receiver is fax
MODEM. Let's also assume, receiving faxmodem is in Class 2, than AFTER
receiving faxdata (in phase C) faxmodem responses <DLE><ETX> to DTE as
a sign of page data is finished. A lot of fax programs now (in this
moment) start to save the data from memory to harddisk. This might
consume a lot of time (we do not have :-).

In the meantime, faxmodem had also responded +FPTS:.... and +FET:0 (the
MPS) and OK which will be either in seriel port device drivers input
buffer or even LOST due to blocked interrupts, when faxprogram stores
data to harddisk!

When disk saving is finished, faxprogram starts again to collect the
responses from the buffer, sees (or even not) the responses and do its
AT+FDR to release frames (in fact the MCF). And in meantime, sender has
repeated MPS three times ....

The last example was a worst case assumption, but this behaviour is
significant for all singlethreaded faxprograms when receive data...

>
>| >113 Error: Unable to transmit page (giving up after 3 attempts)
>| 
>| Not so clear this report. What in detail was the reason ? Could be
>| simply BUSY at remote site ...
>
>These details are available in the big log file that I've left on the
>Website, at http://www.spacenetindia.com/faxerrors.txt. It's a huge
>file, about 1.5MB, so if you have the patience, you'll get the full
>modem dialog for a few hundred failed fax transmission attempts.

Reason might be similar to the above MPS fault, but in this case, the
TSI+DCS+TCF is not (or not errorfree) finished by receiving the CFR
response from receiver.

>| >274 Error: Unspecified Transmit Phase B error
>| 
>| No 'normal' fault :-) Yield of well sent faxes can only be 'increased'
>| by using Class 1 protocol ...

Can be anything :-)

>This is interesting. Is it easier to get good fax transmission using
>Class 1 protocol?

Yes, of course, because programmer of faxprogram is (hope)fully
independent of implementation of  Class 2 protocol in modem.

Programmer can fully implement all situations of T.30 protocol and
react correct. And ECM (ErrorCorrectionMode) is easy to implement in
class1 which normally yields errorfree copies!

>I thought Class 1 was too timing-sensitive and

All timing is dictated by ITU-T.30, so it is quite equal if either
class 2 or class 1 must fullfill the timing!

>therefore more error-prone when driven from multitasking operating
>systems?

Class 1 is only sensefull on multitasking (and multithreading)
operating systems!

>
>| >Mar 25 22:19:41.95: [ 3664]: <-- [24:AT+FLI="SpaceNET India"\r]
>| 
>| Not in accordance to ITU-T.30 !!! Only '0' - '9', '+' and ' ' (blank)
>| !!! Faxsoftware should know this ...
>
>Yes, I know. But the fax software allows this if the modem allows it.

Understand my right: ITU-T.30 DOES NOT ALLOW IT! Therefore it is
absolut meaningless, what modem or software is able to do...

>| >Mar 25 22:20:08.82: [ 3664]: REMOTE best format 2-D MR
>| 
>| I assume, that THIS is the reason for a lot of problems! faxdata makeup
>| for 2-D mod. mod. Read is obviously 'out of spec' from ITU-T.4 and
>| normaly fax machines reject such transmissions (with bad T.4 data) by
>| sending RTN ...
>
>How do you know that the data being generated at the transmitting end is
>out of spec with the ITU standards?

because receiver responses with RTN :-)

>Is there any way I can find out?

Take the original compressed data(file) and decompress it manually (by
using tables in T.4 :-)

>| 09:50:13.69 {01 200} Receive:  2,96E,5,1,0<CR><LF> 0.22s
>| 09:50:13.69 {01 200} Receive:  OK<CR><LF> 0.25s
>| 
>| The above (first) value of '2' means RTN ! Faxmodem can NOT decide,
>| what to do, so fax software starts to send NEXT page (faxsoftware
>| should NEVER RETRANSMIT a page).
>
>This is interesting. When this error occurs, you feel that the fax
>software should proceed with the next page as if there was no error at
>all?

This (proceed with next page) is one possibilty, but I FEEL, that
sender should give up instead ...

>Will some expert users of Hylafax please comment?

How do you define an 'expert user ' ?

>| >|   SM> | I've been trying to send out faxes to Mumbai using "tpc.int" but
>| >|   SM> never got | through to it! However, the recepients complain that th
e
>| >|   SM> receive about 8-12 | copies of the same fax sent to them.
>| 
>| Fax software MUST NOT retransmit any page after RTN ...
>
>This is what I found particularly interesting. If the standard is very
>clear on this, will someone who knows Hylafax comment on these protocol
>issues?

As I mentioned above, the 'equality' of RTN and 'OVER AND OUT' is from
some national telecom specialists and my personal experience.

>Thanks once again. You have been very very patient and thorough and
>you've taken a lot of trouble with my query.

No problem. For Gods sake, fax is not my profession, so I can do all
neccessary studies (which consume a lot of my time :-) and I get from
almost all manufacturers and national telecoms all relevant
informations which commercial requesters will never get.

>I'll see if Hylafax is really the source of the problem as you state it is.

It is not the only source, of course, but if faxsoftware is 100% in
accordance to the standards, almost all faxmodems will work fine ...

There should be NO COMPROMISE in making faxsoftware.

Programmer should collect all neccessary standards and implement them
in a faxsoftware. Than programmer should take all faxmodems and test
their functionalyty. If one brands equipment will fail, well he should
report this (maybe with a copy to the editors of opinion building
computer magacines :-) to the manufacturer.

Irreal ? maybe, but faxmachines are examined in this way ....



Herzliche Gruesse

Dr. Harald Pollack



------- End of Forwarded Message