Re: [hylafax-users] More on Fedora 12 and ghostscript

To: Raman Gupta <rocketraman@xxxxxxxxxxx>
Subject: Re: [hylafax-users] More on Fedora 12 and ghostscript
From: Lee Howard <faxguy@xxxxxxxxxxxxxxxx>
Cc: hylafax-users@xxxxxxxxxxx
Date: Mon, 26 Apr 2010 19:32:13 -0700

Raman,

Thank you for this very useful and well-researched post.

Indeed, the problem has to do with Ghostscript 8.71 now preparing multi-strip TIFFs with MMR encoding (whereas before MMR-encoded images were always single-strip). This is rather silly behavior on the part of Ghostscript, and it seems to go against RFC-1314, but it seems to be legitimate to libtiff and a myriad of other TIFF readers, and so we can cope with it.

I've attached the patch to fix this for HylaFAX+ 5.4.1. I've committed it to the HylaFAX+ CVS, and it will be in the next HylaFAX+ release. This patch will unfortunately not apply cleanly with current hylafax.org releases.

While I do recommend that, to resolve the matter, you either use current HylaFAX+ CVS or patch HylaFAX+ with the attached patch, it is likely possible to hack your current HylaFAX scripting to compensate. To do that edit your /var/spool/hylafax/etc/common-functions script, and at the very end of the gs2fax() function put these two lines of code:

$TIFFBIN/tiffcp -r -1 $out $out.new
$MV -f $out.new $out

Please note that's a minus-one and not the letter "l" after the dash-r.

(The reason that my patch isn't so simple is because it gives us the capability to easily cope with other TIFF-F production problems that Ghostscript may have in the future.)

Thanks,

Lee.

Raman Gupta wrote:

Earlier this month Eliran posted about ghostscript-8.71 on Fedora 12 breaking a working Hylafax setup:

http://www.hylafax.org/archive/2010-03/msg00001.php

I too have experienced the same problem with gs 8.71 on Fedora 12 i.e. the receiver of a fax sent from hylafax receives a page containing only the tagline.

A workaround is to add:

Use2D: no

to the etc/config file, as suggested by the Hylafax troubleshooting document for *earlier* versions of Ghostscript. YMMV.

A couple of debugging notes. Without the Use2D parameter, the conversion executed by hylafax is something like:

/bin/bash bin/pdf2fax -o docq/doc50.pdf;c1 -r 196 -w 1728 -l 297 -m 4096 -U -3 docq/doc50.pdf.47

which results in gs using the tiffg4 device (I added debug statements to the pdf2fax script, giving me the gs command line used).

With the Use2D setting, the command line is changed to pass a -1 rather than a -3 parameter, and gs is configured to use the tiffg3 device.

I manually ran the pdf2fax program with gs 8.71 using both the -1 (tiffg3) and -3 (tiffg4) settings. While the resulting tiffg4 file is smaller, I note that *both* the tiffg3 and tiffg4 files seem to load perfectly fine when viewed in several different programs including Okular, Gimp, Windows Picture and Fax viewer, and others. Here is the tiffinfo output:
foo_g3.tiff:
TIFF Directory at offset 0x870e (34574)
  Subfile Type: multi-page document (2 = 0x2)
  Image Width: 1728 Image Length: 2292
  Resolution: 209.1, 196 pixels/inch
  Bits/Sample: 1
  Compression Scheme: CCITT Group 3
  Photometric Interpretation: min-is-white
  FillOrder: msb-to-lsb
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 1
  Rows/Strip: 37
  Planar Configuration: single image plane
  Page Number: 0-0
  Software: GPL Ghostscript 8.71
  DateTime: 2010:04:13 10:55:28
  Group 3 Options: EOL padding (4 = 0x4)
foo_g4.tiff:
TIFF Directory at offset 0x5a3e (23102)
  Subfile Type: multi-page document (2 = 0x2)
  Image Width: 1728 Image Length: 2292
  Resolution: 209.1, 196 pixels/inch
  Bits/Sample: 1
  Compression Scheme: CCITT Group 4
  Photometric Interpretation: min-is-white
  FillOrder: msb-to-lsb
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 1
  Rows/Strip: 37
  Planar Configuration: single image plane
  Page Number: 0-0
  Software: GPL Ghostscript 8.71
  DateTime: 2010:04:13 10:55:21
  Group 4 Options: (0 = 0x0)
However, despite the tiffg4 file appearing to be correct, the destination still receives a blank fax.

I'm not sure what to check next in order to debug this further, as ghostscript 8.71 appears to be the culprit, and yet the output of pdf2fax using ghostscript 8.71 and the tiffg4 device is fine AFAICT. Is there a way to have hylafax send the manually created tiffg4 file directly for testing?
Cheers,
Raman
____________________ HylaFAX(tm) Users Mailing List _______________________ To subscribe/unsubscribe, click http://lists.hylafax.org/cgi-bin/lsg2.cgi On UNIX: mail -s unsubscribe hylafax-users-request@xxxxxxxxxxx < /dev/null *To learn about expensive HylaFAX(tm) support, mail sales@xxxxxxxxx*

diff -Nru hylafax.orig/util/common-functions.sh.in hylafax/util/common-functions.sh.in
--- hylafax.orig/util/common-functions.sh.in	2010-04-26 18:59:59.324152760 -0700
+++ hylafax/util/common-functions.sh.in	2010-04-26 18:13:07.961544840 -0700
@@ -344,6 +344,138 @@
     if [ "$DITHERING" = "libtiff-fs" ]; then
 	$RM -f $out.1 $out.2 $out.3
     fi
+    if [ "$df" = "g4" ]; then
+	#
+	# Ghostscript 8.71 will create multi-strip MMR pages.
+	# To compensate we must run the tiffCheck process.
+	#
+	CHECK=$SBIN/tiffcheck       # program to check acceptability
+	PS2FAX=                     # null to prevent recursion
+	TIFFCP=$TIFFBIN/tiffcp      # part of the TIFF distribution
+	TIFF2PS=$TIFFBIN/tiff2ps    # ditto
+	TIFFINFO=$TIFFBIN/tiffinfo  # ditto
+    
+	fil=$out
+
+	tiffCheck
+    fi
+}
+
+tiffCheck()
+{
+    CLEARTMP=no
+    if [ "$fil" = "$out" ]; then
+	CLEARTMP=yes
+	fil="$fil.$$.tmp"
+	$MV $out $fil
+    fi
+    #
+    # tiffcheck looks over a TIFF document and prints out a string
+    # that describes what's needed (if anything) to make the file
+    # suitable for transmission with the specified parameters (page
+    # width, page length, resolution, encoding).  This string may
+    # be followed by explanatory messages that can be returned to
+    # the user.  The possible actions are:
+    #
+    # OK		document is ok
+    # REJECT	something is very wrong (e.g. not valid TIFF)
+    # REFORMAT	data must be re-encoded
+    # REVRES	reformat to change vertical resolution
+    # RESIZE	scale or truncate the pages
+    # REIMAGE	image is not 1-channel bilevel data
+    #
+    # Note that these actions may be combined with "+";
+    # e.g. REFORMAT+RESIZE.  If we cannnot do the necessary work
+    # to prepare the document then we reject it here.
+    #
+    RESULT=`$CHECK $opt $fil 2>/dev/null`
+
+    ACTIONS=`echo "$RESULT" | $SED 1q`
+    case "$ACTIONS" in
+    OK)				# no conversion needed
+	#
+	# 1) We don't use hard links because it screws up faxqclean
+	#    logic that assumes the only hard links are used 
+	#    temporarily when document files are being created during
+	#    the job submission process.
+	# 2) We don't use symbolic links because the links get broken
+	#    when the source document is shared between jobs and
+	#    faxq removes the source document before all jobs complete.
+	#
+	# If we ever encounter problems where the client submits corrupt
+	# TIFF and we need to clean it up before transmission, then we
+	# can simply merge OK with REFORMAT.  For now we use $CP instead
+	# of $TIFFCP, however, to provide the client some control.
+	#
+	$CP -f $fil $out
+	if [ "$CLEARTMP" = "yes" ]; then $RM -f $fil; fi
+	exit 0			# successful conversion
+	;;
+    *REJECT*)			# document rejected out of hand
+	echo "$RESULT" | $SED 1d
+	if [ "$CLEARTMP" = "yes" ]; then $RM -f $fil; fi
+	exit 254			# reject document
+	;;
+    REFORMAT)			# only need format conversion (e.g. g4->g3)
+	rowsperstrip="-r 9999 "
+	if [ -n "`$TIFFINFO $fil | $GREP 'Compression Scheme: ISO JBIG'`" ]; then
+	    rowsperstrip=""
+	fi
+	$TIFFCP -i -c $df -f lsb2msb $rowsperstrip$fil $out
+
+	# libtiff 3.5.7 gives exit status 9 when there are unknown tags...
+	exitcode=$?
+	if [ "$CLEARTMP" = "yes" ]; then $RM -f $fil; fi
+	if [ $exitcode != 0 ] && [ $exitcode != 9 ]; then {
+	    $CAT<<EOF
+Unexpected failure converting TIFF document; the command
+
+    $TIFFCP -i -c $df -f lsb2msb $rowsperstrip$fil $out
+
+failed with exit status $?.  This conversion was done because:
+
+EOF
+	    echo "$RESULT" | $SED 1d; exit 254
+	}
+	fi
+	exit 0
+	;;
+    #
+    # REVRES|REFORMAT+REVRES	adjust vertical resolution (should optimize)
+    # *RESIZE			page size must be adjusted (should optimize)
+    # *REIMAGE			maybe should reject (XXX)
+    #
+    *REVRES|*RESIZE|*REIMAGE)
+	if [ -z "$PS2FAX" ]; then
+	    echo "Unable to format with converters."
+	    echo "Preventing ps2fax recursion."
+	    if [ "$CLEARTMP" = "yes" ]; then $RM -f $fil; fi
+	    exit 254
+	fi
+	($TIFF2PS -a $fil | $PS2FAX -o $out -i "$jobid" $opt) || {
+	    $CAT<<EOF
+Unexpected failure converting TIFF document; the command
+
+    $TIFF2PS -a $fil | $PS2FAX $opt
+
+failed with exit status $?.  This conversion was done because
+
+EOF
+	    echo "$RESULT" | $SED 1d; exit 254
+	}
+	if [ "$CLEARTMP" = "yes" ]; then $RM -f $fil; fi
+	exit 0
+	;;
+    *)				# something went wrong
+	echo "Unexpected failure in the TIFF format checker;"
+	echo "the output of $CHECK was:"
+	echo ""
+	echo "$RESULT"
+	echo ""
+	if [ "$CLEARTMP" = "yes" ]; then $RM -f $fil; fi
+	exit 254			# no formatter
+	;;
+    esac
 }
 
 SetupPrivateTmp()
diff -Nru hylafax.orig/util/pdf2fax.gs.sh.in hylafax/util/pdf2fax.gs.sh.in
--- hylafax.orig/util/pdf2fax.gs.sh.in	2010-04-26 18:59:59.162177384 -0700
+++ hylafax/util/pdf2fax.gs.sh.in	2010-04-26 18:11:43.787341264 -0700
@@ -70,25 +70,28 @@
 device=			# to know if it was never specified
 unlimitedlength=no	# default to fixed length-pages
 color=no		# default to monochrome-only
+opt=
+df=
 
 while test $# != 0
 do
     case "$1" in
     -i)	shift; jobid="$1" ;;
     -o)	shift; out="$1" ;;
-    -w) shift; pagewidth="$1" ;;
-    -l) shift; pagelength="$1" ;;
-    -r) shift; vres="$1" ;;
+    -w) shift; pagewidth="$1"; opt="$opt -w $1" ;;
+    -l) shift; pagelength="$1"; opt="$opt -l $1" ;;
+    -r) shift; vres="$1"; opt="$opt -r $1" ;;
     -m) shift;;				# NB: not implemented
-    -U) unlimitedlength=yes ;;
-    -1) device=tiffg3 ;;
+    -U) unlimitedlength=yes; opt="$opt $1" ;;
+    -1) device=tiffg3; opt="$opt $1"; df="g3:1d" ;;
     -2) ($PS -h | grep tiffg32d >/dev/null 2>&1) \
 	    && { device=tiffg32d; } \
 	    || { device=tiffg3; }
-	;;
+	opt="$opt $1"; df="g3:2d" ;;
     -3) ($PS -h | grep tiffg4 >/dev/null 2>&1) \
 	    && { device=tiffg4; } \
 	    || { device=tiffg3; }
+	opt="$opt $1"; df=g4
 	;;
     -color) ($PS -h | grep tiff24nc >/dev/null 2>&1) \
 	    && { color=yes; }
diff -Nru hylafax.orig/util/ps2fax.gs.sh.in hylafax/util/ps2fax.gs.sh.in
--- hylafax.orig/util/ps2fax.gs.sh.in	2010-04-26 18:59:58.718244872 -0700
+++ hylafax/util/ps2fax.gs.sh.in	2010-04-26 18:16:30.312782776 -0700
@@ -70,25 +70,29 @@
 device=			# to know if it was never specified
 unlimitedlength=no	# default to fixed length-pages
 color=no		# default to monochrome-only
+opt=
+df=
 
 while test $# != 0
 do
     case "$1" in
     -i)	shift; jobid="$1" ;;
     -o)	shift; out="$1" ;;
-    -w) shift; pagewidth="$1" ;;
-    -l) shift; pagelength="$1" ;;
-    -r) shift; vres="$1" ;;
+    -w) shift; pagewidth="$1"; opt="$opt -w $1" ;;
+    -l) shift; pagelength="$1"; opt="$opt -l $1" ;;
+    -r) shift; vres="$1"; opt="$opt -r $1" ;;
     -m) shift;;				# NB: not implemented
-    -U) unlimitedlength=yes ;;
-    -1) device=tiffg3 ;;
+    -U) unlimitedlength=yes; opt="$opt $1" ;;
+    -1) device=tiffg3; opt="$opt $1"; df="g3:1d" ;;
     -2) ($PS -h | grep tiffg32d >/dev/null 2>&1) \
 	    && { device=tiffg32d; } \
 	    || { device=tiffg3; }
+	opt="$opt $1"; df="g3:2d"
 	;;
     -3) ($PS -h | grep tiffg4 >/dev/null 2>&1) \
 	    && { device=tiffg4; } \
 	    || { device=tiffg3; }
+	opt="$opt $1"; df=g4
 	;;
     -color) ($PS -h | grep tiff24nc >/dev/null 2>&1) \
 	    && { color=yes; }
diff -Nru hylafax.orig/util/tiff2fax.sh.in hylafax/util/tiff2fax.sh.in
--- hylafax.orig/util/tiff2fax.sh.in	2010-04-26 18:59:59.350148808 -0700
+++ hylafax/util/tiff2fax.sh.in	2010-04-26 17:23:49.837248088 -0700
@@ -55,6 +55,7 @@
     exit 1
 }
 . etc/setup.cache
+. bin/common-functions
 
 CHECK=$SBIN/tiffcheck			# program to check acceptability
 PS2FAX=bin/ps2fax			# for hard conversions
@@ -121,99 +122,4 @@
     $RM -f "$out.color"
 fi
 
-#
-# tiffcheck looks over a TIFF document and prints out a string
-# that describes what's needed (if anything) to make the file
-# suitable for transmission with the specified parameters (page
-# width, page length, resolution, encoding).  This string may
-# be followed by explanatory messages that can be returned to
-# the user.  The possible actions are:
-#
-# OK		document is ok
-# REJECT	something is very wrong (e.g. not valid TIFF)
-# REFORMAT	data must be re-encoded
-# REVRES	reformat to change vertical resolution
-# RESIZE	scale or truncate the pages
-# REIMAGE	image is not 1-channel bilevel data
-#
-# Note that these actions may be combined with "+";
-# e.g. REFORMAT+RESIZE.  If we cannnot do the necessary work
-# to prepare the document then we reject it here.
-#
-RESULT=`$CHECK $opt $fil 2>/dev/null`
-
-ACTIONS=`echo "$RESULT" | $SED 1q`
-case "$ACTIONS" in
-OK)				# no conversion needed
-    #
-    # 1) We don't use hard links because it screws up faxqclean
-    #    logic that assumes the only hard links are used 
-    #    temporarily when document files are being created during
-    #    the job submission process.
-    # 2) We don't use symbolic links because the links get broken
-    #    when the source document is shared between jobs and
-    #    faxq removes the source document before all jobs complete.
-    #
-    # If we ever encounter problems where the client submits corrupt
-    # TIFF and we need to clean it up before transmission, then we
-    # can simply merge OK with REFORMAT.  For now we use $CP instead
-    # of $TIFFCP, however, to provide the client some control.
-    #
-    $CP -f $fil $out
-    exit 0			# successful conversion
-    ;;
-*REJECT*)			# document rejected out of hand
-    echo "$RESULT" | $SED 1d
-    exit 254			# reject document
-    ;;
-REFORMAT)			# only need format conversion (e.g. g4->g3)
-    rowsperstrip="-r 9999 "
-    if [ -n "`$TIFFINFO $fil | $GREP 'Compression Scheme: ISO JBIG'`" ]; then
-	rowsperstrip=""
-    fi
-    $TIFFCP -i -c $df -f lsb2msb $rowsperstrip$fil $out
-
-    # libtiff 3.5.7 gives exit status 9 when there are unknown tags...
-    exitcode=$?
-    if [ $exitcode != 0 ] && [ $exitcode != 9 ]; then {
-	$CAT<<EOF
-Unexpected failure converting TIFF document; the command
-
-    $TIFFCP -i -c $df -f lsb2msb $rowsperstrip$fil $out
-
-failed with exit status $?.  This conversion was done because:
-
-EOF
-	echo "$RESULT" | $SED 1d; exit 254
-    }
-    fi
-    exit 0
-    ;;
-#
-# REVRES|REFORMAT+REVRES	adjust vertical resolution (should optimize)
-# *RESIZE			page size must be adjusted (should optimize)
-# *REIMAGE			maybe should reject (XXX)
-#
-*REVRES|*RESIZE|*REIMAGE)
-    ($TIFF2PS -a $fil | $PS2FAX -o $out -i "$jobid" $opt) || {
-	$CAT<<EOF
-Unexpected failure converting TIFF document; the command
-
-    $TIFF2PS -a $fil | $PS2FAX $opt
-
-failed with exit status $?.  This conversion was done because
-
-EOF
-	echo "$RESULT" | $SED 1d; exit 254
-    }
-    exit 0
-    ;;
-*)				# something went wrong
-    echo "Unexpected failure in the TIFF format checker;"
-    echo "the output of $CHECK was:"
-    echo ""
-    echo "$RESULT"
-    echo ""
-    exit 254			# no formatter
-    ;;
-esac
+tiffCheck

References:
- [hylafax-users] More on Fedora 12 and ghostscript
  - From: Raman Gupta

Prev by Date: [hylafax-users] * FOR SALE ! patton PRI/E1 server card ... reduced price ...
Next by Date: [hylafax-users] Sending blank faxes
Previous by thread: [hylafax-users] More on Fedora 12 and ghostscript
Next by thread: [hylafax-users] Making Hylafax dial X times / X attempts
Index(es):
- Main
- Thread

Project hosted by iFAX Solutions