Roll up, Roll up. Test your backend

Nick Lamb (njl98r@ecs.soton.ac.uk)
Thu, 31 Dec 1998 15:32:43 +0000 (GMT)

This message is in MIME format. The first part should be readable text,
while the remaining parts are likely unreadable without MIME-aware tools.
Send mail to mime@docserver.cac.washington.edu for more info.

--1318610500-70641457-915118363=:29443
Content-Type: TEXT/PLAIN; charset=US-ASCII

A while back I reported that SANE 1.0 has a broken PNM device driver.
While I was waiting for it to be fixed (someone?) I offered to write
a test frontend that would spot this sort of stupidity. Here it is.

I have added functionality to SANE's scanimage so that it can hopefully
detect broken SANE backends. A patch attached to this mail. I can also
make it available for HTTP if anyone asks.

The file scanimage-test-patch is a patch to be applied directly to the
file frontend/scanimage.c.

I would like to see this patch, or something similar, applied before
SANE 1.0.1 is made available, but it should perhaps be enclosed in
#ifdefs so that by default the test features aren't compiled in. This
is especially the case if some hardware+backend combos are still
broken when shipped with 1.0.1 -- I hope that doesn't happen.

What does it do?
----------------

This frontend detects problems with SANE backends which do not fully
meet the SANE specification. In this first version I have concentrated
on a problem which several backends "In the wild" are known to have.
In particular this version checks that the backend correctly handles
sane_read(...) even for unusual and varying buffer length values. It
detects backends which overflow the buffer or incorrectly report the
amount of data read and it also detects backends which sometimes don't
write any data even after a blocking read.

Why should I use it?
--------------------

If you're a SANE end-user you probably don't want to try this. If you
do try it, and it detects a problem with a backend you use, please
report it to the SANE developers mailing list, where hopefully someone
will notice and take appropriate action.

If you're a SANE frontend developer, you probably don't want to try
this. The exception is if your frontend has problems in sane_read(...)
on some backends, but not others. In that case this patch may help
you to show that the problem is in the backend, and you can breathe
a sigh of relief.

If you're a SANE backend developer, and you have working code, you
really should try this frontend. Any properly working backend should
pass with no trouble, but those which don't are a problem. In the
future hopefully a program somewhat like this one will form part of
a self-certification test for new SANE backends...

If you're a SANE backend developer, but you don't yet have working code
you should be reminded that sane_read(...) is used very politely by
the two commonest frontends, scanimage and xscanimage. Your code should
be written to work with /any/ frontend, and that means this one too.

If you're a core developer, please take a look at this and see if you
think the tests done here should be _required_ before a backend is
made part of the core SANE distribution in future. Also please try it
with any backends for which you have appropriate hardware, as a double
check in case the maintainer doesn't see this.

How do I compile it?
--------------------

After applying the patch, simply recompile and (optionally) re-install
the scanimage frontend. The new code should work on all the platforms
for which scanimage itself was tested. It works for me (TM)

How do I use it?
----------------

Simply add the parameter --test or -T to an ordinary scanimage run, and
the test code is invoked in place of the ordinary scanimage sane_read
code. Unlike a normal scan, there is no data written to stdout for the
test scan.

e.g. scanimage -d pnm:0 -T --filename /path/to/some.ppm

After some initial diagnostic messages, there are a series of lines
each containing a description of the test being performed, then either
PASS or FAIL and a brief explanation of the problem.

It is NOT acceptable for a backend to FAIL any of the tests. If it is
not possible to handle the extreme values tried by --test in your
backend (e.g. due to hardware limitations) then you should return a
suitable SANE error status and note the limitation under BUGS in the
manual page for your backend.

Feedback
--------

I have only tested this on an Intel PC with no physical scanners. For
testing I used only the PNM:0 and PNM:1 devices. Therefore I cannot
say for sure that this works with _any_ known scanners in the real world
I hope some of you will help rectify that by sending me bug reports.

I would also appreciate info about other ways in which some backends are
known not to meet the SANE standard. This code should evolve to test
for compliance as thoroughly as possible.

Nick
njl195@zepler.org.uk

--1318610500-70641457-915118363=:29443
Content-Type: TEXT/PLAIN; charset=US-ASCII; name=scanimage-test-patch
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.3.96.981231153243.29443B@chef.ecs.soton.ac.uk>
Content-Description: Scanimage patch

LS0tIHNjYW5pbWFnZS5jLm9sZAlUaHUgRGVjIDMxIDEzOjIzOjU1IDE5OTgN
CisrKyBzY2FuaW1hZ2UuYwlUaHUgRGVjIDMxIDEzOjM4OjA0IDE5OTgNCkBA
IC01OSwxOCArNTksMTkgQEANCiAgIHsibGlzdC1kZXZpY2VzIiwgbm9fYXJn
dW1lbnQsIE5VTEwsICdMJ30sDQogICB7ImhlbHAiLCBub19hcmd1bWVudCwg
TlVMTCwgJ2gnfSwNCiAgIHsidmVyYm9zZSIsIG5vX2FyZ3VtZW50LCBOVUxM
LCAndid9LA0KKyAgeyJ0ZXN0Iiwgbm9fYXJndW1lbnQsIE5VTEwsICdUJ30s
DQogICB7InZlcnNpb24iLCBub19hcmd1bWVudCwgTlVMTCwgJ1YnfSwNCiAg
IHswLCB9DQogfTsNCiANCi0jZGVmaW5lIEJBU0VfT1BUU1RSSU5HCSJkOmhM
dlYiDQorI2RlZmluZSBCQVNFX09QVFNUUklORwkiZDpoTHZWVCINCiAjZGVm
aW5lIFNUUklQX0hFSUdIVAkyNTYJLyogIyBsaW5lcyB3ZSBpbmNyZW1lbnQg
aW1hZ2UgaGVpZ2h0ICovDQogDQogc3RhdGljIHN0cnVjdCBvcHRpb24gKiBh
bGxfb3B0aW9uczsNCiBzdGF0aWMgaW50IG9wdGlvbl9udW1iZXJfbGVuOw0K
IHN0YXRpYyBpbnQgKiBvcHRpb25fbnVtYmVyOw0KIHN0YXRpYyBTQU5FX0hh
bmRsZSBkZXZpY2U7DQotc3RhdGljIGludCB2ZXJib3NlOw0KK3N0YXRpYyBp
bnQgdmVyYm9zZSwgdGVzdDsNCiBzdGF0aWMgaW50IGhlbHA7DQogc3RhdGlj
IGNvbnN0IGNoYXIgKiBwcm9nX25hbWU7DQogc3RhdGljIFNBTkVfT3B0aW9u
X0Rlc2NyaXB0b3Igd2luZG93X29wdGlvblsyXTsNCkBAIC05MzgsNiArOTM5
LDEwNiBAQA0KICAgICBmcmVlIChpbWFnZS5kYXRhKTsNCiB9DQogDQorc3Rh
dGljIHZvaWQgY2xlYW5fYnVmZmVyKFNBTkVfQnl0ZSAqYnVmZmVyLCBpbnQg
c2l6ZSkgew0KKyAgaW50IGk7DQorICBmb3IgKGk9IDA7IGkgPCBzaXplOyAr
K2kpIHsNCisgICAgYnVmZmVyW2ldPSAweDIzOw0KKyAgfQ0KK30NCisNCitz
dGF0aWMgdm9pZA0KK3Bhc3NfZmFpbCAoaW50IG1heCwgaW50IGxlbiwgU0FO
RV9CeXRlICpidWZmZXIsIFNBTkVfU3RhdHVzIHN0YXR1cykgew0KKyAgaWYg
KHN0YXR1cyAhPSBTQU5FX1NUQVRVU19HT09EKSB7DQorICAgIGZwcmludGYo
c3RkZXJyLCAiRkFJTCBFcnJvcjogJXNcbiIsIHNhbmVfc3Ryc3RhdHVzKHN0
YXR1cykpOw0KKyAgfSBlbHNlIGlmIChidWZmZXJbbGVuXSAhPSAweDIzKSB7
DQorICAgIHdoaWxlKGJ1ZmZlcltsZW5dICE9IDB4MjMpIGxlbisrOw0KKyAg
ICBmcHJpbnRmKHN0ZGVyciwgIkZBSUwgQ2hlYXQ6ICVkIGJ5dGVzXG4iLCBs
ZW4pOw0KKyAgfSBlbHNlIGlmIChsZW4gPiBtYXgpIHsNCisgICAgZnByaW50
ZihzdGRlcnIsICJGQUlMIE92ZXJmbG93OiAlZCBieXRlc1xuIiwgbGVuKTsN
CisgIH0gZWxzZSBpZiAobGVuID09IDApIHsNCisgICAgZnByaW50ZihzdGRl
cnIsICJGQUlMIE5vIGRhdGFcbiIpOw0KKyAgfSBlbHNlIHsNCisgICAgZnBy
aW50ZihzdGRlcnIsICJQQVNTXG4iKTsNCisgIH0NCit9DQorDQorc3RhdGlj
IHZvaWQNCit0ZXN0X2l0ICh2b2lkKQ0KK3sNCisgIGludCBpLCBsZW47DQor
ICBTQU5FX1BhcmFtZXRlcnMgcGFybTsNCisgIFNBTkVfU3RhdHVzIHN0YXR1
czsNCisgIEltYWdlIGltYWdlID0gezAsIH07DQorICBzdGF0aWMgY29uc3Qg
Y2hhciAqZm9ybWF0X25hbWVbXSA9DQorICAgIHsgImdyYXkiLCAiUkdCIiwg
InJlZCIsICJncmVlbiIsICJibHVlIiB9Ow0KKw0KKyAgc3RhdHVzID0gc2Fu
ZV9zdGFydCAoZGV2aWNlKTsNCisgIGlmIChzdGF0dXMgIT0gU0FORV9TVEFU
VVNfR09PRCkgew0KKyAgICBmcHJpbnRmIChzdGRlcnIsICIlczogc2FuZV9z
dGFydDogJXNcbiIsDQorICAgICAgICAgICAgIHByb2dfbmFtZSwgc2FuZV9z
dHJzdGF0dXMgKHN0YXR1cykpOw0KKyAgICBnb3RvIGNsZWFudXA7DQorICB9
DQorDQorICBzdGF0dXMgPSBzYW5lX2dldF9wYXJhbWV0ZXJzIChkZXZpY2Us
ICZwYXJtKTsNCisgIGlmIChzdGF0dXMgIT0gU0FORV9TVEFUVVNfR09PRCkg
ew0KKyAgICBmcHJpbnRmIChzdGRlcnIsICIlczogc2FuZV9nZXRfcGFyYW1l
dGVyczogJXNcbiIsDQorICAgICAgICAgICAgIHByb2dfbmFtZSwgc2FuZV9z
dHJzdGF0dXMgKHN0YXR1cykpOw0KKyAgICBnb3RvIGNsZWFudXA7DQorICB9
DQorDQorICBpZiAocGFybS5saW5lcyA+PSAwKSB7DQorICAgIGZwcmludGYg
KHN0ZGVyciwgIiVzOiBzY2FubmluZyBpbWFnZSBvZiBzaXplICVkeCVkIHBp
eGVscyBhdCAiDQorICAgICAgICAgICAgICIlZCBiaXRzL3BpeGVsXG4iLA0K
KyAgICAgICAgICAgICBwcm9nX25hbWUsIHBhcm0ucGl4ZWxzX3Blcl9saW5l
LCBwYXJtLmxpbmVzLA0KKyAgICAgICAgICAgICA4ICogcGFybS5ieXRlc19w
ZXJfbGluZSAvIHBhcm0ucGl4ZWxzX3Blcl9saW5lKTsNCisgIH0gZWxzZSB7
DQorICAgIGZwcmludGYgKHN0ZGVyciwgIiVzOiBzY2FubmluZyBpbWFnZSAl
ZCBwaXhlbHMgd2lkZSBhbmQgIg0KKyAgICAgICAgICAgICAidmFyaWFibGUg
aGVpZ2h0IGF0ICVkIGJpdHMvcGl4ZWxcbiIsDQorICAgICAgICAgICAgIHBy
b2dfbmFtZSwgcGFybS5waXhlbHNfcGVyX2xpbmUsDQorICAgICAgICAgICAg
IDggKiBwYXJtLmJ5dGVzX3Blcl9saW5lIC8gcGFybS5waXhlbHNfcGVyX2xp
bmUpOw0KKyAgfQ0KKyAgZnByaW50ZiAoc3RkZXJyLCAiJXM6IGFjcXVpcmlu
ZyAlcyBmcmFtZSwgJWQgYml0cy9zYW1wbGVcbiIsDQorICAgICAgICAgICBw
cm9nX25hbWUsIGZvcm1hdF9uYW1lW3Bhcm0uZm9ybWF0XSwgcGFybS5kZXB0
aCk7DQorDQorICBpbWFnZS5kYXRhPSBtYWxsb2MocGFybS5ieXRlc19wZXJf
bGluZSAqIDIpOw0KKw0KKyAgY2xlYW5fYnVmZmVyKGltYWdlLmRhdGEsIHBh
cm0uYnl0ZXNfcGVyX2xpbmUgKiAyKTsNCisgIGZwcmludGYgKHN0ZGVyciwg
IiVzOiByZWFkaW5nIG9uZSBzY2FubGluZSwgJWQgYnl0ZXMuLi5cdCIsIHBy
b2dfbmFtZSwNCisgICAgICAgICAgIHBhcm0uYnl0ZXNfcGVyX2xpbmUpOw0K
KyAgc3RhdHVzPSBzYW5lX3JlYWQoZGV2aWNlLCBpbWFnZS5kYXRhLCBwYXJt
LmJ5dGVzX3Blcl9saW5lLCAmbGVuKTsNCisgIHBhc3NfZmFpbChwYXJtLmJ5
dGVzX3Blcl9saW5lLCBsZW4sIGltYWdlLmRhdGEsIHN0YXR1cyk7DQorICBp
ZiAoc3RhdHVzICE9IFNBTkVfU1RBVFVTX0dPT0QpIGdvdG8gY2xlYW51cDsN
CisNCisgIGNsZWFuX2J1ZmZlcihpbWFnZS5kYXRhLCBwYXJtLmJ5dGVzX3Bl
cl9saW5lICogMik7DQorICBmcHJpbnRmIChzdGRlcnIsICIlczogcmVhZGlu
ZyBvbmUgYnl0ZS4uLlx0XHQiLCBwcm9nX25hbWUpOw0KKyAgc3RhdHVzPSBz
YW5lX3JlYWQoZGV2aWNlLCBpbWFnZS5kYXRhLCAxLCAmbGVuKTsNCisgIHBh
c3NfZmFpbCgxLCBsZW4sIGltYWdlLmRhdGEsIHN0YXR1cyk7DQorICBpZiAo
c3RhdHVzICE9IFNBTkVfU1RBVFVTX0dPT0QpIGdvdG8gY2xlYW51cDsNCisN
CisgIGZvciAoaT0yOyBpIDwgcGFybS5ieXRlc19wZXJfbGluZSAqIDI7IGkq
PTIpIHsNCisgICAgY2xlYW5fYnVmZmVyKGltYWdlLmRhdGEsIHBhcm0uYnl0
ZXNfcGVyX2xpbmUgKiAyKTsNCisgICAgZnByaW50ZiAoc3RkZXJyLCAiJXM6
IHN0ZXBwZWQgcmVhZCwgJWQgYnl0ZXMuLi4gXHQiLCBwcm9nX25hbWUsDQor
ICAgICAgICAgICAgIGkpOw0KKyAgICBzdGF0dXM9IHNhbmVfcmVhZChkZXZp
Y2UsIGltYWdlLmRhdGEsIGksICZsZW4pOw0KKyAgICBwYXNzX2ZhaWwoaSwg
bGVuLCBpbWFnZS5kYXRhLCBzdGF0dXMpOw0KKyAgICBpZiAoc3RhdHVzICE9
IFNBTkVfU1RBVFVTX0dPT0QpIGdvdG8gY2xlYW51cDsNCisgIH0NCisNCisg
IGZvciAoaS89MjsgaSA+IDI7IGkvPTIpIHsNCisgICAgY2xlYW5fYnVmZmVy
KGltYWdlLmRhdGEsIHBhcm0uYnl0ZXNfcGVyX2xpbmUgKiAyKTsNCisgICAg
ZnByaW50ZiAoc3RkZXJyLCAiJXM6IHN0ZXBwZWQgcmVhZCwgJWQgYnl0ZXMu
Li4gXHQiLCBwcm9nX25hbWUsDQorICAgICAgICAgICAgIGkgLSAxKTsNCisg
ICAgc3RhdHVzPSBzYW5lX3JlYWQoZGV2aWNlLCBpbWFnZS5kYXRhLCBpIC0g
MSwgJmxlbik7DQorICAgIHBhc3NfZmFpbChpIC0gMSwgbGVuLCBpbWFnZS5k
YXRhLCBzdGF0dXMpOw0KKyAgICBpZiAoc3RhdHVzICE9IFNBTkVfU1RBVFVT
X0dPT0QpIGdvdG8gY2xlYW51cDsNCisgIH0NCisgDQorY2xlYW51cDoNCisg
IHNhbmVfY2FuY2VsIChkZXZpY2UpOw0KKyAgaWYgKGltYWdlLmRhdGEpDQor
ICAgIGZyZWUgKGltYWdlLmRhdGEpOw0KK30NCisNCiBpbnQNCiBtYWluIChp
bnQgYXJnYywgY2hhciAqKmFyZ3YpDQogew0KQEAgLTk3NSw2ICsxMDc2LDcg
QEANCiAJY2FzZSAnZCc6IGRldm5hbWUgPSBvcHRhcmc7IGJyZWFrOw0KIAlj
YXNlICdoJzogaGVscCA9IDE7IGJyZWFrOw0KIAljYXNlICd2JzogKyt2ZXJi
b3NlOyBicmVhazsNCisJY2FzZSAnVCc6IHRlc3Q9IDE7IGJyZWFrOw0KIAlj
YXNlICdMJzoNCiAJICB7DQogCSAgICBpbnQgaTsNCkBAIC0xMDE0LDYgKzEx
MTYsNyBAQA0KIC1kLCAtLWRldmljZS1uYW1lPURFVklDRSAgIHVzZSBhIGdp
dmVuIHNjYW5uZXIgZGV2aWNlXG5cDQogLWgsIC0taGVscCAgICAgICAgICAg
ICAgICAgZGlzcGxheSB0aGlzIGhlbHAgbWVzc2FnZSBhbmQgZXhpdFxuXA0K
IC1MLCAtLWxpc3QtZGV2aWNlcyAgICAgICAgIHNob3cgYXZhaWxhYmxlIHNj
YW5uZXIgZGV2aWNlc1xuXA0KKy1ULCAtLXRlc3QgICAgICAgICAgICAgICAg
IHRlc3QgYmFja2VuZCB0aG9yb3VnaGx5XG5cDQogLXYsIC0tdmVyYm9zZSAg
ICAgICAgICAgICAgZ2l2ZSBldmVuIG1vcmUgc3RhdHVzIG1lc3NhZ2VzXG5c
DQogLVYsIC0tdmVyc2lvbiAgICAgICAgICAgICAgcHJpbnQgdmVyc2lvbiBp
bmZvcm1hdGlvblxuIiwNCiAJICAgIHByb2dfbmFtZSk7DQpAQCAtMTEyMSw3
ICsxMjI0LDcgQEANCiAJICAgIGNhc2UgJz8nOg0KIAkgICAgICBleGl0ICgx
KTsJLyogZXJyb3IgbWVzc2FnZSBpcyBwcmludGVkIGJ5IGdldG9wdF9sb25n
KCkgKi8NCiANCi0JICAgIGNhc2UgJ2QnOiBjYXNlICdoJzogY2FzZSAndic6
IGNhc2UgJ1YnOg0KKwkgICAgY2FzZSAnZCc6IGNhc2UgJ2gnOiBjYXNlICd2
JzogY2FzZSAnVic6IGNhc2UgJ1QnOg0KIAkgICAgICAvKiBwcmV2aW91c2x5
IGhhbmRsZWQgb3B0aW9ucyAqLw0KIAkgICAgICBicmVhazsNCiANCkBAIC0x
MjMxLDcgKzEzMzQsMTAgQEANCiAgIHNpZ25hbCAoU0lHUElQRSwgc2lnaGFu
ZGxlcik7DQogICBzaWduYWwgKFNJR1RFUk0sIHNpZ2hhbmRsZXIpOw0KIA0K
LSAgc2Nhbl9pdCAoKTsNCisgIGlmICh0ZXN0ID09IDApIA0KKyAgICBzY2Fu
X2l0ICgpOw0KKyAgZWxzZQ0KKyAgICB0ZXN0X2l0ICgpOw0KIA0KICAgc2Fu
ZV9jbG9zZSAoZGV2aWNlKTsNCiANCg==
--1318610500-70641457-915118363=:29443--

--
Source code, list archive, and docs: http://www.mostang.com/sane/
To unsubscribe: echo unsubscribe sane-devel | mail majordomo@mostang.com