1
0
mirror of synced 2026-01-16 08:43:50 +00:00
rmkaplan 4238dc3888
Rmk56 filepos with external formats #3 (#827)
* TESTUPF:  Move from internal/ to internal/test/unpackfilename

to be parallel with filepos

* IOCHAR:  FILEPOS respects external format (#3)

* FILEPKG: EDITCALLERS speed up with new FILEPOS

* internal/test/filepos:  Testing jig and cases for new FILEPOS

* EXTERNALFORMAT, IOCHAR: fix external-format glitches
2022-07-16 21:30:54 -07:00
..

FILEPOS testing files

The file FPTESTS has the specifications for testing against the sample files in this directory.  The file TFP has the functions for performing the tests, plus copies of old versions of FILEPOS and FFILEPOS for comparison.

Each entry in FPTESTS is a pair of the form (expected-value filepos-args), with the convention that the pattern and casearray args are evaluated before the the filepos function is applied.  CASEARRAY=T is interpreted as transparent, it just throws test into the slow case (/SLOWFILEPOS) of very careful interpretion of non-ascii characters.

The extension of the testfiles specifies the external format to be asserted of the file before the test is run (defaults to :XCCS) Each test opens and closes the file, so the file position at the start of each test is 0, no carry-over from one test to another.

(TFP TESTNAMES TAGS FN)
TESTNAMES is or contains the names of the variables containing a particular test suite.  NIL means all the names in ALLTESTS.

THe file-pos args of a particular test and the value of FN are passed to TFP1.

(TFP1 FPARGS FN) runs the tests.  

The FN argument of TFP and TFP1 specifies the function to apply:
     NIL or FILEPOS:  New FILEPOS
          (NIL with CASEARRAY=T also means new FFILEPOS)
      FFILEPOS or FF:  new FFILEPOS
     OLDFILEPOS or OF:  Old FILEPOS
     OLDFFILEPOS or OFF:  Old FFILEPOS

Test files:

ZERONETWO.XCCS
      012   (sanity)

GREEK.XCCS
      aÿÿ<C3BF>&e&v&d&e&z&bÿ<62>c
    (Charset shift 255 38 before first ÿÿ<C3BF>&eÿ<65>, not before second: ÿÿ<C3BF>&e&zÿ<7A> fails unless starting at 4)

GREEK.UTF8  (hyphen in UTF-8) can be omitted)
      aÿÿ<C3BF>&e&v&d&e&z&bÿ<62>c coded in UTF-8  All searches should work

BIG.XCCS
       32,764 a's followed by 012345X1245
            (segment size to avoid fixp's is 32768, this crosses over)

BIGGREEK.XCSS
       32762 a's followed by ÿÿ<C3BF>&e&v&d&e&z&bÿ<62>cÿÿ<C3BF>&e&eÿ<65>b
         (Greek runs over segment boundary, second ÿÿ<C3BF>&eÿ<65> has a Greek prefix so XCCS fails in fast case (CASEARRAY=NIL)

BIGGREEK.UTF8
     (in UTF-8)

SHORT.XCCS
	012345X1245
       (two occurrences of "12", " "123 matches early, "124" tests restarting on failure)
       ("1245" tests matching at EOF, "12456" tests overflow failure)

Timing stats: 
(FINDCALLERS "slslsksks" FILEPKG):
Original:  8.3 secs, 5 secs compute time  (Old EDITCALLERS, old FILEPOS
New fast: 5.2 secs, 2.5 secs compute time (new EDITCALLERS, no  case array)
New slow: 15.9secs, 13 secs compute time.  (new EDITCALLERS with case array flips to \SLOWFILEPOS)

(FILEPOS "ldldld" FILEPKG): 
New fast:   .014  563 FIXP   
Original:  .018 secs  10172 FIXP  1.2 times slower
New slow:  .12 secs  562 FIXP    9 times slower than new fast, 6.7 times slower than original