github.com/Interlisp.medley
1
0
Fork 0
mirror of synced 2026-05-01 22:16:57 +00:00
Code Issues Projects Releases Wiki Activity

Merge (rebase) Cleanup-character-IO-interfaces with master (#356)

Browse Source 
* Cleanup  of character IO interface

Committing this branch for further testing.  I know at least that the TTY output stream somehow is defaulting to :XCCS, which is wrong, but I haven't yet found the interface for that.

* Clean out \NSIN etc

No top-level calls to the NS specific functions, just to the generic \OUTCHAR etc.

Updated full.database

* MODERNIZE: added dragging for fixed-menu windows

They can be dragged by their title bars

* UNICODE:  Added Greek to the default set

Also made spelling of default-externalformats consistent with FILEIO

* FASLOAD: EOL conversion in FASL::READ-TEXT

EOL's printed as LF's will be read as EOL

* LLREAD:  Added meta as a CHARACTERSETNAME

meta,a maps to 1,a now.  But slowly propagating this to TEDIT, SEDIT, etc will make it easier to change the coding of meta characters, e.g. as part of a Unicode transition.

* APRINT FILEIO LLREAD: \OUTCHAR now a closed function

Removed the macro

* LLKEY: call CHARCODE.DECODE directory in \KEYACTION1

Minor cleanup, avoid typical user entry and APPLY*

* WHEELSCROLL: re-enable on AFTERMAKESYS/SYSOUT FORMS

Also sets up mappings in the \COMMANDKEYACTIONS, whatever that is

* ABASIC:  NILL and ZERO change from LAMBDA NOBIND to LAMBDA NIL

So that things like Masterscope don't break

* MASTERSCOPE:  Added WHEREIS as last-resort for CONTAINS

Looks at the WHEREIS database, if present, for FNS and FUNCTIONS if it has no other information.  . WHO CONTAINS ANY CALLING FOO works, but not the inverse:  . WHO DOES FUM CONTAIN.  We still need to figure out why the CONTAINS table isn't populated

* POSTSCRIPTSTREAM: use standard \OUTCHAR conventions

Now uses generic \OUTCHAR to get the proper function from the stream (or default)

* Recompile with right EXPORTS.ALL

Some of the macros weren't correct.

* Fix POSTSCRIPTSTREAM

Cleaner separation between external \OUTCHAR and internal BOUT

* POSTSCRIPTSTREAM gets its own external format

* Minor fix

* Compile-time warning about EXPORTS.ALL

* MODERNIZE:  Modern button fn has same args as the original

For Notecards  #343

* Fixed another glitch in the MODERNIZE  arglist thing

\TEDIT.BUTTONEVENTFN actually takes a second STREAM argument.  I don't see where it is ever called with that.  The modernize replacement binds that argument, but it isn't being passed to the original.

* FILEWATCH:  added missing record field

* Update FILEWATCH.LCOM

* Eliminating record/type name conflicts

Mostly just qualifying references, more work to get BIGBITMAP stuff out of ADISPLAY and to eliminate ambiguity of LINE record (now XXLINE in XXGEOM)

* Compile away open calls to \OUTCHAR, add loadups/full.database

Mostly new LCOMS where \OUTCHAR calls were compiled open

* Remove garbage library/XCCS

Old tools for reading wikipedia XCCS tables, sources/XCCS will deal with XCCS external format

* Next step:  Remove open input-character calls, factor XCCS to separate file

XCCS is the default, but can be swapped out (eventually) by setting a few variables, without recompiling everything

* Lots of residual cleanup for XCCS isolation

* Delete old file MACINTERFACE (migrated to MODERNIZE)

* Eliminate straggling NS calls:  LAFITE, READINTERPRESS

* Typo

* READINTERPRESS:  removed CHARSET

* MODERNIZE: Interface to control title-bar response (for Notecards)

* Many changes for external format name consistency

Very close to the end of this

* Put :FORMAT in file info, fix TEDIT plaintext hardcopy

I distributed :FORMAT :XCCS as the default marking, but somehow one of the variables seems to get revert during the loadup.  This is correct, as far as it goes.

* Getting the format in the file-info

This is all very twisty, different variables set in different places.  It now seems to do the right thing, at least for new files.  Marks them with :FORMAT :XCCS.

* Another fileinfo glitch

* CLIPBOARD -UNICODE:  Make UTF8 to UTF-8 to match standards

* MODERNIZE:  fix bug in MODERWINDOW

* External format as MAKEFILE option, LOAD applies the file's format

(MAKEFILE 'XX '((FORMAT :UTF-8)))
  will dump XX as a UTF-8 file.  LOAD will load it back to XCCS internal.

* Compilers respect DEFINE-FILE-INFO format

* MODERNIZE:  little glitch

* Delete old FILEIO.LCOM

* More edge cases of external format thru MAKEFILE, PRETTY, PRETTYFILEINDEX etc.

* FILEBROWSER:  Can SEE UTF-8 Lisp sourcefile

* INSPECT:  Better macro for inspecting readtables

* recompile changed files and do new loadup

Co-authored-by: rmkaplan <ron.kaplan@post.harvard.edu>
This commit is contained in:
Larry Masinter
2021-07-29 17:07:23 -07:00
committed by GitHub
parent 4fac4e3e96
commit 4efe2f93af
159 changed files with 41605 additions and 42552 deletions
Download Patch File Download Diff File Expand all files Collapse all files

297
sources/XCCS Normal file
View File

@@ -0,0 +1,297 @@
(DEFINE-FILE-INFO READTABLE "INTERLISP" PACKAGE "INTERLISP")
(FILECREATED "24-Jun-2021 23:15:05" {DSK}<Users>kaplan>Local>medley3.5>git-medley>sources>XCCS.;36 13925
changes to%: (VARS XCCSCOMS)
previous date%: "24-Jun-2021 16:47:37"
{DSK}<Users>kaplan>Local>medley3.5>git-medley>sources>XCCS.;35)
(PRETTYCOMPRINT XCCSCOMS)
(RPAQQ XCCSCOMS
[(FNS ACCESS-CHARSET)
(FNS \XCCSINCCODE \XCCSPEEKCCODE \XCCSOUTCHAR \XCCSBACKCCODE \XCCSFORMATBYTESTREAM)
(FNS \CREATE.XCCS.EXTERNALFORMAT)
(FNS \NSIN.24BITENCODING.ERROR)
(INITVARS (*SIGNAL-24BIT-NSENCODING-ERROR*))
(GLOBALVARS \DEFAULTINCCODE \DEFAULTOUTCHAR \DEFAULTBACKCCODE \DEFAULTPEEKCCODE)
(INITVARS (\DEFAULTINCCODE '\XCCSINCCODE)
(\DEFAULTOUTCHAR '\XCCSOUTCHAR)
(\DEFAULTPEEKCCODE '\XCCSPEEKCCODE)
(\DEFAULTBACKCCODE '\XCCSBACKCCODE))
(DECLARE%: EVAL@COMPILE DONTCOPY (EXPORT (CONSTANTS (\NORUNCODE 255)
(NSCHARSETSHIFT 255))
(MACROS \RUNCODED)
(OPTIMIZERS ACCESS-CHARSET)))
(DECLARE%: DONTEVAL@LOAD DOCOPY (P (\CREATE.XCCS.EXTERNALFORMAT])
(DEFINEQ
(ACCESS-CHARSET
[LAMBDA (STREAM NEWVALUE) (* ; "Edited 11-Sep-87 15:46 by bvm:")
(FDEVOP 'CHARSETFN (fetch (STREAM DEVICE) of STREAM)
STREAM NEWVALUE])
)
(DEFINEQ
(\XCCSINCCODE
[LAMBDA (STREAM BYTECOUNTVAR BYTECOUNTVAL) (* ; "Edited 21-Jun-2021 15:44 by rmk:")
(* ;;; "Returns a 16 bit character code. SHIFTEDCSET is STREAM's char set left shifted 8.")
(* ;;; "If BYTECOUNTVAR is non-NIL, it is freely incremented by number of bytes read. If BYTECOUNTVAL is given it is the current value, saves the call to EVAL.")
(* ;;; "This doesn't do EOL conversion, \INCHAR does that")
(LET (NUMBYTES (CSET (ACCESS-CHARSET STREAM))
(CHAR (\BIN STREAM))) (* ;
 "Error on EOF unless ENDOFSTREAMOP does something else.")
(* ;; " NUMBYTES tracks the number of \BINs. ")
(IF (EQ CHAR NSCHARSETSHIFT)
THEN (* ;
 "Shifting character sets, toss CHAR")
(SETQ CSET (\BIN STREAM))
(IF (NEQ NSCHARSETSHIFT CSET)
THEN (* ;
 "Shift to new runcode CSET: SH CS CH")
(ACCESS-CHARSET STREAM CSET)
(SETQ CHAR (\BIN STREAM))
(SETQ NUMBYTES 3)
ELSEIF (EQ 0 (\BIN STREAM))
THEN (* ; "SH SH CSH CS CH where CSH is 0")
(* ;;
 "The high-order character set byte must be 0, because we don't support obese characters (24 bit)")
(SETQ CSET (\BIN STREAM))
(SETQ CHAR (\BIN STREAM)) (* ; "To align with below")
(SETQ NUMBYTES 5)
(ACCESS-CHARSET STREAM \NORUNCODE)
ELSE (\NSIN.24BITENCODING.ERROR STREAM))
(* ;; "The stream now knows the new character set, runcoded or not.")
ELSEIF (EQ CSET \NORUNCODE)
THEN (* ; "2-bytes")
(SETQ CSET CHAR)
(SETQ CHAR (\BIN STREAM))
(SETQ NUMBYTES 2)
ELSE
(* ;; "Runcoded CSET and CHAR")
(SETQ NUMBYTES 1))
(AND BYTECOUNTVAR (SET BYTECOUNTVAR (IDIFFERENCE BYTECOUNTVAL NUMBYTES)))
(CL:WHEN CHAR (* ;
 "Typically NIL if ENDOFSTREAMOP returned NIL at EOF ")
(LOGOR (UNFOLD CSET 256)
CHAR))])
(\XCCSPEEKCCODE
[LAMBDA (STREAM NOERROR) (* ; "Edited 21-Jun-2021 23:44 by rmk:")
(* ;;
 "Modeled on \XCCSINCCODE, but peeks at the last byte in the sequence, leaves the stream unchanged")
(LET ((CSET (ACCESS-CHARSET STREAM))
(CHAR (\PEEKBIN STREAM NOERROR)))
(* ;;
 "Returns a 16 bit character code. Doesn't do EOL conversion--\PEEKCCODE does that. ")
(* ;; "We don't change the charset in the stream, put the file ptr back the way it was.")
(CL:WHEN CHAR
(IF (EQ CHAR NSCHARSETSHIFT)
THEN (\BIN STREAM) (* ; "Read the peeked shifting byte")
(SETQ CSET (\BIN STREAM)) (* ; "Consume the char shift byte")
(IF (NEQ CSET NSCHARSETSHIFT)
THEN
(* ;;
 "Shift to new runcode CSET: SH CS CH. We have to BIN what we peeked, BIN, and peek again")
(SETQ CHAR (\PEEKBIN STREAM NOERROR))
(\BACKFILEPTR STREAM)
(\BACKFILEPTR STREAM)
ELSEIF (EQ 0 (\BIN STREAM))
THEN (* ; "SH SH CSH CS CH where CSH is 0")
(* ;;
 "Note: no eof error check on this \BIN -- an eof in the middle of a charset shift is an error")
(SETQ CSET (\BIN STREAM))
(SETQ CHAR (\PEEKBIN STREAM NOERROR))
(\BACKFILEPTR STREAM)
(\BACKFILEPTR STREAM)
(\BACKFILEPTR STREAM)
(\BACKFILEPTR STREAM)
ELSE (\NSIN.24BITENCODING.ERROR STREAM))
ELSEIF (EQ CSET \NORUNCODE)
THEN (* ; "2 byte runs, BIN/PEEK/BACK")
(SETQ CSET CHAR)
(\BIN STREAM)
(SETQ CHAR (\PEEKBIN STREAM NOERROR)) (* ; "One BACKFILEPTR seems OK")
(\BACKFILEPTR STREAM))
(* ;; "No need to back up for the runcoded case")
(CL:WHEN CHAR
(LOGOR (UNFOLD CSET 256)
CHAR)))])
(\XCCSOUTCHAR
[LAMBDA (STREAM CHARCODE) (* ; "Edited 21-Jun-2021 13:28 by rmk:")
(* ;; "Closed function for the :XCCS external format, also called when :XCCS is the default")
(COND
((EQ CHARCODE (CHARCODE EOL))
[COND
[(NOT (\RUNCODED STREAM)) (* ;
 "Charset is a constant 0, we put out the high-order byte.")
(\BOUT STREAM (\CHARSET (CHARCODE EOL]
((EQ (\CHARSET (CHARCODE EOL))
(ffetch (STREAM CHARSET) of STREAM)))
(T (* ;
 "We are runcoded, and not in character set 0, have to shift.")
(\BOUT STREAM NSCHARSETSHIFT)
(\BOUT STREAM (freplace (STREAM CHARSET) of STREAM with (\CHARSET
(CHARCODE EOL]
(* ;; "We are now in the right charset (0) for the first EOL byte. For CRLF, the CR is immediately followed by the LF byte, without the prefix 0 byte even if not runcoded, i.e. the 2 bytes are though of as a composite. The stream is left in CSET0 (the freplace above), read for another shift according to the next shift in a runcoded file.")
(\BOUTEOL STREAM)
(freplace CHARPOSITION of STREAM with 0))
(T [COND
((NOT (\RUNCODED STREAM))
(\BOUT STREAM (\CHARSET CHARCODE))
(\BOUT STREAM (\CHAR8CODE CHARCODE)))
((EQ (\CHARSET CHARCODE)
(ffetch (STREAM CHARSET) of STREAM))
(\BOUT STREAM (\CHAR8CODE CHARCODE)))
(T (\BOUT STREAM NSCHARSETSHIFT)
(\BOUT STREAM (freplace (STREAM CHARSET) of STREAM with (\CHARSET
CHARCODE))
)
(\BOUT STREAM (\CHAR8CODE CHARCODE]
(freplace CHARPOSITION of STREAM with (PROGN
(* ; "Ugh. Don't overflow")
(IPLUS16 (ffetch CHARPOSITION
of STREAM)
1])
(\XCCSBACKCCODE
[LAMBDA (STREAM BYTECOUNTVAR BYTECOUNTVAL) (* ; "Edited 15-Jun-2021 10:26 by rmk:")
(CL:WHEN (\BACKFILEPTR STREAM)
(* ;; "The immediately preceding byte must be a character byte. If it is a byte in a runcode, then we are done, even if the byte before is part of a shift sequence.")
(* ;; "But if we are currently in a nonruncoded file, we have to go back one more to get the character set byte.")
(* ;; "If we can't back up, we are already at the beginning.")
[COND
[(EQ \NORUNCODE (ACCESS-CHARSET STREAM))
(COND
((\BACKFILEPTR STREAM)
(AND BYTECOUNTVAR (SET BYTECOUNTVAR (IPLUS BYTECOUNTVAL 2)))
T)
(BYTECOUNTVAR (SET BYTECOUNTVAR (ADD1 BYTECOUNTVAL]
(BYTECOUNTVAR (SET BYTECOUNTVAR (ADD1 BYTECOUNTVAL])])
(\XCCSFORMATBYTESTREAM
[LAMBDA (STREAM BYTESTREAM) (* ; "Edited 24-Jun-2021 16:47 by rmk:")
(REPLACE (STREAM CHARSET) OF BYTESTREAM WITH (FETCH (STREAM CHARSET) OF
STREAM])
)
(DEFINEQ
(\CREATE.XCCS.EXTERNALFORMAT
[LAMBDA NIL (* ; "Edited 24-Jun-2021 16:45 by rmk:")
(* ;;; "Create the :XCCS external format. Stream's EOL overrides the (vacuous) default here")
(\INSTALL.EXTERNALFORMAT (create EXTERNALFORMAT
NAME _ :XCCS
INCCODEFN _ (FUNCTION \XCCSINCCODE)
PEEKCCODEFN _ (FUNCTION \XCCSPEEKCCODE)
BACKCCODEFN _ (FUNCTION \XCCSBACKCCODE)
OUTCHARFN _ (FUNCTION \XCCSOUTCHAR)
FORMATBYTESTREAMFN _ (FUNCTION \XCCSFORMATBYTESTREAM)
EOL _ LF.EOLC
EOLVALID _ NIL])
)
(DEFINEQ
(\NSIN.24BITENCODING.ERROR
[LAMBDA (STREAM) (* bvm%: "12-Mar-86 15:35")
(DECLARE (USEDFREE *SIGNAL-24BIT-NSENCODING-ERROR*))
(* ;;; "Called if we see the sequence shift,shift on STREAM -- means shift to 24-bit character set, which we don't support. Usually this just means we're erroneously reading a binary file as text. If this function returns, its value is taken as a character set to shift to")
(COND
(*SIGNAL-24BIT-NSENCODING-ERROR* (* ;
 "Only cause error if user/reader cares")
(ERROR "24-bit NS encoding not supported" STREAM)))
(* ; "Return charset zero")
0])
)
(RPAQ? *SIGNAL-24BIT-NSENCODING-ERROR* )
(DECLARE%: DOEVAL@COMPILE DONTCOPY
(GLOBALVARS \DEFAULTINCCODE \DEFAULTOUTCHAR \DEFAULTBACKCCODE \DEFAULTPEEKCCODE)
)
(RPAQ? \DEFAULTINCCODE '\XCCSINCCODE)
(RPAQ? \DEFAULTOUTCHAR '\XCCSOUTCHAR)
(RPAQ? \DEFAULTPEEKCCODE '\XCCSPEEKCCODE)
(RPAQ? \DEFAULTBACKCCODE '\XCCSBACKCCODE)
(DECLARE%: EVAL@COMPILE DONTCOPY
(* "FOLLOWING DEFINITIONS EXPORTED")(DECLARE%: EVAL@COMPILE
(RPAQQ \NORUNCODE 255)
(RPAQQ NSCHARSETSHIFT 255)
(CONSTANTS (\NORUNCODE 255)
(NSCHARSETSHIFT 255))
)
(DECLARE%: EVAL@COMPILE
(PUTPROPS \RUNCODED MACRO (OPENLAMBDA (STREAM)
(* ;; "returns NIL is the stream is not runcoded, that is, if the stream has 16 bit bytes explicitly represented")
(* ;
 "note that neq is ok since charsets are known to be SMALLP's")
(NEQ (fetch CHARSET of STREAM)
\NORUNCODE)))
)
(DEFOPTIMIZER ACCESS-CHARSET (STREAM &OPTIONAL NEWVALUE)
`((OPENLAMBDA (STRM)
(FDEVOP 'CHARSETFN (fetch (STREAM DEVICE) of STRM)
STRM
,NEWVALUE))
,STREAM))
(* "END EXPORTED DEFINITIONS")
)
(DECLARE%: DONTEVAL@LOAD DOCOPY
(\CREATE.XCCS.EXTERNALFORMAT)
)
(DECLARE%: DONTCOPY
(FILEMAP (NIL (1292 1521 (ACCESS-CHARSET 1302 . 1519)) (1522 10652 (\XCCSINCCODE 1532 . 4344) (
\XCCSPEEKCCODE 4346 . 6882) (\XCCSOUTCHAR 6884 . 9422) (\XCCSBACKCCODE 9424 . 10323) (
\XCCSFORMATBYTESTREAM 10325 . 10650)) (10653 11497 (\CREATE.XCCS.EXTERNALFORMAT 10663 . 11495)) (11498
12329 (\NSIN.24BITENCODING.ERROR 11508 . 12327)))))
STOP