* A revision to the font, Unicode, Tedit, and other modules to implement the MCCS character coding as the standard for internal text strings. MCCS is a variant of XCCS with arrows switched with circumflex/underscore and $ switched with currency, and allows for additional code assignments over time. :MCCS replaces :XCCS as the default external format, especially for source files. The file XCCS is removed in favor of the file MCCS, which includes the XCCS external format for backward compatibility.
* This includes a single Medley-font formatted font file for each of the family/size/face display fonts. The glyph assignments correspond to the MCCS character encoding (except for fonts with idiosyncratic encodings--Hippo, Symbol). All charsets from legacy font files are included in each file, and the character sets and glyphs in each file have also been extended by offline coercion from related families (e.g. Glyphs not in legacy Terminal are taken from legacy Modern). There should be fewer black boxes, and character-display shouldn't change when you switch fonts.
* The Unicode mapping tables have been redefined to set up correspondences between Unicode and MCCS, not XCCS. Separate XCCS to/from MCCS mapping functions are provided in the file MCCS; they are no longer included in INTERPRESS.
* TEDIT converts characters in legacy fonts to their new MCCS codes as it reads formatted files, marks the file as MCCS compatible and preserves the new codes on writing.
* Default keyboard assignments produce the MCCS uparrow and leftarrow for shift-6 and shift-hyphen, use Function-6 for circumflex and Function-10 for underscore.
See documentation in FONTCODECHANGES.TEDIT MCCS.TEDIT MEDLEYFONTFORMAT.TEDIT in docs/internal, and library/UNICODE.TEDIT.
CHARSET.DECODE, CHARSET.ENCODE added. CHARACTER.ENCODE moved here from TEDIT-FNKEYS. Some character names moved here from a few other files.
This anticipates the ability to use names for the charset argument to FONTCREATE. IOCHAR needed to move a little earlier so that STRING.EQUAL/UPPERCASEARRAY is available before FONTCREATE created the guaranteed displayfont.
This PR contains a large number of changes in support of the implementation of the Medley Dsplay Fon file format.
The changes are documented in the docs/internal/FONTCHANGES.TEDIT file.
* EXTERNALFORMAT: \CHECKECOLC macro confusedf ANY vs CR EOL convention
* Recompile callers of \CHECKEOLC macro
* If ANY and no CR after LF, return EOL instead of CR
* EXTERNALFORMAT,UNICODE,XCCS: Consistent naming of \PEEKCCODE(.EOL), \BACKCCODE returns the code
It was confusing that the naming of the peek functions was inconsistent with the inccode and backccode functions with respect to the EOLC qualifier. Now uniformaly, the unqualified names (\PEEKCCODE, \BACKCCODE, \INCCODE) do not do any EOL interpretation. The qualified-name functions (\PEEKCCODE.EOLC, \INCCODE.EOLC, \BACKCCODE.EOLC) do EOL interpretation.
Also, the \BACKCCODE interface is changed so that it returns the code corresponding to the bytes that it passed over. This simplifies some of the callers.
* Clients of \PEEKCCODE \BACKCCODE and .EOL
Small changes either to fix inconsistencies (backing, reading, peeking with inconsistent EOL interpretation), updating the names in a few other cases, a few simplifications.
* glitches in ATERM, HARDCOPY updates
* TTYIN: Meta chars defined by Meta charset, not specific value 1
Also, changed \TTYIN.RPEOF to use \FORMATBYTESTREAM generic function so that it does not make assumptions about the stream's external format
* LLKEY: Respecify 535 as "Function,^W" in \ORIGKEYACTIONS
First step in anticipating moving meta and signaling characters to unoccupied parts of the Unicode code space
* Have to commit the build artifacts in order to unstash changes
* CMLSTRING: No change, just upgrade filemap for CL Functions
* LLREAD: A little tighter on CHARCODE.DECODE
Allow hex and unicode in comma-separated specs, but must have 0X or U+ indicator. Don't allow junk characters
* LLKEY: Revert attempt at Meta,x and Function,x
For unknown reason, CHARTABLE isn't declared when LLREAD appears in loadup sequence.
* Unicode: Unicode character-constants weren't properly prefixed
Needed U+ or x0
* THINFILES (again): Add version number for mis-ordered HELP
* ATBL: Change 1,0...1,377 to Meta,0...Meta,377
In anticipation of a future migration of the meta charset.
* TEDITFNKEYS: Convert 1,x 2,x to Meta,x Function,x
In anticipation of eventual deconfliction with Unicode
* FILEBROWSER: remove reference to reader-environment field, add fix-directory-dates
Added a new submenu item under Recompute to make directory dates match the filecreated dates for Medley source/compiled files.
* SEDIT-COMMANDS: 1/2 to Meta/Function in Sedit commands
Codes in the 512+ range are now Function, anticipating eventual code-space translation
* FILEPKG: Consistency of exports.all external format
* Format implementation functions set a known variable *BYTECOUNTER*
The generic functions deal with updating the application variable.
\INCHAR eliminated in favor of \INCCODE.EOLC to make clear what it does.
OPENSTRINGSTREAM streams have their own format, and the string is always fattened.
READBITMAP doesn't mix character and byte reading
* AOFD: Don't execute \STRINGSTREAM.INIT
This creates a file device that is not used anywhere. The function OPENSTRINGSTREAM provides the functionality that this file device suggests that it would provide, but that functionality seems suspect at best. The function is left in the system for now, probably should be deleted at some point in the future so we don't try to maintain it if we trip over it.
* TTYIN: Fix an ancient coding error
but still doesn't solve the (DIRECTORY ?= problem #402
* LLREAD, FILEIO, XCCS: Improve charcode backing, copychars
Added \BACKCCODE.EOLC that backs up over EOL encoding bytes, simplifies \RSTRING2.
\XCCSBACKCCODE returns T/NIL according to whether it succeeded.
\XCCSOUTCHAR uses IPLUS16 for CHARPOSITION
COPYCHARS makes no assumptions about EOL encoding
But still no solution for #402
* LLREAD, TTYIN.LCOM fix#402
The bug showed up in TTYIN, but it was actually a bad edit in the generic backccode.
TTYIN.LCOM is just a recompile--that had never been done with various new declarations.
* DEFINE-FILE-INFO with its own reader environment
Including format. That reader environment also allows colon as a package delimiter, in addition to the funky control and upper panel character.
"Function" is now a synonym for character set 2. WITH-READER-ENVIRONMENT doesn't bind EXTERNALFORMAT.
* Changed *DEFAULT-MAKEFILE-ENVIRONMENT* to use Interlisp rdtbl
* Store FDEV's default externalformat in the FDEV
And related adjustments so that the display output stream (which is created in various places) is always created with the right (CR) EOL convention. Also a little simpler interface for creating external formats.
* Cleanup of character IO interface
Committing this branch for further testing. I know at least that the TTY output stream somehow is defaulting to :XCCS, which is wrong, but I haven't yet found the interface for that.
* Clean out \NSIN etc
No top-level calls to the NS specific functions, just to the generic \OUTCHAR etc.
Updated full.database
* MODERNIZE: added dragging for fixed-menu windows
They can be dragged by their title bars
* UNICODE: Added Greek to the default set
Also made spelling of default-externalformats consistent with FILEIO
* FASLOAD: EOL conversion in FASL::READ-TEXT
EOL's printed as LF's will be read as EOL
* LLREAD: Added meta as a CHARACTERSETNAME
meta,a maps to 1,a now. But slowly propagating this to TEDIT, SEDIT, etc will make it easier to change the coding of meta characters, e.g. as part of a Unicode transition.
* APRINT FILEIO LLREAD: \OUTCHAR now a closed function
Removed the macro
* LLKEY: call CHARCODE.DECODE directory in \KEYACTION1
Minor cleanup, avoid typical user entry and APPLY*
* WHEELSCROLL: re-enable on AFTERMAKESYS/SYSOUT FORMS
Also sets up mappings in the \COMMANDKEYACTIONS, whatever that is
* ABASIC: NILL and ZERO change from LAMBDA NOBIND to LAMBDA NIL
So that things like Masterscope don't break
* MASTERSCOPE: Added WHEREIS as last-resort for CONTAINS
Looks at the WHEREIS database, if present, for FNS and FUNCTIONS if it has no other information. . WHO CONTAINS ANY CALLING FOO works, but not the inverse: . WHO DOES FUM CONTAIN. We still need to figure out why the CONTAINS table isn't populated
* POSTSCRIPTSTREAM: use standard \OUTCHAR conventions
Now uses generic \OUTCHAR to get the proper function from the stream (or default)
* Recompile with right EXPORTS.ALL
Some of the macros weren't correct.
* Fix POSTSCRIPTSTREAM
Cleaner separation between external \OUTCHAR and internal BOUT
* POSTSCRIPTSTREAM gets its own external format
* Minor fix
* Compile-time warning about EXPORTS.ALL
* MODERNIZE: Modern button fn has same args as the original
For Notecards #343
* Fixed another glitch in the MODERNIZE arglist thing
\TEDIT.BUTTONEVENTFN actually takes a second STREAM argument. I don't see where it is ever called with that. The modernize replacement binds that argument, but it isn't being passed to the original.
* FILEWATCH: added missing record field
* Update FILEWATCH.LCOM
* Eliminating record/type name conflicts
Mostly just qualifying references, more work to get BIGBITMAP stuff out of ADISPLAY and to eliminate ambiguity of LINE record (now XXLINE in XXGEOM)
* Compile away open calls to \OUTCHAR, add loadups/full.database
Mostly new LCOMS where \OUTCHAR calls were compiled open
* Remove garbage library/XCCS
Old tools for reading wikipedia XCCS tables, sources/XCCS will deal with XCCS external format
* Next step: Remove open input-character calls, factor XCCS to separate file
XCCS is the default, but can be swapped out (eventually) by setting a few variables, without recompiling everything
* Lots of residual cleanup for XCCS isolation
* Delete old file MACINTERFACE (migrated to MODERNIZE)
* Eliminate straggling NS calls: LAFITE, READINTERPRESS
* Typo
* READINTERPRESS: removed CHARSET
* MODERNIZE: Interface to control title-bar response (for Notecards)
* Many changes for external format name consistency
Very close to the end of this
* Put :FORMAT in file info, fix TEDIT plaintext hardcopy
I distributed :FORMAT :XCCS as the default marking, but somehow one of the variables seems to get revert during the loadup. This is correct, as far as it goes.
* Getting the format in the file-info
This is all very twisty, different variables set in different places. It now seems to do the right thing, at least for new files. Marks them with :FORMAT :XCCS.
* Another fileinfo glitch
* CLIPBOARD -UNICODE: Make UTF8 to UTF-8 to match standards
* MODERNIZE: fix bug in MODERWINDOW
* External format as MAKEFILE option, LOAD applies the file's format
(MAKEFILE 'XX '((FORMAT :UTF-8)))
will dump XX as a UTF-8 file. LOAD will load it back to XCCS internal.
* Compilers respect DEFINE-FILE-INFO format
* MODERNIZE: little glitch
* Delete old FILEIO.LCOM
* More edge cases of external format thru MAKEFILE, PRETTY, PRETTYFILEINDEX etc.
* FILEBROWSER: Can SEE UTF-8 Lisp sourcefile
* INSPECT: Better macro for inspecting readtables
* recompile changed files and do new loadup
Co-authored-by: rmkaplan <ron.kaplan@post.harvard.edu>
UFS changes the the default in \UFSeol to LF.EOLC. LLREAD changes \rprint2 to convert EOL to LF if escaped in a string. NSPROTECTION eliminates literal EOL
