* WIP in changing loadups to tagged nomeclature rather that branch and ability to delete tagged branches as well as thin them.
* In loadup and medley scripts, changed the --branch (-br) command line argument to --tag (-tg). Correspondingly changed the MEDLEYDIR/loadups/branches directory to be MEDLEYDIR/loadups/tagged. Separately, added a + parameter to the --thinw and --thinl command line arguments which causes ALL files to be deleted, not just the versioned files. Added a --thin command line argument that is equivalent to --thinw followed by --thinl. Updated the man page to reflect all these changes.
* Updated medley script man page to reflect the new tag nomenclature instead of branch nomenclature.
* Fix issue#2063 - typo in medley man page
* 1) In loadup script made --help (-h) be a synomym of --man (-z). 2) In both loadup and medley scripts added an optional parameter (+) to the --man (z) command line argument that when specified will display the man page without paging - i.e., sets the pager to cat.
* Fix Issue#2251 LOADUPSDIRECTORIES settings by the MEDLEYDIR fn now takes into account tagged loadups
* Add to loadup and medley scripts a --branch argument whereby which loadup outputs are stored in MEDLEYDIR/loadups/branches/BRANCH, where BRANCH is the name of the currently active git branch of MEDLEYDIR (for loadup) or the name of a BRANCH whereby which a loadup had been run with the --branch option (for medley). Man pages updated accordingly.
* Removed --git-branch synonym for --branch from both medley and loadup scripts. Added optional BRANCH argument to --branch in loadup script. If BRANCH is specified iot is used as the (pseudo)branch name instead of the current active git branch of MEDLEYDIR. If BRANCH is "-", the current git branch name is used.
* When doing loadups, a file called gitinfo is created in the LOADUP_OUTDIR that contains the git commit, git branch and git status of MEDLEYDIR at the time of the loadup. Branch names in loadup and medley scripts now contain only alphanumerics, dahes, underscores, periods. Any other character is coerced to underscore. The branch name is now included in the window title.
* Incorporate changes from PR#2208: Use the loadup step script names as the --id string.
* Add loadups/gitinfo and loadups/branches to .gitignore
* Fix Issue #2155 (loadup -f -b -x fails can't find full.sysout). Reordered loadup so that the sysouts are copied to loadups before loadup-aux and loadup-db are run, and that the product of loadup-aux are copied to loadups before loadup-db is run -- all to make sure that the right prerequisites are available in loadups when needed. Also forced a run of loadup-aux if full.sysout is newer than exports.all when -db is specified.
* Fix Issue 2148 - loadups fail on WSL1 with Xvnc server error. Root cause of 2148 is that certain Medley sessions in loadups run very quickly - less that a second or 2. This causes the medley script to improperly detect an Xvnc server error. Fixed by adding a cli flag --automation that forces the medley script to skip the check for Xvnc server errors. Changed loadup script to call medley with this flag set. Also add a cl flag to loadup --forcevnc that forces loadup to use vnc even on WSL2. This is intended for testing purposes only.
Fix Issue #2155 (loadup -f -b -x fails can't find full.sysout).
Reordered loadup so that the sysouts are copied to loadups before loadup-aux and loadup-db are run, and that the product of loadup-aux are copied to loadups before loadup-db is run -- all to make sure that the right prerequisites are available in loadups when needed.
Also forced a run of loadup-aux if full.sysout is newer than exports.all when -db is specified.
Here is what this PR does:
Fix Issue 2139: (MAKE-INDEX-HTMLS) was not handling pseudohosts correctly and an errant LI pseudohost was causing MAKE-INDEX-HTMLS to terminate early. Adjusted MAKE-INDEX-HTML so it uses psuedohosts only for the top level directory and everything further down in the tree uses the truenames relative to the top-level pseudohost, Results in a MAKE-INDEX-HTMLS run that works in the presence of random pseudohosts and in a collection of index.html files without difficult to understand and out of context references to pseudohosts.
Remove loadups/build directory from all HCFILES runs (on desktop and via github actions)
Added maiko source code and removed maiko lde executables from HCFILES outputs for github actions - thus adding maiko code and removing maiki executables @ files.interlisp.org.
Fixed scripts/clean_hcfiles.sh so that it actually cleans off all of the index.html files - was missing some.
.github/workflow directory was being left out of HCFILES. Put it back it.
Three changes to loadup scripts:
1) dribble files are now copied from the workdir into loadups if loadup completes successfully,
2) if the lock preventing simultaneous runs is already set when loadup starts, it now asks the user if they want to override the lock or to exit (previously it just exited),
3) there is now a --override flag that will automatically override the lock without asking the user.
Man page updated accordingly.
This PR is an extensive revamp of the loadup scripts including the following changes:
New omnibus script scripts/loadup which is meant to be the single interface to the loadup system. The man page for this script can be found here: https://online.interlisp.org/downloads/man_loadup.html
The new loadup script allows you to restart the loadup process from any particular stage - init, mid, lisp, full, etc. For example, you can start the loadup from an existing init-mid.sysout and have it run thru creating the full.sysout. The call for this would be: ./scripts/loadup --target full --start mid. (See man page for all options to the loadup script as well as examples of their use.)
In order to facilitate this target/start feature, the loadup workdir has been moved to a single location per MEDLEYDIR, specifically to MEDLEDIR/loadups/build. (Previously, it was different for every invocation in /tmp/loadups-$$)
When restarting the loadup from, say, lisp.sysout, the script will look for the lisp.sysout to start from first in MEDLEYDIR/loadups and then in MEDLEYDIR/loadups/build. If the starting sysout is found in loadups, it will be copied (non-versioned) into the workdir before doing the loadup, overwriting whatever is already there.
There is now a lock (MEDLEYDIR/loadups/build/lock) that prevents concurrent loadups from running (and very occassionally needs to be manually removed).
At successful completion of a loadup run, the created .sysouts and other files are moved into the loadups directory as before (unless the --nocopy option is specified). BUT the build files - including the dribble files - are left in the working directory (i.e., in loadups/build)
The loadup script now supersedes the former loadup-all.sh, loadup-full.sh and loadup-db.sh scripts. But I have left in their place scripts that call the new loadup script with the right options. loadup-all.sh is now just a link to the single loadup script, who's options are (hopefully) a superset of loadup-all.sh.
All of the worker scripts (e.g., loadup-full-from-lisp.sh) have been updated to use better mechanisms to catch and report errors, including the new (LOGOUT T EXITCODE) feature. The main script has been updated to better catch errors in these worker scripts when they do happen.
You can now specify a MAIKODIR for the loadup, either using the MAIKODIR env variable or thru the --maikodir command line option.
All of the loadup scripts have been moved down one level into scripts/loadups. But there are links from the scripts/ directory into the scripts/loadups directory for all of the top-level legacy scripts - loadup-all.sh, loadup-db.sh, loadup-full.sh so that you do not have to change your own scripts unless you need to use some of the new features. More importantly, none of the github workflows need to change right now.
The SYSOUTCOMMITS mechanism had to be changed since there is no guarantee that e.g., lisp.sysout and full.sysout are built on the same commit. So (ASSOC 'MEDLEY SYSOUTCOMMITS) now returns an assoc list of sysouts, e.g., ((INIT "aaa")(LISP "bbb")(FULL "ccc")(APPS "ddd")), showing the commits for the various layers of the loadup.
The new loadup scheme allows different sysouts to be created from different commits, so there is an issue with RDSYS (and RDSYS.LCOM) being out of sync with one or more of the sysout in loadups. RDSYS(.LCOM) are copied into library when ever a loadup of the Init stage completes successfully (unless the -nocopy option is specified). The only way to solve this issue when it arises is to do a complete loadup from starter.sysout to full.sysout (or apps.sysout) to ensure evrything is built on the same commit.
1961 - RDSYS incomplete path for copying
2061 - delay starting making loadups due to "expensive" operation just to check if connected dir is inside of a git repo
* Fix Issue 2072: update do_hcfiles.sh to use new rem.cm argument to medley script. Also update medley script to print out the REM.CM file used (if any).
* Update buildReleaseIncDocker.yml to account for the change on interlisp/online repo from 'master' to 'main' as primary branch
* Do compile.sh after update to medley-run.sh to add REM.CM to printout when using the medley command.
- Fixed INTERPRET.REM.CM so that it no longer tries to load the file pointed to by LDEINIT and instead loads the file pointed to by LDEREMCM.
LDEINIT remains the file used by greet.
- Adjusted the medley script to have a new argument -cm (or --rem.cm) which sets LDEREMCM as appropriate before launching lde.
- Updated the loadup scripts as required to use this new -cm argument when calling medley.
Finally, added a new feature to the medley script -cc (or --repeat) whereby which when medley finishes it checks for a nonzero file given as the argument to -cc. If that file exists, medley is run again (i.e., repeated) with LDEREMCM set to that file. This repeats until this file no longer exists or is zero-length. The file can be found as the vale of LDEREPEATCM so that each invocation of medley can modify (or delete) this file so as to change the subsequent run of medley.
* Add a call to check for orphaned versions after any checkout
* use == instead of -eq for optional
* Add a scripts/install-repo-checks for things to run after checkout; only this versioning error checked for now
* Add git commit ID to beginning of loadups .dribble files.
The commit ID is put into the .dribble file in the .sh scripts. (using echo ... > name.dribble)
The changes to MEDLEY-UTILS, LOADUP-FULL, and LOADUP-LISP are to enable the passing of APPENDFLG to (DRIBBLE ... APPENDFLG).
* I hadn't saved file before previous commit.
* Add to IL:SYSOUTCOMMITS instead of to the .dribble files.
* Initialize SYSOUTCOMMITS so PUTASSOC has somewhere to put value(s)
In scripts/medley/medley_vnc.sh, update references to Xvnc and vncviewer to their tigervnc-specific versions Xtigervnc and xtigervncviewer. Fixes Issue#1841.
* Add clean-hcfiles.sh; update do_hcfiles.sh to save hcfiles.dribble to loadups and to extract fails to hcfiles-fails.txt in loadups.
* Fix clean_hcfiles.sh to handle pdf files that are tracked by git.
* Tweak output of clean_hcfiles.sh
* First pass at workflow for doing HCFILES on each release
* Finish doHCFILES workflow
* Fix delete of gh-page branch in DoHCFILES workflow
* Redo doHCFILES workflow for files.interlisp.org; add indexing to do_hcfiles script
* Fiddling with workflow names so that I can test doHCFILES.yml on a branch
* in doHCFILES workflow fix use of GH_TOKEN
* Fix typo in doHCFILES workflow
* Debugging doHCFILES workflow
* Fix multiple bugs in do_hcfiles script; fixed multiple bugs in doHCFILES workflow
* Debugging move
* in do_hcfiles.sh add back in Tedit file stoHCFILES run
* Clean up do_hcfiles.sh a bit
* Add debugging code to doHCFILES workflow
* In MAKE-INDEX-HTMLS, add code to ensure that the original case of the files/directory names are preserved since (DIRECTORY) seems to return names ia all-caps, always
* Debugging doHCFILES
* Fiddling with debugging code in doHCFILES workflow
* Add MEDLEY-INIT-VARS to cm file in do_hcfiles.sh
* Undo effect of merging fgh_hcfiles-updates into fgh_hcfiles-workflow. fgh_hcfiles-update will be abadoned
* Add up button to index.html files in MAKE-INDEX-HTMLS
* Update MAKE-INDEX-HTMLS to include an up-on-level button in index.html files. Move fio files to medley instead of source. Streamline doHCFILES workflow
* Debugging
* In MAKE-INDEX-HTMLS, make sure that the up-one button does not appear in the top-level index.html
* In doHCFILES workflow, add difference between development(draft) and production; add doHCFILES workflow into buildReleraseInclDocker workflow
* Update MAKE-INDEX-HTMLS with new onclick script to handle directories properly
* Fix typo in buildRelease workflow
* Polishing up do_hcfiles.sh
* Return buildDocker.yml to original state after using it to test doHCFILES.yml
* Make medley.sh and its associated scripts POSIX compliant - i.e., debashify them
* Added config file for medley script, medley now reads from config file and prepends arguemnts from file to the copmmand line arguments
* WIP. Updates to medley.sh scripts.
* WIP. More on medley.sh and friends update.
* WIP. Medley redo
* WIP. Debugging new medley scripts
* Renamed medley.sh/medley.command to be medley_main.sh. Added code to compile single medley.sh/medley.command script by inlining all of the source'd medley_*.sh files.
* Add temp fix for cygwin Issue #1685
* Minor fixup to medley_utils.sh; take debug code out out of run_medley
* Add README to medley directory to explain how to compile medley.sh (medley.command).
* Ooops. This time really adding the README file to the medley directory explaining how to compile medley.sh (medley.command)
* Update loadup- scripts to use updated medley scripts rather than run-medley
* Fix default setting of $config_file in medley_configfile.sh
* Redo medley compile to pick up last commikt
* Fixing how maiko exe is found and sysout argument error processing - both issues discovered testing on MAcOS
* In medley_configfile, replace echo with printf %s because echo - does not work in zsh
* Supress config file on loadups calls to Medley
* Add oldschool support (use original run-medley) to loadup scripts; improve FAILURE detection so loadup-all won't proceed once one of the components fails
* Add in medley_args.sh add -prog as synonym to --maikoprog to aid in loadup scripts; in medley_run.sh script try to get a good exit code for call to maiko, especially useful for loadup scripts
* Run loadup scripts thru shellcheck and update as necessary to make Posix compliant
* Get rid of -nt comparisons in loadup-setup.sh because they are not posix-complaint. They were not really needed anyway.
* Removing (for now) use of lde exit codes to decide FAILURE case in loadup-setup.sh since exit codes from lde apperar to be inverted on MacOS.
* Update medley man page. Add - functionality to more args is medley_args.sh
* Compile medley.sh with changes from last commit
* Ooops. Left medley_args.sh changes out of last commit. Rectifying here.
* Added support for LDEKEYBOARDTYPE to medley_run to match run-medley
* Add to medley.sh: auto numbered id's and titles with id's inserted
* Cleanup some shellcheck issues in medley_main.sh
* fix maiko args -nh-xxx. were -nethub-xxxx. In medley_run.sh
* Overhaul handling of pass-on args to manage the quoting issues prevelant in the previous implementation
* Cleanup minor shellcheck issues in medley_*.sh scripts
* Add underscore as character allowed in ids - makes things clearer when id used with +
* Add a self-numbering id to medley calls in loadup scripts
* Put workaround in medley_run.sh for Issue #1702 - issues with sysout arg processing in Maiko
* Oops. messed up LDESRCSYSOUT in last commit. should be LDESOURCESYSOUT
* compile medley.sh
* Remove derived PDFs from branch other than gh-pages
* only the gh-pages branch should have the pdfs
* make pdfs and listings
* Update HCFILES and MAKE-INDEX-HTMLS, add preliminary documentation
* add a slashit to avoid angle brackets
* Further cleanup on cpv script; add back a ln_or_cp function and use it to cp only when ln fails for all instances of linking/copying in the script; better handling of case where the unversioned dest file does not exist but version versions of the file DO exst.
* cpv: remove local declarations to be Posix-compliant
* File See operations use full path to loaded file.
Add Manager.WINDOW-ANCHOR to fix corner from which MANAGER-MAIN-WINDOW grows, and (attempt) to keep it on-screen.
* Improved handling of the ICONW for MANAGER-MAIN-WINDOW.
* Fix typo in MasterScope functions (multiple occurrences: LOADBFLG should be LOADDBFLG).
Changed to CL compiler by default (not need to go to submenu). This is my preference, so I should remove it before setting pull request.
* Manual cleanup of multiple "Edited" comments in 4 FNS.
Reverted: Changed to CL compiler by default (not need to go to submenu). (From commit f60c6362)
* Update MANAGER.TEDIT documentation file.
Fix error in previous commit. (Changes that I thought were there, were not.)
Cleanup COMMON-MAKE COMS so it can be handled by the file package, and add .LCOM file to the repo.
* Add cygwin-sdl build to buildLoadup workflow; add installer for cygwin-sdl on windows
* Change how buildLoadup computes latest maiko release to accomodate draft releases
* Fix call to gh release list for maiko
* Debugging call to gh release list for maiko
* Debugging call to gh release list for maiko #2
* Debugging call to gh release list for maiko #3
* Debugging call to gh release list for maiko #4
* Debugging call to gh release list for maiko #5
* Debugging call to gh release list for maiko #6
* Change maiko downloads to accoiunt for draft releases
* Change maiko downloads to account for draft releases #2
* Specify shell (powershell) for Download cygwin installler
* Few cleanup items on cygwin-install
* Update ShellWhich to use command -v instead of which because which returns to much crap on cygwin and command -v is more portable overall
* Switch from using medley-loadup & -runtime tars to medley-full-*.tgz so we get full release incl notecards; delete maiko on install and replace with cygwin maiko
* Make sure Notecards doesn't try to load its HASH fileon PostGreet - for apps.sysout
* Add xdg-utils to cygwin install to support ShellBrowser
* Odds and ends on cygwin build
* Redo medley.iss install script to use tar from Windows rather than cygwin tar because cygwin tar was messing up ACLs in windows. Needed to change creation of medley.bat accordingly.
* Remove junk lines from buildLoadup.yml
* Restore accidently deleted line to buildLoadup.yml
* Fix multiple issues with cygwin_installer filename; arrange to remove placeholder.txt from the release assets at the end of cygwin installer
* Change name of job from windows_installer to cygwin_installer
* Fix missing GH_TOKEN is removal of placeholder.txt; fix naming of output file in medley.iss
* Fiddling with getting cygwin-installer name right
* Redoing merge of medley.sh/medley.command to handle the Darwin plus Cygwin cases; is medley.iss recreate symbolic links surrounding the medley.sh script
* Fix typos/syntrax errors in medley.sh/medley.command
