rmk122--Next round on fonts and MCCS

[rmkaplan]

Many files have changed as I have rearchitected the font interfaces, introduced the Medley font-format files, and worked to normalize to the MCCS encoding of internal data.

This PR is a draft that doesn't yet include the flood of files. For starters, it just has the documentation files that describe the strategy and new and changed features. It will be helpful to get some initial reactions so I can make adjustments before pushing everything. The code and font files will come later.

[pamoroso]

On Linux Mint 22.1 Cinnamon this PR builds with no issues and I didn't notice anything unusual. I tested DInfo and NoteCards and the fonts look fine so far.

[rmkaplan]

I have uploaded all of the medleyformat fonts, and removed the draft status.

I expect/hope the only issue is that filenames with underscores and carets might not look the same in Medley vs Unix. I would like to get this tested and merged, to avoid confusion with the changes to the imagefile/hardcopy interfaces.

[pamoroso]

On Linux Mint 22.1 Cinnamon I tested up to commit d01503c and I haven't noticed anything unusual. Names containing ← or ↑ in TEdit look as expected on Linux, i.e. with _ and ^.

[rmkaplan]

I found that PRETTYFILEINDEX had its own compensation for underscore/arrow, at least for Interpress. Removed it.

[rmkaplan]

@MattHeffron , LOADUPFULLFONTS on internal/loadups/LOADUP-FULL has some special code for building the postscript fontcache. I wonder whether this should be done when POSTSCRIPTSTREAM is loaded, as part of POSTSCRIPT.INIT.

[rmkaplan]

A second thought re (CHARCODE DEL): Another solution would be to simply add (CHARCODE DEL) to MCCS, with an identity mapping into Unicode/ISO. And forget about this glitch.

[pamoroso]

I also noticed some minor screen corruption when pressing Backspace in certain cases but couldn't reproduce it.

[rmkaplan]

Chat with MCCS fonts now works for me, for simple inputs (underscore) and backspace. I haven't done more elaborate testing.

Added DEL as an MCCS known character

[rmkaplan]

One other Chat improvement: I modernized its window, so you can drag it around by the title and reshape by the corners.

[nbriggs]

What do I need to put on CHAT.KEYACTIONS to make shift-6 send ASCII caret and shift-minus send underscore? There's no way I'm going to be able to change my touch typing to use different keystrokes between a regular terminal window and a CHAT window.

[rmkaplan]

Also on Chat, I didn't look at the back-space display glitch. And noticed that now and in the release, when it starts up there are some garbage characters on the screen. This is a snap from the release:

[nbriggs]

The garbage is because the shell it's bringing up thinks it's talking to an xterm-256color, but the CHAT terminal emulation is a dm2500 (by default) - xterm controls that the shell is sending show as garbage on a dm2500 terminal. I suspect that something tried to pass down to the shell info about the terminal emulator type, but it failed.

[rmkaplan]

If you want it to generally switch the bindings of shift-6/f-6 and shift-hyphen/f10 (but not shift-4/f-4), that's in the general keyaction table in LLKEY. If people think it's better that the characters align with the labels on current keyboards, then we can revert that change. I think it was discussed somewhere. You would then have to type f-10 to get the left-arrrow in create expressions--maybe dwim could be fixed to coerce _ to ←.

…

On Oct 11, 2025, at 1:44 PM, Nick Briggs ***@***.***> wrote: nbriggs left a comment (Interlisp/medley#2280) <#2280 (comment)> What do I need to put on CHAT.KEYACTIONS to make shift-6 send ASCII caret and shift-minus send underscore? There's no way I'm going to be able to change my touch typing to use different keystrokes between a regular terminal window and a CHAT window. — Reply to this email directly, view it on GitHub <#2280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJN2YVC5JYHWYMNKTVT3XFTZDAVCNFSM6AAAAACF4OLO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJTGY2DOMBRHA>. You are receiving this because you were mentioned.

[rmkaplan]

Is that something to worry about?

…

On Oct 11, 2025, at 1:51 PM, Nick Briggs ***@***.***> wrote: nbriggs left a comment (Interlisp/medley#2280) <#2280 (comment)> The garbage is because the shell it's bringing up thinks it's talking to an xterm-256color, but the CHAT terminal emulation is a dm2500 (by default) - xterm controls that the shell is sending show as garbage on a dm2500 terminal. I suspect that something tried to pass down to the shell info about the terminal emulator type, but it failed. — Reply to this email directly, view it on GitHub <#2280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJN2TBJ7CF34DVHBART3XFUURAVCNFSM6AAAAACF4OLO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJTGY2TANRRGY>. You are receiving this because you were mentioned.

[nbriggs]

Not generally, just in CHAT, which I think should be doable using the CHAT.KEYACTIONS, right? I'm fine with using the underscore key to get a left-arrow in an exec (or SEdit), I'm not sure how I feel about it typing file name patterns in a file browser, but when I'm typing at a Unix shell through a CHAT window it's got to be like every other Unix shell.

We should fix the way it tries to tell the shell about the terminal type. It works better if you have VTCHAT loaded.

[nbriggs]

We're actually sending the correct info for the terminal type - it gets set to dm2500 in the default chat (implementation in DMCHAT). I have a test for LDESHELL in my .zshrc so that I try to set a simpler prompt -- I guess I need to look at what DMCHAT.HANDLECHARACTER is doing with <esc>[?2004h --

1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 27
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 91
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 63
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 50
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 48
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 48
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 52
1 - Exit DMCHAT.HANDLECHARACTER =>
1 - Enter DMCHAT.HANDLECHARACTER:
          CHAR = 104
1 - Exit DMCHAT.HANDLECHARACTER =>

[nbriggs]

I was wrong about the source of the garbage -- it's the zsh line editor (zle) trying to say that we should "set bracketed paste mode". CSI here is <esc>[, not sure why they used mixed Pm/Ps in the description.

CSI ? Pm h
          DEC Private Mode Set (DECSET).
[...]
            Ps = 2 0 0 4  ⇒  Set bracketed paste mode, xterm.
[...]

This is what gets you the reverse-highlighted text (and no activation if there's a linefeed) when you paste into a terminal window.

We could fix up the DMCHAT code to parse the set/reset modes, but the easiest thing (on a mac, with zsh) is to
unset zle_bracketed_paste, except there are some subtleties (see below)

My ~/.zshrc contains (with a few other things)

ssh-add --apple-use-keychain -q
export OSTYPE=$OSTYPE
[ -v LDESHELL ] && source ~/.zshrc-medley

and my ~/.zshrc-medley now contains

export PROMPT="%n %*> "
zmodload zsh/zle   # load it so we can change parameters
unset zle_bracketed_paste

[pamoroso]

I tested up to commit ab5c281. Now pressing F10, F6, and F4 in a Terminal 10 Chat window yields _, ^, and $, respectively. However, the behavior of Backspace and Del is as described earlier, i.e. deleting characters doesn't work.

[rmkaplan]

On the back space, that now works for me (Mac, DM2500 connection to the shell). (I extended MCCS so that it now includes DEL).

I only have the chat-related files that come in the loadup. Looking at the dribble file, the loadup includes
CHAT, DMCHAT, CHATTERMINAL, UNIXCHAT, TEDIT-CHAT (which is unlikely to work).
Nothing more in my init other than to set DEFAULTCHATHOST to NIL.

Can you say/show again what happens when you push the Delete key?

[pamoroso]

With DM2500 I get your same behavior for both Backspace and Del.

With VT100, after typing 1, 2, and 3 in a Terminal 10 CHAT window this recording shows what hapens when I press Backspace 3 times in succession, then type A and press ENTER:

vtchat-backspace.mp4

[pamoroso]

A clarification. With DM2500 I get your same behavior only for the default Gacha font but with Terminal 10 Backspace still behaves as in the recording. In the recording Medley runs under VNC but the behavior is the same without.

[rmkaplan]

Whatever is going on, I don't think it has to do with the font or MCCS. My bet is that the problem is that DSPFONT for a Chat window doesn't do the right thing. It appears to just temporarily change the current font on the Window's display stream, but doesn't change the internal data that is supposed to correspond to that. In the release system, try doing (DSPFONT '(CLASSIC 18) (WHICHW)), typing some stuff, and then backspaces. You see big characters before the backspace, the screen is garbled after the backspaces. But if you then type a return to go to the next line and type something, the characters show up in the original Gacha, as if you hadn't done the DSPFONT. I think this is old undesirable behavior, not related to MCCS or this round of font changes.

…

On Oct 12, 2025, at 9:12 AM, Paolo Amoroso ***@***.***> wrote: pamoroso left a comment (Interlisp/medley#2280) <#2280 (comment)> With DM2500 I get your same behavior for both Backspace and Del. With VT100, after typing 1, 2, and 3 in a Terminal 10 CHAT window this recording shows what hapens when I press Backspace 3 times in succession, then type A and press ENTER: https://github.com/user-attachments/assets/195d06ff-26d9-4545-b87d-2e5fb0f7a66a — Reply to this email directly, view it on GitHub <#2280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJPYVI34VTRGK432DX33XJ4YRAVCNFSM6AAAAACF4OLO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJUG44DSNJXGI>. You are receiving this because you were mentioned.

[rmkaplan]

If VTCHAT is loaded, I don't see the 2400h. But it is still a little bit odd. When I hit the return, it shows an isolated inverted % on the next line, and then it shows an entire prompt string with another % on the line after that. 

…

On Oct 11, 2025, at 4:54 PM, Nick Briggs ***@***.***> wrote: nbriggs left a comment (Interlisp/medley#2280) <#2280 (comment)> I was wrong about the source of the garbage -- it's the zsh line editor (zle) trying to say that we should "set bracketed paste mode". CSI here is <esc>[, not sure why they used mixed Pm/Ps in the description. CSI ? Pm h DEC Private Mode Set (DECSET). [...] Ps = 2 0 0 4 ⇒ Set bracketed paste mode, xterm. [...] This is what gets you the reverse-highlighted text (and no activation if there's a linefeed) when you paste into a terminal window. We could fix up the DMCHAT code to parse the set/reset modes, but the easiest thing (on a mac, with zsh) is to unset zle_bracketed_paste, except there are some subtleties (see below) My ~/.zshrc contains (with a few other things) ssh-add --apple-use-keychain -q export OSTYPE=$OSTYPE [ -v LDESHELL ] && source ~/.zshrc-medley and my ~/.zshrc-medley now contains export PROMPT="%n %*> " zmodload zsh/zle # load it so we can change parameters unset zle_bracketed_paste — Reply to this email directly, view it on GitHub <#2280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJI4IZVBNAD3BXLIEBD3XGKETAVCNFSM6AAAAACF4OLO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJTG42DKNBWGY>. You are receiving this because you were mentioned.

[nbriggs]

You shouldn't be using DSPFONT to change the chat font -- I think setting CHAT.FONT and restarting the CHAT is the only approved way to do it. DSPFONT is likely almost never the right thing to do unless you're working directly on an image stream (not some application's window)

[pamoroso]

When setting CHAT.FONT to Terminal 10 VTCHAT works as expected with respect to the function keys for underscore and circumfles, as well as deleting with Backspace and Del.

[rmkaplan]

If you set CHAT.FONT to, say, Classic 18 before you start (e.g. a fresh release sysout), it's still bad. You type the first character, it looks good. You type the next character on the same line, the first character reappears after a big run of white space, then the second character. And backspace is still garbage.

…

On Oct 12, 2025, at 11:17 AM, Nick Briggs ***@***.***> wrote: nbriggs left a comment (Interlisp/medley#2280) <#2280 (comment)> You shouldn't be using DSPFONT to change the chat font -- I think setting CHAT.FONT and restarting the CHAT is the only approved way to do it. DSPFONT is likely almost never the right thing to do unless you're working directly on an image stream (not some application's window) — Reply to this email directly, view it on GitHub <#2280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJK3BESOAF4I4KGDQKT3XKLLBAVCNFSM6AAAAACF4OLO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJVGE4DANJQHE>. You are receiving this because you were mentioned.

[pamoroso]

However, pressing the _ and ^ keys makes the CHAT window flash with any font under DM2500 and VT100, not display the left and up arrow.

[rmkaplan]

However, pressing the _ and ^ keys makes the CHAT window flash with any font under DM2500 and VT100, not display the left and up arrow.

Yes, that's to be expected, given the new LLKEY assignments. Those keys transmit the MCCS uparrow and leftarrow codes (0,255, 0,254), which the shell doesn't recognize.

As per a previous note, it may be more intuitive, especially for new users, to have those transmit according their labels, as they used to. Users would then have to know that the function keys are available if they want to type those particular arrows (and that they do need to type F-10 when they want the left-arrow in a create expression (unless someone fixes DWIM).

[rmkaplan]

BTW, none of these Chat/font/backspace problems are new: same behavior in the Venue sysout. Also in the Venue sysout, Tedit mode doesn't work by clicking the TeditMode button in the Chat menu. Something probably needed to be done earlier to set up the textstream.

[nbriggs]

The CHAT.FONT must be a monospace font. It's never going to work with Classic. The underlying shell may be using what it knows about the terminal emulator to move the cursor before erasing the character, and it can directly address each character cell on the (typically 24 x 80 character) display. If the emulator didn't put the characters in that grid you're going to get garbage.

[rmkaplan]

OK, then it's only Gacha and Terminal up to size 12 that should work, and they (mostly) do. In playing around, sometimes I still do see the case where the already echoed first-typed character is repeated after some extra white space, before the second typed character appears. Interestingly, the unwanted black pixel when backspacing over an underscore only appears for Terminal 12. Terminal 8 and 10 are OK, as are all the Gacha sizes. Maybe something is wrong with its underscore bitmap?

…

On Oct 12, 2025, at 2:34 PM, Nick Briggs ***@***.***> wrote: nbriggs left a comment (Interlisp/medley#2280) <#2280 (comment)> The CHAT.FONT must be a monospace font. It's never going to work with Classic. The underlying shell may be using what it knows about the terminal emulator to move the cursor before erasing the character, and it can directly address each character cell on the (typically 24 x 80 character) display. If the emulator didn't put the characters in that grid you're going to get garbage. — Reply to this email directly, view it on GitHub <#2280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQSTUJMWUDFTORZFFMO7EVT3XLCP5AVCNFSM6AAAAACF4OLO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJVGM4DGMJZGA>. You are receiving this because you were mentioned.

[nbriggs]

I'd expect TITAN, LETTERGOTHIC, and BOLDPS to work - they're printwheel fonts and I think they're monospace. TITANLEGAL perhaps too. But the character mappings in those may not really be XCCS.

[nbriggs]

Actually, they do look like XCCS encoding at least in FontSample for character set C0. They really don't work well in a CHAT window though, with what looks like mis-mapped characters.