feat(developer): Discussion on PUA and hacked keyboards

[LornaSIL]

@dyacob had a suggestion that we should maybe recommend PUA keyboards to have the ID with the end of the ID be pua. For example, gff_sbs (which is PUA-encoded) should maybe have been reserved for when the script gets into Unicode. The PUA-encoded keyboard should maybe have been called gff_sbs_pua and then when the script gets added to Unicode it would change back to the ID without _pua. It's too late for this keyboard, but I like that idea.

Then, I've heard complaints about Keyman (me?) not accepting hacked encoded keyboards. It makes it difficult to use a RTL script when we don't let them use Arabic codepoints. I'm still opposed, but there are some who think we should allow it. If we do decide to allow it, I would suggest:

• Adding another stream called custom or hacked
• Allow people to reuse existing Unicode codepoints (for example Arabic codepoints)
• Any keyboard in this stream would have a warning at the top of the keyboard page saying "This keyboard is using hacked codepoints and will only work with the included font"

[LornaSIL]

We may also want to discuss another category for 'non-language' keyboards. Novelty? Symbolic?

[DavidLRowe]

I'm not opposed to the term "hacked", but I think it would need some explanation, lest people think of "hackers" doing illegal things. Maybe expand your warning: "This keyboard is using hacked (non-standard) codepoints and will only work with the included font since it doesn't follow the Unicode standard."

[mcdurdin]

Agree that 'hack' is a loaded term and maybe using 'reserved codepoints' would be better.

@srl295 thoughts?

[LornaSIL]

'reserved codepoints'

They aren't "reserved", they are encoded as something else. So, we need a better term that conveys actual meaning.

[LornaSIL]

reused?

[mcdurdin]

ah, sorry, I misunderstood -- these are encoded codepoints that are being used out-of-spec.

It's an interesting question because we all use characters such as hyphen incorrectly, just like I did in the paragraph above. At what point do we call it "wrong" and "don't do this"?

Some possible boundaries for discussion:

• When the keyboard it doesn't work with widely-distributed fonts
• Glyphs that don't match the expected Unicode chart glyphs
• Use of deprecated codepoints such as Arabic presentation forms
• Use of reserved codepoints
• Use of PUA
• Mixing of scripts
• Directionality or shaping hacks

I am uncertain about encoding this into the keyboard filename -- we have tried to get away from having semantic meaning in keyboard filenames and I think that has been a good thing overall -- better than our situation with lexical models, where we should have also done the same thing (a separate discussion!)

Categories or tags might be the best approach. It's fuzzy, but allows us to use multiple tags.

[srl295]

Maybe pua should be a subdirectory. There's experimental, but this goes beyond experimental.

Let's say if there were a section that was CSUR compliant.

Maybe alternate-encoded is a good term.

[mcdurdin]

Maybe pua should be a subdirectory.

I'm leaning towards tags: release, experimental, legacy, alternate-encoding, conlang (or conscript?), pan-language?, pua, novelty etc. Some of these could be automatically added by the compiler. There's potential overlap with BCP 47 but not a 1:1 match, and I would be hesitant to try and overload BCP 47 because our language tags already bear a heavy load with their current purpose (just as I am hesitant to overload keyboard ids).

[mcdurdin]

Can I shift this discussion into the Keyman repo? It's going to need changes to Keyman Developer in order to work (and probably also to websites and deployment)