Add specialChars in textNoTagsToken, alphaNumToken

This will ensure that alphaNumToken does not lose any of
the previously used tokens in its definition. The same
tokens were added to textNoTagsToken explicitly, because it used to
have alphaNumToken in its definition before I removed it.

Previously, textNoTagsToken and alphaNumToken had many tokens in
common in their definition. To avoid grammar conflicts, the
alphaNumStatement grammar was created. However, I found this
unintuitive and was an extra step just to avoid repetition in
the definitions.

I opted to have repetition in the definitions of textNoTagsToken
and alphaNumToken and it be explicitly clear right away, rather than
have extra grammar statements like alphaNumStatement which don't look
like they do anything at first glance
This commit is contained in:
Ibrahim Wassouf
2023-08-03 21:05:41 -03:00
parent 850513fa3d
commit c9db0ee749

View File

@@ -576,11 +576,11 @@ idStringToken : NUM | NODE_STRING | DOWN | MINUS | DEFAULT | COMMA | COLON | A
textToken : TEXT | TAGSTART | TAGEND | UNICODE_TEXT;
textNoTagsToken: NUM | NODE_STRING | SPACE | MINUS | keywords | START_LINK ;
textNoTagsToken: NUM | NODE_STRING | SPACE | MINUS | AMP | UNICODE_TEXT | COLON | MULT | BRKT | keywords | START_LINK ;
edgeTextToken : EDGE_TEXT | UNICODE_TEXT ;
alphaNumToken : NUM | UNICODE_TEXT | NODE_STRING | DIR | DOWN | MINUS | COMMA;
alphaNumToken : NUM | UNICODE_TEXT | NODE_STRING | DIR | DOWN | MINUS | COMMA | COLON | AMP | BRKT | MULT;
idString
:idStringToken