What is the best way to parse shell-like syntax without a lexer?
Category&#x2F;autonoma&#x2F;lambda theory &amp; discrete math provide human terminology for this.
To keep things simple, intermidiate data storage is not directly discussed (aka stack&#x2F;heap)
For this artile, intermediate data storage just magically happens for this article.
grammar referenced: <a href="https:&#x2F;&#x2F;pubs.opengroup.org&#x2F;onlinepubs&#x2F;9699919799.2018edition&#x2F;utilities&#x2F;V3_chap02.html#tag_18_10" rel="nofollow">https:&#x2F;&#x2F;pubs.opengroup.org&#x2F;onlinepubs&#x2F;9699919799.2018edition...</a><p>Going to try to keep things &quot;abstract&quot;, as parse through tokens.<p>Each grammer to the right of the &quot;:&quot;, &quot;|&quot; or &quot;=&quot; denotes how to arrive at
a single token, the value to the left of &quot;:&quot;,&quot;|&quot; or &quot;=&quot;.
This is commonly refered to as the &quot;l-value&quot;.<p>example grammar statement, program is the l-value.
program : linebreak<p>need to figure out the smallest group of tokens needed to get the l-value of a grammer
Note:  Grammer lays out order to fetch&#x2F;perform operations to arrive at single token, for
a given grammer statement  (withing a given grammer function)
(aka (a + b) -&gt; get a value, get b value, get (a + b) value<p>terminal grammer l-value : token that can not be reduced further, terminal token.
non-terminal l-value     : set of one or more l-values and how to get a single l-value out of this grammer instruction.<p>One throwback, somewhat less abstract way to visualize this is using &quot;punch cards&quot;
(maps to function calls, returns only one token&#x2F;value, &quot;the l-value&quot;).<p>terminal punch cards hold terminal l-value token (character a, numberic value, literal character word)
Keeping things simple, non-terminal punch cards hold one combination of tokens (reducible&#x2F;non-reduceable) with what to do with the token(s) to get a single token&#x2F;value to return.
So, tokens wind up being references to other &quot;punch cards&quot; (terminal -&gt; return card value, non-terminal -&gt; go do other &quot;punch cards&quot;, combine the values of other punch cards to determine the &quot;l-value&quot;)<p>So, structuring the order to parse the tokens can be done by:<p>a) assign numeric values to each of the grammer l-values (translate r-value tokens into l-value reference)
   when read in &quot;shell script&quot; changes each token&#x2F;grammer l-value to corresponding shell grammer reference.
   identified group(s)&#x2F;sets of token values are associated with a &quot;punch card&quot; index (array reference)
   when done, run through &quot;punch card&quot; index starting from
   each array index &quot;calls&quot;  the associated &quot;punch card&quot; to process things.
   note: token&#x2F;value returned from &quot;punch card&quot; is combined with result from next array element.
         when reach end of array, should have just one &quot;token&quot; value!<p>b) variation on this is &quot;dispatch table&quot; aka switch statement, where the &quot;punch card&quot; return value
   determines which switch statement to do next.<p>c) combine d &amp; c via calls to functional array.
   function in function array to be called based on &quot;punch card&quot; return value.<p>d) order &quot;punch cards&quot; as an abstract syntax tree (which if completely a math expression, change parse order to get infix, postfix, suffiex printouts) hashmap to call function in function array.<p>no need to worry about exceeding array length, &quot;array index&quot; is implicit in parse, don&#x27;t have to worry about &quot;growing&quot; array to insert token(s) within array.<p>e) recursive decent parser  (each &quot;punch card&quot; is a function)