Differences between keyword, reserved word, and builtin?

Question

From Make bash use external `time` command rather than shell built-in, Stéphane Chazelas wrote:

There is no time bash builtin. time is a keyword so you can do for instance time { foo; bar; }

We can verify it:

$ type -a time
time is a shell keyword
time is /usr/bin/time

It doesn't show that time can be a built-in command.

1. What is the definition of a "keyword"?

2. Is "keyword" the same concept as "reserved word" in the Bash Reference Manual?

   reserved word

   A word that has a special meaning to the shell. Most reserved words introduce shell flow control constructs, such as for and while.

3. Is a keyword necessarily not a command (or not a built-in command)?

   As a keyword, is time not a command (or not a built-in command)?

   Based on the definitions of keyword and of built-in, why is time not a built-in but a keyword?

4. Why "you can do for instance time { foo; bar; }" because "time is a keyword"?

Answer 1

Keywords, reserved words, and builtins are all the "first word" of a simple command. They can be placed in two groups: Keywords and builtins. The two are mutually exclusive. A word (token) can be either a keyword or a builtin, but not both.

Why the "first word"

From the POSIX definition of "simple command" (emphasis mine):

A "simple command" is a sequence of optional variable assignments and redirections, in any sequence, optionally followed by words and redirections, terminated by a control operator.

2.- The words that are not variable assignments or redirections shall be expanded. If any fields remain following their expansion, the first field shall be considered the command name and remaining fields are the arguments for the command.

After that "first word" has been identified, and after is has been expanded (by an alias, for example) the final word is "the command", there could be only one command in each line. That command word could be a builtin or a keyword.

Keyword

Yes, a keyword is a "reserved word". Load "man bash" and search for keyword or just execute this command: LESS=+/'keyword' man bash.

The first hit on search says this:

keyword Shell reserved words.

It happens in the completion section, but is quite clear.

Reserved words

In POSIX, there is this definition of "reserved words" and some description of what reserved words do.

But the Bash manual has a better working definition. Search for "RESERVED WORDS" (LESS=+/'RESERVED WORDS' man bash) and find this:

RESERVED WORDS Reserved words are words that have a special meaning to the shell. The following words are recognized as reserved when unquoted and either the first word of a simple command or the third word of a case or for command:

! case do done elif else esac fi for function if in select then until while { } time [[ ]]

Builtin

It is not defined in the Bash manual, but it is quite simple:

It is a command that has been implemented inside the shell for essential needs of the shell (cd, pwd, eval), or speed in general or to avoid conflicting interpretations of external utilities in some cases.

Time is a keyword

why is time not a builtin but a keyword?

To allow the existence of a command as the second word.

It is similar as how an if ... then .... fi allow the inclusion of commands (even compound commands) after the first keyword if. Or while or case, etc.

Answer 2

Q1, Q2

What's the definition of a keyword?

Is "keyword" the same concept as "reserved word" in Bash Reference Manual?

Basically, special words important to the syntactic structure. In C there are goto, if, while, enum and so on; in bash you have if, while (these sound familiar..), time, etc.

And yes, they are the same.

I am taking some liberty in interpreting the question, since the basic syntactic elements of POSIX shell and bash are similar. So let's look at the definition in POSIX.1:2013 Shell Command Language:

2.4 Reserved Words

Reserved words are words that have special meaning to the shell; see Shell Commands. The following words shall be recognized as reserved words:

...

This recognition shall only occur when none of the characters is quoted and when the word is used as:

...

See the grammar in Shell Grammar.

Let's take a look at the POSIX grammar to see how the Special Words—now syntatic tokens after lexing—in action:

for_clause       : For name linebreak                            do_group
                 | For name linebreak in          sequential_sep do_group
                 | For name linebreak in wordlist sequential_sep do_group
/* ... */
do_group         : Do compound_list Done           /* Apply rule 6 */

That looks familiar, right? Note that For, Do and Done are really tokens that are supposed to be mapped and recognized by the lexer:

%token  If    Then    Else    Elif    Fi    Do    Done
/*      'if'  'then'  'else'  'elif'  'fi'  'do'  'done'   */


%token  Case    Esac    While    Until    For
/*      'case'  'esac'  'while'  'until'  'for'   */

/* and 'In' too -- They made a mistake! */
%token  In    /*      'in'   */

If you have heard about Yacc or Bison (or well, jison), that's how people might use the grammar. With these parsers generators, they can generate something that figures out what parts of the grammar a given flow of input 'tokens' are spoken in.

Q3

Is a keyword necessarily not a command (or not a builtin command)?

As a keyword, is time not a command (or not a builtin command)?

None of the keywords are treated like commands. But sure, you can have commands/functions with the same name, e.g.:

# make a command to avoid command-not-found for `FOO=BAR time cmd`
time(){ time "$@"; }

Based on the definitions of keyword and of builtin, why is time not a builtin but a keyword?

Because that's how people decided to make it:

// Licensed under GPLv2, bash:parse.y
// Copyright (C) 1989-2012 Free Software Foundation, Inc.
pipeline_command: pipeline
                | BANG pipeline_command
                | timespec pipeline_command
                | timespec list_terminator
                | BANG list_terminator;
pipeline        : pipeline '|' newline_list pipeline
                | pipeline BAR_AND newline_list pipeline
                | command;
timespec        : TIME | TIME TIMEOPT | TIME TIMEOPT TIMEIGN;

And they get extra power from this (see next question).

Q4

Why "you can do for instance time { foo; bar; }" because "time is a keyword"?

As a part of the grammar, people can naturally let the parser handle everything, and make decisions like allowing time before compound commands. If time was implemented as just a command, you will get a syntax error for constructs like such (try echo { foo; bar; }), since it's really parsed with the 'usual' rules.

Also think about [[. If [[ have not been a keyword, constructs like [[ ($a == 2) || ($b != 3) ]] will make the shell find stray parentheses and complain. (Replace [[ with [ and see).

P.S. time is a utility instead of a keyword in POSIX, although the latter is still considered acceptable. The whole time-a-trunk-of-commands thing is a ksh and bash extension.