Testing a grammar

Let’s take the example of the pizza.gram grammar again. There are two ways of testing this grammar without actually using it for audio recognition: a given sentence can be tested to see if the grammar accepts it; or random sentences that follow the rules defined by the grammar can be generated. In the first scenario, the tool to be used is grammar-test, which can be invoked as follows:

$ grammar-test pizza.gram pizza vegetariana por favor

If the entered grammar accepts the sentence – in other words, if the root rule of the grammar is able to produce the sentence –, the grammar-test returns 0 and writes “true” (without the quotation marks) on the default output; otherwise, it returns 1 and returns “false” (without the quotation marks) on the default output. A returned value of 2 indicates that an error occurred (for example, the entered grammar contains a error of syntax).

A grammar can be tested to see if it accepts an empty entry (in other words, if the root rule of the grammar can produce $NULL). To do so, “” should be sent as the second argument to the grammar-test tool:

$ grammar-test pizza.gram ""

When the special reference $GARBAGE is used in a rule, the grammar-test tool allows, in the place where the reference occurs, any amount (including zero) of words, regardless of its presence in the grammar lexicon or not. For example, take the following grammar, stored in the file 123.gram.

#ABNF 1.0 utf-8;
root $digitos;
$digitos = $GARBAGE (um | dois | três)<1->;

For this grammar, the grammar-test tool will return “true” for all the following cases.

$ grammar-test 123.gram um dois três
$ grammar-test 123.gram foo bar um um dois
$ grammar-test 123.gram o que você quiser dois

However, the cases below will return “false” as the answer.

$ grammar-test 123.gram um dois três foo
$ grammar-test 123.gram foo um um dois bar
$ grammar-test 123.gram o que você quiser
$ grammar-test 123.gram ""

The tool is also able to read the default input sentence. In this case, invoke it with only one parameter.

$ echo "um dois três" | grammar-test 123.gram

Note

The grammar-test tool only simulates the recognition of a sentence, with no attempt to take the conditions of the audio or parameters of the acoustic model into consideration. This means that is it possible for a user to utter a sentence correctly and, even so, the recognition engine will be unable to produce the desired output due to the background noise in the audio, or because the acoustic model has been poorly configured. Therefore, if the grammar-test tool says the tool should be accepted, but, even so, the recognition process produces a less-than-satisfactory result, those other factors must be taken into account.

Another way to debug a grammar is by generating random sentences that comply with it. To this end, use the grammar-randgen tool. We have provided an example below of how this utility can be invoked, as well as an example output.

$ grammar-randgen --sents=10 pizza.gram

gostaria de uma vegetariana
pizza queijo por gentileza
queijo
queijo por gentileza
queria pizza vegetariana
quero calabresa
quero de queijo por gentileza
quero uma vegetariana por favor
vegetariana por favor
vegetariana por gentileza

By default, the tool will generate only one sentence; this behavior can be changed by means of the option --sents. When more than one sentence is generated, any duplicates are removed from the tool’s output, meaning that the actual number of generated sentences might be less than the required value.