Fragmental parsing

Sometimes you may have troubles parsing a text, that holds context dependent tokens. For example, you can't write a context free grammar for parsing java files and their included javadoc comments, because the syntax of the javadoc comments is completly different from the java grammar.

To solve the problem, here's a technique that I called 'Fragmental parsing'. To solve the problem you create two grammars, one for the java and one for the javadoc comments. In the java grammar, define a token for the complete javadoc comment. When parsing the java files, mark the token as text, and parse the output a second time using the javadoc grammar. With this technique you can parse most problematic cases.