OpenGrm NGram Library Suggestions

1. ngramapply takes --verbose, but -v=n (e.g. =v=1) is already provided (BER: fixed to use --v)
2. ngramapply appears to be more of a debugging tool than than applicator given its human-readable type output. Perhaps a name that suggests or at least a comment in the quick tour that this is the case and that 'fstintersect' (possiby with a phi matcher) is the heavy-duty way to do this.
3. Improve memory usage of ngramread - uses 14 gb for 1.5gb input
4. allow unk words to contribute to perplexity measure
5. random test suite
6. parameter for unknown word prob
7. normalize based on prefix count, no additional smoothing
8. counting from cyclic
9. giving counts to strings for counting

-- MichaelRiley - 05 Nov 2010