how can i add numbers and some special characters in my alphabet.txt. if we add numbers in alphabet.txt, it is very complex. how can i handle numbers in my large vocab.txt(15 mb).
i was trouble for handling a large vocab.txt to build a language model. thank you sir.
lissyx
((slow to reply) [NOT PROVIDING SUPPORT])
2
I’ve already replied on the other thread for numbers, but for special characters, as you said, you just have to augment alphabet.txt to cover your language’s alphabet. It supports UTF-8 so any language should be fine.
