How can i add number and some special characters in my alphabet.txt? how can i handle numbers in my large vocab.txt(15 mb)

how can i add numbers and some special characters in my alphabet.txt. if we add numbers in alphabet.txt, it is very complex. how can i handle numbers in my large vocab.txt(15 mb).

i was trouble for handling a large vocab.txt to build a language model. thank you sir.

I’ve already replied on the other thread for numbers, but for special characters, as you said, you just have to augment alphabet.txt to cover your language’s alphabet. It supports UTF-8 so any language should be fine.

Mm thank you sir.:slightly_smiling_face: