@bharath.vadithya Please check with this.
python3 -c "import pandas; print(pandas.read_csv('/home/metlife-vad/DeepSpeech/minigir/train/train.csv')['transcript']);"
@bharath.vadithya Please check with this.
python3 -c "import pandas; print(pandas.read_csv('/home/metlife-vad/DeepSpeech/minigir/train/train.csv')['transcript']);"
thanks alot … its a small mistake
can you do something for this also
+ '[' '!' -f DeepSpeech.py ']'
+ python3 -u DeepSpeech.py --train_files /home/metlife-vad/DeepSpeech/minigir/train/train.csv --dev_files /home/metlife-vad/Deepspeech/minigir/train/miniger-train.csv --test_files /home/metlife-vad/Deepspeech/minigir/train/miniger-train.csv --train_batch_size 48 --dev_batch_size 40 --test_batch_size 40 --n_hidden 1024 --epochs 64 --early_stop True --es_steps 6 --es_mean_th 0.1 --es_std_th 0.1 --dropout_rate 0.30 --learning_rate 0.0005 --report_count 100 --export_dir /metlife-models/ --checkpoint_dir /home/metlife-vad/Deepspeech/metlife-models/check_point --alphabet_config_path /home/metlife-vad/metlife-models/alphabet.txt --lm_binary_path /home/metlife-vad/Deepspeech/metlife-models/lm.binary --lm_trie_path /home/metlife-vad/Deepspeech/metlife-models/trie
Traceback (most recent call last):
File "/home/metlife-vad/DeepSpeech/util/text.py", line 33, in _label_from_string
return self._str_to_label[string]
KeyError: '’'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/metlife-vad/DeepSpeech/util/text.py", line 85, in text_to_char_array
transcript = np.asarray(alphabet.encode(series['transcript']))
File "/home/metlife-vad/DeepSpeech/util/text.py", line 47, in encode
res.append(self._label_from_string(char))
File "/home/metlife-vad/DeepSpeech/util/text.py", line 39, in _label_from_string
).with_traceback(e.__traceback__)
File "/home/metlife-vad/DeepSpeech/util/text.py", line 33, in _label_from_string
return self._str_to_label[string]
KeyError: "ERROR: Your transcripts contain characters (e.g. '’') which do not occur in data/alphabet.txt! Use util/check_characters.py to see what characters are in your [train,dev,test].csv transcripts, and then add all these to data/alphabet.txt."
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DeepSpeech.py", line 931, in <module>
absl.app.run(main)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "DeepSpeech.py", line 915, in main
train()
File "DeepSpeech.py", line 435, in train
train_phase=True)
File "/home/metlife-vad/DeepSpeech/util/feeding.py", line 101, in create_dataset
df['transcript'] = df.apply(text_to_char_array, alphabet=Config.alphabet, result_type='reduce', axis=1)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/frame.py", line 6928, in apply
return op.get_result()
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 186, in get_result
return self.apply_standard()
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 292, in apply_standard
self.apply_series_generator()
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 321, in apply_series_generator
results[i] = self.f(v)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 112, in f
return func(x, *args, **kwds)
File "/home/metlife-vad/DeepSpeech/util/text.py", line 91, in text_to_char_array
raise ValueError('While processing: {}\n{}'.format(series['wav_filename'], e))
ValueError: ('While processing: /home/metlife-vad/DeepSpeech/minigir/wav/tmp.wav\n"ERROR: Your transcripts contain characters (e.g. \'’\') which do not occur in data/alphabet.txt! Use util/check_characters.py to see what characters are in your [train,dev,test].csv transcripts, and then add all these to data/alphabet.txt."', 'occurred at index 0')
Please check your alphabet file
hey @lissyx
my alphabet.txt file
# Each line in this file represents the Unicode codepoint (UTF-8 encoded)
# associated with a numeric label.
# A line that starts with # is a comment. You can escape it with \# if you wish
# to use '#' as a label.
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
r
s
t
u
v
w
x
y
z
# The last (non-comment) line needs to end with a newline.
After checking my transcripts …i found these characters
### The following unique characters were found in your transcripts: ###
[' ', 'b', 'c', 'j', 'h', 't', 'x', 's', 'o', 'r', 'f', 'n', 'm', 'q', 'k', 'g', 'u', 'w', 'p', 'e', 'y', 'z', 'a', 'i', 'l', 'v', 'd']
i got the error as … but there are no extra spaces
+ '[' '!' -f DeepSpeech.py ']'
+ python3 -u DeepSpeech.py --train_files minigir/train/train.csv --dev_files minigir/train/train.csv --test_files minigir/train/train.csv --train_batch_size 48 --dev_batch_size 40 --test_batch_size 40 --n_hidden 1024 --epochs 64 --early_stop True --es_steps 6 --es_mean_th 0.1 --es_std_th 0.1 --dropout_rate 0.30 --learning_rate 0.0005 --report_count 100 --export_dir metlife-models/ --checkpoint_dir metlife-models/check_point --alphabet_config_path metlife-models/alphabet.txt --lm_binary_path metlife-models/lm.binary --lm_trie_path metlife-models/trie
Traceback (most recent call last):
File "/home/metlife-vad/DeepSpeech/util/text.py", line 33, in _label_from_string
return self._str_to_label[string]
KeyError: ' '
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/metlife-vad/DeepSpeech/util/text.py", line 85, in text_to_char_array
transcript = np.asarray(alphabet.encode(series['transcript']))
File "/home/metlife-vad/DeepSpeech/util/text.py", line 47, in encode
res.append(self._label_from_string(char))
File "/home/metlife-vad/DeepSpeech/util/text.py", line 39, in _label_from_string
).with_traceback(e.__traceback__)
File "/home/metlife-vad/DeepSpeech/util/text.py", line 33, in _label_from_string
return self._str_to_label[string]
KeyError: "ERROR: Your transcripts contain characters (e.g. ' ') which do not occur in data/alphabet.txt! Use util/check_characters.py to see what characters are in your [train,dev,test].csv transcripts, and then add all these to data/alphabet.txt."
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DeepSpeech.py", line 931, in <module>
absl.app.run(main)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "DeepSpeech.py", line 915, in main
train()
File "DeepSpeech.py", line 435, in train
train_phase=True)
File "/home/metlife-vad/DeepSpeech/util/feeding.py", line 101, in create_dataset
df['transcript'] = df.apply(text_to_char_array, alphabet=Config.alphabet, result_type='reduce', axis=1)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/frame.py", line 6928, in apply
return op.get_result()
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 186, in get_result
return self.apply_standard()
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 292, in apply_standard
self.apply_series_generator()
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 321, in apply_series_generator
results[i] = self.f(v)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/pandas/core/apply.py", line 112, in f
return func(x, *args, **kwds)
File "/home/metlife-vad/DeepSpeech/util/text.py", line 91, in text_to_char_array
raise ValueError('While processing: {}\n{}'.format(series['wav_filename'], e))
ValueError: ('While processing: /home/metlife-vad/DeepSpeech/minigir/wav/tmp.wav\n"ERROR: Your transcripts contain characters (e.g. \' \') which do not occur in data/alphabet.txt! Use util/check_characters.py to see what characters are in your [train,dev,test].csv transcripts, and then add all these to data/alphabet.txt."', 'occurred at index 0')
Make sure this is not some UTF-8 special space. You can also use util/check_characters.py
to build the alphabet from the dataset.
yeah …solved UTF 8 special space
after i run the script… i got something like this… i searched on internet, but iam unable to figure out it.
I STARTING Optimization
Epoch 0 | Training | Elapsed Time: 0:00:00 | Steps: 0 | Loss: 0.000000
Epoch 0 | Validation | Elapsed Time: 0:00:00 | Steps: 0 | Loss: 0.000000 | Dataset: minigir/train/train.csv
Traceback (most recent call last):
File "DeepSpeech.py", line 931, in <module>
absl.app.run(main)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/home/metlife-vad/.local/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "DeepSpeech.py", line 915, in main
train()
File "DeepSpeech.py", line 642, in train
dev_loss = dev_loss / total_steps
ZeroDivisionError: float division by zero
hey @lissyx … i am just training on three examples will it enough for training (this is just to check weather iam getting results or not) so that i can go further training of more number data.
What’s your training flags / command line ?
i mean three audio files transcripts … the csv file has three audio transcripts
Please, reply to what I asked.
i didn’t understand !!
Well then just say it. I need your python DeepSpeech.py [...]
full command-line
iam running this code
#!/usr/bin/env bash
set -xe
if [ ! -f DeepSpeech.py ]; then
echo "Please make sure you run this from DeepSpeech's top level directory."
exit 1
fi;
python3 -u DeepSpeech.py \
--train_files minigir/train/train.csv \
--dev_files minigir/train/train.csv \
--test_files minigir/train/train.csv \
--train_batch_size 48 \
--dev_batch_size 40 \
--test_batch_size 40 \
--n_hidden 1024 \
--epochs 64 \
--early_stop True \
--es_steps 6 \
--es_mean_th 0.1 \
--es_std_th 0.1 \
--dropout_rate 0 \
--log_level 1 \
--learning_rate 0.000025 \
--report_count 100 \
--export_dir metlife-models/ \
--checkpoint_dir metlife-models/check_point \
--alphabet_config_path metlife-models/alphabet.txt \
--lm_binary_path metlife-models/lm.binary \
--lm_trie_path metlife-models/trie \
"$@"
Ok, if you only have three audio files, please try using batch size not above 3.
okay … i got it …well i will try now
hi @lissyx
Loading model from file metlife-models/output_graph.pb
TensorFlow: v1.13.1-10-g3e0cc53
DeepSpeech: v0.5.1-0-g4b29b78
Warning: reading entire model file into memory. Transform model file into an mmapped graph to reduce heap usage.
2019-11-25 15:24:22.279971: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-11-25 15:24:22.320943: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "CPU"') for unknown op: UnwrapDatasetVariant
2019-11-25 15:24:22.321039: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: WrapDatasetVariant
2019-11-25 15:24:22.321083: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "CPU"') for unknown op: WrapDatasetVariant
2019-11-25 15:24:22.321185: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: UnwrapDatasetVariant
Specified model file version (0) is incompatible with minimum version supported by this client (1). See https://github.com/mozilla/DeepSpeech/#model-compatibility for more information
Traceback (most recent call last):
File "/usr/local/bin/deepspeech", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/deepspeech/client.py", line 88, in main
ds = Model(args.model, N_FEATURES, N_CONTEXT, args.alphabet, BEAM_WIDTH)
File "/usr/local/lib/python3.7/dist-packages/deepspeech/__init__.py", line 23, in __init__
raise RuntimeError("CreateModel failed with error code {}".format(status))
RuntimeError: CreateModel failed with error code 8195
i searched whole internet i didn’t find the solution for this and iam using correct deepspeech version
You have the error here … You need to share more details, but it looks like you exported wrongly.
sudo deepspeech --model metlife-models/output_graph.pb --alphabet metlife-models/alphabet.txt --lm metlife-models/lm.binary --trie metlife-models/trie --audio minigir/wav/tmp2.wav
Loading model from file metlife-models/output_graph.pb
TensorFlow: v1.13.1-10-g3e0cc53
DeepSpeech: v0.5.1-0-g4b29b78
Warning: reading entire model file into memory. Transform model file into an mmapped graph to reduce heap usage.
2019-11-25 15:24:22.279971: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-11-25 15:24:22.320943: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "CPU"') for unknown op: UnwrapDatasetVariant
2019-11-25 15:24:22.321039: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: WrapDatasetVariant
2019-11-25 15:24:22.321083: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "WrapDatasetVariant" device_type: "CPU"') for unknown op: WrapDatasetVariant
2019-11-25 15:24:22.321185: E tensorflow/core/framework/op_kernel.cc:1325] OpKernel ('op: "UnwrapDatasetVariant" device_type: "GPU" host_memory_arg: "input_handle" host_memory_arg: "output_handle"') for unknown op: UnwrapDatasetVariant
Specified model file version (0) is incompatible with minimum version supported by this client (1). See https://github.com/mozilla/DeepSpeech/#model-compatibility for more information
Traceback (most recent call last):
File "/usr/local/bin/deepspeech", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.7/dist-packages/deepspeech/client.py", line 88, in main
ds = Model(args.model, N_FEATURES, N_CONTEXT, args.alphabet, BEAM_WIDTH)
File "/usr/local/lib/python3.7/dist-packages/deepspeech/__init__.py", line 23, in __init__
raise RuntimeError("CreateModel failed with error code {}".format(status))
RuntimeError: CreateModel failed with error code 8195
what is model compatability and error code 8195. still did’t understand