Hi,
i still getting res “” blank after continuing from pretrained model.
yk@andromeda:~/DeepSpeech$ python3 DeepSpeech.py --n_hidden 2048 --checkpoint_dir ./checkpoint_old --epochs 3 --train_files data/train/train.csv --dev_files data/dev/dev.csv --test_files data/test/test.csv --learning_rate 0.0001
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py:494: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
options available in V2.
- tf.py_function takes a python function which manipulates tf eager
tensors instead of numpy arrays. It’s easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means tf.py_function
s can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
- tf.numpy_function maintains the semantics of the deprecated tf.py_func
(it is not differentiable, and manipulates numpy arrays). It drops the
stateful argument making all functions stateful.
W1007 19:19:11.504933 140293868754752 deprecation.py:323] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/dataset_ops.py:494: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
options available in V2.
- tf.py_function takes a python function which manipulates tf eager
tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
an ndarray (just call tensor.numpy()) but having access to eager tensors
means `tf.py_function`s can use accelerators such as GPUs as well as
being differentiable using a gradient tape.
- tf.numpy_function maintains the semantics of the deprecated tf.py_func
(it is not differentiable, and manipulates numpy arrays). It drops the
stateful argument making all functions stateful.
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py:348: Iterator.output_types (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_types(iterator)`.
W1007 19:19:11.572091 140293868754752 deprecation.py:323] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py:348: Iterator.output_types (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_types(iterator)`.
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py:349: Iterator.output_shapes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_shapes(iterator)`.
W1007 19:19:11.572299 140293868754752 deprecation.py:323] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py:349: Iterator.output_shapes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_shapes(iterator)`.
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py:351: Iterator.output_classes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_classes(iterator)`.
W1007 19:19:11.572416 140293868754752 deprecation.py:323] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/data/ops/iterator_ops.py:351: Iterator.output_classes (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.data.get_output_classes(iterator)`.
WARNING:tensorflow:
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
* https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
* https://github.com/tensorflow/addons
* https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.
W1007 19:19:12.182626 140293868754752 lazy_loader.py:50]
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
* https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
* https://github.com/tensorflow/addons
* https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W1007 19:19:12.184339 140293868754752 deprecation.py:506] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:Entity <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f986841fcc0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f986841fcc0>>: AttributeError: module 'gast' has no attribute 'Num'
W1007 19:19:12.208325 140293868754752 ag_logging.py:145] Entity <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f986841fcc0>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f986841fcc0>>: AttributeError: module 'gast' has no attribute 'Num'
WARNING:tensorflow:From DeepSpeech.py:232: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1007 19:19:12.278438 140293868754752 deprecation.py:323] From DeepSpeech.py:232: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
I Initializing variables...
I STARTING Optimization
Epoch 0 | Training | Elapsed Time: 0:00:42 | Steps: 2 | Loss: 327.344215
Epoch 0 | Training | Elapsed Time: 0:01:27 | Steps: 4 | Loss: 279.011799
Epoch 0 | Training | Elapsed Time: 0:02:18 | Steps: 6 | Loss: 384.225965
Epoch 0 | Training | Elapsed Time: 0:06:20 | Steps: 14 | Loss: 273.964598
Epoch 0 | Training | Elapsed Time: 0:06:58 | Steps: 15 | Loss: 263.730356
Epoch 0 | Training | Elapsed Time: 0:11:29 | Steps: 21 | Loss: 239.360187
Epoch 0 | Training | Elapsed Time: 0:12:25 | Steps: 22 | Loss: 238.142157
Epoch 0 | Training | Elapsed Time: 0:14:55 | Steps: 24 | Loss: 251.470535
Epoch 0 | Validation | Elapsed Time: 0:00:14 | Steps: 10 | Loss: 105.280766 | Dataset: data/dev/dev.csv
I Saved new best validating model with loss 105.280766 to: ./checkpoint_old/best_dev-24
Epoch 1 | Training | Elapsed Time: 0:00:19 | Steps: 1 | Loss: 72.282936
Epoch 1 | Training | Elapsed Time: 0:01:28 | Steps: 4 | Loss: 88.395905
Epoch 1 | Training | Elapsed Time: 0:01:51 | Steps: 5 | Loss: 86.035843
Epoch 1 | Training | Elapsed Time: 0:05:41 | Steps: 13 | Loss: 105.126752
Epoch 1 | Training | Elapsed Time: 0:08:16 | Steps: 17 | Loss: 114.653208
Epoch 1 | Training | Elapsed Time: 0:14:52 | Steps: 24 | Loss: 147.739715
Epoch 1 | Validation | Elapsed Time: 0:00:14 | Steps: 10 | Loss: 100.421697 | Dataset: data/dev/dev.csv
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py:960: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
W1007 19:49:33.111889 140293868754752 deprecation.py:323] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py:960: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to delete files with this prefix.
I Saved new best validating model with loss 100.421697 to: ./checkpoint_old/best_dev-48
Epoch 2 | Training | Elapsed Time: 0:05:04 | Steps: 12 | Loss: 98.081915
Epoch 2 | Training | Elapsed Time: 0:05:41 | Steps: 13 | Loss: 98.861042
Epoch 2 | Training | Elapsed Time: 0:06:58 | Steps: 15 | Loss: 104.322818
Epoch 2 | Training | Elapsed Time: 0:07:37 | Steps: 16 | Loss: 106.783278
Epoch 2 | Training | Elapsed Time: 0:14:55 | Steps: 24 | Loss: 143.179845
Epoch 2 | Validation | Elapsed Time: 0:00:14 | Steps: 10 | Loss: 99.873046 | Dataset: data/dev/dev.csv
I Saved new best validating model with loss 99.873046 to: ./checkpoint_old/best_dev-72
I FINISHED optimization in 0:45:29.704290
WARNING:tensorflow:Entity <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f97d2f48a58>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f97d2f48a58>>: AttributeError: module 'gast' has no attribute 'Num'
W1007 20:04:43.839351 140293868754752 ag_logging.py:145] Entity <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f97d2f48a58>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method LSTMBlockWrapper.call of <tensorflow.contrib.rnn.python.ops.lstm_ops.LSTMBlockFusedCell object at 0x7f97d2f48a58>>: AttributeError: module 'gast' has no attribute 'Num'
WARNING:tensorflow:From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
W1007 20:04:43.909140 140293868754752 deprecation.py:323] From /home/yk/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
INFO:tensorflow:Restoring parameters from ./checkpoint_old/best_dev-72
I1007 20:04:43.909901 140293868754752 saver.py:1280] Restoring parameters from ./checkpoint_old/best_dev-72
I Restored variables from best validation checkpoint at ./checkpoint_old/best_dev-72, step 72
Testing model on data/test/test.csv
Test epoch | Steps: 7 | Elapsed Time: 0:00:08
Test on data/test/test.csv - WER: 1.000000, CER: 1.000000, loss: 34.051395
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 25.776178
- wav: file:///home/yk/DeepSpeech/data/test/046.wav
- src: "o uu"
- res: ""
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 29.066339
- wav: file:///home/yk/DeepSpeech/data/test/047.wav
- src: "o p "
- res: ""
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 31.200336
- wav: file:///home/yk/DeepSpeech/data/test/041.wav
- src: "oimmyy"
- res: ""
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 35.002659
- wav: file:///home/yk/DeepSpeech/data/test/044.wav
- src: "syuui"
- res: ""
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 36.742115
- wav: file:///home/yk/DeepSpeech/data/test/045.wav
- src: "ohndieya"
- res: ""
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 37.746620
- wav: file:///home/yk/DeepSpeech/data/test/043.wav
- src: "o rrhof i"
- res: ""
--------------------------------------------------------------------------------
WER: 1.000000, CER: 1.000000, loss: 42.825520
- wav: file:///home/yk/DeepSpeech/data/test/042.wav
- src: "pfnry nya"
- res: ""
--------------------------------------------------------------------------------