Decode error

DeepSpeech
#9

[ 8.674325] RAPL PMU: hw unit of domain pp1-gpu 2^-0 Joules GPU

13 gb ram

2000 max clock speed

#10

Hey Everytime i start the training it starts with different epoch number.

I’m posting the text files the training part and the inferences for twodifferent datasets with 2 checkpoint folders.

Can i know why such behavior is everything correct or am i doing something wrong.traininontest2.zip (1.6 KB)

This zip file has 2 text files where you can see the epoch starting number.

My question is deep speech was already trained for certain epochs .so, if i start the training it was supposed to start from there. But why different epoch every time i start the training.

#11

Every time i run the model with different data sets the epochs number displayed are different.

This is expected. If you’re retraining or fine tuning, just use negative epoch numbers: if you want to train for an additional 20 epochs, pass --epoch -20.

#12

Sorry, but if you keep not replying to what is required so we can help you, it’s not going to be going anywhere. This is not documenting the operating system, nor the CPU and others critical components of your system.

1 Like
#13

Hey sorry for that.I Thought you are asking for system config

OS is “Debian GNU/Linux 9.5 (stretch) (GNU/Linux 4.9.0-8-amd64 x86_64\n)”
Intel Haswell series cpu with one NVIDIA Tesla P100

Thanks
Sagar

#14

@reuben
I’m using “epoch -3” and it still starts at different epoch numbers. You can see the command in the text file i used --epoch -3(negative 3).

Thanks
Sagar

#15

Right, that’s expected, just ignore the absolute epoch numbers.

#16

Hello,
@reuben @lissyx
I did the training with my data the overall data is only for 33 minutes.(train part)
The output that i got was not satisfactory.
I have a doubt here if i add up the data my data with the common-voice data and train the model completely from starting will it show better results.

Is it worthwhile to do this ? Please suggest

Assuming the train data that Deep speech used is common cv (correct me if i’m wrong)

Thanks
Sagar

#17

So we definitively need that you share the output of pip install --verbose when it fails, because it is supposed to work on that system.

#18

Hey this is the output.

pip install --verbose
Created temporary directory: /tmp/pip-ephem-wheel-cache-1lb9ze7_
Created temporary directory: /tmp/pip-req-tracker-1vhdpi6n
Created requirements tracker ‘/tmp/pip-req-tracker-1vhdpi6n’
Created temporary directory: /tmp/pip-install-ce9ji0no
Cleaning up…
Removed build tracker ‘/tmp/pip-req-tracker-1vhdpi6n’
ERROR: You must give at least one requirement to install (see “pip help install”)
Exception information:
Traceback (most recent call last):
File “/home/SAGAR/anaconda3/envs/deep-env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py”, line 143, in main
status = self.run(options, args)
File “/home/SAGAR/anaconda3/envs/deep-env/lib/python3.6/site-packages/pip/_internal/commands/install.py”, line 293, in run
self.name, wheel_cache
File “/home/SAGAR/anaconda3/envs/deep-env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py”, line 252, in populate_requirement_set
‘(see “pip help %(name)s”)’ % opts)
pip._internal.exceptions.CommandError: You must give at least one requirement to install (see “pip help install”)
1 location(s) to search for versions of pip:

Thanks
Sagar

#19

Thanks, but it’s incomplete and improperly formatted, so unusable. Please make sure you use code formatting for console output. You also don’t specify any python module to install … You should be doing the same as when you got the error ds_ctcdecoder-0.4.1-cp36-cp36m-manylinux1_x86_64.whl is not a supported wheel on this platform.:

pip3 install --verbose $(python3 util/taskcluster.py --decoder)
#20

@lissyx Thank you
decoder is installed.

#21

Yet you had an error that should not have happened … It would be helpful to others that we know why.

1 Like
#22

@lissyx Yeah the problem is i didn’t had the folder permission where i’m installing.so, i used sudo to install.

when i’m using sudo it is using python 2.7 so the error

Can you please also comment on the another question that i posted today in the same thread.

Thanks Sagar

#23

It would have been nice to document it earlier.

Well, I don’t understand that question.

#24

The question is if i add my data with the original corpus that you are using to train Deepspeech. and train the model from scratch will it show any progress. as, of now the results were not good when using with Pretrained model.

Thanks

#25

It all depends on a lot of parameters … How much data do you add, what are your expectations, what is exactly the current outcome …

#26

@lissyx
Right now i got WER 0.356 so it’s like 75 percent accuracy right. But the thing is it is completely messing up with some sentences.
i want to achieve with 6-8 percent word error rate.

How much does it take if i’m adding up it to the original corpus and train from scratch.?
How much does it take if using pretrained model.?

When i got 0.356 WER i used v0.4.1 pretrained model and trained with my data which has 900 rows in train file sums up to 33 minutes.

Thanks
Sagar

#27

On what ? Your new data ?

#28

It depends on your data … But in both case, 33 minutes is way likely not enough. Remember, pretrained model ingested something like 5000 hours.