Why don’t you use the LJ Speech dataset instead. It is clean to start with and you can add noise as you please.
https://keithito.com/LJ-Speech-Dataset/