Thanks for reporting. This looks to working as designed. We use the font-display css property with the value of fallback for these fonts for performance reasons. That could sometimes mean that the loading of a specific font is abandoned if it takes to long to load. It might swap in later, or the browser can simply decide to abandon it altogether.
The reason it then loads on a refresh, is most likely because the font is cached, and thus loads almost instantly.
I think it would be nice if at least similar common fonts were used as the backup font; for instance, it seems obvious that the fallback for Zilla should be Courier or Courier New since they are similar typewriter-style faces. This at least would be a less jarring difference.