All Toolbar Icon Pop-ups stay open until they lose focus which means you click outside it (e.g. on the page).
It is possible to make it to automatically close after each action but this way multiple Find/Replace can be done without the need to reopen the pop-up each time.
The popup for doing the search/replace goes away, but if there is an error in the search (text is not found), THAT popup is not going away when focus is lost, it stays up for about a minute. That is the popup I meant, sorry if I was not clear. Thanks!
I see… You mean the notification. How long it stays open is set by Firefox. There is a an X to close it if you want.
It is possible for the add-on to force close it but then how long should it stay open? People might have different ideas.
However, if there is a popular demand, I can write the code for it.