I hardly know what to write about the article at this link, so just go read it.
(I think my favorite of the hymns listed is the fourth of the suggested recessional hymns.)
Ir's unlikely that your church celebrated May Fourth in this way, but apparently there were a few...
