Is it possible to customize load more button caption?

Yes, all labels of the front end interface of the gallery is customizable. Label of the load more button could be edit in admin section