How could "Meeting at Night" be considered as a love poem?

The poem appears to be written from the perspective of one lover on their way to reach their beloved.  The first stanza describes the beginning of the journey this lover must make to reach their beloved and have this "Meeting at Night": he or she travels some distance in a boat.  The speaker describes the colors of the sea and land at night as well as the waves rippling away from the boat and seeming to reflect the "yellow half-moon" because they seem "fiery."  It is notable that the speaker identifies the waves' color with fire because fire often symbolizes passion, which is what such a lover would feel. 

In the second stanza, the speaker reaches the beach and must travel by land now, before they can "tap" quietly at the window pane of their beloved. Then, the person inside the house strikes a match, perhaps, to light a candle -- as there seems to be some need for secrecy (perhaps they are unmarried, or perhaps their relationship is otherwise forbidden -- we don't know for sure) -- and, again, the reference to fire could be symbolic of passion.  Then, the last two lines are where we find the most evidence for this interpretation: "And a voice less loud, thro' its joys and fears, / Than the two hearts beating each to each."  Thus, they whisper, joyful, perhaps, at the sight of the lover and fearful of being caught.  These whispering voices are softer than the sound of the two hearts beating for one another.  The pounding of their hearts lets us know that strong emotion is involved in this scene.  Then, the great trouble the speaker has gone to in order to reach this person, as well as the earlier references to fire, lend themselves to the interpretation that this "Meeting at Night" is a poem about two lovers.