Would I be sensible to just let this potential one go, even though I'm a little upset?

I think you're over thinking things and are too focus on a long term idea. He's only 21 and probably not ready for a long term marriage type commitment. If you like being around him, then be around him, see what happens. But if you already know that you're looking for something more and lasting don't get too serious.