I'd be surprised if you could really tell.

One of the best things you can buy if you get started down this rabbit hole is a switcher. That way you switch sources or amps very quick.

I learned how little dacs and amps actually do (tube amps not withstanding)

I put a song or section of song I know very well on repeat and listen switching back and forth. And I really don't hear anything really different ... I guess I'm just not audiophile enough.