With taxable accounts, it is legally your assets, gains or not. You pay taxes when you sell. Just like with a traditional, non-Roth IRA: you pay taxes when you withdraw. The IRS makes the rules.
The IRS doesn't make the rules, but enforces them.
Unlike a somewhat popular opinion on this website, the law isn't an algorithm, but a codification of societal norms, like that we all share the expanses of living in civilized society.
Yes, the TRANSFER of assets happens at time of sell or withdrawal, but the share taxes was painted such when we (the people, through legislation. Not the IRS!) decided that such share should be dedicated to society.
This isn't about letter of the law, and never was: it's being an asshole.
That doesn't even make sense. The "share" is not known until the assets are sold or withdrawn, and also subject to change based on whatever tax law happens to be at the time. If I hold a stock long term, instead of day trading and accruing short term gains and associated taxes, am I being an asshole?