Encasing the circuit board in epoxy. It makes it very difficult to access components without destroying it. It's also great for water proofing and increasing the mechanical robustness.
There is potting compound with high thermal conductivity for things that produce a lot of heat. A YubiKey hardly uses any power, so heat should not be an issue.
The main downsides of potting are that it makes repair practically impossible and it can add a lot of weight if there is a large volume to be filled.
Potting Grrrr. My fancy track lighting has been potted. It sucks because absolutely no place (even China) sells the 48v LED driver with the odd body shape to bypass the internal mounting screws, and the potting means I can't access the board to desolder a resostor or something
Physical anti-tamper, while important for this type of device, wouldn't have helped for this particular attack. It's an electromagnetic side channel, so they don't even have to be touching the the thing to collect data.