I checked my carbonation caps for seal, I noticed little pieces of rubber sitting on the seal.
As it turns out, the seal sits on the very outside edge of the opening, if you tighten the caps beyond hand tight the seal slips inside the opening and looses seal.
See photos below.
Maybe the plastic varieties of the carbonation caps are the better choice, I believe they don't require a seal.
Yes that is correct. The flat washer inside the carbonation caps can easily be pushed out of the seat if they are over tightened. Especially as these have always been made with a fairly soft seal. If you ever wanted to get some spare seals this cam-lock seal is the correct size for the job:
https://www.kegland.com.au/5-pack-camlock-replacement-silicone-seal.html
Basically it comes down to simply making sure you only tighten these up finger tight.
As you have suggested the plastic carbonation caps that you can see here:
https://www.kegland.com.au/plastic-carbonation-line-cleaning-cap-fermzilla-cap.html
These are excellent and are made from a high grade of engineering plastic. They have no seal like the stainless carbonation caps so even when done up very tight they have no seal to roll out of place or drop out and loose. Even though they are significantly cheaper I actually prefer to use these. The plastic in these is durable, chemical resistant, UV resistant. When purchased in a pack of 10 we give a significant discount on these so they are only about $2 each so cheap enough that if you use them on a coke bottle and take it to a party then it's not a big deal if you loose it.
As much as I love stainless steel I think plastic has it's place as long as the grade is the right grade for the job and it's well designed.