The only reason I keep a small amount of cash on me is for a situation when all the global payment networks are down... but if that's occurring, then the problem is much larger than cash.
The main cases against a cashless society are privacy concerns, system failure, and inequality. The only real point I see is a system failure.
Privacy concerns are exaggerated in this topic. The amount of personal data that is freely shared online seems not to worry most people, but their bank knowing their purchases at 2am does?
Inequality is also not accurate in this case. Not having a bank account is a sign of irresponsibility, having a bank account should be mandatory for everyone.
System failure is an interesting one. Anyone who has a credit card has most likely encountered "the payment network is down" issue. Though this rarely arises, it's a real concern.
Right now, the solution to this is avoiding a single point of failure by having access to a card from each network; Visa, Mastercard, AMEX, etc... If you're in a situation where all payment networks are down, then there's a more significant issue at play, and cash is probably not your most pressing concern.