The AMS network is having intermittent packet loss from some ISP's to their network. This is currently being investigated by the provider in question.
Turns out AMS-IX was having weird intermittent issues and by draining AMS-IX it has been fixed.