by John James.
The exception responder message is not detailed enough, You could try turning on detailed debug logs in moodle, or having the ADFS Administrator review their logs as well. It could be misconfigured claims, or a certificate problem etc.
If you want to skip the default login page and have SAML2 as the default option, that is controlled under Authentication settings in Moodle Administration.
Sorry I couldn't be of more help.