Problem

The message received was unexpected or badly formatted.

HRESULT error: SEC_E_ILLEGAL_MESSAGE (0x80090326)

Cause

This error happens during the TLS protocol negotiation with the server.

Possible causes:

1. The server only supports an insecure protocol (SSL 3.0)
2. The server uses TLS 1.2 but only supports insecure cipher suites. E.g. DHE-RSA-AES256-SHA256 (Uses DH 1024-bit key exchange. See Logjam attack).
3. The server uses a ECDSA certificate with a secp521r1 curve. Schannel on Windows 10 no longer supports secp521r1.
   For example, older versions of glftpd have generated a default certificate with this curve. 

Solution

• For cause 1 and 2
  Contact the server administrator and request an upgrade of the server software.
  Enable weak and insecure ciphers on your system. We strongly recommend against this option.
• For cause 3
  Generate a new certificate on the server using a curve different than secp521r1. E.g. secp384r1, secp256r1 or x25519.