FluentFTP
FluentFTP copied to clipboard
robinrodricks
Make TLS1.3 connections work with AutoConnect
By @JosHuybrighs:
I did some further TLS1.3 tests against my Synology FTP server, added the master GitHub version of the FluentFTP project to my Visual Studio solution, and observed the following:
- TLS session negotiation (Client Hello, Server Hello, Keys and Certificate exchange, ..) is always successfull.
- When
m_stream.ActivateEncryptionreturns, there is still 86 bytes of unread data inm_stream. - The next
Execute("USER " + userName)then fails with 'Stale Data'.
I took a look at the stream data of the session negotation using Wireshark, and see there that the FTP server responds with 4 application data records, immediately followed by 1 other application data record. I am not sure, but what I understand from TLS 1.3 is that the first 4 records are used for the session negotiation. I debugged the code for checking stale data and what I see is that the unread data corresponds with the 5th application data record as I see it in Wireshark.
When I configure _ftpClient.StaleDataCheck = false, the connection is successfull and further data transfer (list folders, etc.) is OK. So it looks like the data is required and I assume it will, after the session activation, be consumed by sslstream (sChannel).
So, my question: is the check for 'Stale Data' correct in FluentFTP when using TLS 1.3? The reason why I ask is because I saw this code in ConnectModule.cs:
#if NET50_OR_LATER
if (protocol == SysSslProtocols.Tls13) {
client.StaleDataCheck = false;
}
else {
client.StaleDataCheck = true;
}
#endif
I am using .NET6
Does this work? FluentFTP.39.4.0-BETA1.zip
Try these:
- Connect with manual config for TLS1.3
- AutoConnect with no config settings
I have fully disabled the stale data check for all connection types during the Connect procedure.
39.4.0
- New: Add
SslProtocolActiveproperty to retrieve the negotiated SSL/TLS protocol version - Fix: Support connecting to TLS 1.3 servers using
AutoConnect(.NET 5.0+)
@robinrodricks
Thanks for the quick reaction and the SslProtocolActive property.
I did a test with the 39.4.0-BETA1 .net6 dll and these are the results:
1. Connect with manual config and TLS1.3 works The code I used to test 'normal' connect is:
public StorageDeviceOpenResult Open()
{
_certTrusted = true;
ConditionalFileLogger.Log($"FTPStorageDevice.Open - Host: {Name}, port: {Port}, account: {AccountName}, connectionMode: {FTPConnectionMode}, tlsProtocol: {SslProtocol.ToString()}");
var result = new StorageDeviceOpenResult() { Status = ConnectStatus.DriveNotAvailable };
if (string.IsNullOrEmpty(AccountName) ||
string.IsNullOrEmpty(Password))
{
result.Status = ConnectStatus.CouldNotAuthenticate;
return result;
}
try
{
// Setup FtpClient
NetworkCredential credential = new NetworkCredential(AccountName, Password);
FtpTrace.EnableTracing = false;
FtpTrace.LogToConsole = false;
_ftpClient = new FtpClient(Host, Port, credential);
if (ConditionalFileLogger.Level == ConditionalFileLogger.LoggingLevel.Detail)
{
FtpTrace.EnableTracing = true;
_ftpClient.OnLogEvent = OnFTPLogEvent;
}
var offset = DateTimeOffset.Now.Offset;
_ftpClient.TimeZone = 2; // Time zone of server should (with LIST) and must (with MLSD) be in UTC
_ftpClient.TimeConversion = FtpDate.ServerTime;
if (FTPConnectionMode != 2)
{
// 1. Setup encryption mode
_ftpClient.EncryptionMode = FTPConnectionMode == 0 ? FtpEncryptionMode.Explicit : FtpEncryptionMode.Implicit;
// 2. Setup ssl/tls protocol
// If SslProtocol == TlsProtocol.LetOsDecide then let .NET sslstream pick the highest and most relevant
// TLS protocol (it will propose TLS 1.2 and TLS 1.3).
_ftpClient.SslProtocols = SslProtocol == TlsProtocol.LetOsDecide ? SslProtocols.None : SslProtocols.Tls12;
// 3. Prepare certificate handling
_certValidationPending = true;
_ftpClient.ValidateAnyCertificate = true;
_ftpClient.ValidateCertificate += _ftpClient_ValidateCertificate;
// 4. Get all trusted certificates in _trustedCerts (will be used in _ftpClient_ValidateCertificate)
var folder = Windows.Storage.ApplicationData.Current.LocalFolder;
string filePath = $"{folder.Path}\\trusted_certs.json";
if (File.Exists(filePath))
{
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - Extract trusted certificates list");
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(List<TrustedCert>));
using (FileStream fs = File.OpenRead(filePath))
{
_trustedCerts = (List<TrustedCert>)serializer.ReadObject(fs);
fs.Close();
}
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - File contains {_trustedCerts.Count} entries");
}
else
{
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - There is no trusted certificates list");
}
}
// Connect to the server
_ftpClient.Connect();
_certValidationPending = false;
if (_ftpClient.IsConnected)
{
result.Status = ConnectStatus.Connected;
SelectedSslProtocol = SelectedSslProcolToTlsProtocol(_ftpClient.SslProtocolActive);
FtpRootDirectory = _ftpClient.GetWorkingDirectory();
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - SUCCESS");
}
else
{
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - NOT CONNECTED");
}
}
catch (FtpSecurityNotAvailableException e1)
{
result.Status = ConnectStatus.TLSNotSupported;
result.ErrorCode = e1.HResult;
result.ErrorMessage = e1.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-1, ErrorCode: {e1.HResult}, {e1.Message}");
}
catch (FtpAuthenticationException e2)
{
result.Status = e2.CompletionCode.Equals("421") ? ConnectStatus.DriveNotAvailable : ConnectStatus.AccountNotRegisteredAnymore; // AuthenticateRequestResult.SignInState.ServiceNotAvailable : AuthenticateRequestResult.SignInState.AccountNotRegistered;
result.ErrorCode = e2.HResult;
result.ErrorMessage = e2.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-2, ErrorCode: {e2.HResult}, {e2.Message}");
}
catch (AuthenticationException e3)
{
result.Status = _certTrusted ? ConnectStatus.AccountNotRegisteredAnymore : ConnectStatus.UntrustedCertificate; // AuthenticateRequestResult.SignInState.AccountNotRegistered : AuthenticateRequestResult.SignInState.CertificateError;
result.ErrorCode = e3.HResult;
result.ErrorMessage = e3.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-3, ErrorCode: {e3.HResult}, {e3.Message}");
}
catch (Exception ex)
{
// 0x80131515: tls problem
// 0x80131500: tls problem or wrong address
// 0x80004004: wrong address
if (FTPConnectionMode != 2 &&
(ex.HResult & 0xFFFF00) == 0x131500)
{
result.Status = _certTrusted ? ConnectStatus.CouldNotAuthenticate : ConnectStatus.UntrustedCertificate;
}
else
{
result.Status = ConnectStatus.DriveNotAvailable; // AuthenticateRequestResult.SignInState.Error;
}
result.ErrorCode = ex.HResult;
result.ErrorMessage = ex.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-4, ErrorCode: {ex.HResult}, {ex.Message}");
}
return result;
}
2. AutoConnect does not work This is the log data:
2022.08.22 10:02:38.516 5:FTPStorageDevice.Initialize - Extract trusted certificates list
2022.08.22 10:02:38.563 5:FTPStorageDevice.Initialize - File contains 6 entries
2022.08.22 10:02:38.599 5:--FluentFTP-- > AutoConnect()
2022.08.22 10:02:38.630 5:--FluentFTP-- > AutoDetect(True, False)
2022.08.22 10:02:38.676 5:--FluentFTP-- > Connect()
2022.08.22 10:02:38.767 5:--FluentFTP-- Status: Connecting to ***:21
2022.08.22 10:02:40.866 5:--FluentFTP-- Status: Disposing FtpSocketStream...
2022.08.22 10:02:40.916 5:--FluentFTP-- > Connect()
2022.08.22 10:02:41.014 5:--FluentFTP-- Status: Connecting to ***:990
2022.08.22 10:02:41.105 5:--FluentFTP-- Command: QUIT
2022.08.22 10:02:41.643 5:--FluentFTP-- Warning: FtpClient.Disconnect(): Exception caught and discarded while closing control connection: System.InvalidOperationException: This operation is only allowed using a successfully authenticated context.
at System.Net.Security.SslStream.ThrowNotAuthenticated()
at System.Net.Security.SslStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at FluentFTP.FtpSocketStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at FluentFTP.FtpSocketStream.WriteLine(Encoding encoding, String buf)
at FluentFTP.FtpClient.Execute(String command)
at FluentFTP.FtpClient.Disconnect()
2022.08.22 10:02:41.675 5:--FluentFTP-- Status: Disposing FtpSocketStream...
2022.08.22 10:02:41.715 5:FTPStorageDevice.Initialize - NOT CONNECTED
It looks like immediately after connecting with port 990 something goes wrong. The certicate validation is also not called but I assume that this is because there is no AUTH command yet (I don't see it in the log).
The code I used to test AutoConnect is:
public StorageDeviceOpenResult Open()
{
_certTrusted = true;
ConditionalFileLogger.Log($"FTPStorageDevice.Open - Host: {Name}, port: {Port}, account: {AccountName}, connectionMode: {FTPConnectionMode}, tlsProtocol: {SslProtocol.ToString()}");
var result = new StorageDeviceOpenResult() { Status = ConnectStatus.DriveNotAvailable };
if (string.IsNullOrEmpty(AccountName) ||
string.IsNullOrEmpty(Password))
{
result.Status = ConnectStatus.CouldNotAuthenticate;
return result;
}
try
{
// Setup FtpClient
NetworkCredential credential = new NetworkCredential(AccountName, Password);
FtpTrace.EnableTracing = false;
FtpTrace.LogToConsole = false;
_ftpClient = new FtpClient(Host, Port, credential);
if (ConditionalFileLogger.Level == ConditionalFileLogger.LoggingLevel.Detail)
{
FtpTrace.EnableTracing = true;
_ftpClient.OnLogEvent = OnFTPLogEvent;
}
var offset = DateTimeOffset.Now.Offset;
_ftpClient.TimeZone = 2; // Time zone of server should (with LIST) and must (with MLSD) be in UTC
_ftpClient.TimeConversion = FtpDate.ServerTime;
if (FTPConnectionMode != 2)
{
// 1. Prepare certificate handling
_certValidationPending = true;
_ftpClient.ValidateAnyCertificate = true;
_ftpClient.ValidateCertificate += _ftpClient_ValidateCertificate;
// 2. Get all trusted certificates in _trustedCerts (will be used in _ftpClient_ValidateCertificate)
var folder = Windows.Storage.ApplicationData.Current.LocalFolder;
string filePath = $"{folder.Path}\\trusted_certs.json";
if (File.Exists(filePath))
{
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - Extract trusted certificates list");
DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(List<TrustedCert>));
using (FileStream fs = File.OpenRead(filePath))
{
_trustedCerts = (List<TrustedCert>)serializer.ReadObject(fs);
fs.Close();
}
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - File contains {_trustedCerts.Count} entries");
}
else
{
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - There is no trusted certificates list");
}
}
// Connect to the server
_ftpClient.AutoConnect();
_certValidationPending = false;
if (_ftpClient.IsConnected)
{
result.Status = ConnectStatus.Connected;
SelectedSslProtocol = SelectedSslProcolToTlsProtocol(_ftpClient.SslProtocolActive);
FtpRootDirectory = _ftpClient.GetWorkingDirectory();
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - SUCCESS");
}
else
{
ConditionalFileLogger.Log("FTPStorageDevice.Initialize - NOT CONNECTED");
}
}
catch (FtpSecurityNotAvailableException e1)
{
result.Status = ConnectStatus.TLSNotSupported;
result.ErrorCode = e1.HResult;
result.ErrorMessage = e1.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-1, ErrorCode: {e1.HResult}, {e1.Message}");
}
catch (FtpAuthenticationException e2)
{
result.Status = e2.CompletionCode.Equals("421") ? ConnectStatus.DriveNotAvailable : ConnectStatus.AccountNotRegisteredAnymore; // AuthenticateRequestResult.SignInState.ServiceNotAvailable : AuthenticateRequestResult.SignInState.AccountNotRegistered;
result.ErrorCode = e2.HResult;
result.ErrorMessage = e2.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-2, ErrorCode: {e2.HResult}, {e2.Message}");
}
catch (AuthenticationException e3)
{
result.Status = _certTrusted ? ConnectStatus.AccountNotRegisteredAnymore : ConnectStatus.UntrustedCertificate; // AuthenticateRequestResult.SignInState.AccountNotRegistered : AuthenticateRequestResult.SignInState.CertificateError;
result.ErrorCode = e3.HResult;
result.ErrorMessage = e3.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-3, ErrorCode: {e3.HResult}, {e3.Message}");
}
catch (Exception ex)
{
// 0x80131515: tls problem
// 0x80131500: tls problem or wrong address
// 0x80004004: wrong address
if (FTPConnectionMode != 2 &&
(ex.HResult & 0xFFFF00) == 0x131500)
{
result.Status = _certTrusted ? ConnectStatus.CouldNotAuthenticate : ConnectStatus.UntrustedCertificate;
}
else
{
result.Status = ConnectStatus.DriveNotAvailable; // AuthenticateRequestResult.SignInState.Error;
}
result.ErrorCode = ex.HResult;
result.ErrorMessage = ex.Message;
ConditionalFileLogger.Log($"FTPStorageDevice.Initialize - FAIL-4, ErrorCode: {ex.HResult}, {ex.Message}");
}
return result;
}
I am OK with 39.4.0-BETA1 since, for my application, I see little use for a AutoConnect. The app already has all the necessary stuff in the UI to configure encryption mode, and force TLS1.2 if the default doesn't work. But I can understand that if this would work across all possible FTP/FTPS servers in the world, any application UI would become much simpler.
Possible solution:
Implement https://github.com/whSwitching/TLSHandler
This rather general issue would profit from: https://github.com/robinrodricks/FluentFTP/wiki/FTPS-Connection-using-GnuTLS
