ronf
Async SSH sftp returning error 0x00000010
when I try to connect to a server with async_ssh with the sftp command then I get the following error
`from asyncssh import connect import asyncio
sftp_conn_options = { "host": "EXAMPLE HOST", "username": "EXAMPLE USER", "password": "EXAMPLE PASS", "known_hosts": None, "port": 22, }
async def main(): async with connect(**sftp_conn_options) as conn: async with conn.start_sftp_client() as sftp: listed_files = await sftp.listdir(".") print(listed_files)
if name == "main": loop = asyncio.get_event_loop() loop.run_until_complete(main())
ERROR raise SFTPBadMessage('Unsupported attribute flags: 0x%08x' % asyncssh.sftp.SFTPBadMessage: Unsupported attribute flags: 0x00000010`
Which version of AsyncSSH are you running? This sounds a lot like a known issue where some SFTP servers violate the spec (see #530 for details), but there should be a workaround included in AsyncSSH 2.13.0 and later.
Hi I am using the asyncssh==2.13.2 version with python 3.7.9 on windows
DEBUG:asyncssh:[conn=0] Sending version SSH-2.0-AsyncSSH_2.13.2 DEBUG:asyncssh:[conn=0] Received version SSH-2.0-WS_FTP-SSH_8.8.1.33
Ok - that version should include the workaround, but the workaround only applied when FILEXFER_ATTR_MODIFYTIME was set at the same time as FILEXFER_ATTR_ACMODTIME, and only when using SFTP version 3. Do you know the type of SSH server you are connecting to?
I got this stack trace
DEBUG:asyncio:Using selector: SelectSelector
INFO:asyncssh:Opening SSH connection to "sftp host" , port 22
INFO:asyncssh:[conn=0] Connected to SSH server at "sftp host", port 22
INFO:asyncssh:[conn=0] Local address: 192.168.18.71, port 15718
INFO:asyncssh:[conn=0] Peer address: 3.212.185.206, port 22
DEBUG:asyncssh:[conn=0] Sending version SSH-2.0-AsyncSSH_2.13.2
DEBUG:asyncssh:[conn=0] Received version SSH-2.0-WS_FTP-SSH_8.8.1.33
DEBUG:asyncssh:[conn=0] Requesting key exchange
DEBUG:asyncssh:[conn=0] Key exchange algs: curve25519-sha256,[email protected],curve448-sha512,ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,ecdh-sha2-1.3.132.0.10,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group15-sha512,diffie-hellman-group16-sha512,diffie-hellman-group17-sha512,diffie-hellman-group18-sha512,[email protected],diffie-hellman-group14-sha1,rsa2048-sha256,ext-info-c
DEBUG:asyncssh:[conn=0] Host key algs: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ssh-ed448,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ecdsa-sha2-1.3.132.0.10,rsa-sha2-256,rsa-sha2-512,[email protected],[email protected],[email protected],[email protected],ssh-rsa
DEBUG:asyncssh:[conn=0] Encryption algs: [email protected],[email protected],[email protected],aes256-ctr,aes192-ctr,aes128-ctr
DEBUG:asyncssh:[conn=0] MAC algs: [email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1,[email protected],[email protected],[email protected],[email protected],[email protected]
DEBUG:asyncssh:[conn=0] Compression algs: [email protected],none
DEBUG:asyncssh:[conn=0] Received key exchange request
DEBUG:asyncssh:[conn=0] Key exchange algs: diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256
DEBUG:asyncssh:[conn=0] Host key algs: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss
DEBUG:asyncssh:[conn=0] Client to server:
DEBUG:asyncssh:[conn=0] Encryption algs: blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr
DEBUG:asyncssh:[conn=0] MAC algs: hmac-md5,hmac-sha1,hmac-sha1-96,hmac-md5-96,hmac-sha2-256,hmac-sha2-384,hmac-sha2-512
DEBUG:asyncssh:[conn=0] Compression algs: none
DEBUG:asyncssh:[conn=0] Server to client:
DEBUG:asyncssh:[conn=0] Encryption algs: blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr
DEBUG:asyncssh:[conn=0] MAC algs: hmac-md5,hmac-sha1,hmac-sha1-96,hmac-md5-96,hmac-sha2-256,hmac-sha2-384,hmac-sha2-512
DEBUG:asyncssh:[conn=0] Compression algs: none
DEBUG:asyncssh:[conn=0] Beginning key exchange
DEBUG:asyncssh:[conn=0] Key exchange alg: diffie-hellman-group-exchange-sha256
DEBUG:asyncssh:[conn=0] Client to server:
DEBUG:asyncssh:[conn=0] Encryption alg: aes256-ctr
DEBUG:asyncssh:[conn=0] MAC alg: hmac-sha2-256
DEBUG:asyncssh:[conn=0] Compression alg: none
DEBUG:asyncssh:[conn=0] Server to client:
DEBUG:asyncssh:[conn=0] Encryption alg: aes256-ctr
DEBUG:asyncssh:[conn=0] MAC alg: hmac-sha2-256
DEBUG:asyncssh:[conn=0] Compression alg: none
DEBUG:asyncssh:[conn=0] Requesting service ssh-userauth
DEBUG:asyncssh:[conn=0] Completed key exchange
DEBUG:asyncssh:[conn=0] Request for service ssh-userauth accepted
INFO:asyncssh:[conn=0] Beginning auth for user REDACTED
DEBUG:asyncssh:[conn=0] Remaining auth methods: publickey,password
DEBUG:asyncssh:[conn=0] Preferred auth methods: gssapi-keyex,gssapi-with-mic,hostbased,publickey,keyboard-interactive,password
DEBUG:asyncssh:[conn=0] Trying public key auth with ssh-rsa key
DEBUG:asyncssh:[conn=0] Remaining auth methods: publickey,password
DEBUG:asyncssh:[conn=0] Preferred auth methods: gssapi-keyex,gssapi-with-mic,hostbased,publickey,keyboard-interactive,password
DEBUG:asyncssh:[conn=0] Trying password auth
INFO:asyncssh:[conn=0] Auth for user REDACTED succeeded
CONNECTEd
DEBUG:asyncssh:[conn=0, chan=0] Set write buffer limits: low-water=16384, high-water=65536
INFO:asyncssh:[conn=0, chan=0] Requesting new SSH session
DEBUG:asyncssh:[conn=0, chan=0] Initial recv window 2097152, packet size 32768
DEBUG:asyncssh:[conn=0, chan=0] Initial send window 30000000, packet size 3000000
INFO:asyncssh:[conn=0, chan=0] Subsystem: sftp
INFO:asyncssh.sftp:[conn=0, chan=0] Starting SFTP client
DEBUG:asyncssh.sftp:[conn=0, chan=0] Sending init, version=3
DEBUG:asyncssh:[conn=0, chan=0] Sending 9 data bytes
DEBUG:asyncssh:[conn=0, chan=0] Reading from channel started
DEBUG:asyncssh:[conn=0, chan=0] Received 9 data bytes
DEBUG:asyncssh.sftp:[conn=0, chan=0] Received version=3
DEBUG:asyncssh.sftp:[conn=0, chan=0] Sending opendir for . DEBUG:asyncssh:[conn=0, chan=0] Sending 14 data bytes DEBUG:asyncssh:[conn=0, chan=0] Received 16 data bytes DEBUG:asyncssh.sftp:[conn=0, chan=0] Received handle 323436 DEBUG:asyncssh.sftp:[conn=0, chan=0] Sending readdir for handle 323436 DEBUG:asyncssh:[conn=0, chan=0] Sending 16 data bytes DEBUG:asyncssh:[conn=0, chan=0] Received 32709 data bytes DEBUG:asyncssh.sftp:[conn=0, chan=0] Sending close for handle 323436 DEBUG:asyncssh:[conn=0, chan=0] Sending 16 data bytes DEBUG:asyncssh:[conn=0, chan=0] Received 21 data bytes DEBUG:asyncssh.sftp:[conn=0, chan=0] Received OK DEBUG:asyncssh:[conn=0, chan=0] Sending EOF INFO:asyncssh:[conn=0, chan=0] Received channel close INFO:asyncssh:[conn=0, chan=0] Channel closed INFO:asyncssh.sftp:[conn=0, chan=0] SFTP client exited INFO:asyncssh:[conn=0, chan=0] Closing channel INFO:asyncssh:[conn=0] Closing connection INFO:asyncssh:[conn=0] Sending disconnect: Disconnected by application (11) INFO:asyncssh:[conn=0] Connection closed SFTP operation failed: Unsupported attribute flags: 0x00000010
Ok, so the server is WS_FTP. I haven't seen this problem on that server before. It looks like you're negotiating SFTP version 3 (which is the usual default), but the server is sending an attribute value that's only legal in SFTP version 4 and higher. AsyncSSH supports those higher versions, so you could try requesting that to see if that work around the server issue.
Do you still see a problem if you set sftp_version=6 in the call to start_sftp_client()? If so, what about setting the version to 4, which should be enough to allow this attribute?
If that doesn't work, the next step would probably be to turn up the debug level to 3 so we can see the exact response the server is sending for the SFTPAttrs in the readdir response, To avoid sharing potentially sensitive data you'll probably want to only post the output after the "Sending readdir" message (specifically the packet corresponding to the "Received 32709 data bytes" in the output above), and even there feel free to redact any filenames you see if that's a concern. It's the attribute data after each filename/longname that I really need to see.
I took a closer look at this today, and it looks like the server is sending FILEXFER_ATTR_CREATETIME in its response, where my previous workaround was for servers sending FILEXFER_ATTR_MODIFYTIME when using SFTP version 3. Neither of these is legal in SFTP version 3. According to https://filezilla-project.org/specs/draft-ietf-secsh-filexfer-02.txt, only the following flags are defined:
#define SSH_FILEXFER_ATTR_SIZE 0x00000001
#define SSH_FILEXFER_ATTR_UIDGID 0x00000002
#define SSH_FILEXFER_ATTR_PERMISSIONS 0x00000004
#define SSH_FILEXFER_ATTR_ACMODTIME 0x00000008
#define SSH_FILEXFER_ATTR_EXTENDED 0x80000000
To make matters worse, it's not clear what order additional fields would be in if they were included. New flags can only be added by either using the extension mechanism or by an SFTP version change, such as what happens with SFTP version 4 documented at https://filezilla-project.org/specs/draft-ietf-secsh-filexfer-03.txt, which defines the following:
#define SSH_FILEXFER_ATTR_SIZE 0x00000001
#define SSH_FILEXFER_ATTR_PERMISSIONS 0x00000004
#define SSH_FILEXFER_ATTR_ACCESSTIME 0x00000008
#define SSH_FILEXFER_ATTR_CREATETIME 0x00000010
#define SSH_FILEXFER_ATTR_MODIFYTIME 0x00000020
#define SSH_FILEXFER_ATTR_ACL 0x00000040
#define SSH_FILEXFER_ATTR_OWNERGROUP 0x00000080
#define SSH_FILEXFER_ATTR_EXTENDED 0x80000000
With SFTP version 3, there's a single flag (ACMODTIME) which went set will cause both access time and modify time to be sent in that order. With SFTP version 4, the ACMODTIME flag is replaced with ACCESSTIME and new flags are created for CREATETIME and MODIFYTIME. with the order being access time followed by create time followed by modify time if all three flags are set. So, if SFTP version is 3, would the value 8 in flags mean ACMODTIME (send both access & modify time) or ACCESSTIME, and if MODIFYTIME and/or CREATETIME are set, what order do they go in?
For your specific case, it would be good to see the actual flags value in each of the SFTPAttrs being returned in the directory listing, along with how many additional fields are present. By looking at the timestamps, we can probably also get a sense of the order of these timestamps.
@ronf
just to add my findings, without sftp_version=6 below code will throw SFTP operation failed: Unsupported attribute flags: 0x00000010
import asyncio, asyncssh, sys
async def run_client() -> None:
async with asyncssh.connect(in_sftp_server,username=in_sftp_user, password=in_sftp_pwd, known_hosts=None) as conn:
async with conn.start_sftp_client(sftp_version=6) as sftp:
await sftp.get(fpath,preserve=False)
try:
asyncio.get_event_loop().run_until_complete(run_client())
except (OSError, asyncssh.Error) as exc:
sys.exit('SFTP operation failed: ' + str(exc))
Server: Windows server 2019, Progress WS_SFTP server 8.7.3 Client: asyncssh 2.13.2, Python 3.10.12 on Ubuntu 22.04
Ok - that suggests that the server you are using doesn't actually support SFTP versions greater than 3. If that's the case, though, it is definitely a bug for it to be sending attributes with the 0x00000010 bit set. However, without knowing what exactly the rest of the packet looks like, it's hard for me to suggest possible workarounds.
Closing due to inactivity. Feel free to reopen this or open a new issue if you need anything else.