ColabFold icon indicating copy to clipboard operation
ColabFold copied to clipboard

Timeout while submitting to MSA server

Open zhangjuan233 opened this issue 1 year ago • 8 comments

Hello,

After executing the command colabfold_batch exper/fasta-res/G0_G0.fasta exper/pdbs --num-models 5, I encountered the following errors:

2024-05-06 15:47:16,903 Unable to initialize backend 'rocm': NOT_FOUND: Could not find registered platform with name: "rocm". Available platform names are: CUDA
2024-05-06 15:47:16,905 Unable to initialize backend 'tpu': INTERNAL: Failed to open libtpu.so: libtpu.so: cannot open shared object file: No such file or directory
2024-05-06 15:47:16,927 Running on GPU
2024-05-06 15:47:17,628 Found 5 citations for tools or databases
2024-05-06 15:47:17,628 Query 1/1: G0_G0 (length 1334)
2024-05-06 15:47:24,534 Timeout while submitting to MSA server. Retrying...
2024-05-06 15:47:30,564 Timeout while submitting to MSA server. Retrying...

**Our IP is 172.31.226.55. After trying the command curl https://api.colabfold.com/queue, I received the following error: curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to api.colabfold.com:443.

Your Environment

colabfold 1.5.5

Looking forward to your reply. Thank you.

zhangjuan233 avatar May 06 '24 07:05 zhangjuan233

Could you run the following from the same machine and report what is says?

openssl s_client -connect api.colabfold.com:443

milot-mirdita avatar May 06 '24 08:05 milot-mirdita

Could you run the following from the same machine and report what is says?

openssl s_client -connect api.colabfold.com:443

Following your instructions, I obtained the following information:

Connecting to 147.46.145.74 CONNECTED(00000003) 40252FA6B97F0000:error:0A000126:SSL routines::unexpected eof while reading:ssl/record/rec_layer_s3.c:687: no peer certificate available No client certificate CA names sent SSL handshake has read 0 bytes and written 332 bytes Verification: OK New, (NONE), Cipher is (NONE) This TLS version forbids renegotiation. Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 0 (ok)

zhangjuan233 avatar May 06 '24 08:05 zhangjuan233

not sure I can help here, this looks like a network/ssl issue and anything specific to colabfold.

I would recommend to ask your university/institute IT for guidance.

milot-mirdita avatar May 06 '24 09:05 milot-mirdita

Hi,

I'm also running into a similar issue. When i try the "openssl s_client -connect api.colabfold.com:443" command i get the following output:

00DCCDA7112B0000:error:8000006E:system library:BIO_connect:Connection timed out:crypto/bio/bio_sock2.c:178:calling connect() 00DCCDA7112B0000:error:10000067:BIO routines:BIO_connect:connect error:crypto/bio/bio_sock2.c:180: connect:errno=110

any help is much appreciated.

jodo231 avatar Jun 26 '24 03:06 jodo231

Is this right now? I am doing some maintenance right now. The server will be back in a few minutes. However even now during the maintenance it should not time out and return at least some response

milot-mirdita avatar Jun 26 '24 03:06 milot-mirdita

Yes, it was a few minutes before the original post. I'll try again in around an hour

Thanks!

jodo231 avatar Jun 26 '24 04:06 jodo231

Hi, Is this issue still happenning, I got the same problem here is the result of

openssl s_client -connect api.colabfold.com:443 

bash test.sh 
2024-08-29 12:26:45,482 Running colabfold 1.5.5 (fdf3b235b88746681c46ea12bcded76ecf8e1f76)

WARNING: You are welcome to use the default MSA server, however keep in mind that it's a
limited shared resource only capable of processing a few thousand MSAs per day. Please
submit jobs only from a single IP address. We reserve the right to limit access to the
server case-by-case when usage exceeds fair use. If you require more MSAs: You can 
precompute all MSAs with `colabfold_search` or host your own API and pass it to `--host-url`

2024-08-29 12:26:45,653 Running on GPU
2024-08-29 12:26:45,991 Found 5 citations for tools or databases
2024-08-29 12:26:45,991 Query 1/1: HERC5_MRPS23 (length 190)
2024-08-29 12:26:58,660 Timeout while fetching result from MSA server. Retrying...                                                                   
2024-08-29 12:27:07,694 Timeout while fetching result from MSA server. Retrying...                                                                   
COMPLETE: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 150/150 [elapsed: 00:21 remaining: 00:00]

But still the same issue persits, will appreciate any suggestion
Fabian

Connecting to 147.46.145.74
CONNECTED(00000003)
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=R11
verify return:1
depth=0 CN=api.colabfold.com
verify return:1
---
Certificate chain
 0 s:CN=api.colabfold.com
   i:C=US, O=Let's Encrypt, CN=R11
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Aug 16 15:02:38 2024 GMT; NotAfter: Nov 14 15:02:37 2024 GMT
 1 s:C=US, O=Let's Encrypt, CN=R11
   i:C=US, O=Internet Security Research Group, CN=ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF8zCCBNugAwIBAgISBPRSvqjbrRB1gLPpPl9sYdWKMA0GCSqGSIb3DQEBCwUA
MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
EwNSMTEwHhcNMjQwODE2MTUwMjM4WhcNMjQxMTE0MTUwMjM3WjAcMRowGAYDVQQD
ExFhcGkuY29sYWJmb2xkLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoC
ggIBAIthj8yf2l2esJCHscLSktrfON9p6AojXlaTyUU9BAy8+/v1vbNr4BH6w5/h
DK8USgkMGHJYE+qThP6dgFqv6cjnKX3lON9IHWaY8xf0QXmJ7DM4aEuBHdB5YAkE
bRlCMfdaI5DGUOxkciMZh7PniTbjmyaWJ/Jsg+0zCbwAoXISSIEZWw5zgnhYd6BG
IxIiL+Q73GmrOMN6FjeBrER1P2ajY+vXcRIKFQ0THLZRAn2pmGg0/oGwHnSDIUhy
vVRhoMzk+yv2Ial5juruA40+Rqq+taG8MXL/qpekRn56J0gqG1t/bP5gReL3ufPT
Hqn9xnoedxjj4yCp+MPY280klR3HmWTGZiV5uJNbfegWZE4eNcCSfIURn86vPLfq
ygF+/pv1jVAR8t5xO2nH/lxBBrQxIA7Ed/QFlJA1Rcf9aSMeSxSWz0duKPM9j3cu
AxXrGGLxT/VUQu5ZaXLiKO49/NOOzLRrvh3is99k2O5PVRvfXj7Al7gHqK1VKZyS
k2uFSjEGaAif90+JrmBu4vajdUgukX/RhRBsabmbomUg/Tng3OOxbkNqwaj8elfb
WjchZqIyO3ckQTWERMBbOB0UgwgQe62CdypeK/lc4JJ5cFQ7VSUfN8vKsktLc7/a
3BId4yv8PlGkOswGVZsD0KX7owparZRmOs4XnSJOfx/Xg4Z9AgMBAAGjggIWMIIC
EjAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC
MAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFFQ9Ujf9XRDZJcmBCCHp/BC7/dxSMB8G
A1UdIwQYMBaAFMXPRqTq9MPAemyVxC2wXpIvJuO5MFcGCCsGAQUFBwEBBEswSTAi
BggrBgEFBQcwAYYWaHR0cDovL3IxMS5vLmxlbmNyLm9yZzAjBggrBgEFBQcwAoYX
aHR0cDovL3IxMS5pLmxlbmNyLm9yZy8wHAYDVR0RBBUwE4IRYXBpLmNvbGFiZm9s
ZC5jb20wEwYDVR0gBAwwCjAIBgZngQwBAgEwggEFBgorBgEEAdZ5AgQCBIH2BIHz
APEAdgA/F0tP1yJHWJQdZRyEvg0S7ZA3fx+FauvBvyiF7PhkbgAAAZFb69jMAAAE
AwBHMEUCIGx6uSjmfDg+eJgbTbQCK3dU7ocPGgKlNgCrmqXCYl2eAiEAjPQBPTlt
HFVRluqRF/0mRKvN37jrjnaCh0QYLpRCCIgAdwDuzdBk1dsazsVct520zROiModG
fLzs3sNRSFlGcR+1mwAAAZFb69jZAAAEAwBIMEYCIQD7UJswkvaeF3ECeNrLxbAW
aTue5nAafcTe40+rTT4jAQIhAOZLLRPxgHVSwOfFB40tRl3UAQWRaGYp2M+TRBwh
5yPZMA0GCSqGSIb3DQEBCwUAA4IBAQBYZJ/MzITTqFhja0Kn4jA+/wwMPPGnLYCC
Uyeu3Y+0vV0R6CUCUMCG9TlFhu2zzOep73KiG9D9OR1P5yNCSaBw7msle0SyM6tP
TAWW9AH0erWubPqCzjL39lfbzRA2nbmF681sliM/FIk6VMo1Y7+vIQc6ts3g7M0d
WO7kPnJqutoYIW1q2EvvlvP2BdrRqtFLumc0YQ8cP4cF45+tRqQSAcQh/g1jM0MR
uEhpWA2AFcAzl57rx2LZ/KSWU0/d99QXpgDSYcAYKQFTp4vXE2wNmmB57vJlCgXZ
pxmPC7CDIss9Vxc3dALP98lYs0cgMOSImqo9zJLV2z0nuliMDp4y
-----END CERTIFICATE-----
subject=CN=api.colabfold.com
issuer=C=US, O=Let's Encrypt, CN=R11
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3638 bytes and written 405 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 4096 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: EA99C80030A49AF59DF20E5214326CE6D93A43068D6982D4143383241D730DE1
    Session-ID-ctx: 
    Resumption PSK: 64D1EF0657855F2139FA9A3F12FC06342A2B0813C876F1296B1DB621B5652D6E58ABBF5E5AD0129BF1A1BC15C8BC2C3C
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - c7 97 97 7c 94 03 e7 f3-76 54 2e 3d 8d 23 19 fd   ...|....vT.=.#..
    0010 - 62 54 09 0e 60 ba 78 b5-b2 9b d0 25 39 a5 0b c4   bT..`.x....%9...
    0020 - 48 d4 c9 50 9d dc 4f a3-b7 8b b9 13 7f d9 08 78   H..P..O........x
    0030 - 91 4f 9d 1b ce 83 61 ae-c8 a8 c4 bd 5e a3 9f 8d   .O....a.....^...
    0040 - 32 66 0c 9d 71 bb 5b 0c-99 97 5c bf 2e 46 22 d2   2f..q.[...\..F".
    0050 - 0a 21 5b 38 33 40 ce 03-34 c1 2f 14 0b 21 38 00   .![[email protected]./..!8.
    0060 - 13 8a b0 ed 2b 10 91 e1-49 2e dd 36 7b 12 86 d5   ....+...I..6{...
    0070 - 9c 78 6d b8 cb e7 c8 c4-ca f8 a5 d8 88 86 b1 b1   .xm.............
    0080 - 79 9c dc 8f b1 a6 36 2a-e3 5d d2 93 33 40 5b 35   y.....6*.]..3@[5
    0090 - a4 79 ee 74 ee 22 45 6e-5a 7f ba 63 7d 81 15 09   .y.t."EnZ..c}...
    00a0 - 84 eb 43 96 a9 ca a2 37-17 32 f8 f4 36 9a b4 9b   ..C....7.2..6...
    00b0 - 9d 8a c2 f5 b2 4d da 8f-fb db b4 7c 23 8e 55 57   .....M.....|#.UW
    00c0 - b8 16 e2 de a4 23 4e 4f-f5 d0 e0 33 b2 10 40 aa   .....#NO...3..@.
    00d0 - a5 94 51 b0 59 1c b6 57-b6 1d 6e 87 c9 da 4d cb   ..Q.Y..W..n...M.
    00e0 - 83 6e 95 d4 b3 86 d6 06-5f 70 c3 6c d7 78 7c a9   .n......_p.l.x|.

    Start Time: 1724923513
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: D89B3A7466FC92BFDC19BC5041CFAD15D1B7E07B6ED18D4B30865B3D1E1000C2
    Session-ID-ctx: 
    Resumption PSK: 5575F5E1F5BD4F550FB796B5EB624A9A66FFA3112A970ABB212307A9090E3CA38CABF8601CD5F4E502A18A8891DBF738
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - c7 97 97 7c 94 03 e7 f3-76 54 2e 3d 8d 23 19 fd   ...|....vT.=.#..
    0010 - f4 3e ed 4d 7f b5 ad 54-4d ee a5 1d c2 2a 2d 13   .>.M...TM....*-.
    0020 - ca 40 53 8e 9c f1 8c b6-d2 d2 fb c9 3b 91 7f fa   .@S.........;...
    0030 - 1b c3 d3 6a 0d 08 5d 3f-2a b1 24 75 c1 58 b9 32   ...j..]?*.$u.X.2
    0040 - ef 96 8b 96 bd 58 20 52-ea aa da 2e 36 cc df f0   .....X R....6...
    0050 - ef 7b 2e 47 75 34 f6 9e-c0 d8 f7 b1 02 08 37 70   .{.Gu4........7p
    0060 - 60 fc 68 3d 40 6d 24 dc-d6 7d 38 d6 dc 09 f5 8f   `.h=@m$..}8.....
    0070 - 1a 4f f3 d6 cd 9e 1e 89-0e 48 c5 a8 bb 87 52 02   .O.......H....R.
    0080 - 29 e7 39 20 e2 55 fa e1-2a 0c de aa 3f e4 bd b0   ).9 .U..*...?...
    0090 - f9 e4 92 67 7c c6 e4 8c-9a 8c 3c 54 9f f3 aa e3   ...g|.....<T....
    00a0 - b1 6a 7f 1d 75 e8 75 21-f9 88 f2 83 08 63 c5 d4   .j..u.u!.....c..
    00b0 - 50 d0 ee cf 85 c4 53 96-84 12 60 59 ac 55 09 2d   P.....S...`Y.U.-
    00c0 - 51 36 a1 96 fe ef 93 6b-58 88 60 33 a3 86 ef 6c   Q6.....kX.`3...l
    00d0 - 2e 7f b4 3f 21 fb 01 0b-0b 57 99 21 00 fc 1c c0   ...?!....W.!....
    00e0 - 55 3d 64 d9 b2 34 ce 30-52 2a 71 a8 c7 30 e6 6c   U=d..4.0R*q..0.l

    Start Time: 1724923513
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
111

fglaser avatar Aug 29 '24 09:08 fglaser