Connecting Forever

[Sam]

FYI, we've figured out what's causing the bug where a download gets stuck in 'Connecting' forever. It'll be fixed in the next beta [4.11.2].

[hopalong]

That's fine. Till then I go with 4.10.9.

[hopalong]

LW 4.11.2 : unfortunately 3 files remained in Connecting state using 2 days the new beta.
As I remember these were originally from a shareaza user (rare files).

[Sam]

Oh well, I guess we didn't quite fix the problem. Could you enable logging on the downloader package (com.limegroup.gnutella.downloader -- set log level to ALL) and email me the results? Try to turn the logging on as soon as possible (preferably before the downloads start). My email is my username at limewire.com. Thanks.

[hopalong]

I sent you the log as "LW log, started just before resume"

[rvanhuix]

Gday,
I have the same problem with not being able to connect to a lot of movies and I am wondering if you sam could offer me some assistance as well, I use limewire pro 4.10.9 and can happily download most things but when it comes to popular downloads as in movies it has lots of servers online but sits on connecting and then goeson to need more sources..Any help would be greatly appreciated...Thanks

[Sam]

I cannot offer assistance with LimeWire 4.10.9. I also cannot offer assistance for anything that is copyrighted if you do not have permission to download it.

[hopalong]

... in the search/downloads and in the connections tab.

[zab]

... in the search/downloads and in the connections tab.

If you can reproduce this, could you give us logging on the com.limegroup.gnutella.io.* and com.limegroup.gnutella.downloader.* packages? The log you uploaded is on the gnutella.* package.

The log shows you had 17 downloads. What state were they in? Connecting, waiting for sources, something else?

[hopalong]

All were in connecting state.
I'll try to reproduce the situation.

[jum]

I can always produce this hang after a while. Each time I did svn update in the last week the problems were fewer, but I can always reproduce it after a hour or two uptime. Alle downloads and ultrapeer connections (if I happen to loose one in that time frame) are stuck in connecting state. Is there anything I can look at in the internals of the nio stuff? I have to admit that I am not up to speed with that newfangled thingy yet.

[hopalong]

I reproduced the problem: after a long break i resumed all incomplete download. They are now in connecting state, I have only excellent connection now - 3 UPs connected, 6 servers are trying to connect I tried to pause the downloads, but no change at the servers. when i resume, all downloads are connecting ...
I hope the trace is good.

[hopalong]

attachment ....

[Sam]

jum -- the problem with downloads getting stuck in connecting stuck is likely related to an interaction with the I/O and downloading code, now that downloading is a state machine and all sources are processed on a single thread. We've fixed every likely scenario & have multiple assertions throughout the code to ensure that things aren't going wrong, yet something still seems to be creeping up. The problem as shown in the log that hopalong added is that something is being added into 'activeWorkers' after it is already removed from 'workers' (or it's being removed from allWorkers but not activeWorkers). We haven't seen or heard anyone else reporting that gnutella connections get stuck in connecting state. If that's the case (and downloads/uploads get stuck too) the cause is almost definitely in the I/O code, although I can't really be certain where.

[jum]

Well, as an additional data point from my side is that I have quite a few very old (a year or so) downloads still in the queue that get contacted every time I start LimeWire. I see that the connecting state to a few of these take very long. I did not delete these in the hope that one day the person sharing that particular file might go online again. Could this be related?

[hopalong]

I think the key is that I didn't use the machine for an hour or more, before resumed all downlods.
And the connection was very slow when the user was active, and later too, when i restarted LW. The files are from yesterday, the client was shareaza.

[hopalong]

Here is another trace with 4.12.0.
17 awaiting sources, 25 connecting files,
2 active server connection only, 9 connecting.

[levp]

Attached are my log and screen captures.

Started LW (got about 4 errors - submitted), went to "Turbo-Charged" status for 2-3 minutes, then back to 2 connections for what now is 2 hours so far.

All the while there are 50 downloads that still say "Connecting".

[Sam]

We're looking into it. We won't release 4.12 as live until we either have a solid workaround or the bug itself is fixed. Thanks very much for letting us know about it.

[zab]

I can always produce this hang after a while. Each time I did svn update in the last week the problems were fewer, but I can always reproduce it after a hour or two uptime. Alle downloads and ultrapeer connections (if I happen to loose one in that time frame) are stuck in connecting state. Is there anything I can look at in the internals of the nio stuff? I have to admit that I am not up to speed with that newfangled thingy yet.

1st. Welcome back Jum! Long time no see :)

Are you using windows? Could you check the system error log for the infamous TCP limit message?

[Sam]

Hrmph -- I just realized I read those logs wrong. I think Zlatin's right that the error is related to the TCP socket limit question. Are you doing lots of internet-related stuff while these connecting problems happen?

[levp]

In my case - no TCP messages in the Event Log.

I have previously followed someone's instructions and increased the limit to 100 and didn't have those messages since.


Hrmph -- I just realized I read those logs wrong. I think Zlatin's right that the error is related to the TCP socket limit question. Are you doing lots of internet-related stuff while these connecting problems happen?

[hopalong]

If this the <<EventID 4226: TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts>> , I had increased the limit too, and there was no such event.

[zab]

@hopalong:

could you send us a new log with the latest beta? (4.12.1 )

[hopalong]

I'll try to produce

[hopalong]

Downloads: 23 files (all) connecting
Connections: 2 servers connected,
9 servers connecting.

[Sam]

How long does it take until you're able to reproduce this issue generally?

The problem is one of a few things:
1) A connecting socket isn't being notified when its timeout expires.
2) The connecting socket is being notified, but it doesn't close or trigger an exception.
3) Pending connections waiting for other connecting sockets to finish aren't being notified that they can begin.

If it doesn't take too long to reproduce this, could you edit the 'log4j.properties' file in the LimeWire program directory so that the bottom of it has the following lines:

log4j.logger.com.limegroup.gnutella.downloader=ALL
log4j.logger.com.limegroup.gnutella.io=ALL

Then try and open LimeWire via a command prompt. To do this, go to Start -> Run and type in "cmd". Then go to C:\Program Files\LimeWire (by typing 'cd "C:\Program Files\LimeWire"') and run LimeWire by typing 'java -jar LimeWire.jar > out.txt'. That will redirect all the logging to the file 'out.txt'. If you could mail that out.txt to us, it would be infinitely helpful in figuring out this problem once and for all. You can mail it to the the email sam at limewire dot com.

Thanks.

[hopalong]

I think I can produce in the following way:
place all downloads in awaiting sources state;
wait about one hour, without any activity;
press resume and make some searches;
(all my incomplete files are connecting normally very slowly.)
... and success.
[sorry I forgot to send my previous message one hour before]

[hopalong]

I sent you the trace in PM

[hopalong]

Here is another trace. The situation almost the same, except that now all (5) servers connected, only the downloads are connecting forever.

[Sam]

Good news -- We've figured out what's causing the bug and we're in the process of fixing it. 4.12.2 should have the bug fixed and should be stable for release. :)

[hopalong]

Great, I'm waiting for.

[Sam]

Alright 4.12.2 is released now. Give it a shot? I'm 99.9% positive that we've fixed the bug. Keep in mind that if you have a ton of downloads, things will necessarily connect slowly (we limit only 4 outgoing connection attempts at a time), so things will stay in connecting for a long time. But they won't stay there forever. We'll add an option in a future version to remove that limit.

[hopalong]

ok, i try it. I see all my 23 files connecting, so what is the 4 outgoing connection attempts ?

[Sam]

Everything will look as if it's connecting, but behind-the-scenes LimeWire is limiting outgoing connections to 4 at a time (it's done this for a very long time, so this is nothing new). When one of those 4 fails or succeeds it will let another begin. So your 23 connections are being handled one at a time, in turn. If LimeWire didn't do this, it would hit the XP SP2 socket limit almost immediately, which would cause even more things to break. We built this feature with the intention that most people have not disabled that part of XP, so did not think to offer to the option to disable it. However folks like you who are power-users could certainly use the option of letting LW make as many connections as it wants.

Note that this limit is only when LW is running on XP.

[hopalong]

I've done my routine test and the bug has gone !!

Thanks for the explanation - but you have still created the threads, I think that's why i had to limit the max. number of simultan downloads to 500, to avoid the thread creation error, when I had about 700 incomplete files. Now this preference is perhaps unnecessary because every download is done in one thread if I understand well the modifications in 5.11

[Sam]

Well, kinda/sorta. There's still a thread for "managing" the download -- that'll go away at some point, but is still there for now. There used to be an additional thread for each *source* you were downloading from. So if you had 1 download downloading from 10 sources, that would have been 11 threads -- one for the manager and one for each source. Two downloads with 10 sources each would have been 22 threads. The change was to get rid of the thread-per-source. So now 2 downloads with 10 sources each are just 2 threads -- the two manager threads. So if you have a ton of downloads, it will still use a lot of threads. That's in the pipeline for fixing, but won't be here till the next cycle of betas.

Great to see the bug is gone though. Thanks very much for your help in tracking it down. We couldn't have done it without you. :)

[hopalong]

That will be the paradise - I can keep all or most of my incomplete files for a longer time, because I'm glad to find one source. I've found more (2) sources only two times.

[XKR]

i see that there has been a problem and wanted to know if maybe you have figured anything out yet. i am running Limewire 4.12.3 and it is happening to me. anything that might help? thanks! i'm confused :confused: