Modified queueing behaviour?

[DaisyW]

Hi all,

i recently had the idea to modify Limewire's queue so that one can specify a per source limit instead of a total one. So I browsed throught the relevant code and thought I might start implementing this in DownloadManager -> pumpDownloads(). Unfortunately I am a newbie in the Limewire project and still do know almost nothing about how the downloads are managed ... so I wondered if it is possible to ask the AbstractDownloader instances for their possible sources (IPs) there? Are they stored along with the URN, filename/-path etc.? What method(s) can be used to retrieve this information? Or do I have to begin earlier in the code, when a link is clicked (gui??)?
Many thanks for any hints & guide in advance and sorry if I am asking stupid questions - as already mentioned: I am new to the project. ;-)

With kind regards,

Daisy

[Sam]

Hi Daisy,

Each download generally corresponds to a single URN, possibly multiple filenames (but only a single one that will be used when storing to disk), and many sources. Could you clarify what you mean by a "per source limit"? The 'Downloader' interface is the best place to look for things that are exposed by downloads.

[DaisyW]

Could you clarify what you mean by a "per source limit"?


Hi Sam,

many thanks for your answer. My idea is like this - the user can specify two download limits in my modification:

- the normal one - a total number of concurrent downloads
- the new value - a per source (IP) maximum of concurrent downloads

So when i have for example a total of 8 allowed downloads and specified 3 as my new (per source) value, it was possible to download three times from one IP, three times from a second and two times from a third (or single downloads from 8 different IPs etc. of course ...).

I am aware, that there are downloads with multiple sources and it will mean a lot of checking and comparing in the loop that manages the download queue ...

If you got further ideas, hints ... ;-)
Many thanks in advance.

Daisy

[Sam]

Hi Daisy,

I'm still not sure I grasp what you're suggesting. Is it that the limit would not be a limit on the number of connections to a given IP address, but instead a limit on the number of sources a single download can use? This is an interesting idea, but seems very much like an implementation detail, and not something that the average user would ever want to set or change. LimeWire already keeps such limits, but manages them based on the type of connection you have (modem allows a few sources, cable/dsl allows more, PRO users even more, etc..).

I might be completely misunderstanding what you're suggesting, though. Are you aware that one download can be connected to many different IPs at once, but one download would never connect to the same IP twice. Two different downloads might be reusing the same address, though.

Hmm. Maybe you're suggesting that LimeWire disallow different downloads from connecting to a given IP at the same time (if it has N number of connections already)? That would be a good idea, and a very welcome contribution.

I think it is your example that is confusing me. The current limit on the number of concurrent downloads has nothing to do with the sources of each individual download. It is just a limit on the number of overall downloads that can be active at any given moment. So, if you have 10 downloads queued and are allowing 6 active downloads, then 6 of those would try to become active, and while they were, the other 4 would remain queued. Those 6 would then be able to connect to as many sources as possible (limited only by your connection's strength, or PRO status).

A new number, as you propose, would be somewhat orthogonal to the total number of active downloads. If I'm understanding it correctly (which I think I managed to do with my second guess -- limiting the number of active connections to any given IP, over many downloads), then that would serve to keep more downloads in a queued state if they had no other sources, or cause the download to prefer other sources over the saturated IP.

Sam

[DaisyW]

Hmm. Maybe you're suggesting that LimeWire disallow different downloads from connecting to a given IP at the same time (if it has N number of connections already)? That would be a good idea, and a very welcome contribution.

Hi Sam,

sorry again for my confusion of Limewire terms - i am afraid I confused sources, IPs ... (as stated above I am a newbie in this subject) - but yes, you are absolutely right with your second guess :-)

Meanwhile I managed to build a partially working version of my idea. But since I haven't found out yet how to retrieve the IPs of the active and queued Downloads in pumpDownloads(), I store the information as an extra attribute in ManagedDownloader (this.setAttribute("initialIPs" ...) and ask this info in pumpDownloads using md.getAttribute("initialIPs"). But this of course is no up to date information about the current IPs. That's the point mentioned above where I hoped to get some help ... :-)

Many thanks,

Daisy

[Sam]

Hi Daisy,

Sources & IPs are for the most part the same. There are minor differences in use: an IP address is just the address of a host, whereas a source is the general term for someone you're downloading something from.

Your idea is very good. I don't think that DownloadManager is the place it will go, though, since once a download starts, it's generally on its own about which sources to use. Each download figures out the next source to connect to by calling SourceRanker.getBest (from within ManagedDownloader.fireDownloadWorkers). SourceRanker's an interface that has a few implementations, so it might be difficult to inject logic into each SourceRanker.

DownloadManager is a good place to detect if the download has any initial sources that can be used, though, to prevent a download from starting if its only sources are already at their per-source limits. There doesn't seem to be any method in Downloader that exposes available sources, but that would be a good method to add. Perhaps just something like 'Collection<IpPort> getAvailableSources', and each implementation would then return their available sources. Although that would have a lot of overhead having to construct that collection so often. Maybe it would be better to have a new construct called SourceManager that keeps track of which sources are active, and have a method 'boolean isAnySourceAvailable(SourceManager)' and each download can independently figure out if it has an source that can be used, based on information it gets from the SourceManager.

Does any of this make sense? :)

[DaisyW]

Does any of this make sense? :)

Hi Sam,

many thanks again - yes, I think I basically understood what you mean, but I surely will have to learn a lot and read much of limewire's code until I will be able to implement it ...

with kind regards,

D.