Alo Sarv
lead developer

Donate via

Latest Builds

version 0.3
tar.gz tar.bz2
Boost 1.33.1 Headers
MLDonkey Downloads Import Module Development
Payment completed
Development in progress.
Developer's Diary

Friday, March 18, 2005

GlobStatReq implemented; UDP GetSources ineffective?

As mentioned yesterday, I was only getting 5% responsivness from UDP source-queries, so I implemented GlobStatReq packet, which is used to ping servers via UDP, aquiring some information about the servers (files, users, and so on), as well as ensure that all servers in list are alive. After implementing that, it cleared up the server-lists nicely (after about a hour - server is dropped when 3 UDP queries fail, and query is done every 20 mins), however, it only raised responsivness up to 10-20% region (11.91% on one running client, 18.38% on other running client). When doing general source-acquisition analysis, it turns out that the amount of sources actually received from UDP servers, compared to what we get from local server, is damn low - around 1-2% margin. Some statistics, generated via newly added script in utils/ subdir:

-> Sources acquisition statistics:
-----> From local server: 3572 (34.67%)
-----> From UDP server: 113 ( 1.10%)
-----> Passivly: 6618 (64.23%)
-----> Total: 10303
-> Server communication statistics:
-----> Sent 439 pings, got 409 answers (6.83% lost)
-----> Sent 439 GetSources, got 83 answers (18.91% effectiveness)
-----> Dropped 2 dead servers.

-> Sources acquisition statistics:
-----> From local server: 3505 (50.93%)
-----> From UDP server: 113 ( 1.64%)
-----> Passivly: 3264 (47.43%)
-----> Total: 6882
-> Server communication statistics:
-----> Sent 749 pings, got 701 answers (6.41% lost)
-----> Sent 749 GetSources, got 88 answers (11.75% effectiveness)
-----> Dropped 6 dead servers.

So either UDP queries are indeed very ineffective, or I'm doing something wrong (again). On the protocol side, it's not that complicated at all - three versions of the packet are in use - v0 allows requesting single file, v1 allows requesting multiple files, and v2 adds filesizes to the request. And I seem to be getting respones with all those packets, so the packet construction must be correct ... ideas?

In case you'r wondering about 50% sources gotten from "Passive" - this generally happens when you restart a running core - in which case clients (in our queue) start reasking us, and are thus passivly added to sources/queue. On a freshly started client, the passive sources getting is significently lower amount (I estimate around 20%, but might be even lower).

Anyway, we'r very close to completing ed2k support. Stability isn't an issue anymore, and there are almost no additional protocol features that add to download speeds (directly that is), but we are missing some critical things that are needed to be accepted in the network properly.

Namely, credits is the biggest of 'em - I'v been holding off credits until now 'cos I wanted to base credits stuff on the new sechashes, only falling back to old userhashes as last resort. So next up we really need to impl sechash + credits stuff.

After that, there's only very minor things that need to be done - need wrapper around ZStream for more dyanic unpacking of incoming packed data (currently it's possible to lose the whole 180kb packet if the source disconnects before all of it is here); minor updates on shell side - e.g. more operations to objects, and then we'r pretty much set and can call ed2k support "completed" - sure there's more work to do there, but it's no longer urgent, and can be done later, over time, when other protocols are implemented also.

Madcat, ZzZz

Comments: Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?