Hate to say it, but I spent another day profiling and bugfixing. But then again, I dislike building new stuff on top of things that I can't be sure they work 100% good, so I guess the time spent ensuring the base is good is well spent. The major fix today was finally closing bug #97, that had plagued us since the first implementation of ClientManager - it caused rather odd crashes every few hours; apparently, the bug was that when merging clients in ed2k, the client extension objects were re-parented, but since they kept parent Client pointer internally, this led to crash. This bug didn't surface earlier, since before the m_parent pointer wasn't used for so important stuff - just some logging calls, and such. To be totally honest, it's not really good design there either - having the extension objects keep reference to their parent - it was originally added to allow log messages coming from extensions to print parent object info (ip/port - for debugging purposes); later it was used to simplify some handling of ED2K::DownloadList management; and now it's used to simplify handling of ClientManager integration. Technically, both of the last two could be implemented "externally", e.g. handled in Client class, instead of the extensions, however that would be more error-prone - lot of stuff needs to be done whenever the extension is constructed, or destroyed, and the most maintainance-safe place to do it is in their constructors/destructors (since they are created/destroyed in many places).

gprof and strace-based profiling only resulted in limited success. I was able to reduce time spent in gettimeofday() syscall (on Linux) from 65% down to 16.5%, and inlined a number of functions in different places (Scheduler, ED2K::Client, Partdata), however personally I didn't record any noticable CPU usage drop - still at around 3-5% at 25kb/s upstream, 185kb/s downstream rates (maxing out my link limits) [release build, 2.4ghz p4/ht]

When I first brought in ClientManager, I was expecting to see a lot of mess being shown about ed2k upload-manager, because I considered that part rather weak; however, as it turns out, it's handling it pretty well - keeping slots at low counts (3-5 usually), and splitting bandwidth between those slots correctly (current implementation uses "slot-focus" I think it's called in emule - single slot gets as much as possible, other slots get the leftovers).

Now the big question here is how to handle upload slots across multiple plugins. The base idea detailed in original Hydranode design documents, dating back August/2004, indicated that upload bandwidth should be distributed between plugins based on how useful the plugin has been to use from downloading perspective; hence, if 80% of incoming traffic has come from edonkey plugin, then 80% of upstream bandwidth should go to edonkey plugin. Now, slots don't measure bandwidth, slots are merely used to "fill" the available bandwidth. Plugins have support for priorities right now - adding per-plugin bandwidth limits would be simple.

Basically, I'd like to end up with two upload handling options: dynamic and static. Dynamic way would then, as described above, automatically adjust per-plugin upload limits based on how much download data has been received (per session or overall - that's still open); static way would allow user to specify per-plugin limits by hand, e.g. 5kb/s ed2k, 5kb/s bt. Now, some networks have minimum upstream requirements (e.g. ed2k), so question is how to handle that properly. One way would be to apply the ed2k-specific limiting only when using static bandwidth limiting. If user sets ed2k upstream limit <10kb/s nice behaviour I don't know yet... suggestions?

Madcat, ZzZz