Alo Sarv
lead developer

Donate via

Latest Builds

version 0.3
tar.gz tar.bz2
Boost 1.33.1 Headers
MLDonkey Downloads Import Module Development
Payment completed
Development in progress.
Developer's Diary

Sunday, July 31, 2005

Config Subsystem upgrades: CGComm support, and runtime updates

Been a nice and progressive night, with nearly 900 lines of new code; while the main target of this patchset was new features, some bugs got accidentally killed along the way as well.
In english, this means that you can now do 'config set UpSpeedLimit 10240' from either hnshell or Console, and everything happens as one would expect - within 100-1000ms timeframe, the real speed is adjusted to the new level.

ConnectionLimit changing is somewhat more tricky - if you lower ConnectionLimit below the current open connections count, existing connections are kept open, but new connections aren't accepted until the real ConnectionCount drops below the limit again - this means it might take longer time to completely adjust to the new limit - while speed limits adjust almost instantly, this one can take rather long time to adjust; however, I do not think instantly dropping bunch of random existing connections would be such a good idea either ...

Madcat, ZzZz

Saturday, July 30, 2005

Progress with Console

Improved - now with toolbar and basic lists.

Just playing around with ideas ... in any case, we have Core/GUI comm support in console now (thanks to libcgcomm it's easy to add Core/GUI comm support to any C++ app). I'm not yet sure what to do with the toolbar idea, two lines is obviously too much, and the buttons need some improvements ...

Oh and btw, I set a release date for 0.1.2, to Aug 10th. Refer to 0.1.2 Roadmap page to see what needs to be done for that. Most important requirements are upgrade to Boost 1.33 (will be released on Aug 4th), and finalizing the console. Async DNS support is bit questionable, but I'm hoping it can be finished and tested for that time.

Anyway, it's 7am, been nice 12h+ dev-session, so -> off to sleep.

Madcat, ZzZz

Thursday, July 28, 2005

Introducing Hydranode Console

Many people wonder why I'm heading off towards GUI development instead of bittorrent, gnutella and whatever new networks. To be quite honest, the major reason behind that is that a change is needed - I'm pretty fed up with the core development - it's been what? 14 months now? - and QT development is basically like starting from scratch - all prior knowledge is void, except for base C++ knowledge - in QT, you have nearly no use for STL, Boost or whatever API's we'v written for Hydranode, so it's a clean slate to all prior knowledge base - very nice change. Not to mention that I'v started making really stupid mistakes in core coding already, for example a week ago I checked in a 3-line change into PartData w/o proper testing nor looking at the code, which caused a ton of crashes on file completition (fixed now in r1573). This clearly shows that I should stay off the existing, working code for a while.

So, after many hours, I finally managed to get the first Launcher version up and running:

Hydranode Console with Win2K theme
Hydranode Console with WinXP theme
Hydranode Console with WinXP theme (transparent)
Hydranode Console on Linux

Granted, it's still very early version, so much can change, but I figured I'll drop these screenshots, since I haven't been very active on the blog recently ...

Anyway, the Console will have support for at least partial Core/GUI communication, so one can perform most operations directly from it, without launching full GUI. Rationale: several of my testers on win32 have said "oh, this interface [hydranode shell] is so cool, you HAVE to leave it in!" - so Hydranode Console will become a usable command-line interface, with the addition of some nifty features that we cannot possibly do in telnet-based interface (transparency, for example :P).

Madcat, ZzZz

Monday, July 25, 2005

Progress on GUI design

Past few days have been rather interesting - we'r getting close to finalizing the user interface design. I cannot disclose any design screenshots at this time, but we'v solved large amounts of problems, including dropping several rather tricky-to-implement, but as it turned out, un-necessary features from the design. It shouldn't take long now before I can jump to implementing this thing.

What I can say at this time though, is that basically we'll have a Launcher application (also previously mentioned some months ago in blog posts), that will launch the core as child process, and then bring up user interface as well as child process; which user interface is brought up is determined by command-line arguments. Since the Launcher will be run from a desktop icon, we can, for example, create different icons for different user interfaces, should there be need for this. The Launcher itself will take care of initial startup, splash-screen, but will have no visible windows once the GUI is launched. If the GUI is not launched directly from Launcher (background-mode, for example), the launcher can (optionally) display a tray icon. The launcher will also provide access to a console window of core output, along with an input box for entering commands (technically equivalent to current Hydranode Shell module, except this one would work over Core/GUI communication (hence much more powerful)).

Since the Launcher will have both Core and GUI running as sub-processes, it can monitor the process's state's, and re-start either of them should they crash - while we do not plan on having either of them crashing, in real-world, software still does crash sometimes, so it's nice to have some automated thing behind it, that can, for example, provide automatic bug-data collection for submitting to bugtracker's.

Madcat, ZzZz

Wednesday, July 20, 2005

Hm... upgrades

There has been nearly 60 repository checkins during past few days; I'll try to summarize it now before falling off my chair :)

Most importantly, the new build system was deployed. Converting the BBv1-based Jamfiles to BBv2 only took some 5 hours, and it's very nice and clean now. Currently it has been tested and builds cleanly on Windows/msvc, Windows/mingw and Linux/gcc. Note: there is a small issue open on win32 builds - library paths aren't hardcoded, so manually moving *.dll from lib/ to top-dir is needed; this will be addressed shortly. I'm aware it currently fails on darwin, this will be looked into shortly as well.

All in all, I believe this was the right choice - dumping GNU auto-tools and going for Boost.Build - it's much more manageable and scalable, and it should be easier for end-user (whether a person compiling a piece of software can be considered an end-user is a different question) to handle. The compilation instructions page on wiki got lot shorter as well.

Other miscellaneus changes/fixes that were done during past few days:

Madcat, ZzZz

Sunday, July 17, 2005

New build system: It compiles!

The good news is that the entire Hydranode (4 libraries, 8 plugins and 1 executable) compile cleanly on MSVC, Mingw and linux/gcc now, using the new build system.

The bad news is that I think I discovered a bug in Boost.Filesystem - when compiled with mingw, some paths get broken. This needs bit deeper investigation and a test-app...

Porting this new build system to also work on mac will take a long time, considering some 50-min recompile times on my mac (and you do a LOT of full recompiles when debugging build system), so that's bit of a problem ....

Anyway, this completes phase 1 of the build system upgrade - get-to-know-chit-chat with Boost.Build and the Jam language. The Jamfiles are a rather big mess right now, since I had no prior experience with BB, so one rewrite of all the Jamfiles (using the knowledge gained from first run) is scheduled - this time writing directly to BBv2 syntax, which should be bit simpler as well.

Build configurations need bit working, need to figure out a bunch of command-line args to configure the build (e.g. how to build plugins, which ones to built-in, how to build static binaries, how to build documentation, how to build release tarballs et al) ...

Madcat, ZzZz

Saturday, July 16, 2005

New build system (continued)

Since until now, we'v only used MSVC on win32 to compile hydranode, some additional delays were introduced due to mingw-specific issues.

First of all, linking with boost_program_options library failed due to some odd reasons; even boost regression tests are failing in the case (and have been failing for a year already apparently), namely in dll configuration - static linkage works fine, but that's not an option for us. The developers have said it fails "due to unresearched issues". Anyway, I did some research, and found that we need -Wl,--enable-runtime-pseudo-reloc linker flag to fix it. An email to the library author is on it's way.

Other issue that seems to come up is related to paths handling (using boost_filesystem library). For some reason, boost::filesystem::path outputs proper win32 paths (e.g. \ as separator), but refuses such paths as input, when compiled with mingw. This issue still needs some research.

On Linux, it builds properly (hncore and hnbase library), but running directly from source tree isn't possible yet, since linux doesn't search for libraries in current dir (I recall there were some hacks to get around it, but I'v lost the links - anyone happen to know them?).

Overall, one might argue that having the entire codebase in single hierarchy binds the components too closely together - the current system is much more loose. However, the difference is that when building/developing the app, we want everything to be closely together, to help with maintainance; but when packaging for releases, we prefer to create separate packages - and that's also easier to do from single code tree. So it's a win-win situation.

Madcat, ZzZz

Friday, July 15, 2005

Upgrading the build system

Hydranode current codebase, including all plugins, test-suite and so on consists of over 200 files, containing over 2MB of code, and frankly, it's starting to get a bit over my head. So, in order to counter that, I believe it's time for another build system upgrade (if you remember, last one was like 8 months ago, to GNU autoconf system).

This time, the plan is to upgrade to Boost.Build v1 (maybe version2 in the future). It's very flexible, highly portable and powerful, which suits our needs. The main problem with build systems is that they often fail to handle large number of sub-projects gracefully, but this one seems to handle it very nicely and cleanly.

So first off, I set out a new layout of the code tree, which can be seen here. The biggest change is that hncore and hnbase are now two separate entities - hnbase providing cross-platform API and bunch of useful, general-purpose classes, and hncore implementing the p2p-related things, e.g. FilesList, MetaDb, and so on.

The other change that I'm planning to bring in is get rid of the pesky /testing hierarchy we have in SVN right now, it doesn't seem to justify it's existance. That code shall be moved to the main tree for better overview and maintainance simplification.

GUI code will be in the same code hierarchy as the engine code, and additional user interfaces can be added there as well. With this setup, we get a better overview of things, and make it easier to manage - consider that very soon we'll also have GUI along with it's own testsuite and plugins there as well.

For those ppl out there that are crying out "but I want my configure/make!" - no problem, I will provide a short configure script, as well as a standard Makefile, in order to keep things "standard"; that Makefile/configure will simply wrap around the Boost.Build-based system.

ETA: Few days - Jamfiles (Boost.Build equivalent to Makefiles) are really powerful, but that means a rather steep learning curve, but currently I'm progressing nicely - hncore compiles/links correctly on win32 at least.

In the longer future, I'm thinking about rolling a small light-weight GUI front-end to it, to further simplify configuration / building of the codebase and related libraries. The rationale is that since QT only works with GCC on win32, we basically lost the only IDE that has proper support for many projects (at least I haven't found another one that does it reasonbly good), and + editor just doesn't cut it in long-term on win32 (which is the main target platform, afterall). So a lightweight bjam frontend, which does compiler-output highlighting, acts as gdb frontend and some other minor things - shouldn't be more than cpl days to roll together.

Madcat, ZzZz

Wednesday, July 13, 2005

E-mail notifications and guarding.p2p ipfilter support

Was busy in the morning with some real-life things that needed my attention, and had a long interface-related discussion with our designer in the evening, which left little time for spitting out code. None-the-less, some minor (?) updates:
The email notifier plugin (not compiled by default) is configured from the global config.ini, with the following values:


Pretty straight-forward, however, as you probably noticed, we need to use the IP address of the mailserver currently, since we lack DNS query support. cyberz is working on DNS queries support, and I'v seen initial patch for that (really cool code, can't wait to integrate it :)), but he said it's not ready for integration, and now he's on vacation for a month, so guess that'll have to wait a bit.

In the future, the mailnotifier plugin will support notifications on all kinds of user-defined events, with user-defined message formats/templates.

One might argue that the mailnotifier is useless, and I tend to agree, however, it was requested by Tobias, and since it was a small plugin (215 lines total), I figured I'll just throw it together and be done with it. And due to the modular architecture of hydranode, more plugins != more bloat, so no losses.

Madcat, ZzZz

Tuesday, July 12, 2005

Search results sorting; eMule downloads importing

Been a fairly interesting 18-hour dev-session, with updates in various areas. So, without further delay, he's the listing:
Importing code still needs some work, while it already does it's job, I believe we can take this a lot further. For one thing, I'd like to update the .met files as we progress with the downloads, so the files can be easily imported back to eMule (currently hydranode-downloaded data would be lost, since eMule won't know more data has been downloaded). Additionally, full rehash of the file can be avoided as well, as soon as I figure out the gap-system eMule uses to write completed-ranges to the .met file.

Regarding the possible bug I mentioned in last blog post, seems it was a ghost-bug - when I woke up next morning, it didn't appear anymore, and right now everything seems to work properly, so I'd blame either my broken windows installation, or my ISP.

Oh, and to explain the missing blog posts since 8th - I spent several days re-installing my boxes; for a while there I thought my vidcard had died, but it's Ati's bad drivers to blame again ... anyway, testing the damn vidcard on multiple boxes and multiple drivers, along with like 6 re-installs took quite some time, but now everything seems to be working properly again.

Madcat, ZzZz

Friday, July 08, 2005

New IO methods, added IPFilter support

While one could easily view each day's progress at the DevCenter Timeline section, if everyone would be doing that, then I could just drop this blog. So here we go again:
Now, all this sounds good and so on, however, the new IO patch seems to have introduced some strange bug in networking, which causes ~70% of outgoing connections to fail. It is not known at this time how could that happen, but it was proved that in r1425, we succeed in ~90% of outgoing connections [ed2k], while in r1429 (the next fully working revision), we only succeed in ~40% of outgoing connections. *doh*

Madcat, ZzZz

Wednesday, July 06, 2005

Back again

Sorry for some missing blog posts, there were some real-life things that needed to be taken care of; but I'm back now.

So today, I went over the current pending bugreports in DevCenter, and managed to fix up 4 of them. In addition, cyberz (while implementing DNS resolution API for Hydranode) discovered a rather bad bug in sockets, where "highest" socket number wasn't updated by UDP sockets, so sometimes, UDP sockets weren't properly polled. smiith has been doing some testing of Hydranode on Solaris/Sparc, with nice progress - most things work, but it segfaults in Hello packet parser right now (ed2k module).

The bugs that were closed today were:
There's also a new configure option [on posix platforms], --enable-hexdumps, which enables ALL hexdumps from eDonkey module, should you be interested in the packets contents (helps in debugging endianess, for example). Developer note: per-packet hexdumps can be enabled by hand from packets.cpp anyway, this is just a shortcut to enable all at once.

Madcat, ZzZz

Archives: December 2004 January 2005 February 2005 March 2005 April 2005 May 2005 June 2005 July 2005 August 2005 September 2005 October 2005 November 2005 December 2005 January 2006 February 2006 March 2006 April 2006 May 2006 June 2006 July 2006 August 2006 September 2006 Current Posts

This page is powered by Blogger. Isn't yours?