MLDonkey Downloads Import Module Development
Development in progress.
Tuesday, May 31, 2005
Developers communication medium thoughts
There are several people working on different Hydranode-related things already - I know at least three people writing new plugins for Hydranode, one person writing an interface and one doing black magic. All this leads me to believe that a central communication medium is neccesery to co-ordinate the efforts.
Experience has shown that neither forums, nor IRC are viable communication platforms. Forums tend to fall more towards user-support area, and IRC, as a real-time communication medium, is simply too time-consuming (and many people don't like IRC at all).
So here's the question - what would be the preferred central, developer-oriented communication medium for discussion? Maybe I'm wrong and people/devs do prefer forums, but judging from other projects, it seems mailing-lists are more favoured. Another alternative would be newsgroup, which has the advantage of better threading support...
What say you?
Sunday, May 29, 2005
Implementing Core/GUI comm
Been a pretty progressive 18h-devsession, with some 1000+ new lines of code in lib_cgcomm and mod_cgcomm, as well as a little test-gui-app for testing stuff. Basic searching seems to be working over cgcomm nicely now, but still needs work and debugging.
Proof? Here's a screenshot
. Just a small test-app to test searching, like the ones we have under tests/ dir for testing Core subsystems. No, it will not
become any real interface, in case anyone is getting some strange ideas out there :)
Most likely I'll be doing a bunch of test apps (that will later be for regress-testing core/gui comm), just as I did before for each Core subsystem - can't count the number of times those test apps saved me hours of debugging :). This also gives me chance to experiment with QT library features and limitations before heading off to implement real interfaces.
PS: No, the code isn't in SVN right now - still needs a lot more work :)
Saturday, May 28, 2005
Thoughts on LibCGComm
First of all, why on earth didn't anyone tell me earlier that the download links on front-page were still pointing to v0.1.0 for two days after 0.1.1 release? *doh*. It's fixed now, but I wonder how many people downloaded wrong version during that time...
Anyway, to the topic. I wrote up the preamble and overview of Hydranode Core/GUI Comminication Library document, viewable here
, but moving on some difficulties appeared. First of all - how does one initialize the library? Consider that the library on it's own does not include any networking code (it simply creates the data for sending to core, and parses data received from core, but the sending/receiving must be implemented in client-side code) - this is done in order to be non-intrusive - including networking in the library would create large dependencies and make the library intrusive.
Obviously we need some kind of "main" class in the library, which provides some startup-code, e.g. authentication, some protocol negotiations (compression, endianess etc)... Other classes of the library shouldn't be available until a proper initialization is performed. However, this raises the question of how the other classes behave at all.
One idea was to design them as Singleton's, with lazy initialization (performed by the "main" class). The problem with Singleton pattern is that it becomes rather tiresome to type on constant basis - all major Engine structures are Singeltons, and I find myself rather tired of typing FilesList::instance().someFunc()
instead of simply m_flist->someFunc()
, if it was a variable. Approaching the problem in the object-oriented way, rather than using Singletons (at least not everwhere) allows somewhat more flexible, and intuitive API. For example, searches could be carried out like this:
m_curSearch = new Engine::Search("hello world");
When going with full Singleton-based model, it would instead read something like this:
Clumsy and evil indeed. Or perhaps a combination of those two:
Engine::Search srch = Engine::SubSearch::instance().newSearch("hello world");
Not getting any better either, so seems at least Searching should use the first example.
Actually, the entire API could be implemented without using any Singletons - you want download list? Create a Engine::DownloadList object. Likewise for other subsystems. Personally, I'm rather tired of Singletons everywhere. They do make sense in a program, but in a library, I'm not so sure any more. From the memory usage viewpoint (not that we'd have memory-usage issues nowadays, but worth a note anway), there isn't much of a difference, since lazily initialized singletons don't take up space when they aren't used.
A point, however, is when client code would like to disable some subsystems on runtime - this is where Singletons fall short - you can't really destroy a singleton object once it's created.
So, the bottom line is, singleton's don't really seem to cut it when writing libraries. Looking at the Engine, and notably, HNBase section (cross-platform API layer), there are no singletons there either, since it didn't make sense to create any there.
PS: You might have noticed some changes/confusion in the naming scheme of Hydranode and related names. The most obvious change was HydraNode -> Hydranode casing change (there are still some places using the old casing - if you find some, let me know). Other change was calling the core "the Engine", or even "HydraEngine". This was done to allow "Hydranode Project" to include more standalone-software than simply "Hydranode" itself, considering that we'r releasing separate products, such as the engine itself, LibCGComm, web interface, mini-interface etc etc etc - there's a lot of additional product plans for the future. Yes, I'm aware the HydraEngine tarballs use the name "hydranode" right now - that's intentional, altough it might change in the future.
Hope this made any sense, it's 6am :)
Friday, May 27, 2005
BT protocol parsing, CGComm design
When parsing binary protocols, we'v established an existing practice in Hydranode, using getVal<>/putVal<> methods in Utils namespace, and stringstreams, where the low-level IO methods take care of all the byte-swapping. This is used in eDonkey module, as well as for internal core things such as MetaData / PartData saving.
However, when parsing ascii data, normally one has to resort to either manual hacking using std::string-based tests, scanf() (*cough*), or Boost.Tokenizer
. Neither of those are scalable enough for complex parsing. Thankfully, Boost has just the thing - Boost.Spirit
library - Spirit is an object-oriented recursive-descent parser generator framework implemented using template meta-programming techniques. Expression templates allow us to approximate the syntax of Extended Backus-Normal Form (EBNF) completely in C++.
I'v used Spirit before in Hydranode, namely when parsing ipfilter
files (mldonkey and emule formats), and the result was over 5 times faster than implementations using alternative parsing techniques, so speed isn't the issue. However, Spirit is rather complex framework, so learning it takes a while. The time will be well spent though, considering how much parsing we need to do in various Hydranode modules.
On the Core/GUI comm side, we know that UI programmers would prefer not to know anything about the underlying protocol - ideally, they'd simply interact with a well-defined API, which would hide the underlying implementation (in this case, TCP-based Core/GUI communication). As such, it is clear that a separate library must be created, called libhn_cgcomm
, which would provide an API that User Interfaces can use to interact with the core. Behind the scenes, it shall communicate with the core libcgcomm
module over CGComm Protocol. In the future, Java, Python etc bindings can be made of the library, allowing easy creation of User Interfaces in any programming language.
With that in mind, the first step for setting up proper Core/GUI communication is the design that library, since that is what the User Interface shall "see", and interact with.
Thursday, May 26, 2005
Bittorrent module design
A few days ago, gcostanza
made an assertion
in the forums, indicating:"BT is a complex network too - like ed2k and G1/G2. I just have to know how the object model of Hydranode would hold against that. I somehow have the feeling there are a lot of donkey-isms in it at the current stage."
Well, I'm up for the challenge. It is true that I'v been promising a multi-network client since the beginning, but not backed up that promise with deeds thus far, so the assertion is fully valid and justified.
It is also true that Bittorrent is a complex protocol, but the complexity at the application level is not the protocol itself - but rather from the way it handles files - or more importantly - the lack of it. As I discovered a while ago, Bittorrent doesn't have
files, not at protocol level. Instead, it handles all torrent data in a continuous stream (obviously used as optimization-technique - imagine 5000 files, each 50-bytes in size - how large would a .torrent file be? How many hashes would it require if there was one hash per file? etc).
The issue behind designing Bittorrent plugin is not that there would be lot of donkey-isms in the core right now, but rather the opposite - how to implement Bittorrent in such a way that it would least interfere with the other networks (that use "normal" files handling).
Today, I believe I have resolved the problems related to files handling in Bittorrent module, and thus wrote up an initial design document on it. So, without further talk, here it is:
Hydranode Bittorrent Module Design
Wednesday, May 25, 2005
Release time again
There's been enough fixes since the 0.1.0 release (mainly related to win32 support, but many also affected linux/mac ports), so it's time to release 0.1.1.
The key improvements are, as mentioned, largely improved Win32 port. To summarize:
- UDP reasks now work [tnx to anonymous blog commenter]
- Download performance should be equal to what we have on Linux now [there was a bug in low-level IO routines that broke all download sessions on win32]
- Many crashes were fixed
- View full changelog
The binaries are fully packaged, and uploaded to SourceForge, but just as I went to actually create the release, SF went down for maintainance, so the release is delayed until tomorrow.
On a sidenote, Solaris x86 port is also finished, but not included in this release, since Crypto++ library code (that ed2k module uses) broke when optimizations were turned on, so I had to drop it from this release.
Seems the most time-consuming part of making a release is OSX port - it takes over 2.5 hours to compile release binaries on my little G3/400, and if something goes wrong... so here's a call - any Mac users interested in becoming official Hydranode packager for OSX?
Monday, May 23, 2005
Lost my BlogTitleGenerator again ...
Development progress cannot always be measured in amount of bugs fixed, or features added, or lines of code produced. For example, last 16 hours were very progressive in several aspects, but writing a blog post about that is more complex.
The day started by further debugging on the win32 port crashes; apparently, the last night's patch didn't fix it, so I started suspecting I might be a phantom bug - something I introduced while supposedly trying to fix itself. To verify that, I reverted to SVN snapshot and patched one issue (related to A4AF sources handling). Lo'and'behold, there's no socket bug. *slaps the stupid kitty around*
Following that, I spent a good several hours doing research on how to get controls on the Windows Taskbar - I had heard rumors that it meant some terrible amount of hacks, and after I found the neccesery resources and sample implementations for it, I agree - I believed I had seen hacks before, but this beats it all.
Then we had 6-hour discussion-session with our designer, on the topics of user interface directions. To summarize it, we proved that mini-interface idea (as originally intended) was fundamentally flawed. While it still might be implemented, it will be for fundamentally different reasons than originally, and it's implementation, target audience and objective changed.
The conclude the day, I headed down to attempt to fix the build system problems that the testers have been complaining about for several days already, but I'v never had time to fix them. Basically, the issue is as before - EVERY linux distro ships boost libs as they see fit, with differently mangled library names, so you have to check 4x4 configurations/locations for each lib, which creates a ton of mess. *sighs*
Sunday, May 22, 2005
Hunting same bug for two days?
*sighs*. Apparently, I just spent two days trying to track down a single bug. It all started yesterday morning, when I went to fix some strange crash somewhere in ED2K::Client class, in socket event handler. Some tracking showed that the crash occours after Client calls destroy(), which in turn deletes the socket (if existing). This lead to a chain-reaction of events inside Scheduler, which was supposed to abort all pending requests for the socket, as well as clean everything up, and if new events came in between the time of actual implementation-socket destruction (which is lazily-deleted by SocketWatcher), or new events were already in pending events queue, those events should'v been quietly discarded. However, they weren't, leading to the afore-mentioned crash.
Fixing it wasn't trivial, however. Most obvious solution was to add simple ref-counted boolean in SSocketWrapper (internal Scheduler class for wrapping the frontend/backend handlers together), which could be checked, however, the checks always succeeded (even though even debugger showed that the ref-counted bool was set to false). So I sat there, looking at it rather stupidly - debugger shows all ok, but code behaves differently.
Today, I believe I finally caught it. Apparently, notify() methods in Scheduler were declared const, however, invoking boost::function is (or at least should be) non-const operation. No compilers detected this, but it would seem this lead to the afore-mentioned crash.
Since I don't have verification yet whether it's fixed or not, I'll leave it running over night and see if it appears again, but initial test shows that it should be fixed, so the patch can be merged to SVN soon.
On other news, we'r planning on introducing Hydranode DevCenter, which will include wiki, bug/feature/patch-tracker and svn-browser, designed to give both developers and users a quick and easy overview of the development, and ease the participation in the development process.
I'm also considering creating daily-snapshots section (generated manually, not automatically), to further encourage people (yes, that means you, dear blog reader) to help in the development process.
"Given enough eyeballs, all bugs are shallow.
" - Linus' Law
Friday, May 20, 2005
Win32 performance improvements
Obviously I had left the win32 port un-attended for way too long. I received word yesterday, that Hydranode was not performing well on win32 platform, compared to what it does on linux, so I headed down to investigate.
A quick look at log-analyzer output, and few greps showed two big flaws:
- We'r losing a LOT of connections before they are even established. This is clearly due to the XP / SP2's connection-limiting stuff. While a proper workaround is being developed, lowering connection-limit to 50 seems to clear out the issue.
- We were using 16bit length-value in three of our low-level I/O routines, which caused major breakage for data packets, where data payload exceeded 65k. This never surfaced before, since data packets never exceeded 20k margin, but strangely, recently I'v started receiving up-to 180k data packets, which exposed this issue.
While the first one is still waiting for a proper workaround, the second one was fixed. That second one was causing complete dropping of all clients that transfered us data, which basically meant we only got data from old / non-mule clients.
Other misc changes:
- [ed2k/hnsh] Added add command to modules/ed2k/serverlist dir.
- Save settings on startup right after loading them, to create settings files on first startup.
- Added compile-time self-checks to verify that all our types are of required size (no surprised on ports to wierd hardware, please)
- [ed2k] Cleanup at SourceInfo::addOffered/setReqFile methods, which should lower the de-syncness of various source-counters (and fix few dangling pointer issues between various containers)
Wednesday, May 18, 2005
The 0.1 release and the new site spurred several internal and external discussions, some of them very lengthy, which took a lot of time off real coding. For example, I spent 22 hours in row on tuesday (yes, from 2am to 0am) in discussions ranging from my design choices in the codebase to whether or not the current site is the right site for the project in this phase.
In any case, today I'm finally starting to get back into normal development routine. I discovered a couple of rather bad bugs that were still lurking around, namely:
- [ed2k] There was an unchecked pointer in Client::reqFile() method, causing crash on rare occasions.
- [ed2k] It was possible that dangling pointers were left in Download object, causing crashes during/after file completition.
- [core] Temp files loading from backup (when loading from original fails) wasn't functioning properly.
- [build] Compiling with built-in Boost libraries, but with custom path to Boost sources wasn't functioning.
These issues were resolved today in SVN. Since three of them were rather important, I think a bugfix release is neccesary, some time on this weekend.
Sunday, May 15, 2005
HydraEngine 0.1.0 launched; a look into the future
As you probably noticed, 0.1.0 was released today, along with this brand new website. Special thanks go to our designer Joakim Kuusemaa for the amazing work on the logo and identity concepts.
Today is another important day. Namely, today, exactly one year ago, Hydranode project was born. In a way, all pieces are falling to their right places - all those delays in the release, all the delays related to the website, and logo - it is clear to me know that they weren't supposed to be finished before - they had to be finished exactly at the right time. The site was developed in two nights earlier this week, the logo concept was fully finalized yesterday, and I was still doing hotfixes in the code 12 hours ago. But in the end, everything went well, and it's finally done.
But enough of the past, today is one of the few days that I can honestly talk about the future - as we have a special occasion today here. So, here goes.
The future holds two important branches. On one hand, we'r moving towards the real multi-network downloads, which have been talked about, and promised since the very beginning. To be concrete, the Bittorrent plugin is under development, and will see heavy development over the coming weeks/months. On the other hand, we are also moving towards graphical user interfaces, which is currently blocked due to lacking Core/GUI communication module. As such, the second line of development effort will go towards finalizing the Core/GUI communication module, and writing initial GUI proof-of-concepts.
While doing that, our designer is already working on initial sketches and design of the mini-interface, which will be a small, compact, yet fully functional interface. This is a lead-in for the full user interface, which will most likely use some components from the mini-interface, but will take longer to fully design and develop, and is thus planned for later.
PS: You might be interested in this interview
on RP2P :)
Saturday, May 14, 2005
Seems I missed whole three days of blog posts. Sorry about that. I suddenly realized that the new site that we spent so much time working on won't be launchable on 15th due to various reasons. While the site itself could be launched, I believe it needs a bit more finalizing and artwork, and that the project itself isn't ready for such a site yet. But I didn't want to release 0.1 w/o having a bit more decent site than what we have up right now, so I rolled up my sleeves and quickly cooked up a simple, lightweight site, which will remain until the full site is finished. This took two nights to complete, e.g. wednesday and thursday nights.
Tonight I was making release binaries for all target platforms, with minor touchups here and there. I guess the most important "touchup" was command-line arguments support (using boost::program_options library ofcourse). Arguments of interest might be --background (detatches the process from current tty, backgrounding it), and --disable-status, which disables that status-bar (might be useful for slower computers).
Anyway, now I have binaries for windows, linux (dynamic + static) and mac (*duh*, compiling release binaries on my G3/400 took 2h18min), so everything is ready on that side. Tomorrow, at around 2-3am, when there's least traffic on the site, I'll be doing the site upgrades, during which the site will be either completely down or at least completely unusable.
Wednesday, May 11, 2005
Dynamic release binaries ?
More design-work on core/gui protocol ... also tried to get hydranode binaries "shippable" with dynamic linking - I don't like the idea of having to link all the modularity together into static binary at all - if nothing else, it ruins the chances of loading additional modules. So, in order for the dynamic binaries, I need to ship all libs hydranode depends on, right? This includes libstdc++.so (837kb) and libc (1.2mb), plus few small 50-100k libs, e.g. pthread etc. Technically, when it comes to final release binary size, we don't lose much - tarball ended up being 2.2MB, compared to 1.5MB in case of static binary. Unpacked size is 6.7MB, compared to 4.2MB in case of static binary. I'd say these are reasonable sizes - I think we'r past the time where we need to count in kbytes - and if you really want to run hydranode on your handheld device ... well, we'll deal with that then. Only issue I ran into these dynamic binares was /lib/ld-linux.so.2 - what's that? That one is the only one with hard-coded path, which introduces a problem ... libc and libpthread look for GLIBC_PRIVATE in that library, and don't find it on debian woody...
Looking deeper, seems the issue raises from woody having libc-2.2.5, while nowadays we seem to have libc-2.3.2. If I could say "you need libc 2.3.2 for the binaries to work", will ppl start yelling again? Guess I have to provide full-static binaries for those people...
Anyway, all this core/gui comm work raised a lot of thoughts and ideas on what external apps / interfaces can use it ... lots and lots of ideas, heh, feels just like a year ago, when all this started :)
Tuesday, May 10, 2005
Progress @ Core/GUI comm land
While yesterday I was working on bittorrent-related things, today it turned out to be Core/GUI-communication-related day instead. In the end, it doesn't really matter in which order everything is done, since there's so much to do, and all needs to be done anyway, so ...
I reviewed Hydranode Core/GUI Communication Protocol v1 RFC
, (written ~6 months ago), and found a ton of crap in there, but some valid points, so v2 of the RFC will be put up shortly, based on v1, but heavily modified and simplified. Special thanks for Xaignar in helping in the review process.
I also set up the base layout for cgcomm
module in core, which will perform, as the name says, core/gui communication. At this time only in a 500-line sketch state, the base layout evolves around Client object, which generates a map of SubSystem type objects, to which actual work is directed as needed. For example, when top-level parser (in Client class) detects a search-related packet, it forwards it to Subsystem::Search class, which performs additional parsing, handling and responding as neccesery. This design is somewhat similar to what we have in ed2k module (Client extensions), but with the difference that the Subsystems do ALL of the work, while in ed2k module, the majority of work was still handled by Client class, and only very specific/complex operations were directed to helper-objects.
Monday, May 09, 2005
Research @ BT, timeline for next 2 months, DC module
Been doing a lot of research, proof-of-concept-writing and designing regarding bittorrent module, which is our next bigger target. I have a pretty good understanding about the protocol now, and some initial tests written, but a lot of work still needs to be done regarding actual module design. There are also some things in bittorrent protocol that require slight changes in the engine itself - namely, I discovered that in BT, chunk-hashes may exceed file boundaries
. Yap, that's correct - in BT, the chunkhashes are generated over the entire data of the torrent, possibly overlapping file boundaries where needed. This introduces several problems in our internal designs, which were initally not designed to support such thing (because I simply didn't know about it). I have mostly resolved it, but the design changes still need bit more work before heading to actual implementation. This change affects both PartData, and Hasher. On PartData side, we need a "guardian" PartData, one per torrent, which may contain sub-PartData objects, one for each actual file. The "guardian" PartData handles full-torrent hashing, e.g. being a "virtual file" which includes entire torrent data.
On other news, I realized that all this release-management, as well as porting / cross-platform development is taking up enormous amount of time. However, the thing is, hydranode working on many platforms is completely waste of time if the app itself isn't fully finished. I recently did a partial Solaris x86 port of hydranode (compiles, crashes on startup), and I think we have proven that hydranode IS portable, and can be ported to any OS we want. But all in due time. Right now, porting hydranode, as well as keeping all ports 100% up2date, is a waste of time - I'm still only human, and humans don't scale well. So here's an idea. I go back to full-time development on single-platform (Linux, since it has best development, profiling and debugging tools), fully finalize the base engine including the plugins. If I can stop worrying about other platforms, I can show the fast development speeds you saw during winter. This development would go on for roughly two months, starting 15th May (e.g. after new site launch and 0.1 release), during which I'll write BT plugin, and perhaps get to at least start some fast-mp3-net plugin work. Once that's done, e.g. around 15th July, our, now fulltime, designer has finished research and design of user interface, which I can then implement w/o having to spend additional time on design.
The reason for this scheme is that it speeds up development, and this is a really important point. Hydranode has taken damn long to develop, and it's development has costed quite large sums of money already (bills, hardware, food - all cost money, and full-time free software developing doesn't generate much income), so it's critical I can finish the app and release final version before financial situation gets too out of hand, and I'd have to drop the project just because of that.
To finish on a more positive note, DirectConnect module (developed by HellFire
) has been added to our subversion repository, available at /testing/dc
folder. It's still in rather early development stage, but most important base things are there (including a rather tricky-to-implement fileslist packing algorithm), so personally I'm very interested in how it turns out - being the first 3rd-person-developed module.
is still up and running, I just seem to have forgotten to add links to it. Improvements, updates and fixes are very welcome ;)
Friday, May 06, 2005
Another 2 days spent on build system stuff
Yeap, you read it correctly. Two full days fighting with auto-tools. One of the goals was to be able to use Boost libraries found on system instead of built-in - the built-in-boost support was originally neccesery, since at the time, Boost wasn't available widely yet (or I simply wasn't aware it was). Anyway, I know at least FC3, Debian and Gentoo have Boost libraries available, so ...
Getting autoconf/automake to do what I needed wasn't trivial at all, entire yesterday went to fighting with it on the topic, and I finally managed to resolve it early today morning ("morning" as relative term, e.g. when I woke up). After several more hours of tuning, configure now properly detects boost libraries installed on system, their version, and links against them. There are some limitations tho.
Namely, boost libraries usually come in 4 different configurations - single-thread and multi-thread debug/release variants. Now, the library names are mangled accordingly, e.g. libboost_thread-gcc-mt-d-1-32.so stands for multi-thread debug version 1.32. However, only distro I'v seen so far that get's it right and has properly manged names is Debian. FC3 doesn't use any mangling, and Gentoo is hopeless - they have Boost 1.31 in portage (insufficient for hydranode), and masked 1.32 lacks boost_thread library completely. And it doesn't use mangled names either.
The bottom line is, I can't request the version of boost of my choosing, which can introduce problems, namely, if some distro decides to ship single-thread lib... or when I want to build against debug versions ... but since there's nothing we can do about it, I'm simply linking with -lboost_filesystem, and hope for the best...
It is also now possible to use built-in zlib files if zlib is not found on system (helps for mingw for example).
PS: Thanks for the patch & link on last blog post ;) Useful stuff to know.
Wednesday, May 04, 2005
More build/install system upgrades
OSX lacks lseek64 and O_LARGEFILE define, which broke builds on darwin; this should be fixed now.
Furthermore, libtool lacks proper support for bundles (as noted before). Even further, it simply eats up -bundle arguments passed through it, so I dropped it completely from build system. 40% speedup in configure run and like 5% or so speedup in overall compile time - who needs libtool anyway? :)make install
should work properly now too. Default install prefix is /opt/hydranode, binarie(s) are installed to /opt/hydranode/bin, plugins to /opt/hydranode/lib and headers to /opt/hydranode/include.
I'm currently considering whether to prefix plugins with mod_ instead of lib, and whether to link hydranode itself into one (or two) dynamiclibs instead. The rationale is that additional software could be written (and some already has been written) which uses hydranode libraries, so it would be convenient to have the libraries available. Notably, hnbase.so (providing xplatform API) and hncore.so (the rest of the p2p-related stuff). This would make writing external apps very easy, especially for developers which are already familar with hydranode APIs. Furthermore - want to do something ed2k-protocol-related? Link against libed2k.so, use include/hn/ed2k/ headers, and there you go - full ed2k protocol at your fingertips :)
Only issue with this approach is that it would require users to add /opt/hydranode/lib to their LD_LIBRARY_PATH, which is bit bad. On second thought, only libhydrabase.so and libhydranode.so need to be in LD_LIBRARY_PATH, so those could be symlinked to /usr/lib perhaps?
Tuesday, May 03, 2005
Static binary available; hasher fixes
Well, you asked for it, here it is - static binaries are now available for x86-linux:hydranode-02-05-05-linux-x86.tar.bz2
(1.5mb packed, 4.2mb unpacked)hydranode-02-05-05.tar.gz
(1002kb, source code)Special thanks to bisley for providing additional hosting space; while eventually we'll be using SF for releases, it's useful to have additional mirrors and keep load off the main server. So, thanks :)
The static binary has ed2k and hnsh modules built-in. Additionally, it is impossible to load additional dynamic modules into a static binary (not that there are any other workable modules ready yet, but just so you know).
So, those that haven't tested hydranode yet due to compile times and/or other reasons, there - now you can test it :). While the binary doesn't contain any debugging information (static binary with full debugging info is 97MB), it should (usually) print out automatic stack trace on it's own on crashes.
On other news, I rewrote IO handling in hasher, which now also uses C library calls for reading files (as you might recall, we discovered a while ago that GCC versions 3.4 lacked 64bit support in iostreams library). Hashing, sharing, and downloading of files >2gb should now work flawlessly when compiling with older gcc versions (e.g. 3.2 and 3.3 - anything below 3.2 can't compile hydranode anyway).
PS: Tweaked hnshell code slightly, which now allows piping commands to it via "netcat" (or similar) utilities. Example usage can be seen in utils/hncmd.sh script, which sends it's arguments to hydranode shell (prepending it with "download" command). This one-liner can be used to implement, say, ed2k:// links assoc in browser, by passing them to that script. This means you can now write bash scripts to send commands to hnshell. Enjoy :)
PPS: No, this last change (hnsh stuff) is not included in the binaries uploaded earlier.
Monday, May 02, 2005
built-in modules support
Now it's possible to compile modules into the final binary using --with-builtin=mod1,mod2,mod3 configure flag. Static builds theoretically should work too, by configuring with --enable-static, and running `make static` in src/ dir, but then again, there seems to be some bug left in there that breaks it.
Anyway, it feels kinda stupid ... first I go through all this trouble to build a completely dynamic, modular piece of software, and then, in order to ship it on Linux, a platform which is oh-so-dynamic and oh-so-modular, and where people love modular stuff, I have to create static, non-modular binary, since people insist on using 5-year-old versions of Linux on 10-years-old hardware, where there aren't modern compilers available, and where hydranode compilation can take hours. Where's the logic in that? :(
September 2006 Current Posts