HydraNode Roadmap

 

The original roadmap of HydraNode project was based on the early development stages, where development was streamlined – every new feature depended on previous feature, thus it was clear in which order things needed to be implemented. In addition, after completing each step, it was assumed the development never needed to return to that step. Naturally, it was a naïve assumption, but it worked well – at least for a while.

As the development progressed, more and more subsystems became operational, and thus raised the need for maintaining, upgrading and modifying existing components. For a while, it could be ignored and done in-place, during other things, as the development progressed, the number of subsystems grew exponentially, bringing up the need for a revised roadmap.

 

 

 

Contents

 

1. Hydranode Engine

1.1 Event Subsystem                       

            Partially complete, needs import/export fixes

1.2 Networking Subsystem 

            Partially complete, needs UDP sockets support

1.3 Configuration Subsystem         

            Needs replacement by Boost.Optional library

1.4 Range Management Subsystem

            Completed

1.5 Files Management Subsystem

            Needs complete rewrite of PartData API

1.6 Files Identification Subsystem

            Needs new WorkThread API

1.7 Logging Subsystem

            Needs a robust way of categorizing log messages based on source module

2. ED2K Module

2.1 Secure Identification

            Needs SecIdent implementation using CryptoPP library

2.2 UDP Queries

            Depends on 1.2, needs parser support for ED2K UDP packets

2.3 Unicode

            Needs research on Unicode, possible support library, and implementation

2.4 Miscellanous eMule extended protocol features

            Roughly 5 features added by eMule to the ED2K protocol

3. User Interface

3.1 Launcher application

            Needs UI design, and implementation (easy)
  1. The HydraNode Engine

 

HydraNode Engine itself consists of a number of smaller components. As the first module was being developed, the older components started showing their age, and needed upgrades. This is a normal life cycle of applications, as well as application components, such as HydraNode engine subsystems. The order in which the engines need to be replaced is rather irrelevant; however, the most natural order is the same order they were originally built.

 

1.1  Event Subsystem

 

Event Subsystem was the first subsystem built, and was the first to be replaced. The new implementation is based on Boost.Signals library, and is decoupled from both the event source and event handler. However, the new implementation still has some problems on Microsoft Windows platform regarding symbols exporting and importing, and thus needs more work. Most likely solutions include coupling the event tables again with the event source – sad, but inevitable, it seems.

 

1.2  Networking Subsystem

 

The original networking subsystem lacked any limits checking, as well as packet scheduling, thus a new Scheduler API was built on top of the old subsystem to provide those features. The new Scheduler API currently only supports TCP sockets though, thus needs some additional work to also support UDP Sockets.

 

1.3  Configuration Subsystem

 

At the time of this subsystem implementation, Boost 1.32 was not available yet, and thus an in-house Configuration Subsystem was designed and implemented. However, Boost 1.32 adds Boost.Optional library, which should replace our homemade Configuration Subsystem completely.

 

1.4  Range Management Subsystem

 

Range management manages ranges of arbitrary type. It was originally designed as backend for PartData API, however, it was later realized it would be useful for storing other kinds of information as well – blocked IP ranges for example. The original implementation was error-prone to use and lacked several features, thus it was re-designed and re-implemented into a significantly smaller and robust version.

 

1.5  Files Management Subsystem

 

Managing files is one of the most common operations the HydraNode engine needs to perform. The subsystem contains three main classes – FilesList, SharedFile and PartData. While the former two are simple and robust enough, there is room for improvement in PartData class, which is error-prone to maintain and error-prone to use. Based on the new and improved Range Management Subsystem, a new implementation of PartData API needs to be done, based on our new quality and usability standards.

 

1.6  Files Identification Subsystem

 

The given subsystem deals with hashing files in separate thread. Communication between the main thread and the hasher thread is performed over HashWork type objects. The system was originally intended for hashing only, however, later progress in development showed the need for performing other disk I/O-extensive tasks in separate thread also – file moving for example. As such, a more robust WorkThread API needs to be implemented, which would be capable of performing arbitrary jobs, irrelevant from the jobs contents.

 

1.7  Logging Subsystem

 

The original Logging subsystem was designed as simple and robust API for logging messages of various nature. As the application evolved, the need for separating messages from different modules arose, which was not part of the original Logging subsystem implementation. A simple and easy API must be designed for automatically categorizing log messages based on from which module the message originated, preferably keeping syntactical changes to a minimum.

 

  1. ED2K Module

 

ED2K module performs communication with the eDonkey2000 network (using eMule extended protocol features), and acts as a driver for HydraNode engine. It depends heavily on the HydraNode engine features, thus the upgrades mentioned in section 1 of this document must be performed before continuing any work in the ED2K module.

 

2.1  Secure Identification

 

Arguably, the most important eMule Extended Protocol feature, this allows secure credits verification between two compatible clients, protecting them against credits-stealers. Modern ED2K client must support this feature. The engine is based on public/private key system, using RSA-DSS keys. Two libraries currently provide the functionality:

 

While the original discussion on this topic lead to the conclusion that we should use OpenSSL because it is more commonly available on any POSIX platforms, it is now believed that it would make more sense to use CryptoPP. First, it is a modern library, just as Boost and STL, and secondly, since the existing clients are using CryptoPP, the implementation of Secure Identification is simplified significantly, cutting development time, since the implementation can nearly be copy/pasted from existing clients code. To counter the lack of availability of this library on most POSIX platforms by default, the required source and header files of the library can be distributed with HydraNode code, removing the need for external dependency.

 

2.2  UDP queries

 

UDP traffic is used in ED2K network for global source searches over servers, as well as Queue Ranking queries between clients. UDP traffic is very easy to accidentally abuse, and thus be banned by servers or even by clients, thus great care must be taken during implementation of this feature. Additionally, this feature depends on HydraNode Scheduler having UDP sockets support (see section 1.2 of this document).

 

2.3  Unicode

 

Yet another eMule extended protocol feature, Unicode filenames are used between compatible clients.

 

2.4  Miscellaneous eMule extended protocol features

 

Over time, eMule has further extended the ed2k protocol, adding various new packets to the protocol. The number of those extended features is currently below 10, but is still increasing. A modern ed2k client should support almost all of those features, none of which are very complex to implement, but take time nonetheless.

 

  1. User Interface

 

As the application becomes more and more complex, more and more information is needed to be displayed simultaneously to provide a good overview of what is going on. While it is still rather early to talk about full user interface development at this stage, first steps towards that must be taken to aid in further development.

 

3.1  Launcher application

 

Launcher application is the application that shall launch HydraNode engine as sub-process, taking over it’s standard input/output devices and directing them to suitable UI controls. Additionally, the Launcher application is capable of monitoring the state of the engine, restarting it on demand, notifying the user upon crashes, and similar. It should be designed mainly as an aid for developers for monitoring the engine’s output, and providing minimal set of features for interacting with the engine (plus a full command-set for advanced features). The application will also be used in the final product, providing features such as automatic bug-reports during crashes. In final product, the engine debug output should be hidden behind a nice splash screen, while still allowing verbose startup messages, similar to modern Linux distributions startup – “Press ESC for verbose mode”.