MetaDb Class Reference

#include <metadb.h>

List of all members.


Detailed Description

MetaDb class is container for all Meta Data handled by this application.

It provides a number of lookup functions as well as addition functions publically. Internally, it stores various cross-referenced lists to provide fast lookups. The contents of this class are stored in metadb.xml, and loaded on each runtime.

The purpose of this class is to provide a application-central database through which it is possible to perform cross-referencing between FilesList and this list.

Definition at line 45 of file metadb.h.


Public Member Functions

void load (std::istream &is)
 Load the MetaDb contents from input stream, adding all found entries in the stream to the database, merging duplicate entries.
void save (std::ostream &os) const
 Write the contents of the database into output stream.
void push (MetaData *md)
 Add a metadata object to the database.
void push (MetaData *md, SharedFile *sf)
 Add a metadata object to the database, associating it with a SharedFile.
MetaDatafind (const HashBase &h) const
 Find Metadata by searching with hash.
std::vector< MetaData * > find (const std::string &filename) const
 Find MetaData by searching with file name.
MetaDatafind (SharedFile *sf) const
 Find MetaData by searching with SharedFile.
SharedFilefindSharedFile (const HashBase &h) const
 Locate SharedFile by searching with hash.
std::vector< SharedFile * > findSharedFile (const std::string &filename) const
 Locate SharedFile by searching with file name.

Static Public Member Functions

static MetaDbinstance ()
 This class is a Singleton.

Private Types

typedef std::set< MetaData
* >::iterator 
LIter
 List Iterator.
typedef std::set< MetaData
* >::const_iterator 
CLIter
 Constant List Iterator.
typedef std::map< SharedFile *,
MetaData * >::iterator 
SFMDIter
 SharedFile to MetaData Iterator.
typedef std::map< SharedFile *,
MetaData * >::const_iterator 
CSFMDIter
 Constant SharedFile to MetaData Iterator.
typedef std::multimap< std::string,
MetaData * >::iterator 
FNIter
 File Name Iterator.
typedef std::multimap< std::string,
MetaData * >::const_iterator 
CFNIter
 Constant File Name Iterator.
typedef std::multimap< std::string,
SharedFile * >::iterator 
NTSFIter
 Name To SharedFile Iterator.
typedef std::multimap< std::string,
SharedFile * >::const_iterator 
CNTSFIter
 Constant Name To SharedFile Iterator.
typedef std::map< CGComm::HashTypeId,
std::map< HashWrapper, SharedFile
* > >::iterator 
HTSFIter
 Hash To SharedFile Iterator.
typedef std::map< CGComm::HashTypeId,
std::map< HashWrapper, SharedFile
* > >::const_iterator 
CHTSFIter
 Constant Hash To SharedFile Iterator.
typedef std::map< HashWrapper,
SharedFile * >::iterator 
HWTSFIter
 HashWrapper To SharedFile Iterator.
typedef std::map< HashWrapper,
SharedFile * >::const_iterator 
CHWTSFIter
 Constant HashWrapper To SharedFile Iterator.
typedef std::map< CGComm::HashTypeId,
std::map< HashWrapper, MetaData
* > >::iterator 
HTMDIter
 Hash To Meta Data Iterator.
typedef std::map< CGComm::HashTypeId,
std::map< HashWrapper, MetaData
* > >::const_iterator 
CHTMDIter
 Constant Hash To Meta Data Iterator.
typedef std::map< HashWrapper,
MetaData * >::iterator 
HWTMDIter
 Hash Wrapper To Meta Data Iterator.
typedef std::map< HashWrapper,
MetaData * >::const_iterator 
CHWTMDIter
 Constant Hash Wrapper To Meta Data Iterator.

Private Member Functions

void onMetaDataEvent (MetaData *md, int evt)
 Event handler for MetaData events, called from event table.
void onSharedFileEvent (SharedFile *sf, int evt)
 Event handler for SharedFile events, called from event table.
void tryAddFileName (MetaData *source, const std::string &name)
 Attempt to add entry to m_filenames list.
void tryAddFileName (SharedFile *sf, const std::string &name)
 Attempt to add entry to m_nameToId map.
void tryAddHashSet (MetaData *source, const HashSetBase *hash)
 Attempt to add entry to m_hashes list.
void tryAddHashSet (SharedFile *sf, const HashSetBase *hash)
 Attempt to add entry to m_hashToId map.
void clear ()
 Clears all lists.
Singleton
 MetaDb ()
 MetaDb (MetaDb &)
MetaDboperator= (const MetaDb &)
 ~MetaDb ()

Private Attributes

std::set< MetaData * > m_list
 Primary List ------------ Pure metadata objects.
std::map< SharedFile *, MetaData * > m_sfToMd
 SharedFile To MetaData Map ---------------------- SharedFile* keyed map to locate MetaData objects by searching with SharedFile pointer.
std::multimap< std::string,
MetaData * > 
m_filenames
 File Name to MetaData Map ------------------------- This map allows finding MetaData knowing file name.
std::multimap< std::string,
SharedFile * > 
m_nameToSF
 File Name to SharedFile Map ------------------------ This map allows searching with file name and locating the corresponding SharedFile's.
std::map< CGComm::HashTypeId,
std::map< HashWrapper, SharedFile * > > 
m_hashToSF
 Hash To SharedFile Map ---------------------- A dual map which allows us to locate SharedFiles given a hash.
std::map< CGComm::HashTypeId,
std::map< HashWrapper, MetaData * > > 
m_hashes
 Hash To Meta Data Map --------------------- Last, but not least, a recursive map which allows us to look up MetaData objects given we know one hash.

Friends

std::ostream & operator<< (std::ostream &o, const MetaDb &md)
 Output operator for streams.
void test_metadb ()

Classes

class  HashWrapper
 HashWrapper structure acts as a container for HashBase pointers in order to allow us to have a map of those keyed by the Hash. More...

Member Typedef Documentation

typedef std::multimap<std::string, MetaData*>::const_iterator MetaDb::CFNIter [private]
 

Constant File Name Iterator.

Definition at line 217 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, MetaData*> >::const_iterator MetaDb::CHTMDIter [private]
 

Constant Hash To Meta Data Iterator.

Definition at line 286 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, SharedFile*> >::const_iterator MetaDb::CHTSFIter [private]
 

Constant Hash To SharedFile Iterator.

Definition at line 253 of file metadb.h.

typedef std::map<HashWrapper, MetaData*>::const_iterator MetaDb::CHWTMDIter [private]
 

Constant Hash Wrapper To Meta Data Iterator.

Definition at line 290 of file metadb.h.

typedef std::map<HashWrapper, SharedFile*>::const_iterator MetaDb::CHWTSFIter [private]
 

Constant HashWrapper To SharedFile Iterator.

Definition at line 257 of file metadb.h.

typedef std::set<MetaData*>::const_iterator MetaDb::CLIter [private]
 

Constant List Iterator.

Definition at line 188 of file metadb.h.

typedef std::multimap< std::string, SharedFile*>::const_iterator MetaDb::CNTSFIter [private]
 

Constant Name To SharedFile Iterator.

Definition at line 232 of file metadb.h.

typedef std::map<SharedFile*, MetaData*>::const_iterator MetaDb::CSFMDIter [private]
 

Constant SharedFile to MetaData Iterator.

Definition at line 203 of file metadb.h.

typedef std::multimap<std::string, MetaData*>::iterator MetaDb::FNIter [private]
 

File Name Iterator.

Definition at line 215 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, MetaData*> >::iterator MetaDb::HTMDIter [private]
 

Hash To Meta Data Iterator.

Definition at line 282 of file metadb.h.

typedef std::map< CGComm::HashTypeId, std::map<HashWrapper, SharedFile*> >::iterator MetaDb::HTSFIter [private]
 

Hash To SharedFile Iterator.

Definition at line 249 of file metadb.h.

typedef std::map<HashWrapper, MetaData*>::iterator MetaDb::HWTMDIter [private]
 

Hash Wrapper To Meta Data Iterator.

Definition at line 288 of file metadb.h.

typedef std::map<HashWrapper, SharedFile*>::iterator MetaDb::HWTSFIter [private]
 

HashWrapper To SharedFile Iterator.

Definition at line 255 of file metadb.h.

typedef std::set<MetaData*>::iterator MetaDb::LIter [private]
 

List Iterator.

Definition at line 186 of file metadb.h.

typedef std::multimap<std::string, SharedFile*>::iterator MetaDb::NTSFIter [private]
 

Name To SharedFile Iterator.

Definition at line 228 of file metadb.h.

typedef std::map<SharedFile*, MetaData*>::iterator MetaDb::SFMDIter [private]
 

SharedFile to MetaData Iterator.

Definition at line 201 of file metadb.h.


Constructor & Destructor Documentation

MetaDb::MetaDb  )  [private]
 

Definition at line 27 of file metadb.cpp.

MetaDb::MetaDb MetaDb  )  [private]
 

MetaDb::~MetaDb  )  [private]
 

Definition at line 35 of file metadb.cpp.

References m_list.


Member Function Documentation

void MetaDb::clear  )  [private]
 

Clears all lists.

This is used for debugging purposes in metadata regress-test.

Definition at line 439 of file metadb.cpp.

References m_filenames, m_hashes, m_hashToSF, m_list, m_nameToSF, and m_sfToMd.

MetaData * MetaDb::find SharedFile sf  )  const
 

Find MetaData by searching with SharedFile.

Parameters:
id SharedFile to search with
Returns:
Pointer to found MetaData object, or 0 if not found.

Definition at line 329 of file metadb.cpp.

References m_sfToMd.

std::vector< MetaData * > MetaDb::find const std::string &  filename  )  const
 

Find MetaData by searching with file name.

Note that this function may return any number of MetaData objects in case the filename is ambigious.

Parameters:
filename Name of the file to search for
Returns:
Vector containing all found entries. May be empty.

Definition at line 313 of file metadb.cpp.

References logTrace(), m_filenames, and METADB.

Here is the call graph for this function:

MetaData * MetaDb::find const HashBase h  )  const
 

Find Metadata by searching with hash.

Parameters:
h Reference to HashBase object to be searched for.
Returns:
Pointer to the MetaData object, or 0 if not found.

Definition at line 289 of file metadb.cpp.

References logTrace(), m_hashes, and METADB.

Referenced by SharedFile::findMetaData(), and tryAddHashSet().

Here is the call graph for this function:

std::vector< SharedFile * > MetaDb::findSharedFile const std::string &  filename  )  const
 

Locate SharedFile by searching with file name.

Note that this function may return any number of SharedFiles if the file name is ambigious.

Parameters:
filename Name of file to search for
Returns:
Vector containing all found entries. May be empty.

Definition at line 352 of file metadb.cpp.

References m_nameToSF.

SharedFile * MetaDb::findSharedFile const HashBase h  )  const
 

Locate SharedFile by searching with hash.

Parameters:
h Hash to search for
Returns:
The file being searched for, or 0 if not found.

Definition at line 338 of file metadb.cpp.

References m_hashToSF.

Referenced by SharedFile::isDuplicate().

MetaDb & MetaDb::instance  )  [static]
 

This class is a Singleton.

The only instance of this class may be retrieved through this function. Note that the very first call to this function also initializes this class.

Returns:
The only instance of this class

Definition at line 41 of file metadb.cpp.

Referenced by HydraNode::cleanup(), FilesList::createDownload(), SharedFile::findMetaData(), HydraNode::initFiles(), SharedFile::isDuplicate(), FilesList::loadTempFile(), HydraNode::onEvent(), SharedFile::onHashEvent(), and SharedFile::verify().

void MetaDb::load std::istream &  is  ) 
 

Load the MetaDb contents from input stream, adding all found entries in the stream to the database, merging duplicate entries.

Parameters:
is Input stream to read from.

Definition at line 51 of file metadb.cpp.

References CHECK_THROW, logError(), logMsg(), logTrace(), OP_MDB_VERSION, push(), and TRACE_MD.

Referenced by HydraNode::initFiles().

Here is the call graph for this function:

void MetaDb::onMetaDataEvent MetaData md,
int  evt
[private]
 

Event handler for MetaData events, called from event table.

Parameters:
md Event source triggering the event.
evt Event data specifying the kind of event.

Definition at line 369 of file metadb.cpp.

References CHECK_THROW, MD_ADDED_FILENAME, MD_ADDED_HASHSET, tryAddFileName(), and tryAddHashSet().

Referenced by push().

Here is the call graph for this function:

void MetaDb::onSharedFileEvent SharedFile sf,
int  evt
[private]
 

Event handler for SharedFile events, called from event table.

Parameters:
sf SharedFile object triggering the event.
evt Event data specifying the kind of event.

Definition at line 387 of file metadb.cpp.

References HashSetBase::getFileHash(), HashSetBase::getFileHashTypeId(), MetaData::getFileName(), MetaData::getHashSet(), logWarning(), m_hashToSF, m_nameToSF, m_sfToMd, and SF_DESTROY.

Referenced by push(), tryAddFileName(), and tryAddHashSet().

Here is the call graph for this function:

MetaDb& MetaDb::operator= const MetaDb  )  [private]
 

void MetaDb::push MetaData md,
SharedFile sf
 

Add a metadata object to the database, associating it with a SharedFile.

Parameters:
md MetaData pointer to be added to the database.
id SharedFile to associate with this data.

Definition at line 263 of file metadb.cpp.

References CHECK_THROW, m_sfToMd, onSharedFileEvent(), push(), tryAddFileName(), and tryAddHashSet().

Here is the call graph for this function:

void MetaDb::push MetaData md  ) 
 

Add a metadata object to the database.

Parameters:
md MetaData pointer to be added to the database.

Definition at line 240 of file metadb.cpp.

References CHECK_THROW, m_list, onMetaDataEvent(), tryAddFileName(), and tryAddHashSet().

Referenced by FilesList::createDownload(), load(), FilesList::loadTempFile(), SharedFile::onHashEvent(), push(), and SharedFile::verify().

Here is the call graph for this function:

void MetaDb::save std::ostream &  os  )  const
 

Write the contents of the database into output stream.

Parameters:
os Output stream to write to.

Definition at line 96 of file metadb.cpp.

void MetaDb::tryAddFileName SharedFile sf,
const std::string &  name
[private]
 

Attempt to add entry to m_nameToId map.

Parameters:
name Filename to be added.
id ID of the file the name belongs to.

Definition at line 146 of file metadb.cpp.

References CHECK_THROW, logTrace(), m_nameToSF, METADB, and onSharedFileEvent().

Here is the call graph for this function:

void MetaDb::tryAddFileName MetaData source,
const std::string &  name
[private]
 

Attempt to add entry to m_filenames list.

Parameters:
name Filename to be added.
source Source object the file name belongs to.

Definition at line 119 of file metadb.cpp.

References CHECK_THROW, logTrace(), m_filenames, and METADB.

Referenced by onMetaDataEvent(), and push().

Here is the call graph for this function:

void MetaDb::tryAddHashSet SharedFile sf,
const HashSetBase hash
[private]
 

Attempt to add entry to m_hashToId map.

Parameters:
hash Hash to be added.
id ID of the hash being added.

Definition at line 206 of file metadb.cpp.

References CHECK_THROW, HashBase::getTypeId(), m_hashToSF, and onSharedFileEvent().

Here is the call graph for this function:

void MetaDb::tryAddHashSet MetaData source,
const HashSetBase hash
[private]
 

Attempt to add entry to m_hashes list.

Parameters:
hash Hash to be added.
source Source object where this HashSet belongs to.

Definition at line 172 of file metadb.cpp.

References CHECK_THROW, find(), HashBase::getTypeId(), logTrace(), m_hashes, and METADB.

Referenced by onMetaDataEvent(), and push().

Here is the call graph for this function:


Friends And Related Function Documentation

std::ostream& operator<< std::ostream &  o,
const MetaDb md
[friend]
 

Output operator for streams.

Definition at line 101 of file metadb.cpp.

void test_metadb  )  [friend]
 


Member Data Documentation

std::multimap<std::string, MetaData*> MetaDb::m_filenames [private]
 

File Name to MetaData Map ------------------------- This map allows finding MetaData knowing file name.

The file name key here is only that - file name. No paths may be included in the name. Multiple objects may have the same file name - for this reason, this is a multi-map.

Definition at line 213 of file metadb.h.

Referenced by clear(), find(), and tryAddFileName().

std::map< CGComm::HashTypeId, std::map<HashWrapper, MetaData*> > MetaDb::m_hashes [private]
 

Hash To Meta Data Map --------------------- Last, but not least, a recursive map which allows us to look up MetaData objects given we know one hash.

The outer map is keyed on hash types, so there are generally only 5-6 entries in the outer map. For each of the outer map, there is an inner map, which is keyed on the actual Hash, and returns MetaData pointer. Note that we use HashWrapper wrapper class around HashBase pointer, doing virtual function calls on lookups. While it is known that virtual function calls come with some performance tradeoff, I really don't see other way of doing it. Also keep in mind that when HashSet object (contained in MetaData object) is destroyed, it would invalidate the HashWrapper pointer - make sure to not have any HashWrappers containing the HashBase objects around anymore when deleting MetaData.

Definition at line 278 of file metadb.h.

Referenced by clear(), find(), and tryAddHashSet().

std::map< CGComm::HashTypeId, std::map<HashWrapper, SharedFile*> > MetaDb::m_hashToSF [private]
 

Hash To SharedFile Map ---------------------- A dual map which allows us to locate SharedFiles given a hash.

The outer map is keyed on HashTypeId, so there are generally only 5-6 entries there. For each of the outer maps, there's an inner map, containing hashes of the given type. Lookups in the inner map return the SharedFile of the searched entry.

Definition at line 245 of file metadb.h.

Referenced by clear(), findSharedFile(), onSharedFileEvent(), and tryAddHashSet().

std::set<MetaData*> MetaDb::m_list [private]
 

Primary List ------------ Pure metadata objects.

We really cannot rely on any data existing in the contained MetaData pointer, so we can't turn this into some kind of map or anything. However, other lists contain iterators to this list for faster access - iterate directly on this list only as last resort. This list is also used to save in metadata.xml file, and is also ready for publishing to outside world.

Definition at line 184 of file metadb.h.

Referenced by clear(), push(), and ~MetaDb().

std::multimap<std::string, SharedFile*> MetaDb::m_nameToSF [private]
 

File Name to SharedFile Map ------------------------ This map allows searching with file name and locating the corresponding SharedFile's.

Note that this is a multimap - same file name may point to several SharedFiles.

Definition at line 226 of file metadb.h.

Referenced by clear(), findSharedFile(), onSharedFileEvent(), and tryAddFileName().

std::map<SharedFile*, MetaData*> MetaDb::m_sfToMd [private]
 

SharedFile To MetaData Map ---------------------- SharedFile* keyed map to locate MetaData objects by searching with SharedFile pointer.

This list is populated by FilesList (and contained) classes, and generally contains only part of the entire m_list contents - only entries which are also in FilesList are listed here.

Definition at line 199 of file metadb.h.

Referenced by clear(), find(), onSharedFileEvent(), and push().


The documentation for this class was generated from the following files: