PartData Class Reference

#include <partdata.h>

Collaboration diagram for PartData:

Collaboration graph
[legend]
List of all members.

Implementation data members

void printCompleted ()
 For testing purposes only.
typedef std::map< uint64_t,
std::string >::iterator 
BIter
boost::scoped_ptr< Detail::ChunkMapm_chunks
uint64_t m_size
boost::filesystem::path m_loc
boost::filesystem::path m_dest
std::map< uint64_t, std::string > m_buffer
uint32_t m_toFlush
MetaDatam_md
uint16_t m_pendingHashes
HashWorkPtr m_fullJob
 Pointer to full rehash job (if any) in progress, used for canceling full rehash in case a chunkhash fails while this is in progress.
uint32_t m_sourceCnt
uint32_t m_fullSourceCnt
std::map< uint32_t, std::vector<
bool > > 
m_partStatus
 Caches complete chunks boolmaps, for faster usage by modules.

Public Member Functions

 DECLARE_EVENT_TABLE (PartData *, int)
 PartData (uint64_t size, const boost::filesystem::path &loc, const boost::filesystem::path &dest)
 Construct a NEW temporary file.
 PartData (const boost::filesystem::path &loc)
 Load a previously constructed temporary file.
void addSourceMask (uint32_t chunkSize, const std::vector< bool > &chunks)
 Add an availability chunk mask.
void addFullSource (uint32_t chunkSize)
 Optimized version of addSourceMask(), adds a full source.
void delSourceMask (uint32_t chunkSize, const std::vector< bool > &chunks)
 Remove an availability chunk mask.
void delFullSource (uint32_t chunkSize)
 Remove a full source mask.
Detail::UsedRangePtr getRange (uint32_t size=0)
 Locates a range that PartData considers most important.
Detail::UsedRangePtr getRange (uint32_t size, const std::vector< bool > &chunks)
 Locates a range which is also contained in passed chunkmap.
void write (uint64_t beginOffset, const std::string &data)
 Simply writes data starting at specified offset.
void save ()
 Saves the current state of this file to m_loc.dat file.
void addHashSet (const HashSetBase *hs)
 Adds a hashset with chunkhashes which to test downloaded data against.
void cancelDownload ()
 Cancels this download, discarding ALL downloaded data.
Check for completeness.
Methods to check whether the entire file, or a part of it, is complete.

bool isComplete () const
bool isComplete (const Range64 &subRange) const
bool isComplete (uint64_t begin, uint64_t end) const
Generic accessors
uint32_t getChunkCount (uint32_t chunkSize) const
MetaDatagetMetaData () const
void setMetaData (MetaData *md)
void setDestination (const boost::filesystem::path &p)
uint64_t getSize () const
boost::filesystem::path getLocation () const
boost::filesystem::path getDestination () const
uint64_t getCompleted () const
 Returns number of bytes completed.
uint32_t getSourceCnt () const
 Returns number of known sources.
uint32_t getFullSourceCnt () const
 Returns number of full sources.
std::vector< bool > getPartStatus (uint32_t chunkSize) const

Private Member Functions

 PartData (const PartData &)
 Copying part files is not allowed.
PartDataoperator= (const PartData &)
 ~PartData ()
 Only allowed by SharedFile.
Detail::LockedRangePtr getLock (Detail::UsedRangePtr r, uint32_t size)
 Aquire lock on a subrange of UsedRange.
Implementation functions
void checkAddChunkMap (uint32_t chunkSize)
template<typename Predicate>
Detail::UsedRangePtr doGetRange (uint64_t size, Predicate &pred)
void doWrite (uint64_t begin, const std::string &data)
void flushBuffer ()
void rehashCompleted ()
void onHashEvent (HashWorkPtr p, HashEvent evt)
boost::logic::tribool verifyHashSet (const HashSetBase *hs)
void doComplete ()
void destroy ()
void deleteFiles ()

Private Attributes

Main rangelists.
These lists are exclusive, no Range may exist simultaneously in multiple of these lists.

Also, none of these lists may contain overlapping ranges.

RangeList64 m_complete
 Complete ranges.
RangeList64 m_locked
 Locked ranges.
RangeList64 m_corrupt
 Corrupt ranges.

Friends

class SharedFile
class FilesList
class Detail::UsedRange
class Detail::LockedRange
class Detail::Chunk
std::ostream & operator<< (std::ostream &o, const PartData &p)
 Output operator to streams.
int test_main (int, char[])

Classes

struct  LockError
 Exception class. More...
struct  RangeError
 Exception class. More...

Member Typedef Documentation

typedef std::map<uint64_t, std::string>::iterator PartData::BIter [private]
 

Definition at line 295 of file partdata.h.


Constructor & Destructor Documentation

PartData::PartData uint64_t  size,
const boost::filesystem::path &  loc,
const boost::filesystem::path &  dest
 

Construct a NEW temporary file.

Using this constructor, a new temporary file is constructed, at specified location with specified size.

Parameters:
size Size of the resulting file.
loc Location on disk where to store the temp file
dest Destination where to write the complete file
Note:
The disk space indicated by
Parameters:
size is not allocated on actual disk right away. Instead, the size is allocated dynamically as the file grows. This can be changed from global application preferences though.

Definition at line 374 of file partdata.cpp.

References m_chunks, m_dest, m_fullSourceCnt, m_loc, m_md, m_pendingHashes, m_size, m_sourceCnt, m_toFlush, and PD_ADDED.

PartData::PartData const boost::filesystem::path &  loc  ) 
 

Load a previously constructed temporary file.

This method can be used to resume a previously started download, by reading the necessery data from

Parameters:
loc. 
loc Path to PartData reference file, which contains the data required to resume the download.

Definition at line 385 of file partdata.cpp.

References addHashSet(), CHECK_THROW, doComplete(), HashSetBase::getChunkCnt(), HashSetBase::getChunkSize(), Utils::getModDate(), isComplete(), logMsg(), logTrace(), logWarning(), m_complete, m_dest, m_loc, CGComm::OP_METADATA, CGComm::OP_PARTDATA, CGComm::OP_PD_COMPLETED, CGComm::OP_PD_DESTINATION, CGComm::OP_PD_VER, CGComm::OP_RANGELIST, PD_ADDED, printCompleted(), rehashCompleted(), and TRACE_PARTDATA.

Here is the call graph for this function:

PartData::PartData const PartData  )  [private]
 

Copying part files is not allowed.

PartData::~PartData  )  [private]
 

Only allowed by SharedFile.

Definition at line 465 of file partdata.cpp.


Member Function Documentation

void PartData::addFullSource uint32_t  chunkSize  ) 
 

Optimized version of addSourceMask(), adds a full source.

Similar to addSourceMask(), this adds a source which has the entire file.

Parameters:
chunkSize Size of one chunk

Definition at line 496 of file partdata.cpp.

References __1, checkAddChunkMap(), Detail::ID_Length, m_chunks, and m_fullSourceCnt.

Referenced by addSourceMask().

Here is the call graph for this function:

void PartData::addHashSet const HashSetBase hs  ) 
 

Adds a hashset with chunkhashes which to test downloaded data against.

Parameters:
hs Hashset
Precondition:
hs->getChunkCount() > 0

hs->getPartSize() > 0

Definition at line 688 of file partdata.cpp.

References __1, CHECK_THROW, checkAddChunkMap(), Detail::ID_Length, and m_chunks.

Referenced by PartData(), and setMetaData().

Here is the call graph for this function:

void PartData::addSourceMask uint32_t  chunkSize,
const std::vector< bool > &  chunks
 

Add an availability chunk mask.

Allows modules to register chunk availability maps, so PartData can decide the lowest-available chunk to be returned from get* methods

Parameters:
chunkSize Size of one chunk
chunks Boolean vector, where each true indicates the source having the part, and false the source not having the part.

Definition at line 476 of file partdata.cpp.

References __1, addFullSource(), CHECK_THROW, checkAddChunkMap(), getChunkCount(), Detail::ID_Length, m_chunks, and m_sourceCnt.

Here is the call graph for this function:

void PartData::cancelDownload  ) 
 

Cancels this download, discarding ALL downloaded data.

Definition at line 1053 of file partdata.cpp.

References deleteFiles(), and destroy().

Here is the call graph for this function:

void PartData::checkAddChunkMap uint32_t  chunkSize  )  [private]
 

Definition at line 669 of file partdata.cpp.

References getChunkCount(), Detail::ID_Length, isComplete(), m_chunks, m_partStatus, and m_size.

Referenced by addFullSource(), addHashSet(), and addSourceMask().

Here is the call graph for this function:

PartData::DECLARE_EVENT_TABLE PartData ,
int 
 

void PartData::deleteFiles  )  [private]
 

Definition at line 1058 of file partdata.cpp.

References logDebug(), and m_loc.

Referenced by cancelDownload(), SharedFile::onMoveEvent(), and SharedFile::SharedFile().

Here is the call graph for this function:

void PartData::delFullSource uint32_t  chunkSize  ) 
 

Remove a full source mask.

See also:
addFullSource

Definition at line 529 of file partdata.cpp.

References __1, CHECK_THROW, Detail::ID_Length, m_chunks, and m_fullSourceCnt.

Referenced by delSourceMask().

void PartData::delSourceMask uint32_t  chunkSize,
const std::vector< bool > &  chunks
 

Remove an availability chunk mask.

See also:
addSourceMask

Definition at line 507 of file partdata.cpp.

References __1, CHECK_THROW, delFullSource(), getChunkCount(), Detail::ID_Length, m_chunks, and m_sourceCnt.

Here is the call graph for this function:

void PartData::destroy  )  [private]
 

Definition at line 1049 of file partdata.cpp.

References PD_DESTROY.

Referenced by cancelDownload(), SharedFile::destroy(), SharedFile::isDuplicate(), and SharedFile::onMoveEvent().

void PartData::doComplete  )  [private]
 

Definition at line 950 of file partdata.cpp.

References CHECK_THROW, WorkThread::instance(), isComplete(), m_fullJob, m_loc, onHashEvent(), WorkThread::postWork(), and save().

Referenced by Detail::Chunk::onHashEvent(), PartData(), Detail::Chunk::verify(), and write().

Here is the call graph for this function:

template<typename Predicate>
UsedRangePtr PartData::doGetRange uint64_t  size,
Predicate &  pred
[private]
 

Definition at line 588 of file partdata.cpp.

References Range< T >::begin(), RangeList< RangeType >::empty(), Range< T >::end(), RangeList< RangeType >::end(), RangeList< RangeType >::front(), logDebug(), m_chunks, m_complete, and m_size.

Referenced by getRange().

Here is the call graph for this function:

void PartData::doWrite uint64_t  begin,
const std::string &  data
[private]
 

Definition at line 775 of file partdata.cpp.

References BUF_SIZE_LIMIT, CHECK_THROW, RangeList< RangeType >::contains(), logTrace(), m_buffer, m_complete, m_toFlush, RangeList< RangeType >::merge(), PD_DATA_ADDED, save(), RangeList< RangeType >::size(), and TRACE_PARTDATA.

Referenced by write(), Detail::Chunk::write(), and Detail::LockedRange::write().

Here is the call graph for this function:

void PartData::flushBuffer  )  [private]
 

Definition at line 790 of file partdata.cpp.

References CHECK_THROW, RangeList< RangeType >::clear(), logTrace(), m_buffer, m_dest, m_loc, m_md, m_toFlush, O_BINARY, PD_DATA_FLUSHED, printCompleted(), MetaData::setModDate(), and TRACE_PARTDATA.

Referenced by save().

Here is the call graph for this function:

uint32_t PartData::getChunkCount uint32_t  chunkSize  )  const
 

Definition at line 665 of file partdata.cpp.

References m_size.

Referenced by addSourceMask(), checkAddChunkMap(), and delSourceMask().

uint64_t PartData::getCompleted  )  const
 

Returns number of bytes completed.

Definition at line 996 of file partdata.cpp.

References RangeList< RangeType >::begin(), RangeList< RangeType >::end(), and m_complete.

Referenced by printCompleted().

Here is the call graph for this function:

boost::filesystem::path PartData::getDestination  )  const [inline]
 

Definition at line 185 of file partdata.h.

Referenced by SharedFile::getName(), and printCompleted().

uint32_t PartData::getFullSourceCnt  )  const [inline]
 

Returns number of full sources.

Definition at line 190 of file partdata.h.

boost::filesystem::path PartData::getLocation  )  const [inline]
 

Definition at line 184 of file partdata.h.

Referenced by SharedFile::setPartData().

LockedRangePtr PartData::getLock Detail::UsedRangePtr  r,
uint32_t  size
[private]
 

Aquire lock on a subrange of UsedRange.

Parameters:
r UsedRange to aquire lock within
size Upper limit on the size of the LockedRange requested.
Returns:
Locked range
Exceptions:
PartData::LockError if no lock could be aquired.

Definition at line 718 of file partdata.cpp.

References CHECK_THROW, Range< T >::end(), RangeList< RangeType >::end(), RangeList< RangeType >::getContains(), m_complete, and m_locked.

Referenced by Detail::UsedRange::getLock().

Here is the call graph for this function:

MetaData* PartData::getMetaData  )  const [inline]
 

Definition at line 180 of file partdata.h.

Referenced by FilesList::loadTempFile(), and SharedFile::SharedFile().

std::vector<bool> PartData::getPartStatus uint32_t  chunkSize  )  const [inline]
 

Definition at line 191 of file partdata.h.

References CHECK_THROW.

UsedRangePtr PartData::getRange uint32_t  size,
const std::vector< bool > &  chunks
 

Locates a range which is also contained in passed chunkmap.

This method restricts getRange() call to only chunks which are indicated by a true value in the passed chunkmap (e.g. partial sources).

Parameters:
size Size of a chunk in the chunkmap
chunks The chunks the source has
Returns:
Pointer to a range marked as 'used'.
Exceptions:
PartData::RangeError if no usable range could be found.

Definition at line 568 of file partdata.cpp.

References checkPred, RangeList< RangeType >::clear(), doGetRange(), Range< T >::end(), CheckPred::m_rl, m_size, RangeList< RangeType >::push(), and truepred.

Here is the call graph for this function:

UsedRangePtr PartData::getRange uint32_t  size = 0  ) 
 

Locates a range that PartData considers most important.

The current implementation considers partially completed ranges top priority, followed by rarest chunks, and then lowest used chunks.

Parameters:
size Optional, size of the range to be aquired.
Returns:
Pointer to a range marked as 'used'.
Note:
The size of the given range my be smaller than was requested.

Current implementation ignores the size parameter.

Exceptions:
PartData::RangeError if no usable range could be found.

Definition at line 564 of file partdata.cpp.

References doGetRange(), and truepred.

Here is the call graph for this function:

uint64_t PartData::getSize  )  const [inline]
 

Definition at line 183 of file partdata.h.

Referenced by printCompleted().

uint32_t PartData::getSourceCnt  )  const [inline]
 

Returns number of known sources.

Definition at line 188 of file partdata.h.

bool PartData::isComplete uint64_t  begin,
uint64_t  end
const
 

Definition at line 715 of file partdata.cpp.

References RangeList< RangeType >::containsFull(), and m_complete.

Here is the call graph for this function:

bool PartData::isComplete const Range64 subRange  )  const
 

Definition at line 712 of file partdata.cpp.

References RangeList< RangeType >::containsFull(), and m_complete.

Here is the call graph for this function:

bool PartData::isComplete  )  const
 

Definition at line 707 of file partdata.cpp.

References RangeList< RangeType >::begin(), RangeList< RangeType >::end(), m_complete, m_size, and RangeList< RangeType >::size().

Referenced by checkAddChunkMap(), doComplete(), Detail::Chunk::isComplete(), Detail::UsedRange::isComplete(), onHashEvent(), Detail::Chunk::onHashEvent(), SharedFile::onPartEvent(), PartData(), SharedFile::read(), Detail::Chunk::verify(), and write().

Here is the call graph for this function:

void PartData::onHashEvent HashWorkPtr  p,
HashEvent  evt
[private]
 

Generated hashsets, sorted on chunk-count

Definition at line 824 of file partdata.cpp.

References CHECK_THROW, Utils::decode(), HashBase::decode(), HashSetBase::getChunkHashType(), HashSetBase::getChunkSize(), HashSetBase::getFileHash(), HashSetBase::getFileHashType(), MetaData::getFileName(), MetaData::getFileSize(), MetaData::getHashSet(), HASH_COMPLETE, HASH_FATAL_ERROR, isComplete(), logDebug(), logError(), logTrace(), m_dest, PD_COMPLETE, TRACE_PARTDATA, and verifyHashSet().

Referenced by doComplete().

Here is the call graph for this function:

PartData& PartData::operator= const PartData  )  [private]
 

void PartData::printCompleted  ) 
 

For testing purposes only.

Definition at line 1006 of file partdata.cpp.

References RangeList< RangeType >::begin(), Utils::bytesToString(), COL_GREEN, COL_NONE, RangeList< RangeType >::end(), getCompleted(), getDestination(), getSize(), logTrace(), m_complete, and TRACE_PARTDATA.

Referenced by flushBuffer(), and PartData().

Here is the call graph for this function:

void PartData::rehashCompleted  )  [private]
 

Definition at line 467 of file partdata.cpp.

References Detail::ID_Pos, and m_chunks.

Referenced by PartData().

void PartData::save  ) 
 

Saves the current state of this file to m_loc.dat file.

Definition at line 958 of file partdata.cpp.

References CHECK_THROW, flushBuffer(), m_loc, and m_md.

Referenced by FilesList::createDownload(), doComplete(), doWrite(), FilesList::savePartFiles(), and Detail::Chunk::verify().

Here is the call graph for this function:

void PartData::setDestination const boost::filesystem::path &  p  )  [inline]
 

Definition at line 182 of file partdata.h.

void PartData::setMetaData MetaData md  ) 
 

Definition at line 940 of file partdata.cpp.

References addHashSet(), CHECK_THROW, HashSetBase::getChunkCnt(), HashSetBase::getChunkSize(), MetaData::getHashSet(), and m_md.

Referenced by SharedFile::SharedFile().

Here is the call graph for this function:

boost::logic::tribool PartData::verifyHashSet const HashSetBase hs  )  [private]
 

Definition at line 888 of file partdata.cpp.

References CHECK_THROW, RangeList< RangeType >::erase(), HashSetBase::getChunkCnt(), HashSetBase::getChunkHashTypeId(), HashSetBase::getFileHash(), HashSetBase::getFileHashTypeId(), MetaData::getHashSet(), logError(), m_complete, m_corrupt, m_md, m_size, and RangeList< RangeType >::merge().

Referenced by onHashEvent().

Here is the call graph for this function:

void PartData::write uint64_t  beginOffset,
const std::string &  data
 

Simply writes data starting at specified offset.

Checks will be performed to ensure the validity of the location and that it's not already complete or locked.

Definition at line 753 of file partdata.cpp.

References __1, CHECK_THROW, RangeList< RangeType >::contains(), doComplete(), doWrite(), Detail::ID_Pos, isComplete(), logTrace(), m_chunks, m_complete, m_fullJob, m_locked, m_pendingHashes, and TRACE_PARTDATA.

Here is the call graph for this function:


Friends And Related Function Documentation

friend class Detail::Chunk [friend]
 

Definition at line 234 of file partdata.h.

friend class Detail::LockedRange [friend]
 

Definition at line 233 of file partdata.h.

friend class Detail::UsedRange [friend]
 

Definition at line 232 of file partdata.h.

friend class FilesList [friend]
 

Definition at line 231 of file partdata.h.

std::ostream& operator<< std::ostream &  o,
const PartData p
[friend]
 

Output operator to streams.

Definition at line 980 of file partdata.cpp.

friend class SharedFile [friend]
 

Definition at line 230 of file partdata.h.

int test_main int  ,
char  []
[friend]
 


Member Data Documentation

std::map<uint64_t, std::string> PartData::m_buffer [private]
 

Definition at line 294 of file partdata.h.

Referenced by doWrite(), and flushBuffer().

boost::scoped_ptr<Detail::ChunkMap> PartData::m_chunks [private]
 

Definition at line 290 of file partdata.h.

Referenced by addFullSource(), addHashSet(), addSourceMask(), checkAddChunkMap(), delFullSource(), delSourceMask(), doGetRange(), PartData(), rehashCompleted(), Detail::UsedRange::UsedRange(), write(), Detail::LockedRange::write(), and Detail::UsedRange::~UsedRange().

RangeList64 PartData::m_complete [private]
 

Complete ranges.

Definition at line 281 of file partdata.h.

Referenced by doGetRange(), doWrite(), getCompleted(), getLock(), isComplete(), Detail::Chunk::onHashEvent(), PartData(), printCompleted(), verifyHashSet(), and write().

RangeList64 PartData::m_corrupt [private]
 

Corrupt ranges.

Definition at line 283 of file partdata.h.

Referenced by verifyHashSet().

boost::filesystem::path PartData::m_dest [private]
 

Definition at line 293 of file partdata.h.

Referenced by flushBuffer(), Detail::LockedRange::LockedRange(), onHashEvent(), Detail::Chunk::onHashEvent(), PartData(), Detail::UsedRange::UsedRange(), Detail::Chunk::verify(), Detail::LockedRange::~LockedRange(), and Detail::UsedRange::~UsedRange().

HashWorkPtr PartData::m_fullJob [private]
 

Pointer to full rehash job (if any) in progress, used for canceling full rehash in case a chunkhash fails while this is in progress.

Definition at line 301 of file partdata.h.

Referenced by doComplete(), Detail::Chunk::onHashEvent(), and write().

uint32_t PartData::m_fullSourceCnt [private]
 

Definition at line 303 of file partdata.h.

Referenced by addFullSource(), delFullSource(), and PartData().

boost::filesystem::path PartData::m_loc [private]
 

Definition at line 292 of file partdata.h.

Referenced by deleteFiles(), doComplete(), flushBuffer(), PartData(), save(), and Detail::Chunk::verify().

RangeList64 PartData::m_locked [private]
 

Locked ranges.

Definition at line 282 of file partdata.h.

Referenced by getLock(), Detail::LockedRange::LockedRange(), write(), and Detail::LockedRange::~LockedRange().

MetaData* PartData::m_md [private]
 

Definition at line 297 of file partdata.h.

Referenced by flushBuffer(), PartData(), save(), setMetaData(), and verifyHashSet().

std::map<uint32_t, std::vector<bool> > PartData::m_partStatus [private]
 

Caches complete chunks boolmaps, for faster usage by modules.

Definition at line 305 of file partdata.h.

Referenced by checkAddChunkMap(), and Detail::Chunk::onHashEvent().

uint16_t PartData::m_pendingHashes [private]
 

Definition at line 298 of file partdata.h.

Referenced by Detail::Chunk::onHashEvent(), PartData(), Detail::Chunk::verify(), and write().

uint64_t PartData::m_size [private]
 

Definition at line 291 of file partdata.h.

Referenced by checkAddChunkMap(), doGetRange(), getChunkCount(), getRange(), isComplete(), PartData(), and verifyHashSet().

uint32_t PartData::m_sourceCnt [private]
 

Definition at line 302 of file partdata.h.

Referenced by addSourceMask(), delSourceMask(), and PartData().

uint32_t PartData::m_toFlush [private]
 

Definition at line 296 of file partdata.h.

Referenced by doWrite(), flushBuffer(), and PartData().


The documentation for this class was generated from the following files: