# opentl::modalities::IntensityEdges Class Reference

Class for contour point feature modality computation. ATTENTION: This class is currently not prepared for multi-threaded match...Level() calls related to a single sensor index! More...

Inherits opentl::modalities::Modality.

List of all members.

## Public Types
## Public Member Functions
virtual modalities::Modality * | clone () const
| Clone this class and all potential childs RECURSIVELY!! (deep copy).
opentl::core::cvdata::VisContourSamplePts * | getContourData (int s=0)
virtual void | init ()
| Allocate data structures required by this class.
| IntensityEdges (const IntensityEdges &c)
| Deep Copy Constructor.
| IntensityEdges (const ObjModelPtrVector &objModels, opentl::modelprojection::Warp *warp, opentl::modelprojection::GLScene *scene, int camIdx, modelprojection::ContourSampler *contourSampler=NULL, opentl::modelprojection::GLRenderer *renderer=NULL)
| Constructor ATTENTION: The sensor vector element size/sequence must be identical for the warps of all object models !!!
void | learnBackgroundEdges (const opentl::core::cvdata::Image &imageGray, bool bFirstFrame, double bgDistThreshold, double cannyT1, double cannyT2, double bgRunningAvgCoeff)
| Learn a model of background edges (position and orientation) for subtraction, during pre-processing.
virtual int | matchFeatLevel (const TargetPtrVector &targets, T_MEAS_FEATPtrVector &outputMeas, std::size_t partitionIdx)
| Matching on feature level match projected contour points with detected image edges.
virtual int | matchPixLevel (const TargetPtrVector &targets, T_MEAS_PIXPtrVector &outputMeas, std::size_t partitionIdx)
| Matching on pixel level chamfer matching of projected silhouette on distance transformed edge image.
virtual int | preProcess (const opentl::core::cvdata::Image &image, const std::vector< std::vector< int > > &preProcessROIs)
| Compute the edge map of I (e.g. with Canny), and store it into mImageFeatDetected.
void | readSilhouettesFromFile (std::string &silhouetteFile, TargetPtrVector &targets, std::vector< std::vector< int > > &parentIdxVector)
| Read silhouette data from file.
virtual int | sampleModelFeatures (const TargetPtrVector &targets)
| Do GPU-based visible edge sample point computation.
void | saveSilhouettesToFile (std::string &silhouetteFile, const TargetPtrVector &targets, const std::vector< std::vector< int > > &parentIdxVector)
| Save silhouette data to file.
virtual | ~IntensityEdges ()
| Destructor.
## Public Attributes
opentl::core::cvdata::Image * | m_bgCanny
std::vector< std::vector< int > > | mAngle_ratios
| Ratios (vertical/horizontal) for each angle of the oriented DT.
opentl::core::cvdata::Image * | mImageDebugOutput
| Debug Image output.
std::vector< double > | mNormalsx
| Normal directions for each angle.
std::vector< double > | mNormalsy
std::vector< std::vector < std::vector< std::vector < int > > > > | mSilhNorm
opentl::core::cvdata::Image * | mSilhOutputImage
| Image for restoring the silhouette.
opentl::core::cvdata::Image * | mSilhOutputImageGray
| Image for restoring the silhouette.
std::vector< std::vector < std::vector< std::vector < int > > > > | mSilhPos
## Static Public Attributes
static std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > | mPreProcessOutput_Canny
| Canny edge map.
static std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > | mPreProcessOutput_D1
| Sobel X gradient.
static std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > | mPreProcessOutput_D2
| Sobel Y gradient.
static std::vector < std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > > | mPreProcessOutput_orientedDT
| Oriented distance transform (for each angle).
static std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > | mSilhouetteBinaryImages
| Binary silhouette Image.
static std::vector < boost::shared_ptr < opentl::core::cvdata::Image > > | mSilhouetteImages
| Silhouette Image.

## Detailed Description

Class for contour point feature modality computation. ATTENTION: This class is currently not prepared for multi-threaded match...Level() calls related to a single sensor index!

Author:
  Erwin Roth <eroth@mytum.de>

## Member Enumeration Documentation

enum opentl::modalities::IntensityEdges::FeatDetectFilterType

Possible filter types for image feature detection.

Enumerator:

  CANNY |
  SOBEL |
  CANNY_AND_SOBEL |
  DISTANCE_TRANSFORM |

enum opentl::modalities::IntensityEdges::OfflineParameters

Enumerator:

  generate_contourSampler | (bool) If true, create the contour sampler internally; else, use a pre-defined one (contourSampler)
  construct_loadShaderCodeFromFile | (bool) Load Shader code from File
  construct_sampleSilhouetteEdgesOnly | (bool) Sample silhoutte edged only
  construct_MaxSamplePointsPerTarget | (int) maximum sample points per target
  preProcess_featureFilterType | (FeatDetectFilterType) preprocess filter type
  matchP_glRenderer | Renderer needed if pixel level matching with DISTANCE_TRANSFORM is used.
  subtract_learnedBackground | (bool) If true, subtract the learned background
  OFFLINE_COUNT |

Reimplemented from opentl::modalities::Modality.

enum opentl::modalities::IntensityEdges::OnlineParameters

Enumerator:

  sample_discardBackgroundEdgeWithinPixelRange | (unsigned int) range to discard background edge pixel
  sample_computeScreenPointJacobians | (bool) compute screen point jacobians
  preProcess_cannyThreshold1 | (double) canny threshold1
  preProcess_cannyThreshold2 | (double) canny threshold2
  preProcess_bgOrientThresh | (double) background orient threshold
  preProcess_numofAngles | (unsigned int) number of angles for computing orientedDT
  matchF_alpha | (double) Missing detection rate
  matchF_edgeSearchlengthAlongNormalinPixel | (unsigned int) edge search length along the normal in pixel
  matchF_useRansac | (bool) use RANSAC
  matchF_dotprodThreshold | (double) dot product threshold
  matchF_angleThreshold | (double) angle threshold
  matchF_enableDebugOutput | (bool) enable debug output
  matchFeatLevel_robustFlag | (bool) robust flag
  matchFeatLevel_useFixedCov | (bool) use fixed covariance
  matchF_nearestNeighbor | (bool) nearest neighbour
  matchF_maxMultiHypoAssoc | (unsigned int) Max. number of associations (for multi-hypotheses matching)
  ONLINE_COUNT |

Reimplemented from opentl::modalities::Modality.

## Constructor & Destructor Documentation

opentl::modalities::IntensityEdges::IntensityEdges | ( | const ObjModelPtrVector & | objModels,
|  | opentl::modelprojection::Warp * | warp,
|  | opentl::modelprojection::GLScene * | scene,
|  | int | camIdx,
|  | modelprojection::ContourSampler * | contourSampler = `NULL`,
|  | opentl::modelprojection::GLRenderer * | renderer = `NULL` |
| ) |  |  |

Constructor ATTENTION: The sensor vector element size/sequence must be identical for the warps of all object models !!!

opentl::modalities::IntensityEdges::IntensityEdges | ( | const IntensityEdges & | c | ) |

Deep Copy Constructor.

virtual opentl::modalities::IntensityEdges::~IntensityEdges | ( |  | ) | `[virtual]`

Destructor.

## Member Function Documentation

virtual modalities::Modality* opentl::modalities::IntensityEdges::clone | ( |  | ) | const`[virtual]`

Clone this class and all potential childs RECURSIVELY!! (deep copy).

Implements opentl::modalities::Modality.

opentl::core::cvdata::VisContourSamplePts* opentl::modalities::IntensityEdges::getContourData | ( | int | s = `0` | ) | `[inline]`

virtual void opentl::modalities::IntensityEdges::init | ( |  | ) | `[virtual]`

Allocate data structures required by this class.

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::IntensityEdges::learnBackgroundEdges | ( | const opentl::core::cvdata::Image & | imageGray,
|  | bool | bFirstFrame,
|  | double | bgDistThreshold,
|  | double | cannyT1,
|  | double | cannyT2,
|  | double | bgRunningAvgCoeff |
| ) |  |  |

Learn a model of background edges (position and orientation) for subtraction, during pre-processing.

virtual int opentl::modalities::IntensityEdges::matchFeatLevel | ( | const TargetPtrVector & | targets,
|  | T_MEAS_FEATPtrVector & | outputMeas,
|  | std::size_t | partitionIdx |
| ) |  |  | `[virtual]`

Matching on feature level

- match projected contour points with detected image edges.

Parameters:

  | targets | Vector of targets
  | outputMeas | Feature-space measurements and residuals (one per target and internal state)

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::IntensityEdges::matchPixLevel | ( | const TargetPtrVector & | targets,
|  | T_MEAS_PIXPtrVector & | outputMeas,
|  | std::size_t | partitionIdx |
| ) |  |  | `[virtual]`

Matching on pixel level

- chamfer matching of projected silhouette on distance transformed edge image.

Parameters:

  | targets | Vector of targets
  | outputMeas | Pixel-space measurement (a unique residual value - chamfer distance)

Reimplemented from opentl::modalities::Modality.

virtual int opentl::modalities::IntensityEdges::preProcess | ( | const opentl::core::cvdata::Image & | image,
|  | const std::vector< std::vector< int > > & | preProcessROIs |
| ) |  |  | `[virtual]`

Compute the edge map of I (e.g. with Canny), and store it into mImageFeatDetected.

Parameters:

  | image | Input sensor data (e.g. camera image in RGB)
  | preProcessROIs | Regions of interest (x0,y0,width,height), per target

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::IntensityEdges::readSilhouettesFromFile | ( | std::string & | silhouetteFile,
|  | TargetPtrVector & | targets,
|  | std::vector< std::vector< int > > & | parentIdxVector |
| ) |  |  |

Read silhouette data from file.

Parameters:

  | silhouetteFile | name of the input file
  | targets | Output target vector, containing the grid states (one target per level)
  | parentIdxVector | Output indices of parent cells in the upper level grid (at l=0 it is empty)

virtual int opentl::modalities::IntensityEdges::sampleModelFeatures | ( | const TargetPtrVector & | targets | ) | `[virtual]`

Do GPU-based visible edge sample point computation.

Parameters:

  | targets | Vector of targets

Reimplemented from opentl::modalities::Modality.

void opentl::modalities::IntensityEdges::saveSilhouettesToFile | ( | std::string & | silhouetteFile,
|  | const TargetPtrVector & | targets,
|  | const std::vector< std::vector< int > > & | parentIdxVector |
| ) |  |  |

Save silhouette data to file.

Parameters:

  | silhouetteFile | name of the output file
  | targets | Input target vector, containing the grid states (one target per level)
  | parentIdxVector | Input indices of parent cells in the upper level grid (at l=0 it is empty)

## Member Data Documentation

opentl::core::cvdata::Image* opentl::modalities::IntensityEdges::m_bgCanny

std::vector<std::vector<int> > opentl::modalities::IntensityEdges::mAngle_ratios

Ratios (vertical/horizontal) for each angle of the oriented DT.

opentl::core::cvdata::Image* opentl::modalities::IntensityEdges::mImageDebugOutput

Debug Image output.

std::vector<double> opentl::modalities::IntensityEdges::mNormalsx

Normal directions for each angle.

std::vector<double> opentl::modalities::IntensityEdges::mNormalsy

std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > opentl::modalities::IntensityEdges::mPreProcessOutput_Canny`[static]`

Canny edge map.

std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > opentl::modalities::IntensityEdges::mPreProcessOutput_D1`[static]`

Sobel X gradient.

std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > opentl::modalities::IntensityEdges::mPreProcessOutput_D2`[static]`

Sobel Y gradient.

std::vector<std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > > opentl::modalities::IntensityEdges::mPreProcessOutput_orientedDT`[static]`

Oriented distance transform (for each angle).

std::vector<std::vector<std::vector<std::vector<int> > > > opentl::modalities::IntensityEdges::mSilhNorm

std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > opentl::modalities::IntensityEdges::mSilhouetteBinaryImages`[static]`

Binary silhouette Image.

std::vector<boost::shared_ptr<opentl::core::cvdata::Image> > opentl::modalities::IntensityEdges::mSilhouetteImages`[static]`

Silhouette Image.

opentl::core::cvdata::Image* opentl::modalities::IntensityEdges::mSilhOutputImage

Image for restoring the silhouette.

opentl::core::cvdata::Image* opentl::modalities::IntensityEdges::mSilhOutputImageGray

Image for restoring the silhouette.

std::vector<std::vector<std::vector<std::vector<int> > > > opentl::modalities::IntensityEdges::mSilhPos

Silhouette pixels 1st index: target MODEL (NOTE: two targets with the same model share the same silhouettes) 2nd index: level of the grid 3rd index: index of state cell (at this level) 4th index: mSilhPos: pixel coordinate (as a single array index) / mSilhNorm: normal to the silhouette (index in mAngle_ratios)