com.taleo.integration.client.customstep.csv
Class MergePostStep

java.lang.Object
  extended by com.taleo.integration.client.step.BaseStep
      extended by com.taleo.integration.client.step.BaseCustomStep
          extended by com.taleo.integration.client.customstep.BaseCustomStep
              extended by com.taleo.integration.client.customstep.csv.BaseCustomCsvStep
                  extended by com.taleo.integration.client.customstep.csv.BaseCustomCsvPostStep
                      extended by com.taleo.integration.client.customstep.csv.MergePostStep
All Implemented Interfaces:
com.taleo.integration.client.step.CustomStep, com.taleo.integration.client.step.Step

public class MergePostStep
extends BaseCustomCsvPostStep

Post-processing step to merge result CSV files. Files corresponding to a pattern will be merged with the current result and be set as the current document in the execution pipeline. This step also allows removing duplicates while merging.

This step takes all parameters from its base class BaseCustomCsvPostStep and the following additional parameters:

Note: Removing duplicates requires sorting the file according to key fields. This is an unavoidable side effect of this step.

This class is dependent on:

Author:
Romain Guay, Taleo Corporation

Field Summary
 
Fields inherited from class com.taleo.integration.client.customstep.BaseCustomStep
parameterNames
 
Fields inherited from class com.taleo.integration.client.step.BaseCustomStep
parameters
 
Fields inherited from class com.taleo.integration.client.step.BaseStep
commType, ERROR_NULL_GLOBAL_CONFIG, ERROR_NULL_PIPELINE, ERROR_NULL_STEP_CONFIG, productCode, tempFolder, type, version
 
Constructor Summary
MergePostStep()
          Constructor without arguments.
 
Method Summary
protected  java.io.File childMerge(java.io.File inFile)
          Merge the files corresponding to the filePattern with the provided inFile.
 void execute(com.taleo.ws.integration.client.Pipeline pipeline)
           
 java.lang.String getDescription()
           
 int getExtraFooterRows()
          Get the extra footer rows count.
 int getExtraHeaderRows()
          Get the extra header rows count.
 java.lang.String getFilePattern()
          Get the file pattern.
 java.lang.String getIdentifier()
           
 java.lang.String[] getKeys()
          Get keys.
protected  java.lang.String[] getKeysFromTag(java.io.File queryFile, java.lang.String tagName)
          Get the key indexes from tag attribute of projections in the SQ-XML query.
 java.lang.String getName()
           
 java.lang.String[] getParentKeys()
          Get parent keys.
 void init(com.taleo.ws.integration.client.GlobalConfig config)
           
 boolean isDeleteMergedFiles()
          Get the deleteMergedFiles flag.
 boolean isForceSortMatchingFiles()
          Get the forceSortMatchingFiles flag.
 boolean isRemoveDuplicates()
          Get the removeDuplicates flag.
 boolean isSort()
          Get the sort flag.
protected  java.io.File merge(java.io.File inFile)
          Merge the files corresponding to the filePattern with the provided inFile.
 void setDeleteMergedFiles(boolean deleteMergedFiles)
          Set the deleteMergedFiles flag.
 void setExtraFooterRows(int extraFooterRows)
          Set the extra footer rows count.
 void setExtraHeaderRows(int extraHeaderRows)
          Set the extra header rows count.
 void setFilePattern(java.lang.String filePattern)
          Set the file pattern.
 void setForceSortMatchingFiles(boolean forceSortMatchingFiles)
          Set the forceSortMatchingFiles flag.
 void setKeys(java.lang.String[] keys)
          Set keys.
 void setParentKeys(java.lang.String[] parentKeys)
          Set parent keys.
 void setRemoveDuplicates(boolean removeDuplicates)
          Set the removeDuplicates flag.
 void setSort(boolean sort)
          Set the sort flag.
protected  void sort(java.io.File inFile, java.io.File outFile, int[] indexes, boolean removeDuplicates)
          Sort the inFile and write result to the outFile, removing duplicates if required.
 
Methods inherited from class com.taleo.integration.client.customstep.csv.BaseCustomCsvPostStep
getSupportedPipeline
 
Methods inherited from class com.taleo.integration.client.customstep.csv.BaseCustomCsvStep
getColumnIndex, getColumnName, getCsvDelimiter, getCsvQuoteCharacter, isCsvHeaderPresent, setCsvDelimiter, setCsvHeaderPresent, setCsvQuoteCharacter
 
Methods inherited from class com.taleo.integration.client.customstep.BaseCustomStep
createTempFile, createTempFile, getEncoding, getTempFolder, isActive, registerParameterName, setActive, setEncoding, validateParameterNames
 
Methods inherited from class com.taleo.integration.client.step.BaseCustomStep
getType, init
 
Methods inherited from class com.taleo.integration.client.step.BaseStep
getAllProcessSupportPipeline, getCurrentFile, getCurrentFile, getCurrentMessage, getPostProcessSupportPipeline, getPreProcessSupportPipeline, isOriginalFile, validateLastStepType, validateMessageType, validatePipeline
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface com.taleo.integration.client.step.Step
validatePipeline
 

Constructor Detail

MergePostStep

public MergePostStep()
Constructor without arguments.

Method Detail

getFilePattern

public java.lang.String getFilePattern()
Get the file pattern.

Returns:

setFilePattern

public void setFilePattern(java.lang.String filePattern)
Set the file pattern.

Parameters:
filePattern -

isDeleteMergedFiles

public boolean isDeleteMergedFiles()
Get the deleteMergedFiles flag.

Returns:

setDeleteMergedFiles

public void setDeleteMergedFiles(boolean deleteMergedFiles)
Set the deleteMergedFiles flag.

Parameters:
deleteMergedFiles -

isSort

public boolean isSort()
Get the sort flag.

Returns:

setSort

public void setSort(boolean sort)
Set the sort flag.

Parameters:
sort -

isRemoveDuplicates

public boolean isRemoveDuplicates()
Get the removeDuplicates flag.

Returns:

setRemoveDuplicates

public void setRemoveDuplicates(boolean removeDuplicates)
Set the removeDuplicates flag.

Parameters:
removeDuplicates -

getKeys

public java.lang.String[] getKeys()
Get keys.

Returns:

setKeys

public void setKeys(java.lang.String[] keys)
Set keys.

Parameters:
keys -

getParentKeys

public java.lang.String[] getParentKeys()
Get parent keys.

Returns:

setParentKeys

public void setParentKeys(java.lang.String[] parentKeys)
Set parent keys.

Parameters:
parentKeys -

isForceSortMatchingFiles

public boolean isForceSortMatchingFiles()
Get the forceSortMatchingFiles flag.

Returns:

setForceSortMatchingFiles

public void setForceSortMatchingFiles(boolean forceSortMatchingFiles)
Set the forceSortMatchingFiles flag.

Parameters:
forceSortMatchingFiles -

getExtraHeaderRows

public int getExtraHeaderRows()
Get the extra header rows count.

Returns:

setExtraHeaderRows

public void setExtraHeaderRows(int extraHeaderRows)
Set the extra header rows count.

Parameters:
extraHeaderRows -

getExtraFooterRows

public int getExtraFooterRows()
Get the extra footer rows count.

Returns:

setExtraFooterRows

public void setExtraFooterRows(int extraFooterRows)
Set the extra footer rows count.

Parameters:
extraFooterRows -

getIdentifier

public java.lang.String getIdentifier()
Specified by:
getIdentifier in interface com.taleo.integration.client.step.CustomStep
Specified by:
getIdentifier in interface com.taleo.integration.client.step.Step
Specified by:
getIdentifier in class com.taleo.integration.client.step.BaseCustomStep

getDescription

public java.lang.String getDescription()
Specified by:
getDescription in interface com.taleo.integration.client.step.CustomStep
Specified by:
getDescription in interface com.taleo.integration.client.step.Step
Specified by:
getDescription in class com.taleo.integration.client.step.BaseCustomStep

getName

public java.lang.String getName()
Specified by:
getName in interface com.taleo.integration.client.step.CustomStep
Specified by:
getName in interface com.taleo.integration.client.step.Step
Specified by:
getName in class com.taleo.integration.client.step.BaseCustomStep

init

public void init(com.taleo.ws.integration.client.GlobalConfig config)
Overrides:
init in class BaseCustomCsvStep

execute

public void execute(com.taleo.ws.integration.client.Pipeline pipeline)
             throws com.taleo.integration.client.step.StepException
Specified by:
execute in interface com.taleo.integration.client.step.Step
Overrides:
execute in class BaseCustomCsvStep
Throws:
com.taleo.integration.client.step.StepException

merge

protected java.io.File merge(java.io.File inFile)
                      throws java.io.IOException,
                             InvalidCSVFileFormat
Merge the files corresponding to the filePattern with the provided inFile.

Parameters:
inFile - The input file
Returns:
The merged file.
Throws:
java.io.IOException
InvalidCSVFileFormat

childMerge

protected java.io.File childMerge(java.io.File inFile)
                           throws java.io.IOException,
                                  InvalidCSVFileFormat
Merge the files corresponding to the filePattern with the provided inFile. This method applies the merge considering the parent keys. To do so, it prepares the outFile by making one row per parent and all its children in a single Base64 encoded field. It then sorts on the parent keys and finally expands back the children in the output file.

Note: This strategy is relevant only if duplicates must be removed. If it is not so, a simple merge is performed.

Parameters:
inFile - The input file
Returns:
The merged file.
Throws:
java.io.IOException
InvalidCSVFileFormat

sort

protected void sort(java.io.File inFile,
                    java.io.File outFile,
                    int[] indexes,
                    boolean removeDuplicates)
             throws java.io.IOException,
                    InvalidCSVFileFormat
Sort the inFile and write result to the outFile, removing duplicates if required.

Parameters:
inFile - The input file
outFile - The output file
indexes - The sort column indexes
removeDuplicates - Remove duplicates
Throws:
java.io.IOException
InvalidCSVFileFormat

getKeysFromTag

protected java.lang.String[] getKeysFromTag(java.io.File queryFile,
                                            java.lang.String tagName)
                                     throws javax.xml.transform.TransformerException,
                                            java.io.IOException
Get the key indexes from tag attribute of projections in the SQ-XML query.

Parameters:
queryFile - The query file.
tagName - The tag name identifying the keys.
Returns:
Throws:
javax.xml.transform.TransformerException
java.io.IOException