|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectcom.taleo.integration.client.step.BaseStep
com.taleo.integration.client.step.BaseCustomStep
com.taleo.integration.client.customstep.BaseCustomStep
com.taleo.integration.client.customstep.csv.BaseCustomCsvStep
com.taleo.integration.client.customstep.csv.BaseCustomCsvPostStep
com.taleo.integration.client.customstep.csv.MergePostStep
public class MergePostStep
Post-processing step to merge result CSV files. Files corresponding to a pattern will be merged with the current result and be set as the current document in the execution pipeline. This step also allows removing duplicates while merging.
This step takes all parameters from its base class
BaseCustomCsvPostStep
and
the following additional parameters:
WildcardFileFilter
, in the file
names only. Pattern is case sensitive. If the pattern includes multiple
files, all of them will be merged in sequence. If absent, the current result
filename itself is used, so instead of overwriting the existing result file,
it will be merged with the current one.
Multiple patterns can be set by separating them with a pipe (|). The special dot (.) pattern represents the current result filename.
true
).false
).keys
parameter or by the "key" value being present in the tag
attribute of projections in the SQ-XML query.keys
parameter has priority over the "key" tag and if no key
is specified, no sorting occurs.true
).sort
parameter.parentKeys
parameter or by the "parentKey"
value being present in the tag attribute of projections in the SQ-XML query.parentKeys
parameter has priority over the "parentKey" tag.
Also, if the keys
parameter is present, "parentKey" tag is
ignored.filePattern
, those files should be named so that
sorting them alphabetically also means chronologically, using a time stamp
like "yyyyMMdd HHmmss". This way the newer records will be further down in
the merged file and if duplicates are removed the most recent records will be
kept. The current result is always the most recent so will be necessarily at
the end, replacing any previous duplicate. removeDuplicates
parameter.removeDuplicates
parameter.filePattern
have been
produced by this same step and are thus sorted already. If this is not the
case, then set this parameter to true
to force sorting them
appropriately. (Default: false
).Note: Removing duplicates requires sorting the file according to key fields. This is an unavoidable side effect of this step.
This class is dependent on:
Field Summary |
---|
Fields inherited from class com.taleo.integration.client.customstep.BaseCustomStep |
---|
parameterNames |
Fields inherited from class com.taleo.integration.client.step.BaseCustomStep |
---|
parameters |
Fields inherited from class com.taleo.integration.client.step.BaseStep |
---|
commType, ERROR_NULL_GLOBAL_CONFIG, ERROR_NULL_PIPELINE, ERROR_NULL_STEP_CONFIG, productCode, tempFolder, type, version |
Constructor Summary | |
---|---|
MergePostStep()
Constructor without arguments. |
Method Summary | |
---|---|
protected java.io.File |
childMerge(java.io.File inFile)
Merge the files corresponding to the filePattern with the provided inFile. |
void |
execute(com.taleo.ws.integration.client.Pipeline pipeline)
|
java.lang.String |
getDescription()
|
int |
getExtraFooterRows()
Get the extra footer rows count. |
int |
getExtraHeaderRows()
Get the extra header rows count. |
java.lang.String |
getFilePattern()
Get the file pattern. |
java.lang.String |
getIdentifier()
|
java.lang.String[] |
getKeys()
Get keys. |
protected java.lang.String[] |
getKeysFromTag(java.io.File queryFile,
java.lang.String tagName)
Get the key indexes from tag attribute of projections in the SQ-XML query. |
java.lang.String |
getName()
|
java.lang.String[] |
getParentKeys()
Get parent keys. |
void |
init(com.taleo.ws.integration.client.GlobalConfig config)
|
boolean |
isDeleteMergedFiles()
Get the deleteMergedFiles flag. |
boolean |
isForceSortMatchingFiles()
Get the forceSortMatchingFiles flag. |
boolean |
isRemoveDuplicates()
Get the removeDuplicates flag. |
boolean |
isSort()
Get the sort flag. |
protected java.io.File |
merge(java.io.File inFile)
Merge the files corresponding to the filePattern with the provided inFile. |
void |
setDeleteMergedFiles(boolean deleteMergedFiles)
Set the deleteMergedFiles flag. |
void |
setExtraFooterRows(int extraFooterRows)
Set the extra footer rows count. |
void |
setExtraHeaderRows(int extraHeaderRows)
Set the extra header rows count. |
void |
setFilePattern(java.lang.String filePattern)
Set the file pattern. |
void |
setForceSortMatchingFiles(boolean forceSortMatchingFiles)
Set the forceSortMatchingFiles flag. |
void |
setKeys(java.lang.String[] keys)
Set keys. |
void |
setParentKeys(java.lang.String[] parentKeys)
Set parent keys. |
void |
setRemoveDuplicates(boolean removeDuplicates)
Set the removeDuplicates flag. |
void |
setSort(boolean sort)
Set the sort flag. |
protected void |
sort(java.io.File inFile,
java.io.File outFile,
int[] indexes,
boolean removeDuplicates)
Sort the inFile and write result to the outFile, removing duplicates if required. |
Methods inherited from class com.taleo.integration.client.customstep.csv.BaseCustomCsvPostStep |
---|
getSupportedPipeline |
Methods inherited from class com.taleo.integration.client.customstep.csv.BaseCustomCsvStep |
---|
getColumnIndex, getColumnName, getCsvDelimiter, getCsvQuoteCharacter, isCsvHeaderPresent, setCsvDelimiter, setCsvHeaderPresent, setCsvQuoteCharacter |
Methods inherited from class com.taleo.integration.client.customstep.BaseCustomStep |
---|
createTempFile, createTempFile, getEncoding, getTempFolder, isActive, registerParameterName, setActive, setEncoding, validateParameterNames |
Methods inherited from class com.taleo.integration.client.step.BaseCustomStep |
---|
getType, init |
Methods inherited from class com.taleo.integration.client.step.BaseStep |
---|
getAllProcessSupportPipeline, getCurrentFile, getCurrentFile, getCurrentMessage, getPostProcessSupportPipeline, getPreProcessSupportPipeline, isOriginalFile, validateLastStepType, validateMessageType, validatePipeline |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface com.taleo.integration.client.step.Step |
---|
validatePipeline |
Constructor Detail |
---|
public MergePostStep()
Method Detail |
---|
public java.lang.String getFilePattern()
public void setFilePattern(java.lang.String filePattern)
filePattern
- public boolean isDeleteMergedFiles()
public void setDeleteMergedFiles(boolean deleteMergedFiles)
deleteMergedFiles
- public boolean isSort()
public void setSort(boolean sort)
sort
- public boolean isRemoveDuplicates()
public void setRemoveDuplicates(boolean removeDuplicates)
removeDuplicates
- public java.lang.String[] getKeys()
public void setKeys(java.lang.String[] keys)
keys
- public java.lang.String[] getParentKeys()
public void setParentKeys(java.lang.String[] parentKeys)
parentKeys
- public boolean isForceSortMatchingFiles()
public void setForceSortMatchingFiles(boolean forceSortMatchingFiles)
forceSortMatchingFiles
- public int getExtraHeaderRows()
public void setExtraHeaderRows(int extraHeaderRows)
extraHeaderRows
- public int getExtraFooterRows()
public void setExtraFooterRows(int extraFooterRows)
extraFooterRows
- public java.lang.String getIdentifier()
getIdentifier
in interface com.taleo.integration.client.step.CustomStep
getIdentifier
in interface com.taleo.integration.client.step.Step
getIdentifier
in class com.taleo.integration.client.step.BaseCustomStep
public java.lang.String getDescription()
getDescription
in interface com.taleo.integration.client.step.CustomStep
getDescription
in interface com.taleo.integration.client.step.Step
getDescription
in class com.taleo.integration.client.step.BaseCustomStep
public java.lang.String getName()
getName
in interface com.taleo.integration.client.step.CustomStep
getName
in interface com.taleo.integration.client.step.Step
getName
in class com.taleo.integration.client.step.BaseCustomStep
public void init(com.taleo.ws.integration.client.GlobalConfig config)
init
in class BaseCustomCsvStep
public void execute(com.taleo.ws.integration.client.Pipeline pipeline) throws com.taleo.integration.client.step.StepException
execute
in interface com.taleo.integration.client.step.Step
execute
in class BaseCustomCsvStep
com.taleo.integration.client.step.StepException
protected java.io.File merge(java.io.File inFile) throws java.io.IOException, InvalidCSVFileFormat
inFile
- The input file
java.io.IOException
InvalidCSVFileFormat
protected java.io.File childMerge(java.io.File inFile) throws java.io.IOException, InvalidCSVFileFormat
Note: This strategy is relevant only if duplicates must be removed. If it is not so, a simple merge is performed.
inFile
- The input file
java.io.IOException
InvalidCSVFileFormat
protected void sort(java.io.File inFile, java.io.File outFile, int[] indexes, boolean removeDuplicates) throws java.io.IOException, InvalidCSVFileFormat
inFile
- The input fileoutFile
- The output fileindexes
- The sort column indexesremoveDuplicates
- Remove duplicates
java.io.IOException
InvalidCSVFileFormat
protected java.lang.String[] getKeysFromTag(java.io.File queryFile, java.lang.String tagName) throws javax.xml.transform.TransformerException, java.io.IOException
queryFile
- The query file.tagName
- The tag name identifying the keys.
javax.xml.transform.TransformerException
java.io.IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |