Batch

File – Data or No Data

A lot of the work we do for integration is of course automated. Sometimes we have to extract a data set and then use that export as the import to another integration. This is fine and well if there is data but if you have to run it every, let’s say, 5 minutes, there may very well be a situation where there isn’t anything to pick up. The matter is complicated by the fact that even if there isn’t any information there will be a header row.
This file is now completely useless and if left in place it will run through the process and wind up in the archive and produce a results file meaning you have to dig through empty files to try to find what’s going on.
To fix this I came across a batch file that will check the file and delete it if it has only a header row, a further check makes sure that that if any data is there is ignores the file. For your consideration and my reference here is the file that does this.

@echo off
setlocal enableExtensions disableDelayedExpansion

for %%a in ("..\..\data\inbound\*.csv") do (
   call :checkFile "%%~a"
   if errorlevel 1 del %%~a 
   echo File contains no content
)
endlocal
goto :eof


:checkFile
:: %~1   name of the file to check
:: returns 0 if file contains data, else 1 (== header only).

:: if file contains more than one line: data found
for /F "tokens=2 delims=:" %%a in ('find /c /v "" "%~1"') do (
   for /F "delims= " %%b in ("%%~a") do (
      if not 1 == %%~b exit /b 0
   )
)

:: if only line does not start with "Ident", then line contains data
for /F %%a in ('findstr /v "^Ident" "%~1"') do (
   exit /b 0
)

:: header only
exit /b 1
TCC

TCC – Date Certain Step/Status Achieved

I ran into an issue trying to get everyone within the last 24 hours that had achieved a certain step/status combination. The hitch was that they could have been moved out of the target step/status so we have to muck around in the history to get it. As it’s a date you need to make sure the projected value you’re comparing to is a date as well since the last 24 hours is always going to be date time. This did the trick. Remember to pay attention to the value types in sub queries like this.

<quer:filtering xmlns:quer="http://www.taleo.com/ws/integration/query">
	<quer:greaterThan>
		<quer:query projectedClass="Application" alias="sqLocation" preventDuplicates="true" pagingsize="1">
			<quer:projections>
				<quer:projection alias="AnotherDate">
					<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingHistoryItem.CreationDate"/>
				</quer:projection>
			</quer:projections>
			<quer:filterings>
				<quer:filtering>
					<quer:equal>
						<quer:field path="Number"/>
						<quer:field ownerQuery="MainQuery" path="Number"/>
					</quer:equal>
				</quer:filtering>
				<quer:filtering>
					<quer:equal>
						<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingCSWItem.Step,Mnemonic"/>
						<quer:string>RequiredStep</quer:string>
					</quer:equal>
				</quer:filtering>
				<quer:filtering>
					<quer:equal>
						<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingCSWItem.Status,Mnemonic"/>
						<quer:string>RequiredStatus</quer:string>
					</quer:equal>
				</quer:filtering>
				<quer:filtering>
					<quer:equal>
						<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingHistoryItem.CreationDate"/>
						<quer:query projectedClass="Application" alias="sqMotives" preventDuplicates="true" pagingsize="1">
							<quer:subQueries/>
							<quer:projections>
								<quer:projection>
									<quer:maximum>
										<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingHistoryItem.CreationDate"/>
									</quer:maximum>
								</quer:projection>
							</quer:projections>
							<quer:filterings>
								<quer:filtering>
									<quer:equal>
										<quer:field path="Number"/>
										<quer:field ownerQuery="MainQuery" path="Number"/>
									</quer:equal>
								</quer:filtering>
								<quer:filtering>
									<quer:equal>
										<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingCSWItem.Step,Mnemonic"/>
										<quer:string>RequiredStep</quer:string>
									</quer:equal>
								</quer:filtering>
								<quer:filtering>
									<quer:equal>
										<quer:field path="ProfileInformation,HistoryItems,ApplicationTrackingCSWItem.Status,Mnemonic"/>
										<quer:string>RequiredStatus</quer:string>
									</quer:equal>
								</quer:filtering>
							</quer:filterings>
						</quer:query>
					</quer:equal>
				</quer:filtering>
			</quer:filterings>
		</quer:query>
		<quer:castAsDate>
			<quer:addDays>
				<quer:date type="now"/>
				<quer:integer>-1</quer:integer>
			</quer:addDays>
		</quer:castAsDate>
	</quer:greaterThan>
</quer:filtering>
PowerShell

PowerShell – Replace in Files

One of the hard parts about working on remote servers is they don’t usually have the proper toolkit to fix things as needed, meaning you have to make the changes in the development workstation, repackage and post, pull, copy, extract and paste in order to get anything done. Fine if it’s the initial load but when you only need to change something in a dozen files or so it can be tedious. So in the spirit of making due with what you have, you can use PowerShell to emulate the replace in file normally invoked from Notepad++ to work around it.
Here is the example:

foreach ($file in Get-ChildItem FileMask*.bat)
{
powershell -Command "(gc $file) -replace 'Text1', 'Text2' | Out-File -encoding UTF8 $file"
}
TCC

TCC – Setting a Select Value Back to ‘Not Specified’

This is a common request I run across. You’ve got a selection list and you need to reset it back to ‘Not Specified’ for whatever reason.
Although I swear I’ve done this before, every time it comes up I’m scratching my head trying to figure out what I did the last time it came up.
As I was looking for help on another issue I ran across Doc ID 2478146.1 on MOS that spells out the ‘correct’ way to do it, leaving it here for my reference: read more »

TCC

TCC – Following a CSW

I’m sure we’ve all had that wonderful fun time trying to figure out what steps and statuses and what order they are supposed to go in. With same named CSW’s and the other craziness it can certainly be frustrating. But there is hope, a properly crafted CSWWorkflow export will reveal all. Simply name a blank file: CSWWorkflowExport_sq.xml
Then add the following code:

<?xml version="1.0" encoding="UTF-8"?>
<quer:query productCode="RC1704" model="http://www.taleo.com/ws/tee800/2009/01" projectedClass="CSWWorkflow" locale="en" mode="CSV" csvheader="true" largegraph="true" preventDuplicates="false" xmlns:quer="http://www.taleo.com/ws/integration/query">
	<quer:subQueries/>
	<quer:projections>
		<quer:projection>
            <quer:field path="Number"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="Mnemonic"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="Name"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="Description"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="Available"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="IsDefault"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Active"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,ConditionalEntryAllowed"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Mandatory"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,NewEntryNotificationNumber"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Restricted"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Sequence"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,Number"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,Mnemonic"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,Name"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,Description"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,Available"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,ShortName"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Active"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,AllConstraintsMustBeValid"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,AutoFlow"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,ConditionnalCompletenessAllowed"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Initial"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,MeetingCompliant"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,NewEntryNotificationNumber"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,QueueAndRouteShortcutNumber"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,RsCSWAskForCommentModeNumber"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,RsCSWAskForMotiveModeNumber"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Sequence"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Status,CSWStatus.Mnemonic"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Status,CSWStatus.Name"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Status,CSWStatus.Description"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Status,CSWStatus.ActiveEventDate"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Status,CSWStatus.Available"/>
        </quer:projection>
		<quer:projection>
            <quer:field path="StepUsage,Step,StatusUsages,Status,CSWStatus.Number"/>
        </quer:projection>
	</quer:projections>
	<quer:projectionFilterings/>
	<quer:filterings/>
	<quer:sortings>
		<quer:sorting ascending="true">
            <quer:field path="Mnemonic"/>
        </quer:sorting>
		<quer:sorting ascending="true">
            <quer:field path="StepUsage,Sequence"/>
        </quer:sorting>
		<quer:sorting ascending="true">
            <quer:field path="StepUsage,Step,StatusUsages,Sequence"/>
        </quer:sorting>
	</quer:sortings>
	<quer:sortingFilterings/>
	<quer:groupings/>
	<quer:joinings/>
</quer:query>
TCC

TCC – Data Dictionaries

These are the last full data dictionaries for Taleo. Unfortunately 15a is the last version available but I’ve honestly rarely, if ever, had to look any further than this documentation.

MySQL

MySQL – New Database > New User

I’m trying to get a little more granular on DB security in Linux and to that end used the MariaDB root user for all LAMP installations is probably not the brightest idea. I’ll try to commit this to memory but until then I’ll follow these steps to create a new database and then a new user with full access to that DB thereby isolating the databases from each other so one breach doesn’t take down the system.

mysql -u root -p

CREATE DATABASE newDatabase;

GRANT ALL PRIVILEGES ON newDatabase.* TO 'newUser'@'localhost' IDENTIFIED BY 'newPassword' WITH GRANT OPTION;

FLUSH PRIVILEGES;

Python

Python – CSV to Pipe delimiter

So why would a perfectly sane person want to have a pipe delimited CSV verses a properly escaped comma separated value file? Yeah beats me too, glutton for punishment, not working with the right parser, drug addiction? All could be reasons. But fear not, if this is the road you’d like to go down there’s a python script for it and here it is so I don’t have to recreate the wheel next time I need it.

import glob
import csv
import os

for entry in glob.glob('applicationResumeAttachmentsManifest/*.csv'):
    #outputFile = (entry.strip(".csv")+"-Pipe.csv")
    outputFile = ("C:\\python\\pipe\\"+entry.strip(".csv")+"-Pipe.csv")
    os.makedirs(os.path.dirname(outputFile), exist_ok=True)
    #print(entry)
    #print(outputFile)
    with open(entry, encoding='utf-8') as inputFile:
        with open(outputFile, 'w', encoding='utf-8', newline='') as writeFile:
            reader = csv.DictReader(inputFile, delimiter=',')
            writer = csv.DictWriter(writeFile, reader.fieldnames, delimiter='|')
            writer.writeheader()
            writer.writerows(reader)
print("Conversion complete.")
Python

Python

I know that the current buzzword for IT is python and I’m going to have to jump on that bandwagon. In my role as an integration engineer, there are a lot of times that I have to manipulate file data and I’m super impressed by the ease and power of this language.

So to this end I’ll be creating a category for python on this blog and storing code snippets that I find handy and useful. Hope you find it as helpful and enjoyable as I do.

I know a lot of people hear python and think, yeah, a big snake that will constrict you until you’re dead and then swallow you whole. A perfect simile for programming. But here’s the truth, Guido van Rossum developed the language starting in 1989 and the name python comes not from the snake but Monty Python’s Flying Circus which the developers loved to watch while they were coming up with the language. Knowledge is power and sometimes funny too.

TCC

TCC – ORA-01791: not a SELECTed expression

Love running into this error. What happens here is that I’m doing an export and I want it to return distinct rows and I want to sort it by a date, but I’ve modified the date to format it how I want to see it. For some reason when we preventDuplicates=”true” it sees that the date (having been converted) as not a selected expression.

So the trick is, on your sorting, you have to use the same custom function toChar to get it to sort by the order you want it to. Kind of a pain but in the end of the equation it makes sense that both the projection and the sort by are the same, too bad you can’t just use the alias…