Page 1 of 2
Split from flat file
Posted: Thu Sep 27, 2012 4:14 pm
by trotta
I have a project where I need to split a .txt file. Each line is a different record so it can not be split anywhere in the line. The first character represents part of a set. A complete set has as it's first character of the record either a 1, 2, 3, 4, or 5, always in that order. There may be many 2s in one set. The sets can not be split either, only after a 5.
In addition it would have at least 20,000 lines in each split file.
Can RoboTask do this?
Split from flat file
Posted: Thu Sep 27, 2012 4:45 pm
by Oleg
Can RoboTask do this?
I think yes.
But I need the example of the file, and examples of the result parts in order to describe algorithm for this.
Split from flat file
Posted: Thu Sep 27, 2012 5:02 pm
by trotta
The data in the first file looks simular to this only longer and more 2 lines:
112 $0.561133096 S KENNEDY
2000107JOHN & nbsp; DOE
2000108JOHN & nbsp; DOE
2000109JUDD & nbsp; DOE
3000110
4000111
5000112
134 $0.681133096 S WEST SE
2000114JANE & nbsp; DOE
2000115JANE & nbsp; DOE
3000222
4000223
5000224
105 $0.441133096 S ALBERT
2000226JOHN & nbsp; DOE
2000227JOHN & nbsp; DOE
2000228JANE & nbsp; DOE
2000229JOHN & nbsp; DOE
3000252
4000253
5000254
140 $0.851133096 S BA ACAD
2000256JOHN & nbsp; DOE
3000410
4000411
5000412
The split file would look the same except not having as many sets. Instead of having 100,000 lines, it would have approximately 20,000 lines but the file can only be split between the 5 line and the 1 line. Like this:
File 1
112 $0.561133096 S KENNEDY
2000107JOHN & nbsp; DOE
2000108JOHN & nbsp; DOE
2000109JUDD & nbsp; DOE
3000110
4000111
5000112
134 $0.681133096 S WEST SE
2000114JANE & nbsp; DOE
2000115JANE & nbsp; DOE
3000222
4000223
5000224
File 2
105 $0.441133096 S ALBERT
2000226JOHN & nbsp; DOE
2000227JOHN & nbsp; DOE
2000228JANE & nbsp; DOE
2000229JOHN & nbsp; DOE
3000252
4000253
5000254
140 $0.851133096 S BA ACAD
2000256JOHN & nbsp; DOE
3000410
4000411
5000412
File 3
112 $0.561133096 S KENNEDY
2000107JOHN & nbsp; DOE
2000108JOHN & nbsp; DOE
2000109JUDD & nbsp; DOE
3000110
4000111
5000112
And so on.
Split from flat file
Posted: Thu Sep 27, 2012 7:45 pm
by Oleg
I'm sorry, I haven't understood why file1 and file2 contains two blocks of information but file3 contains only one block.
I think than one block contains lines from 1 to 5.
Should result files contain two blocks per file? Or maybe I have missed something in logic...
Split from flat file
Posted: Thu Sep 27, 2012 7:54 pm
by trotta
An actual file may have hundreds of blocks from 1 to 5.
The file I would like to split has thousands of the blocks. I need them in hundreds of blocks rather than thousands.
Does that help?
Split from flat file
Posted: Thu Sep 27, 2012 8:08 pm
by Oleg
In other words you want only to decrease the size of file.
It doesn't matter how many blocks will be contained in result file.
For example you can specify this before task.
Do I understand correctly?
If so I'll try to write an example a bit later.
Split from flat file
Posted: Fri Sep 28, 2012 1:03 pm
by trotta
Yes that is correct.
Split from flat file
Posted: Fri Sep 28, 2012 3:01 pm
by Oleg
See the example below.
This task splits file D:\Temp\txt\big.txt
to files:
D:\Temp\txt\small1.txt
D:\Temp\txt\small2.txt
...
D:\Temp\txt\smallN.txt
You can set maximum amount of blocks at variable MAXBLOCK. (Now it is 5)
Code: Select all
;*****************************
;* RoboTask Task file
;* Do not edit in text editor!
;*****************************
[Root]
ActionAfterRun=INTEGER|0
Actions=FOLDER
Automat=INTEGER|-1
CatID=INTEGER|2099193306
Comment=STRINGLIST
ContinueOnError=INTEGER|0
ExternalName=STRING|"Task20"
Hide=INTEGER|0
ID=INTEGER|1216194314
LocalVariables=STRING|"line,numblock,parttext,type,maxblock, numfile"
LogOnAsUser=INTEGER|1
Name=STRING|"Split files"
OnErrorTaskID=INTEGER|0
Priority=INTEGER|3
RunOnClose=INTEGER|0
RunOnStartup=INTEGER|0
ToLog=INTEGER|3
WriteGeneralLog=INTEGER|0
[Actions]
Action1=FOLDER
Action10=FOLDER
Action11=FOLDER
Action12=FOLDER
Action13=FOLDER
Action14=FOLDER
Action15=FOLDER
Action16=FOLDER
Action17=FOLDER
Action18=FOLDER
Action19=FOLDER
Action2=FOLDER
Action20=FOLDER
Action21=FOLDER
Action3=FOLDER
Action4=FOLDER
Action5=FOLDER
Action6=FOLDER
Action7=FOLDER
Action8=FOLDER
Action9=FOLDER
[Actions\Action1]
ActionID=STRING|"A_ROBOTASK_MINIMIZE"
Enabled=INTEGER|-1
Name=STRING|"Minimize RoboTask"
[Actions\Action10]
ActionID=STRING|"A_DIALOG_MESSAGE"
Enabled=INTEGER|0
Name=STRING|"Show ""{parttext}"""
Params=FOLDER
[Actions\Action10\Params]
icon=STRING|"1"
msg0=STRING|"{parttext}"
msgcount=STRING|"1"
playsound=STRING|"0"
showmessage=STRING|"1"
[Actions\Action11]
ActionID=STRING|"A_FILE_TEXTWRITE"
Enabled=INTEGER|-1
Name=STRING|"Create text file D:\Temp\txt\small{numfile}.txt"
Params=FOLDER
[Actions\Action11\Params]
fileexists=STRING|"0"
filname=STRING|"D:\Temp\txt\small{numfile}.txt"
line0=STRING|"{parttext}"
linecount=STRING|"1"
suppress=STRING|"1"
[Actions\Action12]
ActionID=STRING|"A_VARIABLES_INCREMENT"
Enabled=INTEGER|-1
Name=STRING|"Increment variable ""numfile"""
Params=FOLDER
[Actions\Action12\Params]
incement=STRING|"1"
vaiable=STRING|"numfile"
[Actions\Action13]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""NUMBLOCK"" with value ""1"""
Params=FOLDER
[Actions\Action13\Params]
expand=STRING|"0"
varname=STRING|"NUMBLOCK"
varvalue=STRING|"1"
[Actions\Action14]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""PARTTEXT"" with value """""
Params=FOLDER
[Actions\Action14\Params]
expand=STRING|"0"
varname=STRING|"PARTTEXT"
[Actions\Action15]
ActionID=STRING|"A_FLOW_ENDIF"
Enabled=INTEGER|-1
Name=STRING|"End If"
[Actions\Action16]
ActionID=STRING|"A_FLOW_ENDIF"
Enabled=INTEGER|-1
Name=STRING|"End If"
[Actions\Action17]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""PARTTEXT"" with value ""{parttext}{line}{eol}"""
Params=FOLDER
[Actions\Action17\Params]
expand=STRING|"1"
varname=STRING|"PARTTEXT"
varvalue=STRING|"{parttext}{line}{eol}"
[Actions\Action18]
ActionID=STRING|"A_LOOP_END"
Enabled=INTEGER|-1
Name=STRING|"End Loop"
[Actions\Action19]
ActionID=STRING|"A_DIALOG_MESSAGE"
Enabled=INTEGER|0
Name=STRING|"Show ""{parttext}"""
Params=FOLDER
[Actions\Action19\Params]
icon=STRING|"1"
msg0=STRING|"{parttext}"
msgcount=STRING|"1"
playsound=STRING|"0"
showmessage=STRING|"1"
[Actions\Action2]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""NUMBLOCK"" with value ""0"""
Params=FOLDER
[Actions\Action2\Params]
expand=STRING|"0"
varname=STRING|"NUMBLOCK"
varvalue=STRING|"0"
[Actions\Action20]
ActionID=STRING|"A_FILE_TEXTWRITE"
Enabled=INTEGER|-1
Name=STRING|"Create text file D:\Temp\txt\small{numfile}.txt"
Params=FOLDER
[Actions\Action20\Params]
fileexists=STRING|"0"
filname=STRING|"D:\Temp\txt\small{numfile}.txt"
line0=STRING|"{parttext}"
linecount=STRING|"1"
suppress=STRING|"1"
[Actions\Action21]
ActionID=STRING|"A_DIALOG_MESSAGE"
Enabled=INTEGER|-1
Name=STRING|"Show ""Done!"""
Params=FOLDER
[Actions\Action21\Params]
icon=STRING|"1"
msg0=STRING|"Done!"
msg1=STRING|"Total {numfile} files"
msgcount=STRING|"2"
playsound=STRING|"0"
showmessage=STRING|"1"
[Actions\Action3]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""MAXBLOCK"" with value ""5"""
Params=FOLDER
[Actions\Action3\Params]
expand=STRING|"0"
varname=STRING|"MAXBLOCK"
varvalue=STRING|"5"
[Actions\Action4]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""NUMFILE"" with value ""1"""
Params=FOLDER
[Actions\Action4\Params]
expand=STRING|"0"
varname=STRING|"NUMFILE"
varvalue=STRING|"1"
[Actions\Action5]
ActionID=STRING|"A_LOOP_TEXT"
Enabled=INTEGER|-1
Name=STRING|"Text Loop"
Params=FOLDER
[Actions\Action5\Params]
destvar=STRING|"LINE"
file=STRING|"D:\Temp\txt\big.txt"
linecount=STRING|"0"
sourcetext=STRING|"0"
[Actions\Action6]
ActionID=STRING|"A_STR_EXTRACT"
Enabled=INTEGER|-1
Name=STRING|"STR Extract"
Params=FOLDER
[Actions\Action6\Params]
amount=STRING|"1"
from=STRING|"1"
source=STRING|"{line}"
variable=STRING|"type"
[Actions\Action7]
ActionID=STRING|"A_FLOW_IF"
Enabled=INTEGER|-1
Name=STRING|"If Then"
Params=FOLDER
[Actions\Action7\Params]
case=STRING|"0"
operator=STRING|"0"
type=STRING|"0"
value1=STRING|"{type}"
value2=STRING|"1"
[Actions\Action8]
ActionID=STRING|"A_VARIABLES_INCREMENT"
Enabled=INTEGER|-1
Name=STRING|"Increment variable ""numblock"""
Params=FOLDER
[Actions\Action8\Params]
incement=STRING|"1"
vaiable=STRING|"numblock"
[Actions\Action9]
ActionID=STRING|"A_FLOW_IF"
Enabled=INTEGER|-1
Name=STRING|"If Then"
Params=FOLDER
[Actions\Action9\Params]
case=STRING|"0"
operator=STRING|"3"
type=STRING|"0"
value1=STRING|"{numblock}"
value2=STRING|"{maxblock}"
Split from flat file
Posted: Fri Sep 28, 2012 6:09 pm
by trotta
This is great! It works except that I can not change the maxblock to any number. The resulting files only have five blocks in them. Where exactly do I need to change it?
Thank you!
Split from flat file
Posted: Sat Sep 29, 2012 3:49 am
by Oleg
Open the task in task editor.
Look at the step 3 (Set variable "MAXBLOCK")