Page 2 of 2
Split from flat file
Posted: Mon Oct 01, 2012 7:49 pm
by trotta
I tried 400 the first time. Not sure what is going on but when I change the number I get different results.
For 50 I get 5 blocks
For 400 I get 5 blocks
For 90 I get 90 blocks
For 99 I get 989 blocks
For 10 I get 1 blocks
For 60 I get 6 blocks
For 70 I get 7 blocks
Is it possible to get over 100 blocks?
Is is possible for it to put the remainder in a file also?
Split from flat file
Posted: Tue Oct 02, 2012 5:32 am
by Oleg
I'm sorry, I made mistake.
Look at the step #9 (If then)
It's necessary to compare {numblock} and {maxblock} "As Number" (not "as string"). Fix it please or download corrected example here.
What happened:
if we compare values as strings then
"1" < "400"
"2" < "400"
"3" < "400"
"4" < "400"
BUT
"5" > "400"
because strings are being compared in alphabetical order (not as numeric values)
Split from flat file
Posted: Tue Oct 02, 2012 5:37 am
by Oleg
Is is possible for it to put the remainder in a file also?
The remainder will be written in last "SMALL" file in any case.
Split from flat file
Posted: Wed Oct 03, 2012 7:30 pm
by trotta
It is working pretty well. But when I try to split large files, over 20,000 KB or over 800 blocks it will split the first four then does not produce an additional file after that. I'm looking to split a file of over 80,000 blocks into smaller files of 2,000 blocks.
Split from flat file
Posted: Thu Oct 04, 2012 6:24 am
by Oleg
Step #20 (Create text file ) of the example always writes the remainder of the text into file. It is placed after the loop and writes last blocks.
Pack your large file into ZIP or RAR or 7Z and send to me to oleg(at)robotask.com
I want to test it myself
It works correctly with my test examples. I need to test it on real file.
Split from flat file
Posted: Thu Oct 04, 2012 7:49 am
by Oleg
I made the test on file 38Mbyte (1 833 721 lines)
I tried to split this file to small files 800 blocks per file.
The task created 350 small files correctly and wrote the remainder into file #350.
Maybe the problem in your file. Probably there are some nuances which algorithm doesn't take into consideration.
Split from flat file
Posted: Tue Jan 22, 2013 5:14 am
by trotta
We were able to get this working well and now have another challenge.
We need to split the file the same as before but now use the number of lines or the split file size in KB in addition to splitting after a 5.
Split from flat file
Posted: Wed Jan 23, 2013 6:08 am
by Oleg
To count number of lines you can use line counter in the text loop. Counter is a simple variable that increases by 1 at each iteration of
the loop.
To calculate the size of part-file use system variable {StrLength({PARTTEXT})}
This variable calculates the length of variable PARTTEXT in bytes
Full list of system variables here.
Split from flat file
Posted: Wed Jan 23, 2013 10:07 am
by trotta
It needs to split on line 2,500 if the first number of the line is 5. If it is not 5 it goes to the next line.
If the next line is 5, then it splits, if not then it needs to go to the next line and repeat until it gets to a line that starts with 5.
Then continues counting lines until line number 5,000 and split again if the first number of the line is 5.
If the next line is 5, then it splits if not then it needs to go to the next line and repeat until it gets to a line that starts with 5.
The line splits are at 2,500, 5,000, 7,500, 10,000, 12,500 and so on until the end of the file.
Split from flat file
Posted: Fri Jan 25, 2013 7:53 am
by Oleg
See the example below.
You can change the name of source file and number of lines of each part at steps #1 and #2
Code: Select all
;*****************************
;* RoboTask Task file
;* Do not edit in text editor!
;*****************************
[Root]
ActionAfterRun=INTEGER|0
Actions=FOLDER
Automat=INTEGER|-1
CatID=INTEGER|2099193306
Comment=STRINGLIST
ContinueOnError=INTEGER|0
ExternalName=STRING|"Task14"
Hide=INTEGER|0
ID=INTEGER|509484789
LogOnAsUser=INTEGER|1
Name=STRING|"Split by number of lines"
OnErrorTaskID=INTEGER|0
Priority=INTEGER|3
RunOnClose=INTEGER|0
RunOnStartup=INTEGER|0
ToLog=INTEGER|3
WriteGeneralLog=INTEGER|0
[Actions]
Action1=FOLDER
Action10=FOLDER
Action11=FOLDER
Action12=FOLDER
Action13=FOLDER
Action14=FOLDER
Action15=FOLDER
Action16=FOLDER
Action17=FOLDER
Action18=FOLDER
Action19=FOLDER
Action2=FOLDER
Action20=FOLDER
Action21=FOLDER
Action22=FOLDER
Action23=FOLDER
Action24=FOLDER
Action3=FOLDER
Action4=FOLDER
Action5=FOLDER
Action6=FOLDER
Action7=FOLDER
Action8=FOLDER
Action9=FOLDER
[Actions\Action1]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""SOURCEFILE"" with value ""D:\Temp\txt\big.txt"""
Params=FOLDER
[Actions\Action1\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"SOURCEFILE"
varvalue=STRING|"D:\Temp\txt\big.txt"
[Actions\Action10]
ActionID=STRING|"A_VARIABLES_SET"
Enabled=INTEGER|-1
Name=STRING|"Set variable ""TEXT"" with value ""{text}{cline}{EOL}"""
Params=FOLDER
[Actions\Action10\Params]
expand=STRING|"1"
varname=STRING|"TEXT"
varvalue=STRING|"{text}{cline}{EOL}"
[Actions\Action11]
ActionID=STRING|"A_MISC_COMMENT"
Enabled=INTEGER|-1
Name=STRING|"//If more then MAXLINES"
Params=FOLDER
[Actions\Action11\Params]
comment=STRING|"If more then MAXLINES"
[Actions\Action12]
ActionID=STRING|"A_FLOW_IF"
Enabled=INTEGER|-1
Name=STRING|"If Then"
Params=FOLDER
[Actions\Action12\Params]
case=STRING|"0"
operator=STRING|"2"
type=STRING|"1"
value1=STRING|"{cnt}"
value2=STRING|"{maxlines}"
[Actions\Action13]
ActionID=STRING|"A_STR_EXTRACT"
Enabled=INTEGER|-1
Name=STRING|"STR Extract"
Params=FOLDER
[Actions\Action13\Params]
amount=STRING|"1"
from=STRING|"1"
source=STRING|"{cline}"
variable=STRING|"FirstChar"
[Actions\Action14]
ActionID=STRING|"A_MISC_COMMENT"
Enabled=INTEGER|-1
Name=STRING|"//If found 5"
Params=FOLDER
[Actions\Action14\Params]
comment=STRING|"If found 5"
[Actions\Action15]
ActionID=STRING|"A_FLOW_IF"
Enabled=INTEGER|-1
Name=STRING|"If Then"
Params=FOLDER
[Actions\Action15\Params]
case=STRING|"0"
operator=STRING|"0"
type=STRING|"0"
value1=STRING|"{FirstChar}"
value2=STRING|"5"
[Actions\Action16]
ActionID=STRING|"A_FILE_TEXTWRITE"
Enabled=INTEGER|-1
Name=STRING|"Create text file {sourcefile}.{numfile}"
Params=FOLDER
[Actions\Action16\Params]
fileexists=STRING|"0"
filname=STRING|"{sourcefile}.{numfile}"
line0=STRING|"{text}"
linecount=STRING|"1"
suppress=STRING|"1"
[Actions\Action17]
ActionID=STRING|"A_VARIABLES_INCREMENT"
Enabled=INTEGER|-1
Name=STRING|"Increment variable ""numfile"""
Params=FOLDER
[Actions\Action17\Params]
incement=STRING|"1"
vaiable=STRING|"numfile"
[Actions\Action18]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""TEXT"" with value """""
Params=FOLDER
[Actions\Action18\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"TEXT"
[Actions\Action19]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""CNT"" with value ""0"""
Params=FOLDER
[Actions\Action19\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"CNT"
varvalue=STRING|"0"
[Actions\Action2]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""MAXLINES"" with value ""2500"""
Params=FOLDER
[Actions\Action2\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"MAXLINES"
varvalue=STRING|"2500"
[Actions\Action20]
ActionID=STRING|"A_FLOW_ENDIF"
Enabled=INTEGER|-1
Name=STRING|"End If"
[Actions\Action21]
ActionID=STRING|"A_FLOW_ENDIF"
Enabled=INTEGER|-1
Name=STRING|"End If"
[Actions\Action22]
ActionID=STRING|"A_LOOP_END"
Enabled=INTEGER|-1
Name=STRING|"End Loop"
[Actions\Action23]
ActionID=STRING|"A_MISC_COMMENT"
Enabled=INTEGER|-1
Name=STRING|"//Write reminder"
Params=FOLDER
[Actions\Action23\Params]
comment=STRING|"Write reminder"
[Actions\Action24]
ActionID=STRING|"A_FILE_TEXTWRITE"
Enabled=INTEGER|-1
Name=STRING|"Create text file {sourcefile}.{numfile}"
Params=FOLDER
[Actions\Action24\Params]
fileexists=STRING|"0"
filname=STRING|"{sourcefile}.{numfile}"
line0=STRING|"{text}"
linecount=STRING|"1"
suppress=STRING|"1"
[Actions\Action3]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""CNT"" with value ""0"""
Params=FOLDER
[Actions\Action3\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"CNT"
varvalue=STRING|"0"
[Actions\Action4]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""CLINE"" with value """""
Params=FOLDER
[Actions\Action4\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"CLINE"
[Actions\Action5]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""TEXT"" with value """""
Params=FOLDER
[Actions\Action5\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"TEXT"
[Actions\Action6]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""FIRSTCHAR"" with value """""
Params=FOLDER
[Actions\Action6\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"FIRSTCHAR"
[Actions\Action7]
ActionID=STRING|"A_VARIABLES_CREATE"
Enabled=INTEGER|-1
Name=STRING|"Create local variable ""NUMFILE"" with value ""1"""
Params=FOLDER
[Actions\Action7\Params]
expand=STRING|"0"
varkind=STRING|"1"
varname=STRING|"NUMFILE"
varvalue=STRING|"1"
[Actions\Action8]
ActionID=STRING|"A_LOOP_TEXT"
Enabled=INTEGER|-1
Name=STRING|"Text Loop"
Params=FOLDER
[Actions\Action8\Params]
destvar=STRING|"CLINE"
file=STRING|"{sourcefile}"
linecount=STRING|"0"
sourcetext=STRING|"0"
[Actions\Action9]
ActionID=STRING|"A_VARIABLES_INCREMENT"
Enabled=INTEGER|-1
Name=STRING|"Increment variable ""cnt"""
Params=FOLDER
[Actions\Action9\Params]
incement=STRING|"1"
vaiable=STRING|"cnt"