| Computers Forum Index » Computer - Databases - Pick » D3 Compile (o... |
|
Page 1 of 1 |
|
| Author |
Message |
| Peter McMurray... |
Posted: Mon Oct 05, 2009 5:01 am |
|
|
|
Guest
|
I did not bother with the O compile option because of the initial issues
that I perceived and because there did not appear to be any speed issues
that required addressing.
However last week a confused Cobol programmer who could not produce a TAB
delimited file sent me a 2 megabyte file containing 10,000+ fixed length
transactions with a comma delimited Header and Trailer. I mused over the
best way to import it. Normally I would just use COPY DOS: however I could
not guarantee the source position with a Citrix managed server over the
Internet so I used Hostaccess as that is their emulator.
Once received I tossed up over the best way to convert it. Initially I used
EQU NOVAL TO ""
EQU AM TO CHAR(254)
CONVERT CR TO NOVAL IN REC
CONVERT LF TO AM IN REC
this gets round the issue of Unix files not having proper line endings
Then I tried
CONVERT CRLF TO AM IN REC
as the file proved to be Windows compatible after I got him to change future
ones to dottxt instead of dotcsv. Windows Excel expects a csv file to mean
it and he had commas in some of the fixed length data with disastrous
results for the user checking it. On the other hand Open Office will not
open a txt file with calc but will open it and allow one to specify column
splits etc if it is called csv even if it is in fact fixed length, tab or
some other method.
to my surprise both took the same amount of time - 1 minute 6seconds on an
XP machine.
I knew that a COPY DOS: was instantaneous so I thought try COMPILE MMBP CHK
(O)
Incredible the result was instantaneous as per the TCL. So my apologies to
the engineers I should have paid more attention to the optimise option. My
reluctance is based on the fact that my entire suite has to be optimised or
none of it and I hate big hits like that. I can see that it is worth two
steps, one for the debugging phase and then optimise for release.
--
Peter McMurray
Excalibur21 at (no spam) bigpond.com |
|
|
| Back to top |
|
|
|
| Kevin Powick... |
Posted: Mon Oct 05, 2009 5:01 am |
|
|
|
Guest
|
Uhh, wut?
--
Kevin Powick |
|
|
| Back to top |
|
|
|
| Tony Gravagno... |
Posted: Mon Oct 05, 2009 12:03 pm |
|
|
|
Guest
|
Kevin Powick wrote:
Yeah, I thought I was tired.
What the heck did he just say? |
|
|
| Back to top |
|
|
|
| Tony Gravagno... |
Posted: Tue Oct 06, 2009 5:15 am |
|
|
|
Guest
|
Quote: Summary Fact: CONVERT works extremely fast on large items if COMPILED with
(O)
No argument. That's what it's for.
Quote: however you have to jump through the hoop that an O Compiled program
will only run as optimised if called from an optimised program. It falls
back to standard Basic if called from a standard BASIC compile.
What you're describing is what I explain in page 3 of the following
blog:
nospamNebula-RnD.com/blog/tech/mv/2008/04/d3flash1.html
Quote: Simple solution in a program not optimised is EXECUTE COPY it to
WINDOWS and COPY it back.
Peter, you're confusing movement of data from OS into D3 with the
processing of the data after it's in D3. FlashBASIC has nothing to do
with a COPY command because COPY is done outside of BASIC. The speed
of copy from Windows to D3 or back will be exactly the same because
you're dealing with the file tier, not BASIC workspace. Your
performance is attained when you're doing string operations on the
dynamic array because now you're operating in a single block of fast
memory with C++ rather than in paged frame space where links need to
be chased by assembler code.
Now, if you are doing the following:
OPEN 'C:/path' TO FV ELSE STOP
READ INFO FROM FV,'filename.ext' ELSE STOP
I could be wrong but I believe the performance of that READ will be
better with Flashed code than with non-flashed because as stated above
INFO is in memory and you're not grabbing a new frame of overflow
every 4k or so. But again, this is not related to this:
EXECUTE "COPY C:/path file.ext": at (no spam) AM:"(MYFILE"
Part 2 of the above blog discusses some nuances of %functions which I
mentioned earlier:
nospamNebula-RnD.com/blog/tech/mv/d3/2009/06/d3flash2.html
HTH
T |
|
|
| Back to top |
|
|
|
| Peter McMurray... |
Posted: Tue Oct 06, 2009 12:14 pm |
|
|
|
Guest
|
Hi Tony
It is you that is confused not I. My solution works perfectly because of
the reasons given. End of story. The simple fact is that the COPY is using
Flashed code to do the translate after it has moved the data block which an
unflashed master program cannot do.
Peter McMurray
"Tony Gravagno" <address.is.in.posts at (no spam) removethis.com.invalid> wrote in
message news:jiblc5dln957r9mql1h6uhp09asa12fdi0 at (no spam) 4ax.com...
Quote: Summary Fact: CONVERT works extremely fast on large items if COMPILED
with
(O)
No argument. That's what it's for.
however you have to jump through the hoop that an O Compiled program
will only run as optimised if called from an optimised program. It falls
back to standard Basic if called from a standard BASIC compile.
What you're describing is what I explain in page 3 of the following
blog:
nospamNebula-RnD.com/blog/tech/mv/2008/04/d3flash1.html
Simple solution in a program not optimised is EXECUTE COPY it to
WINDOWS and COPY it back.
Peter, you're confusing movement of data from OS into D3 with the
processing of the data after it's in D3. FlashBASIC has nothing to do
with a COPY command because COPY is done outside of BASIC. The speed
of copy from Windows to D3 or back will be exactly the same because
you're dealing with the file tier, not BASIC workspace. Your
performance is attained when you're doing string operations on the
dynamic array because now you're operating in a single block of fast
memory with C++ rather than in paged frame space where links need to
be chased by assembler code.
Now, if you are doing the following:
OPEN 'C:/path' TO FV ELSE STOP
READ INFO FROM FV,'filename.ext' ELSE STOP
I could be wrong but I believe the performance of that READ will be
better with Flashed code than with non-flashed because as stated above
INFO is in memory and you're not grabbing a new frame of overflow
every 4k or so. But again, this is not related to this:
EXECUTE "COPY C:/path file.ext": at (no spam) AM:"(MYFILE"
Part 2 of the above blog discusses some nuances of %functions which I
mentioned earlier:
nospamNebula-RnD.com/blog/tech/mv/d3/2009/06/d3flash2.html
HTH
T
|
|
|
| Back to top |
|
|
|
| Ross Ferris... |
Posted: Wed Oct 07, 2009 1:40 am |
|
|
|
Guest
|
On Oct 6, 7:14 pm, "Peter McMurray" <excalibu... at (no spam) bigpond.com> wrote:
Quote: Hi Tony
It is you that is confused not I. My solution works perfectly because of
the reasons given. End of story. The simple fact is that the COPY is using
Flashed code to do the translate after it has moved the data block which an
unflashed master program cannot do.
Peter McMurray
Peter,
COPY is a TCL2 verb
:ed md copy
top
..p
001 vz
002 3]e
003 uz
eoi 003
No Basic code, flashed or otherwise, to be seen |
|
|
| Back to top |
|
|
|
| Kevin Powick... |
Posted: Wed Oct 07, 2009 5:13 am |
|
|
|
Guest
|
On Oct 6, 10:28 pm, "Peter McMurray" <excalibu... at (no spam) bigpond.com> wrote:
Quote:
It seems to me that people do not read very carefully.
That's right... Blame the readers. :-)
--
Kevin Powick |
|
|
| Back to top |
|
|
|
| Peter McMurray... |
Posted: Wed Oct 07, 2009 5:15 am |
|
|
|
Guest
|
Hi Ross
It seems to me that people do not read very carefully. I am talking about
in and out of the Windows environment as I made quite clear in the original
post.
It is the DOS: part that is doing the conversion.
Just to clear the airwaves here is the code
KEYIT = "(DOS:D:/)"
DATA KEYIT
EXECUTE 'COPY SSTRANS ':RECKEY:" (O)"
DATA "(SSTRANS) ":RECKEY
EXECUTE "COPY DOS:D:/ ":RECKEY:" (O)"
I wrote the original record into SSTRANS after bringing it in through
Hostaccess. I did not use the standard Hostaccess record split because the
confused Cobol guy had mixed record types amongst other stuffups. Plus I
wanted a copy available for Excel on the server, I actually extended the
path to another directory.
This does the conversion approximately 100 times faster than a CONVERT
without optimise which I find significant.
However I decided to try and confirm or deny my firm belief that Disk IO is
dependent on hardware rather than software. A test of 32000 by 380
character records showed no advantage in optimise apparently proving my
belief.
I know you do extensive testing and am happy to accept your call on the
situation. I was unable to read Tony's blog because page 3 selection
crashed it.
The pain with optimise is that it is all or nothing. An optimised program
cannot call a standard one and a standard one calling an optimised one will
drop it back to standard. Plus I believe debug vanishes although I have not
tested recently.
I intend to try EXECUTE of an optimised item instead of a call as it should
work and could be useful sometimes.
Peter McMurray
"Ross Ferris" <rossf at (no spam) stamina.com.au> wrote in message
news:1e252e41-9774-4d8e-b31b-fdab9484203e at (no spam) k13g2000prh.googlegroups.com...
On Oct 6, 7:14 pm, "Peter McMurray" <excalibu... at (no spam) bigpond.com> wrote:
Quote: Hi Tony
It is you that is confused not I. My solution works perfectly because of
the reasons given. End of story. The simple fact is that the COPY is using
Flashed code to do the translate after it has moved the data block which
an
unflashed master program cannot do.
Peter McMurray
Peter,
COPY is a TCL2 verb
:ed md copy
top
..p
001 vz
002 3]e
003 uz
eoi 003
No Basic code, flashed or otherwise, to be seen |
|
|
| Back to top |
|
|
|
| Peter McMurray... |
Posted: Wed Oct 07, 2009 9:23 am |
|
|
|
Guest
|
Who else indeed :-)
Peter McMurray
"Kevin Powick" <kpowick at (no spam) gmail.com> wrote in message
news:48572f69-f2a3-4185-a1a6-95adf5451471 at (no spam) j28g2000vbl.googlegroups.com...
On Oct 6, 10:28 pm, "Peter McMurray" <excalibu... at (no spam) bigpond.com> wrote:
Quote:
It seems to me that people do not read very carefully.
That's right... Blame the readers. :-)
--
Kevin Powick |
|
|
| Back to top |
|
|
|
| Tony Gravagno... |
Posted: Thu Oct 08, 2009 10:44 am |
|
|
|
Guest
|
"Peter McMurray" wrote:
Quote: Feel free to believe that the code used to do the conversion in the COPY
DOS: routine is different to that called by an optimised FlashBasic program
if that makes you happy.
I don't suppose you recall that I was QA Manager and later Product
Manager for the software that we're discussing? I don't make this
crap up, I do have some clue.
The OSFI driver is not related to the compilation method of code
invoking it. A simple proof is that you can use all OSFI functions
from TCL, entirely outside of BASIC.
Please feel free to contact your support provider or TL Support
directly, and ask them to explain how it works. Until then, please
stop insisting that you know the internals of this stuff when you just
discovered it a couple days ago! C'mon, how rational is that?
T |
|
|
| Back to top |
|
|
|
| x... |
Posted: Thu Oct 08, 2009 5:33 pm |
|
|
|
Guest
|
Quote: EXECUTE "COPY DOS:D:/ ":RECKEY:" (O)"
The compile method should have no bearing over executed or chained
programs as they are outside the scope of current code.
Well, he is correct that flashed code must call to flashed code and
unflashed must call to unflashed. See page 3 of the mentioned blog.
This is an "execute" not a "call".
"Execute" just passes control to another program and works regardles
of program type, be it Basic, Proc, Access or a non-PICK program as in
EXECUTE CHAR(255):"sh"
Lucian |
|
|
| Back to top |
|
|
|
| Tony Gravagno... |
Posted: Fri Oct 09, 2009 11:08 am |
|
|
|
Guest
|
x <lucian_pata at (no spam) yahoo.com> wrote:
Quote: EXECUTE "COPY DOS:D:/ ":RECKEY:" (O)"
The compile method should have no bearing over executed or chained
programs as they are outside the scope of current code.
Well, he is correct that flashed code must call to flashed code and
unflashed must call to unflashed. See page 3 of the mentioned blog.
This is an "execute" not a "call".
"Execute" just passes control to another program and works regardles
of program type, be it Basic, Proc, Access or a non-PICK program as in
EXECUTE CHAR(255):"sh"
Lucian
No disagreement. You are correct in what you are saying, as he was
correct in what he was saying about this specific point: In addition
to Executing a copy where flash has no bearing, he discovered that
flash must call to flash and non-flash must call to non-flash.
T |
|
|
| Back to top |
|
|
|
| Peter McMurray... |
Posted: Tue Oct 13, 2009 10:52 am |
|
|
|
Guest
|
Hi
Bet you thought I had gone quiet. No way! As one client said I could talk
under six feet of wet concrete with a mouthful of marbles, he then bought me
another beer.
Two things happened.
On Friday a had an epiphany on three fronts.
I found a reason to look at Linux - the ARM chip
I found a reason to look at Windows 7 - the built in touch screen facilities
I decided to investigate FlashBasic optimise on my own without looking at
other people's opinions.
Then the second thing happened.
New Zealand puts out the "Gone Fushun" sign every Friday whereas Australia
has three unmissable religious festivals during which an invasion would not
be noticed.
The Grand Final - finally won by Geelong thanks to the son of god aka Gary
Ablett.
Bathurst - The greatest motor race in the world. The Jesus entry was black
flagged early on and the Hot Holden Red Devils totally destroyed the cool
blue fords.
Melbourne Cup is next - I look forward to it given the momentous events of
the first two.
Anyway I have done my testing and may post it for more flame throwing,
However I find it quite amazing that people who one would believe are
analysts by trade seem incapable of understanding a simple statement of
fact.
It is extremely unlikely that a system that is absolutely based on string
handling would have multiple low level versions of code to do the same job.
At no time did I say that the conversion of text was done at the OSFI level.
I stated the opposite that in my opinion the switch of characters from CRLF
to AM would be done after the block was moved and therefore in the D3
environment. It believe that this is the same routine that optimised code
would use.
If anyone has actual knowledge of the details of the operation I am willing
to listen.
With regard to the previous poster's comments I did it as an EXECUTE because
that provides a separate environment without requiring the Basic to be
optimised. I did that because the client managed to get himself into a hole
with other suppliers and I had to dig him out ASAP.
Peter McMurray
<address.is.in.posts at (no spam) removethis.com.invalid> wrote in message
news:p50rc5tkl1ptbcberelr08r92g7htdbl5c at (no spam) 4ax.com...
Quote: "Peter McMurray" wrote:
Feel free to believe that the code used to do the conversion in the COPY
DOS: routine is different to that called by an optimised FlashBasic
program
if that makes you happy.
I don't suppose you recall that I was QA Manager and later Product
Manager for the software that we're discussing? I don't make this
crap up, I do have some clue.
The OSFI driver is not related to the compilation method of code
invoking it. A simple proof is that you can use all OSFI functions
from TCL, entirely outside of BASIC.
Please feel free to contact your support provider or TL Support
directly, and ask them to explain how it works. Until then, please
stop insisting that you know the internals of this stuff when you just
discovered it a couple days ago! C'mon, how rational is that?
T
|
|
|
| Back to top |
|
|
|
|