| |
 |
|
| Computers Forum Index » Computer Languages (Misc) » Have modern programmers lost the plot? |
|
Page 1 of 3 Goto page 1, 2, 3 Next |
|
| Author |
Message |
| David Paterson |
Posted: Tue Nov 02, 2004 11:40 pm |
|
|
|
Guest
|
I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
'We used to call it a rigged demo. Now it's called Java'.
Last week a MapBasic programmer (who also programs Delphi) complained
that it took 20 cpu hours to compute 27,000 points. I rewrote it in
Fortran and it took 2 minutes for 7.7 million points. That's a
speed-up of at least 170,000 (more if speed depends on the square of
the points).
Have modern programmers completely lost the plot?
The MapBasic/Delphi programmer was also startled to see that I opened,
read and closed a csv file (format i,j,h) for two dimensional array h
and discarded the i and j in only 3 lines of Fortran code.
Don't modern programmers realise that it's always easier to debug 3
lines of code than 30? (OK, I admit that some lines of C++ code are
undebuggable).
PS. A correctly used 'goto' is far less dangerous than a misused
'while' statement.
'A computer language without a goto statement runs like a dog with
three legs.' |
|
|
| Back to top |
|
|
|
| Marco van de Voort |
Posted: Tue Nov 02, 2004 11:40 pm |
|
|
|
Guest
|
On 2004-11-02, David Paterson <David.Paterson@csiro.au> wrote:
Quote: Don't modern programmers realise that it's always easier to debug 3
lines of code than 30? (OK, I admit that some lines of C++ code are
undebuggable).
You already admit yourself it is not _always_. |
|
|
| Back to top |
|
|
|
| Guest |
Posted: Wed Nov 03, 2004 3:39 pm |
|
|
|
|
David.Paterson@csiro.au (David Paterson) wrote in message news:<93e7f970.0411021442.72b27a43@posting.google.com>...
Quote: I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
'We used to call it a rigged demo. Now it's called Java'.
Last week a MapBasic programmer (who also programs Delphi) complained
that it took 20 cpu hours to compute 27,000 points. I rewrote it in
Fortran and it took 2 minutes for 7.7 million points. That's a
speed-up of at least 170,000 (more if speed depends on the square of
the points).
Have modern programmers completely lost the plot?
The MapBasic/Delphi programmer was also startled to see that I opened,
read and closed a csv file (format i,j,h) for two dimensional array h
and discarded the i and j in only 3 lines of Fortran code.
Don't modern programmers realise that it's always easier to debug 3
lines of code than 30? (OK, I admit that some lines of C++ code are
undebuggable).
PS. A correctly used 'goto' is far less dangerous than a misused
'while' statement.
'A computer language without a goto statement runs like a dog with
three legs.'
I thought the c.l.f. crowd would appreciate this. To the OP: Fortran
95 is essentially backwards compatible with Fortran 77, and it fills
in some gaps. |
|
|
| Back to top |
|
|
|
| David Paterson |
Posted: Wed Nov 03, 2004 11:38 pm |
|
|
|
Guest
|
torbenm@diku.dk (Torben Ęgidius Mogensen) wrote in message news:<7zactze0ou.fsf@pc-032.diku.dk>...
Quote: David.Paterson@csiro.au (David Paterson) writes:
I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
Last week a MapBasic programmer (who also programs Delphi) complained
that it took 20 cpu hours to compute 27,000 points. I rewrote it in
Fortran and it took 2 minutes for 7.7 million points. That's a
speed-up of at least 170,000 (more if speed depends on the square of
the points).
I suspect that has more to do with the algorithms than with the
language. Yes, Java is slower than Fortran, but not 15,000 times
slower (unless you are running on a vector processor or other parallel
machine that the Fortran compiler optimises for). I don't know
MapBasic, so I can't comments on its speed, but even so, I doubt it is
170,000 times slower than Fortran.
Torben
I think that MapBasic programmer was trying to search through a full
matrix of 27,000 * 27,000 points rather than a banded matrix of 27,000
* 85 points. Aren't modern programmers taught algorithms? |
|
|
| Back to top |
|
|
|
| Torben Ęgidius Mogensen |
Posted: Thu Nov 04, 2004 11:38 am |
|
|
|
Guest
|
David.Paterson@csiro.au (David Paterson) writes:
Quote: I think that MapBasic programmer was trying to search through a full
matrix of 27,000 * 27,000 points rather than a banded matrix of 27,000
* 85 points. Aren't modern programmers taught algorithms?
Why should they: How often do you see a job add naming knowledge of
algorithms as a requirement? For many programmers, any skill that can
not match keywords in job adds is irrelevant. So the time they use to
learn Java, ASP, VB, XML, SQL, C++, etc., is more valuable than time
spend on learning even basic algorithms.
Also, managers have very little idea of how long time something should
take, so if a programmer makes a program that runs slow, the manager
is likely to think that the job actually requires this time. Only if
the timing is "mission critical" (don't you love buzzwords like
these?) will they spend money on making code run faster.
In Denmark, the standard programmer education (called "datamatiker")
is 2.25 years and here algorithms is an elective course. At the
university we often get such people (who often find it hard to get
jobs) wanting to study for a bachelors or masters degree. When we
process their merit transfer applications, we find that very few have
elected this course. And these are the datamatikers who have
sufficient academic bent to see university education as a possibility,
so I guess even fewer of the rest elect this course.
Another anecdotal evidence is from a friend of mine who lectures
algorithms. A company was making software to find the optimal route
for ambulances to reach the places they need to go. The software
worked fine, except that it took several minutes to find the optimal
route, which was a serious delay for an ambulance. Hence, the company
contacted the university and got in contact with my friend. It turned
out that all that was needed was using Dijkstras shortest path
algorithm (which you find in any algorithm textbook, often as the
first example of dynamic programming). So this company had invested a
lot of programmer hours into a complex system, but they had not
bothered to make sure any of their programmers knew even basic
algorithms, and none of them apparently did.
Torben |
|
|
| Back to top |
|
|
|
| Marco Manfredini |
Posted: Thu Nov 04, 2004 3:38 pm |
|
|
|
Guest
|
David Paterson wrote:
Quote: I think that MapBasic programmer was trying to search through a full
matrix of 27,000 * 27,000 points rather than a banded matrix of 27,000
* 85 points. Aren't modern programmers taught algorithms?
Of course they are. Ever heard of "recursive bubblesort"?
http://thedailywtf.com/archive/2004/10/13/2557.aspx |
|
|
| Back to top |
|
|
|
| James Harris |
Posted: Thu Nov 04, 2004 11:39 pm |
|
|
|
Guest
|
"David Paterson" <David.Paterson@csiro.au> wrote in message
news:93e7f970.0411021442.72b27a43@posting.google.com...
Quote: I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
Care to post the spec or the Fortran code?
Quote: Have modern programmers completely lost the plot?
I don't know but as an eye opener see a book called Inner Loops -
http://ourworld.compuserve.com/homepages/rbooth/ - which, with a variety of
techniques, manages to get some astonishing performance figures. For
example, hash every word in War & Peace in less than a second, compress the
same book using Huffman compression in less than a second, sort 1 million
random 32-bit values in just over two seconds - at the time IIRC on a lowly
100MHz Pentium. It shows that the machines with which we work are capable
of far greater performance than their lowly performance under modern
operating systems and apps would suggest. The challenge for me (and
relevant to this NG) is to produce a language that can come close to being
as efficient - which is why it would be useful to see the Fortan code you
mentioned. Figures like these make programming 'fun' again and make
computers much more usable.
--
Cheers,
James |
|
|
| Back to top |
|
|
|
| David Paterson |
Posted: Fri Nov 05, 2004 3:38 am |
|
|
|
Guest
|
Marco Manfredini <ok_nospam_ok@phoyd.net> wrote in message news:<5364981.9bj0n9W5a3@technoboredom.net>...
Quote: David Paterson wrote:
I think that MapBasic programmer was trying to search through a full
matrix of 27,000 * 27,000 points rather than a banded matrix of 27,000
* 85 points. Aren't modern programmers taught algorithms?
Of course they are. Ever heard of "recursive bubblesort"?
http://thedailywtf.com/archive/2004/10/13/2557.aspx
O(n^n) for a sort! What a laugh, I needed that.
The java programmer had an O(n!) algorithm and I changed it to O(n),
but that speed-up would have been in addition to the factor of 15,000
already mentioned because my O(n) only applied for n>3 and his java
never made it past n=3. |
|
|
| Back to top |
|
|
|
| gswork |
Posted: Fri Nov 05, 2004 11:41 am |
|
|
|
Guest
|
"James Harris" <no.email.please> wrote in message news:<418ab1aa$0$27534$db0fefd9@news.zen.co.uk>...
Quote: "David Paterson" <David.Paterson@csiro.au> wrote in message
news:93e7f970.0411021442.72b27a43@posting.google.com...
I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
Care to post the spec or the Fortran code?
Have modern programmers completely lost the plot?
I don't know but as an eye opener see a book called Inner Loops -
http://ourworld.compuserve.com/homepages/rbooth/ - which, with a variety of
techniques, manages to get some astonishing performance figures. For
example, hash every word in War & Peace in less than a second, compress the
same book using Huffman compression in less than a second, sort 1 million
random 32-bit values in just over two seconds - at the time IIRC on a lowly
100MHz Pentium. It shows that the machines with which we work are capable
of far greater performance than their lowly performance under modern
operating systems and apps would suggest. The challenge for me (and
relevant to this NG) is to produce a language that can come close to being
as efficient - which is why it would be useful to see the Fortan code you
mentioned. Figures like these make programming 'fun' again and make
computers much more usable.
Speed is more in the algorithm than in the language and/or compiler.
It's also in the bottlenecks - modern software often spends time
waiting on networked files, the internet, polling some distant
peripheral etc. Office type software typically supplies some 'in the
background' features like 'spellcheck as you type' and so on, all
eating up memory and generating plenty of competing threads for the
CPU to choke on. Add a great big OS (like MacOS X, Windows XP or a
modern Linux with KDE) and the threads are everywhere!
Some of the anecdotes in this thread point to a lack of sensible
algorithm usage, and why this might be. A good illustration of
algorithm speed is fibonnaci numbers, the recursive 'simple' version
vs one that stores some values locally to speed things up. It shows
what a difference algorithm choice can make. I'm not sure any
language design can mae so big an impact.
A 100Mhz pentium is indeed a powerhouse. It chews through millions
of instructions second after all. You only have to see a game of
Quake (a reasonably tuned game) flawlessly running mid action
sequence, no video card GPU backing it up, to realize that even early
pentiums are pretty mighty! |
|
|
| Back to top |
|
|
|
| David Paterson |
Posted: Fri Nov 05, 2004 11:39 pm |
|
|
|
Guest
|
"James Harris" <no.email.please> wrote in message news:<418ab1aa$0$27534$db0fefd9@news.zen.co.uk>...
Quote: "David Paterson" <David.Paterson@csiro.au> wrote in message
news:93e7f970.0411021442.72b27a43@posting.google.com...
I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
Care to post the spec or the Fortran code?
Have modern programmers completely lost the plot?
I don't know but as an eye opener see a book called Inner Loops -
http://ourworld.compuserve.com/homepages/rbooth/ - which, with a variety of
techniques, manages to get some astonishing performance figures. For
example, hash every word in War & Peace in less than a second, compress the
same book using Huffman compression in less than a second, sort 1 million
random 32-bit values in just over two seconds - at the time IIRC on a lowly
100MHz Pentium. It shows that the machines with which we work are capable
of far greater performance than their lowly performance under modern
operating systems and apps would suggest. The challenge for me (and
relevant to this NG) is to produce a language that can come close to being
as efficient - which is why it would be useful to see the Fortan code you
mentioned. Figures like these make programming 'fun' again and make
computers much more usable.
Form the link: "The market for sophisticated assembly language
programming books melted down in the early 1990's -- a victim of the
emergence of RAD tools such as Visual Basic and Delphi, the increasing
complexity of operating environments, the fragmented installed base of
32-bit platforms, the general decline in Western Civilization, and
last but hardly least, the precedent-shattering piggishness of
Microsoft MASM version 6. What little a person could still find to
read on this subject in the last few years was pretty much the sole
output of Michael Abrash, the 500-pound-gorilla of cycle-shaving. "
Sounds like great fun. I'll definitely get a copy of this, although I
very seldom have a problem with Fortran algorithm speed, storage is
more my problem. |
|
|
| Back to top |
|
|
|
| David Paterson |
Posted: Fri Nov 05, 2004 11:39 pm |
|
|
|
Guest
|
gswork@mailcity.com (gswork) wrote in message news:<81f33a98.0411050107.351973ca@posting.google.com>...
Quote: "James Harris" <no.email.please> wrote in message news:<418ab1aa$0$27534$db0fefd9@news.zen.co.uk>...
"David Paterson" <David.Paterson@csiro.au> wrote in message
news:93e7f970.0411021442.72b27a43@posting.google.com...
I like Fortran 77.
Recently I specified some maths for a Java programmer, a good one. The
program was running too slow so he optimised for speed. After
optimisation it took more than 2 cpu days to get no more than 1/10 of
the way through. I rewrote it in Fortran and it took 1 1/2 minutes.
That's a speed-up of 15,000.
Care to post the spec or the Fortran code?
Have modern programmers completely lost the plot?
I don't know but as an eye opener see a book called Inner Loops -
http://ourworld.compuserve.com/homepages/rbooth/ - which, with a variety of
techniques, manages to get some astonishing performance figures. For
example, hash every word in War & Peace in less than a second, compress the
same book using Huffman compression in less than a second, sort 1 million
random 32-bit values in just over two seconds - at the time IIRC on a lowly
100MHz Pentium. It shows that the machines with which we work are capable
of far greater performance than their lowly performance under modern
operating systems and apps would suggest. The challenge for me (and
relevant to this NG) is to produce a language that can come close to being
as efficient - which is why it would be useful to see the Fortan code you
mentioned. Figures like these make programming 'fun' again and make
computers much more usable.
Speed is more in the algorithm than in the language and/or compiler.
It's also in the bottlenecks - modern software often spends time
waiting on networked files, the internet, polling some distant
peripheral etc. Office type software typically supplies some 'in the
background' features like 'spellcheck as you type' and so on, all
eating up memory and generating plenty of competing threads for the
CPU to choke on. Add a great big OS (like MacOS X, Windows XP or a
modern Linux with KDE) and the threads are everywhere!
Some of the anecdotes in this thread point to a lack of sensible
algorithm usage, and why this might be. A good illustration of
algorithm speed is fibonnaci numbers, the recursive 'simple' version
vs one that stores some values locally to speed things up. It shows
what a difference algorithm choice can make. I'm not sure any
language design can mae so big an impact.
A 100Mhz pentium is indeed a powerhouse. It chews through millions
of instructions second after all. You only have to see a game of
Quake (a reasonably tuned game) flawlessly running mid action
sequence, no video card GPU backing it up, to realize that even early
pentiums are pretty mighty!
Quake runs fast on a 100Mhz pentium? I must get a copy then, my home
computer is a 133MHz pentium. Duke Nukem 3D and MDK both run amazingly
well on it. |
|
|
| Back to top |
|
|
|
| James Harris |
Posted: Fri Nov 05, 2004 11:39 pm |
|
|
|
Guest
|
"David Paterson" <David.Paterson@csiro.au> wrote in message
news:93e7f970.0411051441.76378ac3@posting.google.com...
<snip>
Quote: Sounds like great fun. I'll definitely get a copy of this, although I
very seldom have a problem with Fortran algorithm speed, storage is
more my problem.
I notice you politely declined to post the code Have fun with the book,
though. Not sure what you mean by storage - enough of it, perhaps. How much
would you like? Modern machines have quite a lot - or are you looking for
more than 2 or 3 Gbyte? |
|
|
| Back to top |
|
|
|
| James Harris |
Posted: Fri Nov 05, 2004 11:39 pm |
|
|
|
Guest
|
"gswork" <gswork@mailcity.com> wrote in message
news:81f33a98.0411050107.351973ca@posting.google.com...
<snip>
Quote: Some of the anecdotes in this thread point to a lack of sensible
algorithm usage, and why this might be. A good illustration of
algorithm speed is fibonnaci numbers, the recursive 'simple' version
vs one that stores some values locally to speed things up. It shows
what a difference algorithm choice can make. I'm not sure any
language design can mae so big an impact.
How do you mean? If I had to generate Fibonacci numbers I think I'd use a
loop, not recursion. (Never had to do it though so have never needed a fast
version.) I'm interested in the idea of storing values "locally." Why
couldn't the compiler work out the need for that?
--
James |
|
|
| Back to top |
|
|
|
| Arthur J. O'Dwyer |
Posted: Sat Nov 06, 2004 3:19 am |
|
|
|
Guest
|
On Fri, 5 Nov 2004, James Harris wrote:
Quote:
"gswork" <gswork@mailcity.com> wrote...
Some of the anecdotes in this thread point to a lack of sensible
algorithm usage, and why this might be. A good illustration of
algorithm speed is fibonnaci numbers, the recursive 'simple' version
vs one that stores some values locally to speed things up. It shows
what a difference algorithm choice can make. I'm not sure any
language design can mae so big an impact.
How do you mean? If I had to generate Fibonacci numbers I think I'd use a
loop, not recursion. (Never had to do it though so have never needed a fast
version.) I'm interested in the idea of storing values "locally." Why
couldn't the compiler work out the need for that?
I bet gswork is talking about something like memoization. It occurs
to me that we can speed things up a tiny bit even without keeping a
whole memoization array mapping each input to its Fibonacci number;
consider
int fib(int n)
{
static last_n, last_fib;
if (n < 2) return n;
if (n == last_n) return last_fib;
last_fib = fib(n-2);
last_fib += fib(n-1);
last_n = n;
return last_fib;
}
which performs fewer recursive calls in total than the straightforward
int fib(int n)
{
if (n < 2) return n;
return fib(n-2) + fib(n-1);
}
I haven't really bothered to figure out how things line up to make it
fewer calls, or how many, though.
Generally, compilers for imperative and/or pointery languages can't do
memoization as an optimization. I would be surprised if nobody's ever
tried such an optimization in a compiler for a functional language at
some point.
(Of course, the iterative solution
int fib(int n)
{
int a, b, t;
if (n < 2) return n;
for (a=b=1; n > 2; --n) {
t = a+b;
a = b;
b = t;
}
return b;
}
is still faster than any of those.)
-Arthur |
|
|
| Back to top |
|
|
|
| gswork |
Posted: Mon Nov 08, 2004 1:53 pm |
|
|
|
Guest
|
"James Harris" <no.email.please> wrote in message news:<418c0b3a$0$3035$db0fefd9@news.zen.co.uk>...
Quote: "gswork" <gswork@mailcity.com> wrote in message
news:81f33a98.0411050107.351973ca@posting.google.com...
snip
Some of the anecdotes in this thread point to a lack of sensible
algorithm usage, and why this might be. A good illustration of
algorithm speed is fibonnaci numbers, the recursive 'simple' version
vs one that stores some values locally to speed things up. It shows
what a difference algorithm choice can make. I'm not sure any
language design can mae so big an impact.
How do you mean? If I had to generate Fibonacci numbers I think I'd use a
loop, not recursion. (Never had to do it though so have never needed a fast
version.) I'm interested in the idea of storing values "locally." Why
couldn't the compiler work out the need for that?
I didn't express myself very clearly, but Arthur explained it very
well indeed!
In general it's the algorithm, or just the design choice, that makes
the difference not the langauge or compiler. Even a 'shortcut' is
valid. Let's say you wanted to simulate lottery winnings for a given
number of tickets. This can involve simulating the lottery turn by
turn - actually generating millions of sets of random numbers and
comparing them to your ticket values.
Alternatively you can derive the overall chance of getting specific
winnings and apply them to you tickets, no need to generate all those
numbers. Games do this, for example. In a game with random 'splash
damage' from an explosion, for instance, instead of trying to
calculate the path of each particle and damage incurred if struck, you
can just derive probable damage at a given distance. Modelling can be
much enhanced by using these kinds of approaches. |
|
|
| Back to top |
|
|
|
|
|
All times are GMT
The time now is Sun Nov 08, 2009 5:54 pm
|
|