| |
 |
|
|
Science Forum Index » Compression Forum » a new product which perpetually compress'es
Page 2 of 2 Goto page Previous 1, 2
|
| Author |
Message |
| Dale King |
Posted: Thu Sep 11, 2003 8:37 am |
|
|
|
Guest
|
"CF" <knightchris2001@yahoo.com> wrote in message
news:e7ced33c.0309101847.587f741f@posting.google.com...
Quote: jg@medg.lcs.mit.edu (Jules Gilbert) wrote in message
news:<496bb05e.0309091351.2240ffa5@posting.google.com>...
I assume that most of the reader's of this newsgroup are familiar with
the usual compression model, a SEND side, a RCVE side, and a channel.
That's not the model my program employs.
My program is organized in such a way that the SEND side operator and
the RCVE side operator must agree on a file which already exists on
both side of the channel. This file is read by both programs, the
SEND side as well as the RCVE side. Obviously, this file acts as a
synchronized random source for each process. I call this the MAPKEY
file.
This file should be at least as large as the file to compressed. If
this is not the case, the file is rewound and re-read when EOF is
reached.
I must ask. Even if you could compress any file using this method
(which actually sounds believeable after you hear what I'm about to
ask), doesn't it occur to you that the MAPKEY file must be saved? I
assume the MAPKEY file is required for decompression. Thus your real
compression is the size of your compressed file plus the size of the
MAPKEY file. Just a thought.
It depends whether you have to count the size of the MAPKEY file or not. If
it is fixed, then you do not as it is just part of the decompression program
(unless one is talking about Kolmogorov complexity). Even if you have
several to choose from and you simply tell the decompressor which is the one
to use, then you only have to count the encoding of which is selected. But
there are not enough MAPKEY files that can fit on any current machine to be
able to compress all files.
--
Dale King |
|
|
| Back to top |
|
| SuperFly |
Posted: Thu Sep 11, 2003 10:41 am |
|
|
|
Guest
|
On Thu, 11 Sep 2003 09:33:52 -0500, "Dale King" <KingD@tmicha.net>
wrote:
Quote: "David A. Scott" <daVvid_a_scott@email.com> wrote in message
news:Xns93F33D2F67A55H110W296LC45WIN3030R@130.133.1.4...
SuperFly <fake@email.com> wrote in
news:38e0mv48r3u9abu6a7nvihm3574n5p52t4@4ax.com:
But in the end it's almost like religion. The really want to believe
it's possible and don't accept counter arguments.
No its not like religion it is religion,
This coming from the high priest of bijective compression.
Yeah. Let's start another one of those pointless bijective threads
next to the recursive compression can't be done thread. That will
really push up the intellectual level of this thread :)
BTW: I'm not referring to bijection itself but to the 'flamewars'
between you and David. Can't you come to some kind of consensus and
move on .. |
|
|
| Back to top |
|
| Dale King |
Posted: Thu Sep 11, 2003 11:26 am |
|
|
|
Guest
|
"SuperFly" <fake@email.com> wrote in message
news:2o81mv8hhouvk6mplg4fg03ph2q2p802qa@4ax.com...
Quote: On Thu, 11 Sep 2003 09:33:52 -0500, "Dale King" <KingD@tmicha.net
wrote:
"David A. Scott" <daVvid_a_scott@email.com> wrote in message
news:Xns93F33D2F67A55H110W296LC45WIN3030R@130.133.1.4...
SuperFly <fake@email.com> wrote in
news:38e0mv48r3u9abu6a7nvihm3574n5p52t4@4ax.com:
But in the end it's almost like religion. The really want to believe
it's possible and don't accept counter arguments.
No its not like religion it is religion,
This coming from the high priest of bijective compression.
Yeah. Let's start another one of those pointless bijective threads
next to the recursive compression can't be done thread. That will
really push up the intellectual level of this thread
Not likely since I am back in David's kill-file.
Quote: BTW: I'm not referring to bijection itself but to the 'flamewars'
between you and David. Can't you come to some kind of consensus and
move on ..
That is sort of the point I was making. To David it is a religion. He really
wants to believe he has actually achieved something significant and doesn't
accept counter arguments. How do you come to a consensus with someone who
firmly believes in such a false religion? I certainly do not come to a
consensus by accepting something that is not true, particularly in the area
of mathmatics.
How do you come to a consensus with someone like Timo claiming that they can
losslessly compress anything to 8K. I do not keep an open mind on issues
like that where the claim is easily proven to be not only unbelievable, but
a logical contradiction.
I've tried several times to come to a consensus with David, with little
result.
--
Dale King |
|
|
| Back to top |
|
| David A. Scott |
Posted: Thu Sep 11, 2003 1:07 pm |
|
|
|
Guest
|
SuperFly <fake@email.com> wrote in
news:2o81mv8hhouvk6mplg4fg03ph2q2p802qa@4ax.com:
Quote: On Thu, 11 Sep 2003 09:33:52 -0500, "Dale King" <KingD@tmicha.net
wrote:
"David A. Scott" <daVvid_a_scott@email.com> wrote in message
news:Xns93F33D2F67A55H110W296LC45WIN3030R@130.133.1.4...
SuperFly <fake@email.com> wrote in
news:38e0mv48r3u9abu6a7nvihm3574n5p52t4@4ax.com:
But in the end it's almost like religion. The really want to believe
it's possible and don't accept counter arguments.
No its not like religion it is religion,
This coming from the high priest of bijective compression.
Yeah. Let's start another one of those pointless bijective threads
next to the recursive compression can't be done thread. That will
really push up the intellectual level of this thread :)
BTW: I'm not referring to bijection itself but to the 'flamewars'
between you and David. Can't you come to some kind of consensus and
move on ..
The problem is King knows a little bit about stream compression and
a little bit about what Shannon said. Its sad he doesn't have the
intelligence to apply it out side of his narrow views. He falsely
contines to say I call Shannon wrong when he does not know how to
apply it. My code is open to all to view. Yet King will continue
to belittle it. He has a weak grasp as to what bijective arithmetic
compression is. THere are several versions Matts mine and Tims. I
think Matt is the best spokesman for bijective compression but he is
smarter than me since he knows its a foolish waste of time to get
into on unending argument with the likes of King who will mot even
take an honest like at the code. Matt writes code like what most
call a real program. Mine is less organized. If King really wants
respect from one like me then find a file that when uncompressed
by Matts code will not comprees back to its self. Or find a file
that when compressed will not decompress back to its self. He can't
becuase he will not even look. But he will waste thousands of messages
saying one either can't do it. Or one is cheating or one should never
view compression as a process done to files. He like a very religious
person can not see the other side of an issue. To one like him
there is only one side his. And people like Matt see both sides
and quickly learn not to waste time trying to educate such a narrow
minded person.
Here again is Matts code for a bijective arithmetic file compressor
http://www3.sympatico.ca/mt0000/biacode/biacode.html
It does not waste bits in data file for length it does not use an EOF
symbol in probability table. It does not waste 2 bits for a quadrant
determinenation like other arithmetic file compressor since there are
a waste of space and not needed yet few have the intelligence to grasp
these simple facts.
The pointer is to Matts many have looked at mine. I write not nearly
as well as he. I wish he would continue to post here and maybe explain
more but I think the likes of King have sucessfully discouraged him into
thinking its a waste of time. I would also like to see him relase a better
version of BICOM. I have been tempted to realease my on changes to it but
then America has far fewer freedoms than what is allowed canadians so
I continue to work more on compression that could be used as a first pass
for encryption.
David A. Scott
--
My Crypto code
http://cryptography.org/cgi-bin/crypto.cgi/Misc/scott19u.zip
http://cryptography.org/cgi-bin/crypto.cgi/Misc/scott16u.zip
http://www.jim.com/jamesd/Kong/scott19u.zip old version
My Compression code http://bijective.dogma.net/
**TO EMAIL ME drop the roman "five" **
Disclaimer:I am in no way responsible for any of the statements
made in the above text. For all I know I might be drugged.
As a famous person once said "any cryptograhic
system is only as strong as its weakest link" |
|
|
| Back to top |
|
| SuperFly |
Posted: Thu Sep 11, 2003 2:31 pm |
|
|
|
Guest
|
On 11 Sep 2003 19:07:39 GMT, "David A. Scott"
<daVvid_a_scott@email.com> wrote:
[snip]
Quote: The pointer is to Matts many have looked at mine. I write not nearly
as well as he. I wish he would continue to post here and maybe explain
more but I think the likes of King have sucessfully discouraged him into
thinking its a waste of time. I would also like to see him relase a better
version of BICOM. I have been tempted to realease my on changes to it but
then America has far fewer freedoms than what is allowed canadians so
I continue to work more on compression that could be used as a first pass
for encryption.
Why not benchmark sets of regular files like raw pictures, dct'ed
pictures, sensory input, etc.. and compare the results with other
coders? Throw the results on a webpage, and you have hard facts which
speak for themselves.
Just my 2 cents .. |
|
|
| Back to top |
|
| Mikael Lundqvist |
Posted: Thu Sep 11, 2003 2:42 pm |
|
|
|
Guest
|
SuperFly wrote:
Quote: No its not like religion it is religion,
This coming from the high priest of bijective compression.
Yeah. Let's start another one of those pointless bijective threads
next to the recursive compression can't be done thread. That will
really push up the intellectual level of this thread :)
BTW: I'm not referring to bijection itself but to the 'flamewars'
between you and David. Can't you come to some kind of consensus and
move on ..
Hey, don't spoil the great entertainment!
Seriously, I hoped they would be able to achieve something constructive
together in their private emails recently. Not so, unfortunately. :-(
Since they are so interested in *arguing* about it, should they also be
interested in *creating* something together, that's the rational and
adult thing to do in their case.
"Winds of truth should blow away heresies, and clear the airs of holy
church, which is now full troubled." -- John Wyclif (1380)
--
Mikael Lundqvist
http://www.geocities.com/mikaellq/
Occam's Razor:
"Keep things simple!" |
|
|
| Back to top |
|
| SuperFly |
Posted: Thu Sep 11, 2003 4:19 pm |
|
|
|
Guest
|
On Thu, 11 Sep 2003 22:42:58 +0200, Mikael Lundqvist
<mlqnospam@telia.com> wrote:
[snip]
Quote: BTW: I'm not referring to bijection itself but to the 'flamewars'
between you and David. Can't you come to some kind of consensus and
move on ..
Hey, don't spoil the great entertainment!
:)
Quote: Seriously, I hoped they would be able to achieve something constructive
together in their private emails recently. Not so, unfortunately. :-(
Since they are so interested in *arguing* about it, should they also be
interested in *creating* something together, that's the rational and
adult thing to do in their case.
Wouldn't it be an idea to come up with some kind of framework for this
NG based on some test-cases, and try to come up with the best
modelling and coding scheme for each case. Might be a good way to
focus the know-how and energy of everyone in one direction.
Just a thought .. |
|
|
| Back to top |
|
| David A. Scott |
Posted: Thu Sep 11, 2003 5:16 pm |
|
|
|
Guest
|
Mikael Lundqvist <mlqnospam@telia.com> wrote in news:bjqmp4$ma40f$1@ID-
87439.news.uni-berlin.de:
Quote:
Since they are so interested in *arguing* about it, should they also be
interested in *creating* something together, that's the rational and
adult thing to do in their case.
"Winds of truth should blow away heresies, and clear the airs of holy
church, which is now full troubled." -- John Wyclif (1380)
--
Mikael Lundqvist
http://www.geocities.com/mikaellq/
Occam's Razor:
"Keep things simple!"
Mikael its not likely King and I would ever do that. He doesn't think
FILE COMPRESSORS are worthly of being called compressors so its hard to
belive that could happen. However take your BWT compressor as an example
I could make it compress better by removing some of the non bijective
parts. Are you game. It would be a slower but it would compress
better.
David A. Scott
--
My Crypto code
http://cryptography.org/cgi-bin/crypto.cgi/Misc/scott19u.zip
http://cryptography.org/cgi-bin/crypto.cgi/Misc/scott16u.zip
http://www.jim.com/jamesd/Kong/scott19u.zip old version
My Compression code http://bijective.dogma.net/
**TO EMAIL ME drop the roman "five" **
Disclaimer:I am in no way responsible for any of the statements
made in the above text. For all I know I might be drugged.
As a famous person once said "any cryptograhic
system is only as strong as its weakest link" |
|
|
| Back to top |
|
| Kelsey Bjarnason |
Posted: Fri Sep 12, 2003 12:23 am |
|
|
|
Guest
|
[snips]
On Tue, 09 Sep 2003 14:51:29 -0700, Jules Gilbert wrote:
Quote: Hello folks:
Most of the regulars around comp.cpx have heard of me -- I'm the guy
who claims to do what most of the rest of the regulars believe to be
impossible.
I build working, practical compressors which routinely further
compress the output produced by processing an uncompressed file, and
thus, perform iterative compression.
Which has _never_ been stated as being impossible; rather, it is _either_
impossible or pointless. If the algorithm is "optimal", it will produce
maximally compressed results; only by losing data or using a different
algorithm will further compression be achieved. So either the algorithm
is optimal, in which case iterating won't help, or it is suboptimal and
should be "tightened" to remove the inefficiencies. If it uses multiple
algorithms on subsequent passes, it would make more sense to modify the
core program to apply all the relevant portions of all the algorithms in a
single pass, increasing efficiency.
The "impossible" argument stems from the above; it is generally assumed
that the compressor is in some manner optimal - that it doesn't produce
wasted space in sufficient quantities that it itself can compress the data
further. With such an assumption in place, the notion of iterative
compression becomes one of compressing already optimally-compressed
data... which, pretty much by definition, cannot be done.
Assuming your mechanism works, then, we'll have to simply assume that the
code is inefficiently compressing during each pass. Inefficient
compressors are nothing new... nor are they anything to build empires
upon; were it so, I'd spew forth a variant of RLE, patent it, and get rich
from the proceeds. At least, unlike yours, it would have the advantage of
being screamingly fast.  |
|
|
| Back to top |
|
| Dale King |
Posted: Thu Sep 18, 2003 9:35 pm |
|
|
|
Guest
|
"David A. Scott" <daVvid_a_scott@email.com> wrote in message
news:Xns93F3B022B2526H110W296LC45WIN3030R@130.133.1.4...
Quote: Mikael its not likely King and I would ever do that. He doesn't think
FILE COMPRESSORS are worthly of being called compressors so its hard to
belive that could happen.
No, I have no trouble with file compressors. I just know that all other
compressors are not file compressors. You are the one that assumes
everything is a file compressor and call it inefficient if it is not.
Quote: However take your BWT compressor as an example
I could make it compress better by removing some of the non bijective
parts. Are you game. It would be a slower but it would compress
better.
There may be parts that you can make better, just as I have complemented you
in the past in some improvements you made to a RLE. But the key thing is
that turning it into a file-only compressor does not mean it compresses
better.
--
Dale King |
|
|
| Back to top |
|
| Dale King |
Posted: Fri Sep 19, 2003 12:05 pm |
|
|
|
Guest
|
"David A. Scott" <daVvid_a_scott@email.com> wrote in message
news:Xns93F385E0EA620H110W296LC45WIN3030R@130.133.1.4...
Quote: SuperFly <fake@email.com> wrote in
news:2o81mv8hhouvk6mplg4fg03ph2q2p802qa@4ax.com:
On Thu, 11 Sep 2003 09:33:52 -0500, "Dale King" <KingD@tmicha.net
wrote:
"David A. Scott" <daVvid_a_scott@email.com> wrote in message
news:Xns93F33D2F67A55H110W296LC45WIN3030R@130.133.1.4...
SuperFly <fake@email.com> wrote in
news:38e0mv48r3u9abu6a7nvihm3574n5p52t4@4ax.com:
But in the end it's almost like religion. The really want to believe
it's possible and don't accept counter arguments.
No its not like religion it is religion,
This coming from the high priest of bijective compression.
Yeah. Let's start another one of those pointless bijective threads
next to the recursive compression can't be done thread. That will
really push up the intellectual level of this thread :)
BTW: I'm not referring to bijection itself but to the 'flamewars'
between you and David. Can't you come to some kind of consensus and
move on ..
The problem is King knows a little bit about stream compression
No, I know a lot about stream compression and file compression. Whereas you
only know about file-only compression and falsely assume everyone else only
deals with file-only compression.
Quote: and
a little bit about what Shannon said.
No, I know a lot about what Shannon said. I at least know enough to know
that he said the Entropy for N equally probable messages is log(N), which
you called wrong.
Quote: Its sad he doesn't have the
intelligence to apply it out side of his narrow views.
I don't have narrow views on the subject. In fact, they aren't even really
my views. They are facts established through 50+ years of information
theory. And they apply equally to whatever kind of compression you are
talking about.
You are the one with the narrow view that every thing in compression is
about producing files, which is not true.
Quote: He falsely
contines to say I call Shannon wrong when he does not know how to
apply it.
Once again Shannon said that the entropy for N equally probable messages
(all with probability of 1/N) is:
log( N )
This group was presented with a case where N = 2.16 * 10^16 and the answer
needed was for units of digits. Thomas Richter simply applied Shannon's
result and said that the average encoding length was:
log10( 2.16 * 10^16 ) = 16.33 digits
Your response to that was "Sorry your wrong in how you calculated. Close but
wrong." "In your eample you got 16.33 changing this to account for 20% drop
in numbers gives 15.33 + .903 which is still 16.23 sonmthing. meaning hay
the true avaerage is bewteen 16.33 and 16.23"
If the true average is less than 16.334 then Shannon was wrong. You said the
average was less than 16.33, therefore you said Shannon was wrong.
Quote: My code is open to all to view. Yet King will continue
to belittle it.
I have never once belittled your code or said anything bad about it. My
problem has never been with your code. The problem is the way you
characterize other compressors. You belittle other compressors as
inefficient when they are not at all inefficient for what they are designed
for. You just can't see that they weren't designed to only compress to
files.
Quote: He has a weak grasp as to what bijective arithmetic
compression is.
I've got a pretty strong grasp of what you mean by it. But once again the
question is your view of non-bijective compressors.
Quote: THere are several versions Matts mine and Tims. I
think Matt is the best spokesman for bijective compression but he is
smarter than me since he knows its a foolish waste of time to get
into on unending argument with the likes of King who will mot even
take an honest like at the code.
And once again there is nothing to be gained by studying your code. I agree
that it maps to the set of all files. The only thing to be gained would be
to disprove this fact.
Quote: Matt writes code like what most
call a real program. Mine is less organized. If King really wants
respect from one like me
I am certainly not after respect, but for you to be enlightened. For you to
see that there is a world of information theory beyond file compression. I
don't care if you want to work only in that narrow view point, but the
problem is that you demean any one that won't put your set of blinders on.
Quote: then find a file that when uncompressed
by Matts code will not comprees back to its self. Or find a file
that when compressed will not decompress back to its self. He can't
becuase he will not even look.
Once again that would not further either argument. I have already accepted
as a given that your assertion that there is no such file. We both agree
that there is no such file. Even if there were such a file that would only
reflect a bug in the program that could be corrected.
The issue is not whether or not your compressors meet this criteria, but
whether other compressors are inefficient if they do not meet this criteria.
That is the difference.
If we are talking about file-only compression where the output of the
compressor is known to always have an end of file indication accompanying
it, then I would agree with you that the compressor would be smart to take
advantage of it. But the fact is when you talk about standard Huffman or
standard Arithmetic you are not talking about file-only compression.
Quote: But he will waste thousands of messages
saying one either can't do it.
Please post a reference to even one. You can't because I never said that. It
is trivial to prove that one can do it. The set of outputs from any
compressor is countably infinite as are the set of all files therefore you
can define a bijection between them.
Quote: Or one is cheating or one should never
view compression as a process done to files.
Once again things I have never said and things that I do not believe. The
problem is your viewpoint that compression is only a process done to files.
No other compressor sees it that way. That does not make them inefficient as
you claim, that simply means that they aren't file compressors.
Quote: He like a very religious
person can not see the other side of an issue. To one like him
there is only one side his.
That sounds like a better description of you and is certainly not true of
me.
I see your side perfectly. But I see the implicit assumption underlying your
entire viewpoint. That implicit assumption so permeates your thought
processes that you cannot even see that it is there or that there is a world
where that assumption is not true.
Quote: And people like Matt see both sides
and quickly learn not to waste time trying to educate such a narrow
minded person.
I guess I am a bit foolish trying to educate you, but it is really only
because I do care about you.
Quote: Here again is Matts code for a bijective arithmetic file compressor
http://www3.sympatico.ca/mt0000/biacode/biacode.html
It does not waste bits in data file for length it does not use an EOF
symbol in probability table. It does not waste 2 bits for a quadrant
determinenation like other arithmetic file compressor
What other arithmetic file compressor are you referring to. Please post a
reference to any other arithmetic compressor that is designed to work only
in an file environment. I certainly don't know of any.
This is an example of that implicit assumption underlying your thought
process. You assume every compressor is designed for compressing to files.
You think they are wasteful for not using the file length that is always
there.
But no other compressors are designed for compressing to only files.
Standard Huffman and arithmetic are designed to produce a sequence of bits,
not files.
It basically depends on your model of what they communication channel is
between the compressor and decompressor. Your model is that it consists of a
sequence of bytes followed by an EOF indication. That is certainly a valid m
odel. But it is not the only model and you cannot see that.
The model used by standard Huffman and Arithmetic is that it is a sequence
of bits and if the receiver tries to read more bits than what were written
by the compressor you either get some unknown values or block forever
waiting on more data. That is a valid model too and in fact is the standard
one used .
The issue is not whether one can design for your model, you certainly can.
And I agree that a compressor designed with your characteristics is optimal
for that model. The problem is that you say a compressor is inefficient if
it is not designed for that model, which is what is not true.
Standard Huffman is provably optimal for *its* model. There is no
inefficiency in its design, it was just designed for a different model than
yours. When you switch a standard Huffman to be optimal for your model you
haven't made it any more efficient, you've just switched models.
Consider if I have a new fangled communication system that is trinary
instead of binary. If I took a compressor that was designed for binary and
sent in on this channel it would not be very efficient. The trivial way
would be to use only 2 of the values out of each trit. I could redesign the
compressor to use a model where it is outputting trinary instead of binary
and this will make better use of the trinary channel.
Have you increased the compression by doing this? No. Was it inefficient
before? No. The difference is that you just switched models. You cannot make
a compressor that is optimal for more than one model.
Quote: since there are
a waste of space and not needed yet few have the intelligence to grasp
these simple facts.
They are not needed in your model, but they are needed in the model it was
designed for.
--
Dale King |
|
|
| Back to top |
|
| |
Page 2 of 2 Goto page Previous 1, 2
All times are GMT - 5 Hours
The time now is Sat Nov 22, 2008 7:49 pm
|
|