 |
|
| Science Forum Index » Compression Forum » Microsoft HDPhoto algorithm exposed... |
|
Page 1 of 1 |
|
| Author |
Message |
| LiloLilo... |
 Posted: Mon Oct 19, 2009 9:47 am |
|
|
|
Guest
|
|
| Back to top |
|
|
|
| LiloLilo... |
| Posted: Mon Oct 19, 2009 11:50 pm |
|
|
|
Guest
|
[quote]Short: DO NOT DEVELOP FROM THIS CODE ANYMORE. WAIT FOR THE REFERENCE
SOFTWARE.
Thanks,
Thomas
[/quote]
Sorry, this post was not intended as 'NEWS' (it was probably a news
only for me  )
I am interested in your point of view about HDPhoto technology, the
provided link was only intended to be a information source for
discussion.
Regards, D. |
|
|
| Back to top |
|
|
|
| Thomas Richter... |
| Posted: Tue Oct 20, 2009 12:45 am |
|
|
|
Guest
|
LiloLilo schrieb:
[quote]Hi all,
at this link you can download the HD Photo Device Porting Kit 1.0. It
contains a document that explains the HDPhoto bitstream and
compression algorithm. I'd like to know your point of view about
it
[/quote]
Folks, this is *old* news. First of all, forget the device porting kit.
It is obsolete ever since. As you might or might not know, HDPhoto is
currently undergoing, and is short to complete, standardization as JPEG
XR. A couple of changes have been made since then, and the HDPhoto DPK
is only partially backwards compatible; it doesn't support "hard tile
boundaries", the POT filter is broken, causing a slight loss in PSNR,
and the overall portability of the software isn't quite what you need
(64 bit support is broken, too, for example, and a subtle bug in a macro
prevents the DPK from operating properly under anything but VS)
Microsoft and the JPEG committee will release a new reference software
really soon now that addresses all the defects found, that is written
from scratch and is much more portable and clearer than the DPK ever was.
Short: DO NOT DEVELOP FROM THIS CODE ANYMORE. WAIT FOR THE REFERENCE
SOFTWARE.
Thanks,
Thomas |
|
|
| Back to top |
|
|
|
| Thomas Richter... |
| Posted: Tue Oct 20, 2009 4:35 am |
|
|
|
Guest
|
LiloLilo schrieb:
[quote]Short: DO NOT DEVELOP FROM THIS CODE ANYMORE. WAIT FOR THE REFERENCE
SOFTWARE.
Thanks,
Thomas
Sorry, this post was not intended as 'NEWS' (it was probably a news
only for me )
I am interested in your point of view about HDPhoto technology, the
provided link was only intended to be a information source for
discussion.
[/quote]
Oh well. For the time being, let's stick to the facts:
The complexity is somewhere between JPEG and JPEG 2000, and clearly much
closer to JPEG than to JPEG 2000. The break-even between the two for
well-written software is around 1bpp (1:24 compression), but you usually
want higher quality, and then JPEG XR is definitely faster.
Quality: Question is - what is it. If you define quality in terms of
PSNR, then JPEG 2000 is definitely better, but JPEG XR is closer to JPEG
2000 than to JPEG. With a couple of neat tricks, one can improve JPEG XR
somewhat in terms of MSE, but not much. Typically, improvements are
below 0.4dB, except for special images like "compound images".
Visual quality: Hot topic, and again, "a depends". If you just use the
software "as is", with the options as given, and then compare, then I'd
rather say that its visual quality is best described as "not
impressive". Somewhere near JPEG, sometimes above, sometimes below. JPEG
2000 clearly advanced. However, the story doesn't end there. There are a
couple of tricks (again) that can improve the visual performance
considerably (but also the complexity) that brings it closer to, and in
some cases even better than JPEG 2000. But so can the old JPEG be
optimized in several ways.
Features: No chance JPEG XR can beat JPEG 2000. It does have a couple of
interesting features, though. Probably important is that it is
"idempotent", i.e. recompression with *exactly the same* settings does
not cause additional loss (as it would for JPEG or JPEG 2000). If you
change settings slightly, or modify the image slightly in between
(geometrical transformations, rendering to a different color space),
things are no longer quite as good, but still ok. Elementary geometric
transformations like rotation and clipping are supported (as they are in
JPEG or JPEG 2000).
So my conclusion is that *personally* do not quite see the need for it -
the overall codec design is a bit behind and not as modern as one would
probably expect from a 21st century compression codec (e.g. Huffman
instead of arithmetic coding) It lacks some functionalities I would
consider useful for improving the quality (i.e. modifying quantization
per frequency and not only per band).
As always, its some kind of a compromise between features, functionality
and complexity. I would have done things a bit differently, and it
clearly doesn't go "over the edge" as JPEG 2000 did - clearly - it's a
more pragmatic design, but IMHO took the wrong decisions here and there.
Nevertheless, probably the market will adapt it - chances are that your
computer already decoded it if you're running Vista or Windows 7.
So long,
Thomas |
|
|
| Back to top |
|
|
|
| LiloLilo... |
| Posted: Sun Nov 01, 2009 3:39 am |
|
|
|
Guest
|
[quote]
So my conclusion is that *personally* do not quite see the need for it -
the overall codec design is a bit behind and not as modern as one would
probably expect from a 21st century compression codec (e.g. Huffman
instead of arithmetic coding) It lacks some functionalities I would
consider useful for improving the quality (i.e. modifying quantization
per frequency and not only per band).
As always, its some kind of a compromise between features, functionality
and complexity. I would have done things a bit differently, and it
clearly doesn't go "over the edge" as JPEG 2000 did - clearly - it's a
more pragmatic design, but IMHO took the wrong decisions here and there.
[/quote]
I see your point. It seems more like a commercial hit than a real
technical improvement,
so one can't really understand how prefer this format instead of
others already out there. I agree.
In your opinion, how a new image file format would be really 'new'?
Which features do you need/expect form a new image format we can
define 'userful'?
Which ones you really can't miss?
Regards, Danilo |
|
|
| Back to top |
|
|
|
| Thomas Richter... |
| Posted: Sun Nov 01, 2009 3:17 pm |
|
|
|
Guest
|
LiloLilo wrote:
[quote]So my conclusion is that *personally* do not quite see the need for it -
the overall codec design is a bit behind and not as modern as one would
probably expect from a 21st century compression codec (e.g. Huffman
instead of arithmetic coding) It lacks some functionalities I would
consider useful for improving the quality (i.e. modifying quantization
per frequency and not only per band).
As always, its some kind of a compromise between features, functionality
and complexity. I would have done things a bit differently, and it
clearly doesn't go "over the edge" as JPEG 2000 did - clearly - it's a
more pragmatic design, but IMHO took the wrong decisions here and there.
I see your point. It seems more like a commercial hit than a real
technical improvement,
so one can't really understand how prefer this format instead of
others already out there. I agree.
In your opinion, how a new image file format would be really 'new'?
[/quote]
By providing functionality current formats don't support, while keeping
compatibility to what is in use. I'm not clear whether compression
factors really matter too much. Or rather, if we could 30% better at the
same complexity, it might - which isn't likely to happen. JPEG 2000
probably got like 10-20% better at ten times the complexity, JPEG-XR
didn't get much better (or at most something like 10%) at about twice to
three times the complexity.
[quote]Which features do you need/expect form a new image format we can
define 'userful'?
[/quote]
Must be backwards compatible to JPEG. I'm using that, I don't want to
loose it. Simple, isn't it? (-;
[quote]Which ones you really can't miss?
[/quote]
A considerably useful feature would be to provide a format that would
allow me to store "relatively unprocessed" images, aka "raw" images.
These usually aren't "raw" at all, and a certain amount of processing
already applied to them; however, if I take "raw" images today, I need
the tool of the camera vendor, and I can be sure that the tools I got
with the camera today are useless in five years. Thus, basically, my
images are lost by then. Oh great! I depend on some proprietary software
from which I don't know what it does.
Sure, JPEG-XR could provide something like that. JPEG 2000 could.
JPEG-LS plus a suitable wrapper could. Except that the camera vendors
don't want such a format; for them, it is more convenient to "lock-in"
their consumers. Tough luck. It is about time consumers notice this
problem and stand up.
Hence, one potentially useful format would be applicable to raw images,
would allow lossless compression, would be documented well enough to
allow archival of images, and would include all the necessary metadata
currently supported by cameras. Security of content, and of technology.
Of course, the JPEG has an open call for such technology.
Another interesting topic would be that of medical or other scientific
databases. Loss *might* be tolerable here, but it must be controlled.
That is, I must be able to specify the amount of loss that is tolerable,
and must ensure that image quality is not degraded more than necessary
or allowable. It must be suitable to represent the corresponding data
sources, including high bitdepths, higher precision, additional data.
JPEG 2000 is something like it, close to it. Medical industry still has
to learn not to specify a compression factor, but a quality. Now quality
as they need it must be defined, another important issue. Here, we have
mostly an educational problem that needs to be attacked. Apparently,
people there think image compression is something like LZW. (No, really!)
And of course, JPEG has an open call for such technology.
How do I know? Because I wrote the calls. (-;
Thus, if anyone has such technology, approach your national body.
Failing that, approach me.
So long,
Thomas |
|
|
| Back to top |
|
|
|
| Pete Fraser... |
| Posted: Sun Nov 01, 2009 4:10 pm |
|
|
|
Guest
|
"Thomas Richter" <thor at (no spam) math.tu-berlin.de> wrote in message
news:hckqdn$lok$1 at (no spam) infosun2.rus.uni-stuttgart.de...
[quote]if I take "raw" images today, I need the tool of the camera vendor, and I
can be sure that the tools I got with the camera today are useless in five
years. Thus, basically, my images are lost by then. Oh great! I depend on
some proprietary software from which I don't know what it does.
[/quote]
Agreed, but do you knnow about dcraw.
The source is available, and I have used it to decode a variety of raw
formats.
[quote]Another interesting topic would be that of medical or other scientific
databases. Loss *might* be tolerable here, but it must be controlled. That
is, I must be able to specify the amount of loss that is tolerable, and
must ensure that image quality is not degraded more than necessary or
allowable.
[/quote]
Is JPEG-LS not suitable for this?
Pete |
|
|
| Back to top |
|
|
|
| Thomas Richter... |
| Posted: Sun Nov 01, 2009 5:11 pm |
|
|
|
Guest
|
Pete Fraser wrote:
[quote]"Thomas Richter" <thor at (no spam) math.tu-berlin.de> wrote in message
news:hckqdn$lok$1 at (no spam) infosun2.rus.uni-stuttgart.de...
if I take "raw" images today, I need the tool of the camera vendor, and I
can be sure that the tools I got with the camera today are useless in five
years. Thus, basically, my images are lost by then. Oh great! I depend on
some proprietary software from which I don't know what it does.
Agreed, but do you knnow about dcraw.
The source is available, and I have used it to decode a variety of raw
formats.
[/quote]
I do know about dcraw. However, parts of this program are pretty naive,
and surely the program doesn't take the same steps the firmware does. It
is the result of a brave reverse-engineering and some guesswork. Nothing
bad about this, but this is not a long-term strategy. Or rather, who
would like to go through all the hassle reverse engineering the
internals of the next upcoming "inventions" of our beloved camera brands.
[quote]Another interesting topic would be that of medical or other scientific
databases. Loss *might* be tolerable here, but it must be controlled. That
is, I must be able to specify the amount of loss that is tolerable, and
must ensure that image quality is not degraded more than necessary or
allowable.
Is JPEG-LS not suitable for this?
[/quote]
As an underlying compression technology, maybe. However, lots of
additional data has to go into a container to make the image as it is
useful and deployable. The raw image data is only one part of it. You
also probably want to have some additional features JPEG-LS doesn't
provide, like random-access or resolution scalability. The type of loss
JPEG-LS introduces is also only suitable for compression very close to
lossless, otherwise its striping artifacts are pretty irritating.
There's much more about image compression than just compressing the
image data. (-;
So long,
Thomas |
|
|
| Back to top |
|
|
|
|
|
All times are GMT - 5 Hours
The time now is Mon Dec 07, 2009 10:46 am
|
|