 |
|
| Computers Forum Index » Computer - Databases - Ingres » DDS for MVCC Support now Available... |
|
Page 1 of 1 |
|
| Author |
Message |
| Ingres Forums... |
Posted: Tue Oct 27, 2009 1:05 am |
|
|
|
Guest
|
|
| Back to top |
|
|
|
| Jonah H. Harris... |
Posted: Wed Oct 28, 2009 5:15 am |
|
|
|
Guest
|
On Mon, Oct 26, 2009 at 5:05 PM, Ingres Forums <
info-ingres at (no spam) kettleriverconsulting.com> wrote:
Quote:
The Design Specifications for MVCC Support in Ingres is available for
review at 'MVCC - Ingres Community Wiki'
(http://community.ingres.com/wiki/MVCC)
It's nice to see such a fairly detailed specification. However, with very
few exceptions, the architecture of this is almost identical to Oracle's
MVCC design and implementation for acquiring a CR version of a block, which
is heavily patented for not only a single-server software architecture, but
also multi-server clustered architectures as well. For example, 4.1.9.1
lists CR buffers as being linked from the current version. While this is a
seemingly obvious implementation for all chained hash table-based DB buffer
management implementations, it is no different for Oracle. In Oracle, all
versions of a buffer similarly hash to the same bucket, and all are chained
to the current version. Likewise, the process listed in 4.1.9.4 is quite
close to Oracle's implementation in that they determine whether a CR version
needs to be created (by comparison of the SCN [similar to the comparison
against crib_low_lsn]), and perform iterative UNDO on that block
accordingly. While the high-level design is similar, in addition to the
coarse-grained block-level UNDO, Oracle can also follow a single tuple back
through multiple versions, which I don't see any conflict with here. Both
Oracle and SQL Server now support Postgres-like tuple-based MVCC for various
things, and both are using a fairly basic multi-version timestamp ordering
(MVTO) algorithm. Oracle's buffer manager still relies on UNDO-based MVRC,
but they use MVTO (rowscn) in a few other areas.
In my experience, UNDO-based MVRC is much more preferable to MVTO from both
a performance and manageability standpoint. It's just that Oracle has quite
a few patents on optimized MVRC designs whereas MVTO is so blatantly obvious
that it's an easy choice to make from a legality standpoint. Though, I'd be
happy to see Ingres get MVRC.
Regardless, I just wanted to make sure you guys are aware of the striking
similarities in the design, especially having referenced Oracle's own
documentation in the DDS. The only other system I know of which performed a
similar UNDO-to-reach-a-CR-version-of-the-block (albeit a little different
than Oracle's traditional method) is InnoDB, which of course is now Oracle's
IP as well. I'm not sure whether Oracle's patent policy has changed, but I
believe they have 3-6 fairly detailed patents for their MVCC implementation.
--
Jonah H. Harris, Senior DBA
myYearbook.com |
|
|
| Back to top |
|
|
|
| rthdavid... |
Posted: Wed Oct 28, 2009 10:04 am |
|
|
|
Guest
|
Hi Jonah,
I must admit it never crossed my mind that implementing a feature on
ones own architecture would somehow infringe a patent based on someone
else's!
I can think of numerous Ingres features that have crept into Oracle
over the years and I never got to hear about any lawyers turning up at
Oracle HQ to slap a lawsuit on Larry.
I also wonder how much Ingres Intellectual Property got included into
Oracle by the late Bob Kooi, and those that left (or got fired from)
Ingres to join Oracle at the CA acquisition?
Oracle and Ingres are like chalk and cheese.
Oracle's storage is broken down into Tablespaces, Segments, Extents
and Blocks where a tablespace must have at least one be datafile.
Physical objects like tables and indexes are stored in Segments.
Ingres has locations which have datafiles belonging to physical
objects like tables and indexes. Each object must have at least one
datafile.
Oracle has redo log groups and undo tablespaces whereas Ingres has
circular transaction log files
Quite different architectures you'll agree?
I am not a lawyer, but how on earth would one go about enforcing a
patent against an Open Source Community? Goodness knows how tough it’s
been for the EU to bring Micro$oft to heel.
I suspect that Oracle would have nothing to gain from such a pursuit
in as much as they would have little to gain from acquiring Ingres
Corp. as a company. The Ingres code is already out there and it would
be terribly hard to close the stable door after the horse has bolted.
As far as I'm concerned, imitation is the greatest form of flattery.
Oracle have much bigger fish to fry.
Go MVCC Ingres!!!!!
It’s been an awful long time coming - even if some reckon that it does
resemble Oracle's implementation! ;-P |
|
|
| Back to top |
|
|
|
| Roy Hann... |
Posted: Wed Oct 28, 2009 2:11 pm |
|
|
|
Guest
|
Jonah H. Harris wrote:
Quote: On Mon, Oct 26, 2009 at 5:05 PM, Ingres Forums
info-ingres at (no spam) kettleriverconsulting.com> wrote:
The Design Specifications for MVCC Support in Ingres is available for
review at 'MVCC - Ingres Community Wiki'
(http://community.ingres.com/wiki/MVCC)
It's nice to see such a fairly detailed specification. However, with very
few exceptions, the architecture of this is almost identical to Oracle's
MVCC design and implementation for acquiring a CR version of a block, which
is heavily patented for not only a single-server software architecture, but
also multi-server clustered architectures as well. For example, 4.1.9.1
lists CR buffers as being linked from the current version. While this is a
seemingly obvious implementation for all chained hash table-based DB buffer
management implementations, it is no different for Oracle. In Oracle, all
versions of a buffer similarly hash to the same bucket, and all are chained
to the current version. Likewise, the process listed in 4.1.9.4 is quite
close to Oracle's implementation in that they determine whether a CR version
needs to be created (by comparison of the SCN [similar to the comparison
against crib_low_lsn]), and perform iterative UNDO on that block
accordingly. While the high-level design is similar, in addition to the
coarse-grained block-level UNDO, Oracle can also follow a single tuple back
through multiple versions, which I don't see any conflict with here. Both
Oracle and SQL Server now support Postgres-like tuple-based MVCC for various
things, and both are using a fairly basic multi-version timestamp ordering
(MVTO) algorithm. Oracle's buffer manager still relies on UNDO-based MVRC,
but they use MVTO (rowscn) in a few other areas.
In my experience, UNDO-based MVRC is much more preferable to MVTO from both
a performance and manageability standpoint. It's just that Oracle has quite
a few patents on optimized MVRC designs whereas MVTO is so blatantly obvious
that it's an easy choice to make from a legality standpoint. Though, I'd be
happy to see Ingres get MVRC.
Regardless, I just wanted to make sure you guys are aware of the striking
similarities in the design, especially having referenced Oracle's own
documentation in the DDS. The only other system I know of which performed a
similar UNDO-to-reach-a-CR-version-of-the-block (albeit a little different
than Oracle's traditional method) is InnoDB, which of course is now Oracle's
IP as well. I'm not sure whether Oracle's patent policy has changed, but I
believe they have 3-6 fairly detailed patents for their MVCC implementation.
Jonah, this is interesting stuff that deserves to be widely discussed,
and a newsgroup is a fine place to do it, but I doubt very many
people are going to take the time I just took to look up some of the
acronyms. Any chance you could post a more easily comprehended
version?
--
Roy
UK Ingres User Association Conference 2010 will be on Tuesday June 8 2010
Go to http://www.iua.org.uk/join to get on the mailing list. |
|
|
| Back to top |
|
|
|
| Roy Hann... |
Posted: Wed Oct 28, 2009 2:23 pm |
|
|
|
Guest
|
rthdavid wrote:
Quote: I am not a lawyer, but how on earth would one go about enforcing a
patent against an Open Source Community? Goodness knows how tough it¢s
been for the EU to bring Micro$oft to heel.
I believe the practice in the past has been to go after end-users.
Hence the need for legal indemnification, which is a very important
and rarely mentioned benefit of a support subscription.
Quote: I suspect that Oracle would have nothing to gain from such a pursuit
[...].
There are probably intangible benefits, as well as the legal principle
that silence implies consent. If you don't sue the little guy who
infringes your patent, you can't later sue the dominant player for the
same infringement because you are presumed to have already consented to
the infringement. IANAL of course, but patent issues are probably worth
taking very seriously as part of this discussion.
On the bright side I expect the legal team has already looked over the
DDS and decided it is unlikely to attract any action.
Quote: Go MVCC Ingres!!!!!
Hear hear. Amen to that. etc.
--
Roy
UK Ingres User Association Conference 2010 will be on Tuesday June 8 2010
Go to http://www.iua.org.uk/join to get on the mailing list. |
|
|
| Back to top |
|
|
|
| Jonah H. Harris... |
Posted: Wed Oct 28, 2009 7:00 pm |
|
|
|
Guest
|
On Wed, Oct 28, 2009 at 6:04 AM, rthdavid <rthdavid at (no spam) googlemail.com> wrote:
Quote: Oracle has redo log groups and undo tablespaces whereas Ingres has
circular transaction log files
Quote: Quite different architectures you'll agree?
From a storage manager architecture, I agree that they are different to some
degree. Oracle's storage format was designed to act similarly to a file
system and allow for near-transparent operation on both raw storage and on
top of pre-existing file systems; hence the extent-based design.
Ingres/Postgres share the same file-per-object notion. But in this respect,
the MVCC design has nothing to do with the on-disk storage format, but
instead, it has to do with buffer management and the method by which a
consistent read version of a block is acquired.
Quote: I am not a lawyer, but how on earth would one go about enforcing a
patent against an Open Source Community? Goodness knows how tough it’s
been for the EU to bring Micro$oft to heel.
There are several examples out there...
As far as I'm concerned, imitation is the greatest form of flattery.
Quote: Oracle have much bigger fish to fry.
That's likely.
Quote: Go MVCC Ingres!!!!!
Indeed.
Quote: It’s been an awful long time coming - even if some reckon that it does
resemble Oracle's implementation! ;-P
:-)
--
Jonah H. Harris, Senior DBA
myYearbook.com |
|
|
| Back to top |
|
|
|
| Jonah H. Harris... |
Posted: Wed Oct 28, 2009 7:13 pm |
|
|
|
Guest
|
On Wed, Oct 28, 2009 at 6:11 AM, Roy Hann
<specially at (no spam) processed.almost.meat>wrote:
Quote: Jonah, this is interesting stuff that deserves to be widely discussed,
and a newsgroup is a fine place to do it, but I doubt very many
people are going to take the time I just took to look up some of the
acronyms. Any chance you could post a more easily comprehended
version?
Hey Roy,
I may have time for that a bit later.
For reference:
- MVCC = Multi-version Concurrency Control (a concurrency method which is
based on versioning rather than locking)
- MVRC = Multi-version Read Consistency (an isolation level component of
MVCC)
- CR = Consistent Read (a version of the block applicable to an individual
transaction)
- MVTO = Multi-version Timestamp Ordering (an algorithm for versioning
tuples using timestamp-based transaction snapshots)
- UNDO-based MVRC = A method by which UNDO recovery data is used to convert
blocks from their current state to a state relevant to an individual
transaction.
If anyone is interested, Transactional Information Systems: Theory,
Algorithms, and the Practice of Concurrency Control and Recovery (
http://www.amazon.com/Transactional-Information-Systems-Algorithms-Concurrency/dp/1558605088)
is a pretty good reference which describes several different algorithms for
performing MVCC.
--
Jonah H. Harris, Senior DBA
myYearbook.com |
|
|
| Back to top |
|
|
|
| Laframboise, Andr¨¦... |
Posted: Wed Oct 28, 2009 11:01 pm |
|
|
|
Guest
|
Take a gander at the data file names ..... Thought it was funny....
1> select * from iifile_info where table_name='s_cache_marc_xml_2'
©°©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ð©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ð©¤©¤©¤©¤©¤©¤©¤©¤©Ð©¤©¤©¤©¤
©¦table_name ©¦owner_name ©¦file_nam©¦file
©À©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©à©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©à©¤©¤©¤©¤©¤©¤©¤©¤©à©¤©¤©¤©¤
©¦s_cache_marc_xml_2 ©¦dbadev ©¦aaaeieio©¦t00
©¦s_cache_marc_xml_2 ©¦dbadev ©¦aaaeieio©¦t01
©¦s_cache_marc_xml_2 ©¦dbadev ©¦aaaeieio©¦t02
©¸©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ø©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ø©¤©¤©¤©¤©¤©¤©¤©¤©Ø©¤©¤©¤©¤
(3 rows)
2> commit |
|
|
| Back to top |
|
|
|
| rthdavid... |
Posted: Thu Oct 29, 2009 10:01 am |
|
|
|
Guest
|
Hi Jonah,
You've really got me thinking about patents, intellectual property
issues and Open Source.
Would you know how EnterpriseDB protect themselves from Larry lawyers,
because EnterpriseDB's Advanced Server Oracle compatibility is a
blatant rip-off, If you ask me?!
- - - - - - - - - - - - - - - - - -
Would anyone from Ingres Corp. care to comment on Jonah's concerns?? |
|
|
| Back to top |
|
|
|
| Martin Bowes... |
Posted: Thu Oct 29, 2009 12:24 pm |
|
|
|
Guest
|
Is this 'gander' part of the livestock on Old McDonalds Farm?
Marty
From: info-ingres-bounces at (no spam) kettleriverconsulting.com [mailto:info-ingres-bounces at (no spam) kettleriverconsulting.com] On Behalf Of Laframboise, Andr¨¦
Sent: 28 October 2009 19:01
To: Ingres and related product discussion forum
Subject: [Info-Ingres] Ingres does have a sense of humor ....
Take a gander at the data file names ..... Thought it was funny....
1> select * from iifile_info where table_name='s_cache_marc_xml_2'
©°©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ð©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ð©¤©¤©¤©¤©¤©¤©¤©¤©Ð©¤©¤©¤©¤
©¦table_name ©¦owner_name ©¦file_nam©¦file
©À©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©à©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©à©¤©¤©¤©¤©¤©¤©¤©¤©à©¤©¤©¤©¤
©¦s_cache_marc_xml_2 ©¦dbadev ©¦aaaeieio©¦t00
©¦s_cache_marc_xml_2 ©¦dbadev ©¦aaaeieio©¦t01
©¦s_cache_marc_xml_2 ©¦dbadev ©¦aaaeieio©¦t02
©¸©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ø©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©¤©Ø©¤©¤©¤©¤©¤©¤©¤©¤©Ø©¤©¤©¤©¤
(3 rows)
2> commit |
|
|
| Back to top |
|
|
|
| Jonah H. Harris... |
Posted: Thu Oct 29, 2009 4:19 pm |
|
|
|
Guest
|
On Thu, Oct 29, 2009 at 6:01 AM, rthdavid <rthdavid at (no spam) googlemail.com> wrote:
Quote: Would you know how EnterpriseDB protect themselves from Larry lawyers,
because EnterpriseDB's Advanced Server Oracle compatibility is a
blatant rip-off, If you ask me?!
Having worked at EnterpriseDB (as employee #9) from 2005-2008 in the role of
Sr. Software Architect working on Oracle Compatibility, I *can* tell you
exactly how they protect themselves: EnterpriseDB does not violate any
Oracle patents. Similarly, EnterpriseDB does not architect or develop
features based on Oracle documentation. Feature development is based only
on publicly available third-party sources, such as books, articles, etc. In
that respect, software specs are already one degree away from Oracle and is
along the lines of clean room design.
EnterpriseDB's PL/SQL Compatibility
EnterpriseDB implemented a PL/SQL-compatible programming language called SPL
(the Superset Procedural Language). Aside from the fact that PL/SQL is
basically just Ada, it's a programming language and as such, is not
patentable. Anyone write their own compiler for PL/SQL, just as there are
many compilers for BASIC, C, C++, Pascal, etc. Now, Oracle's PL/SQL user
guide, reference guide, etc. is copyrighted to Oracle, so EnterpriseDB has
to write their own SPL user documentation, but the language itself is
generally fair game.
EnterpriseDB's Data Access Compatibility
If you're familiar with Oracle, the main C/C++ data access interface used is
OCI (the Oracle Call Interface). If you look, EnterpriseDB has a product
called OCL, which they originally bought from me in order to allow some
applications written against the OCI API to work with EnterpriseDB. While
the API is *copyrighted* to Oracle, because it's used to access the
database, copying the function call signatures is allowed for
interoperability purposes (there's a long history of legal precedence for
that).
EnterpriseDB's Oracle-compatible Utilities
EDB*Plus is EnterpriseDB's version of SQL*Plus. EDB*Plus was written to be
compatible with SQL*Plus commands and report formatting. There's nothing
(in patent law) that says you can't have a program that looks like another
program or acts like another program to the end user.
The main problem is not in what the program *looks like*, it's in how it
works. For example, I architected a complete redesign of MVCC for Postgres
(on which EnterpriseDB is based) to avoid doing MVTO (due to the significant
overhead in both performance and manageability) and prefer a more
block-based read consistency approach. The problem I ran into was the
number of patents Oracle (and IBM) have on buffer management and recovery,
which made it difficult to come up with a scheme that didn't violate them in
some way.
Personally, I think software patents are stupid. Unfortunately, not
everyone shares my opinion :-(
--
Jonah H. Harris, Senior DBA
myYearbook.com |
|
|
| Back to top |
|
|
|
|
|
All times are GMT
The time now is Wed Nov 25, 2009 2:59 pm
|
|