Main Page | Report Page

 

  Computers Forum Index » Computer Languages (IDL-PVWAVE) » Still missing features in IDL 8...

Author Message
Paulo Penteado...
Posted: Tue Oct 12, 2010 9:35 pm
 
I was showing IDL's lists and hashes to a friend, who immediately
noticed two important missing points:

1) A new method (or a new keyword, like, /pointer, to the existing
toarray()), that would create from an arbitrary list an array of
pointers, where each element in the array is a pointer to the
corresponding element of the list. That is, something equivalent to

function topointerarray,list
ret=ptrarr(n_elements(list))
foreach el,list,i do ret[i]=ptr_new(el)
return,ret
end

And a new keyword to list() and list::add (such as /dereference) that
would do the opposite: given a pointer array, would add to the list
the variables pointed to by each element of the pointer array (or !
null in case the element is a null pointer or a pointer to an
undefined variable).

Also, in some cases it would be desirable for list::toarray() and
hash::tostruct() to have a no_copy keyword, that would empty the list/
hash when making the array/structure, instead of making a copy. This
is already present in list() and list::add, but is also missing in
hash().

Why is (1) so needed? Besides better handling those conversions
(particularly when interfacing with old code that returns/expects
pointer arrays), for getting around the lack of:

2) This is not a simple additional feature as (1), but rather a change
to the language: Make a way for a list/hash element not be an
expression. One example of the problem is this error:

IDL> a=list()
IDL> a.add,bindgen(3)
IDL> a.add,-dindgen(2)
IDL> print,a
0 1 2
-0.0000000 -1.0000000
IDL> print,(a[1])[0]
-0.0000000
IDL> (a[1])[0]=1.0
% Expression must be named variable in this context: <DOUBLE
Array[2]>.
% Execution halted at: $MAIN$

This error happend because (a[1]) is an expression:

IDL> help,(a[1])
<Expression> DOUBLE = Array[2]

And I can see why it is, as I guess it is the result of a call to
list::overloadbracketsrightside(). But this makes it very inconvenient
to change pieces of list/hash elements, like in the line that caused
the error above. It would be necessary to retrieve the element,
assigning it to some variable, change that variable, then put it back
in the list/hash. Very awkward, and unnecessary if a pointer array is
used instead of the list:

IDL> a=ptrarr(2)
IDL> a[0]=ptr_new(bindgen(3))
IDL> a[1]=ptr_new(-dindgen(2))
IDL> print,(*a[1])[0]
-0.0000000
IDL> (*a[1])[0]=1.0
IDL> print,(*a[1])[0]
1.0000000

Which obviously works because (*a[1]) is a variable:

IDL> help,(*a[1])
<PtrHeapVar2> DOUBLE = Array[2]

I realize that this is a change to the language, and somewhat tricky
to implement. But it is probably easier than was the addition of
operator overloading.
 
Chris Torrence...
Posted: Wed Oct 13, 2010 4:38 pm
 
Hi Paulo,

Great ideas! I just added #1 to the list (ha!) of potential features
for IDL 8.1.

Regarding #2, what if you could use additional indices to access array
elements within lists?

For example:

IDL> a = LIST(FINDGEN(10), BYTARR(5,3))
IDL> help, a[0]
<Expression> FLOAT = Array[10]
IDL> help, a[0,3] ; currently throws an error in IDL8.0
<Expression> FLOAT = 3.00000
IDL> a[0,3] = !pi ; currently throws an error in IDL8.0

IDL> help, a[1]
<Expression> BYTE = Array[5, 3]
IDL> help, a[1,4,2] ; currently throws an error in IDL8.0
<Expression> BYTE = 0
IDL> a[1,4,2] = 255 ; currently throws an error in IDL8.0

So the first index would give the list element, and the remaining
indices would index into the array itself. Obviously you could only
have up to 7 dimensions in your contained array, but that probably
isn't a huge limitation.

Thoughts?

-Chris
ITTVIS
 
Paulo Penteado...
Posted: Fri Oct 15, 2010 12:26 am
 
On Oct 13, 1:38 pm, Chris Torrence <gorth... at (no spam) gmail.com> wrote:
Quote:
Regarding #2, what if you could use additional indices to access array
elements within lists?

For example:

IDL> a = LIST(FINDGEN(10), BYTARR(5,3))
IDL> help, a[0]
Expression>    FLOAT     = Array[10]
IDL> help, a[0,3]   ; currently throws an error in IDL8.0
Expression>    FLOAT     =       3.00000
IDL> a[0,3] = !pi   ; currently throws an error in IDL8.0

IDL> help, a[1]
Expression>    BYTE      = Array[5, 3]
IDL> help, a[1,4,2]   ; currently throws an error in IDL8.0
Expression>    BYTE      =    0
IDL> a[1,4,2] = 255   ; currently throws an error in IDL8.0

So the first index would give the list element, and the remaining
indices would index into the array itself. Obviously you could only
have up to 7 dimensions in your contained array, but that probably
isn't a huge limitation.

Thoughts?

It took me a while to answer because I was considering the
implications. It is a very clever idea. As I wrote above, I thought
there would be no way to do this without changing the interpreter, and
thus this would be a big problem to implement. But using additional
indices means no new syntax, it is just a regular call to
overloadbracketsrightside(). It seems it could even be implemented
today, with IDL code, with a class inheriting from list. And this
addition would be a very small break in compatibility: it would only
break in the sense that what now throws an error (like a[0,3] above)
would become valid. Small break, and dependent on a recent feature - a
reason to implement this soon, while the likelihood of breaking code
is still low.

I had been trying to figure out what the downsides could be. The clear
limitation is the loss of one dimension to subscript the array, as you
mentioned. How relevant this is in part depends on whether there are
any discussions about changing the two 8D limits (the limit for
arrays, which is probably deeply ingrained in many places in the
interpreter and the API, and the limit for the overloadbrackets
methods, which was introduced now, and can probably change much more
easily).

My impression so far is that it would be good to implement this now.
8D arrays are probably very rare, and when one is dealing with 8
indices (or ranges of indices), things are probably somewhat unwieldy
anyway, as in any situation where one is counting positional things
(indices or positional arguments). The more likely situation for this
problem to appear seems to be with a few nested lists, with 3-4D
arrays, where each list would consume one index, but again, it would
be an unwieldy situation without the limitation anyway, and other data
structures, or separating the code in more than one expression, might
be more appropriate.

Maybe I will write a derived class now and start using it, to see if I
encounter any unforeseen problems. Of course, for this to come with
IDL's lists, it would be better to have it not written in IDL, as was
the case with the implementation of lists and hashes. By the way, are
the current lists and hashes like DLMs? Are they in the interpreter?
 
Bob...
Posted: Thu Nov 04, 2010 6:29 pm
 
On Nov 1, 9:30 am, Chris Torrence <gorth... at (no spam) gmail.com> wrote:
Quote:
On Oct 31, 6:00 pm, Paulo Penteado <pp.pente... at (no spam) gmail.com> wrote:



On Oct 13, 2:38 pm, Chris Torrence <gorth... at (no spam) gmail.com> wrote:

Regarding #2, what if you could use additional indices to access array
elements within lists?

For example:

IDL> a = LIST(FINDGEN(10), BYTARR(5,3))
IDL> help, a[0]
Expression>    FLOAT     = Array[10]
IDL> help, a[0,3]   ; currently throws an error in IDL8.0
Expression>    FLOAT     =       3.00000
IDL> a[0,3] = !pi   ; currently throws an error in IDL8.0

IDL> help, a[1]
Expression>    BYTE      = Array[5, 3]
IDL> help, a[1,4,2]   ; currently throws an error in IDL8.0
Expression>    BYTE      =    0
IDL> a[1,4,2] = 255   ; currently throws an error in IDL8.0

So the first index would give the list element, and the remaining
indices would index into the array itself. Obviously you could only
have up to 7 dimensions in your contained array, but that probably
isn't a huge limitation.

I was writing a class like that, inheriting from list, and that
brought me a question: Should the extra dimension (of the list index)
be on the left, as above, or on the right?

The notation (already valid for retrieving values) (a[1])[0] suggests
that the array index should come on the left. However, writing a[1,0]
suggests array dimensions, in which case the list index would make
more sense on the right, as the list dimension is the slowest-varying
one.

Tough it would be a bit incoherent with the array dimension order, it
seems to me that it is better to have the list index on the left. That
way,

print,(a[1])[0] ;already valid

would be the same as

print,a[1,0]

instead of the more confusing

print,a[0,1]

Any thoughts on that?

Yes, that is exactly what I was thinking.

Back to your original thread - if we added this way of subscripting,
does that eliminate the need to convert a list to/from a pointer
array? I'd rather not add more functionality if we don't have to.

-Chris
ITTVIS

It seems to me that adding array subscripting to the list object is
wrong since lists can take many more types of objects than arrays.
This seems to be a product of IDL think that "everything is an array",
which does not make sense anymore in 8.0. I think the proper way to
implement the feature Paulo is asking for is to have an array_list
object, which would only take arrays and could have overloaded
brackets to get the extra subscripting. I think this is what Paulo is
implementing already and it would seem it could be completely
implemented with no changes to the language. Perhaps if you wanted to
implement it in the language, an ARRAY keyword could be added to the
list::init to force the list to only accept arrays and only if that is
set then the bracket overloading could be added. It still seems
cleaner to have a subclass object, however.

Bob
 
Paulo Penteado...
Posted: Thu Nov 04, 2010 10:28 pm
 
On Nov 4, 4:29 pm, Bob <bobnnamt... at (no spam) gmail.com> wrote:
Quote:
It seems to me that adding array subscripting to the list object is
wrong since lists can take many more types of objects than arrays.
This seems to be a product of IDL think that "everything is an array",
which does not make sense anymore in 8.0.  I think the proper way to
implement the feature Paulo is asking for is to have an array_list
object, which would only take arrays and could have overloaded
brackets to get the extra subscripting.  I think this is what Paulo is
implementing already and it would seem it could be completely
implemented with no changes to the language.  Perhaps if you wanted to
implement it in the language, an ARRAY keyword could be added to the
list::init to force the list to only accept arrays and only if that is
set then the bracket overloading could be added.  It still seems
cleaner to have a subclass object, however.

I disagree. I find bracket overloading to be one of the best points in
IDL 8, and that brackets are proper for anything that may make use of
them: arrays, lists, hashes, and whatever other classes, like those
used by Graphics.

For that reason one important consideration in the class I am writing
is to make sure that the other subscripts (those that come after the
subscript for the list) will be passed on to whatever is in the list
make use of them. I was initially thinking about only arrays in lists
because that was the first need I encountered. But I realized that
other subscriptable things should make use of that too. When that list
class gets done and tested, I will probably extend that to a hash
class, using some intermediate class to be inherited, which could be
used for any other class that wanted that kind of forwarding for extra
indices.

It is up to the user to know in what way the things being accessed can
be subscripted, in the same way that the user must know, for instance,
not to use more dimensions than an array has when subscripting it.
 
 
Page 1 of 1    
All times are GMT
The time now is Tue Jul 29, 2014 8:46 am