 |
|
| .NET DotNet Forum Index » VB.NET Forum (Visual Basic .NET) » Hashtable or Datatable?... |
|
Page 1 of 1 |
|
| Author |
Message |
| Andrius B.... |
Posted: Mon Oct 26, 2009 9:29 am |
|
|
|
Guest
|
Hi all.
I used Hashtable to store data loaded from Access (the db can contain from
1000 till 100000 records), and that worked very well, because of fast data
retrieving from Hashtable using key.
The main problem with Hashtable is sorting. I have to convert all the
Hashtable to array list or smth, sort it with the help of ICompairer
interface, and then converting back to hashtable. That takes much time.
Now I am trying to modify the app code and started to use Datatable objects
instead of Hashtable. Datatable has a much faster sorting mechanism, and
that is very important for me, because I have to display the items from the
db in Listview - not all of them, just some (up to 50), but the items must
be ordered by certain key, t.i. the user should have possibility ti order
the items in the way he wishes. That's why fast sorting is neccessary.
ListView is in Virtual mode.
But on the other hand, datatable is not the one I am looking for. The item
retrieving from datatable (using datatable.rows.find (by key) is much slower
than getting items from hashtable. When for sorting and displaying, it is
not very bad, because, as I have said, I need at a moment dislpay only up to
50 items in Listview. But also my program has to make some calculations and
analysis with the data, so when it tries to retrieve data from datatable,
the speed of analyse decreases very markebly, in compairing to hashtable.
So, what could I do, if want to have fast sorting and also fast data
retrieving in my app? Storing the data both in datatable and in hashtable
could be a solution, but it consumes to much memory. Is there a class or
smth, that could provide this? Or should I try to write my own class
implementing many interfaces (for sorting etc.)?
I use VB.Net 2005.
Thanks for any help. |
|
|
| Back to top |
|
|
|
| Mr. Arnold... |
Posted: Mon Oct 26, 2009 10:22 am |
|
|
|
Guest
|
Andrius B. wrote:
Quote: Hi all.
I used Hashtable to store data loaded from Access (the db can contain from
1000 till 100000 records), and that worked very well, because of fast data
retrieving from Hashtable using key.
The main problem with Hashtable is sorting. I have to convert all the
Hashtable to array list or smth, sort it with the help of ICompairer
interface, and then converting back to hashtable. That takes much time.
Now I am trying to modify the app code and started to use Datatable objects
instead of Hashtable. Datatable has a much faster sorting mechanism, and
that is very important for me, because I have to display the items from the
db in Listview - not all of them, just some (up to 50), but the items must
be ordered by certain key, t.i. the user should have possibility ti order
the items in the way he wishes. That's why fast sorting is neccessary.
ListView is in Virtual mode.
But on the other hand, datatable is not the one I am looking for. The item
retrieving from datatable (using datatable.rows.find (by key) is much slower
than getting items from hashtable. When for sorting and displaying, it is
not very bad, because, as I have said, I need at a moment dislpay only up to
50 items in Listview. But also my program has to make some calculations and
analysis with the data, so when it tries to retrieve data from datatable,
the speed of analyse decreases very markebly, in compairing to hashtable.
Yes, a datatable is as slow as it gets and not very scalable.
http://msdn.microsoft.com/en-us/library/dd364983.aspx
<copied>
At the end of the day, however, if you’re using DataTables, the Select
method is one of the worst things you could possibly do to limit the
scalability of your application. We’ve discussed several alternatives
here, including LINQ, indexing tables to make use of Rows.Find, and
morphing the DataTable into a generic collection. Any of these options
will be orders of magnitude faster than Select for a set of records of
reasonable size. You also want to consider paring your set of records
down as much as possible to speed up both aggregation and selection of
records.
<end copy>
Quote: So, what could I do, if want to have fast sorting and also fast data
retrieving in my app? Storing the data both in datatable and in hashtable
could be a solution, but it consumes to much memory. Is there a class or
smth, that could provide this? Or should I try to write my own class
implementing many interfaces (for sorting etc.)?
I use VB.Net 2005.
The example is in C# in the linq, but you can do the same in VB using
List<T>, a generic collection, which can be done VB2005.
http://dotnetslackers.com/Community/blogs/simoneb/archive/2007/06/20/How-to-sort-a-generic-List_3C00_T_3E00_.aspx
It's get/set in VB as opposed to let/set in C#.
http://www.java2s.com/Tutorial/VB/0120__Class-Module/PropertieswithGetterandSetter.htm |
|
|
| Back to top |
|
|
|
| Scott M.... |
Posted: Mon Oct 26, 2009 12:05 pm |
|
|
|
Guest
|
"Mr. Arnold" <Arnold at (no spam) Arnold.com> wrote in message
news:utnHehlVKHA.4704 at (no spam) TK2MSFTNGP06.phx.gbl...
Quote: It's get/set in VB as opposed to let/set in C#.
C# uses get/set.
-Scottt |
|
|
| Back to top |
|
|
|
| Andrius B.... |
Posted: Mon Oct 26, 2009 12:09 pm |
|
|
|
Guest
|
Thanks Mr. Arnold!
I tried to realize Your idea to use only List (of type) to store the data
and then to sort it using to List.Sort(IComparer). Sorting became a little
bit faster then using Hashtable and the indirect sorting (by convert to
arraylist adn back to Hashtable, as I explained in my question). Of course,
the new conception coud not reach the speed of sorting produced by DataTable
I thing I could find some changes in then inner code witch does the
comparing (inside the Compare function) to make the sorting faster.
Thanks again for the suggestion. Nevertheless, it would be better to have
some kind of class that could combine the good things of DataTable and
generic objects :)
Regards.
"Mr. Arnold" <Arnold at (no spam) Arnold.com> wrote in message
news:utnHehlVKHA.4704 at (no spam) TK2MSFTNGP06.phx.gbl...
Quote: Andrius B. wrote:
Hi all.
I used Hashtable to store data loaded from Access (the db can contain
from
1000 till 100000 records), and that worked very well, because of fast
data
retrieving from Hashtable using key.
The main problem with Hashtable is sorting. I have to convert all the
Hashtable to array list or smth, sort it with the help of ICompairer
interface, and then converting back to hashtable. That takes much time.
Now I am trying to modify the app code and started to use Datatable
objects
instead of Hashtable. Datatable has a much faster sorting mechanism, and
that is very important for me, because I have to display the items from
the
db in Listview - not all of them, just some (up to 50), but the items
must
be ordered by certain key, t.i. the user should have possibility ti order
the items in the way he wishes. That's why fast sorting is neccessary.
ListView is in Virtual mode.
But on the other hand, datatable is not the one I am looking for. The
item
retrieving from datatable (using datatable.rows.find (by key) is much
slower
than getting items from hashtable. When for sorting and displaying, it is
not very bad, because, as I have said, I need at a moment dislpay only up
to
50 items in Listview. But also my program has to make some calculations
and
analysis with the data, so when it tries to retrieve data from datatable,
the speed of analyse decreases very markebly, in compairing to
hashtable.
Yes, a datatable is as slow as it gets and not very scalable.
http://msdn.microsoft.com/en-us/library/dd364983.aspx
copied
At the end of the day, however, if you’re using DataTables, the Select
method is one of the worst things you could possibly do to limit the
scalability of your application. We’ve discussed several alternatives
here, including LINQ, indexing tables to make use of Rows.Find, and
morphing the DataTable into a generic collection. Any of these options
will be orders of magnitude faster than Select for a set of records of
reasonable size. You also want to consider paring your set of records down
as much as possible to speed up both aggregation and selection of records.
end copy
So, what could I do, if want to have fast sorting and also fast data
retrieving in my app? Storing the data both in datatable and in hashtable
could be a solution, but it consumes to much memory. Is there a class or
smth, that could provide this? Or should I try to write my own class
implementing many interfaces (for sorting etc.)?
I use VB.Net 2005.
The example is in C# in the linq, but you can do the same in VB using
List<T>, a generic collection, which can be done VB2005.
http://dotnetslackers.com/Community/blogs/simoneb/archive/2007/06/20/How-to-sort-a-generic-List_3C00_T_3E00_.aspx
It's get/set in VB as opposed to let/set in C#.
http://www.java2s.com/Tutorial/VB/0120__Class-Module/PropertieswithGetterandSetter.htm |
|
|
| Back to top |
|
|
|
| Göran Andersson... |
Posted: Mon Oct 26, 2009 12:27 pm |
|
|
|
Guest
|
Scott M. wrote:
Quote: "Mr. Arnold" <Arnold at (no spam) Arnold.com> wrote in message
news:utnHehlVKHA.4704 at (no spam) TK2MSFTNGP06.phx.gbl...
It's get/set in VB as opposed to let/set in C#.
C# uses get/set.
-Scottt
Besides, Let and Set is the same thing. VB6 uses Get and Let, but VB.NET
uses Get and Set.
--
Göran Andersson
_____
http://www.guffa.com |
|
|
| Back to top |
|
|
|
| Tom Shelton... |
Posted: Mon Oct 26, 2009 1:13 pm |
|
|
|
Guest
|
On 2009-10-26, Göran Andersson <guffa at (no spam) guffa.com> wrote:
Quote: Scott M. wrote:
"Mr. Arnold" <Arnold at (no spam) Arnold.com> wrote in message
news:utnHehlVKHA.4704 at (no spam) TK2MSFTNGP06.phx.gbl...
It's get/set in VB as opposed to let/set in C#.
C# uses get/set.
-Scottt
Besides, Let and Set is the same thing. VB6 uses Get and Let, but VB.NET
uses Get and Set.
VB6 uses GET/LET/SET
--
Tom Shelton |
|
|
| Back to top |
|
|
|
| Andrius B.... |
Posted: Tue Oct 27, 2009 12:55 pm |
|
|
|
Guest
|
By the way, I thing the binary search using IComparer interface (witch is
using for sorting such objects as Arraylist) is not the fastest one. Is
there
smth better? I mean, some kind of faster algorithm for sorting custom data
types? |
|
|
| Back to top |
|
|
|
| Tom Shelton... |
Posted: Tue Oct 27, 2009 1:40 pm |
|
|
|
Guest
|
On 2009-10-27, Andrius B. <andriusbl at (no spam) mail.lt> wrote:
Quote: By the way, I thing the binary search using IComparer interface (witch is
using for sorting such objects as Arraylist) is not the fastest one. Is
there
smth better? I mean, some kind of faster algorithm for sorting custom data
types?
The standard sort methods do not use Binary search - they use QuickSort.
--
Tom Shelton |
|
|
| Back to top |
|
|
|
| Andrius B.... |
Posted: Wed Oct 28, 2009 3:06 pm |
|
|
|
Guest
|
Quote: Why not try a SortedList or a SortedDictionary? Just a thought!
Thats a good idea, I tried it, but the problem is, that I must have a
possibility to sort the Items by different key.
E.G. there are 3 ListViewItems (Text and two subitems, let's imagine that we
see Details - View with 3 columns) :
Name: ID: Town:
Item #1: AAA 333 OOO
Item #2: BBB 111 SSS
Item #3: AAA 222 EEE
So, if want to sort those items by ID, it is not difficult. I will create a
New SortedList(Of String, ListViewItem), and add all the items, specifying
the Item.Subitems(1).text as a key. The sorting will be performed as I
expect (t.i., after sort: Item2, Item3, Item1), because the "ID's" are
unique.
But if I want to sort the Items by "Name", things get worse. "Name" is not
unique. So I have to "whisper" to the SortedList, how I would like to order
items, if they have equal "Name" - then ID's schould be compared; if ID's
are also equal - compare the "Town". And how to tell that to SortedList?
SortedList has only one key, not two or three.
So, I have to combine the neccessary subitems when creating a key for
sorting by "Name"
key = Item.text & Item.Subitems(1).text & Item.Subitems(2).text.
Thats works for me (sorting is adequate), but that's the problem - keys
become quite "long". And, of course, when adding a new value to the
SortedList, as far as I know, the key of the new value is being compared
with the keys of values already in SortedList. Am I not right with the idea,
that the comparing of "long" keys takes more time than of "short" ones? And
of course it is not very "logic" to compare by all subitems' text: if the
one item's "Name" differs from the second item's Name, the Sortedlist should
not pay any attention to the ID's or Town's of the Items. |
|
|
| Back to top |
|
|
|
|
|
All times are GMT - 5 Hours
The time now is Tue Dec 01, 2009 9:45 am
|
|