Hi Greg, Long time no chat... What I have done is have a list of objects and had multiple dictionaries indexing it. An object that is in a dozen data structures only exists in memory once, so it is cheap to setup the extra indexes. You can get some great performance from it. As soon as you put a database in there, you have to get the data in and out of the database, that can be very time consuming depending on what you are doing. It sounds like you are going to need an dictionary of lists, to solve the duplicate non-unique key problem. All the best Greg H Greg Harris Sydney Australia On Sat, Sep 28, 2024 at 9:47 AM Greg Keogh via ozdotnet < ozdotnet@ozdotnet.com> wrote:
Hello folks, we've found a performance bottleneck searching for items in a large in-memory collection. Most of the code was converted from 15+ year old C++ code into C# so a lot of weird looking stuff came across. There are lots of sequential searches for items in arrays and List<> which have complexity O(n) and get really slow for large collections. I previously converted one bottleneck List<> into a Dictionary<> and it was a miracle cure, but it was a simple case with one property value being a unique key.
The next bottleneck collection needs to be searched efficiently on different property values, and I'm wondering if anyone knows of a collection which supports multiple indexes (possibly non-unique).
An in-memory SQLite table with multiple indexes would do the job, but it's a bit heavyweight and like hammering a screw. Using multiple Dictionaries or Sets to index the same objects won't work due to non-unique values. Maybe there is some lightweight package around for this sort of thing? It's a classic computer science problem.
*Greg Keogh* -- ozdotnet mailing list To manage your subscription, access archives: https://codify.mailman3.com/