Linq of the TValue of a Persistent Dictionary, a bad idea?

Apr 3, 2013 at 11:55 AM
I was testing Persistent Dictionary in the following scenario:
A dictionary with a couple of millions of records, with a simple structure as value (string Tag, string Cluster, int Count).
I wanted to make some hard-coded queries using Linq over that dictionary.
I have tested some simple queries (where(), count() over some of the properties of the structure value) and it takes several minutes. It is terribly slow.
 using (var dict = new PersistentDictionary<string, UseStat>("EsentTestUseStatDb"))
            {
                clustered = dict.Count(x => x.Value.Cluster.Equals("3"));
            }
I just wanted to check with you this is not a proper scenario for Persistent Dictionary, and that I should use "raw esent" in order to get some satisfactory performance.
Apr 3, 2013 at 12:21 PM
Edited Apr 3, 2013 at 12:23 PM
Instead or raw ESENT, you might take a look at the ORM framework I've built on top of it:
http://esentserialize.codeplex.com/

It doesn't support LINQ's IQueriable, however you can define secondary index over your table, and use e.g. Recordset<tRow>.filterFindEqual() method to query the index for the records you need. And then e.g. Recordset<tRow>.Count() to cound the records without de-serializing them.
Apr 3, 2013 at 7:12 PM
Linq is only efficient if you are querying the keys of the dictionary not the values -- there is no index over the values so they all have to be enumerated for each query.
Apr 3, 2013 at 9:29 PM
Thank you both. If I eventually need more query support, I will go with a "higher level" database.