How to avoid / handle VersionStoreOutOfMemory

Jul 4, 2016 at 3:56 PM
I'm new to Esent, and am trying to use PersistentDictionary, but I keep hitting VersionStoreOutOfMemory. Are there any existing docs that describe the conditions that lead to this, what steps can be taken to avoid this, and what can be done at runtime when this occurs?

I've found that increasing MaxVerPages avoids it in my test app, but I'm writing a server app so need a robust recovery mechanism when it does occur.
Jul 4, 2016 at 4:22 PM
Increasing MaxVerPages is the way to avoid VersionStoreOutOfMemory. The more concurrent/high-throughput your application is the more version store you will need.
Jul 4, 2016 at 7:03 PM
Thanks for your prompt response, that's helpful. If you may, I have some further questions. What exactly is the version store? Is it an in-memory buffer that gets filled only when the rate of incoming data exceeds the disk writing throughput? I.e. if the version store fills up due to a period of high activity, will it empty out when the load drops? Or is it some finite resource that, once full, is full forever? Either way, what can I do at runtime if the limit is reached - as is bound to happen at some point?

My usage is as a local cache of remote data. As such, it'll receive a massive influx of data upon first use, then a steady trickle feed as data changes over time. The process needs to continue running for weeks.
Jul 4, 2016 at 7:11 PM
The version store is an in-memory list of changes made by database transactions. The version store is used for rollback and MVCC, specifically:
  • Rollback: when a transaction wants to abort it uses the version store entries to undo all its operations.
  • Concurrency control: version store entries are used to lock records.
  • Snapshot isolation: a session should only see changes that were committed before the current transaction began. The version store is used to generate the correct view of the database as transactions read records.
If a transaction starts but never commits the version store will always fill up because esent has to keep track of all modifications made to the database since the transaction began. This should not be the case with PersistentDictionary because it uses short transactions. In your case I believe that a high throughput rate is creating a lot of versions. They are removed from memory by a background thread so you just need a larger buffer to let the background thread catch up.