ESENT Managed Interface

Closed Issue: Windows7Grbits - Compressed bits? [10214]

martinc — Thu, 16 May 2013 10:56:24 GMT

Seems like Compressed and Uncompressed is the same value?

/// <summary>
/// Try to compress the data when storing it.
/// </summary>
public const SetColumnGrbit Compressed = (SetColumnGrbit)0x20000;

/// <summary>
/// Don't compress the data when storing it.
/// </summary>
public const SetColumnGrbit Uncompressed = (SetColumnGrbit)0x20000;

Closed Issue: Windows7Grbits - Compressed bits? [10215]

martinc — Thu, 16 May 2013 10:56:23 GMT

New Post: Out of table cursors - error

bayganik — Thu, 09 May 2013 20:29:40 GMT

What does this error mean? Too many JetSessions opened at same time?

Our library creates ONE instance and allows for multiple Sessions.

Error Message = "Out of table cursors"
at Microsoft.Isam.Esent.Interop.Api.Check(Int32 err) in c:\Users\aclark\Documents\tfs\VisualDesignManager\Trunk\ESE_Managed\EsentInterop\Api.cs:line 2856
\r\n at Microsoft.Isam.Esent.Interop.Api.JetGetObjectInfo(JET_SESID sesid, JET_DBID dbid, JET_OBJECTLIST& objectlist) in c:\Users\aclark\Documents\tfs\VisualDesignManager\Trunk\ESE_Managed\EsentInterop\Api.cs:line 1510
\r\n at Microsoft.Isam.Esent.Interop.TableNameEnumerator.OpenTable() in c:\Users\aclark\Documents\tfs\VisualDesignManager\Trunk\ESE_Managed\EsentInterop\TableNameEnumerator.cs:line 46
\r\n at Microsoft.Isam.Esent.Interop.TableEnumerator`1.MoveNext() in c:\Users\aclark\Documents\tfs\VisualDesignManager\Trunk\ESE_Managed\EsentInterop\TableEnumerator.cs:line 102
\r\n at eXpedite.WinRTLib.DBOps.JetSession..ctor(JetInstance dbInstance) in c:\Users\aclark\Documents\tfs\VisualDesignManager\Trunk\Common\eXpedite.WinRTLib\DBOps\JetSession.cs:line 70
\r\n at eXpedite.ViewModel.TaskViewModel.OpenFormRelation(FormRelationMessage msg) in c:\Users\aclark\Documents\tfs\VisualDesignManager\Trunk\eXpedite.Project\eXpedite.Client\eXpedite\ViewModel\TaskViewModel.cs:line 684"

New Post: Dealing with column default values

martinc — Thu, 02 May 2013 22:53:34 GMT

Now I'm picturing you in Thames House in a Spooks episode, or working at Bletchley Park or GCHQ or something, all James Bond-like. :)

You should try the procmon route to figure out which DLL.

By the way I asked around for another person on another issue, and the answers were:
'IIRC, Microsoft has a specialized team that deals with Law Enforcement Support. '
'The special team that assists law enforcement with these questions is generally CSS, after law enforcement has gone through the LE portal run by the DCU. '

And the glossary:
To translate:
CSS -- product support
LE -- law enforcement
DCU -- ???

But MS in UK may have different policies and procedures.

-martin

New Post: Nuget package targeted 4.0 runtime

martinc — Thu, 02 May 2013 22:40:30 GMT

Thanks for the report!

I received conflicting comments about whether the newest ManagedEsent bits actually needed 4.5 or not. Some people reported that despite it being compiled against 4.5 that it didn't actually need 4.5 assemblies.
Apparently those reports are inaccurate after all. 4.5 assemblies ARE needed after all.

-martin

New Post: Nuget package targeted 4.0 runtime

fcschow — Thu, 02 May 2013 18:49:36 GMT

1.8.3.2 seems to require 4.5 runtime. Got following runtime error when installed from NuGet package.

Could not load type 'System.Reflection.IntrospectionExtensions' from assembly 'mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'.

New Post: Dealing with column default values

phil_h — Tue, 30 Apr 2013 14:49:09 GMT

Hi martin,

Sorry I should have explained myself a bit better - I actually work for a Forensic Services provider in the UK, so we are actually involved in investigating criminal cases for Law Enforcement etc. Our interest, particularly in the "Windows.edb" database but also other relevant ESE databases, is in being able to examine these databases to produce evidence in a suitable format. I believe that Microsoft do indeed have their own forensic team, but we wouldn't typically engage them ourselves :-)

Cheers,

Phil H

New Post: Dealing with column default values

martinc — Tue, 30 Apr 2013 11:06:29 GMT

You mentioned forensics. If you have an actual criminal case, then you really should contact Microsoft Support. They offer forensic services for Law Enforcement purposes.

To be honest I have no idea how much they charge. It might be a few hundred dollars, which in a criminal case, you'll hopefully have budget for. There is also a chance that there is no charge.

-martin

New Post: Dealing with column default values

phil_h — Tue, 30 Apr 2013 09:28:31 GMT

Thanks for all all the comments, I'll take a look at the JET_USERDEFINEDDEFAULT structure and see if I can make any useful sense out of it. I see what you mean about the different security context - it may be better to simply ignore (or flag) these columns, but do nothing with them. In fact I should've thought of martinc's comment re:native code - it makes sense that this would be the case, in which case there is very little likelihood of being able to integrate any such DLL correctly.

With regards to martinc's comment re:the Search Team frowning sincerely on digging through their database etc; I have no issue with them changing the internal format, after all they've done it a number of times already when moving between different versions of the OS :-) As for digging through their database, the reason for my interest is the potential forensic information that may be contained therein (e.g. filenames that have been indexed, etc) - I'm certainly not planning on creating anything that interferes with the Search process, I'm purely interested in being able to identify forensically relevant information that may be contained within the database :-)

Cheers,

Phil H

New Post: PersistentDictionary and Transactions

tronda — Tue, 30 Apr 2013 08:47:39 GMT

Agreed. TransactionScope should really be used and supported by ManagedEsent.

New Post: Dealing with column default values

martinc — Tue, 30 Apr 2013 04:48:26 GMT

First of all, I'm pretty sure that the Search Team frowns sincerely on digging through their database. And they reserve the right to change the internal format at any time.

You can examine the DLLs loaded by the Search process.
You can also use procmon.exe to see which mysterious DLLs are attempted to be loaded by your program, specifically at the moment you try the RetrieveColumn call.

If by 'adding a reference to the DLL' you mean that it's a CLR DLL, I'm pretty sure it would be written in native code.

(Hmm, I can't remember if I wrote that bit on MSDN or not... I must be getting old. :)

-martin

New Post: Dealing with column default values

Const_me — Tue, 30 Apr 2013 04:34:33 GMT

Hi Phil,

The byte[] value is JET_USERDEFINEDDEFAULT structure. Please read this article on MSDN.

As you can see, it recommends specifying the absolute path to the DLL containing the callback. ESENT loads that DLL itself, adding a reference will do nothing.

You might get that exception because of the different security contexts: remember the windows search service is normally running in the local system security context.

Or, the DLL implementing the callback might interoperate with anything external to provide that value: another component of windows search, another windows service, device driver, LAN/WAN, etc.. In this case, I won't be surprised if you won't be able to use that callback in your application.

New Post: Dealing with column default values

phil_h — Mon, 29 Apr 2013 09:55:47 GMT

martinc wrote:

Correct, it's a 'User defined value'. It can be thought of as a 'virtual column', and when ESE tries to retrieve the value it instead calls a user-supplied function.

You could grab the DLL and copy it somewhere so that it gets loaded properly, but I'm assuming that this is just experimentation, in which case it may just be easier to use a different database altogether -- one that doesn't use callback functions. :)

-martin

Hi Martin,

Thanks for that information - so that'd explain why I get the "A callback failed" exception, since the callback function isn't there. So what is the byte[] value that the ManagedEsent library has stored in the "ColumnInfo.DefaultValue" ?

Although this is just experimentation at the moment, I will be wanting to parse the "Windows.edb" database eventually so I guess I need to hunt down the DLL where I can find this callback function and add a reference to it ...

Phil H

New Post: JET_errAttachedDatabaseMismatch

admin503 — Mon, 29 Apr 2013 09:13:46 GMT

Sorry to post it here, but it is related to disconnected or attached mailbox that's why i refereed this.

New Post: JET_errAttachedDatabaseMismatch

martinc — Mon, 29 Apr 2013 08:50:03 GMT

admin503, how is your URL related to the topic? It seems to be promoting a separate piece of software...

-martin

New Post: Dealing with column default values

martinc — Mon, 29 Apr 2013 08:44:26 GMT

Correct, it's a 'User defined value'. It can be thought of as a 'virtual column', and when ESE tries to retrieve the value it instead calls a user-supplied function.

You could grab the DLL and copy it somewhere so that it gets loaded properly, but I'm assuming that this is just experimentation, in which case it may just be easier to use a different database altogether -- one that doesn't use callback functions. :)

-martin

New Post: Dealing with column default values

phil_h — Mon, 29 Apr 2013 08:16:48 GMT

Hmm,

On further testing it appears that the "A callback failed" exception is thrown for these columns regardless of which "Api.RetrieveColumnAs..." method is used. Is this because these columns, for whatever reason, don't have a value and the "DefaultValue" needs to be used instead - this still appears to be the only difference between the three failing column retrievals, and the other working column retrievals ?

Phil H

New Post: JET_errAttachedDatabaseMismatch

admin503 — Sat, 27 Apr 2013 17:08:53 GMT

Just check the my post, you will get real solution to overcome here:

http://www.edb-pst.net/export-disconnected-mailbox-to-pst.html

New Post: Dealing with column default values

phil_h — Fri, 26 Apr 2013 09:39:56 GMT

Hi all,

I'm just starting to develop an application using the ManagedEsent library and have been testing some functionality using a copy of a "Windows.edb" database file. I've encountered a problem when attempting to retrieve three particular columns from one of the available tables - specifically the "A callback failed" exception is thrown when attempting to retrieve the column data.

In all three cases the column data type is "LongText" and the exception is thrown when I attempt to retrieve the column data using the "Api.RetrieveColumnAsString(...)" method. I've stepped through the code and other columns, whose data type is "LongText", are retrieved correctly so I don't believe that I have a fundamental problem with this data type - the only difference I've noted is that the three columns have a byte[] specified as a "DefaultValue" (as observed by inspecting the "ColumnInfo" object for each column), whereas all the other columns have this "DefaultValue" set to null.

So I guess my question is two-fold - why is this "DefaultValue" causing the failure of the "Api.RetrieveColumnAsString(...)" method, and how can I trap for this and deal with it accordingly ?

Phil H

New Post: Running ManagedEsent with Linq support on .NET framework 4.0

laurionb — Tue, 16 Apr 2013 18:27:17 GMT

The tests in the solution do give very good test coverage. If you want to run the complete test suite you will need IronPython installed. The test_persistentdictionary.py script (in the EsentCollectionsTests directory) gives exhaustive, multithreaded coverage.