This project has moved and is read-only. For the latest updates, please go here.

Attaching a db that needs defraging fails (EnableIndexChecking = false)

Jan 27, 2013 at 8:49 AM

Hi,

We are trying to run the following code to defrag a database:

JET_INSTANCE instance;
Api.JetCreateInstance(out instance, "defrag");

new InstanceParameters(instance)
{
	EnableIndexChecking = false
};

Api.JetInit(ref instance);

JET_SESID sesid;
Api.JetBeginSession(instance, out sesid, null, null);

Api.JetAttachDatabase(sesid, @"C:\Work\ravendb\Raven.Server\bin\Debug\Data\Databases\234\Data",
					  AttachDatabaseGrbit.None);

JET_DBID dbid;
Api.JetOpenDatabase(sesid, @"C:\Work\ravendb\Raven.Server\bin\Debug\Data\Databases\234\Data", null, out dbid,
                    OpenDatabaseGrbit.None);

int passes = 10;
int seconds = 600;
Api.JetDefragment2(sesid, dbid, null, ref passes, ref seconds, (jetSesid, jetDbid, tableid, cbtyp, o, o1, context,
                                                                unused) =>
{
	Console.WriteLine(cbtyp);
	return JET_err.Success;
}, DefragGrbit.BatchStart);

Console.WriteLine("Wait until OnlineDefragCompleted is printed");
Console.ReadLine();

The problem is that we are getting the following error:

Unhandled Exception: Microsoft.Isam.Esent.Interop.EsentSecondaryIndexCorruptedException: Secondary index is corrupt. The database must be defragmented

As I am trying to defrag the db for that purpose exactly, I am somewhat at a loss. The docs for EnableIndexChecking and JetAttachDatabase say that this should work.

We are testing on Windows 8, by the way. And it was created on a Windows 2008 R2 machine, so I expected to have to defrag, it is just that I don't know how to make it work.

I know I can run esentutl -d, but I want to do this using the API, and I am pretty sure that I am missing something. The database size is about 600MB.

Jan 27, 2013 at 9:14 AM

The problem is that the API names are a complete mess. The function you want to call is JetCompact, which makes an out-of-place copy of a database.

If you don't want to double the disk space you could try dropping all the indexes that contain unicode columns and then recreating them. That will also fix any sort order issues.

Jan 27, 2013 at 9:16 AM

Would Compact also defrag things and sort the indexes?

Jan 27, 2013 at 9:18 AM

Yes, JetCompact defrags and sorts all the indexes.

Jan 28, 2013 at 6:23 AM

There is still a problem.

According to this: http://managedesent.codeplex.com/discussions/361315

I need to call JetAttachDatabase before calling JetCompact.

And that is the part that fails for me.

Jan 28, 2013 at 6:30 AM

When calling JetCompact without first calling JetAttachDatabase, we get :"No such database" error.

Jan 28, 2013 at 8:13 AM

JET_paramEnableIndexChecking is pretty messy. It actually defaults to false. You'll want to also set JET_paramEnableIndexCleanup to false as well. Those two parameters should be set to the same value (either both true; or both false).

Here's what the header file says about these parameters. Hmm, just noticed that 'behaviour' is misspelled. Time to fix that...

//	Index-checking parameters
//
//	After Windows 7, it was discovered that JET_paramEnableIndexCleanup had some implementation limitations, reducing its effectiveness.
//	Rather than update it to work with locale names, the functionality is removed altogether.
//
//	Unfortunately JET_paramEnableIndexCleanup can not be ignored altogether. JET_paramEnableIndexChecking defaults to false, so if
//	JET_paramEnableIndexCleanup were to be removed entirely, then by default there were would be no checks for NLS changes!
//
//	The current behavious (when enabled) is to track the language sort versions for the indices, and when the sort version for that
//	particular locale changes, the engine knows which indices are now invalid. For example, if the sort version for only "de-de" changes,
//	then the "de-de" indices are invalid, but the "en-us" indices will be fine.
//
//	IN SUMMARY:
//	New code should explicitly set both IndexChecking and IndexCleanup to the same value.
//
//
//	OLDER NOTES (up to and including Windows 7)
//
//	Different versions of windows normalize unicode text in different ways. That means indexes built under one version of Windows may
//	not work on other versions. Windows Server 2003 Beta 3 introduced GetNLSVersion() which can be used to determine the version of unicode normalization
//	that the OS currently provides. Indexes built in server 2003 are flagged with the version of unicode normalization that they were
//	built with (older indexes have no version information). Most unicode normalization changes consist of adding new characters -- codepoints
//	which were previously undefined are defined and normalize differently. Thus, if binary data is stored in a unicode column it will normalize
//	differently as new codepoints are defined.
//
//	As of Windows Server 2003 RC1 ESENT tracks unicode index entries that contain undefined codepoints. These can be used to fixup an index when the
//	set of defined unicode characters changes.
//
//	These parameters control what happens when ESENT attaches to a database that was last used under a different build of the OS (the OS version
//	is stamped in the database header).
//
//	If JET_paramEnableIndexChecking is TRUE JetAttachDatabase() will delete indexes if JET_bitDbDeleteCorruptIndexes or return an error if
//	the grbit was not specified and there are indexes which need deletion. If it is set to FALSE then JetAttachDatabase() will succeed, even
//	if there are potentially corrupt indexes.
//
//	If JET_paramEnableIndexCleanup is set, the internal fixup table will be used to fixup index entries. This may not fixup all index corruptions
//	but will be transparent to the application.
//


-martin