Problem querying a IE 11 WebCacheV01.dat file

Feb 12, 2015 at 10:35 AM
Hi,
I am trying to query some tables on a IE 11 WebCacheV01.dat file but looks like ESENTManaged cannot find records on some of them. I have opened the same db file on NirSoft ESEDatabaseView and it can find records on them.

For both History tables my script cannot go to the first record using TryMoveFirst, so it thinks the table is empty.

Here is my code:
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Isam.Esent.Interop;

namespace ConsoleESEDBApplication
{
    class Program
    {
        static void Main(string[] args)
        {

            int pageSize;
            String fileName = @"C:\Users\dummy\Downloads\WebCacheV01.dat";

            try
            {
                Api.JetGetDatabaseFileInfo(fileName, out pageSize, JET_DbInfo.PageSize);
                SystemParameters.DatabasePageSize = pageSize;

                using (var instance = new Instance("SuperTimeline"))
                {
                    var param = new InstanceParameters(instance);
                    param.Recovery = false;
                    instance.Init();
                    using (var session = new Session(instance))
                    {
                        Api.JetAttachDatabase(session, fileName, AttachDatabaseGrbit.ReadOnly);
                        JET_DBID dbid;
                        Api.JetOpenDatabase(session, fileName, null, out dbid, OpenDatabaseGrbit.ReadOnly);

                        using (var tableContainers = new Table(session, dbid, "Containers", OpenTableGrbit.ReadOnly))
                        {
                            IDictionary<string, JET_COLUMNID> columnIds = Api.GetColumnDictionary(session, tableContainers);

                            if (Api.TryMoveFirst(session, tableContainers))
                            {
                                do
                                {

                                    var containerId = (int)Api.RetrieveColumnAsInt32(session, tableContainers, columnIds["ContainerId"]);
                                    var name = Api.RetrieveColumnAsString(session, tableContainers, columnIds["Name"], Encoding.Unicode);

                                    using (var table = new Table(session, dbid, "Container_" + containerId, OpenTableGrbit.ReadOnly))
                                    {
                                        if (Api.TryMoveFirst(session, table))
                                        {
                                            Console.WriteLine(name + " | Container_" + containerId + " | has records!!!");
                                        }
                                        else
                                        {
                                            Console.WriteLine(name + " | Container_" + containerId + " | does not have records!!!");
                                        }
                                    }

                                } while (Api.TryMoveNext(session, tableContainers));
                            }
                        }
                    }
                }

            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
            }

            Console.ReadLine();

        }
    }
}
Developer
Feb 12, 2015 at 6:36 PM
I'm not sure whether it's good news or bad news, but your code works fine for me:
History\0 | Container_1 | has records!!!
Content\0 | Container_2 | has records!!!
iecompat\0 | Container_3 | does not have records!!!
No such table or object
And the 'Container_3' table actually has zero records, so that is correct.
It fails trying to open 'Container_4', and for my database, there is no 'Container_4'. So that's expected, too.


-martin
Feb 15, 2015 at 12:12 PM
Hi,

Thanks for testing the script. I have uploaded the WebCacheV01.dat I am experiencing problems to parse. Please try to open it on both my script and Nirsoft ESEdb viewer. You will see that Container_4 has data, but my script cannot see it.

https://dl.dropboxusercontent.com/u/11506499/Administrator_WebCacheV01.zip

Thanks in advance.
Developer
Feb 17, 2015 at 7:10 PM
You code is correct. Nirsoft is not.
esentutl -mm -v Administrator_WebCacheV01.dat
******************************* META-DATA DUMP *******************************
Name                                               Type    ObjidFDP    PgnoFDP
==============================================================================
Admin                                               Db            1          1
...
Container_4                                         Tbl          11         45
  HashEntryIdIndex                                  Pri          11         45
From that, we see that that the first page is page #45. So let's dump that:
esentutl -mm -v Administrator_WebCacheV01.dat -p45
TAG   0    cb:0x0010    ib:0x0000    offset:0x0050-0x0060    flags:0x0000 (   )
TAG   1    cb:0x0210    ib:0x02b9    offset:0x0309-0x0519    flags:0x0003 (vd )
TAG   2    cb:0x01ed    ib:0x04c9    offset:0x0519-0x0706    flags:0x0003 (vd )
TAG   3    cb:0x02a9    ib:0x0010    offset:0x0060-0x0309    flags:0x0003 (vd )
TAG   4    cb:0x02a9    ib:0x115b    offset:0x11ab-0x1454    flags:0x0003 (vd )
TAG   5    cb:0x02a1    ib:0x1404    offset:0x1454-0x16f5    flags:0x0003 (vd )
TAG   6    cb:0x02a1    ib:0x06b6    offset:0x0706-0x09a7    flags:0x0003 (vd )
TAG   7    cb:0x02a1    ib:0x0957    offset:0x09a7-0x0c48    flags:0x0003 (vd )
TAG   8    cb:0x02c2    ib:0x0bf8    offset:0x0c48-0x0f0a    flags:0x0003 (vd )
The 'd' in the flags means 'deleted'. ESE will 'soft-delete' the records first, and then later clean them up. These rows have been 'soft-deleted', and are therefore not valid.

Nirsoft says "esent.dll (The dll file of Extensible Storage Engine) is not required to read the database". That means they've written their own code to access the database file, and can get differing results.

-martin
Feb 17, 2015 at 11:51 PM
Great. Thanks for your reply.

Do you know if ESENT Managed can retrieve soft-deleted records? If so, how?

As I work as a computer forensic analyst, it would be very helpful to have a script that is able to extract even soft-deleted records from a IE 10+ database file.

Thanks again for all your help.
Developer
Feb 17, 2015 at 11:59 PM
No. The esent.dll code hides it completely. You'd need to reverse-engineer the page structure (as Nirsoft has apparently done). Of course, the legality of reverse-engineering is out of scope for me to comment on. :)

I can see how it would be useful for forensics, but there just isn't much customer demand to expose it.

-martin
Feb 18, 2015 at 12:01 AM
I agree.

Thank you very much.