As already mentioned, the compound file system you've been looking at is implemented in the OLE32 DLL. The structured storage interfaces are flexible enough, however, to permit you to change the underlying implementation. The key to this flexibility is the ILockBytes interface. The StgCreateDocfile and StgOpenStorage global functions use the default Windows file system. You can write your own file access code that implements the ILockBytes interface and then call StgCreateDocfileOnILockBytes or StgOpenStorageOnILockBytes to create or open the file, instead of calling the other global functions.
Rather than implement your own ILockBytes interface, you can call CreateILockBytesOnHGlobal to create a compound file in RAM. If you wanted to put compound files inside a database, you would implement an ILockBytes interface that used the database's blobs (binary large objects).