Pismo File Mount Development Kit build 184

Pismo Technic Inc. Copyright 2006-2017 Joe Lowe
2017.01.04

Preface

This document is targeted towards software developers who are utilizing Pismo File Mount to implement or control file systems or pseudo file systems.

Contents

Pismo File Mount

Description

Pismo File Mount is a portable operating system extension that enables the development of file systems and file system extensions using common user mode application development tools.

Applications can utilize Pismo File Mount to expose data through the file system as mounted volumes. Volumes are exposed through drive letters, UNC paths, or through a mount point on the system volume.

PFM is cross platform, allowing implementation of file systems for Windows, Mac, and Linux operating systems.

Architecture

PFM fundamentally implements a client-server architecture where the client sends file system requests to a local or remote server for processing.

The client is the PFM kernel module (driver), and integrates with the local operating system to allow the system and applications to access the file system provided by the server.

The server is the local or remote application that utilizes PFM to expose file system data.

Portability

PFM is available for Windows, Mac, and Linux operating systems. The PFM API and interfaces are consistent across all three platforms.

The PFM API and interfaces can be used and implemented via a variety of programming languages, including:

PFM operating system support and programming language flexibility make it an efficient solution for implementing portable file system projects and products.

PFM Kernel Module

The core of the Pismo File Mount system extension is a kernel module that interfaces with the various core operating system interfaces, providing an installable file system that redirects application requests to a user mode or remote file system implementation.

PFM Protocol

The communication between the PFM kernel module and the file system is done over a socket using the PFM Protocol. Most developers have no need to deal directly with the PFM Protocol.

Servers

PFM Servers are any applications that utilize PFM to expose file system data.

The PFM Audit Package is an application that contains a variety of PFM server imlpementations. These servers are referred to as "Formatters", each implemented in a loadable DLL that can decode container and/or archive files and expose the contents through the file system.

Other applications will typically implement PFM servers directly using the PFM API, not using the PFM Audit Package formatter architecture.

PFM Application Programming Interface

Applications interact with and control PFM using the PfmApi interface and the associated interfaces PfmMount, PfmFileMount , and PfmMarshaller.

Command Line Interface

Pismo File Mount includes a command line interface. This is implemented as the "pfm" command, usable from a command or terminal prompt. Using pfm it is possible to:

The command line interface includes basic help information. Examples:

>pfm -h
>pfm mount -h
>pfm mount myphotos.zip
>pfm unmount myphotos.zip

Installer

The PFM Audit Package and 3rd party applications that incorporate PFM install the core PFM files using the provided installer "pfminst". Example:

>pfminst install
>pfminst uninstall pfm-license-someapplication.txt

The PFM installer has been designed to reduce points of failure during install/upgrade/uninstall, to simplify integration with 3rd party applications, and to reduce support costs.

Pismo File Mount Audit Package

The Pismo File Mount Audit Package (PFMAP) is a utility application that utilizes PFM. It allow users to mount the contents of container files to the filesystem as read-write or read-only volumes. PFMAP is built using the same PFM interfaces available to 3rd party application developers.

In addition to being a useful stand-alone application, PFMAP can be used with the PFM Developer Kit by software developers.

Explorer shell extension

PFMAP includes a Windows Explorer shell extension. This shell extension allows convenient control of PFMAP directly from Explorer.

Mount Control

The Mount Control application, pfmcontrol, is the graphical user interface to PFMAP. All functionality in PFMAP can be accessed through this application, independent of explorer or the PFM command line interface.

Formatters

PFMAP supports a variety of container file formats. Each supported container format is implemented as a separate "formatter" DLL. When a container file is mounted, PFMAP locates the matching formatter DLL and uses it to create a PFM server for exposing the container file contents.

Third party developers can implement new formatters and register them for use with PFMAP.

CD/DVD image file reader

Pfmisofs.dll is a read-only DVD and CD image file formatter. It supports all or part of the following formats and extensions:

Zip reader

Pfmzipfs is a read-only ZIP archive file formatter. It supports the following formats and compression modes:

The implementation is missing the following potentially useful features:

RAM file system

Pfmramfs.dll implements a read-write temporary virtual file system.

The RAM file system stores all file data in system memory. This limits the maximum amount of stored file data based on available physical memory and swap file space. On 32 bit systems the amount of stored file data cannot exceed 2GB due to limited process address space.

Private Folder file system

Pfmpfolderfs implements a read, write, encrypted, compressed container file formatter. It allows storing sensitive files in a secure container file, accessible only with the correct password.

The user supplied password is converted to an encryption key using the PKCS5V2 algorithm. Data is encrypted using the AES encryption algorithm in XTS chaining mode.

Data compression uses the Zlib implementation of the deflate compression format. This is the same compression used in PNG images and ZIP archives.

The Private Folder data format documentation is not yet ready for public release. Pre-release information will be provided to interested parties upon request. For more information contact Pismo Technic Inc. support.

PFM Development Kit

The PFM Development Kit allows developers to build file systems using PFM. The kit also can be used to integrate existing PFMAP file systems into 3rd party applications.

Programming Languages

PFM supports development in a variety of programming languges, including:

A primary goal with PFM programming language support is to have a largely similar API across all programming languages. This is to allow for consistent documentation across all languages, as well as to allow sharing of support information and documentation between developers and projects using different programming languages.

A consequence of the PFM API portability is that it is not customized for each programming language to follow common language specific practices. As an example, PFM API methods return error codes for all programming languages, as opposed to throwing exceptions. Developers are encouraged to wrap the PFM API as they see fit to make it fit their style of programming.

This documentation describes the API using C++ syntax. Developers using other languages will need to refer to the class, assembly, or header files, for the syntactic differences.

C Header Files

The PFM API is defined in a C header file.

The C PFM API linkage is handled using inline code. Applications do not need to link to any PFM library at build time, and applications can load on systems where PFM is not installed.

C++ Header Files

The PFM API is defined in a C++ header file.

Note that if you include the C versions of the above header in C++ code, it will automatically include the C++ version.

The C++ PFM API linkage is handled using inline code. Applications do not need to link to any PFM library at build time, and applications can load on systems where PFM is not installed.

CLR (C#, VB) API Assembly

The CLR (C#, VB) PFM API is implemented in a portable assembly DLL and with a number of platform specific native DLLs. These files are all located in the "clr" folder in the PFM Developer Kit. Applications using the CLR PFM API will distribute the API assembly and DLLs with their application and will not install them to or load them from a shared location on the system.

The PFM API assembly is compatible with the Microsoft .NET runtime on Windows, and with the Mono runtime on Mac and Linux.

Though the PFM API assembly and helper libraries are versioned with a PFM build number, they can be used with any matching or newer PFM build.

Java API Jar

The Java PFM API is implemented in a portable jar file.

This jar contains all needed classes and JNI libraries for interfacing with PFM from Java applications. Applications using the Java PFM API will distribute the API jar with their application and will not install it to or load it from a shared location on the system.

The Java PFM API is compatible with a variety of JVM implementations and platforms. It includes the following JNI libraries:

Though the Java PFM API is versioned with a PFM build number, it can be used with any matching or newer PFM build.

Samples

Mounter application in C, C++, C#, and Java

The mounter sample demonstrates integrating PFMAP formatters into an application to allow mounting and unmounting container files such as zip and ISO.

Hello World file system application in C, C++, C#, and Java

Hellofs is a very simple virtual file system application. When run, this application presents a virtual file system containing a single read-only file, named readme.txt, containing the text "hello world".

Temp file system application in C++ and C#

Tempfs an application that presents a read-write temporary virtual file system volume. When executed it creates an empty volume in which new files and folders can be created. All contained data is discarded when the application exits.

Application Development

The PFM API allows direct integration of PFM with applications. Integration with PFM allows applications to perform a number of useful functions.

Concepts

Errors

The PFM API methods return system error codes such as ERROR_ACCESS_DENIED on Windows or EACCESS on Mac and Linux.

Mount ID

Each time a new mount is created it is assigned a unique identifier. This identifier is used in the various interfaces to identify the mount. Mount-ids are not reused until after the system is restarted.

Mount Source Name

Each mount is given an application chosen unique mount-source-name when it is created. The mount-source-name is formatted as a file-name or URL, where slashes are considered special characters.

Mount End Name

For each mount, PFM generates a unique mount-end-name. This name is the portion of the mount-source-name following the last slash character, with a number suffix appended when necessary to make it unique. The mount-end-name is used in the mount-point and UNC name.

Mount Point

Each mount is accessible through a mount point created in the \Volumes folder on Windows, the /Volumes folder on Mac, and the /media folder on Linux. The mount-end-name is used as the name of the mount-point.

Mount UNC Name

On Windows, each mount is accessible through a UNC name in the format "\\-\mount-end-name".

Change Instances

PFM maintains a change-instance count for the mount list and for each individual mount. Each time mount status changes the mounts change-instance is incremented. Each time a mount is created or destroyed the mount list change-instance is incremented. These change instances are used with PfmApi::MountIterate and PfmMountMonitor::Wait to allow applications to efficiently monitor for mount status and mount list changes.

PfmApi interface

The PfmApi interface allows applications to interact with and control PFM.

PfmApiFactory

int /*error*/ PfmApiFactory (
   PfmApi** api )

This function retrieves an instance of the PfmApi interface.

PfmApi::Release

void PfmApi::Release ( )

Free the interface instance and any associated resources.

PfmApi::MountCreate

int /*error*/ PfmApi::MountCreate (
   const PfmMountCreateParams* params,
   PfmMount**                  mount )

Create a new mount whose data will be served using the PFM Protocol through the mountCreateParams.toFormatterWrite and mountCreateParams.fromFormatterRead pipes or socket.

The params structure should be initialized as follows:

   PfmMountCreateParams params;
   PfmMountCreateParams_Init(&params); // C only.
   params.mountSourceName = ...;
   params.mountFlags |= ...;
   params.toFormatterWrite = ...;
   params.fromFormatterRead = ...;
   ...

Applications that only want to initiate mounts, but not create their own virtual file systems, do not use this function. Instead they would use PfmApi::FileMountFactory .

PfmApi::MountSourceNameOpen

int /*error*/ PfmApi::MountSourceNameOpen (
   const wchar_t* mountSourceName ,
   PfmMount**     mount )

Opens the mount with the specified mount source name. The mount source name is an arbitrary file-name formatted string supplied when the mount was created. Mounts created through PfmApi::FileMountFactory use the name of the mounted container file as the mount source name.

PfmApi::MountEndNameOpen

int /*error*/ PfmApi::MountEndNameOpen (
   const wchar_t* mountEndName ,
   PfmMount**     mount )

Opens the mount with the specified mount end name. The mount end name is the unique PFM generated name for the mount, used for constructing the mount-point and UNC names.

PfmApi::MountPointOpen

int /*error*/ PfmApi::MountPointOpen (
   const wchar_t* mountPoint ,
   PfmMount**     mount )

Opens the mount at the specified mount point.

PfmApi::MountIdOpen

int /*error*/ PfmApi::MountIdOpen (
   int        mountId ,
   PfmMount** mount )

Open the mount identified by the mountId parameter. The mountId parameter is typically retrieved from the PfmMountIterator::Next function.

PfmApi::MountIterate

int /*error*/ PfmApi::Iterate (
   int64_t       startChangeInstance ,
   int64_t*      nextChangeInstance ,
   PfmMountIterator** iterator )

This function creates and returns an instance of the PfmMountIterator interface, which allows applications to query a complete or partial list of mounts.

To iterate all mounts, zero should be passed for the startChangeInstance parameter.

The nextChangeInstance parameter is the location to store the current change instance of the mount list that will be returned by the iterator. This change instance can be used as the startChangeInstance parameter in future calls to iterate only mounts that have changed.

PfmApi::MountMonitorFactory

int /*error*/ PfmApi::MountMonitorFactory (
   PfmMountMonitor** monitor )

This function creates and returns an instance of the PfmMountMonitor interface, which allows applications to efficiently maintain an updated list of all mounts and mount states.

PfmApi::FileMountFactory

int /*error*/ PfmApi::FileMountFactory (
   PfmFileMount** fileMount )

Create a file mount interface to allow initiating a mount of a container file using a PFMAP formatter.

PfmFileMount interface

This interface allows applications to mount a container file system via a PFM Audit Package formatter. This interface is returned from the PfmApi::FileMountFactory function.

PfmFileMount::Release

void PfmFileMount::Release ( )

Free the interface instance and any associated resources. If the file mount has completed and was not detached, an unmount will be performed on the resulting mount.

PfmFileMount::Cancel

void PfmFileMount::Cancel ( )

Asynchronously cancel the ongoing file mount operation and close any related user interface dialogs.

PfmFileMount::Start

int /*error*/ PfmFileMount::Start (
   const PfmFileMountCreateParams* params )

Start a file mount operation. This will result in user interface dialogs being displayed as necessary.

The params structure should be initialized as follows:

   PfmFileMountCreateParams params;
   PfmFileMountCreateParams_Init(&params); // C only
   params.mountFileName = ...;
   params.mountFlags |= ...;
   params.fileMountFlags |= ...;
   ...

PfmFileMount::Send

void PfmFileMount::Send (
   const wchar_t* data,
   int/*bool*/    newLine);

This function can be used to send data back to the formatter during authentication. This may be needed for user interface that is mounting formatters with authentication that require more than simple password queries.

PfmFileMount::Status

void PfmFileMount::Status (
   const wchar_t* data,
   int/*bool*/    newLine )

This function is used to print mount status messages to the file mount user interface. It generally results in a matching call to PfmFileMountUi::Status . Any status prefixed with the string "ERROR: " may be used as the message in the final error dialog that is displayed if the mount fails.

PfmFileMount::WaitReady

int /*error*/ PfmFileMount::WaitReady ( )

This function will block until the mount operation completes successfully or fails due to an error or is cancelled.

PfmFileMount::GetMount

PfmMount* PfmFileMount::GetMount ( )

This function returns an unreferenced ptr to the PfmMount object associated with a successful file mount operation. The caller is responsible to add a reference to the object if it will be using it beyond the lifetime of the PfmFileMount object. If called before the file mount is ready, or on a file mount that has failed, it will return null.

PfmFileMount::Detach

void PfmFileMount::Detach ( )

This function detaches a successful out-of-process file mount from the current process. This allows the mount to survive after the file mount object is releases and after the current process exits.

If this function is called on an in-process file mount then it will block until an unmount occurs.

PfmFileMountUi

This interface is implemented by applications that are performing file mount operations, to allow the display of a custom user interface.

PfmFileMountUi::Start

void PfmFileMountUi::Start ( )

PfmFileMountUi::Complete

void PfmFileMountUi::Complete (
   const wchar_t* errorMessage )

PfmFileMountUi::Status

void PfmFileMountUi::Status (
   const wchar_t* data ,
   int/*bool*/    newLine )

PfmFileMountUi::QueryPassword

const wchar_t* PfmFileMountUi::QueryPassword (
   int count )

PfmFileMountUi::ClearPassword

void PfmFileMountUi::ClearPassword ( )

PfmMount interface

This interface allows applications to query information about, and control, an existing mount. This interface is returned from the PfmApi::MountCreate, PfmApi::MountSourceNameOpen, PfmApi::MountWEndNameOpen, PfmApi::MountIdOpen, and PfmFileMount::GetMount methods.

PfmMount::Release

void PfmMount::Release ( )

Free the interface instance and any associated resources.

Once this method is called, any previous data returned from this interface should no longer be used. In particular, any pointers to strings are no longer valid.

PfmMount::Refresh

int /*error*/ PfmMount::Refresh ( )

Update to match any changes to the state of the mount.

Once this method is called, any previous data returned from this interface should no longer be used. In particular, any pointers to strings will no longer be valid.

PfmMount::Unmount

int /*error*/ PfmMount::Unmount (
   int unmountFlags )

End an existing file mount. The mount will remain visible through the PfmApi::MountIterate method until all PfmMount instances referring to the mount are released.

PfmMount::GetMountId

int /*mountId*/ PfmMount::GetMountId ( )

PfmMount::GetMountFlags

int /*mountFlags*/ PfmMount::GetMountFlags ( )

PfmMount::GetStatusFlags

int /*statusFlags*/ PfmMount::GetStatusFlags ( )

PfmMount::GetVolumeFlags

int /*volumeFlags*/ PfmMount::GetVolumeFlags ( )

PfmMount::GetChangeInstance

int64_t /*changeInstance*/ PfmMount::GetChangeInstance ( )

PfmMount::GetMountSourceName

const wchar_t* PfmMount::GetMountSourceName ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh mathod or until the PfmMount instance is released.

PfmMount::GetMountEndName

const wchar_t* PfmMount::GetMountEndName ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh method or until the PfmMount instance is released.

PfmMount::GetMountPoint

const wchar_t* PfmMount::GetMountPoint ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh method or until the PfmMount instance is released.

PfmMount::GetUncName

const wchar_t* PfmMount::GetUncName ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh method or until the PfmMount instance is released.

PfmMount::GetDriveLetter

wchar_t PfmMount::GetDriveLetter ( )

PfmMount::GetOwnerId

const wchar_t* PfmMount::GetOwnerId ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh method or until the PfmMount instance is released.

PfmMount::GetOwnerName

const wchar_t* PfmMount::GetOwnerName ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh method or until the PfmMount instance is released.

PfmMount::GetFormatterName

const wchar_t* PfmMount::GetFormatterName ( )

The returned pointer to a string is only valid until the next call to the PfmMount::Refresh method or until the PfmMount instance is released.

PfmMount::Flush

int /*error*/ PfmMount::Flush ( )

PfmMount::Control

int /*error*/ PfmMount::Control (
   int         controlCode ,
   const void* input ,
   size_t      inputSize ,
   void*       output ,
   size_t      maxOutputSize ,
   size_t*     outputSize )

Send a server specific control code through to the server. Servers using PfmMarshaller will see a the control code via a call to their implementation of PfmFormatterDispatch::Control .

Since control codes are server specific, applications must identify the server before sending control codes. This can be done using the PfmMount::GetFormatterName method.

Servers should number their control codes starting at zero or one and not leave gaps. The range of available control codes is limited and is not guaranteed to remain constant.

PfmMount::WaitReady

int /*error*/ PfmMount::WaitReady (
   int timeoutMsecs )

Wait for the mount to become ready.

PfmMountIterator interface

This interface is returned from the PfmApi::MountIterate method. It allows applications to retrieve the mountId and current changeInstance of all file mounts.

PfmMountIterator::Release

void PfmMountIterator::Release ( )

Free the interface instance and any associated resources.

PfmMountIterator::Next

int /*mountId*/ PfmMountIterator::Next (
   int64_t* changeInstance )

PfmMountMonitor interface

This interface is returned from the PfmApi::MountMonitorFactory method. It allows applications to monitor for the creation and deletion of mounts.

PfmMountMonitor::Release

void PfmMonitor::Release(void)

Free the interface instance and any associated resources.

PfmMountMonitor::Wait

int /*error*/ PfmMountMonitor::Wait (
   int64_t nextChangeInstance ,
   int     timeoutMsecs )

Wait for the change instance of the mount list to be different from the nextChangeInstance parameter, or until another thread calls the PfmMountMonitor::Cancel function.

PfmMountMonitor::Cancel

void PfmMountMonitor::Cancel ( )

Return early from any calls to PfmMountMonitor::Wait .

File System Development

Getting Started

New file system server developers should first build and test one of the samples .

Testing

When debugging applications that are exposing PFM file system mounts, it is important that the debugger never access the PFM mount. If the debugger does access the PFM mount then it will deadlock. If a deadlock occurs, you can perform an unmount of the mount from another command prompt to get things moving again, but you will probably need to restart your debugging session. You also can kill the process to resolve a deadlock.

The PfmMarshaller interface provides some built in tracing functionality that is useful during development. To see these traces you will need to download and install Pismo Trace Monitor, available on the Pismo Technic Inc. website, http://www.pismotechnic.com/ .

You start the trace monitor on Windows using the "Start Menu - Programs - Pismo Trace Monitor" menu link, or by running tracemon.exe . New trace channels are hidden by default. You will need to use the unhide command from the trace monitor window menu to make channels visible. Do this after you have mounted a file.

On Mac and Linux you can view traces using the tracecmd command line tools.

Concepts

Portable Errors

The PFM Protocol and PfmFormatterDispatch interface return portable errors such as pfmErrorAccessDenied. The other interfaces return system error codes such as ERROR_ACCESS_DENIED or EACCESS. The protocol is remotable and portable, so the use of system error codes is not appropriate.

Folders are Files

The term file is regularly used in this document to mean file or folder. Unless stated otherwise, all PFM Protocol requests and related marshalled methods work the same for files and folders.

File Names

The PFM Protocol represents all file names in UTF8. The marshaller converts UTF8 file names to and from wchar_t file names for use by the servers.

The protocol and the driver support multiple named files, or hard links. Servers that support hard links make use of the client maintained parentFileId and endName in order to identify which name for a file is being manipulated.

The protocol allows the server to return a case corrected spelling for the last element of the name of opened files. This case corrected name is used by the driver when emulating short name aliases (DOS 8.3 file names) on Windows. Short names are still used by many applications, most notably by portions of Windows itself. Servers that do not provide the case corrected end name will have reduced compatibility.

OpenId and OpenSequence

OpenId and openSequence are integral parts of the PFM protocol. The openId is the mechanism used to identify open files and folders. OpenSequence is maintained by the driver and used by the server to identify when the last reference to an open file or folder has closed.

OpenIds are not file IDs. In particular the openId values for new files is assigned by the driver, where file IDs are assigned by the server.

The use of openIds and openSequence allows PFM to provide atomicity guarantees during file system name space changes and share mode checks. Without this mechanism the driver would have to make assumptions about the servers name space, and hold locks while waiting for the server to process many requests.

Deleted Files

The PFM Protocol handles file deletion using a unix model, where deletion applies to file names as opposed to underlying file data. After a file has been deleted, the file data must remain accessible until the file is finally closed.

The unix file deletion model requires extra code for some servers. For other servers it is trivially supported. Regardless, it is a requirement for servers and is utilized by the client.

Server developers should understand that the NT file system model is more complex than is apparent to casual users of the Win32 API. Using a unix model for file delete allows the client to achieve a high level of application compatibility in a more portable and supportable way than if the native NT delete model were directly supported by the protocol.

Concurrency

The Windows, Mac, and Linux kernels are massively multi-threaded. The PFM kernel module runs in the kernel, and therefore must also be multi-threaded. Likewise, the PFM Protocol used by the driver to communicate with servers supports requests being processed in parallel and in any order, so servers can also be implemented multi-threaded.

Servers are not required to be multi-threaded. Both the PFM kernel module and the PFM protocol have been carefully designed to support multi-threaded servers and servers that serially process requests.

PfmMarshaller???Op objects

All the methods in PfmFormatterDispatch receive an "op" object as a parameter. The various methods all have a unique interface for the related op object. These op objects represent the request from the client that is being processed. The various request parameters are available via op object methods.

All op objects have a Complete() method. The result for the associated request is returned to the client once the server calls the op->Complete() method. Once completed, the op object is freed and can no longer be used by the server.

Single-threaded servers will generally call op->Complete() from the related PfmFormatterDispatch method, before returning to the marshaller. Multi-threaded or concurrent formatters may save the op object and complete it later from another thread or work queue.

PfmFormatterSerializeOpen

interface PfmFormatterSerializeOpen
{
   void SerializeOpen (
      int64_t  openId ,
      int64_t* openSequence)
}

Some op->Complete() methods take an optional PfmFormatterSerializeOpen object as a parameter. This interface can be implemented by servers to allow generation of proper openSequence values when requests are being completed concurrently from multiple threads. The marshaller calls the SerializeOpen methods in the order that results are being sent back to the client. When called, the server should increment the openSequence value for the related file and return the new value to the marshaller.

If the server guarantees that it calls the related op->Complete() methods serially, either because the server is single-threaded or because the server uses its own serialization when completing requests, then this interface is redundant and does not need to be implemented. This is because the marshaller will send request results back to the client in the same order they are completed.

void* formatterUse

The C and C++ PfmFormatterDispatch methods also receive a formatter-use parameter. This is a pointer to a block of untyped memory associated with each request, that is available for use by the server. This can be useful to avoid extra heap memory allocation in concurrent servers. For example, it can be used to hold intrusive container links to facilitate storing op objects into queues.

PfmMarshaller interface

This interface is used to convert the PFM Protocol into the PfmFormatterDispatch interface that is implemented by servers.

PfmMarshallerFactory

int /*error*/ PfmMarshallerFactory (
   PfmMarshaller** marshaller )

This method creates an instance of the PfmMarshaller interface.

PfmMarshaller::Release

void PfmMarshaller::Release ( )

Servers should call this method when they are finished using an instance of the PfmMarshaller interface.

PfmMarshaller::SetTrace

void PfmMarshaller::SetTrace (
   const wchar_t* traceChannelName )

Formaters can optionally call this method to initialize a diagnostic trace channel to help with testing and field troubleshooting.

The Pismo Trace Monitor application must be installed to view the traces that are generated when this method is used. This applications is available on the Pismo Technic Inc. website, http://www.pismotechnic.com/ .

PfmMarshaller::SetStatus

void PfmMarshaller::SetStatus (
   HANDLE write )

Servers can call this method to set the handle (or file descriptor) of the pipe to which to send status text written using the status methods (Print, Vprintf, Printf, Line).

The marshaller does not duplicate or reference the supplied handle. Servers must make a second call to this method, specifying INVALID_HANDLE_VALUE or -1.

PfmMarshaller::ConvertSystemError

int /*pfmError*/ PfmMarshaller::ConvertSystemError (
   int error )

This method can optionally be used by servers to convert system errors (such as ERROR_OUTOFMEMORY or errnomem) to the equivalent PFM error codes needed with the PfmFormatterDispatch interface.

PfmMarshaller::GetPassword

int /*error*/ PfmMarshaller::GetPassword (
   HANDLE          read ,
   const wchar_t*  prompt ,
   const wchar_t** password )

Servers that require passwords can use this method to query the user to enter a password. The prompt value is the text to display with the prompt, typically "user name:" or "password:". The returned password is valid until the next call to either PfmMarshaller::GetPassword or PfmMarshaller::ClearPassword .

PfmMarshaller::ClearPassword

void PfmMarshaller::ClearPassword ( )

This method should be called after PfmMarshaller::GetPassword, when the returned password is no longer needed. This clears the password from system memory, reducing the chance that it will persist in the page file or appear in subsequent uninitialized memory allocations.

PfmMarshaller::ServeDispatch

void PfmMarshaller::ServeDispatch (
   PfmMarshallerServeParams* params )

This method is called by servers when they are ready to begin processing requests from the client. The method sends the PFM Protocol ready string to the client, which with file mounts will result in the mount status dialog closing. It then goes into a loop, reading protocol requests from the client, calling the associated methods in the PfmFormatterDispatch interface, and sending results back to the client.

The method will return when the driver disconnects from the server by closing its end of the pipe/socket.

The params structure should be initialized as follows:

   PfmMarshallerServeParams params;
   PfmMarshallerServeParams_Init(&params); // C only
   params.dispatch = ...;
   params.volumeFlags |= ...;
   params.formatterName = ...;
   params.toFormatterRead = ...;
   params.fromFormatterWrite = ...;

The volumeFlags and formatterName parameters are provided to the client, which uses the information to satisfy volume information queries made by applications.

PfmMarshaller::Print

void PfmMarshaller::Print (
   const wchar_t* data )

This method can be used by PFMAP formatters during identify and by servers during mount initialization, to send status data to the mount status dialog through the handle specified in an earlier call to PfmMarshaller::SetStatus .

This method can be used during processing of requests from the driver to send traces to the trace channel specified in an earlier call to PfmMarshaller::SetTrace .

PfmMarshaller::Vprintf

void PfmMarshaller::Vprintf (
   const wchar_t* format ,
   va_list        args )

Similar to PfmMarshaller::Print .

PfmMarshaller::Printf

void PfmMarshaller::Printf (
   const wchar_t* format ,
   ... )

Similar to PfmMarshaller::Print .

PfmMarshaller::Line

void PfmMarshaller::Line (
   const wchar_t* data ,
   int/*bool*/    newLine )

Similar to PfmMarshaller::Print .

PfmFormatterDispatch

This interface is implemented by servers, to process file system requests from the client that are marshalled through PfmMarshaller::ServeDispatch .

PfmFormatterDispatch::Open

void PfmFormatterDispatch::Open (
   PfmMarshallerOpenOp* op )

interface PfmMarshallerOpenOp
{
   PfmNamePart const* NameParts ( )
   size_t  NamePartCount ( )
   int8_t  CreateFileType ( )
   uint8_t CreateFileFlags ( )
   int64_t WriteTime ( )
   int64_t NewCreateOpenId ( )
   int8_t  ExistingAccessLevel ( )
   int64_t NewExistingOpenId ( )
   void    Complete (
      int                        perr ,
      bool                       existed ,
      PfmOpenAttribs const*      openAttribs ,
      int64_t                    parentFileId ,
      wchar_t const*             endName ,
      size_t                     linkNamePartCount ,
      void const*                linkData ,
      size_t                     linkDataSize ,
      PfmFormatterSerializeOpen* serializeOpen )
}

This method is called by the marshaller to process file open and create requests from the client. Correct use of newCreateOpenId parameter, the newExistingOpenId parameter, the existed result, the openAttribs->openId result, and the openAttribs->openSequence result, are critical to proper functioning of the atomicity guarantees provided by the client.

If the server returns no error (0) then it must return the information about the newly opened/created file through the openAttribs parameter, the existed parameter, and optionally through the parentFileId parameter the endName parameter. All of the openAttribs->attribs fields must be filled. The four time fields in openAttribs->attribs should be filled with valid times, or with constant pfmTimeInvalid.

The name of the file being opened is indicated by the nameParts and namePartCount parameters.

If the server determines that the indicated file exists but has not already been opened then it must associate the client specified newExistingOpenId with the file, and return this same value in openAttribs->openId. The openAttribs->openSequence value must be initialized to a non zero positive value, and the value must be associated with the open file for use during subsequent close processing. The variable referenced by the existed parameter must be set to true.

If the server determines that the indicated file is already open then it must return in openAttribs->openId the same openId that is already associated with the open file. The openAttribs->openSequence value must be initialized to a positive value that is greater than the openSequence returned by any previous open of the same file. The new openSequence value must be associated with the open file for use during subsequent close processing. The variable referenced by the existed parameter must be set to true.

If the parent folder of the indicated file does not exist then the server should return pfmErrorParentNotFound.

If the indicated file does not exist and the newCreateOpenId parameter is zero then the server should return pfmErrorNotFound.

If the indicated file does not exist and the newCreateOpenId parameter is non-zero then the server should create the file. The createFileType, createFileFlags, and writeTime parameters should be used to to initialize the new file. The server must associate the client specified newCreateOpenId with the file, and return this same value in openAttribs->openId. The openAttribs->openSequence value must be initialized to a non zero positive value, and the value must be associated with the open file for use during subsequent close processing. The variable referenced by the existed parameter must be set to false.

If the server is opening a file with different spelling for the last name component than was specified by the client then it must return the correct spelling through the endName parameter.

The openAttribs->accessLevel field should be filled with the highest access level currently available for the file. For most servers this field can always be initialized with pfmAccessWriteData. For redirectors, when opening existing files, the existingAccessLevel parameters can be used to avoid opening files with higher access levels than are needed for the current request.

PfmFormatterDispatch::Replace

void PfmFormatterDispatch::Replace (
   PfmMarshallerReplaceOp* op )

interface PfmMarshallerReplaceOp
{
   int64_t            TargetOpenId ( )
   int64_t            TargetParentFileId ( )
   PfmNamePart const* TargetEndName ( )
   uint8_t            CreateFileFlags ( )
   int64_t            WriteTime ( )
   int64_t            NewCreateOpenId ( )
   void               Complete (
      int                        perr ,
      PfmOpenAttribs const*      openAttribs ,
      PfmFormatterSerializeOpen* serializeOpen )
}

The marshaller calls this method when processing a replace request from the client. This request is made when an existing file is being replaced by a new file with the same name. The replaced file name becomes deleted.

The file being replaced has already been opened and is identified with the targetOpenId parameter.

If the target file type is a folder and the server does not support replace for folders then the server should return pfmErrorInvalid. Servers must support replace for files.

If the target file type is a folder and the folder is not empty then the server should return pfmErrorNotEmpty.

Servers that support multiple names for a single file (hard links) should use the targetParentFileId and targetEndName parameters to identify which name is being replaced.

The createFileFlags and writeTime parameters should be used to initialize the new file. The file type is always the same as the target.

The newCreateOpenId and openAttribs results should be treated the same as is described for the PfmFormatterDispatch::Open method when a new file is created.

PfmFormatterDispatch::Move

void PfmFormatterDispatch::Move (
   PfmMarshallerMoveOp* op )

interface PfmMarshallerMoveOp
{
   int64_t            SourceOpenId ( )
   int64_t            SourceParentFileId ( )
   PfmNamePart const* SourceEndName ( )
   PfmNamePart const* TargetNameParts ( )
   size_t             TargetNamePartCount ( )
   bool               DeleteSource ( )
   int64_t            WriteTime ( )
   int8_t             ExistingAccessLevel ( )
   int64_t            NewExistingOpenId ( )
   void               Complete (
      int                        perr ,
      bool                       existed ,
      PfmOpenAttribs const*      openAttribs ,
      int64_t                    parentFileId ,
      wchar_t const*             endName ,
      size_t                     linkNamePartCount ,
      void const*                linkData ,
      size_t                     linkDataSize ,
      PfmFormatterSerializeOpen* serializeOpen )
}

This method is called by the marshaller to process move requests from the client. This request is made when a file is being renamed. Proper handling of the newExistingOpenId parameter, exists result, openAttribs->openId result, and openAttribs->openSequence result, are critical to proper functioning of the atomicity guarantees provided by the client.

The sourceOpenId parameter specifies the previously opened file that is being renamed. The sourceParentFileId and sourceEndName parameters can be used by servers that support hard links to determine which name for the file is being renamed.

The targetNameParts and targetNamePartCount parameters specified the new file name (target) for the file.

If the parent folder of the target file name does not exist then the server should return pfmErrorParentNotFound.

If a file already exists with the target file name, then the existing target file should be opened. In this case the source file is left unmodified and no rename or move operation is performed. The variable pointed to by the existed parameter must be set to true. The newExistingOpenId, openAttribs, parentFileId, and endName parameters should be used in the same manner as is described for the PfmFormatterDispatch::Open method when an existing file is opened.

If the source file name has been deleted then the move request is the equivalent of an undelete. Servers must support this for files, but can return pfmErrorInvalid for folders.

If the deleteSource parameter is false, and the server does not determine that the source file name is deleted, then the move request is creating an additional name for the file (hard link). Servers that do not support hard links should fail the request with pfmErrorInvalid.

If the server is able to create the new name for the file then updated information about the source file is returned through the openAttribs and parentFileId parameters, openAttribs->openId must contain sourceOpenId, openAttribs->openSequence must contain a value equal to or higher than the largest openSequence value returned in any previous open for the file. The variable pointed to by the existed parameter must be set to false.

PfmFormatterDispatch::MoveReplace

void PfmFormatterDispatch::MoveReplace (
   PfmMarshallerMoveReplaceOp )

interface PfmMarshallerMoveReplaceOp
{
   int64_t            SourceOpenId ( )
   int64_t            SourceParentFileId ( )
   PfmNamePart const* SourceEndName ( )
   int64_t            TargetOpenId ( )
   int64_t            TargetParentFileId ( )
   PfmNamePart const* TargetEndName ( )
   bool               DeleteSource ( )
   int64_t            WriteTime ( )
   void               Complete (
      int perr )
}

The marshaller calls this method when processing a move-replace request from the client. This request is made when a an opened source file is being renamed to the same name as an opened target file. The replaced target file name becomes deleted.

Some processing for this method is similar to PfmFormatterDispatch::Replace, except that the source file is guaranteed to exist since it is already open.

Some processing for this method is similar to PfmFormatterDispatch::Move, specifically with handling of deleted source files and the deleteSource parameter.

PfmFormatterDispatch::Delete

void PfmFormatterDispatch::Delete (
   PfmMarshallerDeleteOp* op )

interface PfmMarshallerDeleteOp
{
   int64_t            OpenId ( )
   int64_t            ParentFileId ( )
   PfmNamePart const* EndName ( )
   int64_t            WriteTime ( )
   void               Complete (
      int perr )
}

The marshaller calls this method when processing a delete request from the client. This request is made when a file name is being deleted.

Servers that support multiple file names for a single file (hard links) can use the parentFileId and endName parameters to identify which file name is being deleted.

PfmFormatterDispatch::Close

void PfmFormatterDispatch::Close (
   PfmMarshallerCloseOp* op )

interface PfmMarshallerCloseOp
{
   int64_t OpenId ( )
   int64_t OpenSequence ( )
   void Complete (
      int perr )
}

There is not a one to one correspondence between open and close requests. The client generates a close request when it believes what may be the last reference to an open file has been released. The server decides if the last reference is actually gone by comparing the openSequence parameter against the openSequence associated with the file on the most recent successful open.

If the openSequence parameter is less than the openSequence of the file then the file is still open. The server should perform no action.

If the openSequence parameter is greater than or equal to the openSequence of the file then the client no longer has any references to the file. The server can free resources associated with the open file.

After all references to a file are gone, the server has the choice of forgetting the openId that was associated with the file, or saving the openId for use if the file is re-opened. Saving the openId may allow the client to re-associate cache data with the file.

Errors from this method are ignored by the client.

PfmFormatterDispatch::FlushFile

void PfmFormatterDispatch::FlushFile (
   PfmMarshallerFlushFileOp* op )

interface PfmMarshallerFlushFileOp
{
   int64_t     OpenId ( )
   uint8_t     FlushFlags ( )
   uint8_t     FileFlags ( )
   uint8_t     Color ( )
   int64_t     CreateTime ( )
   int64_t     AccessTime ( )
   int64_t     WriteTime ( )
   int64_t     ChangeTime ( )
   void const* LinkData ( )
   unsigned    LinkDataSize ( )
   void        Complete (
      int                        perr ,
      PfmOpenAttribs const*      openAttribs ,
      PfmFormatterSerializeOpen* serializeOpen )
}

Updated attributes for modified files are supplied to the server in the flush file request.

If the fileFlags parameter is the value pfmFileFlagsInvalid then the server should skip updating the file flags.

If any of the time parameters are the value pfmTimeInvald then the server should skip updating the associated time value.

Errors returned from this method may be logged, but are otherwise ignored by the client.

PfmFormatterDispatch::List

void PfmFormatterDispatch::List (
   PfmMarshallerDispatchOp* op )

interface PfmMarshallerListOp
{
   int64_t        OpenId ( )
   int64_t        ListId ( )
   bool /*added*/ Add (
      PfmAttribs const* attribs ,
      wchar_t const*    endName )
   bool /*added*/ Add8 (
      PfmAttribs const* attribs ,
      char const*       endName )
   void           Complete (
      int  perr ,
      bool noMore )
}

This method is called by the marshaller to generate list results for the contents of folders.

The behavior of this method for non-folder files is undefined. Servers are free to handle this in whatever way is convenient.

The server must create and maintain state information for each unique listId parameter that is used to list contents of a folder. The server should create this state when it sees a new listId. The server frees this state when it receives a call to PfmFormatterDispatch::ListEnd with a matching openId/listId, or when it frees resources associated with an open folder while processing a call to PfmFormatterDispatch::Close for the same openId.

Folder contents are added to the results by repeatedly calling the op->Add() or op->Add8() methods.

PfmFormatterDispatch::ListEnd

void PfmFormatterDispatch::ListEnd (
   PfmMarshallerListEndOp* op )

interface PfmMarshallerListEndOp
{
   int64_t OpenId ( )
   int64_t ListId ( )
   void    Complete (
      int perr )
}

See PfmFormatterDispatch::List.

PfmFormatterDispatch::Read

void PfmFormatterDispatch::Read (
   PfmMarshallerReadOp* op )

interface PfmMarshallerReadOp
{
   int64_t  OpenId ( )
   uint64_t FileOffset ( )
   void*    Data ( )
   size_t   RequestedSize ( )
   void     Complete (
      int    perr ,
      size_t actualSize )
}

The behavior of this method for folders is undefined. Servers are free to handle this in whatever way is convenient.

PfmFormatterDispatch::Write

void PfmFormatterDispatch::Write (
   PfmMarshallerWriteOp* op )

interface PfmMarshallerWriteOp
{
   int64_t     OpenId ( )
   uint64_t    FileOffset ( )
   void const* Data ( )
   size_t      RequestedSize ( )
   void        Complete (
      int perr ,
      size_t actualSize )
}

The behavior of this method for folders is undefined. Servers are free to handle this in whatever way is convenient.

The behavior of zero length writes is undefined. Servers are free to handle this in whatever way is convenient.

PfmFormatterDispatch::SetSize

void PfmFormatterDispatch::SetSize (
   PfmMarshallerSetSizeOp* op )

interface PfmMarshallerSetSizeOp
{
   int64_t  OpenId ( )
   uint64_t FileSize ( )
   void     Complete (
      int perr )
}

PfmFormatterDispatch::Capacity

void PfmFormatterDispatch::Capacity (
   PfmMarshallerCapacityOp* op )

struct/*interface*/ PfmMarshallerCapacityOp
{
   int64_t OpenId ( )
   void    Complete (
      int      perr ,
      uint64_t totalCapacity ,
      uint64_t availableCapacity )
}

PfmFormatterDispatch::FlushMedia

void PfmFormatterDispatch::FlushMedia (
   PfmMarshallerFlushMediaOp* op )

struct/*interface*/ PfmMarshallerFlushMediaOp
{
   void Complete (
      int perr ,
      int msecFlushDelay )
}

The client will generate a flush media request shortly after a preiod of inactivity (~2 seconds). The server can use this request to perform whatever updates are applicable based on the file system characteristics.

If the last request was a media flush then the client will wait the previously returned delay before sending another media flush.

Errors returned from this method may be logged, but are otherwise ignored by the client.

PfmFormatterDispatch::Control

void PfmFormatterDispatch::Control (
   PfmMarshallerControlOp* op )

interface PfmMarshallerControlOp
{
   int64_t     OpenId ( )
   int8_t      AccessLevel ( )
   int         ControlCode ( )
   void const* Input ( )
   size_t      InputSize ( )
   void*       Output ( )
   size_t      MaxOutputSize ( )
   void        Complete (
      int    perr ,
      size_t outputSize )
}

PfmFormatterDispatch::MediaInfo

void PfmFormatterDispatch::MediaInfo (
   PfmMarshallerMediaInfoOp* op )

interface PfmMarshallerMediaInfoOp
{
   int64_t               OpenId ( )
   void Complete (
      int                 perr ,
      PfmMediaInfo const* mediaInfo ,
      wchar_t const*      mediaLabel )
}

The client sends a media info request to retrieve media identification information such as media label and media id. Servers are free to fail this request, or to return only partial information. The returned information can be accessed by client applications using the Win32 GetVolumeInformation() system call.

The supplied open ID may be zero, or may be the id of an open file or folder. The server is free to ignore the open id or to return different information based on what file or folder the open id refers to.

PfmFormatterDispatch::Access

void PfmFormatterDispatch::Access (
   PfmMarshallerAccessOp* op )

interface PfmMarshallerAccessOp
{
   int64_t OpenId ( )
   int8_t  AccessLevel ( )
   void    Complete (
      int                        perr ,
      PfmOpenAttribs const*      openAttribs ,
      PfmFormatterSerializeOpen* serializeOpen )
}

This method is called by the marshaller when a higher access level is needed for an open file.

The openAttribs result should be treated the same as is described for the PfmFormatterDispatch::Open method when an existing file is opened.

PfmFormatterDispatch::ReadXattr

void PfmFormatterDispatch::ReadXattr (
   PfmMarshallerReadXattrOp* op )

interface PfmMarshallerReadXattrOp
{
   int64_t            OpenId ( )
   PfmNamePart const* Name ( )
   unsigned           Offset ( )
   void*              Data ( )
   size_t             RequestedSize ( )
   void               Complete (
      int perr ,
      unsigned xattrSize ,
      size_t transferredSize )
}

PfmFormatterDispatch::WriteXattr

void PfmFormatterDispatch::WriteXattr (
   PfmMarshallerWriteXattrOp* op )

interface PfmMarshallerWriteXattrOp
{
   int64_t            OpenId ( )
   PfmNamePart const* Name ( )
   unsigned           XattrSize ( )
   unsigned           Offset ( )
   void const*        Data ( )
   size_t             RequestedSize ( )
   void               Complete (
      int perr ,
      size_t transferredSize )
}

Data Types

mountFlags

unmountFlags

statusFlags

pfmError

fileMountFlags

fileType

fileFlags

color

time

All file times used in the PFM Protocol and with the PfmFormatterDispatch interface are in the Windows FILETIME format, 64 bit count of 100 nano second units since Jan 1 1601 UTC.

accessLevel

volumeFlags

flushFlags

PfmAttribs

struct PfmAttribs
{
   int8_t   fileType
   uint8_t  fileFlags
   int64_t  fileId
   uint64_t fileSize
   int64_t  createTime
   int64_t  accessTime
   int64_t  writeTime
   int64_t  changeTime
}

PfmOpenAttribs

struct PfmOpenAttribs
{
   int64_t    openId
   int64_t    openSequence
   int8_t     accessLevel
   PfmAttribs attribs
}

PfmNamePart

struct PfmNamePart
{
   const wchar_t* name
   size_t         len
   const char*    name8
   size_t         len8
}

This structure only used in the C and C++ API. For other programming languages, the native string type is used.

The name8 field points to a UTF8 string, not to an ANSI or OEM codepage string.

The name and name8 fields point to the same string in two different encodings. Formatters should use the name that is most convenient. Both names are zero terminated with the length available in the respective len field.

PfmMediaInfo

struct PfmMediaInfo
{
   GUID     mediaUuid
   uint64_t mediaId64
   uint32_t mediaId32
   uint8_t  mediaFlags
   int64_t  createTime
}

PfmFileMountCreateParams

struct PfmFileMountCreateParams
{
   const wchar_t* mountFileName
   int            mountFlags
   int            fileMountFlags
   wchar_t        driveLetter
   const wchar_t* password
   PfmFileMountUi* ui
}

PfmMountCreateParams

struct PfmMountCreateParams
{
   const wchar_t* mountFileName
   int            mountFlags
   wchar_t        driveLetter
   HANDLE         toFormatterWrite
   HANDLE         fromFormatterRead
}

PfmMarshallerServeParams

struct PfmMarshallerServeParams
{
   PfmFormatterDispatch* dispatch
   int                   volumeFlags
   const char*           formatterName
   HANDLE                toFormatterRead
   HANDLE                fromFormatterWrite
}