Repository File System Selection and Optimization
Posted by Mont Rothstein on 07 January 2016 08:33 PM
Foray Adams uses the file system as its primary data store. This offers both performance and durability benefits. Performance characteristics and limitations are therefore directly related to the underlying file system.
Following are file system specific guidelines.
This has been the file system for Windows operating systems starting with NT (server) and XP (desktop).
However as the number of files and folders increase there are some thresholds to pay attention to.
Adams stores a directory for each Case/Folder in a single parent directory. Microsoft advises that when the number of files and subdirectories in a single parent directory reaches 300K that, depending on how the files and folders are named, there may be a significant impact on performance. This is due to the generation of short names (8.3) for each file and folder. The recommendation is therefore to disable generation of short file names.
When the total number of Cases/Folders approaches 10 million or more the index file for the parent directory, internal to the working of NTFS, may exceed the maximum number of fragments. The recommendation is therefore to periodically defrag this file.
A Global Files System (GFS), also know as a Distributed Files System, aggregates multiple local file systems. These are commonly used by storage vendors with each implementation being proprietary to the vendor.
These systems can provide access to very large storage solutions across multiple devices. They help to facilitate the management, access, and partitioning of large storage solutions.
When evaluating a GFS for use with the Adams solution the following areas should be discussed with the storage vendor. Be sure to fully understand how the specific storage solution addresses each of these items before making a decision as to whether a specific storage solution will meet your needs.
What hard and/or practicle (ex: causes performance degredation) limitations does the system have? This should include at least: Max volume size, Max file size, Max number of files and folders per volume, Max number of files and folders per directory. You should determine from the vendor not just the theoretical max but the numbers at which performance degrades and any mitigation options other than simply keeping below the recommended max.
The Foray Adams solution has a mixture of large files (photos, videos, etc.) as well as very small files (< 1K). Does fetching and/or writing small files present any issues for the selected storage solution? If so, are there configuration changes that can be made to address this issue and if there are what are the tradeoffs of those changes? The specific concern here is that for some solutions small files will get stuck in a cache until another fetch fills the cache.
Access Control Lists (ACLs)
Does the storage solution fully support Microsoft's implementation of ACLs as implemented in NTFS? If not, what specific differences are there? ACLs are key to the security and integrity of evidence stored in the Foray Adams solution and some storage solutions use limited ACL implementations.
After having reviewed the available options, and gathered the necessary information, if you are unsure which file system is right for you please contact firstname.lastname@example.org