Configuring metaSAN for an image file sequence based workflow
Applications such as Assimilate Scratch and Iridas FrameCycler do not work with QuickTime movies but instead use image file sequences (ex: DPX, TIFF, etc.). In a SAN environment, this leads to specific challenges. A QuickTime movie often contains thousands of frames that are stored in a single file. This means that large chunks of the file are stored in contiguous segments on the drive. When a SAN member needs to playback a QuickTime movie, it sends a single metadata request to the master to understand where the file is located on the disk.
An image file sequence, on the other hand, requires one file per frame. A few minutes of video will easily generate thousands of files. In a SAN, every frame requires metadata requests to be processed before a frame can be located on disk and accessed. Instead of one metadata request for an entire QuikTime movie, a DPX sequence requires 24 metadata requests per seconds of video. A 5 min clip generates 7,200 requests. Given this huge amount of metadata, it is very difficult to ensure reliable playback as any delay in processing the request will lead to drop frames*. Tiger recommends creating a dedicated volume for the workstation needing realtime DPX playback. This workstation should be configured to become the master (MDC) of this volume (see diagram). As MDC of the volume, it doesn’t depend on any other workstation to process metadata requests. It can access the files directly. Playback performance is identical to that of a direct-attach drive. Each workstation in the SAN that is destined to perform image file sequence capture or playback, should be configured to be master of its own volume. From a workflow standpoint, all SAN members can still access all volumes containing image file sequences. But during DPX playback, the color grading workstation (that is also acting as the master) will only serve requests to other SAN members when it is able to do so. It is therefore normal that other users experience some sluggishness when accessing a volume currently used for playback. It is therefore recommended to maintain one generic SAN volume (to store QuickTime and other files) that is always available to all clients. *) Note that even on local disks without metaSAN there is still a large number of metadata requests made to the file system. When you access local storage, the system cache is helping, but still you can experience problems playing back sequences because volumes easily get fragmented. That is why many people working with image sequences often re-format their drives prior to beginning a new project. In a SAN environment the fragmentation problem is aggravated because multiple threads are writing to the volume at the same time, requesting free clusters one after the other.




