Cookies improve the way our website works, by using this website you are agreeing to our use of cookies. For more information see our privacy policy. OK

NY_FileStorePlugIn

The plug-in is required to provide files to other services. It is also necessary to download files from other nodes.

Directory

The FileStore directory is located by default in the storage directory. It includes three other directories:

  • The indexed directory contains all files and folders that have been imported into the FileStore. The files in this directory should not be manually altered, as it is not monitored. If files need to be deleted, the node must be restarted afterward so that the FileStore can report the changes to the FileRegistry.
  • The incomplete directory is used for downloading files. It stores files that have not yet been completely downloaded.
  • The incoming directory is monitored by the node. Files and folders copied into this directory are immediately indexed. Subsequently, they are removed from this directory.

Registry

All files are reported to a central file registry with their relative path. The relative path refers to the location of the file relative to the indexed directory.

Example

If the absolute path of a file on Windows is C:\NodeName\storage\FileStore\indexed\a\b\c\file.txt, then the relative path is a\b\c\file.txt.

The relative path serves as a search term for the file. The file can also be searched using its hash value. Alongside the path and hash value, the timestamp of the last modification is reported to the registry. This enables the registry to return the most recent file when there are multiple matches and the search is based solely on the path.

File Hashes

The hash value is automatically generated by the system. We use MD5 hashes as they are fast and meet our requirements. The generated hash is converted into a string using a CrockfordBase32 algorithm, suitable for storage in the filesystem. The files are stored in the FileStore\indexed directory and retain their relative paths.

The file name extension for files in the index is ._32_[Base32-MD5].
Example: annotations-24.1.0.jar._32_71KKNFM7T4JGHK85967FY2WQ64

File Transfer

Files are transported in chunks across the network. This method is used to prevent slowing down other traffic between nodes. No separate TCP channel is opened for the transportation. The messages are assigned a low priority. Files retain their timestamps, both for creation and last modification, during import and transport.

Execution of the Plugin

The plugin should be run on each node that provides files or requires files from other FileStore services.