How new software is installed on a node remotely

The software of a node can be updated semi-automatically.

A node consists of a kernel and the plugins that this kernel loads. Additionally, there is third-party software in the form of libraries. All components are located in two directories. The lib directory contains the kernel, the API JARs of the plugins, and the third-party software. The classpath contains only one entry, namely the lib directory. The plugins directory contains the plugin JAR files.

During a software update, either the files in the lib directory, the files in the plugins directory, or the files in both directories are updated. Subsequently, a reboot of the node is initiated. In the shell script (or batch file) that starts the node, the files are replaced before the start. Finally, the node boots the new software.

There is another variant of the software update. In this case, only the list of plugins to be loaded is changed, followed by a normal reboot of the node. No files are downloaded. This variant is preferably chosen when only the characteristics of the node need to be changed. After all, the plugins to be loaded determine which services the node offers and uses.

1. Preparation of the update

We need a local node that is connected to the network. This node must have the NY_NetSoftwareUpdaterPlugIn plugin loaded.

The remote nodes that are to receive a software update must have the NY_LocalSoftwareUpdaterPlugIn plugin loaded.

2. Preparation of the JARS

Local directories (lib and plugins) should be created to contain all the necessary files. A local node should be started with this software, and the log should be checked to identify any missing libraries. These files are copied into the incoming directory of the local node's FileStore. As a result, the files are incorporated into the FileStore and are available for download.

3. Creation of a JSON control file

Next, a JSON file is created that contains the filenames and possibly the hashes of the files to be copied.

Example file

{
  "jar_list": {
    "nodes": [
      "moon"
    ],
    "reboot": true,
    "sessionToken": "78febe5e-eae1-4c72-816a-a74114c3376a",
    "libraries": [
      "annotations-24.1.0.jar(71KKNFM7T4JGHK85967FY2WQ64)",
      "h2-2.2.224.jar(ETENN1EHKK62P1K21Y683NPRZ0)",
      "hamcrest-core-1.1.jar(PSPGRJ71Y7E59N12FPTJA4P1BC)",
      "json-simple-1.1.1.jar(BK1C8Y6Q7T259D63D77ECV2VRW)",
      "junit-4.10.jar(D0W000DRG03EQS4VWM6EYPXJ78)",
      "miglayout-core-5.2.jar(AWJP3ENGQQFHM1WXD4WKG71RY4)",
      "miglayout-swing-5.2.jar(M34QAEFTSPMV3Z13FR0J7QJFEG)",
      "NY_Kernel-2.0.1.jar(ARC2G32GVEK0HJQET4VDN9YR84)",
      "NY_KernelStarter-2.0.1.jar(WF0NKRG3MAWZKYK777TS216A2R)",
      "NY_RecordDatabase-2.0.1.jar(VCYWM45ZRMF53BR6STT2C7ZBSC)",
      "NY_RecordParserJson-2.0.1.jar(CP2VSZ0FBKTJF104TR25XG2CSW)",
      "NY_RecordParserXml-2.0.1.jar(K8TJAG76VFK0SN7QB2NG6Q2EDW)"
    ],
    "plugins": [
      "NY_ApplicationAPI-2.0.1.jar(8WHT39RZBDMJ8TZCRAAV5DXA1W)",
      "NY_ApplicationFactoryCollectorAPI-2.0.1.jar(DC9KPWMN9D593NVSNBYYTM7NY8)",
      "NY_ApplicationFactoryCollectorPlugIn-2.0.1.jar(QXV5SH3CSTRBHV6A0456BDAK5R)",
      "NY_ApplicationNetworkPlugIn-2.0.1.jar(04DRQB28JD9CW6B3KJ7EDAGHVC)",
      "NY_ApplicationPlugIn-2.0.1.jar(9SQ1YQ0GBA025DRV3ASYDZDNXR)",
      "NY_AppMonitor-2.0.1.jar(5MHV918RRF8C12XAR79B0R1JY4)",
      "NY_AppNodeManager-2.0.1.jar(8Z9VFMPDXTPC6RQ52K9DPR26JW)",
      "NY_AppUserManager-2.0.1.jar(AHF60TTFBRD3KD0099HG2FVR14)",
      "NY_AppWebLogIn-2.0.1.jar(DYQZPJDCYT6D4WDC5CYKJ56EJ8)",
      "NY_AppWidgetShowCase-2.0.1.jar(SAWVPCJF02MGEKJ3V4X7EG47YG)",
      "NY_ClientNodePlugIn-2.0.1.jar(F5W7YSN65GM6ED7X44NC0Y0ZR4)",
      "NY_ConfigurationProviderPlugIn-2.0.1.jar(F68A7H7PXK2CJW8MFJ4ETVPASW)",
      "NY_DevCheckPlugIn-2.0.1.jar(ZBRABSH8YW9AQCPXXP79WQTYQW)",
      "NY_DialogMessagePanelAPI-2.0.1.jar(1GRM5K52FFHJ3QN3712W9Y7KK8)",
      "NY_DialogMessagePanelPlugIn-2.0.1.jar(0K4XQNVHNP0AJHQQVPSZFSRHH0)",
      "NY_DialogNodeListAPI-2.0.1.jar(7A9VWTHZF65M58D4V50M9X97F8)",
      "NY_DialogNodeListPlugIn-2.0.1.jar(QCQ7W4GRX8XWES4AF3SZ073Z70)",
      "NY_DialogPasswordChangeAPI-2.0.1.jar(ZPQ0MZZ5EM3Q5NHGP49QV770E4)",
      "NY_DialogPasswordChangePlugIn-2.0.1.jar(3M181ACPFCEXRHZ5X6RBV5D4PG)",
      "NY_DialogPresetAPI-2.0.1.jar(WZQRET3MSW489006SKXDE2PA50)",
      "NY_DialogPresetPlugIn-2.0.1.jar(ET3Z1KYWKG1YN7A559KZ8ZM4GC)",
      "NY_DialogPropertyEditorAPI-2.0.1.jar(E3HEASDH1QQKMJD3NRREJSDHWM)",
      "NY_DialogPropertyEditorPlugIn-2.0.1.jar(EBY7JSFZGH6634723NZVKC0TH4)",
      "NY_DialogUserAPI-2.0.1.jar(G2X20HENG162X4YZDE8EG47NN4)",
      "NY_DialogUserPlugIn-2.0.1.jar(5BG0CEN93E7YZKQYXJMNKV7QFG)",
      "NY_Examples-1.1.0.jar(6768RVH4ASC0TDSH2NX2VQYAQW)",
      "NY_FileCachePlugIn-2.0.1.jar(T02HCKZ2CGKCJN7SFSPGCZY9N8)",
      "NY_FileRegistryAPI-2.0.1.jar(TQMNVX2YZXV0MF4HJ4K6KQJ60R)",
      "NY_FileRegistryPlugIn-2.0.1.jar(2EE147FRZ0ABBDV17VQHPH30MW)",
      "NY_FileStoreAPI-2.0.1.jar(GTTD79F7RF1XJSC4T2ENSM5V8G)",
      "NY_FileStorePlugIn-2.0.1.jar(41B0EPJS4E3T18VARYEJ2BF190)",
      "NY_GateKeeperAPI-2.0.1.jar(71XKWEWJVH7EQD0KHZK5SMTR1M)",
      "NY_GateKeeperPlugIn-2.0.1.jar(AEQWH19QJF57MCR84N0Z52BC10)",
      "NY_H2DatabasePlugIn-2.0.1.jar(YZY88WEW40Y926D1Y9BF1BEJZ0)",
      "NY_IntegrationTestPlugIn-2.0.1.jar(WS10PP1WSMX4262T0ZKDWG8KH4)",
      "NY_LinkCollectorAPI-2.0.1.jar(QMAGPNG5JGBKR1TCZNJCE9TTV8)",
      "NY_LinkCollectorPlugIn-2.0.1.jar(T2RVJPZYR8RPHGCEEE5T8MBR7G)",
      "NY_LinkCostCollectorAPI-2.0.1.jar(0F5KRFQM1Q9BRC22SGXZK1Y2TG)",
      "NY_LinkCostCollectorPlugIn-2.0.1.jar(MB3T7A3CBJSJZCATC43B5X5MYG)",
      "NY_LinkPrinterPlugIn-2.0.1.jar(HVM1VCEDHM3PN9FC4Z8YGN4N8M)",
      "NY_LocalSoftwareUpdaterPlugIn-2.0.1.jar(84C4DGTCXBPBV5B1FZ3KATNPQC)",
      "NY_LogMonitorPlugIn-2.0.1.jar(28NZ5E9G5TA1AXS0JWP10MXGDW)",
      "NY_MicroServiceAPI-2.0.1.jar(YW8RHX1M8ANZT3RCW2R654YVJ8)",
      "NY_MicroServicePlugIn-2.0.1.jar(3MK93C4RGKSMRH44PVVWNZ8W94)",
      "NY_MicroServiceRegistryAPI-2.0.1.jar(A5V4J2S0X5MEGC7M8A9HT1MYJ8)",
      "NY_MicroServiceRegistryPlugIn-2.0.1.jar(GXTJVBRWJ8Q9K0PXSF71WCG7JM)",
      "NY_MonitorPlugIn-2.0.1.jar(1BW8DECSYV71SJQBDW4RYB5YB0)",
      "NY_MonitorTestPlugIn-2.0.1.jar(Z06EVX63NPWDSZ0MKA8KC2TSHM)",
      "NY_NetSoftwareUpdaterPlugIn-2.0.1.jar(60SHJYAC4ACMCPWR7J2DMCXBV8)",
      "NY_NetworkAPI-2.0.1.jar(JMVXGP803CKX0EA792DA0SE9MW)",
      "NY_NetworkIndexPlugIn-2.0.1.jar(C08XZYJJ4X90MK3M81EEXPF060)",
      "NY_NetworkPlugIn-2.0.1.jar(XHBKRDN3VFD8M5KW1AAPMKY8XR)",
      "NY_NodeCheckerPlugIn-2.0.1.jar(KQJAB6R1A8VJYTE29YVCAQTZQ8)",
      "NY_NodeStarterPlugIn-2.0.1.jar(0DBRYKV3Z92BJ321E7S1FS8MAR)",
      "NY_NodeUiPlugIn-2.0.1.jar(KQH79E9SCH139KBF2799TE1Q74)",
      "NY_RemoteSkinClientPlugIn-2.0.1.jar(KJ340BT1NADBDYFA9ST52EYTQW)",
      "NY_RemoteSkinDialogMonitorAPI-2.0.1.jar(5ES375NXSZN01BQ6JKQC5TAXP0)",
      "NY_RemoteSkinDialogMonitorPlugIn-2.0.1.jar(EEN62YTMHDN34PF7WKS7C8HCG0)",
      "NY_RemoteSkinModelPlugIn-2.0.1.jar(HETVMVS669KTSBYQ1RE80R12EM)",
      "NY_RemoteSkinMonitorConfigurationPlugIn-2.0.1.jar(583AJ3HQ7PYQJ83T0JH24QEY88)",
      "NY_RemoteSkinServerPlugIn-2.0.1.jar(CZ2AZQXVZX61ZB8AB48FNTMN6C)",
      "NY_RemoteSkinSwingPlugIn-2.0.1.jar(51QFQ28M2D7DZ9M42CDZ5HJA54)",
      "NY_RemoteSkinTest-2.0.1.jar(4Z0BS0R2HT5XA2PJEFGXE79FZ4)",
      "NY_RemoteSkinWebProtocolPlugIn-2.0.1.jar(FDV7J60BME6ESRAJS2A8HDSEB0)",
      "NY_RemoteSkinWebServerAPI-2.0.1.jar(3Q6TBPQJF031YXADJF3WBR6KJR)",
      "NY_RemoteSkinWebServerPlugIn-2.0.1.jar(3JT62S1RA171N8A60PV1DEN2GG)",
      "NY_SequenceMonitorPlugIn-2.0.1.jar(Y55RVNH84AQS9AMP84H564MY64)",
      "NY_Session2API-2.0.1.jar(TG6XGJZ6ACCFM2VZCNEY74WNE8)",
      "NY_Session2PlugIn-2.0.1.jar(3SV1HZYSPJWJH5YAHY8WX35068)",
      "NY_SoftwareUpdaterAPI-2.0.1.jar(AP6XMV7V2ERV9MZAHNBWF8K5J8)",
      "NY_TcpAPI-2.0.1.jar(12E2Q5WHVN0D63QJRN38R5A458)",
      "NY_TcpPlugIn-2.0.1.jar(9ESJPXD06G8Y96ZVR5QBS44CER)",
      "NY_TestSenderPlugIn-2.0.1.jar(RTF914XQF0ZQB5ZR7BZH68REHG)",
      "NY_TransportStatisticAPI-2.0.1.jar(7PD1A4Q6QS87NNED4M59APF670)",
      "NY_TransportStatisticPlugIn-2.0.1.jar(T6DEHCSCT6ECV55P9XCJ6TYY90)",
      "NY_WebSocketAPI-2.0.1.jar(MD70N6X6HS9KMNP1B3CYB2QAFG)",
      "NY_WebSocketPlugIn-2.0.1.jar(5F6Q3AE2KF1M4YJVHTD2XMD1S8)"
    ],
    "pluginsToLoad": [
      "NY_NetworkPlugIn",
      "NY_TcpPlugIn",
      "NY_MicroServicePlugIn",
      "NY_FileStorePlugIn",
      "NY_RemoteSkinModelPlugIn",
      "NY_RemoteSkinServerPlugIn",
      "NY_RemoteSkinWebProtocolPlugIn",
      "NY_WebSocketPlugIn",
      "NY_AppWebLogIn",
      "NY_LocalSoftwareUpdaterPlugIn"
    ]
  }
}
  • nodes : The list of nodes to update.
  • reboot : The default here is true for rebooting the nodes after the update. The user must have the right to shut down the node (NY_Shutdown).
  • sessionToken : You get the session token when you log in to the console. You must manually enter it in the JSON file.
  • libraries : Optional: The list of libraries and the kernel JAR to download. The hash is optional and is only required if you want to distinguish between files with the same name but different versions. The hash value of the file is displayed on the console when the file is imported into the FileStore. Only the files that do not already exist will be downloaded. Files that existed before the update and no longer exist will be imported into the remote FileStore and deleted from the directory. If this list is missing, the kernel file and libraries will not be modified. Updating the library files requires the NY_SoftwareUpdate permission.
  • plugins : Optional: The list of plugins in the plugins directory. The hash is optional and is only required if you want to distinguish between files with the same name but different versions. The hash value of the file is displayed on the console when the file is imported into the FileStore. Only the files that do not already exist will be downloaded. Files that existed before the update and no longer exist will be imported into the remote FileStore and deleted from the directory. If the list is missing, the files in the plugins directory will not be modified. Updating the plugins requires the NY_SoftwareUpdate permission.
  • pluginsToLoad : Optional: The list of plugins to load when the node is started. If this list is missing, the node's existing list is not modified. Preferably use filenames without version numbers here. The kernel automatically selects the latest version when there are multiple versions of a plugin. This way, you avoid updating the list when installing new software.

4: Obtaining the necessary permissions and the session token

The user must now log in to the local node's console. Upon successful login, a session token is generated and displayed on the console. This token must be entered into the JSON file. It is valid for half an hour.

5: Start of the update process

This file must now be copied to the storage/incoming_jar_lists directory of the local node. This directory is monitored by the NY_NetSoftwareUpdaterPlugIn plugin. The file is read and transmitted to the affected remote nodes.

6. What happens

The current software is checked there. All JARs from the lib directory are copied to the update/old_files/lib directory. All JARs from the plugins directory are copied to the update/old_files/plugins directory. JARs that are no longer needed are imported into the FileStore. They are available there in case a rollback is necessary. JARs that are still needed are copied to the update/new_files/lib and update/new_files/plugins directories. Missing JARs are downloaded from the FileStore and copied to the same directories.

The list of plugins to be loaded is adopted, if provided.

If everything went well and a reboot is to be performed, it is done with an exit code. The following exit codes are important:

  • 1: No file is copied by the shell script. The node is restarted. This variant is chosen when only the list of plugins to be loaded has been changed.
  • 2: The lib directory is emptied, and the files in the update/new_files/lib directory are adopted. The node is restarted. This variant is chosen when the library list is provided but the plugin list is missing.
  • 3: The plugins directory is emptied, and the files in the update/new_files/plugins directory are adopted. The node is restarted. This variant is chosen when the plugin list is provided but the library list is missing.
  • 4: Both directories are emptied, and the files in the "update/new_files" directory are adopted. The node is restarted. This variant is chosen when both lists are included.

Errors

In case of an error (e.g., a file is not found, the session token is invalid, or permissions are missing), the process is aborted. Please check the log of the affected nodes.

Rollback

You can undo the process at any time if you still have the old control file. The update process is then simply repeated with the old control file. However, you must enter a current session token in the file.

You can obtain the old control file by connecting to the node with the local console before the update (connect [node-id]) and typing version. The output will be displayed on standard out. Then, you simply need to copy the output into a new file. The name of the file does not matter.

Hint

Create the control file as described in the previous paragraph. Only change the filenames, versions, or hashes that you want to update.

Don't forget the session token!

nyssr.net - Innovative Distributed System