File Operations

File Operations#

Systems map to folders on the XCOMPUTE-SERVER host.

  • Each virtual system is a branch of your project; organizing files (e.g. code, design, geometry, data, docs, etc). Systems and sub-systems (children) form a filesystem hierarchy on the host server. This can and should be your project’s digital source-of-truth (as the informational embodiment).

  • Files (and folders!) can be imported and exported from a permitted user’s computer to/from a connected host server. Supported files are interpreted into high-performance objects for computation and/or rendering. Unrecognized file types go along for the ride and can also be exported & launched directly from our GUI. See interface.cfg to assign extension commands.

I/O Configuration#

The client interface can import/export external files to/from the server, while the server host saves/loads native files to/from its filesystem:

Input

Output

Server

Load from host

Save to host

Client

Import from remote

Export to remote

To define the import/export/save/load path, set input and output in client.cfg and server.cfg:

input=/path/to/project
output=/path/to/project

  • Usually, input and output are set to the same directory. Make sure you or user xcompute has ownership with rw (read/write) permissions, else you will get a critical error when trying to save or load. On Unix OS, check with:

ls -al /path/to/project

  • XCOMPUTE-SERVER saves the project state (setup, geometry, and data) of each system using native XCS/JSON, XCG, and XCO files. These files can be accessed via almost any programming language with XC-Messages.

  • Unless otherwise named, the top-level “root” system defaults to the session’s launch timestamp (to avoid accidental overwriting of work).

The Active System#

Within an XCOMPUTE-CLIENT session, file operations are performed within the active system (unless something is selected).

  • The interface’s active system can be changed by clicking its name in the Navigation Tree (top left in-scene), or by double-clicking on its spatial representation in-scene.

  • Typically, only one system is operated upon at a time, dictating the information and operations available. This optimizes concurrent teamwork for multiple people (or agents) as projects get more complex.

Importing Files#

Importing typically transmits a file from user’s XCOMPUTE-CLIENT to a connected team’s XCOMPUTE-SERVER filesystem.

Recognized file types include:

  • System: XCS, JSON

  • Data: XCO, CSV, VTU

  • Geometry: XCG, STL, MSH, SDF

Note: All file types can be attached and launched from an XC project. See interface.cfg to assign extension commands.

  • No matter the file type, currently, an 8 GB/file (1<<34) limit is enforced on XC networks. Please contact XC to propose an increase to transmission limits.

Geometry Files#

To import a geometry file, in the file menu click Import Geometry, or right click in the scene to show the context menu via Import>Geometry.

  • To provide a common basis for CAE workflows, XC recognizes tesselated formats such as STL, OBJ, and MSH formats. The reason is that essentially every CAD program can export to these simplistic formats. This provides a common input to both spatial numerics and visualization (both of which utilize massive parallelism with linear scaling). As such, these discrete representations are most appropriate for downstream SDF-based physics and optimization.

  • Imported surfaces are represented as discrete meshes that are processed into volumetric signed-distance fields (SDF). Many numerical methods (including mesh-gen) direct utilize the SDF field; it is possible to bypass an explicit surface representation by utilizing an SDF file. Its schema defined by Messages::SparseMatrix in vector.proto (Note: this SDF file cannot be directly rendered. Work-in-progress.)

Other Files#

In the XCOMPUTE-CLIENT file menu click Import File, or right click in the scene to show the context menu via Import>File.

  • All file types can be imported (aka attached) into systems to organize a source-of-truth for the team’s digital project. All files can then be exported via XCOMPUTE-CLIENT and worked upon with other software.

Exporting Files#

When the export command is received by XCOMPUTE-SERVER, the server host creates (or finds) the requested file and stages it in its output path. The file is then transmit over network to XCOMPUTE-CLIENT and written to the output path defined in client.cfg. Export file types include:

  • System: XCS, JSON

  • Data: XCO, CSV, VTU

  • Geometry: XCG, STL, MSH, SDF

External Files#

From XCOMPUTE-CLIENT, it is possible to graphically navigate, export and launch files in external software. To do so, set the file type extension association in interface.cfg to the program command. For example, if Paraview, GMsh, Excel, and Matlab are installed, one might add:

vtu=paraview
msh=gmsh --some-option
csv=excel
mat=matlab --headless

  • Ensure that the other software is installed and in your OS’s path (e.g. /usr/bin/). Everything following the = is evaluated as a shell command. Upon the next launch or load interface.cfg, the context menu will then show an entry open in .... This is an easy way for XCOMPUTE to act as a project hypervisor.

Save#

When save is invoked, XCOMPUTE-SERVER writes native JSON text (or XCS binary) setup, XCG geometry binary, and XCO binary data files for the given system and its subsystems to the output path defined in server.cfg. These native files can be accessed via XC-Messages.

  • When loading a resource, XCOMPUTE-SERVER firsts look into the designated system folder. If the expected resource is not there it will look in the top-level of the input path. This prioritizes hierarchical (local) resources whilst permitting referencing of flat (global) resources as desired.

System#

From the XCOMPUTE-CLIENT file menu, the save button invokes the host’s filesystem write action upon the active system and its subsystems. The save all button inokes the write action starting at the root system and progressing through all systems.

  • Within a system (folder), a XCS/JSON file act as the nucleus; it specifies the setup and parameters of the associated files and subsystems. This includes field associated with: project management, physics, conditions, and solver sequences or algorithm names. Associated geometry, data, and other files are referenced externally by filepath.

  • XCS/JSON files are the std::filesteam of the serialized Messages::System found in XC-Messages’ concept.proto. For convenience, human-readable JSON is set as default over binary XCS. XCS/JSON files should have the same name as folder containing it.

Data#

When a solver or sequence of algorithms are executed, XCOMPUTE-SERVER invokes the instructions one or more times. Solvers can execute iteratively defined by the execution_count. Depending on the save_interval, XCO “data frames” are produced in the server’s output path.

  • Data files are organized by iteration and Property Key, handy for humans and machines whilst faciliating optimal parallel write/read. The XCO file is simply a standard file stream of the serialized Messages::Vector64 found in vector.proto.

  • Each XCS/JSON system and XCG geometry can reference XCO data stored within each system as a hidden folder, such as _system_data or _geometry_data; else it may be named _default_data.

Example: Illustration of filesystem organization#

The preceding _ provides a directive to load these folders as data rather than subsystems.

For brevity here, PropertyKey names are replaced with *.

Suppose, system data is organized for each iteration 0, 1, 2, ... , I:

_system_data/0/*.xco
_system_data/1/*.xco
_system_data/2/*.xco
...
_system_data/I/*.xco

Geometry data would follow suit, albeit perhaps only up to J iterations:

_geometry_data/0/*.xco
_geometry_data/1/*.xco
_geometry_data/2/*.xco
...
_geometry_data/J/*.xco

The materials database is saved in similar fashion, but under folder _material_data/:

_material_data/<MATERIAL>/_default_data/0/*.xco

  • System XCO files can be deleted manually by the server admin or via XCOMPUTE-CLIENT’s interface in the Data tab. Likewise, the current system’s data state can be changed by loading XCO files IFF they have the same number of nodes.

Geometry#

Geometric assets are saved in the native geometry format, following the patterns established above except the following:

  • Due to its potential size, the produced XCG file is encoded stream of Messages::Topology (found in spatial.proto). The geometry file contains the nodal positions and element indices, and/or other embedded spatial data.

Load#

System#

Data#

Geometry#