Results and associated datafiles

To any Experiment (Simulation or PostProcessingRun), you can attach results of your scientific analysis. There are two kinds of result available in astrophysix:

  • GenericResult : result not strictly related to a particular instant in the dynamical evolution of your numerical experiment (e.g. star formation history, solar activity cycles, planetary orbital decay, etc.),

  • Snapshot : result corresponding to a specific moment during the numerical experiment (galactic pericentric passage, solar activity peak, star formation burst, etc.).

../_images/result_inheritance.png

Generic result

Here is a full example on how to create a GenericResult object with all optional parameters and add it into an Experiment, using the Simulation.generic_results or PostProcessingRun.generic_results property :

>>> from astrophysix.simdm.results import GenericResult
>>>
>>> res1 = GenericResult(name="Star formation history",
...                      directory_path="/my/path/to/result",
...                      description="""This is the star formation history during
...                                     the 2 Myr of the galaxy major merger""")
>>> simu.generic_results.add(res1)

Snapshot

A Snapshot derives from a GenericResult object but with additional optional properties :

Here is a full example on how to create a Snapshot object and add it into any Experiment, using Simulation.snapshots or PostProcessingRun.snapshots property :

>>> from astrophysix.simdm.results import Snapshot
>>> from astrophysix import units as U

>>> sn34 = Snapshot(name="Third pericenter",
...                 time=(254.7, U.Myr),
...                 physical_size=(400.0, U,kpc),
...                 decription="""This snapshot corresponds to the third pericentric
...                               of the galactic major merger simulation, occurring
...                               around $t\simeq255 \; \\textrm{Myr}$.""",
...                 data_reference="34;Big_endian",
...                 directory_path="/my/path/to/galactic_merger/simu/outputs/output_00034")
>>> simu.snapshots.add(sn34)

Note

The Snapshot.data_reference property is only used by the Galactica web application to provide a reference on your raw simulation data files to the on-demand post-processing services (see Terminus documentation.

Datafiles

One of the most important feature implemented in the astrophysix package is the possibility to insert documents into a SimulationStudy and to describe each one of them with meta-information.

../_images/simdm_datafiles.png

To do so, you must create a Datafile (the name attribute is the only one mandatory) and then add it into your Snapshot (or GenericResult) using the Snapshot.datafiles (or GenericResult.datafiles) property :

>>> from astrophysix.simdm.datafiles import Datafile, PlotType, PlotInfo, image, file
>>> from astrophysix.utils.file import FileType, FileUtil
>>>
>>> imf_df = Datafile(name="Initial mass function",
...                   description="This is my IMF plot detailed description...")
>>> snapshot_100.datafiles.add(imf_df)

Attached files

Once created, a single Datafile can contain different files, but at most one per FileType. The available FileType values are :

To add files from your filesystem in a Datafile, you can do it in 2 steps (create first a AssociatedFile and then put it in the Datafile):

>>> import os
>>> from astrophysix.simdm.datafiles import image, file
>>>
>>> # JPEG image
>>> jpeg_filepath = os.path.join("/data", "path", "to", "my", "plots", "IMF_plot.jpg")
>>> jpeg_image_file = image.JpegImageFile.load_file(jpeg_filepath)
>>> imf_df[FileType.JPEG_FILE] = jpeg_image_file
>>>
>>> # HDF5 file
>>> hdf5_filepath = os.path.join("/data", "path", "to", "raw", "datasets", "all_stars.h5")
>>> hdf5_file = file.HDF5File.load_file(hdf5_filepath)
>>> imf_df[FileType.HDF5_FILE] = hdf5_file

or if you are in a hurry, you can do it in a single one :

>>> import os
>>> from astrophysix.simdm.datafiles import image, file
>>>
>>> imf_df[FileType.JPEG_FILE] = os.path.join("/data", "path", "plots", "IMF_plot.jpg")
>>> imf_df[FileType.HDF5_FILE] = os.path.join("/data", "path", "datasets", "all_stars.h5")

To delete a file from a Datafile use the del Python operator:

>>> del imf_df[FileType.HDF_FILE]

The AssociatedFile contains your file raw byte array and has information on the original file (filename, last modification time). It can be used to re-export your file from your SimulationStudy to save it on your local filesystem (it even preserves the last modification time of the original file):

>>> jpeg_image_file = imf_df[FileType.JPEG_FILE]
>>> jpeg_image_file.last_modified
datetime.datetime(2020, 9, 22, 10, 42, 18, tzinfo=datetime.timezone.utc)
>>> saved_path = os.path.join("/home", "user", "Desktop", jpeg_image_file.filename)
>>> jpeg_image_file.save_to_disk(saved_path)
File '/home/user/Desktop/IMF_plot.jpg' saved
>>>
>>> import filecmp
>>> # Is the file saved identical to the original one ?
>>> filecmp.cmp(saved_path, jpeg_filepath, shallow=False)
True
>>>
>>> from astrophysix.utils.file import FileUtil
>>> from astrophysix.utils import DatetimeUtil
>>> # Was the file 'last modification time' preserved ?
>>> last_mod_tms = FileUtil.last_modification_timestamp(fpath)
>>> last_mod_dt = DatetimeUtil.utc_from_timestamp(last_mod_tms)
>>> last_mod_dt == jpeg_image_file.last_modified
True

Note

Since you can embed all your reduced data files into a SimulationStudy, you can safely remove your datafiles from your local filesystem and use the SimulationStudy HDF5 file as a self-contained, portable filesystem that you can exchange with your scientific collaborators.

Plot information

A Datafile can also have additional meta-information on a scientific plot for which you may already have attached PNG files (or JPEG, etc.). This meta-information can be used by other users to reproduce your plot or by the Galactica web application to display an interactive version of your plot online.

>>> from astrophysix.simdm.datafiles import PlotType, PlotInfo
>>> from astrophysix import units as U
>>>
>>> imf_df.plot_info = PlotInfo(plot_type=PlotType.LINE_PLOT, title="My plot title",
...                             xaxis_values=N.array([10.0, 20.0, , 22.0, 24.2, 30.0]),
...                             yaxis_values=N.array([1.2, 35.2, 5.2, 21.2, 14.9]),
...                             xaxis_log_scale=False, yaxis_log_scale=True,
...                             xlabel="x-axis label", ylabel="y-axis label",
...                             xaxis_unit="Myr", yaxis_unit=U.kpc)