.. This file is part of the 'astrophysix' Python package documentation. ----------------------------------------------------------------------------------- Copyright © Commissariat a l'Energie Atomique et aux Energies Alternatives (CEA) ----------------------------------------------------------------------------------- ----------------------- FREE SOFTWARE LICENCING ----------------------- This software is governed by the CeCILL license under French law and abiding by the rules of distribution of free software. You can use, modify and/or redistribute the software under the terms of the CeCILL license as circulated by CEA, CNRS and INRIA at the following URL: "http://www.cecill.info". As a counterpart to the access to the source code and rights to copy, modify and redistribute granted by the license, users are provided only with a limited warranty and the software's author, the holder of the economic rights, and the successive licensors have only limited liability. In this respect, the user's attention is drawn to the risks associated with loading, using, modifying and/or developing or reproducing the software by the user in light of its specific status of free software, that may mean that it is complicated to manipulate, and that also therefore means that it is reserved for developers and experienced professionals having in-depth computer knowledge. Users are therefore encouraged to load and test the software's suitability as regards their requirements in conditions enabling the security of their systems and/or data to be ensured and, more generally, to use and operate it in the same conditions as regards security. The fact that you are presently reading this means that you have had knowledge of the CeCILL license and that you accept its terms. ----------------------- COMMERCIAL SOFTWARE LICENCING ----------------------------- You can obtain this software from CEA under other licencing terms for commercial purposes. For this you will need to negotiate a specific contract with a legal representative of CEA. Quick-start guide ================= This quick-start guide will walk you into the main steps to document your numerical project. If you want to skip the tour and directly see an example, see :ref:`full_example_script`. Project and study ----------------- Creation and persistency ^^^^^^^^^^^^^^^^^^^^^^^^ To create a :class:`~astrophysix.simdm.Project` and save it into a :class:`~astrophysix.simdm.SimulationStudy` HDF5 file, you may run this minimal Python script : .. code-block:: python >>> from astrophysix.simdm import SimulationStudy, Project, ProjectCategory >>> proj = Project(category=ProjectCategory.Cosmology, project_title="Extreme Horizons cosmology project") >>> study = SimulationStudy(project=proj) >>> study.save_HDF5("./EH_study.h5") Loading a study ^^^^^^^^^^^^^^^ To read the :class:`~astrophysix.simdm.SimulationStudy` from your HDF5 file, update it and save the updated study back in its original file : .. code-block:: python >>> from astrophysix.simdm import SimulationStudy >>> study = SimulationStudy.load_HDF5("./EH_study.h5") >>> proj = study.project >>> proj.short_description = "This is a short description (one-liner) of my project." >>> # Saves your updated study back in the same file './EH_study.h5' >>> study.save_HDF5() Study information ^^^^^^^^^^^^^^^^^ A :class:`~astrophysix.simdm.SimulationStudy` object contains details on when it has been created (:attr:`~astrophysix.simdm.SimulationStudy.creation_time` property) and last modified (:attr:`~astrophysix.simdm.SimulationStudy.last_modification_time` property) : .. code-block:: python >>> study.creation_time datetime.datetime(2020, 9, 4, 14, 05, 21, 84601, tzinfo=datetime.timezone.utc) >>> study.last_modification_time datetime.datetime(2020, 9, 18, 15, 12, 27, 14512, tzinfo=datetime.timezone.utc) Full project initialization example ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To initialize a :class:`~astrophysix.simdm.Project`, you only need to set :attr:`~astrophysix.simdm.Project.category` and :attr:`~astrophysix.simdm.Project.project_title` properties. Here is a more complete example of a :class:`~astrophysix.simdm.Project` initialization with all optional attributes : Available project categories are : * :attr:`ProjectCategory.SolarPhysics `, * :attr:`ProjectCategory.PlanetaryAtmospheres `, * :attr:`ProjectCategory.StellarEnvironments `, * :attr:`ProjectCategory.StellarEnvironments `, * :attr:`ProjectCategory.StarFormation `, * :attr:`ProjectCategory.StellarPhysics `, * :attr:`ProjectCategory.InterstellarMedium `, * :attr:`ProjectCategory.Magnetohydrodynamics `, * :attr:`ProjectCategory.Supernovae `, * :attr:`ProjectCategory.HighEnergyAstrophysics `, * :attr:`ProjectCategory.GalacticDynamics `, * :attr:`ProjectCategory.Cosmology `. .. code-block:: python >>> proj = Project(category=ProjectCategory.StarFormation, alias="FRIG", ... project_title="Self-regulated magnetised ISM modelisation", ... short_description="Short description of my 'FRIG' project", ... general_description="""This is a pretty long description for my project spanning over multiple lines ... if necessary""", ... data_description="The data available in this project...", ... acknowledgement="""Please cite this project using the following reference : [Hennebelle & Iffrig 2014].""", ... directory_path="/path/to/project/data/", thumbnail_image="~/images/project_highlight.jpg") .. note:: On the :galactica:`Galactica <>` web application, these properties will be displayed as follows : * the :attr:`~astrophysix.simdm.Project.project_title` property will be displayed in the project page top title bar, * the :attr:`~astrophysix.simdm.Project.short_description` and :attr:`~astrophysix.simdm.Project.thumbnail_image` properties will be displayed in the project thumbnail panel within the project category listing, * the :attr:`~astrophysix.simdm.Project.general_description` and :attr:`~astrophysix.simdm.Project.data_description` properties will be used as the Project page body, * the :attr:`~astrophysix.simdm.Project.acknowledgement` property will be displayed in the ``Cite me`` window in **all the project pages**, * the :attr:`~astrophysix.simdm.Project.directory_path` optional property will not be displayed on the web application but only used to get to the simulation raw data on the Terminus server. .. warning:: Setting the :attr:`~astrophysix.simdm.Project.alias` property is necessary only if you wish to upload your study on the :galactica:`Galactica simulation database <>`. See :ref:`why_an_alias` and :ref:`check_galactica_validation` .. seealso:: * :class:`~astrophysix.simdm.SimulationStudy`, :class:`~astrophysix.simdm.Project` and :class:`~astrophysix.simdm.ProjectCategory` API references. Simulation codes and runs ------------------------- To add a simulation run into your project, the definition of a :class:`~astrophysix.simdm.protocol.SimulationCode` is mandatory. Once defined, you can create a :class:`~astrophysix.simdm.experiment.Simulation` based on that simulation code: .. code-block:: python >>> from astrophysix.simdm.protocol import SimulationCode >>> from astrophysix.simdm.experiment import Simulation >>> ramses = SimulationCode(name="Ramses 3.1 (MHD)", code_name="RAMSES") >>> simu = Simulation(simu_code=ramses, name="Hydro run full resolution") To add the simulation to the project, use the :attr:`Project.simulations ` property (:class:`~astrophysix.simdm.utils.ObjectList`): .. code-block:: python >>> proj.simulations.add(simu) >>> proj.simulations Simulation list : +---+---------------------------+----------------------------------------+ | # | Index | Item | +---+---------------------------+----------------------------------------+ | 0 | Hydro run full resolution | 'Hydro run full resolution' simulation | +---+---------------------------+----------------------------------------+ | 1 | Hydro run full resolution | 'MHD run full resolution' simulation | +---+---------------------------+----------------------------------------+ .. seealso:: * :class:`~astrophysix.simdm.experiment.Simulation`, :class:`~astrophysix.simdm.protocol.SimulationCode` API references, * :ref:`usage_codes` detailed section. * :ref:`usage_runs` detailed section. * :class:`~astrophysix.simdm.utils.ObjectList` API reference. Post-processing runs -------------------- Optionally, you can add :class:`~astrophysix.simdm.experiment.PostProcessingRun` into a :class:`~astrophysix.simdm.experiment.Simulation` using the :attr:`Simulation.post_processing_runs ` property (:class:`~astrophysix.simdm.utils.ObjectList`). To create a :class:`~astrophysix.simdm.experiment.PostProcessingRun`, you must first define a :class:`~astrophysix.simdm.protocol.PostProcessingCode` : .. code-block:: python >>> from astrophysix.simdm.protocol import PostProcessingCode >>> from astrophysix.simdm.experiment import PostProcessingCodeRun >>> >>> radmc = PostProcessingCode(name="RADMC-3D", code_name="RADMC-3D", code_version="2.0") >>> pprun = PostProcessingCodeRun(ppcode=radmc, name="Synthetic observable creation of pre-stellar cores") >>> >>> # Add post-pro run into the simulation >>> simu.post_processing_runs.add(pprun) .. seealso:: * :class:`~astrophysix.simdm.experiment.PostProcessingRun`, :class:`~astrophysix.simdm.protocol.PostProcessingCode` API references, * :ref:`usage_codes` detailed section. * :ref:`usage_runs` detailed section. * :class:`~astrophysix.simdm.utils.ObjectList` API reference. Results and snapshots --------------------- Experiment-wide ^^^^^^^^^^^^^^^ You can add results into any simulation or post-processing run. If it is an experiment-wide result, create a :class:`~astrophysix.simdm.results.generic.GenericResult` and use the :attr:`Simulation.generic_results ` or :attr:`PostProcessingRun.generic_results ` property (:class:`~astrophysix.simdm.utils.ObjectList`) to add it into your run : .. code-block:: python >>> from astrophysix.simdm.results import GenericResult >>> >>> res1 = GenericResult(name="Star formation history", directory_path="/my/path/to/result", ... description="""This is the star formation history during the 2 ... Myr of the galaxy major merger""") >>> simu.generic_results.add(res1) Time-specific ^^^^^^^^^^^^^ Otherwise, if it is time-specific result, create a :class:`~astrophysix.simdm.results.snapshot.Snapshot` and use the :attr:`Simulation.snapshots ` or :attr:`PostProcessingRun.snapshots ` property (:class:`~astrophysix.simdm.utils.ObjectList`) to add it into your run : .. code-block:: python >>> from astrophysix.simdm.results import Snapshot >>> from astrophysix import units as U >>> sn34 = Snapshot(name="Third pericenter", time=(254.7, U.Myr), ... directory_path="/my/path/to/simu/outputs/output_00034") >>> simu.snapshots.add(sn34) .. seealso:: * :class:`~astrophysix.simdm.results.generic.GenericResult`, :class:`~astrophysix.simdm.results.snapshot.Snapshot` API references, * :ref:`usage_runs` detailed section. * :ref:`usage_results` detailed section. * :class:`~astrophysix.simdm.utils.ObjectList` API reference. Object catalogs --------------- *New in version 0.5.0* You can add object catalogs into any result (:class:`~astrophysix.simdm.results.generic.GenericResult` or :class:`~astrophysix.simdm.results.snapshot.Snapshot`), using the :attr:`~astrophysix.simdm.results.generic.GenericResult.catalogs` property : .. code-block:: python >>> from astrophysix.simdm.catalogs import TargetObject, ObjectProperty, Catalog, CatalogField,\ ... PropertyFilterFlag >>> from astrophysix.simdm.utils import DataType >>> from astrophysix import units as U >>> >>> cluster = TargetObject(name="Galaxy cluster") >>> x = tobj.object_properties.add(ObjectProperty(property_name="x", unit=U.Mpc, ... filter_flag=PropertyFilterFlag.BASIC_FILTER)) >>> y = tobj.object_properties.add(ObjectProperty(property_name="y", unit=U.Mpc, ... filter_flag=PropertyFilterFlag.BASIC_FILTER)) >>> z = tobj.object_properties.add(ObjectProperty(property_name="z", unit=U.Mpc, ... filter_flag=PropertyFilterFlag.BASIC_FILTER)) >>> # You may group properties in a property group >>> pos = ObjectPropertyGroup(group_name="position") >>> pos.group_properties.add(x) >>> pos.group_properties.add(y) >>> pos.group_properties.add(z) >>> >>> m = tobj.object_properties.add(ObjectProperty(property_name="M500", unit=U.Msun, ... filter_flag=PropertyFilterFlag.BASIC_FILTER)) >>> >>> # Define a catalog >>> cat = Catalog(target_object=tobj, name="Galaxy cluster catalog") >>> # Add the catalog fields into the catalog (100 clusters) >>> cat.catalog_fields.add(CatalogField(x, values=N.random.uniform(size=100))) >>> cat.catalog_fields.add(CatalogField(y, values=N.random.uniform(size=100))) >>> cat.catalog_fields.add(CatalogField(z, values=N.random.uniform(size=100))) >>> cat.catalog_fields.add(CatalogField(m, values=N.random.uniform(size=100))) >>> >>> # Add the catalog in the snapshot >>> sn34.catalogs.add(cat) .. seealso:: * :class:`~astrophysix.simdm.catalogs.catalog.Catalog` and :class:`~astrophysix.simdm.catalogs.field.CatalogField` API references, * :class:`~astrophysix.simdm.catalogs.targobj.TargetObject`, :class:`~astrophysix.simdm.catalogs.targobj.ObjectProperty` and :class:`~astrophysix.simdm.catalogs.targobj.ObjectPropertyGroup` API references, * :ref:`usage_catalogs` detailed section. * :class:`~astrophysix.simdm.utils.ObjectList` API reference. Datafiles --------- You can add :class:`~astrophysix.simdm.results.Datafile` objects into any :class:`~astrophysix.simdm.results.snapshot.Snapshot` or :class:`~astrophysix.simdm.results.generic.GenericResult`, or even (*new in version 0.5.0*) in an object :class:`~astrophysix.simdm.catalogs.catalog.Catalog` : .. code-block:: python >>> from astrophysix.simdm.datafiles import Datafile >>> >>> # Add a datafile into a snapshot >>> imf_df = Datafile(name="Initial mass function", ... description="This is my IMF plot detailed description...") >>> sn34.datafiles.add(imf_df) And then embed files from your local filesystem into the :class:`~astrophysix.simdm.results.Datafile` (the file raw byte array will be imported along with the original file `name` and `last modification date`): .. code-block:: python >>> from astrophysix.simdm.datafiles import image >>> from astrophysix.utils.file import FileType >>> >>> # Attach files to datafile >>> imf_df[FileType.PNG_FILE] = os.path.join("/data", "io", "datafiles", "plot_image_IMF.png") >>> jpg_fpath = os.path.join("/data", "io", "datafiles", "plot_with_legend.jpg") >>> imf_df[FileType.JPEG_FILE] = image.JpegImageFile.load_file(jpg_fpath) >>> imf_df[FileType.HDF5_FILE] = os.path.join("/data", "io", "HDF5", "stars.h5") Anyone can reopen your :class:`~astrophysix.simdm.SimulationStudy` HDF5 file, read the attached files and re-export them on their local filesystem to retrieve a carbon copy of your original file: .. code-block:: python >>> imf_df[FileType.JPEG_FILE].save_to_disk("/home/user/Desktop/export_plot.jpg") File '/home/user/Desktop/export_plot.jpg' saved .. seealso:: * :class:`~astrophysix.simdm.datafiles.Datafile` API references, * :ref:`usage_results` detailed section. Data-processing services ------------------------ You can define bindings between :class:`~astrophysix.simdm.results.snapshot.Snapshot` objects (or even :class:`~astrophysix.simdm.catalogs.catalog.Catalog` objects) and :galactica:`Galactica <>` online data-processing services to allow visitors to request the execution of post-processing jobs on your raw astrophysical simulation data : .. code-block:: python >>> from astrophysix.simdm.results import Snapshot >>> from astrophysix import units as U >>> from astrophysix.simdm.services import DataProcessingService >>> sn_Z2 = Snapshot(name="Z~2", data_reference="output_00481") >>> simu.snapshots.add(sn_Z2) >>> >>> # Add data processing services to a snapshot >>> dps = DataProcessingService(service_name="column_density_map", ... data_host="My_Dept_Cluster") >>> sn.processing_services.add(dsp) .. seealso:: * Data-processing server : see `Terminus `_ * :class:`~astrophysix.simdm.services.process.DataProcessingService`, * :ref:`processing_services` detailed section. .. _full_example_script: Full example script ------------------- .. literalinclude:: ../../example.py :language: python