Target objects and object catalogs

New in version 0.5.0

Introduction

To any Result (GenericResult or Snapshot), you can attach catalogs of objects identified into your simulation data. The object types identified in any Catalog are described as TargetObject :

Each TargetObject is qualified with a set of properties (ObjectProperty instances) that are optionally grouped in property groups (ObjectPropertyGroup instances) ,

a Catalog gather a list of fields (CatalogField objects), each one referring to an ObjectProperty of the associated TargetObject.

A strict binding is enforced between a Catalog’s CatalogField and its TargetObject’s ObjectProperty, see Strict target object/catalog bindings for more details.

The content of a Catalog can be represented as a table where :

the rows correspond to the collection of TargetObject instances,

the columns correspond to the different CatalogField objects.

As a consequence, a Catalog can be quite naturally converted into a pandas.DataFrame, using the Catalog.to_pandas() method.

Example study structure

Target object definition

Target objects define the type of items you will be able to characterize into a Catalog. To initialize a TargetObject, you only need to set its name property. Here is a more complete example of a TargetObject initialization with all optional attributes :

>>> from astrophysix.simdm.catalogs import TargetObject
>>> cluster = TargetObject(name="Galaxy cluster", description="My cluster object full description")

See also

TargetObject API reference.

Target object properties

You can insert ObjectProperty into a TargetObject using its object_properties attribute :

>>> from astrophysix.simdm.catalogs import ObjectProperty
>>> # One-liner
>>> pos_x = cluster.object_properties.add(ObjectProperty(property_name="pos_x",
...                                                      description="x-axis position of the cluster center of mass"))
# Or
>>> pos_x = ObjectProperty(property_name="pos_x",
...                        description="x-axis position of the cluster center of mass")
>>> cluster.object_properties.add(pos_x)

To initialize an ObjectProperty, only the ObjectProperty.property_name property must be set :

>>> # Object properties should be initialised with at least a 'property_name' attribute.
>>> unknown = ObjectProperty()
AttributeError : ObjectProperty 'property_name' attribute is not defined (mandatory).

Other optional ObjectProperty attributes are :

>>> from astrophysix.simdm.catalogs import PropertyFilterFlag, PropertySortFlag
>>> from astrophysix.simdm.utils import DataType
>>> from astrophysix import units as U
>>>
>>> mass = ObjectProperty(property_name="$M_{500}$", description="Cluster mass",
...                       filter_flag=PropertyFilterFlag.BASIC_FILTER,
...                       sort_flag=PropertySortFlag.BASIC_SORT,
...                       dtype=DataType.REAL, unit=1.0e9*U.Msun)

Available property filter flag enum values are :

PropertyFilterFlag.NO_FILTER,

PropertyFilterFlag.BASIC_FILTER,

PropertyFilterFlag.ADVANCED_FILTER

and available property sort flag enum values are :

PropertySortFlag.NO_SORT,

PropertySortFlag.BASIC_SORT,

PropertySortFlag.ADVANCED_SORT

Warning

Only CatalogFields associated to ObjectProperty instances with :

filter_flag set to PropertyFilterFlag.BASIC_FILTER or PropertyFilterFlag.ADVANCED_FILTER will appear as filter fields in the online Galactica object catalog search form.

sort_flag set to PropertyFilterFlag.BASIC_SORT or PropertyFilterFlag.ADVANCED_SORT will appear as sortable fields in the online Galactica object catalog search form.

See also

ObjectProperty API reference,
PropertyFilterFlag, PropertySortFlag and DataType API references.

Object property groups

Optionally, you can group ObjectProperty instances in groups for better clarity in your workflow or enhanced display on the Galactica web application. You can insert ObjectProperty objects into a ObjectPropertyGroup with the ObjectPropertyGroup.group_properties attribute and include the ObjectPropertyGroup into your TargetObject using its TargetObject.property_groups attribute :

>>> from astrophysix.simdm.catalogs import ObjectPropertyGroup
>>>
>>> # Additional object properties (positions along y/z axes)
>>> pos_y = cluster.object_properties.add(ObjectProperty(property_name="pos_y"))
>>> pos_z = cluster.object_properties.add(ObjectProperty(property_name="pos_z"))
>>>
>>> # Velocity properties along x/y/z axes
>>> v_x = cluster.object_properties.add(ObjectProperty(property_name="v_x"))
>>> v_y = cluster.object_properties.add(ObjectProperty(property_name="v_y"))
>>> v_z = cluster.object_properties.add(ObjectProperty(property_name="v_z"))
>>>
>>> # Position property group
>>> pos = ObjectPropertyGroup(group_name="position", description="Cluster position")
>>> pos.group_properties.add(pos_x)
>>> pos.group_properties.add(pos_y)
>>> pos.group_properties.add(pos_z)
>>> cluster.property_groups.add(pos)
>>>
>>> # Veloicity property group
>>> vel = ObjectPropertyGroup(group_name="velocity", description="Cluster velocity")
>>> vel.group_properties.add(v_x)
>>> vel.group_properties.add(v_y)
>>> vel.group_properties.add(v_z)
>>> cluster.property_groups.add(vel)

To initialize an ObjectPropertyGroup, only the ObjectProperty.property_name property must be set :

>>> # Object property groups should be initialised with at least a 'group_name' attribute.
>>> unknown = ObjectPropertyGroup()
AttributeError : ObjectPropertyGroup 'group_name' attribute is not defined (mandatory).

See also

ObjectPropertyGroup API reference.

Catalog construction

To define a Catalog, only two attributes are mandatory :

name : a non-empty string value,

target_object : a TargetObject instance.

Here is a more complete example of a Catalog initialization with all optional attributes :

>>> from astrophysix.simdm.catalogs import TargetObject, Catalog
>>>
>>> cat = Catalog(target_object=cluster, name="Galaxy cluster catalog at $Z \simeq 2$",
...               description="This is the catalog of all galaxy clusters identified in the "
...               "simulation at redshift $Z\simeq 2$")

Setting catalog fields

Once the Catalog has been created, you can insert CatalogField objects in it, using the Catalog.catalog_fields property. To define a CatalogField, you must set :

obj_prop : catalog’s TargetObject’s ObjectProperty instance quantified in the field,

values : 1D numpy.ndarray (numeric type) containing the values of the field for each object.

>>> from astrophysix.simdm.catalogs import CatalogField
>>>
>>> cat.catalog_fields.add(CatalogField(obj_prop=pos_x, values=N.random.uniform(size=200))
>>> cat.catalog_fields.add(CatalogField(obj_prop=pos_y, values=N.random.uniform(size=200))
>>> cat.catalog_fields.add(CatalogField(obj_prop=pos_z, values=N.random.uniform(size=200))
>>> cat.catalog_fields.add(CatalogField(obj_prop=mass, values=N.array([...]))

Warning

Once the first CatalogField is inserted into a Catalog, it sets the number of objects contained in the Catalog (values 1D numpy.ndarray size). Every subsequent CatalogField addition into that Catalog MUST contain the same number of objects :

>>> cat.catalog_fields.add(CatalogField(obj_prop=vel_x, values=N.random.uniform(size=300))
AttributeError: Cannot add a catalog field with 300 item values in a catalog containing
200 items.

See also

Catalog and CatalogField API references,
How to add/remove objects in an existing Catalog ?

Strict target object/catalog bindings

When you manipulate TargetObject and their ObjectProperty and ObjectPropertyGroup, and you link them to a Catalog and its list of CatalogField objects), astrophysix makes sure for you that your entire study hierarchical structure remains consistent at all times :

upon object creation,
upon object addition into another object,
upon object deletion from another object.

Upon object creation

You are free to create any kind of astrophysix object, even those linked to another object :

>>> from astrophysix.simdm.catalogs import ObjectProperty
>>> from astrophysix.simdm.catalogs import CatalogField
>>>
>>> posy = ObjectProperty(property_name="pos_y")
>>> field_posy = CatalogField(obj_prop=posy, values=N.array([0.5, 0.6, 0.48, 0.65, 0.49]))

There is no risk of breaking your study hierarchical structure consistency.

Upon object addition

To avoid dangling references into the study hierarchical structure, astrophysix will not let you :

add a CatalogField object into the catalog_fields list of a Catalog if its associated ObjectProperty does not already belong to the Catalog’s TargetObject’s object_properties list,

add an ObjectPropertyGroup into the property_groups list of a TargetObject if all the ObjectProperty it contains does not already belong to the TargetObject’s object_properties list.

add a ObjectProperty object into a ObjectPropertyGroup contained in the property_groups list of a TargetObject if the ObjectProperty does not already belong to the TargetObject’s object_properties list.

Here are the examples :

>>> from astrophysix.simdm.catalogs import TargetObject, ObjectProperty, CatalogField, \
                                           Catalog
>>> # -------------------------------------------------------------------------------- #
>>> tobj = TargetObject(name="Spiral galaxy")
>>> x = tobj.object_properties.add(ObjectProperty(property_name="pos_x"))
>>> y = tobj.object_properties.add(ObjectProperty(property_name="pos_y"))
>>> z = ObjectProperty(property_name="pos_z")  # z not added in target object
>>> cat = Catalog(target_object=tobj, name="Spiral galaxy catalog")
>>> # Tries to add a catalog field with z property => error
>>> cat.catalog_fields.add(CatalogField(z, values=N.random.uniform(size=5))) # => Error
AttributeError: Cannot add catalog field. 'pos_z' target object property
is not a property of 'Spiral galaxy' target object.
>>>
>>> # Add first the 'pos_z' property into the 'Spiral galaxy' target object,
>>> tobj.object_properties.add(z)
>>> # THEN add the catalog field into the catalog
>>> cat.catalog_fields.add(CatalogField(z, values=N.random.uniform(size=5))) # => Ok
>>> # -------------------------------------------------------------------------------- #
>>>
>>> # -------------------------------------------------------------------------------- #
>>> alpha = ObjectProperty(property_name='alpha')
>>> delta = ObjectProperty(property_name='delta')
>>> beta = ObjectProperty(property_name='beta')
>>> g = ObjectPropertyGroup(group_name="Group1")
>>> # Tries to add properties to a group and then insert the group before the property
>>> # have been added to the TargetObject property list => raises an error
>>> g.group_properties.add(alpha)
>>> g.group_properties.add(delta)
>>> tobj.property_groups.add(g)
AttributeError: 'alpha' target object property does not belong to this TargetObject
object property list.
>>> # Add the properties in the TargetObject first
>>> tobj.object_properties.add(alpha)
>>> tobj.object_properties.add(delta)
>>> # THEN add the group
>>> tobj.property_groups.add(g)  # => Ok
>>> # -------------------------------------------------------------------------------- #
>>>
>>> # -------------------------------------------------------------------------------- #
>>> # Tries to add a property to an already registered group in the TargetObject
>>> # property group list while the property has not added yet to the TargetObject
>>> # property list => raises an error
>>> g.group_properties.add(beta)
AttributeError: 'beta' target object property does not belong to this TargetObject object
property list.
>>> # Add the property in the TargetObject first
>>> tobj.object_properties.add(gamma)
>>> # THEN insert it into the group
>>> g.group_properties.add(gamma)  # Ok
>>> # -------------------------------------------------------------------------------- #

Upon object deletion

To avoid missing references into the study hierarchical structure, astrophysix will also prevent you from deleting an ObjectProperty from a TargetObject.object_properties list if :

the ObjectProperty is included in one of the ObjectPropertyGroup of the TargetObject.property_groups,

any Catalog associated to that TargetObject contains a CatalogField that refers to the ObjectProperty to be deleted.

Here are the examples :

>>> from astrophysix.simdm.protocol import SimulationCode, InputParameter, Algorithm, \
                                           PhysicalProcess, AlgoType, Physics
>>> from astrophysix.simdm.experiment import Simulation, ParameterSetting, \
                                             AppliedAlgorithm, ResolvedPhysicalProcess
>>> tobj = TargetObject(name="Pre-stellar core")
>>> x = tobj.object_properties.add(ObjectProperty(property_name="pos_x"))
>>> y = tobj.object_properties.add(ObjectProperty(property_name="pos_y"))
>>> z = tobj.object_properties.add(ObjectProperty(property_name="pos_z"))
>>> pos = tobj.property_groups.add(ObjectPropertyGroup(group_name="position"))
>>> pos.group_properties.add(x)
>>> pos.group_properties.add(y)
>>> pos.group_properties.add(z)
>>>
>>> # ---------------------------------------------------------------------------------- #
>>> # Tries to delete x, while it is in 'position' group
>>> del tobj.object_properties[x.property_name]
AttributeError: ''pos_x' target object property' cannot be deleted, the following items
depend on it (try to delete them first) : ['Pre-stellar core' target object - 'position'
property group - 'pos_x' target object property].
>>>
>>> # Delete property from 'position' group first
>>> del pos.group_properties[x.property_name]
>>> # THEN delete 'pos_x' property from TargetObject => Ok
>>> del tobj.object_properties[x.property_name]
>>>
>>> # ---------------------------------------------------------------------------------- #
>>> cat = Catalog(target_object=tobj, name="Core catalog")
>>> fy = cat.catalog_fields.add(CatalogField(y, values=N.random.uniform(size=10)))
>>> # Tries to delete 'pos_y' property, while it is linked to fy catalog field
>>> del tobj.object_properties[y.property_name]
AttributeError: ''pos_y' target object property' cannot be deleted, the following items
depend on it (try to delete them first) : ['Core catalog' catalog 'pos_y' catalog field].
>>>
>>> # Delete CatalogField from Catalog first
>>> del cat.catalog_fields[y.property_name]
# THEN delete Property from TargetObject => Ok
del tobj.object_properties[y.property_name]