Back to home page

sPhenix code displayed by LXR

 
 

    


Warning, /acts/docs/examples/python_bindings.rst is written in an unsupported language. File is not indexed.

0001 .. _python-bindings:
0002 
0003 Python bindings for the Examples
0004 ================================
0005 
0006 The examples part of ACTS ships with python bindings using the ``pybind11``
0007 library. Building these bindings can be enabled via
0008 ``-DACTS_BUILD_EXAMPLES_PYTHON_BINDINGS=ON``, and requires a python installation
0009 including the development files to be installed. You can then build the special
0010 target `ActsPythonBindings` to build everything that can be accessed in python.
0011 
0012 The build will create a setup script in ``$BUILD_DIR/python/setup.sh`` which
0013 modifies ``$PYTHONPATH`` so that you can import the ``acts`` module in python.
0014 
0015 Here is a minimal example of a python script using the example bindings, which
0016 sets up the particle propagation and runs a few events.
0017 
0018 .. code-block:: python
0019 
0020    import os
0021 
0022    import acts
0023    import acts.examples
0024 
0025    detector, trackingGeometry, contextDecorators  = acts.examples.GenericDetector.create()
0026    s = acts.examples.Sequencer(events=10)
0027 
0028    rnd = acts.examples.RandomNumbers(seed=42)
0029 
0030    nav = acts.Navigator(trackingGeometry=trackingGeometry)
0031 
0032    field = acts.ConstantBField(acts.Vector3(0, 0, 2 * acts.UnitConstants.T))
0033    stepper = acts.EigenStepper(field)
0034 
0035    prop = acts.examples.ConcretePropagator(acts.Propagator(stepper, nav))
0036 
0037    alg = acts.examples.PropagationAlgorithm(
0038        propagatorImpl=prop,
0039        level=acts.logging.INFO,
0040        randomNumberSvc=rnd,
0041        ntests=1000,
0042        sterileLogger=False,
0043        propagationStepCollection="propagation-steps",
0044    )
0045 
0046    s.addAlgorithm(alg)
0047 
0048    outputDir = "."
0049    objDir = outputDir + "/obj"
0050    if not os.path.exists(objDir):
0051       os.mkdir(objDir)
0052    
0053    s.addWriter(
0054        acts.examples.ObjPropagationStepsWriter(
0055            level=acts.logging.INFO,
0056            collection="propagation-steps",
0057            outputDir=objDir,
0058        )
0059    )
0060 
0061    s.addWriter(
0062        acts.examples.RootPropagationStepsWriter(
0063            level=acts.logging.INFO,
0064            collection="propagation-steps",
0065            filePath=outputDir + "/propagation_steps.root",
0066        )
0067    )
0068 
0069    s.run()
0070 
0071 Python based example scripts
0072 ----------------------------
0073 
0074 The repository contains a set of example scripts that can be used to execute various workflows.
0075 They can be found in ``$REPO_ROOT/Examples/Scripts/Python``. Make sure you have run
0076 
0077 .. code-block:: console
0078 
0079    source $BUILD_DIR/python/setup.sh
0080 
0081 to make sure python can find the ``acts`` module.
0082 
0083 Python based unit tests
0084 -----------------------
0085 
0086 A number of unit tests based on the ``pytest`` library are shipped with the
0087 repository. They are located under ``$REPO_ROOT/Examples/Python/tests``, and
0088 intend to cover the public API of the python bindings. A set of tests also
0089 executed the standalone example scripts.
0090 
0091 To run these python based tests, ``pytest`` and a few other dependencies need
0092 to be installed. They can be installed via ``pip install -r
0093 Examples/Python/tests/requirements.txt`` from the repository root.  You can
0094 then simply run ``pytest`` from the repository root.
0095 
0096 .. tip::
0097    :name: python-virtualenv
0098 
0099    It is **strongly recommended** to use a `virtual environment`_ for
0100    this purpose! For example, run 
0101 
0102    .. code-block:: console
0103 
0104       $ python -m venv venv
0105       $ source venv/bin/activate
0106 
0107    to create a local virtual environment, and then run the `pip` command above.
0108 
0109 
0110 .. _virtual environment: https://realpython.com/python-virtual-environments-a-primer/
0111 
0112 .. _root_hash_checks:
0113 
0114 ROOT file hash regression checks
0115 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
0116 
0117 In a number of cases, the python based test suite will run hash based regression tests against ROOT files that are
0118 written by the test workloads. These tests use a custom hash algorithm written in python, which hashes each individual
0119 entry of each ``TTree`` found in a file. These entry hashes are then sorted, concatenated and hashed again for the final output.
0120 This procedure ensures that if the ROOT file content changes, the hash changes, while also giving the same hash when the events
0121 stored in the file are reordered.
0122 
0123 The tests are implemented by looking up a reference hash from a central data file ``$REPO_ROOT/Examples/Python/tests/root_file_hashes.txt``
0124 that looks like
0125 
0126 .. code-block:: none
0127 
0128    test_ckf_tracks_example_full_seeding__performance_seeding_trees.root: 938bcc9b9425b12c620f5d0efa2c592817dfe92a18c309e97aa9d87412918620
0129    test_ckf_tracks_example_full_seeding__trackstates_ckf.root: 2faceafd4a521ff4030557301723e29c3d870edad052965eb644b824b57e2146
0130    test_ckf_tracks_example_truth_estimate__performance_seeding_trees.root: 5c0cf9e84af64e6814ab1ddf4cbaf4be6008ad8b2371b5b0241085b19d0fc52c
0131    test_ckf_tracks_example_truth_estimate__performance_seeding_trees.root: 5c0cf9e84af64e6814ab1ddf4cbaf4be6008ad8b2371b5b0241085b19d0fc52c
0132    test_ckf_tracks_example_truth_estimate__trackstates_ckf.root: ac4485c09a68fca3d056cb8d9adb81695e68d822629e48c71fd2b6d2bbd31f88
0133    # ...
0134 
0135 where the left side before the ``:`` indicates the test in which the check is performed and the name of the ROOT file 
0136 that is checked. The right side is the reference hash.
0137 
0138 .. note:: The file from which reference hashes are loaded can be changed by setting the environment variable ``ROOT_HASH_FILE``
0139           to the desired file.
0140 
0141 These checks have two purposes:
0142 
0143 1. Detect regressions in the algorithms: if an algorithm produces different output, the test will catch it. This also means that 
0144    if algorithmic changes are made that intentionally change the output, the reference hashes also have to be updated.
0145 
0146    .. warning:: Please make sure to check the contents of a changed file are correct/reasonable before updating the reference hash!
0147 
0148 2. Detect potential reproducibility issues. Tests that run with multiple threads should produce the same output every run,
0149    event ordering aside. If a test workload has a thread-reproducibility issue, the output hash should also change.
0150 
0151 Running the hash checks locally and how to update the reference hashes
0152 """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
0153 
0154 By default, the hash checks are not executed when the ``pytest`` command is run. To enable them, you need to set the environment
0155 variable ``ROOT_HASH_CHECKS`` needs to be set to ``ON``, for example like:
0156 
0157 .. code-block:: console
0158 
0159    ROOT_HASH_CHECKS=ON pytest
0160 
0161 If any hash mismatches are observed, the corresponding tests will fail, and ``pytest`` will print a summary at the end that looks like
0162 
0163 .. code-block:: console
0164 
0165    ------------------------------------------- RootHashAssertionErrors -----------------------------------------------------
0166    The ROOT files produced by tests have changed since the last recorded reference.
0167    This can be be expected if e.g. the underlying algorithm changed, or it can be a test failure symptom.
0168    Please manually check the output files listed below and make sure that their content is correct.
0169    If it is, you can update the test reference file Examples/Python/tests/root_file_hashes.txt with the new hashes below.
0170 
0171    test_seeding__estimatedparams.root: 8bbc97cb3d4777c61dd0b483a1c8268fc8411ad182c35bc731e5ed222450deca
0172    test_material_recording__geant4_material_tracks.root: 019ce62ce378efa5c02a94768039686ed3cdfbd60c115c1f0cab2cbc53def57b
0173    test_material_mapping__material-maps_tracks.root: c03215e8b53733a3a7d7a0a5f9aec5bf2df20e8e40cc0492a8fa22400491d216
0174    test_material_mapping__propagation-material.root: a15a5c1e92fc3b848efb232eea1d40c422ee3a1d9ef1f7140294415621a04ce5
0175    test_ckf_tracks_example_full_seeding__tracksummary_ckf.root: 9e4d14169f20961be38d0305853a7cf7eeea4a647f0c94a48aada22c3c2c7a51
0176    test_ckf_tracks_example_truth_estimate__tracksummary_ckf.root: 3d56b26788163852e2c1f7288920f60a505bd14deeabb6f9189b680fcd90bfc5
0177    test_ckf_tracks_example_truth_smeared__tracksummary_ckf.root: ca2ce4069d2a2388c3d3c826dec8bea9f9d1e622239a20f8b985784d6c546c6e
0178    =========================================== short test summary info =====================================================
0179    FAILED Examples/Python/tests/test_examples.py::test_seeding
0180    FAILED Examples/Python/tests/test_examples.py::test_material_recording
0181    FAILED Examples/Python/tests/test_examples.py::test_material_mapping
0182    FAILED Examples/Python/tests/test_examples.py::test_ckf_tracks_example_full_seeding
0183    FAILED Examples/Python/tests/test_examples.py::test_ckf_tracks_example_truth_estimate
0184    FAILED Examples/Python/tests/test_examples.py::test_ckf_tracks_example_truth_smeared
0185    ================================== 6 failed, 183 passed in 199.82s (0:03:19) ============================================
0186 
0187 Here, we see that 7 hash checks have failed. The error output conveniently has the same format as the reference hashes found in ``root_file_hashes.txt``.
0188 To update the reference hashes, simply replace the corresponding entries in ``root_file_hashes.txt`` with the output from the ``pytest`` run.
0189 
0190 .. note:: The CI runs the ROOT hash checks. However, we have observed the hashes to change between different machines.
0191           This is believed to be due to differences in math libraries producing slightly different outputs. As a consequence,
0192           locally obtained file hashes might cause CI failures, as the CI hashes are different. 
0193 
0194           For local testing, it is therefore advisable to use ``ROOT_HASH_FILE`` to use a different file for the reference hashes
0195           and populated it with known-good reference hashes from the ``main`` branch, before testing your developments.
0196 
0197           To make the CI succeed if it obtains different hashes than you get locally: make sure that the output is correct, and then
0198           update the central ``root_file_hashes.txt`` with the hashes reported in the failed CI job.