2024 Data Taking#

Overview#

This page provides information for users involved in the 2024 data taking. It includes details on running productions in parallel with data taking, configuration options to control this behavior, and the implications of archiving active productions.

Running Productions in Parallel to Data Taking#

Productions can be left running in parallel to data taking, allowing them to collect data as it is taken. Typically, a production is made up of two steps:

  • DaVinci: The system waits for 10 files from the sprucing, all stored at the same site, before submitting jobs.

  • Merging: The system waits for approximately 6GB worth of input data to be available before merging it into one file.

Data is only available after the merging has been completed. To ensure that data arrives frequently, productions are “flushed” approximately once a week to override these criteria and force the processing and merging of whatever data is currently available.

Controlling Parallel Data Collection#

To control whether productions should continue collecting data in parallel with data taking, use the keep_running configuration option in your info.yaml file.

By default, productions are set to continue running and collect new data as it becomes available. To prevent this behavior, set keep_running to false.

Example:

my_production_job:
  input:
    bk_query: /some/bookkeeping/path
    keep_running: false
  options:
    - my_options_file.py
  output: my_output_file.root

Archiving Active Productions#

If you archive a production that is still active, it will be stopped automatically. This ensures that resources are not wasted on collecting data that is no longer needed.

Steps to Archive an Active Production:

  1. Navigate to your production on the Analysis Productions webpage.

  2. Select the datasets you wish to archive.

  3. Confirm your selection and proceed with archiving.

Important

Archiving a production will stop it from collecting new data and mark the existing data as safe for removal. Ensure that you no longer need data from this production before archiving it.

For more detailed instructions on archiving, refer to the Housekeeping section.

Next Steps#

For additional information on configuring productions and managing outputs, refer to the following sections:

If you have any questions, feel free to reach out on the DPA WP2 Analysis Productions mattermost channel.