pyAML organization until next workshop

[GamelinAl]

Now that the development of prototypes has started I think it would be very good to identify better the different tasks we need to meet our goals and share the workload accordingly.

Could we discuss these points in a (near) future maintainer meetings to agree on which direction we want to go and who is responsible a given task?

It would help to reach the use case targets we have for the February workshop and prepare future development.

Tentative roadmap by @gubaidulinvadim (to remind us of the set goals for next workshop and where we stand now):

Task list proposition (most of the big items can be worked on somewhat independently):

1. Develop pyAML core
   1. control system abstraction (epics, tango, doocs)
   2. hardware abstraction (classes for magnets, bpms, ElementArrays, ...)
   3. measurements (algorithms, data management, compatibility with measurements engines, ...)
2. Setting up a shared virtual accelerator instances
   1. Ideally available for every control system we support (could be HZB twin, a modified version of ESRF twin, ...)
   2. Distribute pyAML prototypes with virtual accelerator instances in docker image to ease testing to provide fast feedback to the pyAML core developer team
   3. Integrate in CI/CD for testing
   4. Add more tests and examples
3. Starting building blocks for future control modes
   1. virtual accelerator components (should give the basic common features of a virtual accelerator, may be used in the future to run all pyAML tests)

For me the 2nd item is the most urgent at this stage, if the HZB twin solution proposed in https://github.com/orgs/python-accelerator-middle-layer/discussions/13 could be set-up easily that would be perfect (at least for EPICS and TANGO) but otherwise it's probably possible to setup a modified version of the ESRF twin for TANGO quite fast.

You are all very welcome to propose alternative (or more detailed) task lists. I think it would be more efficient to discuss this here in written form and only then we can meet an agree to a proposition.

[TeresiaOlsson]

The steering committee and the coordinators discussed today to put together a list of features to define the scope of the prototype. @gubaidulinvadim will call to a coordinators meeting where they will make this list.

Perhaps we can come back to this discussion after that when we can see what suggestion they come up with?

[gubaidulinvadim]

Dear maintainers,

At the last coordinator meeting, Jean-Luc Pons, Waheed Sulaimankhail, and I discussed the scope of the prototype use cases (on the request from the steering committee) that will allow a good demonstration of pyAML. Please let us know here if you have any feedback on the scope of the use cases. I've also put the relevant chapters of User interface specification and requirement specification documents.

pyAML 'devices'

• Sleeps and waits are handled on the level of pyAML devices, not in the measurements
• Independent access of devices to the simulator / digital twin/control system 
• Configuration of the simulator / digital twin/control system is not required to run another mode
• Binding for TANGO and EPICS. DOOCS is up to DESY colleagues to contribute if they wish, at the stage of a prototype
• Grouping of devices into families/arrays
  pyAML 'measurements'
• Measurements can be run either in hardware or in physics units
• Measurement and analysis of the measurement are separate and independent
• Metadata generated from the config file and pyAML devices
• Saving of metadata in a standardised format

Tune correction use case ([Section 8.1.1] User interface specification); [appendix A3.6](Requirement specification))

Tune correction method: only SVD-based
Required devices:
      - Quadrupole correctors
            - Magnet type
                  - Simple quadrupolar corrector
                  - (optional) combined function magnet
            - Magnet models
                  - Linear model
                  - A calibration curve is necessary
      - Tune monitor (horizontal, vertical,  possibility to extend to include longitudinal)

User interface
      - Choice of the response matrix file (or lattice-based calculation) + setting a golden value + errors if file not found
      - Correction to a specific value
      - Step correction of the tune

• Exceptions/alternative flows
        - Possibility to select a subset of quads
        - Exclusion of a quad corrector if not working properly
        - Stop if the tune detection is not working
  Orbit response matrix measurement use case ([Section 8.1.2](User interface speficiation); [appendix A3.6](Requirement specification))
  Devices required
        - H/V dipole correctors
        - RF corrector
        - BPMs -> only slow acquisition / no turn-by-turn options; (optional: offset and tilt for the BPMs)
  Orbit correction
• Only SVD-based single-step correction
        - No Tikhonov regularisation
        - No truncation (cleaning of unwanted mode in the corrector pattern eventually induced by FOFB)
        - In the case of measured ORM, the coupling part is cancelled. We assume that the machine optics is already well corrected.
  Devices required
        - H/V dipole correctors
        - RF corrector
        - BPMs -> only slow acquisition / no turn-by-turn options; (optional: offset and tilt for the BPMs)
• User interface
        -  Choice of the response matrix  (or lattice-based calculation)  + setting a golden value + errors if file not found
        - Normalisation of ORM and correction matrix calculation from the provided (calculated or measured) ORM
        -  BPM weighting
        -  H/V corrector and RF corrector weighting
        -  HCorrector sum control to 0 (no off energy orbit, no bump orbit)
• Exceptions/alternative flows
        - Possibility to select a subset of BPMs
        - Possibility to select a subset of H/V correctors
        - Exclusion of a corrector if it is not working properly

[TeresiaOlsson]

I think we first should focus on the hardware abstraction layer. For me the first, simple test to make is to be able to configure and create a single device and do the following with it:

• Read a value from the live machine, virtual accelerator and simulator
• Do it for both TANGO and EPICS using the same user interface
• Read it in either hardware or physics units depending on the choice of the user

Before we have a robust implementation of that which we have tested for different machines, I think it will be a waste of time to implement the application part of the use cases because it will likely have to rewritten anyway as long as the implementation of the hardware abstraction layer hasn't been settled and keeps changing.