2.1   settingVersions¶

The settingVersions event is a Generic event that is to be implemented by every CSC. It contains three parameters: recommendedSettingsLabels, recommendedSettingsVersion and settingsUrl. This event must be published by all CSCs upon entering the STANDBY state and broadcast information about “recommended” settings (e.g. configurations). The combination of recommendedSettingsLabels and recommendedSettingsVersion allow users to uniquely identify a set of configurations. A name/version setting must always result in the same configuration parameters. Any changes in the configuration repository or database should produce a new version.

The settingsUrl attribute is a URL indicating how the CSC connects to its settings. It will start with “file:” if it is a clone of a git repo, or the standard URL, if a database.

The recommendedSettingsLabels contain a comma separated list of labels, each label maps to a configuration. The same label can point to different name/version pair over time. This information should be available in the CSC configuration repository or database and must match the value in recommendedSettingsLabels published to SAL. Labels must be human readable strings that clearly state the purpose of that configuration (e.g. current, nighttime, daytime). Labels should avoid having version numbers or dates in them. They are group classifiers and have some relative permanence. Transient labels with Jira ticket numbers may be used for developing new configurations. They should be moved to standard type labels at the earliest opportunity. The order of the labels is important, as the first label in the list will be the one selected by the high-level control system for any configurable CSC.

The recommendedSettingsVersion will be filled with the version information about the local configuration repository or database. For configurations stored in git repositories the following branch description[1] is used:

git describe --all --long --always --dirty --broken

The repository branch (or tag) name forms the first part of the branch description. It may take any form necessary to convey the appropriate information. They are individual identifiers and can change rapidly.

The configuration repository or database may contain any number of different configurations with different labels. Configurable CSCs must specify a list of recommended labels. How they implement this is up to the CSC. It should be noted that not all configurations need to be associated with a label. For instance, old configuration files that are still valid can be kept with the repository without a label. This will allow knowledgeable users to use them if needed.

At minimum, all configurable CSCs should pass at least one label in the recommendedSettingsLabels attribute, which can be explicitly referenced in the settingsToApply attribute of the start command. The CSC should understand how to use this label to retrieve the correct configuration. See caveats to this process below.

2.2   Settings Applied¶

The settingsApplied event is a Generic event that is to be implemented by every CSC. It currently contains two parameters: settingsVersion and otherSettingsEvent. This event should be published between the start command response starts to execute and before it finishes.

When the configuration is managed using git, settingsVersion will contain the SHA of the repository. For a database configuration settingsVersion will have TBD.

The otherSettingsEvents is a comma-separated list of other specific CSC configuration events. This may be blank if no other specific CSC events are necessary. If otherSettingsEvents is not blank, then those event(s) must be published by the CSC alongside the settingsApplied event. The CSC is allowed to publish as many events as necessary to convey the information.

2.3   Other Settings Applied events¶

Since it is not possible to provide a generic way for CSCs to output detailed information about the configuration parameters they are loading, it is recommended to create additional events which are particular to each CSC to carry that information.

Although it is not required, for clarity, we suggest that these events be preceded by settingsApplied followed by some description of the content, e.g., settingsAppliedLUT or settingsAppliedController.

3.1   Salobj Derived CSCs¶

Salobj is the framework provided by Telescope & Site to develop CSCs in Python. Extensive development documentation is available, especially on how to create configurable CSCs.

Components that are written using the framework will automatically inherit the standard behavior implemented in the library. The main points regarding Salobj CSCs are:

1. Definition of the configuration repo. In general CSC configuration should be grouped according to the overall system architecture. For instance, ts_config_attcs hosts configurations for all the ATCS configurable components.
2. The configuration package is specified in the CSC code by overriding the method get_config_pkg.
3. The CSC defines a schema for its configuration, which lives with the CSC repository.

The configuration for a CSC is stored in the configuration repository in a directory with the same name as the CSC, e.g. ATAOS in ts_config_attcs stores the configuration files for the ATAOS CSC.

The first level inside a CSC configuration package will have the schema version, e.g., ATAOS/v1 and ATAOS/v2.

Inside a schema version the user can find the available configurations and a labels file. The labels will provide the mapping between the recommendedSettingsLabels and the configuration.

Note that some configuration files are not linked to any label. They can be either removed from the most recent version of the configuration or kept there for historical or testing purposes. Since the repository setup is published by the CSC in the settingVersions event, the user can aways go back to a set of configurations.

3.2   Camera CSCs¶

These CSCs will also specify a set of labels to recommendedSettingsLabels. A given label will point to N available versions that will be published via recommendedSettingsVersion. As an example, if a label called normal is present, that label may be present as the following versions: normal-1.1, normal-1.2, normal-2.0, normal-3.0.

3.3   Other Handcrafted CSCs¶

Unfortunately, not all CSCs provided by Telescope and Site are developed with a framework like Salobj that handles most of the system architecture details. Some CSCs where developed by external vendors which did not have a framework to work with at the time the contract started. In other cases the CSC was developed in-house using a different programming language due to performance requirements.

In these “handcrafted CSCs” the developer is in charge of constructing their own solution to the problem. Here we gather some information about those CSCs.

3.4   Non-Configurable CSCs¶

Some CSCs will not be configurable at all. Examples are sparse in our current architecture but, the from Salobj point of view, a CSC can be developed on top of a BaseCSC which makes it a non-configurable component.

A non-configurable CSC will ignore the settingsToApply attribute of the start command, as it does not contain any true meaning to it. Likewise these CSCs will not output any of the configuration-related events.

As can be seen from previous sections, most of the handcrafted CSCs written in C++ or LabVIEW are not “Configurable CSCs”, in the sense that they either ignore the settingsToApply value on the start command or does not output all the appropriate events.

3.5   Examples¶

The most simple (and probably most common) case is for those where the CSC has only a single recommended setting. For example, for the ATDome CSC we have:

recommendedSettingsLabels: test
recommendedSettingsVersion: v0.3.0-0-g6fbe3c7
settingsUrl: file:///home/saluser/repos/ts_config_attcs/ATDome/v1

Some CSCs may also have multiple recommended settings, one of them being the preferred or default and another being secondary and so on. In this case, the purpose of those configurations should be spelled out. As an example, the ATAOS has a couple of available options for look-up tables. In this case, we may have something like:

recommendedSettingsLabels: current,constant_hex,high_degree
recommendedSettingsVersion: v0.3.0-0-g6fbe3c7
settingsUrl: file:///home/saluser/repos/ts_config_attcs/ATAOS/v2

Note how the recommendedSettingsVersion from both CSCs have the same value. Both configurations reside in the same repository: ts_config_attcs.

Imagine now that during a test run, someone connects to the computer running the ATAOS CSC and edits the configuration. The recommendedSettingsVersion would reflect that change with something like:

recommendedSettingsVersion: v0.3.0-0-g6fbe3c7-dirty

Even though it may be useful to edit configurations on the fly for testing, the process should be avoided as much as possible. When this happen, it prevents us from precisely identifying what configuration was used. Alternatively, the user could create a branch on their work machine, make the required changes, commit, push it to GitHub and pull/check out the new configuration in the CSC machine.

For a CSC that uses a configuration database, like the ATCamera, we may have something like:

recommendedSettingsLabels: normal,highgain_fast,lowgain_fast,highgain_slow,lowgain_slow
recommendedSettingsVersion: 1.1,1.2,2.0,3.0
settingsUrl:  sqlite:///home/camuser/config/config.db

It might be the case where the configuration is hosted in a sql database which enables remote connection. Is this case, we could have something like:

settingsUrl: mysql://10.0.100.104:3306/CONFIG

4.1   Topics/attributes renaming¶

The interface would be much clear if we rename some generic topics and attributes to reflect more closely their true meaning.

For instance, one of the things to point out is the use of words like “recommended” and “settings” in attributes that are related to configuration information. Users will usually count on being able to easily enable a component with appropriate defaults first and then, what different configurations they have available to fine tune the behavior of the system. The use of recommended gives the impression that not everything that is shown is what is available (which is true in some cases), and also means users must look into the configuration repository to know what else is available. On the other hand settings really seems like a misnomer for configuration.

The following is a renaming suggestion for discussion:

1. Rename settingsVersions to configurationsAvailable.

   This topic presents all the available configurations that can be loaded by the CSC (see the proposal to change the way CSC handles configuration).

   1. Rename recommendedSettingsLabels to labels.
   2. Rename recommendedSettingsVersion to versions.
   3. Rename settingsUrl to url.
   4. Add mapping. For a configuration repository mapping will reflect the content of the _labels.yaml file. For a configuration database mapping will explicitly show the mapping of label to label:version.

2. Rename settingsApplied to configurationApplied

   1. Add label.
   2. Add version.
   3. Add mapping.
   4. Add url.
   5. Rename otherSettingsEvent -> otherInfo.

   The event will publish the selected values once the CSC is configured.

3. In the start command, rename attribute settingsToApply to configuration.

4.2   Continuous monitoring configuration repository¶

Right now CSCs are required to publish configurationsAvailable (former settingsVersions, see renaming proposal) when they transition to STANDBY state. Nevertheless, while in STANDBY state it is possible for someone to update the available configuration, which would make the information out of sync. We propose that, while in STANDBY state, CSCs continuously monitor the configuration repository and update the information if needed. If an invalid configuration is made available while the CSC is in STANDBY the CSC should transition to FAULT state and publish an appropriate error message saying which file caused the issue and how to resolve it. This monitoring should only happen while the CSC is in STANDBY and should not interfere with any other state. For instance, when transitioning from DISABLE to STANDBY, the CSC shall not start monitoring until the transition is completed and the command acknowledged.

4.3   Base configuration (handling default configuration values)¶

This is mainly a proposal to update how Salobj manages default configuration values. Other handcrafted CSCs are encouraged to follow this proposal as closely as possible to maintain uniformity across the system.

As described above, CSCs written with Salobj define a configuration schema (e.g. ts_atdome). The configuration schema contains default values for the configuration which are loaded if the start command is sent with an empty configuration attribute (the default value). Nevertheless, the values in the schema are seldom valid beyond a unit testing environment, which requires users to provide some kind of operational defaults or default label. One can see how this can cause confusion when operating the system since now “default” has two different meanings, e.g.; schema default and operational default. Furthermore, it is usually enough to override a small subset of the schema defaults for operations. Therefore, to get a full set of applied configurations, users must look at two distinct repositories; the configuration repository (for the modified parameters) and the CSC repository (for the schema defaults).

The proposal to improve this aspect of the system is:

1. Remove default values from the configuration schema.
2. On the configuration repository there must be a _base.yaml file defining all the base configuration values (we use “base” instead of “default”).
3. The labels file (e.g. _labels.yaml) will continue to exist with the same format and purpose.
4. Additional configuration files can provide new values for individual configuration parameters.
5. If a CSC receives a start command with an empty configuration (see renaming proposal) attribute, it will load the values in base.yaml.
6. If a CSC receives a start command with a configuration attribute equal to a label in _labels.yaml, it will load the values in base.yaml first and override those values defined in the mapped configuration file.
7. The name default should not be used for labels.
8. All valid configurations must have a label to be loaded by the CSC. Configurations will not be loaded by filename.
9. The configuration repository should be kept clean of configurations used for unit testing or for different purposes (e.g. test stand configurations). Configurations tailored for test stand can be kept in different branches/tags. Configurations needed for unit testing should be added to the test directory in the CSC repository and use the override feature in CSCs (see Salobj documentation).
10. All configuration files should have a header explaining that they are loading basic values from _base.yaml.

5.1   LSE-60¶

Requirement TLS-REQ-0065, in section 2.8.1.3 from the Telescope & Site Subsystem Requirements [4] states that:

The Telescope and Site shall publish telemetry using the Observatory specified protocol (Document-2233) containing time stamped structures of all command-response pairs and all technical data streams including hardware health, and status information. The telemetry shall include all required information (metadata) needed for the scientific analysis of the survey data as well as, at a minimum, the following: Changes in the internal state of the system, Health and status of operating systems, and Temperature, rate, pressure, loads, status, and conditions at all sensed system components.

This is a broad requirement specifying that components must publish operational status information.

5.2   LSE-62¶

The LSST Observatory Control System Requirements Document [3] contains three requirements regarding system configuration:

Requirement OCS-REQ-0045 in section 3.4.4 (Subsystem Latest Configuration) states that:

Specification: The Configuration Database shall manage the latest configuration for each subsystem, for the different observing modes.

Discussion: The Configuration Database maintains also the latest configuration utilized during operations that can be utilized for rapid restoration of service in case of failure.

Requirement OCS-REQ-0069 in section 3.4.4.1 (Subsystem Parameters) state that:

Specification: The Configuration Database shall manage the subsystem parameters for the different observing modes.

Requirement OCS-REQ-0070 in section 3.4.4.2 (Subsystem History) state that:

Specification: The Configuration Database shall manage subsystem history for the different observing modes.

See furthermore details about the adopted definition of “configuration database” in the context of the control software architecture and more details about the proposed implementation.

5.3   LSE-150¶

Section 2.4 of the LSST Control Software Architecture [2] describes how to perform configuration management. The document provides two valid alternatives for managing configuration in the LSST system; through a configuration database or version control system.

For a configuration database, any solution is acceptable as long as the technology allows versioning of the database.

For version control systems the adopted solution is git. The document also specifies that configurations must be stored in a separate repository from that of the component source code, to allow the configuration to evolve independently of the main code base. The configuration for different components can be stored individually or in groups of components to facilitate maintainance.

References