Evolution of the Configuration mechanism

[chaen]

As of now, the Config object is generated from a single default.yaml file in a git repository. The code is generic enough to allow for a different backend. This object also handles the caching at the server and client level.

As of now, the /config endpoint used by the client returns everything contains in the default.yaml (although augmented a bit, for example in the Operations section which is duplicated by VO).

---

The caching mechanism used here could be reused for other purposes, like the RSS status, and they could be served by the /config router.

A first step would be to split the config endpoint like this:

/config -> serve what it does now
# Clients do not need anything else that the storage status (for example Compute) 
/config/resource-status/storage?vo={inferred_from_credentials}  -> dict[se_name, StorageElementStatus]

The vo can be inferred from the token used to talk to the service.

From the code point of view, split server-side ConfigSource into

• CacheableSource: for data that can be cached by clients
• ConfigSource: Subclass of CacheableSource that interacts with git
• ResourceStatusSource: Subclass of CacheableSource that interacts with the resource status database

class ResourceStatusSource(CacheableSource):
    def get(self, resource_type: ResourceType, vo: str) -> dict[str, StorageElementStatus]:

Update from @aldbr: done in #850

---

We may also think about implementing the following endpoint, to make sure a client from a given VO only see the configuration relevant to them.

/config/client?vo={inferred_from_credentials}  # Checks client identity

The whole /config endpoint could eventually be vo dependent, as there's no usecase of a client seeing the full configuration.

---

Another needed feature is the possibility to have extra configuration file in the repository, for example tasks.yaml which are used only by tasks. We would have a TaskConfig object, behaving just like the current Config object

[fstagni]

The whole /config endpoint could eventually be vo dependent, as there's no usecase of a client seeing the full configuration.

What about the admin VO?

Another needed feature is the possibility to have extra configuration file in the repository, for example tasks.yaml which are used only by tasks. We would have a TaskConfig object, behaving just like the current Config object

I would first like to see the tasks plan of work :) before determining if this is an actual requirement.

[chrisburr]

What about the admin VO?

The admin vo can have the ability to request each VO's config one-by-one if it needs this information. It's a niche case so it doesn't matter if it has the overhead of N calls.