Add an Inventories class for loading data, similar to the Wordlist class

[LinguList]

The Wordlist class is peculiar, since it allows to merge several CLDF datasets, since wordlists are typically offered not for large cross-linguistic collections but rather for smaller data for one language family and the like.

The Inventories class would therefore not need to merge several CLDF datasets, but rather load a datasets like Phoible, or Saphon and then offer direct access to the Language.sound_inventory methods.

[xrotwang]

Hm. Without the aggregation of multiple CLDF datasets, there isn't too much added on top of pycldf.orm, I think.

[LinguList]

Well, in this case, it would be useful to show how one can use the code in cltoolkit to compute the features that can be computed in the phonology section of cltoolkit for all languages in phoible on the one hand and for all languages in saphon on the other hand (I select them because one uses contributions and one not).

The code would require:

1. to load the data
2. to assemble the sound values per language variety (work with contributions in phoible, work with values directly in saphon)
3. to create language objects and assemble them in some kind of a DictTuple
4. to compute the features for the language objects

This code, I think could be handled by a very small class:

from cltoolkit import Inventories

inv = Inventories.from_dataset("cldfl/Structure-Dataset-metadata.json", contributions=False)
for language in inv:
    print(language, len(language.sound_inventory))

[LinguList]

Sorry, @xrotwang, I do not know if I read you note correctly: did you want to say that "since there is not much to be added, it is not needed" (as I read it first, which is why I then brought up my argument to add it) or "it is easy, and won't be difficult to do"?

[LinguList]

One more thought that may be important here: we have mini-repos, like allenbai and liusinitic, and also zhoubizic, where I started to routinely add phoneme inventories in case they are provided along with the data. So the multi-data-usecase seems in fact to be valid, since they all represent different language families. One could also routinely load phoible and saphon into the same database and later make a comparison of different datasets, as we did in the study with Cormac, with one Inventories instance.

[xrotwang]

To be honest, I'm still chewing on the Wordlist class, trying to make up my mind about all the data denormalization happening there (with everything available from everywhere). To me that looks like a re-invention of SQLAlchemy's ORM (with back-references, etc.) - just with a lot less testing (and with less architecture, e.g. no explicit identity map).

So I think I'm not ready to jump right into implementing the next class like this, before we have some experience with the wordlist - beyond the experience of convenience :)

[LinguList]

Well, from my perspective, the importance of having these classes is that they allow for the kind of analyses with features and the like which were not possible before, since it took me so much time to make up my mind of how to actually load a class in CLDF and which packages to load, how to represent them, and the like. So for a developmental perspective, these classes are indispensable for me, as I feel really that a larger part of my energy flows into custom constructs of how to load phoible data here, how to load saphon data there, instead of loading a larger kind of datasets once and then being able to play around with the data.

It is again a post-lexibank issue, nothing one needs to discuss now in detail, but from my experience with cltoolkit Wordlist and how it helped me to rapidly start programming these rather simple lines that allow us to model various kinds of features, I think on the long run, it will be important to have facilitated access to the data where a lot of the checking is already done and one (this "one" being myself mostly, I assume) can start to develop new methods to work with the checked data.

[LinguList]

But I think there is no direct conflict here, as I don't need to have this implemented now, and I will just experiment with the implementation of such a class for inventories at some later point myself and can wait until we have more experience with the usefulness of the wordlist class.

[xrotwang]

I understand the idea - and I agree that this will make a lot of things possible. I just want to make sure we don't paint us into a corner - or if we do, we do it with less investment :)

As I said, this looks a lot like SQLAlchemy ORM, so at some point soon, I'd like to look into implementing this via SQLAlchemy ORM on top of data in SQLite. If nothing elase, this will also give us access to all the remaining data in the datasets in a unified way.