Fabian Offert, in his article “Are Nearest Neighbors Good Neighbors?”, talks about similarity as a way to understand how AI models work with images. Offert's main argument is that AI systems cannot look for similarities between artworks in the same way that we do. He uses as an example for comparison throughout his article the Atlas of Aby Warburg (Bilderatlas Mnemosyne), noting how with Warburg's groupings of works of art throughout history, it is sometimes straightforward to identify the similarities (the logic he was using to relate the works), but with some groupings the reasoning is more subtle, requiring some knowledge of the works or the period, or some guessing (so open to interpretation). The key for Offert seems to be that there isn't a clear rule that we can follow to replicate the grouping of images with some new images of our choice.
In Warburg's themes, sometimes we can see that the matches are formal (compositional layout, painting style, colour scheme, etc), while in others it seems to be the content or meaning of the works that attracted them together, and yet in others, the intent of the artists might be the link. And surely this is part of the attraction of the Atlas, because if the matches were obvious to us every time, their interest would be little. There is also the openness of it that I touched on: it allows for different interpretations or at least a multitude of possible thoughts that can spark in each viewer when looking at the panels.
In contrast to the Altas, Offert describes how latent spaces (encodings) of AI models (diffusion models, I believe he is talking of), all images are related. This, from a purely mathematical perspective is true, as he also states: a vector space is defined by the possibility of arriving at any vector in it from combinations of the basis vectors. But why are all images related? We can create independent vectors in different spaces that cannot be related. The reason why those image-vectors live in the same vector space, is that they share their attributes, and those come from their physical visual properties (either pixel descriptions or their statistical equivalent in encoded space). That same reality applies to Warburg's images of course: we could scan them and embedded in an already trained model we happen to work with, and the images of the Atlas will now be related to all the other images in the model training dataset.
We need to differentiate between different models that are used in AI image generation: the diffusion model itself only uses a visual model, it learns about colour, grain, lines, shapes, etc. But all image-generation models, to be usable, need (at least) a second model (in reality it can all be the same model), an LLM/CLIP model that translates images to words and words to images. So we have a formal/syntactic space and a semantic space, and we combine them to generate the images we want. This also means that if we say that all images are related in embedding/latent space, we need to think about both syntax (form) and semantics (meaning relationships.
Back to the Atlas for a moment: why did Warburg chose those images and not others? Clearly there must exist many other possibilities for each individual image in a group that would fit the relationship. Any other person would probably have made a different selection, so we can agree that the curation is based on some parameters, subjective and objective, rational or unconscious, but there are clearly some boundaries to the choices made that separate them from other possible choices.
In the world of AI models, we need to define any such boundaries or preferences in very clear terms, and even if we do so in natural language (e.g.: select images that convey a feeling of sublime), this is translated into numbers in order to then translate it into images. Even subjective and open-ended parameters are translated into hard boundaries, i.e., numbers.
So if there are parameters bounding the choices in both human selection and machine selection, where is the difference? It seems to be in the way we see them: we don't try to turn Warburg's parameters for his selection into numbers, because we all probably realise without even test it, that the effort would be helpless. The magic of the Atlas is in its openness, its multiple possible readings and the variety, the way some selections feel obvious while others we might contest and yet others feel puzzled about. We might think as well that this is part of what makes the Atlas the work of one person and not some kind of survey: it is subjective because we want it to be subjective. There is little point in gathering together the works that on average are selected by a large number of people as being representative of the sublime. The value is in being a subjective, partly explainable and partly unconscious, choice by one "expert judge" about whom's opinion we are interested.
So yes, the Atlas is not based on hard parameters that define which images are in one group and which are not. Warburg made his choices subjectively, but that subjectivity could well be put into numbers, or at least written down in a way that follows logical rules. After all, there must be a reason for one choice and not another, whether or not Warburg let us know about it, or knows himself about it. Would it make sense to do it? No, it wouldn't, because the magic of it is in the subjective unknowns that make the curation both subject and object: it defines itself as a thing but it is also open to our interpretation of it.