Pretrained models
The hidden layers of pretrained foundation models, such as ChatGPT, contain useful and abstract summaries of data. From an information-theoretic perspective, they might compress the data. From a machine learning perspective, they compute useful features of the data. From a statistics perspective, they might be sufficient statistics for a parameter of interest.
Probabilistic models