Being Bayesian about Large Language Models to Address Truthfulness

Several explanations for hallucination exist, but perhaps the main reason is that they are trained to be plausible.  There is no element of truth-seeking in their construction.  The training content can be filtered to support this, but in many areas truth is not established.   Many text sources are opinions, may contain argumentation and indeed subtle or not so subtle propaganda, written in many dfferent styles, and some reflect misinformed views.   Even Wikipedia sources are known to have bias (so called "establishment bias").  LLMs reproduce this mess, with a sycophancy excacerbating lack of truthtfulness.  Moreover, the training and theory of LLMs has no notion of epistemic uncertainty.  Recently AI researchers pushing world models address this problem and one alteBeinrnative is training objectives like JEPA focusing on semantic predictability rather than prediction of sensory input.  However, this does not address the handling of opinions.  We should attempt to understand sources and their viewpoint biases/opinions and support epistemic uncertainty.  Can we disentangle the biases/opinions of a diversity of sources?  We could adapt Bayesian meta-reasoning and work with the LLM in an agentic manner, or we could adapt alternative forms of LLMs like JEPA.