Along the lines of last Thursday, I wonder if the layers of an LLM could help identify the text that is most useful for identifying a topic. In particular, I’m thinking of Jay Alamar’s work on using NNMF to visualize what’s going on in the layers of a model (Interfaces for Explaining Transformer Language Models)

Added this thought to the project documentation and tweaked the layout so there is now a “prompts and stories” appendix. Makes things read better.