Run “The Swim” through the GPT as a New Yorker editor and get some feedback.

SBIRs

• 9:00 Standup
• 2:00 BMD SBIR Meeting
• 2:30 AI ethics?
• Going to continue reading the papers I found yesterday.
• Eliciting Latent Predictions from Transformers with the Tuned Lens
  • We analyze transformers from the perspective of iterative inference, seeking to understand how model predictions are refined layer by layer. To do so, we train an affine probe for each block in a frozen pretrained model, making it possible to decode every hidden state into a distribution over the vocabulary. Our method, the tuned lens, is a refinement of the earlier “logit lens” technique, which yielded useful insights but is often brittle.
    We test our method on various autoregressive language models with up to 20B parameters, showing it to be more predictive, reliable and unbiased than the logit lens. With causal experiments, we show the tuned lens uses similar features to the model itself. We also find the trajectory of latent predictions can be used to detect malicious inputs with high accuracy. All code needed to reproduce our results can be found at this https URL.
• Toy Models of Superposition
  • Neural networks often pack many unrelated concepts into a single neuron – a puzzling phenomenon known as ‘polysemanticity’ which makes interpretability much more challenging. This paper provides a toy model where polysemanticity can be fully understood, arising as a result of models storing additional sparse features in “superposition.” We demonstrate the existence of a phase change, a surprising connection to the geometry of uniform polytopes, and evidence of a link to adversarial examples. We also discuss potential implications for mechanistic interpretability.

GPT Agents

• At 54 responses, so yay! If nothing comes in today, I’m going to download the csv files, put them in Box, and start creating graphs.
• Create a project that stores the files, loads them into the DB, and produces excel files. Use private SVN repo