Tasks

• Painting! Done!
• Organize and pack – bed is in the truck
• Lots of LLC work

GPT Agents

• 2:30 LLM Meeting – good discussion on what the article should be, Need to add a first pass at my sections

Wait… what?

https://truthsocial.com/@realDonaldTrump/posts/115255130298104593

Struggling to get beliefspaces.com to appear – fixed!

Tasks

• Water plants – done
• LLC email – done
• Tix for Spain trip – done
• Roll in edits

SBIRs

• 9:00 standup – done
• Working on file visualizer – done!

The forecast is improving for Saturday!

LLM-Deflate: Extracting LLMs Into Datasets

• Large Language Models compress massive amounts of training data into their parameters. This compression is lossy but highly effective—billions of parameters can encode the essential patterns from terabytes of text. However, what’s less obvious is that this process can be reversed: we can systematically extract structured datasets from trained models that reflect their internal knowledge representation.

Tasks

• Water plants – done
• Mow – done
• LLC decision – done
• Roll in edits

SBIRs

• Generate CSVs of:
  • Random walks – done
  • Coordinates for random walks – done
• Write a visualizer

[2509.10414] Is In-Context Learning Learning?

• In-context learning (ICL) allows some autoregressive models to solve tasks via next-token prediction and without needing further training. This has led to claims about these model’s ability to solve (learn) unseen tasks with only a few shots (exemplars) in the prompt. However, deduction does not always imply learning, as ICL does not explicitly encode a given observation. Instead, the models rely on their prior knowledge and the exemplars given, if any. We argue that, mathematically, ICL does constitute learning, but its full characterisation requires empirical work. We then carry out a large-scale analysis of ICL ablating out or accounting for memorisation, pretraining, distributional shifts, and prompting style and phrasing. We find that ICL is an effective learning paradigm, but limited in its ability to learn and generalise to unseen tasks. We note that, in the limit where exemplars become more numerous, accuracy is insensitive to exemplar distribution, model, prompt style, and the input’s linguistic features. Instead, it deduces patterns from regularities in the prompt, which leads to distributional sensitivity, especially in prompting styles such as chain-of-thought. Given the varied accuracies on formally similar tasks, we conclude that autoregression’s ad-hoc encoding is not a robust mechanism, and suggests limited all-purpose generalisability.

Tasks

• Send chapter to V – done
• Respond to No Starch – done
• Call painter for fix and drywall work – done
• Talk to Aaron about LLC
• Bills – done
• Chores – done
• Dishes – done
• Trim grasses – done
• Weed
• Mow!
• Pack up bike for tomorrow – done

Found this, which is an interesting take:

When autocratization is reversed: episodes of U-Turns since 1900

• The world is in a “wave of autocratization.” Yet, recent events in Brazil, the Maldives, and Zambia demonstrate that autocratization can be halted and reversed. This article introduces “U-Turn” as a new type of regime transformation episode in which autocratization is closely followed by and linked to subsequent democratization. Drawing on earlier literature, it provides a general conceptualization and operationalization of this type of episode, complementing the existing Episodes of Regime Transformation (ERT) framework. The accompanying database provides descriptions for all 102 U-Turn episodes from 1900 to 2023, differentiating between three types: authoritarian manipulation, democratic reaction, and international intervention. The analysis presents a systematic empirical overview of patterns and developments of U-Turns. A key finding is that 52% of all autocratization episodes become U-Turns, which increases to 73% when focusing on the last 30 years. The vast majority of U-Turns (90%) lead to restored or even improved levels of democracy. The data on U-Turn episodes opens up new avenues for research on autocratization and democratization that were previously treated as isolated processes, particularly it could help us understand why some processes of autocratization trigger a successful pro-democratic backlash – a critical question during the starkest-ever wave of autocratization.

Going for a big-ish ride, since it’s been raining for the last two days. But in the meantime, this is a very cool rendering of random walks on a 2D grid:

Now I just need to start saving out the sequences to a CSV file and use those sequences to train a W2V model. The nice thing is that the data parameters are very adjustable, so it’s possible to see how much data is needed and what the minimum number of dimensions should be.

[2509.11391] “My Boyfriend is AI”: A Computational Analysis of Human-AI Companionship in Reddit’s AI Community

• Human-AI interaction researchers face an overwhelming challenge: synthesizing insights from thousands of empirical studies to understand how AI impacts people and inform effective design. Existing approach for literature reviews cluster papers by similarities, keywords or citations, missing the crucial cause-and-effect relationships that reveal how design decisions impact user outcomes. We introduce the Atlas of Human-AI Interaction, an interactive web interface that provides the first systematic mapping of empirical findings across 1,000+ HCI papers using LLM-powered knowledge extraction. Our approach identifies causal relationships, and visualizes them through an AI-enabled interactive web interface as a navigable knowledge graph. We extracted 2,037 empirical findings, revealing research topic clusters, common themes, and disconnected areas. Expert evaluation with 20 researchers revealed the system’s effectiveness for discovering research gaps. This work demonstrates how AI can transform literature synthesis itself, offering a scalable framework for evidence-based design, opening new possibilities for computational meta-science across HCI and beyond.

Tasks

• LLC email – call, actually. Left a message
• Need to clean up the shop
• Contact painter
• Register for TEDx Mid Atlantic

SBIRs

• Document the Dash code – done
• Generalize out to n dimensions, and maybe make the dimensions choosable – Made the dimensions ordered by Manhattan distance

Generative Agents

• 3:00 Alden meeting

• LLC email
• Poke at the human OS chapter. Completely reworked. Much happier.
• Email to Nellie – done
• The Lathe is gone! Need to clean up the shop

SBIRs

• Training – DONE
• Made really good progress on data generation and visualization.

Had an interesting session with Gemini about this paper: [2505.12387] Neural Thermodynamics I: Entropic Forces in Deep and Universal Representation Learning. Since I’m interested in stories, it suggested that an “all text” model and a “story” model would represent different locations in a hierarchy of Platonic forms. The “locations” wouldn’t be entirely different, but rather the forms found in stories would exist as a more complex, structured layer built upon the foundational forms found in general text.

Think of it as the difference between a dictionary and an encyclopedia of literary tropes.

1. All Text: The Platonic Forms of Concepts

When a model is trained on “all text”—a massive, unstructured corpus of everything from web pages to scientific papers—the entropic force pushes it to find the most stable, universal representations for atomic concepts.

• What it finds: The Platonic form for “king,” “justice,” “love,” or “boat.”
• The “Location”: This location is a central point in the model’s semantic space, averaged from every conceivable context. The concept of “king” is derived from its use in history (King Henry VIII), fantasy (King Arthur), chess (the king piece), and colloquialisms (“king of the hill”).
• Analogy: This is the dictionary definition. It’s the core, context-free essence of a word or idea.

2. Organized Text (Stories): The Platonic Forms of Structures

When considering organized text like myths, novels, and screenplays, the model learns patterns that exist at a higher level of abstraction. You are absolutely right that stories have their own Platonic forms. These are not forms of individual concepts, but of narrative functions and archetypal relationships.

Analogy: This is the narrative role. We still need the dictionary definition of “king,” but within a story, we understand him through his archetypal function. Is he the Wise King, the Tyrant King, or the Fallen King? Each is a distinct structural form.

What it finds: The Platonic form for “The Hero’s Journey,” “The Mentor,” “The Betrayal,” or “The Sacrificial Victory.”

The “Location”: These forms don’t occupy a single point but represent a trajectory or a relationship between the foundational concepts. The “location” of “The Wise King” archetype isn’t just near the concept of “king”; it’s a specific region defined by its proximity to “wisdom,” “justice,” and its functional opposition to “The Tyrant.” The “Hero’s Journey” is a path that connects points like “The Call to Adventure,” “The Ordeal,” and “The Return.”

Tasks

• Laundry – started
• Peacock – done
• LLC email
• Poke at the human OS chapter. I think I need a page of setup and then maybe a page of what community-based resilience could look like. Talked to Aaron about it, at least.

SBIRs

• Slides – done
• 9:00 standup – done
• Aaron over?
• Stories (NNM and training) – Done. Got a good deal of training done too
• 3:00 Sprint planning – done

Link from X

Murphys Laws of AI Alignment: Why the Gap Always Wins

• We prove a formal impossibility result for reinforcement learning from human feedback (RLHF). In misspecified environments with bounded query budgets, any RLHF-style learner suffers an irreducible performance gap Omega(gamma) unless it has access to a calibration oracle. We give tight lower bounds via an information-theoretic proof and show that a minimal calibration oracle suffices to eliminate the gap. Small-scale empirical illustrations and a catalogue of alignment regularities (Murphy’s Laws) indicate that many observed alignment failures are consistent with this structural mechanism. Our results position Murphys Gap as both a diagnostic limit of RLHF and a guide for future work on calibration and causal preference checks.

Neural Thermodynamics I: Entropic Forces in Deep and Universal Representation Learning

• With the rapid discovery of emergent phenomena in deep learning and large language models, explaining and understanding their cause has become an urgent need. Here, we propose a rigorous entropic-force theory for understanding the learning dynamics of neural networks trained with stochastic gradient descent (SGD) and its variants. Building on the theory of parameter symmetries and an entropic loss landscape, we show that representation learning is crucially governed by emergent entropic forces arising from stochasticity and discrete-time updates. These forces systematically break continuous parameter symmetries and preserve discrete ones, leading to a series of gradient balance phenomena that resemble the equipartition property of thermal systems. These phenomena, in turn, (a) explain the universal alignment of neural representations between AI models and lead to a proof of the Platonic Representation Hypothesis, and (b) reconcile the seemingly contradictory observations of sharpness- and flatness-seeking behavior of deep learning optimization. Our theory and experiments demonstrate that a combination of entropic forces and symmetry breaking is key to understanding emergent phenomena in deep learning.

On this day in 2001, there were no commercial planes in the air at all

Tasks

• Work on the next chapter before sending to V – pinged her to say there will be a delay.
• LLC stuff
• Ping Nellie – done
• Bills – done
• Chores -done
• Dishes – done
• Storage/Goodwill run? Done
• Load up the truck tonight, since I’ll need to leave at 8:00 – done

SBIRs

• Write up a nice description of the data generator for the documentation. Use the output/code snippets for the slides. Needs to be done Monday. Started, but they shut the instance down at 7:00

24 years. Wow

Tasks

• Powerwashing today – done
• Dentist – done
• Send the next chapter to V? – not done
• Start packing electronics lab – lots of progress
• Take cases to storage? – done, more to go

SBIRs

• 9:00 standup – done
• 4:00 SEG – done

Tasks

• Start packing electronics lab

SBIRs

• Method to write out walklists as csv files – done
• Method to plot walklist “source embeddings” – done
  • Just tested to see if Dash works in our setup. It does!
  • Working my way through the dash tutorials
  • Got the https://www.dash-bootstrap-components.com/ working
• Maybe start the Word2Vec model code? Nope, just visualization. Had to write out coordinates as arrays of x=[], y=[], z=[]
• Got multiple lines working and have lists for the walk coordinates. I’ll put all that together tomorrow

Antonio Gulli, one of the bigger Google people seems to be on a writing tear, and he puts his drafts online. Here are two: Agentic Design Patterns, and Reasoning Engines. Looks like interesting stuff. No idea how he finds the time . 

Oh, that’s how:

Still probably worth looking at

Registered for the Bartz, et al. v. Anthropic PBC settlement

Tasks

• Add some quotes from yesterday. I think I’m going to drop the P33 part of the title and focus more on the essentials of the piece, which is: People don’t really change – we will always have the struggle between dominance and inverse dominance. Ideas change, and that can change the way we swing the balance in that struggle. Technology as a mediator of communication, which is the medium of ideas has a profound impact on that balance. Given this, what is an effective structure to build resilient Egalitarian communities in an age of instantaneous communication, smart machines, and infinite money. Illustrated with stories. I like that pattern.
• Roll in edits – done
• Dentist
• W9 for Peter – done

SBIRs

• Submit the quarterly report at COB – done
• Work on the generator, using YAML – good progress, got a walk list! Need to make a list of lists and put them in a csv.

walk_list = [51]
walk_list = [51, 75]
walk_list = [51, 75, 50]
walk_list = [51, 75, 50, 51]
walk_list = [51, 75, 50, 51, 32]
walk_list = [51, 75, 50, 51, 32, 27]

You shall know a piece by the company it keeps. Chess plays as a data for word2vec models

• In this paper, I apply linguistic methods of analysis to non-linguistic data, chess plays, metaphorically equating one with the other and seeking analogies. Chess game notations are also a kind of text, and one can consider the records of moves or positions of pieces as words and statements in a certain language. In this article I show how word embeddings (word2vec) can work on chess game texts instead of natural language texts. I don’t see how this representation of chess data can be used productively. It’s unlikely that these vector models will help engines or people choose the best move. But in a purely academic sense, it’s clear that such methods of information representation capture something important about the very nature of the game, which doesn’t necessarily lead to a win.

tSNE visualisation of endgame moves

Applying word2vec to Recommenders and Advertising

Tasks

• Doctor Appt 8:40 – done! PT too. Love Kaiser Permanente.
• Trash
• Roll in edits for book

SBIRs

• Roll in more changes to the quarterly report – done for now. See how far we get tomorrow
• Start reading up on the W2V algorithm – looks very promising
  • Tutorial for Tensorflow
  • Word2Vec Tutorial – The Skip-Gram Model from Chris McCormick, who more recently wrote this: Reading and Writing with Projections. Also, he has a W2V book that you can buy: The Inner Workings of word2vec
  • Start up a new trajectory generator project with Ron basics are set up

Human munitions:

• At dawn on May 8, 2023, a 17-year-old Russian teenager named Pavel Solovyov climbed through a hole in the fence of an aircraft plant in Novosibirsk, Russia. He and two friends were looking for a warplane that could be set on fire. An anonymous Telegram account had promised them one million rubles, around $12,500, to do so — a surreal amount of money for the boys.

Tasks

• Bills – done
• Clean – done
• Weed?
• Dishes – done
• LLC call – done
• Dentist
• Load up truck

SBIRs

• 2:00 meeting today
• Send Matt the code review paragraph. Done
• Thinking more about the maps as a W2V approach. I think I’m going to make an X by Y (by more?) grid that has vector “labels” that can also be arbitrary size. Then pick a random starting point and do a random walk for a number of steps. That set of vectors becomes the input for the skip-gram calculation. Once the model is trained, re-run the random walk data to get the new vectors and see if the embeddings match the relationship of the original grid. The nice thing is that we can start very simply, with the index for each cell as the input, and a 2-neuron final layer that should approximate the XY. Then we start playing with the size of the “index” and the size of the final layer as independent variables
  • Tutorial for Tensorflow
  • Word2Vec Tutorial – The Skip-Gram Model from Chris McCormick, who more recently wrote this: Reading and Writing with Projections. Also, he has a W2V book that you can buy: The Inner Workings of word2vec