“Grokipedia is a copy of Wikipedia but one where in each instance that Wikipedia disagrees with the richest man in the world, it’s ‘rectified’ so that it’s congruent with them.”
Tasks
Send Nellie a response on lead and keys – done
Vanessa xfer – done!
See if there is anything to pull from the garden
SBIRs
Did some clustering on the sentence embedding data and see what that looks like. It seems as though the lowest number of dimensions results in the best clustering. Not surprising, but good to know the curse of dimensionality intuition holds
Here’s a set of screenshots for each of the UMAP/HDBSCAN variations:
Write a class that reads in one scenario and then plots lines for each version. See how to animate dots that move along the lines.
This campaign has substantial implications for cybersecurity in the age of AI “agents”—systems that can be run autonomously for long periods of time and that complete complex tasks largely independent of human intervention. Agents are valuable for everyday work and productivity—but in the wrong hands, they can substantially increase the viability of large-scale cyberattacks.
Tasks
4:00 Meeting with Nellie – done
SBIRs
Slides! Done!
9:00 Sprint demos – done
3:00 Sprint planning – done
Got sentence-level embeddings done
Now I need to see how clustering looks. Definitely some different regions, though there may just be a big blob too.
Along the lines of last Thursday, I wonder if the layers of an LLM could help identify the text that is most useful for identifying a topic. In particular, I’m thinking of Jay Alamar’s work on using NNMF to visualize what’s going on in the layers of a model (Interfaces for Explaining Transformer Language Models)
Added this thought to the project documentation and tweaked the layout so there is now a “prompts and stories” appendix. Makes things read better.
Had some interesting thoughts about the embedding space results from yesterday. I want to look at how each variation of a particular scenario relates to the others within the scenario. That could be interesting and a way of showing the “probability cone” of LLM narratives.
The other thing to try is to do an embedding at the sentence level and see what that looks like. Since all the tools are in place and embedding is ludicrously inexpensive, this should be straightforward and affordable
Tasks
Bills – done
Chores – done
Dishes – done
Print and sign things this afternoon, maybe. Nope
Submitted ticket for broken BeliefSpaces email – fixed
There may be a GPT5.1? Need to check the available models
9:00 Standup – done
10:00 Ron’s meeting – done
3:00 SEG – done. Fast Matt apparently forgot. I need to read his notes later
4:00 ADS. Mention that I won’t be able to make it next week – canceled
UMAP! Working!
These are embeddings of 5 scenarios that should be in a roughly similar space. I’m a bit surprised that they don’t overlap. Probably need a lot more scenarios. I’ll make a few more and see how that changes things
OpenRouter is “the first LLM marketplace, OpenRouter has grown to become the largest and most popular AI gateway for developers. We eliminate vendor lock-in while offering better prices, higher uptime, and enterprise-grade reliability.” They have all kinds of interesting data about models they are serving (rankings), and piles of big-name and obscure models.
This article examines how digital historians can use large language models (LLMs) as research tools while critically assessing their limitations through source criticism of their underlying training data. Case studies of LLM performance on historical knowledge benchmarks, oral history transcriptions, and OCR corrections reveal how these technologies encode patterns of whose history has been digitised and made computationally legible. These variations in performance across linguistic and temporal domains reveal the uneven terrain of knowledge encoded within generative AI systems. By mapping this “jagged frontier” of AI capabilities, historians can evaluate LLMs not just as tools but as historical sources shaped by the scale and diversity of their training. The article concludes by examining how historians can develop new forms of source criticism to navigate generative AI’s uneven potential while contributing to broader debates about these technologies’ societal impact.
Tasks
Finish slides – scroll through notes for links – done
Check in and ping Sande – done
4:30 class – done!
SBIRs
Change df so that cluster id is a column and see if I can get that to work
That works nicely. Here’s the code that creates the df:
num_populations = 5
num_samples = 1000*num_populations
l = []
scalar = 5.0
for i in range(num_samples):
c = np.random.randint(0, num_populations)
d = {'cluster': f"c{c}", 'x':np.random.normal()+(float(c)-num_populations/2.0)*scalar, 'y': np.random.normal(), 'z':np.random.normal()}
l.append(d)
df = pd.DataFrame(l)
Large language models (LLMs) are increasingly used in the social sciences to simulate human behavior, based on the assumption that they can generate realistic, human-like text. Yet this assumption remains largely untested. Existing validation efforts rely heavily on human-judgment-based evaluations — testing whether humans can distinguish AI from human output — despite evidence that such judgments are blunt and unreliable. As a result, the field lacks robust tools for assessing the realism of LLM-generated text or for calibrating models to real-world data. This paper makes two contributions. First, we introduce a computational Turing test: a validation framework that integrates aggregate metrics (BERT-based detectability and semantic similarity) with interpretable linguistic features (stylistic markers and topical patterns) to assess how closely LLMs approximate human language within a given dataset. Second, we systematically compare nine open-weight LLMs across five calibration strategies — including fine-tuning, stylistic prompting, and context retrieval — benchmarking their ability to reproduce user interactions on X (formerly Twitter), Bluesky, and Reddit. Our findings challenge core assumptions in the literature. Even after calibration, LLM outputs remain clearly distinguishable from human text, particularly in affective tone and emotional expression. Instruction-tuned models underperform their base counterparts, and scaling up model size does not enhance human-likeness. Crucially, we identify a trade-off: optimizing for human-likeness often comes at the cost of semantic fidelity, and vice versa. These results provide a much-needed scalable framework for validation and calibration in LLM simulations — and offer a cautionary note about their current limitations in capturing human communication.
Tasks
Bills – done
Dishes – done
Chores – done
LLC stuff -skimmed and found a typo. Need to decide on text for “purpose.” I’m thinking of something along the lines of the development and application of ethical machine-learning and generative AI solutions, and to promote awareness of malicious and nefarious uses of these technologies. Sent draft to Aaron – done
In 1954, Dorothy Martin predicted an apocalyptic flood and promised her followers rescue by flying saucers. When neither arrived, she recanted, her group dissolved, and efforts to proselytize ceased. But When Prophecy Fails (1956), the now-canonical account of the event, claimed the opposite: that the group doubled down on its beliefs and began recruiting—evidence, the authors argued, of a new psychological mechanism, cognitive dissonance. Drawing on newly unsealed archival material, this article demonstrates that the book’s central claims are false, and that the authors knew they were false. The documents reveal that the group actively proselytized well before the prophecy failed and quickly abandoned their beliefs afterward. They also expose serious ethical violations by the researchers, including fabricated psychic messages, covert manipulation, and interference in a child welfare investigation. One coauthor, Henry Riecken, posed as a spiritual authority and later admitted he had “precipitated” the climactic events of the study.
For nine months, Sky News’ Data and Forensics team has been investigating whether X’s algorithm amplifies right-wing and extreme content. It does.Read our full methodology here.
Tasks
Water plants – done
Storage run – done
Looks like the first freeze will be next Monday night Tuesday morning. See what can be pulled in from the garden
Run the story generator for all variants. I realize that I want to try some political trajectories too, if the results for this look good. Also, because the number of walks will be low, this should be a 2D map at first. – done
Started on the extractor/embedder
LLM Agents
2:30 meeting – fun! Spent a lot of time talking about coffee. And I have a talk I need to give next Tuesday
Once pitched as dispassionate tools to answer your questions, A.I. chatbots are now programmed to reflect the biases of their creators.
Tasks
Ping Nellie – done
Fix fixee flat
Pay Michael – done
Pinged Aaron
Groceries – done
SBIRs
9:00 Standup
Assemble a list of ChatUnits as the story develops and see how that works – It works very well! Generated a bunch of stories and tweaked the prompts. Created two story methods – one for RAG and one control.
A new startup backed by one of the biggest venture capital firms in Silicon Valley, Andreessen Horowitz (a16z), is building a service that allows clients to “orchestrate actions on thousands of social accounts through both bulk content creation and deployment.” Essentially, the startup, called Doublespeed, is pitching an astroturfing AI-powered bot service, which is in clear violation of policies for all major social media platforms.
Tasks
Ping Nellie
Water plants – done
Fix fixee flat
Email Michael after tweaking the first refusal response – done
Ping Nathan and/or Edwins – done
SBIRs
Keep on working with the story prompting. I think I need to graduate the story objects to their own file as they are getting to be good sized.
Add binary saving and loading
Finish a full story
See if I can tweak the model to do less reasoning for these tests
Start playing around with getting embeddings and clustering. The goal will be to some good ratio of number of clusters to cluster size. Somehow, I think this is related to the value of the skip-gram acting as an attractor/repulsor. There is probably some kind of magic ratio that I can look for
In a democracy, the framework of constitutions, laws, and institutions provides structure, but substance only exists when citizens can trust that truth is tested, that their voices count, and that those in power can be held responsible for what they do. These three conditions, verification, deliberation, and accountability, form the structural minimum of democracy. They are not aspirational goals, but the foundations on which all other democratic values depend.
viztales 
You must be logged in to post a comment.