Got the lawn done! Keeping the grass setting high so the grass stays healthy through the heat wave

Dropped off books at Book Thing. Next time, bring coffee for the workers. Sat, Jul 12

Following news on social media boosts knowledge, belief accuracy and trust | Nature Human Behaviour

• Many worry that news on social media leaves people uninformed or even misinformed. Here we conducted a preregistered two-wave online field experiment in France and Germany (N = 3,395) to estimate the effect of following the news on Instagram and WhatsApp. Participants were asked to follow two accounts for 2 weeks and activate the notifications. In the treatment condition, the accounts were those of news organizations, while in the control condition they covered cooking, cinema or art. The treatment enhanced current affairs knowledge, participants’ ability to discern true from false news stories and awareness of true news stories, as well as trust in the news. The treatment had no significant effects on feelings of being informed, political efficacy, affective polarization and interest in news or politics. These results suggest that, while some forms of social media use are harmful, others are beneficial and can be leveraged to foster a well-informed society.

Tasks

• Ping Carlos – done
• SINTEF has a neat position
• Ping Daniel Thilo Schroeder about How Malicious AI Swarms Can Threaten Democracy
• Help Barbara with her rugs? – done
• Print tix! – done
• Movie & Dinner with Suz & Viki – done

Chores

• Bills – done
• Dishes – done
• Cleaning – done
• Lawn – rained. Maybe tomorrow or Sunday?
• WEED – started. There are a LOT of weeds

SBIRs

• Write up notes from yesterday’s meeting, in particular the SYNAPSE->socket module, sizing & timing for a “hello world” and that I put together Matt’s Overleaf

Computer-vision research powers surveillance technology

• An increasing number of scholars, policymakers and grassroots communities argue that artificial intelligence (AI) research—and computer-vision research in particular—has become the primary source for developing and powering mass surveillance^{1,2,3,4,5,6,7}. Yet, the pathways from computer vision to surveillance continue to be contentious. Here we present an empirical account of the nature and extent of the surveillance AI pipeline, showing extensive evidence of the close relationship between the field of computer vision and surveillance. Through an analysis of computer-vision research papers and citing patents, we found that most of these documents enable the targeting of human bodies and body parts. Comparing the 1990s to the 2010s, we observed a fivefold increase in the number of these computer-vision papers linked to downstream surveillance-enabling patents. Additionally, our findings challenge the notion that only a few rogue entities enable surveillance. Rather, we found that the normalization of targeting humans permeates the field. This normalization is especially striking given patterns of obfuscation. We reveal obfuscating language that allows documents to avoid direct mention of targeting humans, for example, by normalizing the referring to of humans as ‘objects’ to be studied without special consideration. Our results indicate the extensive ties between computer-vision research and surveillance.

And look it this: ICE Is Using a New Facial Recognition App to Identify People, Leaked Emails Show

• Immigration and Customs Enforcement (ICE) is using a new mobile phone app that can identify someone based on their fingerprints or face by simply pointing a smartphone camera at them, according to internal ICE emails viewed by 404 Media. The underlying system used for the facial recognition component of the app is ordinarily used when people enter or exit the U.S. Now, that system is being used inside the U.S. by ICE to identify people in the field. 

AI and Data Voids: How Propaganda Exploits Gaps in Online Information

• The pattern became apparent as early as July 2024, when NewsGuard’s first audit found that 31.75 percent of the time, the 10 leading AI models collectively repeated disinformation narratives linked to the Russian influence operation Storm-1516. These chatbots failed to recognize that sites such as the “Boston Times” and “Flagstaff Post” are Russian propaganda fronts powered in part by AI and not reliable local news outlets, unwittingly amplifying disinformation narratives that their own technology likely assisted in creating. This unvirtuous cycle, where falsehoods are generated and later repeated by AI systems in polished, authoritative language, has demonstrated a new threat in information warfare. 

I just came across something that might be useful for belief maps. Meta has been fooling around with “large concept models” (LCM) You could compare the output of the model in question to the embeddings in the concept space and see if measures of the output in the embedding space change meaningfully.

• Paper and github repo.

Tasks

• Ping Alston again
• Find out who the floor refinisher is again
• Mow?

GPT Agents

• Got the trustworthy info edits back from Sande and rolled them in.
• I should also send the new proposal to Carlos today and see if there is any paperwork that I can point to for proposal writing
• Follow up on meeting times

SBIRs

• 9:00 standup
• Waiting on T for casual time
• I think we’re still waiting on Dr. J for approval of the modified SOW?

Worked on the title a bit:

Here’s a good long read on AI and drones from The Guardian: https://www.theguardian.com/world/2025/jun/25/ukraine-russia-autonomous-drones-ai

• “We didn’t know the Terminator was Ukrainian,” Fedoryshyn jokes. “But maybe a Terminator is not the worst thing that can happen? How can you be safe in some city if somebody tried to use a drone to kill you? It’s impossible. OK, you can use some jamming of radio connection, but they can use some AI systems that know visually how you look, and try to find you and kill you. I don’t think that the Terminator and the movie is the worst outcome. If this war never started, we will never have this type of weapon that is too easy to buy and is very easy to use.”

Tasks

• Ping Alston again
• Get the paperwork on painting starts
• Find out who the floor refinisher is again

GPT Agents

• Got a first pass on the edits to vignette 1. Need to do a readthrough to see if everything – particularly early parts of the story – are working right in present tense.
• Sent the trustworthy info off to Sande. I should also send to Carlos today and see if there is any paperwork that I can point to for proposal writing
• Ping Shimei and Jimmy for new meeting time?

SBIRs

• Waiting on T for casual time
• Need to add PB’s email to the PII project doc
• I think we’re still waiting on Dr. J for approval of the modified SOW?

Nice technical analysis on election fraud. I’d like to do an embedding analysis of left and right-wing conspiracy claims about recent elections and compare it to this thread.

GPT Agents

• Add How Malicious AI Swarms Can Threaten Democracy to the WH AI section – done
• Ping Carlos – after Sande has a chance to take a look
• Start rolling in edits to KA – done! Need to do a full read through

SBIRs

• Write up notes from yesterday – done
• 10:30 Meeting with Orest – done
• Negotiating with T

Hot:

House

• Property taxes – done?
• Ping painter people for estimate – done
• Ping Alston for schedule – done
• Ping floor people for estimate

SBIRs

• Letter for Orest – done
• 9:00 Demos – done
• 3:00 Sprint planning – done

A friend and I decided to test out our new fitness and do a tough ride yesterday. No records, but it went well. I have an exercise hangover today.

Tasks

• Weed
• Mail – done
• Assemble bike – done
• KA Book
• Add some of the new P33 content
• Ping Andrea – done

Tasks

• Bills
• Sheets
• Kitty Litter!

SBIRs

• Catch up at work – done kind of. More next week
• Write quarterly report – done
• COI Disclosure
• No afternoon meeting?

How Malicious AI Swarms Can Threaten Democracy

• Advances in AI portend a new era of sophisticated disinformation operations. While individual AI systems already create convincing—and at times misleading—information, an imminent development is the emergence of malicious AI swarms. These systems can coordinate covertly, infiltrate communities, evade traditional detectors, and run continuous A/B tests, with round-the-clock persistence. The result can include fabricated grassroots consensus, fragmented shared reality, mass harassment, voter micro-suppression or mobilization, contamination of AI training data, and erosion of institutional trust. With increasing vulnerabilities in democratic processes worldwide, we urge a three-pronged response: (1) platform-side defenses—always-on swarm-detection dashboards, pre-election high-fidelity swarm-simulation stress-tests, transparency audits, and optional client-side “AI shields” for users; (2) model-side safeguards—standardized persuasion-risk tests, provenance-authenticating passkeys, and watermarking; and (3) system-level oversight—a UN-backed AI Influence Observatory.
• Check out the organizations. Maybe there is an opening?

How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?

• Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning, which is crucial in spotting mistakes and arriving at accurate solutions. A natural question that arises is how effectively models can perform such self-reevaluation. We tackle this question by investigating how well reasoning models identify and recover from four types of unhelpful thoughts: uninformative rambling thoughts, thoughts irrelevant to the question, thoughts misdirecting the question as a slightly different question, and thoughts that lead to incorrect answers. We show that models are effective at identifying most unhelpful thoughts but struggle to recover from the same thoughts when these are injected into their thinking process, causing significant performance drops. Models tend to naively continue the line of reasoning of the injected irrelevant thoughts, which showcases that their self-reevaluation abilities are far from a general “meta-cognitive” awareness. Moreover, we observe non/inverse-scaling trends, where larger models struggle more than smaller ones to recover from short irrelevant thoughts, even when instructed to reevaluate their reasoning. We demonstrate the implications of these findings with a jailbreak experiment using irrelevant thought injection, showing that the smallest models are the least distracted by harmful-response-triggering thoughts. Overall, our findings call for improvement in self-reevaluation of reasoning models to develop better reasoning and safer systems.
• I think this might be helpful for white hat AI applications as well. Conspiracy theories and runaway social realities are also unhelpful thoughts, and there is a need for social “meta-cognitive awareness.”

A Statistical Physics of Language Model Reasoning

• Transformer LMs show emergent reasoning that resists mechanistic understanding. We offer a statistical physics framework for continuous-time chain-of-thought reasoning dynamics. We model sentence-level hidden state trajectories as a stochastic dynamical system on a lower-dimensional manifold. This drift-diffusion system uses latent regime switching to capture diverse reasoning phases, including misaligned states or failures. Empirical trajectories (8 models, 7 benchmarks) show a rank-40 projection (balancing variance capture and feasibility) explains ~50% variance. We find four latent reasoning regimes. An SLDS model is formulated and validated to capture these features. The framework enables low-cost reasoning simulation, offering tools to study and predict critical transitions like misaligned states or other LM failures.
• I think this might be important for working out LLM topic projections for maps

As counterpoint to Apple’s paper from yesterday, there is this article on the absolutely phenomenal burn rate for OpenAI

“The parallels to the 2007-2008 financial crisis are startling. Lehman Brothers wasn’t the largest investment bank in the world (although it was certainly big), just like OpenAI isn’t the largest tech company (though, again, it’s certainly large in terms of market cap and expenditure). Lehman Brothers’ collapse sparked a contagion that would later spread throughout the global financial services industry, and consequently, the global economy. “

“I can see OpenAI’s failure having a similar systemic effect. While there is a vast difference between OpenAI’s involvement in people’s lives compared to the millions of subprime loans issued to real people, the stock market’s dependence on the value of the Magnificent 7 stocks (Apple, Microsoft, Amazon, Alphabet, NVIDIA and Tesla), and in turn the Magnificent 7’s reliance on the stability of the AI boom narrative still threatens material harm to millions of people, and that’s before the ensuing layoffs.”

And here’s a direct counterpoint to the Apple paper: Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity

• Shojaee et al. (2025) report that Large Reasoning Models (LRMs) exhibit “accuracy collapse” on planning puzzles beyond certain complexity thresholds. We demonstrate that their findings primarily reflect experimental design limitations rather than fundamental reasoning failures. Our analysis reveals three critical issues: (1) Tower of Hanoi experiments systematically exceed model output token limits at reported failure points, with models explicitly acknowledging these constraints in their outputs; (2) The authors’ automated evaluation framework fails to distinguish between reasoning failures and practical constraints, leading to misclassification of model capabilities; (3) Most concerningly, their River Crossing benchmarks include mathematically impossible instances for N > 5 due to insufficient boat capacity, yet models are scored as failures for not solving these unsolvable problems. When we control for these experimental artifacts, by requesting generating functions instead of exhaustive move lists, preliminary experiments across multiple models indicate high accuracy on Tower of Hanoi instances previously reported as complete failures. These findings highlight the importance of careful experimental design when evaluating AI reasoning capabilities.

The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity

• Recent generations of frontier language models have introduced Large Reasoning Models (LRMs) that generate detailed thinking processes before providing answers. While these models demonstrate improved performance on reasoning benchmarks, their fundamental capabilities, scaling properties, and limitations remain insufficiently understood. Current evaluations primarily focus on established mathematical and coding benchmarks, emphasizing final answer accuracy. However, this evaluation paradigm often suffers from data contamination and does not provide insights into the reasoning traces’ structure and quality. In this work, we systematically investigate these gaps with the help of controllable puzzle environments that allow precise manipulation of compositional complexity while maintaining consistent logical structures. This setup enables the analysis of not only final answers but also the internal reasoning traces, offering insights into how LRMs “think”. Through extensive experimentation across diverse puzzles, we show that frontier LRMs face a complete accuracy collapse beyond certain complexities. Moreover, they exhibit a counterintuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget. By comparing LRMs with their standard LLM counterparts under equivalent inference compute, we identify three performance regimes: (1) low complexity tasks where standard models surprisingly outperform LRMs, (2) medium-complexity tasks where additional thinking in LRMs demonstrates advantage, and (3) high-complexity tasks where both models experience complete collapse. We found that LRMs have limitations in exact computation: they fail to use explicit algorithms and reason inconsistently across puzzles. We also investigate the reasoning traces in more depth, studying the patterns of explored solutions and analyzing the models’ computational behavior, shedding light on their strengths, limitations, and ultimately raising crucial questions about their true reasoning capabilities.

Operational Art and the Salvation of Ukraine

Yesterday was kind of a blur. Worked with Aaron quite a bit:

• I think that synchronizing the different folders requires make venv
• Need to change the order of the inferred and actual curves to see what’s going on with only one inferred curve being drawn
• Change the code so config list is generated, but not written out

Meeting with Seg

• Lots of interesting information on how the system works together, and where we might fit in.
• Operational debris seems like an easy win, and something to focus on

Nice dinner!

Forgot to mow the lawn and it rained last night

GPT Agents

• No word from the NY Times, so no OpEd. Refactoring for The Conversation
• 4:15 Meeting

Tasks

Tasks

• Water plants – done
• Fix hose
• JB-weld rings
• Emily email
• Signed this: https://www.standupforscience.net/open-letter-in-support-of-science

SBIRs

• 9:00 standup – done
• 10:00 RTAT tagup
  • Get the data generation running for Ron. I think we have to make more and longer trajectories
• 10:30 RTX followup
• 3:00 Tradeshow demo