Tasks

• Sign things in blue for Wolfram
• Pick up Stacey’s car and get the van if it’s not too wet

SBIRs

• Scan cards
• Expense report
• Put together slides for sprint demo
• USNA interns final presentation
• Conference debrief to Orest
  • The conference was almost exclusively props and videos. No one could really show anything unless you were a manufacturer (“look at this beautiful planetary gear!”)
    • The golf idea was universally well received, and there was one other booth with an interceptor game. I just don’t think it went far enough, and was too easy to defeat

• We need videos that work with no audio. Also the lighting on Aaron makes his head distractingly shiny.
• The booths that had demos had more people as a rule (Blue Halo and Axient were the only ones I saw)
• A monitor in the conference-facing desk would be better than one on the desk
• Chairs need to work on deeply-padded tradeshow carpets. The dis-based chairs we had wobbled. A lot.
• Sequentially numbered/barcoded batches of bears that when scanned link to ASRC branded polar bear cams (or even better an updated curated feed)

GPT Agents

• Write next review and read a paper

Anointed with Oil

• Anointed with Oil places religion and oil at the center of American history. As prize-winning historian Darren Dochuk reveals, from the earliest discovery of oil in America during the Civil War, citizens saw oil as the nation’s special blessing and its peculiar burden, the source of its prophetic mission in the world. Over the century that followed and down to the present day, the oil industry’s leaders and its ordinary workers together fundamentally transformed American religion, business, and politics — boosting America’s ascent as the preeminent global power, giving shape to modern evangelical Christianity, fueling the rise of the Republican Right, and setting the terms for today’s political and environmental debates.

Finishing the conference and heading home:

GPT Agents

• Finished one more paper review and started the fourth one. Pretty good so far!
• Jimmy put in a big chunk of writing on the Consumer-First Approach to AI paper!

Transformer Layers as Painters

• Despite their nearly universal adoption for large language models, the internal workings of transformers are not well understood. We aim to better understand the impact of removing or reorganizing information throughout the layers of a pretrained transformer. Such an understanding could both yield better usage of existing models as well as to make architectural improvements to produce new variants. We present a series of empirical studies on frozen models that show that the lower and final layers of pretrained transformers differ from middle layers, but that middle layers have a surprising amount of uniformity. We further show that some classes of problems have robustness to skipping layers, running the layers in an order different from how they were trained, or running the layers in parallel. Our observations suggest that even frozen pretrained models may gracefully trade accuracy for latency by skipping layers or running layers in parallel.

SBIRs

• At a dumb conference and trade show that I just don’t need to be at.

GPT Agents

• Getting some progress on paper reviews though

Good ride on Saturday and a nice party on Sunday.

GPT Agents

• Finished two reviews! Printed out the other three for the flight

SBIRs

• Going down to Huntsville for a few days to be a booth babe. Hot but dry there, rain here.

From BlueSky

No interns today, so reviewing papers instead. One down!

Tweaked the Overleaf document.

Applied for the Reddit Research API

Liars know they are lying: differentiating disinformation from disagreement

• Mis- and disinformation pose substantial societal challenges, and have thus become the focus of a substantive field of research. However, the field of misinformation research has recently come under scrutiny on two fronts. First, a political response has emerged, claiming that misinformation research aims to censor conservative voices. Second, some scholars have questioned the utility of misinformation research altogether, arguing that misinformation is not sufficiently identifiable or widespread to warrant much concern or action. Here, we rebut these claims. We contend that the spread of misinformation—and in particular willful disinformation—is demonstrably harmful to public health, evidence-informed policymaking, and democratic processes. We also show that disinformation and outright lies can often be identified and differ from good-faith political contestation. We conclude by showing how misinformation and disinformation can be at least partially mitigated using a variety of empirically validated, rights-preserving methods that do not involve censorship.

Dog days of summer for sure

Need to start on the tasks Wolfram asked for

SBIRs

• Really interesting discussion with Aaron on CI Agents. It might be possible for token trajectories to maintain their “identity” by looking at the distance between a set of agent responses, and selecting those that have the greatest distance.
• Got some good NNM work done yesterday. I’m almost ready to generate ring buffer text along with activations by token at each layer of the model. And in case I haven’t really mentioned it anywhere else, I think training ring-buffer GPT-2 (or bigger, local models that can quickly be finetuned. You need to be able to access the layers) models via finetuning from a big model with extensive prompt tuning might be a very good way to create local maps.
• 9:00 standup
• Lunchtime ride in this gap!
• 11:00 M30
• 2:00 Conference prep
• 2:30 Hall research
• 4:30 Book club

GPT Agents

• Finish TiiS review – done!
• ICTAI – Downloaded papers. Deadline is August 18, so that’s a bit over 3 days per paper
• 3:00 Meeting
• Can Artificial Intelligence be Open Sourced?
  • This paper explores the potential of open source models to match or even surpass proprietary models in the future. The dialogue reflects a broader debate on the implications of open sourcing AI, weighing the benefits of democratization against the risks of misuse.
• The Public Interest Internet
  • Allow me to open with a wildly speculative question: What if the internet were public interest technology? I mean “internet” the way most people understand it, which is to say our whole digital sphere, and by “public interest” I don’t mean tinkering at the margins to reduce harm from some bad actors or painting some glossy ethics principles atop a pile of exploitative rent-seeking — I mean through and through, warts and all, an internet that works in support of a credible, pragmatic definition of the common good.¹