Saw Dune Part 2 on Saturday. Good, but not as good as part 1. I have some thoughts about how the Dune universe lays out the conflict between hierarchical rule (the Emperor and the Harkonnens) and egalitarian communities (The Fremen). I draw on the work by Goodall, de Wall, Boehm, and Scott for this and it hangs together pretty well. Interestingly, I’ve been rubber ducking this with Gemini and that’s been quite helpful.

SBIRs

• The ICC tolls came in, so I submitted my travel expenses for the NIST talk
• Today is pretty much working on the white paper. Still no response from Matt about our questions
  • Finished the intro
  • Finished the synthetic logs section

Well that certainly explains the sense of warmer, snowless winters:

The big snow of 2010 here in Baltimore:

The big snow of 2024 here in Baltimore. We had no snow in 2022 and 2023:

LMSYS Chatbot Arena Leaderboard

• LMSYS Chatbot Arena is a crowdsourced open platform for LLM evals. We’ve collected over 300,000 human preference votes to rank LLMs with the Elo ranking system.

New (tubeless!) wheels for the fixee! For those unfamiliar, Cannondale Aluminum means “harsh ride.” Bigger, lower-pressure tires really help with that

SBIRs

• Filling out expenses for the NIST talk. EZ pass has no record of me on the ICC? Going to wait a bit for that to burble to the surface.

GPT Agents

• Finish poster!

A good example of Bostrom Pollution (https://bsky.app/profile/justicar.xyz/post/3kn4eim4f622c)

This is what I’ve been calling The Pancake Printer Economy, which I’ve been dreading since seeing this:

Lying Blindly: Bypassing ChatGPT’s Safeguards to Generate Hard-to-Detect Disinformation Claims at Scale

• As Large Language Models (LLMs) become more proficient, their misuse in large-scale viral disinformation campaigns is a growing concern. This study explores the capability of ChatGPT to generate unconditioned claims about the war in Ukraine, an event beyond its knowledge cutoff, and evaluates whether such claims can be differentiated by human readers and automated tools from human-written ones. We compare war-related claims from ClaimReview, authored by IFCN-registered fact-checkers, and similar short-form content generated by ChatGPT. We demonstrate that ChatGPT can produce realistic, target-specific disinformation cheaply, fast, and at scale, and that these claims cannot be reliably distinguished by humans or existing automated tools.

SBIRs

• Work on the white paper
• 9:00 Standup
• 10:00 SimAccel code review
• 11:30 SST dataset tagup
• 3:30 USNA

GPT Agents

• Working on the poster, and expanding the discussion section in the KA paper to talk about White Hat AI, since that went over well at NIST
• 2:00 Meeting with Shimei to go over SIGCHI reviews. I do want to discuss the idea of the construction of White Hat AI’s that take an understanding of individual and group psychology to detect dangerous manipulation from AI. And human actors, since this will soon get to the point that human and AI manipulation will be indistinguishable. Also, we need to do this in a way that respects agency for the individuals, with controls and opt-out/in approaches. There could easily be a “bias knob” that could be built to

Dentist

Ping Tim, Dave

SBIRs

• Talk went well yesterday. The White Hat AI seems to be reasonable. Need to put that on the poster
• White paper. Start to fill in the stuff that I remember the best

GPT Agents

• 3:00 Meeting with Alden

Started the day off on the wrong foot by dropping my breakfast. Grumble

SBIRs

• Starting LM white paper
• NIST AI COE presentation. Slides are done! Need to copy over to ppt and copy.

GPT Agents

• Need to make a 10-minute version of the presentation
• Need to see how the upstairs TV could work as a monitor
• Need to put together a new poster
• And I really need to add a AI White Hat section to the KillerApps paper based on the reception of the idea today
• The paper is up on ArXiV: RAGged Edges: The Double-Edged Sword of Retrieval-Augmented Chatbots
  • Large language models (LLMs) like ChatGPT demonstrate the remarkable progress of artificial intelligence. However, their tendency to hallucinate — generate plausible but false information — poses a significant challenge. This issue is critical, as seen in recent court cases where ChatGPT’s use led to citations of non-existent legal rulings. This paper explores how Retrieval-Augmented Generation (RAG) can counter hallucinations by integrating external knowledge with prompts. We empirically evaluate RAG against standard LLMs using prompts designed to induce hallucinations. Our results show that RAG increases accuracy in some cases, but can still be misled when prompts directly contradict the model’s pre-trained understanding. These findings highlight the complex nature of hallucinations and the need for more robust solutions to ensure LLM reliability in real-world applications. We offer practical recommendations for RAG deployment and discuss implications for the development of more trustworthy LLMs.

Some links on the relative environmental impact of computing:

AI is like a very tiny hamburger

The Staggering Ecological Impacts of Computation and the Cloud

Submitted the HAI-GEN paper!

Got rejected on the SIGCH late-breaking. Put the RAG paper up on ArXiv. Should be live Monday.

Undermining Ukraine: How Russia widened its global information war in 2023

• As the full-scale war in Ukraine enters its third year, Russia has doubled down on its worldwide efforts to undermine Kyiv’s international standing in an attempt to erode Western support and domestic Ukrainian morale. Years of close monitoring of not only state-sponsored media such as Russia Today (RT) and Sputnik, but also Russian activity on Telegram, TikTok, X, and other social platforms, points to one conclusion: In the propaganda war, Russia remains fully committed to conducting information operations around the globe, playing the long game to outlast any unity among Ukraine’s allies and persist until Ukraine loses its will to fight.

Going to work on the slides for the NIST talk – good progress