Dont you (forget NLP): Prompt injection with control characters in ChatGPT

• As part of this work, we recently observed some unusual behavior with two popular large language models from OpenAI, in which control characters (like backspace) are interpreted as tokens. This can lead to situations where user-controlled input can circumvent system instructions designed to constrain the question and information context. In extreme cases, the models will also hallucinate or respond with an answer to a completely different question.

SBIRs

• 2 hour meeting in Moorestown with a 2.5 hour drive each way. Not sure it’s worth it, but the meeting went well,
• Closed out my stories, made slides, and created new stories. Still need to do one for the IS Dept presentation on the 22nd

5:00 power washer

CUI 2024 paper:

• Distorting the truth versus blatant lies: The effects of different degrees of deception in domestic and foreign political deepfakes
• Detecting Video Speed Manipulation
• Situational Awareness of Deepfake Technology
• AI Is Not a Unitary Actor: My Response to the UN Interim Report Consultation: “the limits of the almost supernaturalist framing of AI as a unitary actor, rather than a suite of techniques. More useful for the report overall would be of AI as an extension of our human capacities of reasoning, action, and perception that alters what we can do, and therefore our obligations, especially as governments.“
• Made good progress on “The Voice in the Machine” section

SBIRs

• Slide deck and stories for Monday – put in a story to make a sandbox project using a makefile. cookiecutter, name, venv, and source.

GPT Agents

• 3:00 Alden

Need to move the crown appointment

Review board faults Microsoft for ‘cascade’ of errors in China hack

• The report details what it calls a “cascade of avoidable errors.” For instance, Microsoft had not noticed the presence of an old signing key from 2016 that should have been disabled but wasn’t. “That one just sat for years, kind of forgotten,” a second person said. Part of the problem was that Microsoft was supposed to switch from a manual key rotation to an automated system that minimized the chance of human error. But that switch never happened. “They never prioritized fixing the problem,” the first person said.

SBIRs

• More CUI paper. Bring in papers on dark patterns in CUIs and CSCW
• 9:00 Standup
• Ron’s in today? Maybe more setup. Done! Wrote up a story too.
• AI Ethics?
• MCML?

Call powerwasher

SBIRs

• Working on the CUI paper
• Did the vague letter thing. Still not sure why anything should happen, but it’s an encantation maybe?

Pick up the truck if the rain stays away this morning – done!

Good Organizational Lobotomy writeup: The Descent of Elon Musk

SBIRs

• Made good progress on the LLM notes yesterday. Waiting for the AWS instance to get turned on so I can play with UMAP. Got a lot done, though there are resolution issues that need to be worked out.
• 9:00 Standup – done
• 11:30 CSC touchpoint – done
• 1:00 Cyber COP presentation – done, and went well.

GPT Agents

• LLM Meeting – went over the NNM work with Shimei
• Started on CUI provocation. As part of that, I’ve started reading Tim O’Reily’s biography(?) of Frank Herbert. It’s really good. He also says that Dune is an alternate Foundation, and Paul is The Mule. Fascinating.
  • In Dune, each of the players—the Emperor, the Baron Harkonnen (archenemy of the Atreides), the monopolistic Spacing Guild, even the seemingly wise Bene Gesserit gene manipulators—tries either to dominate the situation or to control it in such a way as to minimize his own risks. And in the end all are overwhelmed. The elemental forces of history can only be ridden, not controlled. Paul alone is victorious, because he chooses to ride the whirlwind. He risks everything. His initiation by the Fremen into riding the sandworms is symbolic of his choice. These predators represent all the elemental forces of Arrakis: their native name means “maker,” and they are the heart of the ecological matrix of the planet, source of the spice, the sand, and thief of water. And, like nature itself, they abhor artificial boundaries; they are drawn irresistibly to destroy the protective energy shields relied on by off-worlders. They close the desert to all who try to isolate themselves from it; only the Fremen “sandriders,” who move with the rhythms of the desert, and mount the fearsome worm, can brave its wilds.
• Start White Hat AI section – tomorrow?

AI experts: Racist audio of Baltimore County principal’s voice is fake – The Baltimore Banner

• The Eiswert audio, Lyu said, shows the danger of the degree to which artificial intelligence could be used to harm individuals. When deepfakes are used on celebrities or well-known political figures, they are easier to detect, both because there’s an abundance of video and audio of their voices and so the public is more likely to believe and spot a fake. “If they are focusing on less prominent people … the damage they are causing is bigger.”

Example: Pandas Excel output with a chart

##############################################################################
#
# An example of converting a Pandas dataframe to an xlsx file with a chart
# using Pandas and XlsxWriter.
#
# SPDX-License-Identifier: BSD-2-Clause
# Copyright 2013-2024, John McNamara, jmcnamara@cpan.org
#

import pandas as pd


# Create a Pandas dataframe from some data.
df = pd.DataFrame({"Data": [10, 20, 30, 20, 15, 30, 45]})

# Create a Pandas Excel writer using XlsxWriter as the engine.
writer = pd.ExcelWriter("pandas_chart.xlsx", engine="xlsxwriter")

# Convert the dataframe to an XlsxWriter Excel object.
df.to_excel(writer, sheet_name="Sheet1")

# Get the xlsxwriter workbook and worksheet objects.
workbook = writer.book
worksheet = writer.sheets["Sheet1"]

# Create a chart object.
chart = workbook.add_chart({"type": "column"})

# Get the dimensions of the dataframe.
(max_row, max_col) = df.shape

# Configure the series of the chart from the dataframe data.
chart.add_series({"values": ["Sheet1", 1, 1, max_row, 1]})

# Insert the chart into the worksheet.
worksheet.insert_chart(1, 3, chart)

# Close the Pandas Excel writer and output the Excel file.
writer.close()

SBIRs

• Spent some time adding references to the slide deck
• Add a “to_excel()” method – done
• Meet with Aaron to set up AWS for UMAP? Things are working, but I don’t have an instance. Blocked for now
  • Install ecco
  • Install HFace
  • Maybe langchain?
• 1:30: LLM UI Tools – done

GPT Agents

• Meeting with Alden

Here’s Google’s generative API/playground/documentation site: Vertex AI

SBIRs

• 9:00 standup. Going to go over the IUI conference
• Working on the NNM again. Just getting back into things and generated a cosine similarity (by layer) to the average vector for each layer:

• Although it looks interesting, I think it would make more sense to export as a spreadsheet to show numbers and colors. Also, I’m not convinced that this is the right way to go. I’d like to try UMAP, but can’t get it to behave on my box. Going to try spinning up an instance on AWS

On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial

• The development and popularization of large language models (LLMs) have raised concerns that they will be used to create tailor-made, convincing arguments to push false or misleading narratives online. Early work has found that language models can generate content perceived as at least on par and often more persuasive than human-written messages. However, there is still limited knowledge about LLMs’ persuasive capabilities in direct conversations with human counterparts and how personalization can improve their performance. In this pre-registered study, we analyze the effect of AI-driven persuasion in a controlled, harmless setting. We create a web-based platform where participants engage in short, multiple-round debates with a live opponent. Each participant is randomly assigned to one of four treatment conditions, corresponding to a two-by-two factorial design: (1) Games are either played between two humans or between a human and an LLM; (2) Personalization might or might not be enabled, granting one of the two players access to basic sociodemographic information about their opponent. We found that participants who debated GPT-4 with access to their personal information had 81.7% (p < 0.01; N=820 unique participants) higher odds of increased agreement with their opponents compared to participants who debated humans. Without personalization, GPT-4 still outperforms humans, but the effect is lower and statistically non-significant (p=0.31). Overall, our results suggest that concerns around personalization are meaningful and have important implications for the governance of social media and the design of new online environments.

Tasks

• Dentist!
• Truck? – Nope. Should be ready Friday
• Eclipse email

SBIRs

• Expense report
• Put together a set of slides that covered what I thought was interesting at the conference
• CUI Provocation
• Killer Apps post for HCAI Medium Magazine(?)
• 4:00 meeting with Bob S. to go over the MP part of the LM white paper
• Need to get back to NNM

Just realized that Fritz Lang’s Metropolis is an example of a deepfake/KillerApp:

 …the false Maria unleashes chaos throughout Metropolis, driving men to murder and stirring dissent among the workers.

https://en.wikipedia.org/wiki/Metropolis_(1927_film)

Deepfake Kari Lake video shows coming chaos of AI in elections

• They brainstormed ideas for about a week and enlisted the help of a tech-savvy friend. On Friday, Stephenson published the piece, which included three deepfake clips of Lake.

Drove back from Greenville NC yesterday. Eleven hours or so. Electric cars are still not common on the long stretches between cities. I live in the Baltimore/DC region, and I’d say 1 in 20 cars that I see on the roads at this point is electric, with a big mix of manufacturers. Still mostly Teslas. On my drive, I only saw Teslas, and not very many of them. Charging infrastructure is solid. Leaving with a full charge, it required 2 SuperCharger recharges to cover the 535 miles. Probably $20 in electricity? And autopilot makes everything much nicer, though you always have to be ready for phantom braking.

Need to ping Mario, Ossi, and James about Eclipse coordination

IUI 2024 Notes

Information visualization and Visual Analytics

• Assessing User Trust in Active Learning Systems: Insights from Query Policy and Uncertainty Visualization – Active learning, where the machine asks for help if uncertain. This is human ESDT. Numeric labels seem to help the most over other graphical visualizations. Like a picture with a xx% confidence seems best.

25% confident

• iScore: Visual Analytics for Interpreting How Language Models Automatically Score Summaries – rates model rating of essays. Word replacement with synonyms allows the model to detect the salience of particular words in the grades. It would be interesting to try this with tests like GPTZero to see what they are looking for. Yes, they do have an API, so this would be straightforward?
• SlopeSeeker: A Search Tool for Exploring a Dataset of Quantifiable Trends. A good point here for our conversational interfaces. We may need to train (or prompt-tune?) models to be able to discriminate behaviors from the simulation in ways that can be articulated by the LLM
• Visual Analytics of Co-Occurrences to Discover Subspaces in Structured Data – a tool for exploring really large datasets. Like the Blackrock problem? It sort of resembles the way that I set up spreadsheets some times? And in fact, I think you could process the data and make it spreadsheet compatible.
• AutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning. Looks like a useful tool for building and evaluating models. I’d be curious how it would work with the JSC data. Worth watching the video, if its available. Shown using AutoSKLearn. Not sure about parallelization like raytune. Code is online.

Applications of Language Models

• Empirical Evidence on Conversational Control of GUI in Semantic Automation – conversational interface to access db. JSON-2-JSON transformer.
• FigurA11y: AI Assistance for Writing Scientific Alt Text – this looks like a really nice tool to generate alt-text and the code exists on GitHub. Not sure how easy to build, but worth trying. It would be interesting to see how the text could be incorporated in the body of the text as well, particularly in cases where the image needs to be pulled for length.

Generative AI: Theory and Applications

Got accepted for my talk at the 92nd MORS symposium!

IUI 2024 Notes

I had an interesting chat yesterday with Ossi about Active Measures Leading up to and since October 7. We need some time to sit down and talk. My sense is that all sides have been under prolonged external influence for a long time with the specific intent to raise the political temperature so that exactly this situation happened.

Keynote: Prof. Krzysztof Gajos (check Google Scholar for references)

• Predictive text can manipulate the users, who wind up reflection the biases of the predictive text model. Change the organization’s model, change the bias of the organization.
• The mere presence of an explanation increases the credibility of AI assistance, regardless of the content. Fact-like assertions increase the perceived competence of the AI. This is a dark pattern that needs to be detected.
• Learning means that cognitive engagement occurs, but AI answers vs. cognitive forcing does not impact the amount of learning
• Providing the material to support a decision but not a decision suggestion, worked better than any answer-based decision aids. This may be a key for complementation
• Denial of a request is treated emotionally, not cognitively. This is another vector that needs to be recognized and adapted to. A source could be paper submission rejects.
• Critical techical practice – question assumptions

HCAI, Bias and Fairness in AI

• BiasEye: A Bias-Aware Real-time Interactive Material Screening System for Impartial Candidate Assessment – really good visualizations for decision support. An interesting use of an LLM to take the data from a form. The rest is mostly statistical comparisone between students. I wonder if this would be another way to have operators rate models. One of the most interesting visualizations was to use TSNE to cluster similar students on a canvas, where each student was a set of concentric pie charts that showed the difference between actual and expected performance. Also, lot’s of culture-specific measures such as “honor”
• Understanding Users’ Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Level – looks like it might be a really good source of working out a framework for evaluating effective conversational interfaces – what works, what doesn’t, etc.

Lunch ride!

AI Tools, User Interfaces and Interaction

• SpaceEditing: A Latent Space Editing Interface for Integrating Human Knowledge into Deep Neural Networks – This is an interesting take on hand-tweaking the manifold projection. It might be very good for DTA

AI for Health

• How Do Users Experience Traceability of AI Systems? Examining Subjective Information Processing Awareness in Automated Insulin Delivery (AID) Systems – Informational awareness. Nice perspective on AI Operators

Dinner Banquet – fun 🙂

Had a thought for the day. For a learning assignment, have students build a context prompt that lets an LLM answer a question on the rubric correctly. Bonus points if the models is able to answer a question that is outside the domain where a raw LLM has struggled. This way you have a project that requires students learning the topic, and also exposes them to weaknesses and strengths of LLMs. Not sure if this is a good idea, but it could be worth poking at.

Did my talk and demo at HAI-GAN yesterday. It went well, but I had forgotten my poster! I ad-hoc-ed on with the hotel printer and my slide deck:

Some really interesting talks:

• ExpressEdit: Video Editing with Natural Language and Sketching
  • Neat work on a very hard problem
• Intent Elicitation in Mixed-Initiative Co-Creativity
  • Collaborative storytelling with LLMs and Stable Diffusion at Midjourney.
• Deriving Desirable Artistic Generative Distributions from Individual Identity Statements
  • This one in particular had some really novel work. One idea that she mentioned almost in passing was the idea of latent mode collapse, which is a technique that I think I could use to recognize stampede behavior.
• Teaching Middle Schoolers about the Privacy Threats of Tracking and Pervasive personalization: A Classroom Intervention Using Design-Based Research
• Though no one made a presentation about it, the idea of prompt swarms in some form came up multiple times. Still nothing about maps or even about studying the structures that contain knowledge/narratives in LLMs

Main conference starts today. I’ll post notes here:

• Keynote – pathsto effective AI for diverse real people
  • Stumble forward empirracally
  • Draw from the field of education (e.g learning goals imply associated measures)
  • Add “after actin review” to your AI design – one of the things that seems to help is letting users come up with their own labels for things, which helps building abstractions that help overall understanding
  • Measure inclusiveness, and determine the why (GenderMag survey? Also SocioeconomicMag, AgeMag, InclusiveMag)
    • More inclusive designs of humans+AI ecosystem
    • Persona-based AI debugging

Got to chat with Mauro Martino about maps a bit

Sesssion 1:AI in Personalization, Recommendation and Search

Multimodal Models and Interaction

• Conan’s Bow Tie: A Streaming Voice Conversion for Real-Time VTuber Livestreaming – uses RNN for real-time predictive voice conversions. This could be generalizable? Less lag in video chat, or even remote robotics. Need to reach out possibly.
• ReviewFlow: Intelligent Scaffolding to Support Academic Peer Reviewing
• Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure
• Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern? Nice chat. Maybe write a provocation for CUI?
• Slicing, Chatting, and Refining: A Concept-Based Approach for Machine Learning Model Validation with ConceptSlicer – definitely something for AI operators
• VMS: Interactive Visualization to Support the Sensemaking and Selection of Predictive Models – nice visualization for AI Operators. And a demo!

Poster session. Met a bunch of nice folks, and had a longer chat with Mauro. Need to follow up.

No idea what to make of this:

• Red Dragon 1949.com is the premiere global web location for all updated information focused on the People’s Republic of China. Our intent is to develop a mutual cooperation and understanding of how the Internet and connected systems can be used by a nation state as a military weapon system.

Dammit

• De Waal, Charles Howard Candler Professor Emeritus of Psychology and former director of the Living Links Center for the Advanced Study of Ape and Human Evolution at the Emory National Primate Research Center, was 75.

Chores

• Drop off truck
• Haircut
• Slide backup
• Laundry
• Get gas for mower
• Weed
• Mow lawn
• Clean house
• Pack!
• Note!
• Trash

Kind of an interesting take on a way that white hat AI could work. Something like AI-generated annotations for all the manipulations we are being exposed to and the way that it is supposed to make you feel. And a knob that replaces the manipulative text with neutral text (possibly with whitelist fact checks):