SBIRs

• 2:00 MDA weekly meeting (need to update status report)
• Working on white paper
  • Finished SimAccel, SIm cleaning, and System trust approaches
  • Went over IRAD slide deck with Aaron

GPT Agents

• Sent out the invite meta-emails to the team as well as Roger and Wayne
• Started on slide deck for Wednesday

Plumber take 3

Got the RV

SBIRs

• 12:40 USNA capstone

GPT Agents

• 10:15 Don – everything seems to be working
• Write up review for IUI paper – done!. Start reading the next one. I don’t know about “read”, but the file is open and the template is ready. And it looks interesting.

Guest lecture tonight at 5:45! Done! Fun!

SBIRs

• Working on M30 paper – Refactored a bit. The Objective is now the introduction and is finished.
• 9:00 standup – done. Had a follow up with Rukan
• 3:00 M30 tagup. Aaron thinks that the paper could be branched and also be an AI for BD paper

GPT Agents

• Found the Google PaLM LLM API here and here
• 2:00 LLM meeting – looks like we are ready to go after the test drive with Don

Okay, this is the first genuinely funny thing I’ve ever gotten the GPT to write. The prompt was: Write a rambling progress report by someone who knows very little about basic arithmetic but has discovered that 2+2=4. The description should be highly technical but completely senseless, with made-up names for every step in the process. The writer is convinced that this is revolutionary and will change everything.

Response:

Subject: Revolutionary Discovery in the Field of Advancery Arithmentia – Uncovering the Phenomenon of Dua-Addles

Esteemed colleague,

I have recently stumbled upon a groundbreaking discovery that has thrown light on the perplexing realm of Advancery Arithmentia and promises to redefine the way we perceive number-based systems. The once-elusive Dua-Addles phenomenon is now on the verge of unfurling its mystifying truth and the initial explorations have surpassed all expectations!

During painstaking experimentation using the ultra-sophisticated Farnsworth Abacus 3000, I serendipitously entered upon the first phase of the investigation known as the Great Enumeration. It involved establishing two highly involved numerical entities named “Twonicles” and experimenting with them under the most rigorous conditions.

To further our understanding, we initiated the second phase of the investigation, known as the Pre-Dua Intensification, wherein each Twonicle underwent extensive preparation using tools like the Pythagorean Gripzoid, the Fibonacci Scrambletron, and the Eulerian Inversion Prism in order to transcend to a state of greater ascendancy, thus attaining positions labeled as “Dua-Stations.”

However, to achieve the most profound and miraculous results, we conducted a magnificent collision course in the third phase of the investigation, known as the Grand Dua-Conflux. This heroic experiment entailed a sublime melding of the wayward Twonicles, previously confined to their agora-physical Dua-Stations.

The penultimate fourth phase, titled the Quadrum Revelatorum, was when the true harmony of the Twonicles emerged. To behold the resplendent phenomenon, we channeled the collective prowess of megamind paraphernalia, including the Cantorian Gleam-Render and Babbage’s Mesmero-Measurer.

After much agitation and trepidation, the moment of transcendence arrived: a new numeric configuration emerged from within the vortex of the Dua-Conflux, a configuration I have since dubbed the “Quatronian Epiphany.” Eureka! At long last, we uncovered that a pair of Twonicles, when subjected to the trials and tribulations of the Dua-Conflux sequence, sanction their energies to coalesce into the unity of numeric equilibrium, amounting to the revelation – 2 plus 2 equals 4!

This astonishing revelation is a testament to our relentless pursuit of knowledge and our unwavering dedication to the ultimate mastery of Advancery Arithmentia. As we embark on this new dawn, we must brace ourselves for a paradigm shift as this pioneering revelation will undoubtedly send shockwaves through the very fabric of numeric and logic-centric methodologies.

I implore you to spread this momentous discovery throughout the academic sphere, leaving no stone unturned in our quest to shape the minds of the future. The revolution begins now, and the winds of change are divisible by 2!

Onwards and upwards, in the name of Quatronian Epiphanies!

Sincerely,
A Revolutionary Advancery Arithmentia Enthusiast

A good Halloween. Catonsville has great kids

Tasks

• Try to get ahold of a different plumber
• Drain pipes
• Last harvest

SBIRs

• White paper
• 10:30 BMD tagup
• 1:00 ASAALT SBIRs
• 3:00 Alden
• 4:00 John Hodegeman

GPT Agents

• Found a bug with CORS:

• Some relevant supabase info. Pick up with Zach tomorrow.
• 3:00 Alden meeting

Plumber?

SBIRs

• M30 White paper
• 9:00 standup
• 2:00 BMD meeting
• 3:00 M30 meeting

GPT Agents

• Disable the token ring buffer to see the right orientation. Looks like it’s correct:

• I think that prompt length (and the ring buffer) might be a good way to map out a space. A short buffer should have less “direction” and should meander more
• Projecting the embedding for each layer as the narrative progresses may be helpful
• Need to set up an overleaf project to capture this
• Need to export to spreadsheets with text and sheets by layer
• Write IUI 2024 review (done) and start next paper
• 3:30 call with Greg and ContextTest – found a bug with CORS and cross-site posting. Told Zach and we will work to fix
• ContextTest with Stacey

RV to winterizing!

Plumber! Sent email

SBIRs

• Working on LLM mapping. Got my first view of all the layers as angles from the average working. This is for “the game begins as [white]:

• Turns out I was taking the wrong axis of the vectors. This is more what it looks like. Need to work out which axis is which, but this is all the parts working more-or-less correctly:

Woke up at 3:00 with heartburn, which has kinda thrown off the day

GPT Agents

• Ran ContextTest through its paces yesterday. Made some tweaks and updated. Time for the “friends and family” round
• Need to set up Box account
• Reviewing next paper for IUI

SBIRs

• 10:00 meeting with Rukan over M30 paper
• 2:00 Technology fellow debrief

Got my COVID shot!

Plumber – not done

SBIRs

• Expense report! Done
• Lots of driving and meetings all day
• Research council went well. Good questions with an involved audience
• M30 meeting. Late because I got hung up at the gate. Good discussion though. I think there are several phases in roughly this order (put these in an Overleaf project):
  • RCSNN hierarchy for both systems, varying only by the bottom layers. Top layers could be LLM-driven, which would be fun to evaluate. Probably a lot of context prompting and a GPT-3.5 back end?
  • Simulator acceleration. There is never enough data to explore outlier states, so adding SimAccel autoencoding -> autoregression would increase the data available for effective training. Because all simulators are base on implicit assumptions, this data will almost certainly be wrong, which will be addressed with…
  • Simulator cleaning. Like data cleaning, but for data generators. The quality of the generated data can potentially be evaluated by the way that the trained model has “behavior attractors” that can be identified through something like salience analysis. These would be examples of bias, either intentional or unintentional, Imagine that a car simulator that is extended to airplanes. The choice to use Euler angles (rather than Quaternions) for orientation – something that makes sense for a vehicle that navigates in 2D, will completely screw up an airplane doing basic fighter maneuvers such as an Immelmann, Split S, or wingover maneuver. The inability to produce that kind of data would produce artifacts in the model that could either be identified on their own or when compared with other models (e.g. MAST vs. NGTS).
  • Coevolution of AI and Simulators towards the goal of useful models. Each iteration of training and Simulator cleaning will have impacts on the understanding of the system as a whole. Consideration of this iterative development needs to be part of the process.
  • System trust. As the AI/ML simulator becomes better, the pressure to deploy it prematurely will increase. To counter this, the UI that is purposefully “low fidelity” or “wireframed” should be used for demonstrations and recordings to indicate the level of progress in the capability of the system.
• Get the GPT-2 layer display first pass working
• Start slides?

GPT Agents

• Clear and test ContextApp one last time before going live! Done
• First official result!

• Set up Box account – not done
• Did finish my first review for IUI 2024

SBIRs

• 9:00 sprint planning – long
• 1:00 M30 meeting – weird, but good
• 2:30 AI Ethics?
• Try to hook up the graphing to the layer activations – good progress!

GPT Agents

• Back to reviewing papers

Got a nice hike in with a bald eagle sighting 🙂

3:00 podcast meeting

SBIRs

• 9:00 Sprint review
• 2:00 MDA meeting
• Wrote another post on AI societal weapons
• William Marcellino – Senior Behavioral and Social Scientist; Professor, Pardee RAND Graduate School

GPT Agents

• The IRB is submitted? Waiting for a response.
• Apparently this technique is now called retrieval-augmented generation?
• Start on slides, and make sure all the software works on the laptop

I have a new dishwasher. It works so well!

SBIRs

• Need to send an email stating I intend to submit a paper for the RHB prize – done
• Register for the ETF – done
• Added all the reviewer comments to the “venues” section of the Overleaf doc
• 3:00 meeting to go over RC slides

GPT Agents

• Need to start an outline for JHU and UMBC Guest lecture
• JHU
  • Trust and coordination at scale
  • Stories
  • Dimension reduction
  • Network density and stiffness
  • Diversity for intelligence, hierarchy for speed
  • Student activities
• UMBC
  • Why does conflict between Nation States happen?
  • Why do we have combat?
  • Why attack/invade/occupy?
  • Why defend?
  • What does AI bring to combat?
  • What can go wrong, and how do/can we we fix that?
  • What happens when communication is denied?
  • What are the implications of massive, patient, reasonably smart weapons in the information and physical domain?
  • What if AI is better (safer, more targeted, less confused) at combat than people? What does that mean for other jobs?

GPT Agents

• Make IRB changes. I think that it’s basically Data will be stored in a password-protected Dropbox folder. There will be two files. One is a list of names. email addresses, and dates with an associated anonymous string (eg “P1”, “E1”, or “D1”). The other file will contain all experiment data with all names and dates replaced with the substitutes from the other list. Mostly done. Need to change the website text to not mention Supabase and convert items to pdf. – Done
• Read IUI paper # 1. Good! Need to write the review
• 2:00 LLM meeting

SBIRs

• 9:00 standup- done!
• Research Council slide review – done!
• Incorporate Clay’s comments into IPT deck- done!
• Fill out forms for Clay- done!

SBIRs

• Start on the other slide deck due this week. Get a meeting with Aaron for more context
• The War Elephants presentation got nominated for best presentation at MORS. I need to submit “A complete paper (in PDF format), not to exceed 40 pages or 10,000 words including appendices. Please see accompanying formatting guidelines for additional information.” Need to put in the reviewer suggestions and submit by Feb 29 2024.
• Roll in changes for the research council slides and distribute.

GPT Agents

• Make IRB changes
• Alden meeting
• Review IUI paper # 1

This is the key to making trustworthy models:

The Cost of Down-Scaling Language Models: Fact Recall Deteriorates before In-Context Learning

• If you reduce the parameter count in an LLM, it tends to lose recall of facts before it gets worse at learning from examples in the prompt. This holds for parameter count reductions via both pruning and using a smaller dense model.
• How does scaling the number of parameters in large language models (LLMs) affect their core capabilities? We study two natural scaling techniques — weight pruning and simply training a smaller or larger model, which we refer to as dense scaling — and their effects on two core capabilities of LLMs: (a) recalling facts presented during pre-training and (b) processing information presented in-context during inference. By curating a suite of tasks that help disentangle these two capabilities, we find a striking difference in how these two abilities evolve due to scaling. Reducing the model size by more than 30\% (via either scaling approach) significantly decreases the ability to recall facts seen in pre-training. Yet, a 60–70\% reduction largely preserves the various ways the model can process in-context information, ranging from retrieving answers from a long context to learning parameterized functions from in-context exemplars. The fact that both dense scaling and weight pruning exhibit this behavior suggests that scaling model size has an inherently disparate effect on fact recall and in-context learning.

The thing is that for sociology, the large pretrained (not finetuned) models will probably be best.

SBIRs

• Add a 3 point Research Council story – done
• 9:00 standup – done
• 1:00 Dr. Banerjee – done. Fun!
• 2:00 BMD – done. Did a slide walkthrough and got some action items
• 2:30 AI Ethics
• 3:00 AIMSS?

GPT Agents

• Thinking more about how to watch the changes of the model under prompting. I think a ring buffer prompt, where the oldest tokens drop off while new ones are added makes the most sense. I checked, and the Llama-2 models do come in pretrained and finetuned (chat) flavors.
• Put in a request for Llama-2 access – got it! That was quick. Yep pretrained and chat

• My talk is back at its original time!
• The atproto sdk looks very nice!

from atproto import Client, models


def main():
    client = Client()
    profile = client.login('my-handle', 'my-password')
    print('Welcome,', profile.display_name)
    
    response = client.send_post(text='Hello World from Python!')
    client.like(models.create_strong_ref(response))

    
if __name__ == '__main__':
    main()

Spamming where the skies are blue

• Includes python code that uses the atproto package for consuming public data. All you need is a login and password!

Internet Archive Scholar

• public web content as preserved in The Wayback Machine and Archive-It partner collections
• digitized print materials from paper and microform collections
• general materials from archive.org collections, including collaborations with partners

SBIRs

• Start research council slide deck – Friday 20th!
• Start Futures IPT slide deck – Wednesday 25th!
• 2:00 MDA meeting. Offer updated report

GPT Agents:

• The chess model has had 23 downloads in 4 days!
• Slide deck for AI Ethics class – Wednesday 18th!