GPT Agents

• Creating the 4 and 5 star models currently. Done!
• Run each through the “vegetarian” options. I’m really curious how LIWC will look at the outputs of the models with relation to each other, and to the ground truth. Also get the counts of the occurrences of each prompt in the GT by star rating. My guess is that it won’t show up in some of the cases, which sets up the Twitter section really well.
• 4:15 Meeting

SBIRs

• Fix duplicate entries in the DB topic file – done
• Back up db – done
• Create superclass that has most of the parts and then subclass the various implementations (Full, BuildView, ViewScript) – done
• Work with Aaron on the stories/maps? Also, what is our plan for the paper? In process
• Ping Antonio? Next week, after the demo

Book

• Why we’re polarized review

WordPress has some serious lag. Need to back this up

GPT Agents

• Delayed meeting until Friday. That should give me time to get the balanced data working and compare baseline models to the baseline data (American)
• And the balanced data still isn’t working. I think that there are more paths to good reviews, so the GPT, even when fed balanced data generates unbalanced results. Training up single star models to verify this
• Also, write a first pass on the introduction that uses the vegetarian Yelp as an example, and then set up to explain the method
• Do I still need to train x-star models?

SBIRs

• 9:15 Standup
• 11:00 LAIC
• Get the script running for the current map to show at the meeting today – done!

Book

• Review The Revolt of The Public

…clearly Biden was a net drag on McAuliffe. Overall, Virginians disapproved of Biden’s handling of the presidency by a 10-point margin, with nearly half saying they “strongly disapprove” — double the percentage who strongly approved. Nearly 3 in 10 Virginia voters said their vote was meant to express opposition to Biden, network exit polls found, compared to the 2 in 10 who said their vote was to express support for Biden. The economy was by far the most important issue driving Virginia voters, and people who put the economy at the top of their list favored Youngkin by a dozen percentage points. (Washington Post)

I just found this: https://github.com/google-research/tiny-differentiable-simulator
It appears to be a NN-enhanced physics sim: “TDS can run thousands of simulations in parallel on a single RTX 2080 CUDA GPU at 50 frames per second:“
Here are the relevant papers:

• “NeuralSim: Augmenting Differentiable Simulators with Neural Networks”, Eric Heiden, David Millard, Erwin Coumans, Yizhou Sheng, Gaurav S. Sukhatme. PDF on Arxiv
• “Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap”, RSS 2020 sim-to-real workshop, Eric Heiden, David Millard, Erwin Coumans, Gaurav Sukhatme. PDF on Arxiv and video
• “Interactive Differentiable Simulation”, 2020, Eric Heiden, David Millard, Hejia Zhang, Gaurav S. Sukhatme. PDF on Arxiv

I also found this MIT thesis from 2019: Augmenting physics simulators with neural networks for model learning and control

GPT Agents

• Finished training the balanced model and am re-running the original prompts
• A really negative prompt will produce a low review distribution. Here’s an example of GPT generating reviews in response to a slightly negative set of prompts ([there are absolutely no vegetarian options], [there is not a single vegetarian option on the menu], [the menu has no vegetarian options]), compared with the ground truth of the Yelp database returning reviews and ratings that match the string ‘%no vegetarian options%‘:

• The distribution of star ratings is obviously different too:

• As you can see on the right, the ground truth is distinctly different. The correlation coefficient between the two distributions on the right is -0.4, while it’s well above 0.9 when comparing any of the three distributions to the left.
• So it’s clear that the model has a bias towards positive reviews. In fact, if you look at the baseline distribution from the first 1,000 reviews of restaurants in the ‘American’ category, we can see the underlying distribution that the model was trained on:

• The new question to answer is what happens to the responses when the training data is balanced for stars? Also, I realize that I need to run a pass through the models with just a ‘review:‘ prompt.
• Dammit, the ‘balanced’ training corpora isn’t. Need to fix that and re-train

• 4:15 Meeting

SBIRs

• MDA costing meeting
• Work on building first pass map. It’s actually working pretty well! Need to write an example script for tomorrow
• Need to create some views

GPT Agents

• Create balanced (20k each) star corpora and train – done
• Create low star corpora and train (1, 2, 3?)
• Installed sentence-transformers, which probably broke sentiment.

SBIRs

• Integrate TextComparePopup and try making a map. I’m pretty sure that there will be issues about putting topics into groups and listing topics from different groups – done, and seems to be working well. Tomorrow we try for real?
• 9:15 standup
• Finished the Great Timesheet Update! Hopefully

Chase Dispute team 9:00 – 9:00 1 888 489 8452

Just Landscaping: (443) 251-2188

BB Infinite: 866.865.3335

Societies change their minds faster than people do

GPT Agents

• Spreadsheets for vegetarian 100k GT vs GT vs synth. Everything is good except for ‘no vegetarian options’ It’s the only options that does not appear in the first 100k rows. Going to try some longer prompts to see if I can nudge the model in a better direction. Do that at lunch
• Hmmm. I can’t seem to produce a negative star distribution:

• Meeting with Andreea? Yup. Good chat

SBIRs

• Integrate text similarity into popup widget

GPT Agents

• Extracting the vegetarian synthetic reviews so I can see if they worked. It actually looks pretty good!

• Going to run a range of prompts:

probe_list = ["no vegetarian options", "some vegetarian options", "several vegetarian options", "many vegetarian options"]

• Building the db table/store for the NZ tweets – done. After screwing up the insert arguments, I’m running a 1k set of synthetic tweets to evaluate the test field.

As a J&J recipient, today is booster day! I will become a cocktail of J&J/Moderna antibodies at 10:00. Hopefully my wifi reception will improve dramatically with these new chips.

• Leave NLT 9:15

GPT Agents

• Run some other variations, or show that the usage of “several vegetarian options” is normal speech. The phrase ‘%vegetarian options%’ has 79 reviews in the training set and 4,911 in the holdout set. Going to do a quick boxplot to see if there’s much difference.
• Pretty much what I expected – very hard to evaluate any difference:

• Running ‘some vegetarian options’ and ‘no vegetarian’ on the 100k American model. It turns out that I ran the default ‘review:’ probe yesterday

SBIRs

• Getting the separate behavior for the cmdr and sbrd nodes to work. Rather than having them bounce around, I need to have them head to targets at a speed. That means rewriting bits of Moveable node. Done!

SBIRs

• Added a boxplot to the TextSimilarity framework. Might show at the meeting today?

• Get the script tied into node display. Having some issues. Thinking about having an animated commander and subordinate node moving across the map
  • Set enum types for ForceNodes and MoveableNodes. Tomorrow I’ll add cmdr and sbrd nodes and try moving them around. This will need a setup() method in ScriptFrame that adds them as MoveableNodes
  • Here’s how to raise these objects. Also put the lines below the nodes: stackoverflow.com/questions/10959858/tkinter-canvas-move-item-to-top-level
• LAIC meeting
• Meeting with Aaron about TextSimilarity

GPT Agents

• Try to get the query and probe for “%several vegetarian options%” running. This data is used in the “Why use this technique?” section of the introduction.

• Run American 100k model with the prompt “several vegetarian options” and pull into spreadsheet – done. It looks good, too:

• Add more content to the paper container

Book

• Add review – done!

Large Language Models: A New Moore’s Law?

• When evaluating models, you should pick the smallest one that can deliver the accuracy you need. It will predict faster and require fewer hardware resources for training and inference. Frugality goes a long way.

GPT Agents

• Added some content to the paper
• Meeting. Need to compare the stars from something like “%ethnic vegan%” that doesn’t appear much in the training set but shows up significantly in the later data and compare that to the gpt for the prompt “ethnic vegan”

SBIRs

• Stories! Done!
  • Work with Aaron on document similarity
  • Add script section
  • Create initial maps
  • Plus one of heatmap
• Fix bug that doesn’t save details
• Sprint planning
• I have a much bigger application:

• Got a primitive script generator working. Next will be to load it and navigate a map

{
    "time": "Wednesday January 2020 (12:00:00)",
    "role": "Commander",
    "text": "Commander says something",
    "mapgroup": "fff"
},
{
    "time": "Wednesday January 2020 (12:00:00)",
    "role": "Subordinate",
    "text": "Subordinate says something",
    "mapgroup": "ccc"
}

Book

• Add review – nope

Jury Room

• Quick meeting with Panos about turning the manifesto into a CACM article by Jan 1

Call Wegmans about J&J Booster – Done! Appt on Friday

Book

• Add review

GPT Agents

• Set up sections and start the intro

SBIRs

• Sprint demos – done
• Split out the classes in GraphNavigator2
• Talked to Aaron about document similarity
• Talked to John about demo

SBIRs

• Topics are being stored redundantly – fixed!
• Connections source and target id’s are broken – fixed!
• Introduction – first pass!

Book

• Do another review, and do something in the introduction that works with the idea that we are barely individuals. How that changes from childhood through adulthood to old age, and how technology has had a huge impact

SBIRs

• 9:15 Standup
• 3:00 Army
• Timesheets!

GPT-Agents

• Run the new LIWC data and generate two spreadsheets. One with the word numbers and one with the default settings. Done

Getting the RV lifted a few inches and getting better wheels and tires. That should help with the road damage that keeps on happening

SBIRs

• Now that I’m writing projects, try reading them back in. Once that’s done, try to update a project to DB
  • Loading nodes!
• Write out GraphML files for John – done

Neural circuit policies enabling auditable autonomy

• A central goal of artificial intelligence in high-stakes decision-making applications is to design a single algorithm that simultaneously expresses generalizability by learning coherent representations of their world and interpretable explanations of its dynamics. Here, we combine brain-inspired neural computation principles and scalable deep learning architectures to design compact neural controllers for task-specific compartments of a full-stack autonomous vehicle control system. We discover that a single algorithm with 19 control neurons, connecting 32 encapsulated input features to outputs by 253 synapses, learns to map high-dimensional inputs into steering commands. This system shows superior generalizability, interpretability and robust-ness compared with orders-of-magnitude larger black-box learning systems. The obtained neural agents enable high-fidelity autonomy for task-specific parts of a complex autonomous system.

Apparently I have to roll over my 401k? We no longer have a retirement plan?

Baselines for Uncertainty and Robustness in Deep Learning

• In “Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning”, we introduce Uncertainty Baselines, a collection of high-quality implementations of standard and state-of-the-art deep learning methods for a variety of tasks, with the goal of making research on uncertainty and robustness more reproducible. The collection spans 19 methods across nine tasks, each with at least five metrics. Each baseline is a self-contained experiment pipeline with easily reusable and extendable components and with minimal dependencies outside of the framework in which it is written. The included pipelines are implemented in TensorFlow, PyTorch, and Jax. Additionally, the hyperparameters for each baseline have been extensively tuned over numerous iterations so as to provide even stronger results.

Book

• Twitter and Tear Gas

SBIRs

• Adding optional buttons to TopicCombo so it’s possible to add a topic and not set a seed.
• Need to check for the case where I am adding a topic to the group that provided the seed. No need to link to yourself
• Save graph to DB, hopefully
• Woohoo!

• Taking a break
• Saving the full graph to the DB!
• How it looks today

• I’m going to add a button to “find closest group” based on text analysis. Probably start with Doc2vec paragraph embeddings