SBIRs 4:30 – 6:30

• Working on RCSNN diagram

SBIRs

• 12:00 FA2 meeting
• 3:30 Present the AI RoE paper to the data science tagup
• 4:30 LAIC meeting

Book

• I finished Social Dominance last night and I think there might be room for a chapter on how SDT and AI/ML could work together to a) Identify and attenuate runaway HE behavior while also identifying and amplifying nascent or stagnant HA behavior.
• Looks like it’s now called Power Basis Theory?
  • Power Basis Theory: A Psychoecological Approach to Power
  • Power dynamics in intergroup relations
  • Need to read these two and then reach out to Dr. Felicia Pratto at uconn.edu

SBIRs

• Cleaning up minGPT for comprehensibility
• Meeting with Rukan and Aaron. Great progress!
• Working on slide deck for presentation tomorrow

Book

• Finished a pass of some kind and sent off to Wajanat and Aaron
• Fixed the chapter headings
• Reworked the proposal so that it has a new intro and the chapters are in the new order

SBIRs

• 10:00 Meeting with Rukan and Aaron
• Need to download the MinGPT project and see if I can build it. It works! Now I need to load and save the model, then start playing around with the mask
  • Save and load the model
  • Create a reverse model

JuryRoom

• Working with Zach a bit on framing out the concept and how much it might cost
• Meeting with Jarod

SBIRs

• 9:10 Standup
• Set up a meeting with Rukan and Aaron to discuss RCSNN
• Continuing Transformers book

JuryRoom

• Talked to Zach about costing out a MCC-style version

GPT-Agents

• Tweaked things for multiple plots:

• 3:30 Meeting

SBIRs

• Continuing with Transformers book
• Quick meeting with Aaron and Rukan to deal with his training problems
• Grabbed some papers for Steve

Book

• Fixing more chapters that I didn’t realize still sounded like a dissertation

This looks really nice:

Downloading the svn backup – Done!. Going to try to install following these directions: www.if-not-true-then-false.com/2012/svn-subversion-backup-and-restore

SBIRs

• 10:00 Meeting with Rukan
• More Transformers book. Need to look more deeply at MinGPT

GPT Agents

• Now that I have the counts working, need to tie that back into the GPT output. I think I need some Parts-of-speech analysis to figure out what to count. The other part is to use the feedback to determine important points in the GPT response

Data Stuff – It’s stuff I made with data! (@erindataviz)

Tasks

• The Planets
• Spanish – done
• So we find out what’s going on with SVN?
• JCS – done

SBIRs

• 9:15 standup
• Meeting with Aaron
• More Transformers book
  • Chapter 3
  • BertViz: Visualize Attention in Transformer Models (BERT, GPT2, T5, etc.)
  • Found (I think) what I’m looking for: MinGPT: “A PyTorch re-implementation of GPT training. minGPT tries to be small, clean, interpretable and educational, as most of the currently available ones are a bit sprawling. GPT is not a complicated model and this implementation is appropriately about 300 lines of code, including boilerplate and a totally unnecessary custom causal self-attention module.“

GPT Agents

• Continue with TwitterV2 count class. Good progress. I have basic functionality:

Need to work on the queries a bit to get phrases. Actually not hard, you just have to use escaped quotes ‘\”happy new year\”‘:

Looking forward to 2.22.22. Almost as exciting as 11.11.11

This IS VERY COOL!! It’s an entire book written using Jupyter Notebooks that you can read on github: GitHub – fastai/fastbook: The fastai book, published as Jupyter Notebooks

GPT Agents.

• Got the counts query working with only a small amount of googling. The cool thing is that the items come back with a granularity, so this call (which has a default granularity of “day”:

query = "from:twitterdev"
start_time = "2021-05-01T00:00:00Z"
end_time = "2021-06-01T00:00:00Z"
url = create_counts_url(query, start_time, end_time)
json_response = connect_to_endpoint(url)
print_response("Get counts", json_response)

• returns a json object that has the daily volume of tweets from @twitterdev (There were 24 total time periods, and then the total tweet count was 22) :

response:
{
    "data": [
        {
            "end": "2021-05-02T00:00:00.000Z",
            "start": "2021-05-01T00:00:00.000Z",
            "tweet_count": 0
        },

        {
            "end": "2021-05-13T00:00:00.000Z",
            "start": "2021-05-12T00:00:00.000Z",
            "tweet_count": 6
        },
        {
            "end": "2021-05-14T00:00:00.000Z",
            "start": "2021-05-13T00:00:00.000Z",
            "tweet_count": 1
        },
        {
            "end": "2021-05-15T00:00:00.000Z",
            "start": "2021-05-14T00:00:00.000Z",
        {
            "end": "2021-05-21T00:00:00.000Z",
            "start": "2021-05-20T00:00:00.000Z",
            "tweet_count": 8
        },
        {
            "end": "2021-05-29T00:00:00.000Z",
            "start": "2021-05-28T00:00:00.000Z",
            "tweet_count": 2
        },
        {
            "end": "2021-06-01T00:00:00.000Z",
            "start": "2021-05-31T00:00:00.000Z",
            "tweet_count": 0
        }
    ],
    "meta": {
        "total_tweet_count": 22
    }
}

• This is very nice! I’m looking forward to doing some interesting things with the GPT. We can scan through responses to prompts and look at word-by-word Twitter frequencies after stop words, and then use those sentences for further prompting. We can also compare embeddings, cluster and other interesting things

SBIRs

• Working through Natural Language Processing with Transformers
  • Very nice. You can easily get training data here

• Datasets simplifies this process by providing a standard interface for thousands of datasets that can be found on the Hub. It also provides smart caching (so you don’t have to redo your preprocessing each time you run your code)and avoids RAM limitations by leveraging a special mechanism called memory mapping that stores the contents of a file in virtual memory and enables multiple processes to modify a file more efficiently.
• Imbalanced-learn (imported as imblearn) is an open source, MIT-licensed library relying on scikit-learn (imported as sklearn) and provides tools when dealing with classification with imbalanced classes.
• Nice NW job fair

Ack! Dreamhost has deleted my SVN repo. Very bad. Working on getting it back. Other options include RiouxSVN, but it may be moribund. Assembla hosts for $19/month with 500 GB, which is good because I store models. Alternatively, make a svn server, fix the IP address, and have it on Google Drive, OneDrive, or DropBox.

Spanish

SBIRs

• Stories
• 9:15 Sprint planning
• Going to spend most of the sprint building a GPT-2 from scratch and then figuring out how to reverse it
• Got Natural Language Processing with Transformers 1st Edition, Kindle Edition
• Downloaded notebooks from GitHub

GPT Agents

• 3:30 Meeting
  • I think the upshot is to 1) Get the embedding topic narrative thing working, then run the gpt to generate keywords and count their occurrence on Twitter

Sharpened Cosine Similarity (CosSim) is an alternative to Convolution for building features in neural networks. It performs as well as ConvNets with 10x-100x more parameters.

Current Stereotypes: A Little Fading, a Little Faking

• Examined the possibility that social-desirability-tainted responses emerge in the study of stereotypes. 60 white male undergraduates were randomly assigned to 1 of 4 experimental conditions. Ss were asked to indicate how characteristic each of 22 adjective traits was of either “Americans” or “Negroes.” 1/2 the Ss responded in a rating situation in which they were presumably free to distort their responses. The remaining Ss responded under “bogus pipeline” conditions; i.e., they were led to believe that the experimenter had an accurate, distortion-free physiological measure of their attitudes, and were asked to predict that measure. Results support the expectation that the stereotype ascribed to Negroes would be more favorable under rating than under bogus pipeline conditions. Americans were more favorably stereotyped under bogus pipeline than under rating conditions. A number of explanations for these results are discussed, and consideration is given to the relationship between verbally expressed attitudes and other, overt, behavior.

Social physics

• Recent decades have seen a rise in the use of physics methods to study different societal phenomena. This development has been due to physicists venturing outside of their traditional domains of interest, but also due to scientists from other disciplines taking from physics the methods that have proven so successful throughout the 19th and the 20th century. Here we characterise the field with the term ‘social physics’ and pay our respect to intellectual mavericks who nurtured it to maturity. We do so by reviewing the current state of the art. Starting with a set of topics that are at the heart of modern human societies, we review research dedicated to urban development and traffic, the functioning of financial markets, cooperation as the basis for our evolutionary success, the structure of social networks, and the integration of intelligent machines into these networks. We then shift our attention to a set of topics that explore potential threats to society. These include criminal behaviour, large-scale migration, epidemics, environmental challenges, and climate change. We end the coverage of each topic with promising directions for future research. Based on this, we conclude that the future for social physics is bright. Physicists studying societal phenomena are no longer a curiosity, but rather a force to be reckoned with. Notwithstanding, it remains of the utmost importance that we continue to foster constructive dialogue and mutual respect at the interfaces of different scientific disciplines.

This is really clever: How does fake news spread? Understanding pathways of disinformation spread through APIs

• What are the pathways for spreading disinformation on social media platforms? This article addresses this question by collecting, categorizing, and situating an extensive body of research on how application programming interfaces (APIs) provided by social media platforms facilitate the spread of disinformation. We first examine the landscape of official social media APIs, then perform quantitative research on the open‐source code repositories GitHub and GitLab to understand the usage patterns of these APIs. By inspecting the code repositories, we classify developers’ usage of the APIs as official and unofficial, and further develop a four‐stage framework characterizing pathways for spreading disinformation on social media platforms. We further highlight how the stages in the framework were activated during the 2016 US Presidential Elections, before providing policy re-commendations for issues relating to access to APIs, algorithmic content, advertisements, and suggest rapid response to coordinate campaigns, development of collaborative, and participatory approaches as well as government stewardship in the regulation of social media platforms.

The Wikipedia folks have produced a very clear Precision/Recall diagram!

SBIRs

• Slides for demo
• 9:00 Demos
• 2:00 Meeting with Rukan
• Natural Language Processing with Transformers Book
  • Train transformers from scratch and learn how to scale to multiple GPUs and distributed environments

Book

• More work on intro

GPT Agents

• Work on Twitter queries

SBIRs

• Had a nice chat with Rukan about how to partition models. Which made me think about RCS again. Maybe make a pyRCS library? I have the code written already, just need to pull it out of the PyBullet project
• Finished the LAIC roadmap paper
• Need to do some more book new chapter one still

Dump and shop before 4:00

America Has Split, and It’s Now in ‘Very Dangerous Territory’

• Polarization has become a force that feeds on itself, gaining strength from the hostility it generates, finding sustenance on both the left and the right. A series of recent analyses reveals the destructive power of polarization across the American political system.

GPT Agents

• I think OpenAI’s embeddings may have gone public – Yes!
  • Introducing Text and Code Embeddings in the OpenAI API
  • Documentation
  • New names for models
  • Get_embeddings.ipynb
• Here’s an example of calling the embedding

oai = OpenAIComms()
result = oai.get_embedding('hello, world', 'text-similarity-ada-001')
print(result)

• Here’s the result

[0.012463847, 0.02531687, -0.0059246803, 0.022367332, 0.037196957, 0.013784995, 0.019438276, -0.0075837956, 0.012187328, -0.014604311, 0.013569924, -0.022551678, 0.025398802, -0.015515801, -0.005586712, -0.04231768, -0.046905853, -0.025583148, 0.006472598, -0.0036203535, 0.036684882, 

--------------- Lines removed because we don't need to see every embedding ----

-0.015310971, 0.00073034357, -0.013856685, -0.00026291728, -0.0049056555, 0.024436105, -0.0086181825, -0.023248097, 0.008290456, 0.012443365, -0.020278076, 0.024169827, -0.012361433, -0.057515997, 0.045103356, 0.04752034, -0.008510647, -0.05014215, 0.012279501, 0.013979582, 0.05182175, 0.03209671, -0.008920305]

• Also, you can store JSON object in MySql/MariaDB as text. Here’s an example table:

• Here’s how you insert, create a view, and then select an element from that view

insert into table_json (embedding) values ('{"foo":12, "bar":2}');

create or replace view view_json as
    select id, json_value(embedding, '$.foo') as foo, json_value(embedding, '$.bar') as bar from table_json;

select id, foo from view_json;

• Which results in this:

SBIRs

• Need to create a story for GSAW deck
• 9:15 standup
• LAIC prep
• 11:00 LAIC discussion
• Aligning Language Models to Follow Instructions
  • We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API.

Book?

GPT Agents

• Had a long and winding talk about quality in Twitter data and whether using thread is a way to increase that. Shimei’s thought is that it will bias the data towards a different population. I think that’s reasonable, but I’m not sure that matters as long as you specify what population you’re polling.
• Got the recent conversation search working
• Working on historical queries
  • Getting historical Tweets using the v2 full-archive search endpoint

SBIRs

• Slides for GSAW