MuZero: The Walkthrough (Part 1/3)

• Teaching a machine to play games using self-play and deep learning…without telling it the rules

This looks super interesting for building domain-specific belief maps:

Here’s a link to the paper: DEMix Layers: Disentangling Domains for Modular Language Modeling

• We introduce a new domain expert mixture (DEMix) layer that enables conditioning a language model (LM) on the domain of the input text. A DEMix layer is a collection of expert feedforward networks, each specialized to a domain, that makes the LM modular: experts can be mixed, added or removed after initial training. Extensive experiments with autoregressive transformer LMs (up to 1.3B parameters) show that DEMix layers reduce test-time perplexity, increase training efficiency, and enable rapid adaptation with little overhead. We show that mixing experts during inference, using a parameter-free weighted ensemble, allows the model to better generalize to heterogeneous or unseen domains. We also show that experts can be added to iteratively incorporate new domains without forgetting older ones, and that experts can be removed to restrict access to unwanted domains, without additional training. Overall, these results demonstrate benefits of explicitly conditioning on textual domains during language modeling.
• Git repo: github.com/kernelmachine/demix

GPT Agents

• Get the review extraction working and produce some content. Got everything running and generating 10,000 reviews. We’ll see how the pattern of stars looks first, and then do a sentiment run on the stored data
• Export the DB and run sentiment analysis

SBIR(s)

• Had a long talk yesterday with Aaron about what to do with MARE. I think it becomes the framework for training and using our enhanced simulation scenario explorer. Basically AlphaZero but for physics-based games like tennis.
• Got Andrew to buy off on the LAIC stories and show me how to put them properly(!) in Jira, so I’ll do that today
• Endless, mind-numbing training
• EXPENSE REPORT

Book

• Skipping this week – Michelle has meetings

Just back from a conference in Huntsville. Lots of very expensive ways to deliver energy to a point in space at a particular time. I need to write up my thoughts in more detail later. Also EXPENSE REPORT!

Announcing AI21 Studio and Jurassic-1 Language Models

• We are thrilled to announce the launch of AI21 Studio, our new developer platform where you can use our state-of-the-art Jurassic-1 language models to build your own applications and services. Jurassic-1 models come in two sizes, where the Jumbo version, at 178B parameters, is the largest and most sophisticated language model ever released for general use by developers. AI21 Studio is currently in open beta, allowing anyone to sign up and immediately start querying Jurassic-1 using our API and interactive web environment.

Research community dynamics behind popular AI benchmarks

• The widespread use of experimental benchmarks in AI research has created competition and collaboration dynamics that are still poorly understood. Here we provide an innovative methodology to explore these dynamics and analyse the way different entrants in these challenges, from academia to tech giants, behave and react depending on their own or others’ achievements. We perform an analysis of 25 popular benchmarks in AI from Papers With Code, with around 2,000 result entries overall, connected with their underlying research papers. We identify links between researchers and institutions (that is, communities) beyond the standard co-authorship relations, and we explore a series of hypotheses about their behavior as well as some aggregated results in terms of activity, performance jumps and efficiency. We characterize the dynamics of research communities at different levels of abstraction, including organization, affiliation, trajectories, results and activity. We find that hybrid, multi-institution and persevering communities are more likely to improve state-of-the-art performance, which becomes a watershed for many community members. Although the results cannot be extrapolated beyond our selection of popular machine learning benchmarks, the methodology can be extended to other areas of artificial intelligence or robotics, and combined with bibliometric studies.

The Learning on Graphs and Geometry Reading Group

Alpha Zero’s “Alien” Chess Shows the Power, and the Peculiarity, of AI

• What’s also remarkable, though, Hassabis explained, is that it sometimes makes seemingly crazy sacrifices, like offering up a bishop and queen to exploit a positional advantage that led to victory. Such sacrifices of high-value pieces are normally rare. In another case the program moved its queen to the corner of the board, a very bizarre trick with a surprising positional value. “It’s like chess from another dimension,” Hassabis said.

SBIR

• Standup – done
• Respond to Steve – done multiple
• Schedule story time with Andrew – done. Now I just need to put them in Jira
• Schedule golf with Aaron? Done! Sim first (using MARE and enhanced sim), then prototype, then build a trade show version (indoor so no weather), then try fielding at some willing golf course? Paul could probably help with that

Nice ride on Saturday. An 18mph average pace and I still got dropped by the lead group! But I did hang on for over 40 miles

Book

• Want a \TODO{write something here} that can disappear as needed? Use these two versions of TODO:

%\newcommand\TODO[1]{\textcolor{red}{(TODO: #1)}} % show
\newcommand\TODO[1]{} % hide

SBIRs

• Go over stories with Aaron?
• MARCOM meeting
• Off to the SMD symposium

GPT Agents

• Setting up the DB to handle sentiment and PoS – done
• Generating and parsing the review/stars model. When there is an exception thrown while debugging, the IDE loses the ability to edit?

There is a version of DALL-E at huggingface for image to text! (huggingface.co/spaces/flax-community/dalle-mini)

A man in a room:

A woman in a room:

Need to fix my timesheet for Monday

A Network Framework of Cultural History

• The emergent processes driving cultural history are a product of complex interactions among large numbers of individuals, determined by difficult-to-quantify historical conditions. To characterize these processes we have reconstructed aggregate intellectual mobility over two millennia through the birth and death locations of more than 150,000 notable individuals. The tools of network and complexity theory were then used to identify characteristic statistical patterns and determine the cultural and historical relevance of deviations. The resulting network of locations provides a macroscopic perspective of cultural history, which helps us to retrace cultural narratives of Europe and North America using large-scale visualization and quantitative dynamical tools and to derive historical trends of cultural centers beyond the scope of specific events or narrow time intervals.

Had to get my truck serviced yesterday (oil change and recalls) which took a bunch of hours, so I brought my bike and went on a really nice ride on a wonderful day

Speaking of the truck. These folks (103 Creek Ridge Road,Greensboro, North Carolina 27406) will install lift kits from these folks. I could also do wheels and tires. Stay at Haw River State Park?

Book

• Put the proposal in the Overleaf folder using proposal.tex as the root document
• 2:00 Meeting. We are getting very close! I need to make the TODOs vanish

SBIRs

• The Delta tix did not save to PDF worth a damn, so I created a new document with screenshots that didn’t suck. Delta is horrible and expensive. I think if I have to go to Huntsville again I’ll try to take the train
• Steve has many questions. Did some answering and pointed him at Microsoft Flight Simulator, which is getting more amazing all the time. It’s over 40 years old!

More stupid travel stuff. Clay suggests American for next time

Finished Stewardship of global collective behavior. It’s quite good and a nice way to frame all this research

• Collective behavior provides a framework for understanding how the actions and properties of groups emerge from the way individuals generate and share information. In humans, information flows were initially shaped by natural selection yet are increasingly structured by emerging communication technologies. Our larger, more complex social networks now transfer high-fidelity information over vast distances at low cost. The digital age and the rise of social media have accelerated changes to our social systems, with poorly understood functional consequences. This gap in our knowledge represents a principal challenge to scientific progress, democracy, and actions to address global crises. We argue that the study of collective behavior must rise to a “crisis discipline” just as medicine, conservation, and climate science have, with a focus on providing actionable insight to policymakers and regulators for the stewardship of social systems.

Put my bids in for ICTAI-2021 reviews

GPT Agents

• Building 6-epoch review, stars model – done! Need to verify they work

SBIR

• Phase II presubmission meeting (One hr)
• Working on Epic/stories for map app
• React and Flask
  • blog.miguelgrinberg.com/post/how-to-create-a-react–flask-project
  • towardsdatascience.com/build-deploy-a-react-flask-app
  • medium.com/analytics-vidhya/simple-chat-app-with-react-flask
• Django
  • realpython.com/get-started-with-django
  • betterprogramming.pub/prettify-python-django-with-beautiful-visual-charts
• Had a good discussion with Zack, who suggested a stack of Svelt, Sapper, Python and MySQL all connected with ZeroMQ

Examining the consumption of radical content on YouTube

• Daily share of news consumption on YouTube, a social media platform with more than 2 billion monthly users, has increased in the last few years. Constructing a large dataset of users’ trajectories across the full political spectrum during 2016–2019, we identify several distinct communities of news consumers, including “far-right” and “anti-woke.” Far right is small and not increasing in size over the observation period, while anti-woke is growing, and both grow in consumption per user. We find little evidence that the YouTube recommendation algorithm is driving attention to this content. Our results indicate that trends in video-based political news consumption are determined by a complicated combination of user preferences, platform features, and the supply-and-demand dynamics of the broader web.

GPT Agents

• I now have 3 and 6 epoch runs for name, review, stars models.
• Evaluate stars to see how much has changed
• Maybe try to train up a bigger model? Start with the xl model and step back to find the largest model that will fit. Then train that with the name, review, stars corpora
  • Nope, the 117m model is the biggest that will fit. When I’ve got the time try the Huggingface Course and see how to do cloud training
• 3:00 Meeting. Went over results and the mapping tool proposal
  • Need to adjust the counts to relative percent for easier compare
  • Try training a model from scratch on the stars/votes corpora? Thaty way we could see if it learns the ratios better. This could be an artifact from finetuning
  • Create models for review+star since the name sets up the review

SBIRs

• Sprint planning
  • Plan LM Epic – DSR-646
  • SMD conference – DSR-645
• Long-ish chat with Rukan about transforms in scene graphs

Set up oil change and recall service

GPT Agents

• Running the ensemble of 3-epoch models to see how much variation there is
• Create review corpora
• Train model(s?). Start with the default 3 epoch since that seems to work well

SBIR

• Sprint demos
• Meeting with Steve and Rukan about next steps
• Last-second changes about the SMD trip. I swear that I am never going to speak at a conference that doesn’t have a clear and easy to find schedule
• Need to come up with stories

I had some perplexing runs trying different epoch counts on corpora recently. Here’s the 6k:

As you can see, only the 32 epoch can figure out there are no 0 star ratings, though the best overall match is the 16 epoch.

The 100k is weirder, and makes more sense when you look at the raw data:

As you can see, the 16 and 32 epoch miss the two star rating entirely, though aside(!) from that the fit isn’t bad:

Looking at this, I’d say that 2-4 epochs seem to work well, and when I take out the explicit epoch, run_clm determines the epochs should be 3:

[INFO|trainer.py:1164] 2021-07-31 06:31:02,447 >> ***** Running training *****
[INFO|trainer.py:1165] 2021-07-31 06:31:02,447 >>   Num examples = 1649
[INFO|trainer.py:1166] 2021-07-31 06:31:02,448 >>   Num Epochs = 3
[INFO|trainer.py:1167] 2021-07-31 06:31:02,448 >>   Instantaneous batch size per device = 1
[INFO|trainer.py:1168] 2021-07-31 06:31:02,449 >>   Total train batch size (w. parallel, distributed & accumulation) = 1
[INFO|trainer.py:1169] 2021-07-31 06:31:02,449 >>   Gradient Accumulation steps = 1
[INFO|trainer.py:1170] 2021-07-31 06:31:02,449 >>   Total optimization steps = 4947
{'loss': 0.1737, 'learning_rate': 4.494643218111988e-05, 'epoch': 0.3}

I’m going to try an ensemble of these 3-epoch models to see what that looks like

Pointer Value Retrieval: A new benchmark for understanding the limits of neural network generalization

• The successes of deep learning critically rely on the ability of neural networks to output meaningful predictions on unseen data — generalization. Yet despite its criticality, there remain fundamental open questions on how neural networks generalize. How much do neural networks rely on memorization — seeing highly similar training examples — and how much are they capable of human-intelligence styled reasoning — identifying abstract rules underlying the data? In this paper we introduce a novel benchmark, Pointer Value Retrieval (PVR) tasks, that explore the limits of neural network generalization. While PVR tasks can consist of visual as well as symbolic inputs, each with varying levels of difficulty, they all have a simple underlying rule. One part of the PVR task input acts as a pointer, giving the location of a different part of the input, which forms the value (and output). We demonstrate that this task structure provides a rich testbed for understanding generalization, with our empirical study showing large variations in neural network performance based on dataset size, task complexity and model architecture. The interaction of position, values and the pointer rule also allow the development of nuanced tests of generalization, by introducing distribution shift and increasing functional complexity. These reveal both subtle failures and surprising successes, suggesting many promising directions of exploration on this benchmark.

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

• This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub “prompt-based learning”. Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P(y|x), prompt-based learning is based on language models that model the probability of text directly. To use these models to perform prediction tasks, the original input x is modified using a template into a textual string prompt x’ that has some unfilled slots, and then the language model is used to probabilistically fill the unfilled information to obtain a final string x, from which the final output y can be derived. This framework is powerful and attractive for a number of reasons: it allows the language model to be pre-trained on massive amounts of raw text, and by defining a new prompting function the model is able to perform few-shot or even zero-shot learning, adapting to new scenarios with few or no labeled data. In this paper we introduce the basics of this promising paradigm, describe a unified set of mathematical notations that can cover a wide variety of existing work, and organize existing work along several dimensions, e.g.the choice of pre-trained models, prompts, and tuning strategies. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured typology of prompt-based concepts, but also release other resources, e.g., a website this http URL including constantly-updated survey, and paperlist.

GPT Agents

• This looks interesting for paragraph clustering: Sentence Transformers in the Hugging Face Hub
  • Sentence Transformers is a framework for sentence, paragraph and image embeddings. This allows to derive semantically meaningful embeddings (1) which is useful for applications such as semantic search or multi-lingual zero shot classification. As part of Sentence Transformers v2 release, there are a lot of cool new features:
    • Sharing your models in the Hub easily.
    • Widgets and Inference API for sentence embeddings and sentence similarity.
    • Better sentence-embeddings models available (benchmark and models in the Hub).
• Finished the 6k epoch tests yesterday. Maybe finish creating models for 100k today?

SBIR

• Abstract! Done!
• See how Rukan is doing. Tell him about cubes and other shapes – done. Looks good

Book

• Meeting with Michelle at 2:00. Worked on the positioning statement. It’s almost ready to go to an agent!

4:00 NLP Meeting – cancelled

When the Echo Chamber Shatters: Examining the Use of Community-Specific Language Post-Subreddit Ban

• Community-level bans are a common tool against groups that enable online harassment and harmful speech. Unfortunately, the efficacy of community bans has only been partially studied and with mixed results. Here, we provide a flexible unsupervised methodology to identify in-group language and track user activity on Reddit both before and after the ban of a community (subreddit). We use a simple word frequency divergence to identify uncommon words overrepresented in a given community, not as a proxy for harmful speech but as a linguistic signature of the community. We apply our method to 15 banned subreddits, and find that community response is heterogeneous between subreddits and between users of a subreddit. Top users were more likely to become less active overall, while random users often reduced use of in-group language without decreasing activity. Finally, we find some evidence that the effectiveness of bans aligns with the content of a community. Users of dark humor communities were largely unaffected by bans while users of communities organized around white supremacy and fascism were the most affected. Altogether, our results show that bans do not affect all groups or users equally, and pave the way to understanding the effect of bans across communities.

SBIR

• 9:15 standup – nope
• Write abstract!

GPT Agents

• Creating spreadsheets to see how the different epochs affect accuracy in addition to format.
• Running all the models (1 – 32 epochs) and putting the results in the database

4:30 NLP Meeting

Add this to the Saturday ride calendar

How epidemic psychology works on Twitter

• During 2020, based on changes on the use of language on Twitter, three distinct phases were identified. The first was the refusal phase: people in the US refused to accept reality despite the increasing numbers of deaths in other countries. The second was the anger phase, started after the announcement of the first death in the country: people’s fear translated into anger about the looming feeling that things were about to change. The third phase was the acceptance phase, started after the authorities imposed physical-distancing measures: people found a “new normal” for their daily activities. During the year, as cases surged in waves, so did anger, re-emerging cyclically at each wave. These results suggest the concrete future possibility of embedding epidemic psychology derived from the use of language on social media into more traditional epidemiological models.

Research note: Examining potential bias in large-scale censored data

• We examine potential bias in Facebook’s 10-trillion cell URLs dataset, consisting of URLs shared on its platform and their engagement metrics. Despite the unprecedented size of the dataset, it was altered to protect user privacy in two ways: 1) by adding differentially private noise to engagement counts, and 2) by censoring the data with a 100-public-share threshold for a URL’s inclusion. To understand how these alterations affect conclusions drawn from the data, we estimate the prevalence of fake news in the massive, censored URLs dataset and compare it to an estimate from a smaller, representative dataset. We show that censoring can substantially alter conclusions that are drawn from the Facebook dataset. Because of this 100-public-share threshold, descriptive statistics from the Facebook URLs dataset overestimate the share of fake news and news overall by as much as 4X. We conclude with more general implications for censoring data.

SBIR(s)

• SMD: Register for conference, book hotel and flight – Done. That took hours! There are few flights to Huntsville, and I had to use Delta, which doesn’t integrate with car rental. A lot of hotels were booked, too.
• LAIC: Finish writeup. Had a meeting with Aaron about what was needed, and then we worked on his SPIE presentation
• IRAD: Coordinate with Rukan. We reworked the JSON file so that it is human readable. Rukan will use this as a basis for rendering the targets and platforms, as well as running the scenarios. Once that’s working we’ll add ordinance

GPT Agents

• Set up finetuning folder. See if it works with the current setup and upgrade Pytorch, TF, and Transformers as needed. Could not get that to work, so I went back to the run_clm CI approach. I rebased the transformers project, and found that there are now TF and Pytorch versions. I’m using Pytorch for this.
• Try training using the 6k corpus for fixed epochs.
• Built models for 1, 2, 4, 8, 16, and 32 epochs. You can see the formatting results improve until 16 epochs. I need to build spreadsheets that show the values and compare them to the ground truth to see if this is a better approximation. THen move on to larger corpora

JuryRoom

• 7:00 Meeting

In a fake battle for Taiwan, U.S. forces lost network access almost immediately.

Curation Bubbles: Domain Versus URL Level Analysis of Partisan News Sharing on Social Media

• Empirical inquiries of political news consumption are typically based on analysis at the level of the news source: a given web domain can be assigned a partisanship score reflective of its relative tendency to be shared by Democrats or Republicans. This practical, tractable approach represents an important methodological advance which has allowed for large-scale empirical studies of how democratic citizens consume political information online. However, despite strong evidence that information sharing is dominated by in-group bias, previous work has also found that most users are exposed to information from a balanced variety of mainstream sources. Such conflicting findings around filter bubbles and echo chambers highlights the need to be able to estimate partisanship at the more fine-grained level of individual stories. It may be that individuals tend to consume politically homogeneous content which originates from a relatively heterogeneous collection of sources. Rather than never sharing stories associated with their political opponents, partisans may selectively share out-group content precisely when that information is favorable to them. Using a panel of 1.6 million Twitter users linked to administrative data, we test this dynamic by examining within-domain sharing patterns by user partisanship over time. Consistent with previous work, we find that, in aggregate, partisans do consume news from a variety of sources. However, we find notable story-level differences suggesting that, despite the heterogeneity of sources, the news curated from partisan’s social networks contains politically homogeneous information. Our findings suggest that domain-level analyses of information sharing gives a false impression of exposure to politically diverse content, and raises new concerns regarding polarization in the consumption and sharing of digital media
• This really fits with my experience, where Fox News viewers share links to the NYTimes that are mentioned on Fox, often without reading them

GPT-Agents

• Add 200k data to rollup spreadsheet
• Here’s the 200k added to the stars counts for each model vs the Yelp 75k ground truth

• It seems to be better at the lower star counts, but worse at 5. To make sure this wasn’t an artifact of the training data, here’s a measure of the error vs the specific data used to create the training corpora:

• 3:30 Meeting
  • Have the same size corpora (100k) and the same number of training steps, and prompt with “stars = “.
  • Fine-tuning a pretrained model: In this tutorial, we will show you how to fine-tune a pretrained model from the Transformers library. In TensorFlow, models can be directly trained using Keras and the fit method. In PyTorch, there is no generic training loop so the 🤗 Transformers library provides an API with the class Trainer to let you fine-tune or train a model from scratch easily. Then we will show you how to alternatively write the whole training loop in PyTorch.

SBIRs:

• 9:15 Standup
• More proposal?
• Loop back to the simulator and Rukan
• Start working on LAIC tasking splits for IP

There is this thing called PyTorch Lightning that supports scaling of your models and integrates with Grid.AI

Grid.AI is designed for developing and training deep learning models at scale.

• Create a DATASTORE with your dataset.
• Spin up an interactive SESSION to develop, analyze and prototype models/ideas.
• When you have something that works, train it at scale via RUN.
• Grid allocates all the machines and GPUs you need on demand, so you only pay for what you need when you need it.

SBIR(s)

• Talk to Rukan about the simulator – brief discussion, then pulled back into the proposal
• Work on slides – finished first draft

Book

• Good chat with Michelle on Friday. The proposal is coming along!

GPT-Agents

• Finished the 200k Yelp run
• 4:30 Meeting with Andreea. Nice discussion. We might be able to set up some research that involves understanding the use of metaphor by the GPT