Category Archives: Phil

Phil 6.9.2022

Call about bike! Does it pass inspection? Nope. Selling it to Bobs for very little


  • Finished up the last edits and tweaks for this version and sent out some copies to readers.
  • I need to look at what is needed to submit to Cambridge and Stripe. Might as well try Oxford again. I will need to update the proposal


  • More FMDS

GPT Agents

  • While waiting for Aaron, start getting the pulls working.
    • If NOT randomized, then pull the tweets in sequential order. I think that this can use the query token and just stop when the end is reached.
    • If it is randomized, then randomly select within the span and then pull in sequential order.
    • Make sure that the beginning of the NEXT sequence does not re-use rollover tweets from the previous sequence. If it does, then throw a warning and use the timestamp of the last tweet in the previous pull.
  • Set up the schema and tables. We can start with tweet_table, and add a user table later

Phil 6.8.2022

Towards Learning Universal Hyperparameter Optimizers with Transformers

  • Meta-learning hyperparameter optimization (HPO) algorithms from prior experiments is a promising approach to improve optimization efficiency over objective functions from a similar distribution. However, existing methods are restricted to learning from experiments sharing the same set of hyperparameters. In this paper, we introduce the OptFormer, the first text-based Transformer HPO framework that provides a universal end-to-end interface for jointly learning policy and function prediction when trained on vast tuning data from the wild. Our extensive experiments demonstrate that the OptFormer can imitate at least 7 different HPO algorithms, which can be further improved via its function uncertainty estimates. Compared to a Gaussian Process, the OptFormer also learns a robust prior distribution for hyperparameter response functions, and can thereby provide more accurate and better calibrated predictions. This work paves the path to future extensions for training a Transformer-based model as a general HPO optimizer.


  • Head down on FMDS

Phil 6.6.2022

It was a weekend of perfect weather


  • Continuing through the conspiracy section

GPT Agents

  • Work on getting keyword tweets in fine granular samples (e.g. 5 minutes four times a day at semi-random intervals. Mostly done, though there are all kinds of odd behaviors that involve making too many requests. Working through the options.
  • Compare proportions of samples to full counts for the same time periods
  • Train a model!


  • Sprint review – done
  • Write up stories. Probably go back to RCSNN – done

Phil 6.4.2022

Four levers of reciprocity across human societies: concepts, analysis and predictions

  • This paper surveys five human societal types – mobile foragers, horticulturalists, pre-state agriculturalists, state-based agriculturalists and liberal democracies – from the perspective of three core social problems faced by interacting individuals: coordination problems, social dilemmas and contest problems. We characterise the occurrence of these problems in the different societal types and enquire into the main force keeping societies together given the prevalence of these. To address this, we consider the social problems in light of the theory of repeated games, and delineate the role of intertemporal incentives in sustaining cooperative behaviour through the reciprocity principle. We analyse the population, economic and political structural features of the five societal types, and show that intertemporal incentives have been adapted to the changes in scope and scale of the core social problems as societies have grown in size. In all societies, reciprocity mechanisms appear to solve the social problems by enabling lifetime direct benefits to individuals for cooperation. Our analysis leads us to predict that as societies increase in complexity, they need more of the following four features to enable the scalability and adaptability of the reciprocity principle: nested grouping, decentralised enforcement and local information, centralised enforcement and coercive power, and formal rules.

Doing this today!

Phil 6.2.2022

There is something really deep in that kind of thinking. It would be a micro stampede for sure. Could the AI herd the person into a harmless area? Would that be ethical?

Depolarization of echo chambers by random dynamical nudge

  • In social networks, users often engage with like-minded peers. This selective exposure to opinions might result in echo chambers, i.e., political fragmentation and social polarization of user interactions. When echo chambers form, opinions have a bimodal distribution with two peaks on opposite sides. In certain issues, where either extreme positions contain a degree of misinformation, neutral consensus is preferable for promoting discourse. In this paper, we use an opinion dynamics model that naturally forms echo chambers in order to find a feedback mechanism that bridges these communities and leads to a neutral consensus. We introduce the random dynamical nudge (RDN), which presents each agent with input from a random selection of other agents’ opinions and does not require surveillance of every person’s opinions. Our computational results in two different models suggest that the RDN leads to a unimodal distribution of opinions centered around the neutral consensus. Furthermore, the RDN is effective both for preventing the formation of echo chambers and also for depolarizing existing echo chambers. Due to the simple and robust nature of the RDN, social media networks might be able to implement a version of this self-feedback mechanism, when appropriate, to prevent the segregation of online communities on complex social issues.


  • Finish 1st quarterly status report – done!
  • Help Aaron on JSC – done!

Phil 5.31.2022

Call Barbara after 10:00


  • Working on Money chapter
  • Chat with Mike today? Good discussion. Most important is to put a summary at the end of each chapter


  • Multiple meetings on getting the server up and running
  • Getting content from Rukan and Loren
  • Some discussion with Aaron on the JSC. Need to go over the COPERNICUS paper tomorrow morning

GPT Agents

  • Decided to shelve keywords for a while and get back to pulling tweets. Need to get that part of the API working (keyword list, location, start/stop times). I think it should be possible to get a valid sample by just limiting the duration of the sample, so something like 24 5-minute samples per day? Need to see if that is possible.

Phil 5.27.2022

My contribution to the mass shooting discussion. Let’s try placing taxes on 2nd amendment products (guns, ammo, etc.) based on the number killed and wounded in the last, say, 100 days. For each person (or child) killed, add 10% each, and for each person (or child) wounded add 5% maybe? Seems reasonable, no? If no one is killed or injured in a mass shooting in the last 100 days, no taxes! Just leave it up to the manufacturers and gun owners to decide what they need to do to keep their taxes down. After all, they keep telling us they understand the problem better than anyone.

Maybe we use the proceeds for funding free mental healthcare for all? After all, that’s the current excuse for gun violence.

What would that look like? Well, using the handy list of mass shootings in the USA in 2022, we can work this out:

  • 221 people were killed and 824 wounded in 191 mass shootings in the last 100 days.
  • That means the tax right now for guns and ammo should be 6,330%.
  • A typical .223 bullet, like the ones used in the AR-15, normally costs about $1.75. With the murder tax, those bullets would cost $110.78 each.

I think the problem would be fixed. Probably within 100 days. No other laws required.


  • Finished Interview with a biased Machine and started on The Spacecraft of Babel
  • Made a cool cover


  • Finish writing up RCSNN progress – done!


  • 1:00 Librarian meeting about keyword search

Phil 5.26.2022

Exposing the fetish of right-wing politics and how liberals can fight back

  • Liberals should pay less attention to what right-wingers say and more attention to what they mean. Liberals should presume the principle there is the one that isn’t there. And they should spell it out for normal people in order to ask if this is the kind of country they want to live in.
  • This article is right in line with my thinking on dominance displays


  • Standup
  • Meeting with Rukan. Went over the design of the config mgr and set up for adding autoencoding sections to the quarterly report
  • Refactored the quarterly report to match previous submissions
  • Sent email to Dr. J to see if he’d like a presentation as well
  • JSC meeting with Aaron this afternoon


  • Finished section 1! Need to send it out to some folks

Phil 5.25.2022

Need to write some code that lets me play with a bullet tax based on this insanity

GPT Agents

  • Nice meeting with Jimmy and Shimei. Need to contact a librarian to get insights for overall keyword search, and get a first pass of the protocol done to try next week
  • Got one of those text-to-image accounts:
A “hello world” program written in shapes and light


  • Working on Hierarchies, Networks, and Technology. Last section of Part I!


  • Send email to Dr. J – oops! Tomorrow
  • Continue on JSC proposal – about 2 hours
  • Read the CDRLs and start the quarterly report – done

Phil 5.23.2022

Just finished up a nice bike vacation:

And I got a chapter cleaned up in the book! Need to photoshop a few pix

Simple Annotated implementation of GPT-NeoX in PyTorch

  • This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimizations in GPT-NeoX for simplicity.


  • Catch up with Rukan
  • Sprint review
  • Try to remember what I did and what to do next. Maybe work with Aaron on Embeddings?


  • Work on


  • Read and comment on Jarod’s thesis