Phil 5.23.2022

Just finished up a nice bike vacation:

And I got a chapter cleaned up in the book! Need to photoshop a few pix

Simple Annotated implementation of GPT-NeoX in PyTorch

  • This is a simpler implementation of GPT-NeoX in PyTorch. We have taken out several optimizations in GPT-NeoX for simplicity.


  • Catch up with Rukan
  • Sprint review
  • Try to remember what I did and what to do next. Maybe work with Aaron on Embeddings?


  • Work on


  • Read and comment on Jarod’s thesis