GPT-2 Agents

• Added Queenside castling
• Found an ECO opening file here! Now I really want to parse it into a json file and use it
• Started the to_narrative() method. Here’s the first result!

  In 1987, Fred Van der Vliet played Loek Van Wely. Fred Van der Vliet was the higer-ranked player, with an Elo rating of 2330. Loek Van Wely was unrated. 
  The game began with ECO opening E69: 
  In move 1, Fred Van der Vliet moves white pawn from d2 to d4. Black moves knight from g8 to f6.
  this is a comment
  In move 2, White moves pawn from c2 to c4. Black moves pawn from g7 to g6.
  In move 3, Fred Van der Vliet moves white pawn from g2 to g3. Loek Van Wely moves black bishop from f8 to g7.
  In move 4, Fred Van der Vliet moves white bishop from f1 to g2. Loek Van Wely kingside castles.
  In move 5, White moves knight from g1 to f3. Black moves pawn from c7 to c6.
  In move 6, Fred Van der Vliet kingside castles. Loek Van Wely moves black pawn from d7 to d6.
  In move 7, White moves knight from b1 to c3. Black moves knight from b8 to d7.
  In move 8, White moves pawn from e2 to e4. Loek Van Wely moves black pawn from e7 to e5.
  In move 9, White moves pawn from h2 to h3. Loek Van Wely moves black rook from f8 to e8.
  In move 10, White moves rook from f1 to e1. Loek Van Wely moves black queen from d8 to b6.
  In move 11, White moves pawn from d4 to d5. Black moves knight from d7 to c5.
  In move 12, White moves rook from a1 to b1. Black moves pawn from a7 to a5.
  In move 13, White moves bishop from c1 to e3. Black moves queen from b6 to c7.
  In move 14, Fred Van der Vliet moves white knight from f3 to d2. Loek Van Wely moves black knight from f6 to h5.
  In move 15, White moves queen from d1 to e2. Black moves pawn from h7 to h6.
  In move 16, White moves pawn from d5 to c6. White takes black pawn. Black moves queen from c7 to c6. Black takes white pawn.
  In move 17, White moves knight from c3 to d5. Black moves bishop from c8 to e6.
  In move 18, White moves rook from e1 to c1. Black moves king from g8 to h7.
  In move 19, White moves pawn from b2 to b3. Black moves rook from e8 to b8.
  In move 20, White moves pawn from a2 to a3. Loek Van Wely moves black pawn from b7 to b6.
  In move 21, White moves king from g1 to h2. Black moves queen from c6 to c8.
  In move 22, Fred Van der Vliet moves white pawn from f2 to f4. Black moves pawn from e5 to f4. Black takes white pawn.
  In move 23, Fred Van der Vliet moves white pawn from g3 to f4. White takes black pawn. Black moves rook from a8 to a7.
  In move 24, Fred Van der Vliet moves white knight from d5 to c3. Black moves queen from c8 to d8.
  In move 25, White moves knight from c3 to b5. Black moves rook from a7 to d7.
  In move 26, Fred Van der Vliet moves white pawn from b3 to b4. Black moves pawn from a5 to b4. Black takes white pawn.
  In move 27, White moves pawn from a3 to b4. White takes black pawn. Black moves knight from c5 to a6.
  In move 28, White moves knight from d2 to f3. Loek Van Wely moves black queen from d8 to f6.
  In move 29, White moves queen from e2 to d2. Black moves rook from b8 to c8.
  In move 30, White moves knight from b5 to d6. White takes black pawn. Black moves bishop from e6 to c4. Black takes white pawn.
  In move 31, White moves pawn from e4 to e5. Loek Van Wely moves black queen from f6 to d8.
  In move 32, White moves pawn from b4 to b5. Loek Van Wely moves black knight from a6 to b8.
  In move 33, White moves queen from d2 to b4. Black moves rook from d7 to d6. Black takes white knight.
  In move 34, White moves pawn from e5 to d6. White takes black rook. Black moves bishop from c4 to d3.
  In move 35, White moves rook from c1 to c8. White takes black rook. Black moves queen from d8 to c8. Black takes white rook.
  In move 36, White moves rook from b1 to d1. Black moves bishop from d3 to e2.
  In move 37, White moves rook from d1 to c1. Black moves queen from c8 to e8.
  In move 38, Fred Van der Vliet moves white bishop from e3 to b6. White takes black pawn. Black moves bishop from e2 to f3. Black takes white knight.
  In move 39, White moves bishop from g2 to f3. White takes black bishop. Loek Van Wely moves black knight from h5 to f4. Black takes white pawn.
  In move 40, Fred Van der Vliet moves white rook from c1 to e1. Loek Van Wely moves black bishop from g7 to e5.
  In move 41, White moves rook from e1 to e5. White takes black bishop. Loek Van Wely moves black queen from e8 to e5. Black takes white rook.
  In move 42, White moves queen from b4 to d4. Black moves queen from e5 to b5. Black takes white pawn.
  In move 43, White moves queen from d4 to f4. White takes black knight. Black moves queen from b5 to b6. Black takes white bishop.
  In move 44, Fred Van der Vliet moves white queen from f4 to f7. White takes black pawn. Check. Black moves king from h7 to h8.
  In move 45, White moves queen from f7 to f6. Check. Black moves king from h8 to h7.
  In move 46, White moves bishop from f3 to e4. Black moves queen from b6 to a7.
  In move 47, Fred Van der Vliet wins! Loek Van Wely resigns.
  

GOES

• Adding better plotting of intermediate values
• Tweaking the margins, which is not adjusted by the plt.margins() method. It’s actually plt.subplots_adjust(). That took entirely too long to find.
• Got the plot working, and it is super cool:

Tried a lower latent dimension (8) and fewer neurons in the generator (32). Not really any different. It does take longer to stabilize though.

• Upping the strides on the discriminator to 4. Definitely less mode collapse

• Trying a stride of 8. That seems even better, though there is this funky artifact on the right. Going to try it with 10,000 epochs:

• Nope, no better. Sticking with the stride of 4. And 5,000 epochs seems to be enough to settle. This really looks a lot like the previous stride=4 run above:

• Ok, back to Advanced Deep Learning with Keras and GANS
• The last part of chapter 4 covered conditional GANS, or CGANS. I think that could be very nice to associate a particular type of noise with the state on a sim. For example, daylight vs. eclipse. It looks pretty straightforward, just the concatenation of a state to the input and discriminator vectors:

Graduation today starting at 10:00!

D20

• Looks like ASRC actually wanted us to build a completely different marketing app that would be done by 3 people in 3-4 Fridays. So no sponsorship there

GPT2 Agents

• Adding promotion – done. I have to say that I’m pretty pleased with how the parser handles it. We go from:

  62. a8=Q+ Kg1 63. Qa7 Kg2

  To:

  [62] expanded: 
  	white: F Caruana moves white pawn from a7 to a8. White pawn is promoted to white queen. Check.
  	black: Ding Liren moves black king from g2 to g1.
  
  [63] expanded: 
  	white: F Caruana moves white queen from a8 to a7.
  	black: Ding Liren moves black king from g1 to g2.

   

• Had a bit of trouble figuring out how to deal with the end of a file with a potentially incomplete game.
• Everything works!
• Now I need to write games out to files, do something with the introductions, etc.
• Also add some variability in the language later
• Also need to handle queenside castling (O-O-O)

GOES

• See how the 20,000 epoch run went, and start on Wasserstein Loss
• The 20,000 epoch really wasn’t any better. rerunning the 10k and saving the model

• The 10k version seems to be wider and fits better in the bounds. Looking at the two charts, it looks like following the loss/accuracy of the fake data make be best in this case:

• Reading the GAN chapter in Advanced Deep Learning with Keras (github)
  • Adding a plot of plots to show the appearance of the generator over the epochs. It should help in understanding the evolution of the generator
  • Bless StackOverflow

    side = 4
    fig = plt.figure(1)
    axs = fig.subplots(side, side)
    fig.set_figheight(15)
    fig.set_figwidth(15)
    for i in range(side*side):
        X = gen_data()
        plot_grid(axs, X.T, i, side=side)
    
    fig = plt.figure(2)
    plt.plot([1,2,3,4,3,2,1,2,3], label = "GREEN", color="green")
    plt.plot([1, 2, 3, 4, 5, 6, 7, 8, 9], label = "RED", color="red")
    plt.legend(loc="upper left")
    plt.show()

• 2:00 Meeting
  • Write up a code walkthrough proposal and send to Erik
  • set up 10:00 meeting with Vadim for tomorrow
• 3:00 Log file meeting (T & Isaac)
  • Demo’d LMN and RB

#COVID

• Good discussion about data and paper writing
• Tried to see if the annotated twitter data is in the DB. Some is, some is not
• Tried to get the transformers translate example running again, including downloading the files explicitly. Still didn’t work. Submitted a ticket

Call RV repair – done!

D20

• Aaron started to poke around in his code again and has some thoughts about how to get the sensitivity we need. I realize that the display that we’re working out will be a good dashboard for seeing second waves. We had a good chat on this and the advantage of using multiple sampling windows, possibly of 7, 14, 30 and all days to get a sense of where things are going.
• Georgia seems to be heading back up?

#COVID

• Public Health and Online Misinformation: Challenges and Recommendations
  • The internet has become a popular resource to learn about health and to investigate one’s own health condition. However, given the large amount of inaccurate information online, people can easily become misinformed. Individuals have always obtained information from outside the formal health care system, so how has the internet changed people’s engagement with health information? This review explores how individuals interact with health misinformation online, whether it be through search, user-generated content, or mobile apps. We discuss whether personal access to information is helping or hindering health outcomes and how the perceived trustworthiness of the institutions communicating health has changed over time. To conclude, we propose several constructive strategies for improving the online information ecosystem. Misinformation concerning health has particularly severe consequences with regard to people’s quality of life and even their risk of mortality; therefore, understanding it within today’s modern context is an extremely important task.

GPT-2 Agents

• Finish up PGNtoEnglish?
• Oops, had the black queen in the wrong place
• Got it working for one game. Need to try out a bunch next!

  Moves:
  [1] expanded: 
  	white:  Fred Van der Vliet moves white pawn from d2 to d4.
  	black:  Loek Van Wely moves black knight from g8 to f6.
  	this is a comment
  
  [2] expanded: 
  	white:  Fred Van der Vliet moves white pawn from c2 to c4.
  	black:  Loek Van Wely moves black pawn from g7 to g6.
  
  [3] expanded: 
  	white:  Fred Van der Vliet moves white pawn from g2 to g3.
  	black:  Loek Van Wely moves black bishop from f8 to g7.
  
  [4] expanded: 
  	white:  Fred Van der Vliet moves white bishop from f1 to g2.
  	black:  Loek Van Wely kingside castles.
  
  [5] expanded: 
  	white:  Fred Van der Vliet moves white knight from g1 to f3.
  	black:  Loek Van Wely moves black pawn from c7 to c6.
  
  [6] expanded: 
  	white:  Fred Van der Vliet kingside castles.
  	black:  Loek Van Wely moves black pawn from d7 to d6.
  
  [7] expanded: 
  	white:  Fred Van der Vliet moves white knight from b1 to c3.
  	black:  Loek Van Wely moves black knight from b8 to d7.
  
  [8] expanded: 
  	white:  Fred Van der Vliet moves white pawn from e2 to e4.
  	black:  Loek Van Wely moves black pawn from e7 to e5.
  
  [9] expanded: 
  	white:  Fred Van der Vliet moves white pawn from h2 to h3.
  	black:  Loek Van Wely moves black rook from f8 to e8.
  
  [10] expanded: 
  	white:  Fred Van der Vliet moves white rook from f1 to e1.
  	black:  Loek Van Wely moves black queen from d8 to b6.
  
  [11] expanded: 
  	white:  Fred Van der Vliet moves white pawn from d4 to d5.
  	black:  Loek Van Wely moves black knight from d7 to c5.
  
  [12] expanded: 
  	white:  Fred Van der Vliet moves white rook from a1 to b1.
  	black:  Loek Van Wely moves black pawn from a7 to a5.
  
  [13] expanded: 
  	white:  Fred Van der Vliet moves white bishop from c1 to e3.
  	black:  Loek Van Wely moves black queen from b6 to c7.
  
  [14] expanded: 
  	white:  Fred Van der Vliet moves white knight from f3 to d2.
  	black:  Loek Van Wely moves black knight from f6 to h5.
  
  [15] expanded: 
  	white:  Fred Van der Vliet moves white queen from d1 to e2.
  	black:  Loek Van Wely moves black pawn from h7 to h6.
  
  [16] expanded: 
  	white:  Fred Van der Vliet moves white pawn from d5 to c6. White takes black pawn.
  	black:  Loek Van Wely moves black queen from c7 to c6. Black takes white pawn.
  
  [17] expanded: 
  	white:  Fred Van der Vliet moves white knight from c3 to d5.
  	black:  Loek Van Wely moves black bishop from c8 to e6.
  
  [18] expanded: 
  	white:  Fred Van der Vliet moves white rook from e1 to c1.
  	black:  Loek Van Wely moves black king from g8 to h7.
  
  [19] expanded: 
  	white:  Fred Van der Vliet moves white pawn from b2 to b3.
  	black:  Loek Van Wely moves black rook from e8 to b8.
  
  [20] expanded: 
  	white:  Fred Van der Vliet moves white pawn from a2 to a3.
  	black:  Loek Van Wely moves black pawn from b7 to b6.
  
  [21] expanded: 
  	white:  Fred Van der Vliet moves white king from g1 to h2.
  	black:  Loek Van Wely moves black queen from c6 to c8.
  
  [22] expanded: 
  	white:  Fred Van der Vliet moves white pawn from f2 to f4.
  	black:  Loek Van Wely moves black pawn from e5 to f4. Black takes white pawn.
  
  [23] expanded: 
  	white:  Fred Van der Vliet moves white pawn from g3 to f4. White takes black pawn.
  	black:  Loek Van Wely moves black rook from a8 to a7.
  
  [24] expanded: 
  	white:  Fred Van der Vliet moves white knight from d5 to c3.
  	black:  Loek Van Wely moves black queen from c8 to d8.
  
  [25] expanded: 
  	white:  Fred Van der Vliet moves white knight from c3 to b5.
  	black:  Loek Van Wely moves black rook from a7 to d7.
  
  [26] expanded: 
  	white:  Fred Van der Vliet moves white pawn from b3 to b4.
  	black:  Loek Van Wely moves black pawn from a5 to b4. Black takes white pawn.
  
  [27] expanded: 
  	white:  Fred Van der Vliet moves white pawn from a3 to b4. White takes black pawn.
  	black:  Loek Van Wely moves black knight from c5 to a6.
  
  [28] expanded: 
  	white:  Fred Van der Vliet moves white knight from d2 to f3.
  	black:  Loek Van Wely moves black queen from d8 to f6.
  
  [29] expanded: 
  	white:  Fred Van der Vliet moves white queen from e2 to d2.
  	black:  Loek Van Wely moves black rook from b8 to c8.
  
  [30] expanded: 
  	white:  Fred Van der Vliet moves white knight from b5 to d6. White takes black pawn.
  	black:  Loek Van Wely moves black bishop from e6 to c4. Black takes white pawn.
  
  [31] expanded: 
  	white:  Fred Van der Vliet moves white pawn from e4 to e5.
  	black:  Loek Van Wely moves black queen from f6 to d8.
  
  [32] expanded: 
  	white:  Fred Van der Vliet moves white pawn from b4 to b5.
  	black:  Loek Van Wely moves black knight from a6 to b8.
  
  [33] expanded: 
  	white:  Fred Van der Vliet moves white queen from d2 to b4.
  	black:  Loek Van Wely moves black rook from d7 to d6. Black takes white knight.
  
  [34] expanded: 
  	white:  Fred Van der Vliet moves white pawn from e5 to d6. White takes black rook.
  	black:  Loek Van Wely moves black bishop from c4 to d3.
  
  [35] expanded: 
  	white:  Fred Van der Vliet moves white rook from c1 to c8. White takes black rook.
  	black:  Loek Van Wely moves black queen from d8 to c8. Black takes white rook.
  
  [36] expanded: 
  	white:  Fred Van der Vliet moves white rook from b1 to d1.
  	black:  Loek Van Wely moves black bishop from d3 to e2.
  
  [37] expanded: 
  	white:  Fred Van der Vliet moves white rook from d1 to c1.
  	black:  Loek Van Wely moves black queen from c8 to e8.
  
  [38] expanded: 
  	white:  Fred Van der Vliet moves white bishop from e3 to b6. White takes black pawn.
  	black:  Loek Van Wely moves black bishop from e2 to f3. Black takes white knight.
  
  [39] expanded: 
  	white:  Fred Van der Vliet moves white bishop from g2 to f3. White takes black bishop.
  	black:  Loek Van Wely moves black knight from h5 to f4. Black takes white pawn.
  
  [40] expanded: 
  	white:  Fred Van der Vliet moves white rook from c1 to e1.
  	black:  Loek Van Wely moves black bishop from g7 to e5.
  
  [41] expanded: 
  	white:  Fred Van der Vliet moves white rook from e1 to e5. White takes black bishop.
  	black:  Loek Van Wely moves black queen from e8 to e5. Black takes white rook.
  
  [42] expanded: 
  	white:  Fred Van der Vliet moves white queen from b4 to d4.
  	black:  Loek Van Wely moves black queen from e5 to b5. Black takes white pawn.
  
  [43] expanded: 
  	white:  Fred Van der Vliet moves white queen from d4 to f4. White takes black knight.
  	black:  Loek Van Wely moves black queen from b5 to b6. Black takes white bishop.
  
  [44] expanded: 
  	white:  Fred Van der Vliet moves white queen from f4 to f7. Check. White takes black pawn.
  	black:  Loek Van Wely moves black king from h7 to h8.
  
  [45] expanded: 
  	white:  Fred Van der Vliet moves white queen from f7 to f6. Check.
  	black:  Loek Van Wely moves black king from h8 to h7.
  
  [46] expanded: 
  	white:  Fred Van der Vliet moves white bishop from f3 to e4.
  	black:  Loek Van Wely moves black queen from b6 to a7.
  
  [47] expanded: 
  	white:  Fred Van der Vliet wins!
  	black:  Loek Van Wely resigns.
  
  
  
  Process finished with exit code 0
  

GOES

• See if I can get a GAN running with the new parts. I am not convinced that the input vector and times series are set up right. I probably need to set up for testing that as a parameter as well.
• Still, they seem reasonable. For testing, I have an input vector of 5 and 10 samples, like so:

  [[ 0.99611333]
    [-0.49462109]
    [-0.58444332]
    [ 0.98104957]
    [-0.23207803]]
  
   [[ 0.70418362]
    [ 0.35257367]
    [-0.99762846]
    [ 0.47774618]
    [ 0.60000333]]

• My latent matrix is 4 wide by 10 samples, arranged like so:

  [  2.46776301  17.36723783  -3.05622169 -15.7807726 ]
   [  4.73444423 -13.03790398  -0.04382812 -17.16272464]

• And my predicts seem reasonable, and they match the real data exactly, both from the Dense->CNN and the Dense versions
• Dense (2 steps of 5-feature vector)

  [[-1.        ]
    [ 1.        ]
    [-1.        ]
    [-1.        ]
    [ 1.        ]]
  
   [[-0.845926  ]
    [ 0.98410994]
    [-0.9970485 ]
    [-0.99517673]
    [ 0.910765  ]]

• Dense->CNN (2 steps of 5-feature vector)

  [[ 0.9974974 ]
    [-0.7852714 ]
    [-0.8944576 ]
    [ 0.99998605]
    [-0.34316158]]
  
   [[-0.9874317 ]
    [-0.90552104]
    [ 0.95992273]
    [ 0.8134167 ]
    [-0.94480175]]

• And all the predicts from the detector have the same form of (10, 1):

  [[0.5464545 ]
   [0.5419694 ]
   [0.5464503 ]
   [0.5464593 ]
   [0.5464545 ]
   [0.5462216 ]
   [0.45328763]
   [0.541866  ]
   [0.48022035]
   [0.54643464]]

• So I think I’m just going to try to build a OneDGAN2 class, evaluate that, and then if it’s working well, build a NoiseGAN2 class and use that for the paper/presentation
• Back to being able to draw plots, which means that the model is being created without blowing up!

• So I made some changes, basically added many more samples, played around with the size of the latent space and so forth. And I got a very unusual result when looking at the untrained model:

• I think this may be related to the use of a dense layer between the latent data and the Conv1D. Need to do some more reading, I guess.