Monthly Archives: March 2019

Phil 3.28.19

7:00 – 1:30 ASRC PhD 1:30 – 6:00 NASA AIMES

  • Downloaded an amoeba eating a bacteria. Need to edit
  • Fixed some bugs in the analytic code and downloaded some images and text. There is banding in the space name embeddings, which is pretty interesting
  • Worked on the paper. I think I have an entry point now. It seems to be flowing nicely
  • Dungeons & Dragons Single Volume Edition By Gary Gygax & Dave Arneson
  • GOES-R AI Kickoff meeting
  • More work on slides. Incorporated Wayne’s comments (I think?). Also got the video edited

Phil 3.26.19

7:00 – ASRC PhD

  • BBC Business Daily on “essay mills”. Apparently 7% of students use these?
  • Got 8 results on the survey so far!
  • Add query on place/space/channel(?) terms to find post. It could be an iteration until the one post that returns the most terms is found.
  • Evolving place names. Remarkable how few passes it takes for convergence (of at least the first 3 terms):
    • Group 1: (12 new terms)
      • ‘goblin’, ‘fire’, ‘orc’
      • ‘orb’, ‘statues’, ‘see’
      • ‘grogg’, ‘troll’, ‘box’
      • ‘dragon’, ‘grogg’, ‘something’
    • Group 1, tymora1: (2 new terms)
      • ‘goblin’, ‘orc’, ‘see’ – 1
      • ‘orb’, ‘gate’, ‘statues’ – 1
      • ‘troll’, ‘box’, ‘grogg’ – 0
      • ‘dragon’, ‘coins’, ‘something’ – 0
    • Group 1, tymora1, tymora2 (2 new terms)
      • ‘goblin’, ‘orc’, ‘stairs’ – 1
      • ‘rope’, ‘gate’, ‘orb’ – 2
      • ‘box’, ‘troll’, ‘grogg’ – 0
      • ‘coins’, ‘dragon’, ‘something’ – 0
    • Group 1, tymora1, tymora2, tymora3 (1 new term)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 0
      • ‘troll’, ‘box’, ‘grogg’ – 0
      • ‘dragon’, ‘coins’, ‘barrier’ – 1
    • Group 1, tymora1, tymora2, tymora3, tymora4 (0 new terms)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 0
      • ‘troll’, ‘grogg’, ‘box’ – 0
      • coins’, ‘dragon’, ‘barrier’  – 0
  • Checking to see if changing the order looks different
    • tymora4 (12 new terms)
      • ‘orc’, ‘goblin’, ‘vines’
      • ‘orb’, ‘trap’, ‘statues’
      • ‘grogg’, ‘troll’, ‘box’
      • ‘dragon’, ‘coins’, ‘platform’
    • tymora3, tymora4 (1 new term)
      • ‘vines’, ‘orc’, ‘goblin’ – 0
      • ‘rope’, ‘statues’, ‘orb’ – 1
      • ‘grogg’, ‘troll’, ‘box’ – 0
      • ‘dragon’, ‘coins’, ‘platform’ – 0
    • tymora2, tymora3, tymora4 (4 new terms)
      • ‘goblin’, ‘stairs’, ‘orc’ – 1
      • ‘rope’, ‘gate’, ‘across’ – 2
      • ‘grogg’, ‘troll’, ‘box’ – 0
      • ‘coins’, ‘dragon’, ‘barrier’ – 1
    • tymora1, tymora2, tymora3, tymora4 (1 new term)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 1
      • ‘troll’, ‘box’, ‘grogg’ – 0
      • ‘coins’, ‘dragon’, ‘barrier’ – 0
    • Group 1, tymora1, tymora2, tymora3, tymora4 (0 new terms)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 0
      • ‘troll’, ‘grogg’, ‘box’ – 0
      • coins’, ‘dragon’, ‘barrier’ – 0
  • Made a really nice map. This is entirely based on data and patterns found by the software, just arranged using a human-assisted layout:


  • JuryRoom meeting. Chris is getting the schema right, and is looking at IBM’s cloud service for hosting. We spent some time walking through the map. Panos is eager to read the paper

Phil 3.25.19

7:00 – ASRC PhD

  • Realized that the blue sky papers are only referenced by a link in the preface of the proceedings, so I think very few people will have read it. Going to make the slide deck review the paper first, then add in the rest of the content
  • Need to make a survey for past players asking them if a map would have helped, and what kind of map?
  • Added more mapping types. We now have
    • simplesimple
    • place-edgesplace-edges
    • space-edgesspace_edges
    • all-edgesall_edges
  • Create maps with linked spaces for each room for a “zoomed in” room
  • Need to play around with labeling nodes, but that may be after writing the paper. I think I’ll need a different mapping tool


Phil 3.24.19

I think this is pretty good: Map!

Are conspiracy theories what mysticism looks like when it adopts the trappings of rationalism?

Added some exception handling for the sql calls:

except pymysql.err.OperationalError as oe:
    messagebox.showerror("pymysql.err.OperationalError", oe.args)
    return None

Downloaded the new Slack files. Had to handle the presence of gifs, and a few other db issues. Everything continues to look good.

Working on building the table, then the map. Here’s the first shot.

split_1 places: ['goblin', 'orc', 'stairs', 'vines', 'see', 'hit', 'arrow', 'fire', 'gate', 'behind']
	splt-chn_1of4-tymora1 spaces: ['guys', 'temple', 'guess', 'damage', 'hey', 'spell', 'axe', 'know', 'trees', 'move', 'come']
	splt-chn_1of4-tymora2 spaces: ['short', 'log', 'attack', 'feet', 'stealth', 'damage', 'morc', 'temple', 'axe', 'top', 'mindy']
	splt-chn_1of4-tymora3 spaces: ['coins', 'light', 'damage', 'shards', 'glass', 'steps', 'floor', 'need', 'poison', 'bow', 'sacred']
	splt-chn_1of4-tymora4 spaces: ['spell', 'glass', 'feet', 'perception', 'antechamber', 'shield', 'cast', 'hand', 'log', 'move', 'beyond']
	splt-chn_1of4-Group 1 spaces: ['eyes', 'bow', 'side', 'bit', 'looking', 'carefully', 'halfling', 'arrows', 'near', 'beyond', 'hand']
	common elements: set()
split_2 places: ['rope', 'gate', 'orb', 'statues', 'across', 'see', 'side', 'around', 'pit', 'feet']
	splt-chn_2of4-tymora1 spaces: ['something', 'perception', 'hand', 'statue', 'floor', 'guys', 'ropes', 'switch', 'wall', 'tie', 'pull']
	splt-chn_2of4-tymora2 spaces: ['tie', 'wall', 'open', 'lever', 'gear', 'trap', 'hand', 'pitons', 'lightning', 'javelin', 'cross']
	splt-chn_2of4-tymora3 spaces: ['lever', 'trap', 'metal', 'need', 'floor', 'perception', 'statue', 'dwarves', 'hand', 'bar', 'large']
	splt-chn_2of4-tymora4 spaces: ['trap', 'guys', 'guidance', 'metal', 'statue', 'fountain', 'perception', 'appears', 'investigate', 'spell', 'coins']
	splt-chn_2of4-Group 1 spaces: ['anything', 'metal', 'halfling', 'gear', 'close', 'something', 'spell', 'statue', 'range', 'trap', 'hand']
	common elements: set()
split_3 places: ['troll', 'grogg', 'box', 'chest', 'hand', 'open', 'gate', 'club', 'see', 'want']
	splt-chn_3of4-tymora1 spaces: ['rope', 'oil', 'fire', 'door', 'sleep', 'maybe', 'trolls', 'key', 'nice', 'around', 'feet']
	splt-chn_3of4-tymora2 spaces: ['dream', 'tiny', 'people', 'sleep', 'groggs', 'thing', 'eyes', 'turn', 'lets', 'need', 'group']
	splt-chn_3of4-tymora3 spaces: ['attack', 'damage', 'key', 'fire', 'need', 'lock', 'hit', 'torches', 'attacks', 'dwarves', 'know']
	splt-chn_3of4-tymora4 spaces: ['spell', 'stealth', 'guidance', 'sleep', 'hands', 'burning', 'cast', 'ready', 'attack', 'damage', 'key']
	splt-chn_3of4-Group 1 spaces: ['song', 'sing', 'trolls', 'running', 'feather', 'sleeping', 'looking', 'need', 'creature', 'come', 'face']
	common elements: set()
split_4 places: ['coins', 'dragon', 'barrier', 'platform', 'something', 'light', 'woman', 'see', 'eyes', 'treasure']
	splt-chn_4of4-tymora1 spaces: ['blue', 'figures', 'enormous', 'coin', 'piles', 'appears', 'friend', 'around', 'full', 'wyrm', 'shall']
	splt-chn_4of4-tymora2 spaces: ['body', 'enormous', 'across', 'must', 'offer', 'mortal', 'gold', 'piles', 'die', 'near', 'within']
	splt-chn_4of4-tymora3 spaces: ['bridge', 'choose', 'insight', 'blue', 'know', 'gold', 'bridges', 'appears', 'die', 'enormous', 'game']
	splt-chn_4of4-tymora4 spaces: ['stay', 'across', 'must', 'dragons', 'game', 'boy', 'enormous', 'gold', 'piles', 'cant', 'hear']
	splt-chn_4of4-Group 1 spaces: ['grogg', 'looking', 'anything', 'thing', 'troll', 'dragons', 'moment', 'gate', 'gold', 'breath', 'magical']
	common elements: set()

A really interesting thing happened in the run last night. There was an argument about the shared perception of the belief space in the troll room. This misalignment was with the DM (the longer smear), who is kind of like a ship’s captain and is the final authority of the dungeon. When the player resists adhering with the DM’s perception of the belief space, the DM threatens expulsion. Some explorers are born, but involuntary explorers can be made:


Phil 3.23.19


Are Deep Neural Networks Dramatically Overfitted? (from this blog)

  • I would like to discuss a couple of papers on generalizability and complexity measurement of deep learning models in the post. Hopefully, it could shed light on your thinking path towards the understanding of why DNN can generalize.

Work for the next week:

  • I need to build a set of panalyzers that are room/channel specific. Done!
    • It looks like it’s just creating a key in the channel loop that also has the split information:
      max = len(split_list)-1
      for i in range(0, max):
          for db in db_list:
              name_list = self.run_query('select username as new, id as old from user_view', db_name=db)
              for group in channel_list:
                  key = "splt-chn_({}of{})-{}".format(i+1, max, group)
  • Comparing the counts of place and space terms will (hopefully!) show the movement of the channel population between splits (splits should be similar across channels), and the actions (spaces) that are explored in each space.
    • Table that shows place/space term counts by channel and split
    • I think I can get this by just using the top-n common terms to identify the rooms, and the top-n (excluding room terms) for each splt-chn panalyzer. These room terms are the identifier (row label), and then each column is a different group, which gives us a set of 4 room nodes from which link to action nodes. Take that and output to graph?
    • Refining the table and graph can be done by finding embedding neighbors.
  • Statistical difference in the populations of place terms within each split and within each channel.
  • What is the math that we use to show spaces? Or do we just show that they are different from the place populations?
  • Build a map from the table

Well that’s a nice way to start the day:

subject BOW
tymora1: 3948
tymora2: 732
tymora3: 506
Group: 63
1: 63
post BOW
goblin: 287
room: 239
orc: 227
stairs: 214
vines: 197
hit: 186
fire: 171
arrow: 170
gate: 134
behind: 133
subject BOW
tymora1: 3972
tymora2: 732
tymora3: 376
Group: 83
1: 83
post BOW
rope: 415
gate: 331
room: 324
orb: 309
statues: 251
across: 207
side: 196
around: 162
pit: 150
feet: 148
subject BOW
tymora1: 3004
tymora2: 816
tymora3: 1016
Group: 57
1: 57
post BOW
troll: 618
grogg: 596
chest: 274
hand: 252
room: 244
open: 209
gate: 199
club: 191
want: 169
key: 143
subject BOW
tymora1: 596
tymora2: 162
tymora3: 438
Group: 67
1: 67
post BOW
dragon: 176
coins: 174
room: 148
barrier: 108
light: 102
something: 102
platform: 100
eyes: 82
woman: 82
treasure: 62

Need to fix the “Group” “1” error and add an underscore to subjects in panalyzers – done

Note. This means that you left a paren on your callback: File "C:/Development/AntibubblesDungeon/src/analytics/", line 438, in show_bow
self.main_text.delete('1.0', END)
AttributeError: 'NoneType' object has no attribute 'delete'

Added the beginnings of the table

Phil 3.22.19

7:00 – 5:30 ASRC PhD

  • Morning thought – primordial movement is navigation at a more generalized level. At this level, only broad features are visible, so it’s only easy to jump to a large, common feature like fear-of-the-other. For humans, I think there are very few accessible features at this level.
  • Adversarial attacks on medical machine learning
    • With public and academic attention increasingly focused on the new role of machine learning in the health information economy, an unusual and no-longer-esoteric category of vulnerabilities in machine-learning systems could prove important. These vulnerabilities allow a small, carefully designed change in how inputs are presented to a system to completely alter its output, causing it to confidently arrive at manifestly wrong conclusions. These advanced techniques to subvert otherwise-reliable machine-learning systems—so-called adversarial attacks—have, to date, been of interest primarily to computer science researchers (1). However, the landscape of often-competing interests within health care, and billions of dollars at stake in systems’ outputs, implies considerable problems. We outline motivations that various players in the health care system may have to use adversarial attacks and begin a discussion of what to do about them. Far from discouraging continued innovation with medical machine learning, we call for active engagement of medical, technical, legal, and ethical experts in pursuit of efficient, broadly available, and effective health care that machine learning will enable.
  • Jumping straight into coding
    • Moved the menu bars around
      • Added a “Chains” menu that runs sequences of analytics
      • Added an NLP menu for BOW and TF-IDF
      • Working on parsing the selected listings to add to the various lists
      • Added a “search” button to search for neighbors of terms in the spaces and places text areas
      • Ran into a problem when I was calculating TF-IDF From here, but the way to get the data is here, spoilers – it uses Dataframes. Anyway, artifacts from Slack and PHPBB we interfering with the analytics. Things like [i:3fsgwhpz]. So I added a lot of preprocessing. First, more regexes:
        punctuation_regex = re.compile(r"[\",\-\_\.\d+?!*;“”]")
        bracket_regex = re.compile(r"\[.*?\]")
        amper_semicolon_regex = re.compile(r"\&.*?\;")

        And then some word-by-word testing and more regexing:

        def cleanup_text(self, raw: str) -> str:
            raw = self.bracket_regex.sub("",raw)
            raw = self.amper_semicolon_regex.sub("",raw)
            words = raw.split(" ")
            s = ""
            for word in words:
                if len(word) < 20:
                    word = self.punctuation_regex.sub("",word)
                    word = word.strip()
                    word = word.lower()
                    s += " {}".format(word)
                    print("Threw out {}".format(word))
            return s
      • Anyway, the results are pretty good now with TF-IDF. Here’s the top few terms for each player’s runs
        post TF-IDF
        	javelin: 1.3235824005932413
        	orc: 1.2596337832554234
        	don: 1.2329356851106978
        	mindy: 1.1935877563453061
        	dragon: 1.180695872064165
        	room: 1.16758789676671
        	goblin: 1.1244271157713812
        	gate: 1.0518778198496626
        	muttered: 0.9602704217082824
        	orb: 0.7818369426232508
        	vines: 0.7682713289173773
        	around: 0.7650741399517217
        	believe: 0.7181160263645541
        	set: 0.6960723252519683
        post TF-IDF
        	troll: 1.8877816896391235
        	room: 1.6145159552006425
        	spell: 1.19387373804735
        	orb: 1.031131725198719
        	hand: 1.0226191383574903
        	halfling: 0.882770501608177
        	feather: 0.8321564150194518
        	chest: 0.7956715550842
        	need: 0.7833254070966856
        	without: 0.7774257667568547
        	thought: 0.7698168337310007
        	wasn: 0.7681132279117597
        	grogg: 0.7397095370527041
        	nodded: 0.7283692418929164
        	cricket: 0.7152753845274986
        post TF-IDF
        	room: 1.37664885493529
        	halfling: 1.3095258648568422
        	orb: 1.1635393833341
        	arrow: 1.0958743699311024
        	bow: 1.0833323871838676
        	woman: 0.905661830111989
        	sword: 0.8274886204621582
        	side: 0.8159855961191507
        	door: 0.8030328480633289
        	weapons: 0.8023256227742956
        	bit: 0.7654054685520582
        	armor: 0.6925203949859854
        	compatriots: 0.6877982251257945
        	moment: 0.666106887452693
        	ready: 0.6591955622573524
        	wasn: 0.6407825783880001
        	group: 0.6181005287618395
        	words: 0.6162132133458346
        	statues: 0.6129058701485565
        post TF-IDF
        	grogg: 5.334199523123002
        	room: 3.811509447108922
        	gate: 3.258183021914178
        	orb: 2.618164708893951
        	troll: 2.613186928575063
        	arrow: 2.324879853387413
        	looking: 2.0945608918805405
        	dragon: 2.0386853821484214
        	statues: 2.0307941282615394
        	halfling: 1.9946267677568987
        	goblin: 1.9479046173471883
        	eyes: 1.9250729565572604
        	large: 1.8740265406115013
        	hand: 1.8621985701316857
        	ooc: 1.854167610118333
        	orc: 1.8453651951258079
        Aaron Buchanan-player:
        post TF-IDF
        	smile: 5.028089150947501
        	troll: 4.349229589605341
        	linda: 4.01623419920215
        	damage: 3.5816072459387898
        	let: 3.145522725372224
        	know: 3.0568701804539486
        	jenni: 3.0150241755107605
        	need: 2.9865597817204916
        	don: 2.89736685000583
        	fire: 2.8533063572331154
        	hit: 2.750325190292788
        	elf: 2.61638151392588
        	nothing: 2.5832965974665827
        	rope: 2.5651899037822403
        	want: 2.5516172618521207
        	room: 2.5234046485126127
        	hand: 2.408589994530533
        	key: 2.400913929253641
        	halfling: 2.300372097141645
        	grogg: 2.2976898742276277
        post TF-IDF
        	insight: 3.5333368131417924
        	troll: 3.4738014944618905
        	help: 2.629726435567097
        	grogg: 2.396996336469604
        	need: 2.358944535917952
        	room: 2.25768912386651
        	healing: 2.2382624097615995
        	rolls: 2.2229326490531314
        	gate: 2.195439698041948
        	around: 2.161868029714761
        	flame: 2.132641891485049
        	perception: 2.1149040159363293
        	floor: 2.1054598538974045
        	hands: 1.9310855224062018
        	light: 1.8784320905937355
        	uses: 1.850257078968512
        	catch: 1.793277658871867
        Bjorn Hasseler-player:
        post TF-IDF
        	flame: 2.315315586844817
        	sacred: 2.315315586844817
        	stonecunning: 2.1557147060826947
        	rolls: 2.132883047459771
        	grogg: 1.9660626127710805
        	don: 1.9424906312537291
        	steps: 1.935482241603244
        	bridge: 1.7092785394164918
        	rope: 1.5744166673302291
        	torches: 1.551554596214274
        	stairs: 1.4732750153279608
        	platform: 1.3689503773093064
        	vines: 1.3351891642736244
        	light: 1.3217202403569526
        post TF-IDF
        	troll: 5.780387270246775
        	grogg: 4.59954530049338
        	need: 4.379175783748707
        	attack: 3.7429397050152247
        	don: 3.564218421074024
        	open: 3.503242156009218
        	gate: 3.3068573706012985
        	stealth: 3.1899559813180645
        	traps: 2.965951521209904
        	chest: 2.9426150667239175
        	side: 2.8272991421295277
        	room: 2.8187225708944883
        	dragon: 2.7919898167411095
        	party: 2.784297298562819
        	know: 2.698287503059542
        post TF-IDF
        	grogg: 10.15480371661793
        	damage: 6.745957190399183
        	gate: 4.595526542550383
        	turn: 4.47763106887882
        	side: 4.220054751334396
        	room: 4.0759304935283875
        	troll: 3.6496283755351433
        	persuasion: 3.427787051079431
        	stairs: 3.2452878097049718
        	open: 3.0691978408049714
        	appears: 2.9662151644031236
        	barrier: 2.9526976397873197
        	hit: 2.882801312489094
        	please: 2.756473252393314
        	feet: 2.7553337830423685
        post TF-IDF
        	rope: 2.5925740290266477
        	troll: 2.149767946611558
        	grogg: 2.1373623910153023
        	close: 2.093606121380299
        	goes: 1.9114083131138986
        	thing: 1.879425660287383
        	gate: 1.8498993377390152
        	behind: 1.7759591324283694
        	around: 1.7739744768952832
        	eyes: 1.6015525983573213
        	along: 1.565130385605012
        	face: 1.5465277621378841
        	merric: 1.531838768719999
        	goblin: 1.5017357357398398
        	chest: 1.4793322265307385
        post TF-IDF
        	rope: 2.5574727497157186
        	bit: 2.334695846271241
        	halfling: 2.2917865866720875
        	turn: 2.2913671375617244
        	help: 2.263326099701241
        	want: 2.255517431033907
        	let: 2.2248419816867973
        	hand: 2.05823145874274
        	thing: 2.0252535914010243
        	need: 2.017853031957051
        	trying: 1.9392838304479607
        	won: 1.9074029011042937
        	lever: 1.8940047810992084
        	motions: 1.8791666280630153
        	grogg: 1.8512620533629853
        	know: 1.8460073095362568
        post TF-IDF
        	joy: 2.2090099096592093
        	bow: 1.4708821850714477
        	comrades: 1.3450198066079633
        	agreed: 1.2909672047219294
        	shit: 1.2511351948636689
        	hand: 1.1939893091569087
        	simply: 1.0754065506075219
        	alongside: 1.03095098597318
        	ready: 1.030909228527731
        	moves: 1.0217451884953292
        	follows: 1.019308558744064
        	esvele: 1.0031051305561383
        	eyes: 1.0031051305561383
        	alrighty: 1.0
        	sounds: 1.0
        	wow: 1.0
        	armor: 0.9675296552629622
        	goblin: 0.9675296552629622
        	arrow: 0.959551586099932
        post TF-IDF
        	grogg: 9.574956839446978
        	room: 7.7929146794417985
        	gate: 7.769157727681967
        	open: 5.161416388301517
        	side: 4.9552473292407315
        	hand: 4.920538249739186
        	something: 4.862660052617354
        	troll: 4.756897094197843
        	damage: 4.698981284560832
        	phoenix: 4.506875041306749
        	group: 4.4419312361884575
        	stealth: 4.347602688687722
        	bee: 4.337563548710324
        	stairs: 4.272837824041584
        	perception: 4.240823300482532
        	investigation: 4.214126267526979
        	bit: 4.2116435909201435
        	goblin: 4.16370490562877
        post TF-IDF
        	gate: 3.813331273921538
        	course: 3.3238613690720182
        	light: 3.2357123635315
        	yenadar: 3.206131400462222
        	move: 3.0410330090181983
        	orc: 2.920652421803098
        	dragon: 2.91393405622444
        	casts: 2.910174915050543
        	shelton: 2.781316206653326
        	turn: 2.7696197904765247
        	close: 2.7395876537286137
        	history: 2.7290686528241705
        	rope: 2.533647571484718
        	toss: 2.488833414797614
        post TF-IDF
        	rope: 7.346484814122864
        	mage: 5.1901641703849695
        	hand: 5.169823617707603
        	something: 4.052962289815423
        	room: 3.7977359903069874
        	spell: 3.733564719352958
        	feet: 3.629297463463484
        	gate: 3.4221850444525543
        	need: 3.313446574887807
        	hands: 3.306280386619182
        	casts: 3.239250407373651
        	arrows: 3.094136824719543
        	hahaha: 3.0
        	lorelai: 2.8832894462350955
        	troll: 2.879950620848435
        post TF-IDF
        	javelin: 5.50125987652874
        	sigh: 4.0
        	hmm: 3.534360458898429
        	room: 3.5148283185111264
        	yup: 3.5117826563035313
        	axe: 3.114116974088525
        	wait: 3.0448104923325108
        	course: 2.7955498582374037
        	chest: 2.7578668995086213
        	kk: 2.68753920352695
        	rope: 2.491724054450192
        	hand: 2.409513756019609
        	attack: 2.380401354039185
        	side: 2.2585165820164
        	ft: 2.2506235321034813
        	lead: 2.229579193706388
        	better: 2.2144928823627117
        	move: 2.159570674311593
        	troll: 2.149851721830461
        Shelton Herrington-player:
        post TF-IDF
        	hehe: 12.124909531389427
        	don: 11.505547928323026
        	guess: 10.67778790195941
        	nice: 9.170367618719803
        	rope: 8.698156248650768
        	orb: 8.567297721201122
        	hey: 8.3986603737825
        	hmm: 8.284208790999351
        	guys: 8.195019661596476
        	want: 7.965528793382409
        	door: 7.900199902653034
        	know: 7.817490225187236
        	can: 6.476312939358579
        	maybe: 6.393688899561484
        	around: 6.370346146839613

Phil 3.21.19


ASRC PhD 7:00 – 6:00

  • Worked a little on the iConference slides. Found the section on subcritical/supercritical in Kaufman’s book
  • Working on drawing the player panalyzers
  • Visualizing a NetworkX graph in the Notebook with D3.js
    • In this recipe, we will create a graph in Python with NetworkX and visualize it in the Jupyter Notebook with D3.js.
  • Bokeh is an interactive visualization python library that targets modern web browsers for presentation. Its goal is to provide elegant, concise construction of versatile graphics, and to extend this capability with high-performance interactivity over very large or streaming datasets. Bokeh can help anyone who would like to quickly and easily create interactive plots, dashboards, and data applications.
    • Visualizing Network Graphs
      • Bokeh has added native support for creating network graph visualizations with configurable interactions between edges and nodes.
  • Got the BOW terms out of all the panalyzers.
  • Working on setting child embeddings
  • Scikit has TF-IDF and BOW vectorizers
  • Players plotted in the master embedding space: PlayerChildEmbedding
  • I just realized that I can plot the space and place terms in the same way, which should make some nice figures in the paper.
  • Here’s a plot of the different groups. Note that it is much wider. Something happened? Child_Embeddings
  • Anyway, I’m done for the day. Tomorrow I’ll start to add stop words and look for neighbor terms (And plot them!) SQ_GUI

Phil 3.20.19

ASRC PhD 7:00 – 2:00, NASA 2:00 – 4:00

  • Add Text area for “selected”, with a combobox for “Place”, “Space”, and “Ignore”. Ignore words get split(), added to a sorted list view, and saved in the config.xml
  • Add text area for program output as well as console. Text lists for users can be shown here. Selected terms from here can be selected and added to the ignore list
  • Draw the following as colored points on the embeddings
    • Top Room (bow/tf-idf) terms
    • Top Group (bow/tf-idf)
    • Place terms and Space terms (differentiated by room/group?)


  • Split my time between TL and AIMS through May, see what has the most traction

Meeting with Aaron M.

  • Nice discussion. He thinks I’m on a good track, and agrees with using mixed methods as a step towards full automation

Made some tweaks to Antonio’s reply article

Phil 3.19.19

7:00 – 5:00, 6:30 – 7:15 ASRC PhD

  • Algorithmic Rationality: Game Theory with Costly Computation
    • We develop a general game-theoretic framework for reasoning about strategic agents performing possibly costly computation. In this framework, many traditional game-theoretic results (such as the existence of a Nash equilibrium) no longer hold. Nevertheless, we can use the framework to provide psychologically appealing explanations of observed behavior in well-studied games (such as finitely repeated prisoner’s dilemma and rock-paper-scissors). Furthermore, we provide natural conditions on games sufficient to guarantee that equilibria exist.
  • NVIDIA has an IoT NN chipset as well
  • Recreate DB with file pull. Success! Here’s the code that works. It takes a downloaded Slack chat session and iterates over that. When it finds a message with a “files” subobject, it does the following. Shout out to the wonderful requests library:
    for mf in entry["files"]:
        mf["user"] = entry["user"]
        mf["ts"] = entry["ts"]
        if mf["pretty_type"] == "Post":
            if use_local :
                to_return += mf["preview"]
                url = mf["url_private"]
                    result = requests.get(url, headers={'Authorization': 'Bearer %s' % self.slack_token})
                    dict = json.loads(result.text)
                    root = dict["root"]
                    to_return = ""
                    ch_list = root["children"]
                    for ch in ch_list:
                        if ch["type"] == 'p':
                            to_return += "{} ".format(ch["text"])
                    print("handle_file_entries(): text = {}".format(to_return))
                except requests.exceptions.RequestException as err:
                    print("Got a RequestException: {}".format(err))
                    to_return += mf["preview"]
  • Whoops, tymora3 doesn’t have the “Near the port city of Waterdeep in Faerun” phrase. Switching to “young man in a well crafted tunic”
  • Add explicit file opening and saving – done
  • Add “Embedding Dimensions” field – done
  • Example xml file:
          <channel>Group 1</channel>
          <split>young man in a well crafted tunic</split>
          <split>brightly glowing blue orb</split>
          <split>large scaled troll sleeping</split>
          <split>the sea of gold coins and gems filling it</split>
          <split>Two women lounge on chairs across from each other</split>
  • Reading and writing data are done. Now to start slicing and displaying data interactively


  • Data slicing
    • Build embedding for all docs
    • Starting to add bucketing code. First step is to get ignore text from individual users, second task is to have single, room aligned buckets, gust so we can see what that looks like
  • JuryRoom meeting at 6:30
    • Added my req’s
      • Groups can stay together
      • Threaded questions in sequential or random order
      • Voting for post, rather than yes/no questions
    • Tony raised a bunch of points about how the conversation could be gamed. My response was that we should build something simple and then try to game it to see what affordances we need
    • Extended discussion on display real estate – how do we show the “starred” posts
      • Tony mentioned the idea that starred posts could fade if they languish without additional stars
    • Panos mentioned the idea of a countdown clock to pressure a vote
    • We walked through the implementation issues. The estimated framework is 3-tier, with a relational DB, Node on the server, and a web client running a framework like React. The goal is a “Hello World” application that we can log into and create accounts running by next week
    • I pointed back to the original JuryRoom document’s strawman representation of the schema.
  • Synonyms for fringe
    • Synonyms: Noun
      • border, borderline, bound, boundary, brim, circumference, compass, confines, edge, edging, end, frame, hem, margin, perimeter, periphery, rim, skirt, skirting, verge
    • Synonyms: Verb
      • abut, adjoin, border (on), butt (on or against), flank, join, march (with), neighbor, skirt, touch, verge (on)

Phil 3.18.19

ASRC PhD 7:00 – 6:00

  • SlackToDb
    • Pull down text – Done, I hope. The network here has bad problems with TLS resolution. Will try from home
    • Link sequential posts – done
    • Add word lists for places and spaces (read from file, also read embeddings)
      • Writing out the config file – done
    • Add field for similarity distance threshold. Changing this lists nearby words in the embedding space. These terms are used for trajectory generation and centrality tables.
    • Add plots for place/space words
    • Add phrase-based splitting to find rooms. Buckets work within these splits. Text before the first split and after the last split isn’t used (For embedding, centrality, etc.)
    • Add phrase-based trimming. Test before one and after the other isn’t used
    • Stub out centrality for (embedded) terms and (concatenated, bucketed, and oriented) documents
    • Look into 3d for tkinter (from scratch)


    • Progress for the day:


Phil 3.17.19

Got a really good idea about doing a hybrid coding model using embeddings. We start with a list of “place terms” and a list of “space terms”. We then use the embedded representation (vector) of those terms to find the adjacent terms. This is a sort of automated “snowball sampling”, where terms can lead to other terms. Once we have these terms, we use them as queries into the database to find the campaign and the timestamp for each. We use these to create the trajectories and maps.

This is a pretty straightforward code and a set of queries to write, and I have high confidence that it will work, and provide a novel, understandable method of producing a nice ‘mixed method’ process that is also grounded completely in the corpora.

Phil 3.15.19

7:00 – ASRC

  • Downloaded the JuryRoom spec from Waikato and sent my sympathies for Christchurch
  • Worked on getting cosine distance working – Done. Also created spreadsheets of the distances between posts and list the posts on a tab in the spreadsheet. I strip out the words that aren’t used to make the vectors so the posts look a little funny, but the gist is there: