Category Archives: Writing

Phil 4.2.19

A lot of the weekend – ASRC PhD

  • Most of the past few days has been putting together the iConference talk, which Aaron, Wane and I pulled off yesterday. I need to send Keith Marzullo the paper he was asking about, but I’d like to close the loop with Wayne first.
  • And back to the CHI play paper!
  • Generate permutation table for convergence of place terms by number of groups

Phil 3.28.19

7:00 – 1:30 ASRC PhD 1:30 – 6:00 NASA AIMES

  • Downloaded an amoeba eating a bacteria. Need to edit
  • Fixed some bugs in the analytic code and downloaded some images and text. There is banding in the space name embeddings, which is pretty interesting
  • Worked on the paper. I think I have an entry point now. It seems to be flowing nicely
  • Dungeons & Dragons Single Volume Edition By Gary Gygax & Dave Arneson
  • GOES-R AI Kickoff meeting
  • More work on slides. Incorporated Wayne’s comments (I think?). Also got the video edited

Phil 3.26.19

7:00 – ASRC PhD

  • BBC Business Daily on “essay mills”. Apparently 7% of students use these?
  • Got 8 results on the survey so far!
  • Add query on place/space/channel(?) terms to find post. It could be an iteration until the one post that returns the most terms is found.
  • Evolving place names. Remarkable how few passes it takes for convergence (of at least the first 3 terms):
    • Group 1: (12 new terms)
      • ‘goblin’, ‘fire’, ‘orc’
      • ‘orb’, ‘statues’, ‘see’
      • ‘grogg’, ‘troll’, ‘box’
      • ‘dragon’, ‘grogg’, ‘something’
    • Group 1, tymora1: (2 new terms)
      • ‘goblin’, ‘orc’, ‘see’ – 1
      • ‘orb’, ‘gate’, ‘statues’ – 1
      • ‘troll’, ‘box’, ‘grogg’ – 0
      • ‘dragon’, ‘coins’, ‘something’ – 0
    • Group 1, tymora1, tymora2 (2 new terms)
      • ‘goblin’, ‘orc’, ‘stairs’ – 1
      • ‘rope’, ‘gate’, ‘orb’ – 2
      • ‘box’, ‘troll’, ‘grogg’ – 0
      • ‘coins’, ‘dragon’, ‘something’ – 0
    • Group 1, tymora1, tymora2, tymora3 (1 new term)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 0
      • ‘troll’, ‘box’, ‘grogg’ – 0
      • ‘dragon’, ‘coins’, ‘barrier’ – 1
    • Group 1, tymora1, tymora2, tymora3, tymora4 (0 new terms)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 0
      • ‘troll’, ‘grogg’, ‘box’ – 0
      • coins’, ‘dragon’, ‘barrier’  – 0
  • Checking to see if changing the order looks different
    • tymora4 (12 new terms)
      • ‘orc’, ‘goblin’, ‘vines’
      • ‘orb’, ‘trap’, ‘statues’
      • ‘grogg’, ‘troll’, ‘box’
      • ‘dragon’, ‘coins’, ‘platform’
    • tymora3, tymora4 (1 new term)
      • ‘vines’, ‘orc’, ‘goblin’ – 0
      • ‘rope’, ‘statues’, ‘orb’ – 1
      • ‘grogg’, ‘troll’, ‘box’ – 0
      • ‘dragon’, ‘coins’, ‘platform’ – 0
    • tymora2, tymora3, tymora4 (4 new terms)
      • ‘goblin’, ‘stairs’, ‘orc’ – 1
      • ‘rope’, ‘gate’, ‘across’ – 2
      • ‘grogg’, ‘troll’, ‘box’ – 0
      • ‘coins’, ‘dragon’, ‘barrier’ – 1
    • tymora1, tymora2, tymora3, tymora4 (1 new term)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 1
      • ‘troll’, ‘box’, ‘grogg’ – 0
      • ‘coins’, ‘dragon’, ‘barrier’ – 0
    • Group 1, tymora1, tymora2, tymora3, tymora4 (0 new terms)
      • ‘goblin’, ‘orc’, ‘stairs’ – 0
      • ‘rope’, ‘gate’, ‘orb’ – 0
      • ‘troll’, ‘grogg’, ‘box’ – 0
      • coins’, ‘dragon’, ‘barrier’ – 0
  • Made a really nice map. This is entirely based on data and patterns found by the software, just arranged using a human-assisted layout:

Map-1

  • JuryRoom meeting. Chris is getting the schema right, and is looking at IBM’s cloud service for hosting. We spent some time walking through the map. Panos is eager to read the paper

Phil 3.25.19

7:00 – ASRC PhD

  • Realized that the blue sky papers are only referenced by a link in the preface of the proceedings, so I think very few people will have read it. Going to make the slide deck review the paper first, then add in the rest of the content
  • Need to make a survey for past players asking them if a map would have helped, and what kind of map?
  • Added more mapping types. We now have
    • simplesimple
    • place-edgesplace-edges
    • space-edgesspace_edges
    • all-edgesall_edges
  • Create maps with linked spaces for each room for a “zoomed in” room
  • Need to play around with labeling nodes, but that may be after writing the paper. I think I’ll need a different mapping tool

FlatEarth

Phil 3.11.19

7:00 – 10:00 ASRC PhD. Fun, long day.

Phil 3.10.19

Learning to Speak and Act in a Fantasy Text Adventure Game

  • We introduce a large scale crowdsourced text adventure game as a research platform for studying grounded dialogue. In it, agents can perceive, emote, and act whilst conducting dialogue with other agents. Models and humans can both act as characters within the game. We describe the results of training state-of-the-art generative and retrieval models in this setting. We show that in addition to using past dialogue, these models are able to effectively use the state of the underlying world to condition their predictions. In particular, we show that grounding on the details of the local environment, including location descriptions, and the objects (and their affordances) and characters (and their previous actions) present within it allows better predictions of agent behavior and dialogue. We analyze the ingredients necessary for successful grounding in this setting, and how each of these factors relate to agents that can talk and act successfully.

New run in the dungeon. Exciting!

Finished my pass through Antonio’s paper

Zoe Keating (May 1) or Imogen Heap (May 3)?

Phil 3.9.19

Understanding China’s AI Strategy

  • In my interactions with Chinese government officials, they demonstrated remarkably keen understanding of the issues surrounding AI and international security. It is clear that China’s government views AI as a high strategic priority and is devoting the required resources to cultivate AI expertise and strategic thinking among its national security community. This includes knowledge of U.S. AI policy discussions. I believe it is vital that the U.S. policymaking community similarly prioritize cultivating expertise and understanding of AI developments in China.

Russian Trolls Shift Strategy to Disrupt U.S. Election in 2020

  • Russian internet trolls appear to be shifting strategy in their efforts to disrupt the 2020 U.S. elections, promoting politically divisive messages through phony social media accounts instead of creating propaganda themselves, cybersecurity experts say.

Backup phone

Work on SASO paper – started

Rachel’s dungeon run is tomorrow! Maybe cross 10,000 posts?

Look at using BERT and the full Word2Vec model for analyzing posts

The Promise of Hierarchical Reinforcement Learning

  • To really understand the need for a hierarchical structure in the learning algorithm and in order to make the bridge between RL and HRL, we need to remember what we are trying to solve: MDPs. HRL methods learn a policy made up of multiple layers, each of which is responsible for control at a different level of temporal abstraction. Indeed, the key innovation of the HRL is to extend the set of available actions so that the agent can now choose to perform not only elementary actions, but also macro-actions, i.e. sequences of lower-level actions. Hence, with actions that are extended over time, we must take into account the time elapsed between decision-making moments. Luckily, MDP planning and learning algorithms can easily be extended to accommodate HRL.

Phil 3.6.19

5:00 – ASRC TL

  • Got a lot done on the BAA on the flight yesterday
  • Wrote up a description of LMN and CM for Eric V.
  • Reading more of the Handbook of Latent Semantic Analysis. It’s giving me some good ideas for calculating similarities of posts using Word2Vec and comparing the average vector for each post
  • Antonio got an extension to the 12th. Need to see what he’s up to. Wow, there’s a lot there now. Made some comments about what I’d like to see. I’ll pull down the document to read later
  • Continued to tweak the slides
  • TF Dev conference main sessions today. Breakouts tomorrow.

Phil 3.4.19

7:00 – 5:00 ASRC

  • Build an interactive SequenceAnalyzer. The adjustments are
    • Number of buckets
    • Percentages for each analytic (percentages to keep/discard
    • Selectable skip words that can be added to a list (in the db?)
  • Algorithm
    1. Find the most common words across all groups, these are skip_words
    2. Find the most common words along the entire series of posts per player and eliminate them
    3. Find the most common/central words across all sequences and keep those as belief places
    4. For each sequence by group, find the most common/central words after the belief places. These are the belief spaces.
    5. Build an adjacency matrix of players, groups, places and spaces
    6. Build submatrices for centrality calculations? This could be rather than finding the most common
    7. Possible word2vec variations?
      1. It seems to me that I might be able to use direction cosines and dynamic time warping to calculate the similarity of posts and align them better than the overall scaling that I’m doing now. DM posts introducing a room should align perfectly, and then other scaling could happen between those areas of greatest alignment
  • Display
    • Menu:
      • Save spreadsheet (includes config, included words, posts(?), trajectories)
      • load data
      • select database
      • select group within db
      • load/save config file
      • clear all
    • Fields
      • percent for A1, A2, A3, A4
      • Centrality/Sum switch
      • BOW/TF-IDF switch
      • Word2vec switch?
    • Textarea (areas? tabbed?)
      • Table with rows as sequence step. Columns are grouped by places, spaces, groups, and players
    • Work on Antonio’s paper got a first draft on introduction and motivation
    • BAA
      • Upload latex and references to laptop
    • Haircut! Pack!
    • Model-Based Reinforcement Learning for Atari
      • Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction — substantially more, in fact, than a human would need to learn the same games. How can people learn so quickly? Part of the answer may be that people can learn how the game works and predict which actions will lead to desirable outcomes. In this paper, we explore how video prediction models can similarly enable agents to solve Atari games with orders of magnitude fewer interactions than model-free methods. We describe Simulated Policy Learning (SimPLe), a complete model-based deep RL algorithm based on video prediction models and present a comparison of several model architectures, including a novel architecture that yields the best results in our setting. Our experiments evaluate SimPLe on a range of Atari games and achieve competitive results with only 100K interactions between the agent and the environment (400K frames), which corresponds to about two hours of real-time play.

 

Phil 3.1.19

7:00 – ASRC

  • Got accepted to the TF dev conference. The flight out is expensive… Sent Eric V. a note asking for permission to go, but bought tix anyway given the short fuse
  • Downloaded the full slack data
  • Working on white paper. The single file was getting unwieldy, so I broke it up
  • Found Speeding up Parliamentary Decision Making for Cyber Counter-Attack, which argues for the possibility of pre-authorizing automated response
  • Up to six pages. IN the middle of the cyberdefense section

Phil 2.28.19

7:00 – very, very, late ASRC

  • Tomorrow is March! I need to write a few paragraphs for Antonio this weekend
  • YouTube stops recommending alt-right channels
    • For the first two weeks of February, YouTube was recommending videos from at least one of these major alt-right channels on more than one in every thirteen randomly selected videos (7.8%). From February 15th, this number has dropped to less than one in two hundred and fifty (0.4%).
  • Working on text splitting Group1 in the PHPBB database
    • Updated the view so the same queries work
    • Discovered that you can do this: …, “message” as type, …. That gives you a column of type filled with “message”. Via stackoverflow
    • Mostly working, I’m missing the last bucket for some reason. But it’s good overlap with the Slack data.
    • Was debugging on my office box, and was wondering where all the data after the troll was! Ooops, not loaded
    • Changed the time tests to be > ts1 and <= ts2
  • Working on the white paper. Deep into strategy, Cyberdefense, and the evolution towards automatic active response in cyber.
  • Looooooooooooooooooooooooooong meeting of Shimei’s group. Interesting but difficult paper: Learning Dynamic Embeddings from Temporal Interaction Networks
  • Emily’s run in the dungeon finishes tonight!
  • Looks like I’m going to the TF Dev conference after all….

Phil 2.27.19

7:00 – 5:30 ASRC

  • Getting closer to the goal by being less capable
    • Understanding how systems with many semi-autonomous parts reach a desired target is a key question in biology (e.g., Drosophila larvae seeking food), engineering (e.g., driverless navigation), medicine (e.g., reliable movement for brain-damaged individuals), and socioeconomics (e.g., bottom-up goal-driven human organizations). Centralized systems perform better with better components. Here, we show, by contrast, that a decentralized entity is more efficient at reaching a target when its components are less capable. Our findings reproduce experimental results for a living organism, predict that autonomous vehicles may perform better with simpler components, offer a fresh explanation for why biological evolution jumped from decentralized to centralized design, suggest how efficient movement might be achieved despite damaged centralized function, and provide a formula predicting the optimum capability of a system’s components so that it comes as close as possible to its target or goal.
  • Nice chat with Greg last night. He likes the “Bones in a Hut” and “Stampede Theory” phrases. It turns out the domains are available…
    • Thinking that the title of the book could be “Stampede Theory: Why Group Think Happens, and why Diversity is the First, Best Answer”. Maybe structure the iConference talk around that as well.
  • Guidance from Antonio: In the meantime, if you have an idea on how to structure the Introduction, please go on considering that we want to put the decision logic inside each Autonomous Car that will be able to select passengers and help them in a self-organized manner.
  • Try out the splitter on the Tymora1 text.
    • Incorporate the ignore.xml when reading the text
    • If things look promising, then add changes to the phpbb code and try on that text as well.
    • At this point I’m just looking at overlapping lists of words that become something like a sand chart. I wonder if I can use the Eigenvector values to become a percentage connectivity/weight? Weights
    • Ok – I have to say that I’m pretty happy with this. These are centrality using top 25% BOW from the Slack text of Tymora1. I think that the way to use this is to have each group be an “agent” that has cluster of words for each step: Top 10
    • Based on this, I’d say add a “Evolving Networks of words” section to the dissertation. Have to find that WordRank paper
  • Working on white paper. Lit review today, plus fix anything that I might have broken…
    • Added section on cybersecurity that got lost in the update fiasco
    • Aaron found a good paper on the lack of advantage that the US has in AI, particularly wrt China
  • Avoiding working on white paper by writing a generator for Aaron. Done!
  • Cortex is an open-source platform for building, deploying, and managing machine learning applications in production. It is designed for any developer who wants to build machine learning powered services without having to worry about infrastructure challenges like configuring data pipelines, continuous deployment, and dependency management. Cortex is actively maintained by Cortex Labs. We’re a venture-backed team of infrastructure engineers and we’re hiring.