Monthly Archives: February 2019

Phil 2.16.19

Command Dysfunction: Minding the Cognitive War

  • This paper analyzes the factors and conditions of command dysfunction from the cognitive, or mental, perspective of command and control warfare (C2W). The author examines the limitations of rational decision making and the tension that exists between rational and intuitive processes. Next, the paper examines the vulnerabilities of rational and intuitive processes in order to build a cognitive warfare framework. The framework consists of three categories: the command baseline, stressors, and deception. The stressor and deception categories act on the command baseline. The analysis also suggests that there are a number of possible interactions that exist between the stressor and deception categories. The paper then uses the framework to analyze evidence of command dysfunction in three historical campaigns. The historical analyses study the German command during the Normandy invasion, the Allied command during the first week of the Battle of the Bulge, and the Israeli command during the first half of the Arab-Israeli October 1973 War. In addition to showing that there are interactions between stressors and deception, the analyses highlight the importance of understanding the adversary’s command baseline. The paper concludes that effective C2W is not so much what is done to an adversary’s command, but rather what he does to himself, perhaps with a little help.

2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Streaming Tutorials, June 2nd. The following tutorials have been accepted for NAACL 2019 and will be held on Sunday, June 2nd, 2019. Exact timings will be posted later as part of the official schedule.


Phil 2.14.19

7:00 – 7:00 ASRC

  • Worked on the whitepaper. Going down the chain of consequences with respect to adding AI to military systems in the light of the Starcraft2 research.
  • Maps of Meaning: The Architecture of Belief
    • A 1999 book by Canadian clinical psychologist and psychology professor Jordan Peterson. The book describes a comprehensive theory for how people construct meaning, in a way that is compatible with the modern scientific understanding of how the brain functions.[1] It examines the “structure of systems of belief and the role those systems play in the regulation of emotion”,[2] using “multiple academic fields to show that connecting myths and beliefs with science is essential to fully understand how people make meaning”.[3] Wikipedia
  • Continuing with Clockwork Muse review. Finished the overview and theoretical takes. Continuing on the notes, which is going slow because of bad text scanning
  • JAX is Autograd and XLA, brought together for high-performance machine learning research. With its updated version of Autograd, JAX can automatically differentiate native Python and NumPy functions. It can differentiate through loops, branches, recursion, and closures, and it can take derivatives of derivatives of derivatives. It supports reverse-mode differentiation (a.k.a. backpropagation) via grad as well as forward-mode differentiation, and the two can be composed arbitrarily to any order. What’s new is that JAX uses XLA to compile and run your NumPy programs on GPUs and TPUs. Compilation happens under the hood by default, with library calls getting just-in-time compiled and executed. But JAX also lets you just-in-time compile your own Python functions into XLA-optimized kernels using a one-function API, jit. Compilation and automatic differentiation can be composed arbitrarily, so you can express sophisticated algorithms and get maximal performance without leaving Python.
  • Working on white paper lit review
    • An Evolutionary Algorithm that Constructs Recurrent Neural Networks
      • Standard methods for simultaneously inducing the structure and weights of recurrent neural networks limit every task to an assumed class of architectures. Such a simplification is necessary since the interactions between network structure and function are not well understood. Evolutionary computations, which include genetic algorithms and evolutionary programming, are population-based search methods that have shown promise in many similarly complex tasks. This paper argues that genetic algorithms are inappropriate for network acquisition and describes an evolutionary program, called GNARL, that simultaneously acquires both the structure and weights for recurrent networks. GNARL’s empirical acquisition method allows for the emergence of complex behaviors and topologies that are potentially excluded by the artificial architectural constraints imposed in standard network induction methods
    • Added Evolutionary Deep Learning and Deep RTS to the references
  • Better Language Models and Their Implications
    • We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training.
  • Shimei seminar – 4:30 – 7:00

Phil 2.13.19

7:00 – 7:00 ASRC IRAD TL

  • The Digital Clockwork Muse: A Computational Model of Aesthetic Evolution
    • This paper presents a computational model of creativity that attempts to capture within a social context an important aspect of the art and design process: the search for novelty. The computational model consists of multiple novelty-seeking agents that can assess the interestingness of artworks. The agents can communicate to particularly interesting artworks to others. Agents can also communicate to reward other agents for finding interesting artworks. We present the results from running experiments to investigate the effects of searching for different degrees of novelty on the artworks produced and the social organisation of the agents.
  • Upload the rest of Slack Tymora.
  • Create some txt files and feed into LMN. I’m thinking of by player and then by slice. Do this for both PHPBB and Slack data. Other ideas
    • Look into coherence measures
    • Are there economic models of attention? (ArXive)
    • TAACO is an easy to use tool that calculates 150 indices of both local and global cohesion, including a number of type-token ratio indices (including specific parts of speech, lemmas, bigrams, trigrams and more), adjacent overlap indices (at both the sentence and paragraph level), and connectives indices.
    • CRAT is an easy to use tool that includes over 700 indices related to lexical sophistication, cohesion and source text/summary text overlap. CRAT is particularly well suited for the exploration of writing quality as it relates to summary writing.
    •  TAALED is an analysis tool designed to calculate a wide variety of lexical diversity indices. Homographs are disambiguated using part of speech tags, and indices are calculated using lemma forms. Indices can also be calculated using all lemmas, content lemmas, or function lemmas. Also available is diagnostic output which allows the user to see how TAALED processed each word.
    • TAALES is a tool that measures over 400 classic and new indices of lexical sophistication, and includes indices related to a wide range of sub-constructs.  TAALES indices have been used to inform models of second language (L2) speaking proficiency, first language (L1) and L2 writing proficiency, spoken and written lexical proficiency, genre differences, and satirical language.
    • SEANCE is an easy to use tool that includes 254 core indices and 20 component indices based on recent advances in sentiment analysis. In addition to the core indices, SEANCE allows for a number of customized indices including filtering for particular parts of speech and controlling for instances of negation.
    • TAASSC is an advanced syntactic analysis tool. It measures a number of indices related to syntactic development. Included are classic indices of syntactic complexity (e.g., mean length of T-unit) and fine-grained indices of phrasal (e.g., number of adjectives per noun phrase) and clausal (e.g., number of adverbials per clause) complexity. Also included are indices that are grounded in usage-based perspectives to language acquisition that rely on frequency profiles of verb argument constructions.
    • GAMET is an easy to use tool that provides incidence counts for structural and mechanics errors in texts including grammar, spelling, punctuation, white space, and repetition errors. The tool also provides line output for the errors flagged in the text.
    • Comparison of Top 6 Python NLP Libraries
      • NLTK (Natural Language Toolkit) is used for such tasks as tokenization, lemmatization, stemming, parsing, POS tagging, etc. This library has tools for almost all NLP tasks.
      • Spacy is the main competitor of the NLTK. These two libraries can be used for the same tasks.
      • Scikit-learn provides a large library for machine learning. The tools for text preprocessing are also presented here.
      • Gensim is the package for topic and vector space modeling, document similarity.
      • The general mission of the Pattern library is to serve as the web mining module. So, it supports NLP only as a side task.
      • Polyglot is the yet another python package for NLP. It is not very popular but also can be used for a wide range of the NLP tasks.
  • Continuing writing Clockwork Muse review
  • Reading Attachment 1 to BAA FA8750-18-S-7014. “While white papers will be considered if received prior to 6:00 PM Eastern Standard Time (EST) on 30 Sep 2022, the following submission dates are suggested to best align with projected funding:” 
    • FY20 – 15 April 2019
  • AIMS/ML Meeting. Not sure what the outcome was, other than folks are covered for this quarter?
  • Long, wide ranging meeting with Wayne at Frisco’s. Gave him an account on And it seems like we won first place for Blue Sky papers?

Phil 2.12.19

7:00 – 4:30 ASRC IRAD

  • Talked with Eric yesterday. going to write up a white paper about teachable AI. Two-three week effort
  • Speaking of which, The Evolved Transformer
    • Recent works have highlighted the strengths of the Transformer architecture for dealing with sequence tasks. At the same time, neural architecture search has advanced to the point where it can outperform human-designed models. The goal of this work is to use architecture search to find a better Transformer architecture. We first construct a large search space inspired by the recent advances in feed-forward sequential models and then run evolutionary architecture search, seeding our initial population with the Transformer. To effectively run this search on the computationally expensive WMT 2014 English-German translation task, we develop the progressive dynamic hurdles method, which allows us to dynamically allocate more resources to more promising candidate models. The architecture found in our experiments – the Evolved Transformer – demonstrates consistent improvement over the Transformer on four well-established language tasks: WMT 2014 English-German, WMT 2014 English-French, WMT 2014 English-Czech and LM1B. At big model size, the Evolved Transformer is twice as efficient as the Transformer in FLOPS without loss in quality. At a much smaller – mobile-friendly – model size of ~7M parameters, the Evolved Transformer outperforms the Transformer by 0.7 BLEU on WMT’14 English-German.
  • Finished running Tymora1 on Slack. Downloaded, though the download didn’t include research_notes. Hmmm. Looks like I can’t make it public, either.
  • Thinking about writing a tagging app, possibly with a centrality capability.
  • Started on the Teachable AI paper. The rough outline is there, and I have a good set of references.

Phil 2.11.19

7:00 – 5:00 ASRC IRAD (TL)

  • Gen Studio is a way to navigate between designs in latent space. It is a prototype concept which was created over a two-day hackathon with collaborators across The Met, Microsoft, and MIT.
  • Write up Clockwork Muse
  • Continue with parsing, storing and report generation of slack data. Aaron had the idea that multiple statements by one person should be combined into a single post. Need to think about how that works in the report generation. Since the retrieved list is ordered by timestamp, the naive implementation is to accumulate text into a single post as long as the same person is “talking”
  • Pinged back to Panos about JuryRoom. The original email evaporated, so I tried again…
  • Setting up a meeting with Wayne for Wednesday?
  • Fika – nope, meeting with Eric instead. The goal is to write up a whitepaper for human in the loop AI

Phil 2.10.19

Reading Clockwork Muse. Machines don’t really have a desire for novelty directly. But they do have the derivative of arousal potential. That comes from us, their creators. Machines on their own won’t change. Yet.  This is a manifestation of their non-evolutionary history. When machines can seek novelty and become bored is when they will become recognizable as entities.

Listening to Hidden Brain – One Head, Two Brains: How The Brain’s Hemispheres Shape The World We See (transcript)

  • This week on Hidden Brain, we dive into Iain’s research on how the left and right hemispheres shape our perceptions. Iain argues that differences in the brain — and Western society’s preference for what one hemisphere has to offer — have had enormous effects on our lives.
    • VEDANTAM: One of the important differences you point out is sort of understanding the role of metaphor in language. For example, which is that the left hemisphere really is incapable of understanding what metaphor is or how it works.
    • MCGILCHRIST: Yes. And that’s no small thing because as some philosophers have pointed out, metaphor is how we understand everything. And they point out that, actually, particularly scientific and philosophical understanding is mediated by metaphors. In other words, the only way we can understand something is in terms of something else that we think we already understand. And it’s making the analogy, which is what a metaphor does, that enables us to go, I see, I get it. Now, if you think that metaphor is just one of those dispensable decorations that you could add to meaning – it’s kind of nice but probably a distraction from the real meaning – you’ve got it upside down. Because if you don’t understand the metaphor, you haven’t understood the meaning. Literal meaning, however, is a peripheral, diminished version of the richness of metaphorical understanding. And what we know is the right hemisphere understands those implicit meanings, those connections of meanings, what we call connotations, as well as just denotations. It understands imagery. It understands humor. It understands all of that.
    • VEDANTAM: Have you ever wondered whether you, yourself, might be captive to your left hemisphere and you, potentially, now can’t see the problems with your own model?
    • MCGILCHRIST: That’s a very good point. And it’s not – I’m not critical of models, actually, in themselves. I’m critical of particular models because, in fact, we can’t understand anything – this is one of my basic points – except by having a model with which we compare it. So that is always a limitation. We don’t move from a world in which we have models to a better one in which we don’t. We move from a bad model to a better one. So every model has its limitations, but some form, simply, a better fit. And that is what the progress of science is.

This thread is wild. It’s how to force a stampede: YouTube

Phil 2.8.19

7:00 – 6:00 ASRC IRAD TL

  • Need to ping Eric about tasking. Suggest time series prediction. Speaking of which, Transformers (post 1 and post 2) may be much better than LSTMs for series prediction.
    • The Transformer model in Attention is all you need:a Keras implementation.
      • A Keras+TensorFlow Implementation of the Transformer: “Attention is All You Need” (Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, arxiv, 2017)
    • keras-transformer 0.17.0
      • Implementation of transformer for translation-like tasks.
    • The other option is “teachable” ML systems using evolution. There is a lot of interesting older work in this area:
      • Particle swarms for feedforward neural network training
      • Evolving artificial neural networks
      • Training Feedforward Neural Networks Using Genetic Algorithms.
        • Multilayered feedforward neural networks possess a number of properties which make them particularly suited to complex pattern classification problems. However, their application to some real world problems has been hampered by the lack of a training algorithm which reliably finds a nearly globally optimal set of weights in a relatively short time. Genetic algorithms are a class of optimization procedures which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum. Hence, they are well suited to the problem of training feedforward networks. In this paper, we describe a set of experiments performed on data from a sonar image classification problem. These experiments both 1) illustrate the improvements gained by using a genetic algorithm rather than backpropagation and 2) chronicle the evolution of the performance of the genetic algorithm as we added more and more domain-specific knowledge into it.
  • Add writing to the db from within the program, download the latest slack bundle, and try storing it!
  • Read in test-dungeon-1 and realized that there is no explicit link between the channel and the message in the data, so I added fields for the current directory and the current file
  • Ok, everything seems to be working. I had a few trips around the block getting a unique id for messages, but that seems ok now.
  • Created view(s), where I learned how to use conditionals and was happy:
    SELECT * FROM t_message;
    SELECT * FROM t_message_files;
    CREATE or REPLACE VIEW user_view AS
    SELECT,, p.real_name,
           (CASE WHEN p.display_name > '' THEN p.display_name ELSE END) as username
    FROM t_user u
           INNER JOIN t_user_profile p ON = p.parent_id;
    select * from user_view;
    CREATE or REPLACE VIEW post_view AS
    SELECT FROM_UNIXTIME(p.ts) as post_time, p.dirname as post_topic, p.text as post_text, u.username,
           (CASE WHEN p.subtype > '' THEN p.subtype ELSE p.type END) as type
    FROM t_message p
           INNER JOIN user_view u ON p.user =;
    select * from post_view order by post_time limit 1000;


  • Need to put together a strawman invitation that also has checkboxes for BB-based and/or Slack-based preferences and why a user might choose one over the other. Nope, not yet
  • Got the Slack academic discount!

Phil 2.7.19

7:00 – 7:00 ASRC IRAD TL

  • Continuing with Slack to DB process. I should be done with channels, and now I need to get conversations done.
    • The secondary tables that point to the primary user and conversation tables and the tertiary tables that point at them need to be looked at based on what happens when we go past the 10k limit (assuming that I can’t get the discount on the Standard Plan). REPLACE INTO won’t work with an auto incrementing primary key
    • Got all the parts working, now I need to automate and try out on Tymora1
    • Need to write up a letter for Don to sign – done
    • I think Emily is having a run tonight? Nope
    • Added a research_notes section to Slack for Aaron and I right now. I think I’ll invite Wayne as well – done! Need to know
  • Submitted expenses for TL trip
  • Was officially not invited to the TF dev conf
  • Shimei’s group meeting 4:30 – 7:00

Phil 2.6.19

7:00 – 5:00 ASRC IRAD (TL)

  • The role of maps in spatial knowledge acquisition
    • The Cartographic Journal
    • One goal of cartographic research is to improve the usefulness of maps. To do so, we must consider the process of spatial knowledge acquisition, the role of maps in that process, and the content of cognitive representations derived. Research from psychology, geography, and other disciplines related to these issues is reviewed. This review is used to suggest potential new directions for research with particular attention to spatial problem solving and geographic instruction. A classroom experiment related to these issues is then described. The experiment highlights some of the implications that a concern for the process of spatial knowledge acquisition will have on questions and methods of cartographic research as well as on the use of maps in geographic instruction. It also provides evidence of independent but interrelated verbal and spatial components of regional images that can be altered by directed map work.
  • It’s Not A Lie If You Believe It: Lying and Belief Distortion Under Norm-Uncertainty
    • This paper focuses on norm-following considerations as motivating behavior when lying opportunities are present. To obtain evidence on what makes it harder/easier to lie, we hypothesize that subjects might use belief-manipulation in order to justify their lying. We employ a two-stage variant of a cheating paradigm, in which subjects’ beliefs are elicited in stage 1 before performing the die task in stage 2. In stage 1: a) we elicit the subjects’ beliefs about majoritarian (i) behavior or (ii) normative beliefs in a previous session, and b) we vary whether participants are (i) aware or (ii) unaware of the upcoming opportunity to lie. We show that belief manipulation happens, and takes the form of people convincing themselves that lying behavior is widespread. In contrast with beliefs about the behavior of others, we find that beliefs about their normative convictions are not distorted, since believing that the majority disapproves of lying does not inhibit own lying. These findings are consistent with a model where agents are motivated by norm-following concerns, and honest behavior is a strong indicator of disapproval of lying but disapproval of lying is not a strong indicator of honest behavior. We provide evidence that supports this hypothesis.
  • Sent a note to Slack, asking for an academic plan. They do, and there are forms to fill out. I need to send Don some text that he can send back to me on letterhead.
  • Looks like I’m not going to the TF Dev Conf this year…
  • Continuing with the INSERT code
  • Meeting in Greenbelt to discuss… what, exactly?
  • Got a cool book: A Programmer’s Introduction to Mathematics
  • Got my converter creating error-free sql! t_user
  • Working on reading channel data into the db. Possibly done, but I’m afraid to run it so late in the day. I have chores!
  • Reviewing proposal for missing citations – done

Phil 2.5.19

7:00 – 5:00 ASRC IRAD

  • Got the parser to the point that it’s creating query strings, but I need to escape the text properly
  • Created and ab_slack mysql db
  • Added “parent_id” and an auto increment ID to any of the arrays that are associated with the Slack data
  • Reviewing sections 1-3 – done
  • Figure out some past performance – done
  • Work on the CV. Add the GF work and A2P ML work. – done
  • Start reimbursement for NJ trip
  •  Accidentally managed to start a $45/month subscription to the IEEE digital library. It really reeks of deceptive practices. There is nothing on the subscription page that informs you that this is a $45/month, 6-month minimum purchase. I’m about to contact the Maryland deceptive practices people to see if there is legal action that can be brought