From last night’s dungeon run: ” the ethics of Facebook is a good point about why killchain AI is inevitable”
And just to reinforce the point: Russia Unveils KUB-BLA “Kamikaze” Drone at IDEX 2019
Command Dysfunction: Minding the Cognitive War (1996)
Author: Arden B. Dahl
Institution: School of Advanced Airpower Studies Air University
- An analysis of Command and Control Warfare (C2W), which aims to create command dysfunction in the adversary.
- When viewed from an asymetric warfare perspective, this closely resembles the Gerazimov Doctrine
- Perception and cognition perform distinct roles in the formation of judgment. Perception answers the question: What do I see? Cognition answers the next question: How do I interpret it? However, general perceptual and cognitive biases cause decision makers to deviate from objectivity and make errors of judgment. Perceptual biases occur from the way the human mind senses the environment and tend to limit the accuracy of perception. Cognitive biases result from the way the mind works and tend to hinder accurate interpretation. These biases are general in that they are thought to be normally present in the general population of decision makers-regardless of their cultural background and organizational affiliations. (Page 14)
- This is also true of machine (or any) intelligence that is not omniscient. There are corollaries for group decision processes
- There are three perceptual biases that affect the accuracy of one’s view of the environment: the conditioning of expectations, the resistance to change and the impact of ambiguity. (Page 14)
- There are three primary areas in which cognitive biases degrade the accuracy of judgment within a decision process: (Page 16)
- the attribution of causality,
- the evaluation of probability and
- availability bias is a rule of thumb that works on the ease with which one can remember or recall other similar instances
- anchoring bias is a phenomenon in which decision makers adjust too little from their initial judgments as additional evidence becomes available.
- overconfidence bias is a tendency for individual decision makers to be subjectively overconfident about the extent and accuracy of their knowledge
- Other typical problems in estimating probabilities derive from the misunderstanding of statistics.
- the evaluation of evidence.
- Decision makers tend to value consistent information from a small data set over more variable information from a larger sample.
- Absence of Evidence bias is when decision makers to miss data in complicated problems. Analysts often do not recognize that data is missing and adjust the certainty of their inferences accordingly.
- The Persistence of Impressions bias follows a natural tendency to maintain first impressions concerning causality. It appears that the initial association of evidence to an outcome forms a strong cognitive linkage.
- AI systems can help with these errors of judgement though, since that can be explicitly programmed or placed in the training set.
- These are all contributors to Normal Accidents
- What about incorporating doctrine, rules of engagement and standard operating procedures? These can change dynamically and at different scales. (Allison Model II – Organizational Processes)
- Also, it should be possible to infer the adversaries’ rules and then find areas in the latent space that they do not cover. They will be doing the same to us. How to guard against this? Diversity?
- While the division of labor and SOP specialization is intended to make the organization efficient, the same division generates requirements to coordinate the intelligent collection and analysis of data.41 The failure to coordinate the varied perceptions and interests within the organization can lead to a number of uncoordinated rational decisions at the lower echelons, which in tum lead to an overall irrational outcome. (Page 20)
- There are two common cultural biases that deserve mention for their role in forming erroneous perceptions: arrogance and projection. Arrogance is the attitude of superiority over others or the opposing side. It can manifest as a national or individual perception. In the extreme case, it forgoes any serious search of alternatives or decision analysis beyond what the decision maker has already decided. It can become highly irrational. The projection bias sees the rest of the world through one’s own values and beliefs, thus tending to estimate the opposition’s intentions, motivations and capabilities as one’s own. (Page 21)
- Again, a good case for well-designed AI/ML. That being said, a commander’s misaligned biases may discount the AI system
- The overconfidence or hubris bias tends toward an overreaching inflation of one’s abilities and strengths. In the extreme it promotes a prideful self-confidence that is self-intoxicating and oblivious to rational limits. A decision maker affected with hubris will in his utter aggressiveness invariably be led to surprise and eventual downfall, The Hubris-Nemesis Complex is dangerous mindset that combines hubris (self intoxicating “pretension to godliness”) and nemesis (“vengeful desire” to wreak havoc and destroy). Leaders possessing this bias combination are not easily deterred or compelled by normal or rational solutions (Page 22)
- Three major decision stress areas include the consequential weight of the decision, uncertainty and the pressure of time (Page 23)
- Crisis settings complicate the use of rational and analytical decision processes in two ways. First, they add numerous unknowns, which in tum create many possible alternatives to the decision problem. Second, they reduce the time available to process and evaluate data, choose a course of action, and execute it.
- As uncertainty becomes severe, decision makers begin resorting to maladaptive search and evaluation methods to reach conclusions. Part of this may stem from a desire to avoid the anxiety of being unsure, an intolerance of ambiguity. It may also be that analytical approaches are difficult when the link between the data and the outcomes is not predictable
- Still true for ML systems, even without stress. Being forced to make shorter searches of the solution space (not letting the results converge, etc. could be an issue)
- The logic of dealing with the time pressure normally follows a somewhat standard pattern. Increasing time pressure first leads to an acceleration of information processing. Decision makers and their organizations will pick up the pace by expending additional resources to maintain existing decision strategies. As the pace begins to outrun in-place processing capabilities, decision makers reduce their data search and processing. In some cases this translates to increased selectivity, which the decision maker biases or weights toward details considered more important. In other cases, it does not change data collection but leads to a shallower data analysis. As the pace continues to increase, decision strategies begin to change. At this point major problems can creep into the process. The problems result from maladaptive strategies (satisficing, analogies, etc.) that save time but misrepresent data to produce inappropriate solutions. The lack of time also prevents critical introspection for perceptual and cognitive biases. In severe time pressure cases, the process may deteriorate to avoidance, denial or panic. (Page 26)
- The goal is to create this in the adversary, but not us. Which makes this in many respects a algorithm efficiency / processing power arms race
- In some decision situations, a timely, relatively correct response is better than an absolutely correct response that is made too late. In other words, the situation generates a tension between analysis and speed. (Page 30)
- The Recognition-Primed Decision (RPD) process works in the following manner. First, an experienced decision maker recognizes a problem situation as familiar or prototypical. The recognition brings with it a solution. The recognition also evokes an appreciation for what additional information to monitor: plausible outcomes, typical reactions, timing cues and causal dynamics. Second, given time, the decision maker evaluates his solution for suitability by testing it through mental simulation for pitfalls and needed adjustments. Normally, the decision maker implements the first solution “on the run” and makes adjustments as required. The decision maker will not be discard a solution unless it becomes plain that it is unworkable. If so, he will attempt a second option, if available. The RPD process is one of satisficing. It assumes that experienced decision makers identify a first solution that is “reasonably good” and are capable of mentally projecting its implementation. The RPD process also assumes that experienced decision makers are able to implement their one solution at any time during the process. (Page 31)
- It should be possible to train systems to approximate this, possibly at different levels of abstraction
- The RPD is a descriptive model that explains how experienced decision makers work problems in high stress decision situations (Page 31)
- It is reflexive, and as such well suited to ML techniques, assuming there is enough data…
- Situations that require the careful deployment of resources and analysis of abstract data, such as anticipating an enemy’s course of action, require an analytical approach. If there is time for analysis, a rational process normally provides a better solution for these kinds of problems (page 31)
- This is not what AI/ML is good at. As reaction requirements become tighter, these actions will have to be run in “slow motion” offline and used to train the system.
- The RPD model provides some insight as to how operational commanders survive in high-load, ambiguous and time pressured situations. The key seems to be experience. The experience serves as the base for what may be seen as an intuitive way to overcome stress. (Page 32)
- This is why training with attribution may be the best way. “Ms. XXX trained this system and I trust her” may be the best option. We may want to build a “stable” of machine trainers.
- Decision makers with more experience will tend to employ intuitive methods more often than analytical processes. This reliance on pattern recognition among experienced commanders may provide an opportunity for an adversary to manipulate the patterns to his advantage in deception operations. (Page 32)
Chapter 3: Considering a Cognitive Warfare Framework
- …an examination of John Boyd’s Observation-Orientation-Decision-Action (OODA) cycle to illustrate the different ways a C2W campaign may attack an adversary’s decision cycle. This sets the stage for analysis of the particular methods of such attacks.
- From Wikipedia: One of John Boyd’s primary insights in fighter combat was that it is vital to change speed and direction faster than the opponent. This may interfere with an opponent’s OODA cycle. It is not necessarily a function of the plane’s ability to maneuver, but the pilot must think and act faster than the opponent can think and act. Getting “inside” the cycle, short-circuiting the opponent’s thinking processes, produces opportunities for the opponent to react inappropriately.
- Once a group is adapting as fast as its arousal potential can tolerate, it will react in a linear way, since any deviation from the plan creates more arousal potential. Creating these stampedes, often simply through adversarial herding can create an extremely brittle vulnerable C2W cognitive framework.
- …degrading the efficiency of the decision cycle by denying the “observation” function the ability to see and impeding the flow of accurate information through the physical links of the loop. Data denial is usually achieved by preventing the adversary’s observation function, or sensors, from operating effectively in one or more channels. (Page 36)
- The second approach attempts to corrupt the adversary’s orientation. The focus is on the accuracy of the opponent’s perceptions and facts that inform his decisions, rather than their speed through the decision cycle. Operations security, deception and psychological operations (PSYOPS) are usually the primary C2W elements in the corruption effort.72 The corruption scheme’s relationship to decision speed is somewhat complicated. In fact, the corruption mechanism may work to vary the decision speed depending on the objective of the intended misperception. For example, the enemy might be induced to speedily make the wrong decision. (Page 37)
- B. H. Liddell Hart wrote: “…it is usually necessary for the dislocating move to be proceeded by a move, or moves, which can be best defined by the term ‘distract’ in its literal sense of ‘to draw asunder’. The purpose of this ‘distraction’ is to deprive the enemy of his freedom of action, and it should operate in the physical and psychological spheres.” (Page 42)
- The issue here is that we are adding a new “psychological” sphere – the domain of the intelligent machine. Since we have limited insight into the high-dimensional function that is the trained network, we cannot know when it is being successfully manipulated until it makes an obvious blunder, at which point it may be too late. This is one of the reasons that diversity needs to be built into the system so that there is a lower chance of a majority of systems being compromised.
- Fundamentally, all deception ploys are constructed in two parts: dissimulation and simulation. Dissimulation is covert, the act of hiding or obscuring the real; its companion, simulation, presents the false. Within this basic construct, deception programs are employed in two variants: A-type (ambiguity) and M-type (misdirection). The A-type deception seeks to increase ambiguity in the target’s mind. Its aim is to keep the adversary unsure of one’s true intentions, especially an adversary who has initially guessed right. A number of alternatives are developed for the target’s consumption, built on lies that are both plausible and sufficiently significant to cause the target to expend resources to cover them. The M-type deception is the more demanding variant. This deception misleads the adversary by reducing ambiguity, that is, attempting to convince him that the wrong solution is, in fact, “right.” In this case, the target positions most of his attention and resources in the wrong place. (Page 44)
- Both Sun-Tzu and Liddell Hart highlighted the dilemma of alternative objectives upon an adversary’s mind made possible by movement. (Page 45)
- Most important is the fact that the overall cognitive warfare approach is dependent upon the enemy’s command baseline–the decision making processes, command characteristics and expectations of the decision makers. The skillful employment of stress and deception against the command baseline may be a principal mechanism to bring about its cognitive dislocation. (Page 50)
- As the baseline becomes automated, then cognitive warfare must factor these aspects in.
Command Dysfunction: Minding the Cognitive War
- This paper analyzes the factors and conditions of command dysfunction from the cognitive, or mental, perspective of command and control warfare (C2W). The author examines the limitations of rational decision making and the tension that exists between rational and intuitive processes. Next, the paper examines the vulnerabilities of rational and intuitive processes in order to build a cognitive warfare framework. The framework consists of three categories: the command baseline, stressors, and deception. The stressor and deception categories act on the command baseline. The analysis also suggests that there are a number of possible interactions that exist between the stressor and deception categories. The paper then uses the framework to analyze evidence of command dysfunction in three historical campaigns. The historical analyses study the German command during the Normandy invasion, the Allied command during the first week of the Battle of the Bulge, and the Israeli command during the first half of the Arab-Israeli October 1973 War. In addition to showing that there are interactions between stressors and deception, the analyses highlight the importance of understanding the adversary’s command baseline. The paper concludes that effective C2W is not so much what is done to an adversary’s command, but rather what he does to himself, perhaps with a little help.
2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Streaming Tutorials, June 2nd. The following tutorials have been accepted for NAACL 2019 and will be held on Sunday, June 2nd, 2019. Exact timings will be posted later as part of the official schedule.
- T1: DEEP ADVERSARIAL LEARNING FOR NLP
- T2: DEEP LEARNING FOR NATURAL LANGUAGE INFERENCE
- T3: MEASURING AND MODELING LANGUAGE CHANGE
- T4: TRANSFER LEARNING IN NATURAL LANGUAGE PROCESSING
- T5: LANGUAGE LEARNING AND PROCESSING IN PEOPLE AND MACHINES
- T6: APPLICATIONS OF NATURAL LANGUAGE PROCESSING IN CLINICAL RESEARCH AND PRACTICE
Fighting disinformation across Google products
- Providing useful and trusted information at the scale that the Internet has reached is enormously complex and an important responsibility. Adding to that complexity, over the last several years we’ve seen organized campaigns use online platforms to deliberately spread false or misleading information. We have twenty years of experience in these information challenges and it’s what we strive to do better than anyone else. So while we have more work to do, we’ve beenworking hard to combat this challenge for many years.
7:00 – 5:00 ASRC IRAD (TL)
- Gen Studio is a way to navigate between designs in latent space. It is a prototype concept which was created over a two-day hackathon with collaborators across The Met, Microsoft, and MIT.
- Write up Clockwork Muse
- Continue with parsing, storing and report generation of slack data. Aaron had the idea that multiple statements by one person should be combined into a single post. Need to think about how that works in the report generation. Since the retrieved list is ordered by timestamp, the naive implementation is to accumulate text into a single post as long as the same person is “talking”
- Pinged back to Panos about JuryRoom. The original email evaporated, so I tried again…
- Setting up a meeting with Wayne for Wednesday?
- Fika – nope, meeting with Eric instead. The goal is to write up a whitepaper for human in the loop AI
Reading Clockwork Muse. Machines don’t really have a desire for novelty directly. But they do have the derivative of arousal potential. That comes from us, their creators. Machines on their own won’t change. Yet. This is a manifestation of their non-evolutionary history. When machines can seek novelty and become bored is when they will become recognizable as entities.
Listening to Hidden Brain – One Head, Two Brains: How The Brain’s Hemispheres Shape The World We See (transcript)
- This week on Hidden Brain, we dive into Iain’s research on how the left and right hemispheres shape our perceptions. Iain argues that differences in the brain — and Western society’s preference for what one hemisphere has to offer — have had enormous effects on our lives.
- VEDANTAM: One of the important differences you point out is sort of understanding the role of metaphor in language. For example, which is that the left hemisphere really is incapable of understanding what metaphor is or how it works.
- MCGILCHRIST: Yes. And that’s no small thing because as some philosophers have pointed out, metaphor is how we understand everything. And they point out that, actually, particularly scientific and philosophical understanding is mediated by metaphors. In other words, the only way we can understand something is in terms of something else that we think we already understand. And it’s making the analogy, which is what a metaphor does, that enables us to go, I see, I get it. Now, if you think that metaphor is just one of those dispensable decorations that you could add to meaning – it’s kind of nice but probably a distraction from the real meaning – you’ve got it upside down. Because if you don’t understand the metaphor, you haven’t understood the meaning. Literal meaning, however, is a peripheral, diminished version of the richness of metaphorical understanding. And what we know is the right hemisphere understands those implicit meanings, those connections of meanings, what we call connotations, as well as just denotations. It understands imagery. It understands humor. It understands all of that.
- VEDANTAM: Have you ever wondered whether you, yourself, might be captive to your left hemisphere and you, potentially, now can’t see the problems with your own model?
- MCGILCHRIST: That’s a very good point. And it’s not – I’m not critical of models, actually, in themselves. I’m critical of particular models because, in fact, we can’t understand anything – this is one of my basic points – except by having a model with which we compare it. So that is always a limitation. We don’t move from a world in which we have models to a better one in which we don’t. We move from a bad model to a better one. So every model has its limitations, but some form, simply, a better fit. And that is what the progress of science is.
This thread is wild. It’s how to force a stampede:
7:00 – 4:00 ASRC IRAD
Teaching a neural network to drive a car. It’s a simple network with a fixed number of hidden nodes (no NEAT), and no bias. Yet it manages to drive the cars fast and safe after just a few generations. Population is 650. The network evolves through random mutation (no cross-breeding). Fitness evaluation is currently done manually as explained in the video.
- This interactive balance between evolution and learning is exactly the sort of interaction that I think should be at the core of the research browser. The only addition is the ability to support groups collaboratively interacting with the information so that multiple analysts can train the system.
- A quick thing on the power of belief spaces from a book review about, of all things, Hell. One of the things that gives dimension to a belief space is the fact that people show up.
- Soon, he’d left their church and started one of his own, where he proclaimed his lenient gospel, pouring out pity and anger for those Christians whose so-called God was a petty torturer, until his little congregation petered out. Assured salvation couldn’t keep people in pews, it turned out. The whole episode, in its intensity and its focus on the stakes of textual interpretation, was reminiscent of Lucas Hnath’s recent play “The Christians,” about a pastor who comes out against Hell and sparks not relief but an exegetical nightmare.
- Web Privacy Measurement in Real-Time Bidding Systems. A Graph-Based Approach to Rtb System Classification.
- In the doctoral thesis, Robbert J. van Eijk investigates the advertisements online that seem to follow you. The technology enabling the advertisements is called Real-Time Bidding (RTB). An RTB system is defined as a network of partners enabling big data applications within the organizational field of marketing. The system aims to improve sales by real-time data-driven marketing and personalized (behavioral) advertising. The author applies network science algorithms to arrive at measuring the privacy component of RTB. In the thesis, it is shown that cluster-edge betweenness and node betweenness support us in understanding the partnerships of the ad-technology companies. From our research it transpires that the interconnection between partners in an RTB network is caused by the data flows of the companies themselves due to their specializations in ad technology. Furthermore, the author provides that a Graph-Based Methodological Approach (GBMA) controls the situation of differences in consent implementations in European countries. The GBMA is tested on a dataset of national and regional European news websites.
- Continuing with Tkinter and ttk
- That was easy!
- And now there is a scrollbar, which is a little odd to add. They are separate components that you have to explicitly link and place in the same ttk.Frame:
# make the frame for the listbox and the scroller to live in
self.lbox_frame = ttk.Frame(self.content_frame)
# place the frame
self.lbox_frame.grid(column=0, row=0, rowspan=6, sticky=(N,W,E,S))
# create the listbox and the scrollbar
self.lbox = Listbox(self.lbox_frame, listvariable=self.cnames, height=5)
lbox_scrollbar = ttk.Scrollbar(self.lbox_frame, orient=VERTICAL, command=self.lbox.yview)
# after both components have been made, have the lbox point at the scroller
self.lbox['yscrollcommand'] = lbox_scrollbar.set
- If you get this wrong, then you can end up with a scrollbar in some other Frame, connected to your target. Here’s what happens if the parent is root:
- And here is where it’s in the lbox frame as in the code example above:
- The fully formed examples are no more. Putting together a menu app with text. Got the text running with a scrollbar, and everything makes sense. Next is the menus…
- Here’s the version of the app with working menus:
- For seminar: Predictive Analysis by Leveraging Temporal User Behavior and User Embeddings
- The rapid growth of mobile devices has resulted in the generation of a large number of user behavior logs that contain latent intentions and user interests. However, exploiting such data in real-world applications is still difficult for service providers due to the complexities of user behavior over a sheer number of possible actions that can vary according to time. In this work, a time-aware RNN model, TRNN, is proposed for predictive analysis from user behavior data. First, our approach predicts next user action more accurately than the baselines including the n-gram models as well as two recently introduced time-aware RNN approaches. Second, we use TRNN to learn user embeddings from sequences of user actions and show that overall the TRNN embeddings outperform conventional RNN embeddings. Similar to how word embeddings benefit a wide range of task in natural language processing, the learned user embeddings are general and could be used in a variety of tasks in the digital marketing area. This claim is supported empirically by evaluating their utility in user conversion prediction, and preferred application prediction. According to the evaluation results, TRNN embeddings perform better than the baselines including Bag of Words (BoW), TFIDF and Doc2Vec. We believe that TRNN embeddings provide an effective representation for solving practical tasks such as recommendation, user segmentation and predictive analysis of business metrics.