Category Archives: Phil

Phil 12.19.18

7:00 – 4:30 ASRC PhD/NASA

  • I think the IEEE paper with Antonio should be something on the math behind diversity/chaos <-> ensembles <-> hierarchies/stampedes
  • Continuing with Normal Accidents
  • Thinking about costly signalling (economics, and more generally) WRT stampede theory. It’s a form of friction. Proof-of-work could be adaptively added to a system to inhibit stampedes?
    • In contract theory, signalling is the idea that one party (termed the agent) credibly conveys some information about itself to another party (the principal). For example, in Michael Spence’s job-market signalling model, (potential) employees send a signal about their ability level to the employer by acquiring education credentials. The informational value of the credential comes from the fact that the employer believes the credential is positively correlated with having greater ability and difficult for low ability employees to obtain. Thus the credential enables the employer to reliably distinguish low ability workers from high ability workers.
  • Tweak optimizer. Change callback method to replace_with_your_method(), and add some documentation on how to use the classes
  • Move clustering paper to LaTex folder – done!
  • Pinged Antonio about the new paper ideas
  • More predictive analytics? Kind of – setting up to read and write out time series to db
  • Upgrading my postgreSQL, to read/write from a table
    • Installed drivers and python packages
    • Reading and writing in the IDE
      • Created and populated a dummy data table
    • Reading in python. Writing is next
  • I was thinking that if we run out of D&D data for other map configurations, that I can use the current data with some repurposing (replace orc with xxx) to create conversations about other environments using human speech. Kind of like PCA for DNA.

Phil 12.17.18

7:00 – 4:30 ASRC NASA/PhD

  • Ted Radio Hour interview with Margaret Heffernan, who spoke about her book, Willful Blindness:
    • “Companies that have been studied for willful blindness can be asked questions like, are there issues at work that people are afraid to raise? And when academics have done studies like this of corporations in the United States, what they find is 85 percent of people say yes. Eighty-five percent of people know there’s a problem, but they won’t say anything. And when I duplicated the research in Europe, asking all the same questions, I found exactly the same number. And what’s really interesting is that when I go to companies in Switzerland, they tell me this is a uniquely Swiss problem. And when I go to Germany, they say, oh yes, this is the German disease. And when I go to companies in England they say, oh yeah, the British are really bad at this. And the truth is, this is a human problem. We’re all, under certain circumstances, willfully blind.”
    • I’ve been thinking about this a lot because when I say, well, why don’t people speak up? What I get is, oh, it’s the culture. And I think, well, what is the culture? The culture is the accumulation of everybody’s actions. And in many of the organizations I work with, change starts in very unexpected places because people just decide, I want to do this or I want to try this. And then they discover they don’t get shot. And then they discover that, actually, now, they’ve got a really exciting project. You know, I think the most dangerous thing in organizations is silence. It’s all those brains whizzing around full of observations and insight and ideas that are not being articulated.
    • I think that that the 15% who do speak out are Nomads. They are mis-aligned with the culture and as such it’s 1) Easier to see problems and solutions. 2) an inability to not behave independently.
  • Bayesian Layers: A Module for Neural Network Uncertainty
    • We describe Bayesian Layers, a module designed for fast experimentation with neural network uncertainty. It extends neural network libraries with layers capturing uncertainty over weights (Bayesian neural nets), pre-activation units (dropout), activations (“stochastic output layers”), and the function itself (Gaussian processes). With reversible layers, one can also propagate uncertainty from input to output such as for flow-based distributions and constant-memory backpropagation. Bayesian Layers are a drop-in replacement for other layers, maintaining core features that one typically desires for experimentation. As demonstration, we fit a 10-billion parameter “Bayesian Transformer” on 512 TPUv2 cores, which replaces attention layers with their Bayesian counterpart.
  • Continuing with Normal Accidents
  • Nice interactive on disinformation on Twitter
  • The universal decay of collective memory and attention
    • Collective memory and attention are sustained by two channels: oral communication (communicative memory) and the physical recording of information (cultural memory). Here, we use data on the citation of academic articles and patents, and on the online attention received by songs, movies and biographies, to describe the temporal decay of the attention received by cultural products. We show that, once we isolate the temporal dimension of the decay, the attention received by cultural products decays following a universal biexponential function. We explain this universality by proposing a mathematical model based on communicative and cultural memory, which fits the data better than previously proposed log-normal and exponential models. Our results reveal that biographies remain in our communicative memory the longest (20–30 years) and music the shortest (about 5.6 years). These findings show that the average attention received by cultural products decays following a universal biexponential function.
  • Zach walkthough
    • Yarn Workspaces
    • NextJS – Tools for developing React Apps – check the github repo to see, for example, how to roll your own web server
    • REACT hooks api
  • Got the basic recursion piece of the optimizer working right. Works for ints, floats, and strings: 
    def cascading_step(self):
    self.cur_val = self.range_array[self.index]
    print("{} cur_val = {}".format(self.name, self.cur_val))

    child_complete = True
    if self.child:
        child_complete = self.child.cascading_step()

    if child_complete:
        self.index += 1
        if self.index >= len(self.range_array):
            self.index = 0
            return True
    return False
  • And here’s the first working test: 
    v3 cur_val = v3_0
v2 cur_val = v2_0
v1 cur_val = v1_0
step 0 -----------
v3 cur_val = v3_0
v2 cur_val = v2_0
v1 cur_val = v1_1
step 1 -----------
v3 cur_val = v3_0
v2 cur_val = v2_0
v1 cur_val = v1_2
step 2 -----------
v3 cur_val = v3_0
v2 cur_val = v2_0
v1 cur_val = v1_3
step 3 -----------
v3 cur_val = v3_0
v2 cur_val = v2_1
v1 cur_val = v1_0

     

Phil 12.14.18

7:00 – 4:30 ASRC PhD/NASA

  • Sent Greg a couple of quick notes on using CNNs to match bacteria to phages.
  • Continuing with Normal Accidents
  • A Digital Test of the News: Checking the Web for Public Facts – Workshop report, December 2018
    • The Digital Test of the News workshop brought together digital sociologists, data visualisation and new media researchers at the Centre for Interdisciplinary Methodologies at the University of Warwick on 8 and 9 May 2018. The workshop is part of a broader research collaboration between the Centre for Interdisciplinary Methodologies and the Public Data Lab which investigates the changing nature of public knowledge formation in digital societies and develops inventive methods to capture and visualise knowledge dynamics online. Below we outline the workshop’s aims and outcomes.
  • Added plots to the NN code. Everything seems to look right. Looking at the individual weights in a layer is very informative. Need to add this kind of plotting to our keras code somehow:
  • Changing the coherence code so that the row values are zero or one. Actually, as the amount of data grows, BOW is getting more useful raw. This spreadsheet shows all posts, including the DM. Note that the word frequency is power law (R squared = .9352): All_posts
  • Started the optimizer and excel utils classes
  • NOAA meeting
  • NOAA meeting 2
  • Some thoughts from Aaron on our initial ML approach
    • I think for the first pass we do 1-2 models based on contract type focused on the $500k+ award contracts (120ish total).
    • We construct the inputs like sentences in the word generation LSTM with padded lengths equal to the longest running contract of that type, sorted by length. The model can be tested against current contracts held out in a test set using point by point prediction so we can show accuracy of the model against existing data and use that to set our accuracy threshold.
    • My guess is this will be at least an ORF/PAC model (two different primary contract types) which we can work on tuning with the learning_optimizer.py to get as accurate as possible in the timeframe we have.
    • One of the things we advertise as “next steps” is a detailed analysis of contracts based on similarity measures to identify a series of more accurate models. We can pair this with additional models such as Exponential Smoothing and ARIMA which use fundamentally the exact same pipeline.
    • The GUI will be plumbed up to show these analytic outputs on a per contract basis and we can show by the end of January a simple linear model and the LSTM model to demonstrate how Exponential Smoothing / ARIMA or model averages could be displayed. Once we have these outputs we can take the top 5 highest predicted UDO and display in a summary page so they can use those as a launching off point.
    • If we do it this way it means we only have to focus on the completion of TimeSeriesML LSTM and it’s data pipeline with a maximum of 2 initial models (contract type). I think that is a far more reasonable thing to complete in the timeframe and should still be really exciting to show off.

Phil 12.13.18

7:00 – 4:00 ASRC PhD/NASA

  • BBC Business Daily on making decisions under uncertainty. In particular, David Tuckett (Scholar), professor and director of the Centre for the Study of Decision-Making Uncertainty at University College London talks about how we reduce our sense of uncertainty by telling ourselves stories that we can then align with. This reminds me of how conspiracy theories develop, in particular the remarkable storyline of QAnon.
  • More Normal Accident review
  • NYTimes on frictionless design being a problem
  • Dungeon processing – broke out three workbooks for queries with all players, no dm, and just the dm. Also need to write up some code that generates the story on html.
  • Backprop debugging. I think it works? class_error
  • Here’s the core of the forward (train) and backpropagation (learn) code: 
    def train(self):
    if self.source != None:
        src = self.source
        self.neuron_row_array = np.dot(src.neuron_row_array, src.weight_row_mat)
        if(self.target != None): # No activation function to output layer
            self.neuron_row_array = relu(self.neuron_row_array) # TODO: use passed-in activation function
        self.neuron_col_array = self.neuron_row_array.T

def learn(self, alpha):
    if self.source != None:
        src = self.source
        delta_scalar = np.dot(self.delta, src.weight_col_mat)
        delta_threshold = relu2deriv(src.neuron_row_array) # TODO: use passed in derivative function
        src.delta = delta_scalar * delta_threshold
        mat = np.dot(src.neuron_col_array, self.delta)
        src.weight_row_mat += alpha * mat
        src.weight_col_mat = src.weight_row_mat.T
  • And here’s the evaluation: 
    • --------------evaluation
input: [[1. 0. 1.]] = pred: 0.983 vs. actual:[1]
input: [[0. 1. 1.]] = pred: 0.967 vs. actual:[1]
input: [[0. 0. 1.]] = pred: -0.020 vs. actual:[0]
input: [[1. 1. 1.]] = pred: 0.000 vs. actual:[0]

Phil 12.12.18

7:00 – 4:30 ASRC NASA/PhD

  • Do a dungeon analytic with new posts and DM for Aaron – done!
  • Send email to Shimei for registration and meeting after grading is finished
  • Start review of Normal Accidents – started!
  • Debug NN code – in process. Very tricky figuring out the relationships between the layers in backpropagation
  • Sprint planning
  • NASA meeting
  • Talked to Zach about the tagging project. Looks good, but I wonder how much time we’ll have. Got a name though – TaggerML

Phil 12.11.18

7:00 – 4:30 ASRC PhD/NASA

mercator_projection

Somehow, this needs to get into a discussion of the trustworthiness of maps

  • I realized that we can hand-code these initial dungeons, learn a lot and make this a baseline part of the study. This means that we can compare human and machine data extraction for map making. My initial thoughts as to the sequence are:
    • Step 1: Finish running the initial dungeon
    • Step 2: researchers determine a set of common questions that would be appropriate for each room. Something like:
      • Who is the character?
      • Where is the character?
      • What is the character doing?
      • Why is the character doing this?
    • Each answer should also include a section of the text that the reader thinks answers that question. Once this has been worked out on paper, a simple survey website (simpler) can be built that automates this process and supports data collection at moderate scales.
    • Use answers to populate a “Trajectories” sheet in an xml file and build a map!
    • Step 3: Partially automate the extraction to give users a generated survey that lets them select the most likely answer/text for the who/where/what/why questions. Generate more maps!
    • Step 4: Full automation
  • Added these thoughts to the analysis section of the google doc
  • The 11th International Natural Language Generation Conference
    • The INLG conference is the main international forum for the presentation and discussion of all aspects of Natural Language Generation (NLG), including data-to-text, concept-to-text, text-to-text and vision to-text approaches. Special topics of interest for the 2018 edition included:
      • Generating Text with Affect, Style and Personality,
      • Conversational Interfaces, Chatbots and NLG, and
      • Data-driven NLG (including the E2E Generation Challenge)
  • Back to grokking DNNs
    • Still building a SimpleLayer class that will take a set of neurons and create a weight array that will point to the next layer
    • array formatting issues. Tricky
    • I think I’m done enough to start debugging. Tomorrow
  • Sprint review

Phil 12.10.18

7:00 – 5:30 ASRC NASA/PhD

  • For my morning academic work, I am cooking delicious things.
  • There is text in the dungeon! Here’s what happened when I ran the analytics against 3 posts and held back the dungeon master. Rather than put up a bunch of screenshots, here’s the spreadsheet: Day_1_Dungeon_1
  • Russell Richie (twitter) (Scholar) One of my favorite results in the paper is that you can compress the embeddings 10x or more while preserving prediction performance, suggesting that the type of knowledge used to make these kind of judgments may only vary along a relative handful of latent dimensions.
    • dtwstpluuaasaa4
    • dtwqzvwv4aa7kot
  • Ok, back to grokking DNNs
    • Building a SimpleLayer class that will take a set of neurons and create a weight array that will point to the next layer
  • Fika and meeting with Wayne
    • Ade might be interested in doing some coding work!
    • Went over the initial results spreadsheet with Wayn. Overall, progress seems on track. He had an additional thought for venues that I didn’t note.
    • Ping Shimei about 899

Phil 12.7.18

7:00 – 4:30 ASRC NASA/PhD

Analyzing Discourse and Text Complexity for Learning and Collaborating

Analyzing Discourse and Text Complexity for Learning and Collaborating

Author: Mihai Dascalu

Notes

  • …informational level, coherence is most frequently accounted by: lexical chains (Morris and Hirst 1991; Barzilay and Elhadad 1997; Lapata and Barzilay 2005) (see 4.3.1 Semantic Distances and Lexical Chains), centering theory (Miltsakaki and Kukich 2000; Grosz et al. 1995) (see 4.2 Discourse Analysis and the Polyphonic Model) in which coherence is established via center continuation, or Latent Semantic Analysis (Foltz et al. 1993, 1998) (see 4.3.2 Semantic Similarity through Tagged LSA) used for measuring the cosine similarity between adjacent phrases
  • Among chat voices there are sequential and transversal relations, highlighting a specific point of view in a counterpointal way, as mentioned in previous work (Trausan-Matu and Rebedea 2009).
  • From a computational perspective, until recently, the goals of discourse analysis in existing approaches oriented towards conversations analysis were to detect topics and links (Adams and Martell 2008), dialog acts (Kontostathis et al. 2009), lexical chains (Dong 2006) or other complex relations (Rose et al. 2008) (see 3.1.3 CSCL Computational Approaches). The polyphonic model takes full advantage of term frequency – inverse document frequency Tf-Jdf (Adams and Martell 2008; Schmidt and Stone), Latent Semantic Analysis (Schmidt and Stone ; Dong 2006), Social Network Analysis (Dong 2006), Machine Learning (e.g., Nai”ve Bayes (Kontostathis et al. 2009), Support Vector Machines and Collin’s perceptron (Joshi and Rose 2007), the TagHelper environment (Rose et al. 2008) and the semantic distances from the lexicalized ontology WordNet (Adams and Martell 2008; Dong 2006). The model starts from identifying words and patterns in utterances that are indicators of cohesion among them and, afterwards, performs an analysis based on the graph, similar in some extent to a social network, and on threads and their interactions.
  • Semantic Distances and Lexical Chains: an ontology consists of a set of concepts specific to a domain and of the relations between pairs of concepts. Starting from the representation of a domain, we can define various distance metrics between concepts based on the defined relationships among them and later on extract lexical chains, specific to a given text that consist of related/cohesive concepts spanning throughout a text fragment or the entire document.
    • Lexicalized Ontologies and Semantic Distances: One of the most commonly used resources for English sense relations in terms of lexicalized ontologies is the WordNet lexical database (Fellbaum 1998; Miller I 995, 2010) that consists of three separate databases, one for nouns, a different one for verbs, and a third one for adjectives and adverbs. WordNet groups words into sets of cognitively related words (synsets), thus describing a network of meaningfully inter-linked words and concepts.
    • Nevertheless, we must also present the limitations of WordNet and of semantic distances, with impact on the development of subsequent systems (see 6 PolyCAFe – Polyphonic Conversation Analysis and Feedback and 7 ReaderBench (I) – Cohesion-based Discourse Analysis and Dialogism): I/ the focus only on common words, without covering any special domain vocabularies; 2/ reduced extensibility as the serialized model makes difficult the addition of new domain-specific concepts or relationships
    • Building the Disambiguation Graph:Lexical chaining derives from textual cohesion (Halliday and Hasan 1976) and involves the selection of related lexical items in a given text (e.g. , starting from Figure 8, the following lexical chain could be generated if all words occur in the initial text fragment: “cheater, person, cause, cheat, deceiver, . .. “). In other words, the lexical cohesive structure of a text can be represented as lexical chaining that consists of sequences of words tied together by semantic relationships and that can span across the entire text or a subsection of it. (Ontology-based chaining formulas on page 63)
    • The types of semantic relations taken into consideration when linking two words are hypernymy, hyponymy, synonymy, antonymy, or whether the words are siblings by sharing a common hypernym. The weights associated with each relation vary according to the strength of the relation and the proximity of the two words in the text analyzed.
  • Semantic Similarity through Tagged LSA: Latent Semantic Analysis (LSA) (Deerwester et al. 1989; Deerwester et al. 1990; Dumais 2004; Landauer and Dumais 1997) is a natural language processing technique starting from a vector-space representation of semantics highlighting the co-occurrence relations between terms and containing documents, after that projecting the terms in sets of concepts (semantic spaces) related to the initial texts. LSA builds the vector-space model, later on used also for evaluating similarity between terms and documents, now indirectly linked through concepts (Landauer et al. 1998a; Manning and Schi.itze 1999). Moreover, LSA can be considered a mathematical method for representing words’ and passages’ meaning by analyzing in an unsupervised manner a representative corpus of natural language texts.
    • In terms of documents size, semantically and topically coherent passages of approximately 50 to 100 words are the optimal units to be taken into consideration while building the initial matrix (Landauer and Dumais 2011).
      • This fits nicely to post size. Also a good design consideration for JuryRoom
    • Therefore, as compromise of all previous NLP specific treatments, the latest version of the implemented tagged LSA model (Dascalu et al. 2013a; Dascalu et al. 2013b) uses lemmas plus their corresponding part-of-speech, after initial input cleaning and stop words elimination.
  • Topic Relatedness through Latent Dirichlet Allocation
    • Starting from the presumption that documents integrate multiple topics, each document can now be considered a random mixture of corpus-wide topics. In order to avoid confusion, an important aspect needs to be addressed: topics within LDA are latent classes, in which every word has a given probability, whereas topics that are identified within subsequently developed systems (A .S.A.P., Ch.A.MP., Po/yCAFe and ReaderBench) are key concepts from the text. Additionally, similar to LSA, LDA also uses the implicit assumption of the bag of words approach that the order of words doesn’t matter when extracting key concepts and similarities of concepts through co-occurrences within a large corpus.
    • Every topic contains a probability for every word, but after the inference phase a remarkable demarcation can be observed between salient or dominant concepts of a topic and all other vocabulary words. In other words, the goal of LDA is to reflect the thematic structure of a document or of a collection through hidden variables and to infer this hidden structure by using a posterior inference model (Blei et al. 2003)
    • there are inevitably estimation errors, more notable when addressing smaller documents or texts with a wider spread of concepts, as the mixture of topics becomes more uncertain

Phil 12.6.18

7:00 – 4:00 ASRC PhD/NASA

  • Looks like Aaron has added two users
  • Create a “coherence” matrix, where the threshold is based on an average of one or more previous cells. The version shown below uses the tf-idf matrix as a source and checks to see if there are any non-zero values within an arbitrary span. If there are, then the target matrix (initialized with zeroes) is incremented by one on that span. This process iterates from a step of one (the default), to the specified step size. As a result, the more contiguous nonzero values are, the larger and more bell-curved the row sequences will be: spreadsheet3
  • Create a “details” sheet that has information about the database, query, parameters, etc. Done.
  • Set up a redirect so that users have to go through the IRB page if they come from outside the antibubbles site
  • It’s the End of News As We Know It (and Facebook Is Feeling Fine)
    • And as the platforms pumped headlines into your feed, they didn’t care whether the “news” was real. They didn’t want that responsibility or expense. Instead, they honed in on engagement—did you click or share, increasing value to advertisers?
      • Diversity (responsibility, expense), Stampede (engagement, share)
  • Finished Analyzing Discourse and Text Complexity for Learning and Collaborating, and created this entry for the notes.
  • Was looking at John Du Bois paper Towards a dialogic syntax, which looks really interesting, but seems like it might be more appropriate for spoken dialog. Instead, I think I’ll go to Claire Cardie‘s presentation on chat argument analysis at UMD tomorrow and see if that has better alignment.
    • Argument Mining with Structured SVMs and RNNs
      • We propose a novel factor graph model for argument mining, designed for settings in which the argumentative relations in a document do not necessarily form a tree structure. (This is the case in over 20% of the web comments dataset we release.) Our model jointly learns elementary unit type classification and argumentative relation prediction. Moreover, our model supports SVM and RNN parametrizations, can enforce structure constraints (e.g., transitivity), and can express dependencies between adjacent relations and propositions. Our approaches outperform unstructured baselines in both web comments and argumentative essay datasets.

Phil 12.5.18

7:00 – 4:30 ASRC PhD/NASA

Phil 12.4.18

7:00 – 8:00 (13 hrs) ASRC NASA/PhD

  • Put my discourse analysis finds here, so they don’t get lost.
  • Adding a bit more to my post that talks about inertial network behavior
  • Added xmlwriter, since Pandas can’t handle writing out dictionaries, though it can plot them just fine…
  • The test dungeon discourse sequence as a matrix. You can clearly see the three rooms in the top rows. Aaron and I agree that this is a cross-correlation signal processing problem. Next is to gather some real-world dataspreadsheet
  • Discussion with Aaron about next steps with Antonio. Basically say that we’re booked through April, but can review and comment.
  • IEEE Talk by Hai “Helen” Li of Duke University: