Author Archives: pgfeldman

Phil 12.15.20

Rolls-Royce’s Aletheia Framework is a practical toolkit that helps organizations to consider the impacts on people of using artificial intelligence prior to deciding whether to proceed. It looks across 32 facets of societal impact, governance and trust, and transparency and requires executives and boards to provide evidence that these have been rigorously considered.

10:00 – 1:00 Meeting with Mike D

GPT-Agents

  • Finish least-squares transform of embeddings to see what it looks like
  • I don’t see a straightforward way to produce and use a 4×4, so I’m starting with quaternions. Translation could be averaging the position of the points to calculate the center of the point cloud. The offset could be applied before the angle finding
  • I wound up using scipy.spatial.transform.Rotation. To apply the results of the matrix directly. Here’s the relevent code: 
from scipy.spatial.transform import Rotation as R
import numpy as np

kv = we.create_embedding_from_query("select content from table_output where experiment_id = 2 and probe = '{}' and tag = 'tweet';".format(probe), key_list)

# calculate the rotations to map one embedding onto the other
r:R
mapping = False
if len(kv_list) > 0:
    kv0 = kv_list[0]
    intersect = list(set(kv.index2word) & set(kv0.index2word))
    primary = []
    secondary = []
    for w in intersect:
        pi = kv0.index2word.index(w)
        si = kv.index2word.index(w)
        primary.append(kv0.vectors[pi])
        secondary.append(kv.vectors[si])
    A, res, rank, s = np.linalg.lstsq(primary, secondary, rcond=None)
    r = R.from_matrix(A)
    mapping = True
kv_list.append(kv)
  • And it seems to work pretty well! Different embeddings seem to line up pretty well, and the February embedding seems to be farther away from the July embedding, with words like “Mike” and “Pence” in Feb, and coronavirus and @govcuomo in July
https://viztales.com/wp-content/uploads/2020/12/embedding.gif
  • I can also do this with the regular data as well. Just limit the number of returns
  • 2:00 Meeting with Mike
  • 3:30 Meeting COVID meeting

GOES

  • Back to trying to figure out what’s going on.
  • Ok, I made some(?) progress. One of the really important things is for the rotation not to overshoot. If it does, there is a lot of oscillation as the algorithm overshoots, then overshoots as it comes back:
  • But when you fix that, everything looks pretty good!
  • Vadim thinks he may have fixed it, based on the behavior between the calculated and ‘physical’ vectors?

MDS

  • 4:30 resource discussion

Phil 12.14.20

I’m very conflicted about global warming in the winter. We just had a lovely weekend, where temperatures reached 60F. That’s very bad, but it sure feels nice. Didn’t stop me from getting my annual load of carbon offsets from the UN

GOES

  • 2:00 Meeting with Vadim

GPT Agents

  • Need to do some 3D embeddings in Gensim and display them.
  • Try doing a least-squares to rotate two embeddings to align with each other.
  • If that works, try seeing if embeddings from each month can be aligned in a reasonable way and what they look like
  • 3:00 meeting with Sim to work on similarity matrix

Phil 12.11.20

Post this ride for tomorrow

Word embeddings quantify 100 years of gender and ethnic stereotypes (2018 paper)

  • Word embeddings are a popular machine-learning method that represents each English word by a vector, such that the geometry between these vectors captures semantic relations between the corresponding words. We demonstrate that word embeddings can be used as a powerful tool to quantify historical trends and social change. As specific applications, we develop metrics based on word embeddings to characterize how gender stereotypes and attitudes toward ethnic minorities in the United States evolved during the 20th and 21st centuries starting from 1910. Our framework opens up a fruitful intersection between machine learning and quantitative social science.

Book

  • Made some good progress on influence and started dominance
  • 2:00 meeting with Michelle

GOES

  • Sent Vadim a note about catching up. Looks like 2:00 on Monday
  • More Plotly
  • I know it’s dumb, but I figured out how to do a favicon and webpage title. It’s quite simple. Put the favicon file in a folder called “assets” in the same directory as the Dash code. Setting the title is even easier. Add a “title=’my title'” to the dash.Dash() call: 
app = dash.Dash(__name__, title = 'Interactive!')
  • It looks like it is very important to load the data before interacting. Updating global data is bad.

GPT-Agents

  • Got 3D scatterplots working as well. Next I need to do some 3D embeddings in Gensim and display them. After that, try doing a least-squares to rotate two embeddings to align with each other. If that works, Try seeing if embeddings from each month can be aligned ina reasonable way and what they look like
  • Also, set up matrix distance code for Monday

Phil 12.10.20

High-Dimensional Data Analysis with Low-Dimensional Models: Principles, Computation, and Applications (textbook preprint)

  • The problem of identifying low-dimensional structures of signals or data in high-dimensional spaces is one of the most fundamental problems that, through a long history, interweaves many engineering and mathematical fields such as system theory, signal processing, pattern recognition, machine learning, and statistics.

Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models

  • Neural sequence models are widely used to model time-series data. Equally ubiquitous is the usage of beam search (BS) as an approximate inference algorithm to decode output sequences from these models. BS explores the search space in a greedy left-right fashion retaining only the top-B candidates – resulting in sequences that differ only slightly from each other. Producing lists of nearly identical sequences is not only computationally wasteful but also typically fails to capture the inherent ambiguity of complex AI tasks. To overcome this problem, we propose Diverse Beam Search (DBS), an alternative to BS that decodes a list of diverse outputs by optimizing for a diversity-augmented objective. We observe that our method finds better top-1 solutions by controlling for the exploration and exploitation of the search space – implying that DBS is a better search algorithm. Moreover, these gains are achieved with minimal computational or memory over- head as compared to beam search. To demonstrate the broad applicability of our method, we present results on image captioning, machine translation and visual question generation using both standard quantitative metrics and qualitative human studies. Further, we study the role of diversity for image-grounded language generation tasks as the complexity of the image changes. We observe that our method consistently outperforms BS and previously proposed techniques for diverse decoding from neural sequence models.

MORS

  • Presentation today at 11:40! Getting ORA-LITE running on the laptop – done. Now being nervous. Yay! Done. Good questions, too!

Phil 7.9.20

Center for Informed Democracy & Social – cybersecurity (IDeaS)

  • The overarching goal of the IDeaS Center at Carnegie Mellon University is to enhance social cyber-security to preserve and support and informed democratic society. The challenge today is that disinformation, hate speech, information warfare, and propaganda are amplified by cyber-technology like social media. We must remain informed, thoughtful members of our communities and countries, despite online and informational challenges.

Social Cybersecurity Working Group Focused on:

  • The science to characterize, understand, and forecast cyber-mediated changes in human behavior, social, cultural and political outcomes,
  • The science and engineering to build the cyber-infrastructure needed for society to persist in its essential character in a cyber-mediated information environment under changing conditions, actual or imminent social cyber-threats

Glamorous Toolkit is the moldable development environment. It is a live notebook. It is a flexible search interface. It is a fancy code editor. It is a software analysis platform. It is a data visualization engine. All in one.

This looks pretty interesting. Need to spend some time digging deeper. Friday?

“The antibody response to the virus has been shown to be transient and these antibodies start to wane after 3 to 4 months,” he said, adding that at 6 months they are “mostly undetectable” in many people who were infected early on in the epidemic,” (via Reuters)

MORS

  • 11:30 – 12:10 New Adaptive Sampling Techniques to Optimally Augment Complex Simulation Models
  • 12:10 – 12:50 Influence Planning Product
  • 12:50 – 1:30 Risk Management for irregular Events
  • 2:00 – 3:00 Social Cybersecurity
  • 3:40 – 4:20 Community Resilience Indicator Analysis

4:00 Meeting with Matthew?

JuryRoom

  • 5:00 Meeting

Phil 12.8.20

Chuck Yeager died today. He was born in 1923. Here’s what was flying the year he was born:

https://www.mainememory.net/artifact/23897

And here’s what is about to fly as early as tomorrow:

https://www.spacex.com/vehicles/starship/

Social cybersecurity: an emerging science

  • With the rise of online platforms where individuals could gather and spread information came the rise of online cybercrimes aimed at taking advantage of not just single individuals but collectives. In response, researchers and practitioners began trying to understand this digital playground and the way in which individuals who were socially and digitally embedded could be manipulated. What is emerging is a new scientific and engineering discipline—social cybersecurity. This paper defines this emerging area, provides case examples of the research issues and types of tools needed, and lays out a program of research in this area.
  • In today’s high tech world, beliefs opinions and attitudes are shaped as people engage with others in social media, and through the internet. Stories from creditable news sources and finding from science are challenged by actors who are actively engaged in influence operations on the internet. Lone wolfs, and large propaganda machines both disrupt civil discourse, sew discord and spread disinformation. Bots, cyborgs, trolls, sock-puppets, deep fakes, and memes are just a few of the technologies used in social engineering aimed at undermining civil society and supporting adversarial or business agendas. How can social discourse without undue influence persist in such an environment? What are the types of tools and theories needed to support such open discourse?
  • Today scientists from a large number of disciplines are working collaboratively to develop these new tools and theories. There work has led to the emergence of a new area of science—social cybersecurity. Herein, this emerging scientific area is described. Illustrative case studies are used to showcase the types of tools and theories needed. New theories and methods are also described.

MORS

  • Email to Dr. Carley – done!
  • Really nice talk by Dr. Michiel Deskevich at OptTek:
  • Information Warfare panel. Started with Gerasimov, which is pretty cool
https://viztales.com/wp-content/uploads/2020/12/image-13.png

GPT-2

  • 3:30 Meeting
    • Need to set up a meeting with Sim to tag-team together a cosine similarity for the GPT embedding.
      • I think it can be lazy, and calculate the CS as it goes.
      • Save the current distance matrix out as a csv, and read it in the next time, so that it continues to grow
      • Can use the training corpora to create a set of words as a baseline matrix
      • For words that have more than 1 embedding, have subsequent distance be specified in the matrix as “foo”, “foo1”, … “fooN”. That lets distance calculations be performed between the variants, and also to point back at the correct usage easily

Phil 12.7.20

Brrrr

5:00 Meeting with Naveen

I looked into the Association of Computational Linguistics as a possible venue for the chess paper. Aside from being a bit shorter (8 pages), the difference between the ACL papers that I looked at and an mine seems to be mostly the amount of explicit math in the description of the algorithm. Here are some examples from 2020 that I think are in roughly the same area:

Adjusting the citations to include some ACL papers (like the last one should be straightforward). The page count will have to be evaluated once the template is made public. Here’s the 2020 template: http://aacl2020.org/calls/papers/#paper-submission-and-templates

MORS

GOES

  • 3:00 Meeting with Vadim

Phil 12.4.20

UMBC now has a subscription to PolicyMap, a GIS tool that allows users to create maps, tables, and reports from a variety of datasets ranging from demographics, income, health, education, more. Maps can be created as single sheets or with multiple layers from the zip code / block level to worldwide.

Users can create individual accounts to save, share, and print work. A suite of tutorials is available to help both new and experienced users work with the tool effectively.

This is a lot of fun, and not related to anything: The Siege of Gondor, Part I: Professionals Talk Logistics

GOES

  • More Plotly

GPT Agents

Book

  • Working on Attention
  • Downloaded some seaturtle data from here: seamap.env.duke.edu
  • Changed over to storks, because I can’t find any example of a female seaturtle choosing a new beach
  • 2:00 Meeting with Michelle

Phil 12.3.20

It’s been a year since we heard about COVID-19 for the first time. Let’s see how things are going. First, the selection of countries that I’ve been tracking:

https://public.flourish.studio/visualisation/4504138/

Ouch. Germany and Finland seem to be doing well in Europe, but the rest… It looks like it’s going to be a bad winter. I think it is interesting how countries like France, Italy and Switzerland that seemed to have things under control are now at USA levels of deaths per million.

Let’s see how the US is doing:

https://public.flourish.studio/visualisation/4303726/

The hard-hit eastern states still look a lot like the parts of Europe that are still on top of the spread. Georgia, Mississippi, and the Dakotas look very bad. Washington and California, which were hit early, are still experiencing very low rates. I guess we’ll see how this chart looks in January. If there is a Thanksgiving-related surge, we should see it by then.

Book

  • Work on attention

GOES

  • 10:00 Meeting with Vadim. Pymoo is much better to install than Pyomo. It’s API seems more straightforward too. Vadim is working on figuring out the examples
  • 2:00 Meeting. Just a quick status
  • Status report
  • Long chat with Aaron

GPT-2 Agents

  • The generated HTML file to make that chart is huge, btw. It’s 2.9MB.
  • And it’s slooooooow if you just use fig.show(). fig.write_html(‘file_name.html’, auto_open=True) is much faster. Ok. That means the figures can be saved as dynamic pages, which is kind of cool.
  • Got dash running, which set up a server for your interactive graphs. Not really sure which one is better, though I’m guessing that data can be live inside dash graphs. I don’t think this will matter too much with the embedding charts, but it’s good to know
  • Hot-reloading is cool, and works with data or text changes. And the management of the html is nice. It appears to be based on a React engine and it’s nice to not have to care!
  • CSS-type styling works! If you make an error in the code, the program bails with an error message
  • 3:30 Meeting – cancelled

Phil 12.2.20

Call stove repair 

IntelliJ has added CodeWithMe!

I love this kind of simplifying, generalizing research:

https://twitter.com/samgreydanus/status/1333887306940387329

Book

  • More on cults, probably. Just need to get started writing again after the break – made a lot of progress!

GPT-2

  • Look at libraries for plotting embeddings interactively. The OpenGL developer in me is digging VisPy

GOES

  • SATERN training
  • Register for MORS!!! – Done!
  • 1:30 meeting with Vadim
    • Went over the Pyomo api, which is very complicated to install. It works, but getting the solvers to work in the API call framework requires all kinds of additional work.
  • 2:00 Meeting

Phil 12.1.20

Language Through a Prism: A Spectral Approach for Multiscale Language Representations (Twitter summary)

  • Language exhibits structure at different scales, ranging from subwords to words, sentences, paragraphs, and documents. To what extent do deep models capture information at these scales, and can we force them to better capture structure across this hierarchy? We approach this question by focusing on individual neurons, analyzing the behavior of their activations at different timescales. We show that signal processing provides a natural framework for separating structure across scales, enabling us to 1) disentangle scale-specific information in existing embeddings and 2) train models to learn more about particular scales. Concretely, we apply spectral filters to the activations of a neuron across an input, producing filtered embeddings that perform well on part of speech tagging (word-level), dialog speech acts classification (utterance-level), or topic classification (document-level), while performing poorly on the other tasks. We also present a prism layer for training models, which uses spectral filters to constrain different neurons to model structure at different scales. Our proposed BERT + Prism model can better predict masked tokens using long-range context and produces multiscale representations that perform better at utterance- and document-level tasks. Our methods are general and readily applicable to other domains besides language, such as images, audio, and video.

A Visual Guide to Regular Expression

https://twitter.com/emollick/status/1333571781727318019

This could be something for diversity injection?

Corporate Reporting in the Era of Artificial Intelligence

  • The researchers find that companies expecting higher levels of machine readership prepare their disclosures in ways that are more readable by this audience. “Machine readability” is measured in terms of how easily the information can be processed and parsed, with a one standard deviation increase in expected machine downloads corresponding to a 0.24 standard deviation increase in machine readability. For example, a table in a disclosure document might receive a low readability score because its formatting makes it difficult for a machine to recognize it as a table. A table in a disclosure document would receive a high readability score if it made effective use of tagging so that a machine could easily identify and analyze the content.

GPT-2 Agents

  • I want to create a database for generated output. There are two tables:
    • table_experiment – done!
      • Contains the experiment details:
        • id (key)
        • Date
        • Probe list
        • all hyperparameters
    • table_output – done!
      • id
      • experiment_id
      • root_id
      • tag (e.g. “raw”, “date”, “location”, “tweet”
      • depth (this is the index of each piece of content. Raw is 0, then each parsed out section increases depth by 1)
      • content
      • regexes
  • Created a gpt_experiments database. I need to make sure that I can read from one db and write to another
  • Good results on the test. Need to try something at a larger scale to test the embeddings:
https://viztales.com/wp-content/uploads/2020/12/image-1.png
  • 3:30 Meeting. Get script for Antonio
    • Getting small models for the long and short training sets
    • Look into embedding visualizer
    • Send Antonio info on the COVID Twitter stream while Sim assembles the scripts

GOES

  • Register for MORS
  • Status report for November

Phil 11.30.20

Call Verizon

Call stove repair – done

GPT-2 Agents

  • Upload db and corporas – done

GOES

  • Slides and presentation – done
  • 2:00 meeting with Vadim – delayed. I’m getting worried that he’s flailing again

SBIR

  • 10:00 meeting – done, waiting for names and emails

COE Meeting

  • I have my doubts if this is going to work. It seems more like an attempt by BD to get us to create a body of text that they can mine for proposals

Meeting with Aaron

  • Caught up on a lot of things

Phil 11.25.20

IRS stuff

Enjoying Google Research’s Verse by Verse

GPT-2 Agents

  • I think I want to put together a small command-line app that allows a discussion with the language model. All text from the ongoing conversation is saved and and used as the input for the next. A nice touch would be to have some small number of responses to choose from, and the conversation follows that branch.
  • Come to think of it, that could be a cool artificial JuryRoom/Eliza
  • Generate compact text for Sim to try training
  • Look into W2V 3D embedding of outputs, and mapping to adjacent outputs (The wo/man walked into the room). We know that there should be some level of alignment

GOES

  • Rehearse/record talk
  • Meeting with Vadim. He found Pyomo

Book

  • Working on the Attention + Dominance section

Phil 11.24.20

Anonymous 4: Alleluia: Gratulemur et letemur is lovely

PyTorch BigGraph is an Open Source Framework for Processing Large Graphs

  • Graphs are one of the fundamental data structures in machine learning applications. Specifically, graph-embedding methods are a form of unsupervised learning, in that they learn representations of nodes using the native graph structure. Training data in mainstream scenarios such as social media predictions, internet of things(IOT) pattern detection or drug-sequence modeling are naturally represented using graph structures. Any one of those scenarios can easily produce graphs with billions of interconnected nodes. While the richness and intrinsic navigation capabilities of graph structures is a great playground for machine learning models, their complexity posses massive scalability challenges. Not surprisingly, the support for large-scale graph data structures in modern deep learning frameworks is still quite limited. Recently, Facebook unveiled PyTorch BigGraph, a new framework that makes it much faster and easier to produce graph embeddings for extremely large graphs in PyTorch models.

GOES

  • Add composite rotation vector to ddict output. It’s kind of doing what it’s supposed to
  • Think about a NN to find optimal contributions? Or simultaneous solution of the scalars to produce the best approximation of the line? I think this is the way to go. I found pymoo: Multi-objective Optimization in Python
    • Our framework offers state of the art single- and multi-objective optimization algorithms and many more features related to multi-objective optimization such as visualization and decision making. Going to ask Vadim to see if it can be used for our needs
  • MORS talk, headshots, slides, etc
  • 11:00 meeting with Vadim

GPT-2 Agents

Phil 11.23.20

Call Jim Donnie’s! Sheesh – they are booked until January. Going to bring it by for someone to look at

Got the reviews back from AAMAS. Not as bad as SASO, but not great. I think I’ll write a rebuttal because why not? And it’s only 800 words

Call for Papers for Special Issue: Artificial Speakers – Philosophical Questions and Implications

  • With the increasing ubiquity of natural language processing (NLP) algorithms, interacting with “conversational artificial agents” such as speaking robots, chatbots, and personal assistants will be an everyday occurrence for most people. In a rather innocuous sense, we can perform a variety of speech acts with them, from asking a question to telling a joke, as they respond to our input just as any other agent would.

Book

  • Write some of the “Attention + Dominance” paper/chapter outline for Antonio. It’s important to mention that these are monolithic models. It could be a nice place for the Sanhedren 17a discussion too.

GOES

  • Rework primary_axis_rotations.py to use least-squares. It’s looking pretty good!
https://viztales.com/wp-content/uploads/2020/11/image-11.png
  • Folding into sim.
https://viztales.com/wp-content/uploads/2020/11/replayer_11_23_20.gif
  • It’s still not right, dammit! I’m beginning to wonder if the rwheels are correct? Wheels 1 and 4 are behaving oddly, and maybe 3. It’s like they may be spinning the wrong way?
  • Nope, it looks like it is the way the reaction wheel contributions are being calculated?
https://viztales.com/wp-content/uploads/2020/11/image-12.png
  • I think this is enlightening. There seems to be some kind of interplay between the computed rotation and the approximation based on the rwheels:
https://viztales.com/wp-content/uploads/2020/11/image-13.png