Category Archives: Keras

Phil 9.12.19

7:00 – 4:30 ASRC GOES

  • FractalNet: Ultra-Deep Neural Networks without Residuals
    • We introduce a design strategy for neural network macro-architecture based on self-similarity. Repeated application of a simple expansion rule generates deep networks whose structural layouts are precisely truncated fractals. These networks contain interacting subpaths of different lengths, but do not include any pass-through or residual connections; every internal signal is transformed by a filter and nonlinearity before being seen by subsequent layers. In experiments, fractal networks match the excellent performance of standard residual networks on both CIFAR and ImageNet classification tasks, thereby demonstrating that residual representations may not be fundamental to the success of extremely deep convolutional neural networks. Rather, the key may be the ability to transition, during training, from effectively shallow to deep. We note similarities with student-teacher behavior and develop drop-path, a natural extension of dropout, to regularize co-adaptation of subpaths in fractal architectures. Such regularization allows extraction of high-performance fixed-depth subnetworks. Additionally, fractal networks exhibit an anytime property: shallow subnetworks provide a quick answer, while deeper subnetworks, with higher latency, provide a more accurate answer.
  • Structural diversity in social contagion
    • The concept of contagion has steadily expanded from its original grounding in epidemic disease to describe a vast array of processes that spread across networks, notably social phenomena such as fads, political opinions, the adoption of new technologies, and financial decisions. Traditional models of social contagion have been based on physical analogies with biological contagion, in which the probability that an individual is affected by the contagion grows monotonically with the size of his or her “contact neighborhood”—the number of affected individuals with whom he or she is in contact. Whereas this contact neighborhood hypothesis has formed the underpinning of essentially all current models, it has been challenging to evaluate it due to the difficulty in obtaining detailed data on individual network neighborhoods during the course of a large-scale contagion process. Here we study this question by analyzing the growth of Facebook, a rare example of a social process with genuinely global adoption. We find that the probability of contagion is tightly controlled by the number of connected components in an individual’s contact neighborhood, rather than by the actual size of the neighborhood. Surprisingly, once this “structural diversity” is controlled for, the size of the contact neighborhood is in fact generally a negative predictor of contagion. More broadly, our analysis shows how data at the size and resolution of the Facebook network make possible the identification of subtle structural signals that go undetected at smaller scales yet hold pivotal predictive roles for the outcomes of social processes.
    • Add this to the discussion section – done
  • Dissertation
    • Started on the theory section, then realized the background section didn’t set it up well. So worked on the background instead. I put in a good deal on how individuals and groups interact with the environment differently and how social interaction amplifies individual contribution through networking.
  • Quick meetings with Don and Aaron
  • Time prediction (sequence to sequence) with Keras perceptrons
  • This was surprisingly straightforward
    • There was some initial trickiness in getting the IDE to work with the TF2.0 RC0 package:
      import tensorflow as tf
      from tensorflow import keras
      from tensorflow_core.python.keras import layers

      The first coding step was to generate the data. In this case I’m building a numpy matrix that has ten variations on math.sin(), using our timeseriesML utils code. There is a loop that sets up the code to create a new frequency, which is sent off to get back a pandas Dataframe that in this case has 10 sequence rows with 100 samples. First, we set the global sequence_length:

      sequence_length = 100

      then we create the function that will build and concatenate our numpy matrices:

      def generate_train_test(num_functions, rows_per_function, noise=0.1) -> (np.ndarray, np.ndarray, np.ndarray):
          ff = FF.float_functions(rows_per_function, 2*sequence_length)
          npa = None
          for i in range(num_functions):
              mathstr = "math.sin(xx*{})".format(0.005*(i+1))
              #mathstr = "math.sin(xx)"
              df2 = ff.generateDataFrame(mathstr, noise=0.1)
              npa2 = df2.to_numpy()
              if npa is None:
                  npa = npa2
              else:
                  ta = np.append(npa, npa2, axis=0)
                  npa = ta
      
          split = np.hsplit(npa, 2)
          return npa, split[0], split[1]

      Now, we build the model. We’re using keras from the TF 2.0 RC0 build, so things look slightly different:

      model = tf.keras.Sequential()
      # Adds a densely-connected layer with 64 units to the model:
      model.add(layers.Dense(sequence_length, activation='relu', input_shape=(sequence_length,)))
      # Add another:
      model.add(layers.Dense(200, activation='relu'))
      # Add a softmax layer with 10 output units:
      model.add(layers.Dense(sequence_length))
      
      loss_func = tf.keras.losses.MeanSquaredError()
      opt_func = tf.keras.optimizers.Adam(0.01)
      model.compile(optimizer= opt_func,
                    loss=loss_func,
                    metrics=['accuracy'])

      We can now fit the model to the generated data:

      full_mat, train_mat, test_mat = generate_train_test(10, 10)
      
      model.fit(train_mat, test_mat, epochs=10, batch_size=2)

      There is noise in the data, so the accuracy is not bang on, but the loss is nice. We can see this better in the plots above, which were created using this function:

      def plot_mats(mat:np.ndarray, cluster_size:int, title:str, fig_num:int):
          plt.figure(fig_num)
      
          i = 0
          for row in mat:
              cstr = "C{}".format(int(i/cluster_size))
              plt.plot(row, color=cstr)
              i += 1
      
          plt.title(title)

      Which is called just before the program completes:

      if show_plots:
          plot_mats(full_mat, 10, "Full Data", 1)
          plot_mats(train_mat, 10, "Input Vector", 2)
          plot_mats(test_mat, 10, "Output Vector", 3)
          plot_mats(predict_mat, 10, "Predict", 4)
          plt.show()
    • That’s it! Full listing below:
import tensorflow as tf
from tensorflow import keras
from tensorflow_core.python.keras import layers
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import timeseriesML.generators.float_functions as FF


sequence_length = 100

def generate_train_test(num_functions, rows_per_function, noise=0.1) -> (np.ndarray, np.ndarray, np.ndarray):
    ff = FF.float_functions(rows_per_function, 2*sequence_length)
    npa = None
    for i in range(num_functions):
        mathstr = "math.sin(xx*{})".format(0.005*(i+1))
        #mathstr = "math.sin(xx)"
        df2 = ff.generateDataFrame(mathstr, noise=0.1)
        npa2 = df2.to_numpy()
        if npa is None:
            npa = npa2
        else:
            ta = np.append(npa, npa2, axis=0)
            npa = ta

    split = np.hsplit(npa, 2)
    return npa, split[0], split[1]

def plot_mats(mat:np.ndarray, cluster_size:int, title:str, fig_num:int):
    plt.figure(fig_num)

    i = 0
    for row in mat:
        cstr = "C{}".format(int(i/cluster_size))
        plt.plot(row, color=cstr)
        i += 1

    plt.title(title)

model = tf.keras.Sequential()
# Adds a densely-connected layer with 64 units to the model:
model.add(layers.Dense(sequence_length, activation='relu', input_shape=(sequence_length,)))
# Add another:
model.add(layers.Dense(200, activation='relu'))
# Add a softmax layer with 10 output units:
model.add(layers.Dense(sequence_length))

loss_func = tf.keras.losses.MeanSquaredError()
opt_func = tf.keras.optimizers.Adam(0.01)
model.compile(optimizer= opt_func,
              loss=loss_func,
              metrics=['accuracy'])

full_mat, train_mat, test_mat = generate_train_test(10, 10)

model.fit(train_mat, test_mat, epochs=10, batch_size=2)
model.evaluate(train_mat, test_mat)

# test against freshly generated data
full_mat, train_mat, test_mat = generate_train_test(10, 10)
predict_mat = model.predict(train_mat)

show_plots = True
if show_plots:
    plot_mats(full_mat, 10, "Full Data", 1)
    plot_mats(train_mat, 10, "Input Vector", 2)
    plot_mats(test_mat, 10, "Output Vector", 3)
    plot_mats(predict_mat, 10, "Predict", 4)
    plt.show()



Phil 9.11 . 19

be7c6582-044a-4a19-aa8b-de388b4a4f83-cincpt_09-11-2016_enquirer_1_b001__2016_09_10_img_xxx_world_trade_11_1_1_9kfm0g4g_l880019336_img_xxx_world_trade_11_1_1_9kfm0g4g

7:00 – 4:00 ASRC GOES

  • Model:DLG3501W SKU:6181264
  • Maryland Anatomy Board Dept of vital records 410 764 2922
  • arxiv-vanity.com  arXiv Vanity renders academic papers from arXiv as responsive web pages so you don’t have to squint at a PDF.
    • It works ok. Tables and cation alignment are a problem for now, but it sounds great for phones
  • DeepPrivacy: A Generative Adversarial Network for Face Anonymization
    • We propose a novel architecture which is able to automatically anonymize faces in images while retaining the original data distribution. We ensure total anonymization of all faces in an image by generating images exclusively on privacy-safe information. Our model is based on a conditional generative adversarial network, generating images considering the original pose and image background. The conditional information enables us to generate highly realistic faces with a seamless transition between the generated face and the existing background. Furthermore, we introduce a diverse dataset of human faces, including unconventional poses, occluded faces, and a vast variability in backgrounds. Finally, we present experimental results reflecting the capability of our model to anonymize images while preserving the data distribution, making the data suitable for further training of deep learning models. As far as we know, no other solution has been proposed that guarantees the anonymization of faces while generating realistic images.
  • Introducing a Conditional Transformer Language Model for Controllable Generation
    • CTRL is a 1.6 billion-parameter language model with powerful and controllable artificial text generation that can predict which subset of the training data most influenced a generated text sequence. It provides a potential method for analyzing large amounts of generated text by identifying the most influential source of training data in the model. Trained with over 50 different control codes, the CTRL model allows for better human-AI interaction because users can control the generated content and style of the text, as well as train it for multitask language generation. Finally, it can be used to improve other natural language processing (NLP) applications either through fine-tuning for a specific task or through transfer of representations that the model has learned.
  • Dissertation
    • Started to put together my Linux laptop for vacation writing
    • More SIH section
  • Verify that timeseriesML can be used as a library
  • Perceptron curve prediction
  • AI/ML status meetings
  • Helped Vadim with some python issues

Phil 9.10.19

ASRC GOES 7:00 – 5:30

  • Got a mention in an article on Albawaba – When the Only Option is ‘Not to Play’? Autonomous Weapons Systems Debated in Geneva 
  • Dissertation – more SIH
  • Just saw this: On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
    • We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization. We perform a simple extractive step before generating a summary, which is then used to condition the transformer language model on relevant information before being tasked with generating a summary. We show that this extractive step significantly improves summarization results. We also show that this approach produces more abstractive summaries compared to prior work that employs a copy mechanism while still achieving higher rouge scores. Note: The abstract above was not written by the authors, it was generated by one of the models presented in this paper.
  • Working on packaging timeseriesML. I think it’s working!

TimeSeriesML

  • I’ll try it out when I get back after lunch
  • Meeting with Vadim
    • Showed him around and provided svn access
  • Model:DLG3501W SKU:6181264

Phil 9.5.19

7:00 –

  • David Manheim (scholar)
    • I work on existential risk mitigation, computational modelling, and epidemiology. I spend time talking about Goodhart’s Law, and have been a #Superforecaster with the Good Judgement Project since 2012.
  • Goodhart’s law is an adage named after economist Charles Goodhart, which has been phrased by Marilyn Strathern as “When a measure becomes a target, it ceases to be a good measure.”[1] One way in which this can occur is individuals trying to anticipate the effect of a policy and then taking actions that alter its outcome
  • Dissertation
  • Continuing TF 2.0 Keras tutorial
    • Had a weird problem where
      from tensorflow import keras

      made IntelliJ complain, but the python interpreter ran fine. I then installed keras, and IJ stopped complaining. Checking the version(s) seems to be identical, even though I can see that there is a new keras directory in D:\Program Files\Python37\Lib\site-packages. And we know that the interpreter and IDE are pointing to the same place:

      "D:\Program Files\Python37\python.exe" D:/Development/Sandboxes/PyBullet/src/TensorFlow/HelloKeras.py
      2019-09-05 11:30:04.694327: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
      tf.keras version = 2.2.4-tf
      keras version = 2.2.4-tf

Keras

    • This has the implication that instead of :
      from tensorflow.keras import layers

      I need to have:

      from keras import layers

      I mean, it works, but it’s weird and makes me think that something subtle may be busted…

Phil 9.4.19

7:00 – 5:00 ASRC GOES

Surrogation

Phil 9.3.19 (including install directions for Tensorflow 2.0rc1 on Windows 10)

7:00 – 4:30ASRC GOES

  • Dissertation – Working on the Orientation section, where I compare Moby Dick to Dieselgate
  • Uninstalling all previous versions of CUDA, which should hopefully allow 10 to be installed
  • Still flailing on getting TF 2.0 working. Grrrrr. Success! Added guide below
  • Spent some time discussing mapping the GPT-2 with Aaron

Installing Tensorflow 2.0rc1 to Windows 10, a temporary accurate guide

  • Uninstall any previous version of Tensorflow (e.g. “pip uninstall tensorflow”)
  • Uninstall all your NVIDIA crap
  • Install JUST THE CUDA LIBRARIES for version 9.0 and 10.0. You don’t need anything else

NVIDIA1

NVIDIA2

  • Then install the latest Nvidia graphics drivers. When you’re done, your install should look something like this (this worked on 9.3.19):

NVIDIA3

Edit your system variables so that the CUDA 9 and CUDA 10 directories are on your path:

NVIDIA4

One more part is needed from NVIDIA: cudnn64_7.dll

In order to download cuDNN, ensure you are registered for the NVIDIA Developer Program.

    1. Go to: NVIDIA cuDNN home page
    2. Click “Download”.
  1. Remember to accept the Terms and Conditions.
  2. Select the cuDNN version to want to install from the list. This opens up a second list of target OS installs. Select cuDNN Library for Windows 10.
  3. Extract the cuDNN archive to a directory of your choice. The important part (cudnn64_7.dll) is in the cuda\bin directory. Either add that directory to your path, or copy the dll and put it in the Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10\bin directory

NVIDIA6

Then open up a console window (cmd) as admin, and install tensorflow:

  • pip install tensorflow-gpu==2.0.0-rc1
  • verify that it works by opening the python console and typing the following:

NVIDIA5

if that works, you should be able to have the following work:

import tensorflow as tf
print("tf version = {}".format(tf.__version__))
mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)

model.evaluate(x_test, y_test)

The results should looks something like:

"D:\Program Files\Python37\python.exe" D:/Development/Sandboxes/PyBullet/src/TensorFlow/HelloWorld.py
2019-09-03 15:09:56.685476: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
tf version = 2.0.0-rc0
2019-09-03 15:09:59.272748: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library nvcuda.dll
2019-09-03 15:09:59.372341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: TITAN X (Pascal) major: 6 minor: 1 memoryClockRate(GHz): 1.531
pciBusID: 0000:01:00.0
2019-09-03 15:09:59.372616: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2019-09-03 15:09:59.373339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-09-03 15:09:59.373671: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-09-03 15:09:59.376010: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: TITAN X (Pascal) major: 6 minor: 1 memoryClockRate(GHz): 1.531
pciBusID: 0000:01:00.0
2019-09-03 15:09:59.376291: I tensorflow/stream_executor/platform/default/dlopen_checker_stub.cc:25] GPU libraries are statically linked, skip dlopen check.
2019-09-03 15:09:59.376996: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-09-03 15:09:59.951116: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-09-03 15:09:59.951317: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2019-09-03 15:09:59.951433: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2019-09-03 15:09:59.952189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 9607 MB memory) -> physical GPU (device: 0, name: TITAN X (Pascal), pci bus id: 0000:01:00.0, compute capability: 6.1)
Train on 60000 samples
Epoch 1/5
2019-09-03 15:10:00.818650: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cublas64_100.dll

   32/60000 [..............................] - ETA: 17:07 - loss: 2.4198 - accuracy: 0.0938
  736/60000 [..............................] - ETA: 48s - loss: 1.7535 - accuracy: 0.4891  
 1696/60000 [..............................] - ETA: 22s - loss: 1.2584 - accuracy: 0.6515
 2560/60000 [>.............................] - ETA: 16s - loss: 1.0503 - accuracy: 0.7145
 3552/60000 [>.............................] - ETA: 12s - loss: 0.9017 - accuracy: 0.7531
 4352/60000 [=>............................] - ETA: 10s - loss: 0.8156 - accuracy: 0.7744
 5344/60000 [=>............................] - ETA: 9s - loss: 0.7407 - accuracy: 0.7962 
 6176/60000 [==>...........................] - ETA: 8s - loss: 0.7069 - accuracy: 0.8039
 7040/60000 [==>...........................] - ETA: 7s - loss: 0.6669 - accuracy: 0.8134
 8032/60000 [===>..........................] - ETA: 6s - loss: 0.6285 - accuracy: 0.8236
 8832/60000 [===>..........................] - ETA: 6s - loss: 0.6037 - accuracy: 0.8291
 9792/60000 [===>..........................] - ETA: 6s - loss: 0.5823 - accuracy: 0.8356
10656/60000 [====>.........................] - ETA: 5s - loss: 0.5621 - accuracy: 0.8410
11680/60000 [====>.........................] - ETA: 5s - loss: 0.5434 - accuracy: 0.8453
12512/60000 [=====>........................] - ETA: 5s - loss: 0.5311 - accuracy: 0.8485
13376/60000 [=====>........................] - ETA: 4s - loss: 0.5144 - accuracy: 0.8534
14496/60000 [======>.......................] - ETA: 4s - loss: 0.4997 - accuracy: 0.8580
15296/60000 [======>.......................] - ETA: 4s - loss: 0.4894 - accuracy: 0.8609
16224/60000 [=======>......................] - ETA: 4s - loss: 0.4792 - accuracy: 0.8634
17120/60000 [=======>......................] - ETA: 4s - loss: 0.4696 - accuracy: 0.8664
17888/60000 [=======>......................] - ETA: 3s - loss: 0.4595 - accuracy: 0.8690
18752/60000 [========>.....................] - ETA: 3s - loss: 0.4522 - accuracy: 0.8711
19840/60000 [========>.....................] - ETA: 3s - loss: 0.4434 - accuracy: 0.8738
20800/60000 [=========>....................] - ETA: 3s - loss: 0.4356 - accuracy: 0.8756
21792/60000 [=========>....................] - ETA: 3s - loss: 0.4293 - accuracy: 0.8776
22752/60000 [==========>...................] - ETA: 3s - loss: 0.4226 - accuracy: 0.8794
23712/60000 [==========>...................] - ETA: 3s - loss: 0.4179 - accuracy: 0.8808
24800/60000 [===========>..................] - ETA: 2s - loss: 0.4111 - accuracy: 0.8827
26080/60000 [============>.................] - ETA: 2s - loss: 0.4029 - accuracy: 0.8849
27264/60000 [============>.................] - ETA: 2s - loss: 0.3981 - accuracy: 0.8864
28160/60000 [=============>................] - ETA: 2s - loss: 0.3921 - accuracy: 0.8882
29408/60000 [=============>................] - ETA: 2s - loss: 0.3852 - accuracy: 0.8902
30432/60000 [==============>...............] - ETA: 2s - loss: 0.3809 - accuracy: 0.8916
31456/60000 [==============>...............] - ETA: 2s - loss: 0.3751 - accuracy: 0.8932
32704/60000 [===============>..............] - ETA: 2s - loss: 0.3707 - accuracy: 0.8946
33760/60000 [===============>..............] - ETA: 1s - loss: 0.3652 - accuracy: 0.8959
34976/60000 [================>.............] - ETA: 1s - loss: 0.3594 - accuracy: 0.8975
35968/60000 [================>.............] - ETA: 1s - loss: 0.3555 - accuracy: 0.8984
37152/60000 [=================>............] - ETA: 1s - loss: 0.3509 - accuracy: 0.8998
38240/60000 [==================>...........] - ETA: 1s - loss: 0.3477 - accuracy: 0.9006
39232/60000 [==================>...........] - ETA: 1s - loss: 0.3442 - accuracy: 0.9015
40448/60000 [===================>..........] - ETA: 1s - loss: 0.3393 - accuracy: 0.9030
41536/60000 [===================>..........] - ETA: 1s - loss: 0.3348 - accuracy: 0.9042
42752/60000 [====================>.........] - ETA: 1s - loss: 0.3317 - accuracy: 0.9049
43840/60000 [====================>.........] - ETA: 1s - loss: 0.3288 - accuracy: 0.9059
44992/60000 [=====================>........] - ETA: 1s - loss: 0.3255 - accuracy: 0.9069
46016/60000 [======================>.......] - ETA: 0s - loss: 0.3230 - accuracy: 0.9077
47104/60000 [======================>.......] - ETA: 0s - loss: 0.3203 - accuracy: 0.9085
48288/60000 [=======================>......] - ETA: 0s - loss: 0.3174 - accuracy: 0.9091
49248/60000 [=======================>......] - ETA: 0s - loss: 0.3155 - accuracy: 0.9098
50208/60000 [========================>.....] - ETA: 0s - loss: 0.3131 - accuracy: 0.9105
51104/60000 [========================>.....] - ETA: 0s - loss: 0.3111 - accuracy: 0.9111
52288/60000 [=========================>....] - ETA: 0s - loss: 0.3085 - accuracy: 0.9117
53216/60000 [=========================>....] - ETA: 0s - loss: 0.3066 - accuracy: 0.9121
54176/60000 [==========================>...] - ETA: 0s - loss: 0.3043 - accuracy: 0.9128
55328/60000 [==========================>...] - ETA: 0s - loss: 0.3018 - accuracy: 0.9135
56320/60000 [===========================>..] - ETA: 0s - loss: 0.2995 - accuracy: 0.9141
57440/60000 [===========================>..] - ETA: 0s - loss: 0.2980 - accuracy: 0.9143
58400/60000 [============================>.] - ETA: 0s - loss: 0.2961 - accuracy: 0.9148
59552/60000 [============================>.] - ETA: 0s - loss: 0.2941 - accuracy: 0.9154
60000/60000 [==============================] - 4s 65us/sample - loss: 0.2930 - accuracy: 0.9158
... epochs pass ...
10000/1 [==========] - 1s 61us/sample - loss: 0.0394 - accuracy: 0.9778

Phil 8.23.19

7:00 – 4:00 ASRC GEOS

  • More Dissertation
    • Continuing lit review
  • Rework BlueSky paper for air traffic? Meeting with T at 10:00
  • Simulation
    • Need to discuss with Aaron the best way to use the data to train the NN and round-trip the outputs so that they can be used to have the ML model issue commands to the RCS system so that given the outputs of one model, the NN can create commands that cause the same outputs in a separate model
  • Wow. It knows/finds syntactically correct Java. From TalkToTransformer.com:
  • Wow

Phil 5.2.19

7:00 – 9:00 ASRC NASA

  • Wrote up my notes from yesterday
  • Need to make an Akido Drone image, maybe even a sim in Zach’s environment?
  • Changed the title of the Dissertation
  • Need to commit the changes to LMN from the laptop – done
  • Need to create an instance of the JASSS paper in overleaf and make sure it runs
  • Put the jasss.bst file in the svn repo – done
  • Thinking about putting my dict find on stackoverflow, but did see this page on xpath for dict that is making me wonder if I just shouldn’t point there.
  • Did meaningless 2019 goal stuff
  • Adding ragged edge argument and generate a set of curves for eval
  • ML seminar 4:30
  • Meeting with Aaron M at 7:00
    • Spent a good deal of time discussing the structure of the paper and the arguments. Aaron wants the point made that the “arc to full autonomy” is really only the beginning, predictable part of the process. In this part, the humans own the “reflective part” of the process, either as a human in the loop, where they decide to pull the trigger, or in the full autonomy mode where they select the training data and evaluation criteria for the reflexive system that’s built. The next part of that sequence is when machines begin to develop reflective capabilities. When that happens, many of the common assumptions that sets of human adversaries make about conflict (OODA, for example), may well be disrupted by systems that do not share the common background and culture, but have been directed to perform the same mission.

Phil 5.1.19

7:00 – 7:00 ASRC NASA AIMS

  • Added lit review section to the dissertation, and put the seven steps of sectarianism in.
  • Spent most of yesterday helping Aaron with TimeSeriesML. Currently working on a JSON util that will get a value on a provided path
  • Had to set up python at the module and not project level, which was odd. Here’s how: www.jetbrains.com/help/idea/2016.1/configuring-global-project-and-module-sdks.html#module_sdk
  • Done!
        def lfind(self, query_list:List, target_list:List, targ_str:str = "???"):
            for tval in target_list:
                if isinstance(tval, dict):
                    return self.dfind(query_list[0], tval, targ_str)
                elif tval == query_list[0]:
                    return tval
    
        def dfind(self, query_dict:Dict, target_dict:Dict, targ_str:str = "???"):
            for key, qval in query_dict.items():
                # print("key = {}, qval = {}".format(key, qval))
                tval = target_dict[key]
                if isinstance(qval, dict):
                    return self.dfind(qval, tval, targ_str)
                elif isinstance(qval, list):
                    return self.lfind(qval, tval, targ_str)
                else:
                    if qval == targ_str:
                        return tval
                    if qval != tval:
                        return None
    
        def find(self, query_dict:Dict):
            # pprint.pprint(query_dict)
            result = self.dfind(query_dict, self.json_dict)
            return result
    
    
  • It’s called like this:
    ju = JsonUtils("../../data/output_data/lstm_structure.json")
    # ju.pprint()
    result = ju.find({"config":[{"class_name":"Masking", "config":{"batch_input_shape": "???"}}]})
    print("result 1 = {}".format(result))
    result = ju.find({"config":[{"class_name":"Masking", "config":{"mask_value": "???"}}]})
    print("result 2 = {}".format(result))
  • Here’s the results:
    result 1 = [None, 12, 1]
    result 2 = 666.0
  • Got Aaron’s code running!
  • Meeting with Joel
    • A quicker demo that I was expecting, though I was able to walk through how to create and use Corpus Manager and LMN. Also, we got a bug where the column index for the eigenvector didn’t exist. Fixed that in JavaUtils.math.Labeled2DMatrix.java
  • Meeting with Wayne
    • Walked through the JASSS paper. Need to make sure that the lit review is connected and in the proper order
    • Changed the title of the dissertation to
      • Stampede Theory: Mapping Dangerous Misinformation at Scale
    • Solidifying defense over the winter break, with diploma in the Spring
    • Mentioned the “aikido with drones” concept. Need to make an image. Actually, I wonder if there is a way for that model to be used for actually getting a grant to explore weaponized AI in a way that isn’t directly mappable to weapons systems, but is close enough to reality that people will get the point.
    • Also discussed the concept of managing runaway AI with the Sanhedrin-17a concept, where unanimous agreement to convict means acquittal.  Cities had Sanhedrin of 23 Judges and the Great Sanhedrin had 71 Judges en.wikipedia.org/wiki/Sanhedrin
      • Rav Kahana says: In a Sanhedrin where all the judges saw fit to convict the defendant in a case of capital law, they acquit him. The Gemara asks: What is the reasoning for this halakha? It is since it is learned as a tradition that suspension of the trial overnight is necessary in order to create a possibility of acquittal. The halakha is that they may not issue the guilty verdict on the same day the evidence was heard, as perhaps over the course of the night one of the judges will think of a reason to acquit the defendant. And as those judges all saw fit to convict him they will not see any further possibility to acquit him, because there will not be anyone arguing for such a verdict. Consequently, he cannot be convicted.

 

Phil 4.19.19

8:00 – 4:00 ASRC TL

  • Updating working copies of the paper based on the discussion with Aaron M last night.
  • Based on the diagrams of the weights that I could make with the MNIST model, I think I want to try to make a layer neuron/weight visualizer. This one is very pretty
  • Need to start on framework for data generation and analysis with Zach this morning
  • Got Flask working (see above for rant on how).
  • Flask-RESTful provides an extension to Flask for building REST APIs. Flask-RESTful was initially developed as an internal project at Twilio, built to power their public and internal APIs.