Category Archives: Dissertation

Phil 6.10.19

ASRC GEOS 7:00 – 3:00

  • I’ve been thinking about the implications of this article: Training a single AI model can emit as much carbon as five cars in their lifetimes
    • There is something in this that has to do with the idea of cost. NN architectures have no direct concept of cost. Inevitably the “current best network” takes a building full of specialized processors 200 hours. This has been true for Inception, AmeoebaNet, and AlphaGo. I wonder what would happen if there was a cost for computation that was part of the fitness function?
    • My sense is that evolution, has two interrelated parameters
      • a mutation needs to “work better” (whatever that means in the context) than the current version
      • the organism that embodies the mutation has to reproduce
    • In other words, neural structures in our brains have an unbroken chain of history to the initial sensor neurons in multicellular organisms. All the mutations that didn’t live to make an effect. Those that weren’t able to reproduce didn’t get passed on.
    • Randomness is important too. Systems that are too similar, like Aspen trees that have given up on sexual reproduction and are essentially all clones reproducing by rhizome. These live long enough to have an impact on the environment, particularly where they can crowd out other species, but the species itself is doomed.
    • I’d like to see an approach to developing NNs that involves more of the constraints of “natural” evolution. I think it would lead to better, and potentially less destructive results.
  • SHAP (SHapley Additive exPlanations) is a unified approach to explain the output of any machine learning model. SHAP connects game theory with local explanations, uniting several previous methods [1-7] and representing the only possible consistent and locally accurate additive feature attribution method based on expectations (see our papers for details).
  • Working on clustering. I’ve been going around in circles on how to take a set of relative distance measures and use them as a basis for clustering. To revisit, here’s a screenshot of a spreadsheet containing the DTW distances from every sequence to every other sequence: DTW
  • My approach is to treat each line of relative distances as a high-dimensional coordinate ( in this case, 50 dimensions), and cluster with respect to the point that defines. This takes care of the problem that the data in this case is very symmetric about the diagonal. Using this approach, an orange/green coordinate is in a different location from the mirrored green/orange coordinate. It’s basically the difference between (1, 2) and (2, 1). That should be a reliable clustering mechanism. Here are the results:
           cluster_id
    ts_0            0
    ts_1            0
    ts_2            0
    ts_3            0
    ts_4            0
    ts_5            0
    ts_6            0
    ts_7            0
    ts_8            0
    ts_9            0
    ts_10           0
    ts_11           0
    ts_12           0
    ts_13           0
    ts_14           0
    ts_15           0
    ts_16           0
    ts_17           0
    ts_18           0
    ts_19           0
    ts_20           0
    ts_21           0
    ts_22           0
    ts_23           0
    ts_24           0
    ts_25           1
    ts_26           1
    ts_27           1
    ts_28           1
    ts_29           1
    ts_30           1
    ts_31           1
    ts_32           1
    ts_33           1
    ts_34           1
    ts_35           1
    ts_36           1
    ts_37           1
    ts_38           1
    ts_39           1
    ts_40           1
    ts_41           1
    ts_42           1
    ts_43           1
    ts_44           1
    ts_45           1
    ts_46           1
    ts_47           1
    ts_48           1
    ts_49           1
  • First-Order Adversarial Vulnerability of Neural Networks and Input Dimension
    • Carl-Johann Simon-Gabriel, Yann Ollivier, Bernhard Scholkopf, Leon BottouDavid Lopez-Paz
    • Over the past few years, neural networks were proven vulnerable to adversarial images: Targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisingly, vulnerability does not depend on network topology: For many standard network architectures, we prove that at initialization, the l1-norm of these gradients grows as the square root of the input dimension, leaving the networks increasingly vulnerable with growing image size. We empirically show that this dimension-dependence persists after either usual or robust training, but gets attenuated with higher regularization.
  • More JASSS paper. Through the corrections up to the Results section. Kind of surprised to be leaning so hard on Homer, but I need a familiar story from before world maps.
  • Oh yeah, the Age Of discovery correlates with the development of the Mercator projection and usable world maps

Phil 6.7.19

7:00 – 4:30ASRC GEOS

  • Expense report
  • learned how to handle overtime
  • Dissertation. At 68 pages into the Very Horrible First Draft (VHFD)
  • Meeting with Wayne. Walked though JASSS paper and CHIPLAY reviews
  • Set arguments to DTW systems so that a specified number of rows can be evaluated to support parallelization – done: Split
  • Start clustering? Mope. Wrote up report instead

Phil 5.14.19

7:00 – 8:00 ASRC NASA GEOS-R

  • More Dissertation
  • Break out the network slides to “island” (initial state), “star” (radio) “cyclic star” (talk radio), “dense” social media
  • MatrixScalar
  • 7:30 Waikato meeting.
    • Walked through today’s version, which is looking very nice
    • Went over tasking spreadsheets

Phil 5.13.19

7:00 – 3:00 ASRC NASA GEOS-R

Phil 5.10.19

7:00 – 4:00 ASRC NASA GOES

  • Tensorflow Graphics? TF-Graphics
  • An End-to-End AutoML Solution for Tabular Data at KaggleDays
  • More dissertation writing. Added a bit on The Sorcerer’s Apprentice and finished my first pass at Moby-Dick
  • Add pickling to MatrixScalar – done!
    def save_class(the_class, filename:str):
        print("save_class")
        # Its important to use binary mode
        dbfile = open(filename, 'ab')
    
        # source, destination
        pickle.dump(the_class, dbfile)
        dbfile.close()
    
    
    def restore_class(filename:str) -> MatrixScalar:
        print("restore_class")
        # for reading also binary mode is important
        dbfile = open(filename, 'rb')
        db = pickle.load(dbfile)
        dbfile.close()
        return db
  • Added flag to allow unlimited input buffer cols. It automatically sizes to the max if no arg for input_size
  • NOTE: Add a “notes” dict that is added to the setup tab for run information

 

Phil 5.9.19

Finished Army of None. One of the deepest, thorough analysis of human-centered AI/ML I’ve ever read.

7:00 – 4:00 ASRC NASA GOES-R

  • Create spreadsheets for tasks and bugs
  • More dissertation. Add Axelrod
  • Add reading and saving of matrices
    • Well, I can write everything, but xlsxwriter won’t read in anything
    • Tomorrow add pickling
  • Price to win analytic?

4:30 – 7:00 ML Seminar

7:00 – 9:00 Meeting with Aaron M

  • Tried to get biber working, but it produces a blank bib file. Need to look into that
  • Got the AI paper uploaded to Aaron’s new account. Arxiv also has problems with biber
  • Spent the rest of the meeting figuring out the next steps. It’s potentially something along the lines of using ML to build an explainable model for different sorts of ML systems (e.g. Humans-on-the-loop <-> Forensic, post-hoc interaction)

Phil 5.7.19

7:00 – 8:00 ASRC NASA GOES-R

  • Via CSAIL: “The team’s approach isn’t particularly efficient now – they must train and “prune” the full network several times before finding the successful subnetwork. However, MIT professor Michael Carbin says that his team’s findings suggest that, if we can determine precisely which part of the original network is relevant to the final prediction, scientists might one day be able to skip this expensive process altogether. Such a revelation has the potential to save hours of work and make it easier for meaningful models to be created by individual programmers and not just huge tech companies.”
    • From the abstract of The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
      : We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the “lottery ticket hypothesis:” dense, randomly-initialized, feed-forward networks contain subnetworks (“winning tickets”) that – when trained in isolation – reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective. 
    • Sounds like a good opportunity for evolutionary systems
  • Finished with text mods for IEEE letter
  • Added Kaufman and Olfati-Sabir to the discussion on Social Influence Horizon
  • Started the draft deck for the tech summit
  • More MatrixScalar
    • Core functions work
    • Change test and train within the class to input and target
    • Create a coordinating class that loads and creates test and train matrices
  • JuryRoom meeting
    • Progress is good enough to start tracking it. Going to create a set of Google sheets that keep track of tasks and bugs

Phil 5.6.19

7:00 – 5:00 ASRC GOES-R

  • Finished the AI/ML paper with Aaron M over the weekend. I need to have him ping me when it goes in. I think it turned out pretty well, even when cut down to 7 pages (with references!! Why, IEEE, why?)
    • Sent a copy to Wayne, and distributed around work. Need to put in on ArXiv on Thursday
  • Starting to pull parts from phifel.com to make the lit review for the dissertation. Those reviews may have had a reason after all!
    • And oddly (though satisfying), I wound up adding a section on Moby-Dick as a way of setting up the rest of the lit review
  • More Matrix scalar class. Basically a satisfying day of just writing code.
  • Need to fix IEEE letter and take a self-portrait. Need to charge up the good camera

Phil 5.2.19

7:00 – 9:00 ASRC NASA

  • Wrote up my notes from yesterday
  • Need to make an Akido Drone image, maybe even a sim in Zach’s environment?
  • Changed the title of the Dissertation
  • Need to commit the changes to LMN from the laptop – done
  • Need to create an instance of the JASSS paper in overleaf and make sure it runs
  • Put the jasss.bst file in the svn repo – done
  • Thinking about putting my dict find on stackoverflow, but did see this page on xpath for dict that is making me wonder if I just shouldn’t point there.
  • Did meaningless 2019 goal stuff
  • Adding ragged edge argument and generate a set of curves for eval
  • ML seminar 4:30
  • Meeting with Aaron M at 7:00
    • Spent a good deal of time discussing the structure of the paper and the arguments. Aaron wants the point made that the “arc to full autonomy” is really only the beginning, predictable part of the process. In this part, the humans own the “reflective part” of the process, either as a human in the loop, where they decide to pull the trigger, or in the full autonomy mode where they select the training data and evaluation criteria for the reflexive system that’s built. The next part of that sequence is when machines begin to develop reflective capabilities. When that happens, many of the common assumptions that sets of human adversaries make about conflict (OODA, for example), may well be disrupted by systems that do not share the common background and culture, but have been directed to perform the same mission.

Phil 5.1.19

7:00 – 7:00 ASRC NASA AIMS

  • Added lit review section to the dissertation, and put the seven steps of sectarianism in.
  • Spent most of yesterday helping Aaron with TimeSeriesML. Currently working on a JSON util that will get a value on a provided path
  • Had to set up python at the module and not project level, which was odd. Here’s how: www.jetbrains.com/help/idea/2016.1/configuring-global-project-and-module-sdks.html#module_sdk
  • Done!
        def lfind(self, query_list:List, target_list:List, targ_str:str = "???"):
            for tval in target_list:
                if isinstance(tval, dict):
                    return self.dfind(query_list[0], tval, targ_str)
                elif tval == query_list[0]:
                    return tval
    
        def dfind(self, query_dict:Dict, target_dict:Dict, targ_str:str = "???"):
            for key, qval in query_dict.items():
                # print("key = {}, qval = {}".format(key, qval))
                tval = target_dict[key]
                if isinstance(qval, dict):
                    return self.dfind(qval, tval, targ_str)
                elif isinstance(qval, list):
                    return self.lfind(qval, tval, targ_str)
                else:
                    if qval == targ_str:
                        return tval
                    if qval != tval:
                        return None
    
        def find(self, query_dict:Dict):
            # pprint.pprint(query_dict)
            result = self.dfind(query_dict, self.json_dict)
            return result
    
    
  • It’s called like this:
    ju = JsonUtils("../../data/output_data/lstm_structure.json")
    # ju.pprint()
    result = ju.find({"config":[{"class_name":"Masking", "config":{"batch_input_shape": "???"}}]})
    print("result 1 = {}".format(result))
    result = ju.find({"config":[{"class_name":"Masking", "config":{"mask_value": "???"}}]})
    print("result 2 = {}".format(result))
  • Here’s the results:
    result 1 = [None, 12, 1]
    result 2 = 666.0
  • Got Aaron’s code running!
  • Meeting with Joel
    • A quicker demo that I was expecting, though I was able to walk through how to create and use Corpus Manager and LMN. Also, we got a bug where the column index for the eigenvector didn’t exist. Fixed that in JavaUtils.math.Labeled2DMatrix.java
  • Meeting with Wayne
    • Walked through the JASSS paper. Need to make sure that the lit review is connected and in the proper order
    • Changed the title of the dissertation to
      • Stampede Theory: Mapping Dangerous Misinformation at Scale
    • Solidifying defense over the winter break, with diploma in the Spring
    • Mentioned the “aikido with drones” concept. Need to make an image. Actually, I wonder if there is a way for that model to be used for actually getting a grant to explore weaponized AI in a way that isn’t directly mappable to weapons systems, but is close enough to reality that people will get the point.
    • Also discussed the concept of managing runaway AI with the Sanhedrin-17a concept, where unanimous agreement to convict means acquittal.  Cities had Sanhedrin of 23 Judges and the Great Sanhedrin had 71 Judges en.wikipedia.org/wiki/Sanhedrin
      • Rav Kahana says: In a Sanhedrin where all the judges saw fit to convict the defendant in a case of capital law, they acquit him. The Gemara asks: What is the reasoning for this halakha? It is since it is learned as a tradition that suspension of the trial overnight is necessary in order to create a possibility of acquittal. The halakha is that they may not issue the guilty verdict on the same day the evidence was heard, as perhaps over the course of the night one of the judges will think of a reason to acquit the defendant. And as those judges all saw fit to convict him they will not see any further possibility to acquit him, because there will not be anyone arguing for such a verdict. Consequently, he cannot be convicted.

 

Phil 11.26.18

7:00 – 5:00ASRC PhD

  • Had a thought that simulation plus diversity might be an effective way of increasing system resilience. This is based on the discussion of Apollo 13 in Normal Accidents
  • Start folding in content from simulation papers. Don’t worry about coherence yet
  • Start figuring out PHPbb
    • Working on the IRB form – done
    • Set user creation to admin-approved – done
    • Create easily identifiable players
      • Asra Rogueplayer
      • Ping Clericplayer
      • Valen Fighterplayer
      • Emmi MonkPlayer
      • Avia Bardplayer
      • Mirek Thiefplayer
      • Lino Magicplayer
      • Daz Dmplayer
    • Some notes on play by post
    • Added Aaron as a founder. He’s set up the overall structure: dungeon
    • Add easily identifiable content. Working. Set up the AntibubblesDungeon as a python project. I’m going to write a script generator that we will then use to paste in content. Then back up and download the database and run queries on it locally.