Category Archives: Python

Phil 5.14.19

7:00 – 8:00 ASRC NASA GEOS-R

  • More Dissertation
  • Break out the network slides to “island” (initial state), “star” (radio) “cyclic star” (talk radio), “dense” social media
  • MatrixScalar
  • 7:30 Waikato meeting.
    • Walked through today’s version, which is looking very nice
    • Went over tasking spreadsheets

Phil 5.13.19

7:00 – 3:00 ASRC NASA GEOS-R

Phil 5.10.19

7:00 – 4:00 ASRC NASA GOES

  • Tensorflow Graphics? TF-Graphics
  • An End-to-End AutoML Solution for Tabular Data at KaggleDays
  • More dissertation writing. Added a bit on The Sorcerer’s Apprentice and finished my first pass at Moby-Dick
  • Add pickling to MatrixScalar – done!
    def save_class(the_class, filename:str):
        print("save_class")
        # Its important to use binary mode
        dbfile = open(filename, 'ab')
    
        # source, destination
        pickle.dump(the_class, dbfile)
        dbfile.close()
    
    
    def restore_class(filename:str) -> MatrixScalar:
        print("restore_class")
        # for reading also binary mode is important
        dbfile = open(filename, 'rb')
        db = pickle.load(dbfile)
        dbfile.close()
        return db
  • Added flag to allow unlimited input buffer cols. It automatically sizes to the max if no arg for input_size
  • NOTE: Add a “notes” dict that is added to the setup tab for run information

 

Phil 5.9.19

Finished Army of None. One of the deepest, thorough analysis of human-centered AI/ML I’ve ever read.

7:00 – 4:00 ASRC NASA GOES-R

  • Create spreadsheets for tasks and bugs
  • More dissertation. Add Axelrod
  • Add reading and saving of matrices
    • Well, I can write everything, but xlsxwriter won’t read in anything
    • Tomorrow add pickling
  • Price to win analytic?

4:30 – 7:00 ML Seminar

7:00 – 9:00 Meeting with Aaron M

  • Tried to get biber working, but it produces a blank bib file. Need to look into that
  • Got the AI paper uploaded to Aaron’s new account. Arxiv also has problems with biber
  • Spent the rest of the meeting figuring out the next steps. It’s potentially something along the lines of using ML to build an explainable model for different sorts of ML systems (e.g. Humans-on-the-loop <-> Forensic, post-hoc interaction)

Phil 5.8.19

7:00 – 5:00 ASRC NASA GOES-R

  • Create spreadsheets for tasks and bugs
  • More dissertation. Add Axelrod
  • Mission Drive today, no meeting with Wayne, I think
  • Good visualization tool finds this morning:
    • Altair: Declarative Visualization in Python
    • deck.gl is a WebGL-powered framework for visual exploratory data analysis of large datasets.
  • Matrix Class
    • Change test and train within the class to input and target
    • Create the test and train as an output with the rectangular matrix with masks included. This means that I have to re-assemble that matrix from the input and target matrices
    • I still like the idea of persisting these values as excel spreadsheets
    • And now a pile of numbers that makes me happy:
      MatrixScalar:
      	rows = 5
      	input_size = 5
      	target_size = 5
      	mask_value(hex) = -1
      	tmax_cols = 6
      	mat_min = 0.13042279608514273
      	mat_max = 9.566827711787509
      
      input_npmat = 
      [4.384998306058251, 6.006494724381491, 7.061283542583833, 7.817876758859971, 7.214499436254831]
      [0.15061642402352393, 2.818956354589415, 5.04113793598655, 6.31250083574919]
      [2.8702355283795837, 5.564035171373476, 7.81403258383623, 8.590265450278785, 9.566827711787509]
      [0.1359688602006689, 0.8005043254115471, 2.080391037187722, 1.9828746089685887, 2.4669996344853677]
      [0.33676501126574077]
      
      target_npmat = 
      [6.529725859535821, 4.8702784287160075, 3.677355933557321, 1.5184287945320327, -0.5429800453619322]
      [7.629655798004273, 8.043579124885415, 7.261429015491849, 7.137935661381686, 5.583232751491164]
      [8.997538924797388, 8.32502866049641, 6.5215023090524085, 4.725363596736856, 1.3761131232325439]
      [2.270623038824647, 2.430147101210101, 2.0903103552937132, 1.6846416494136842, 1.4289540998497225]
      [1.897999998722116, 1.9054555934093833, 2.883358420829866, 3.703791108487346, 4.011103843736698]
      
      scaled_input_npmat = 
      [0.5608937619909073, 0.7683025595887466, 0.9032226729055693, 0.9999999999999999, 0.9228208193584869]
      [0.023860024409113324, 0.44656728417761093, 0.798596002940291, 1.0]
      [0.30001956916639155, 0.5815966733171017, 0.8167840813322457, 0.8979220394754807, 1.0]
      [0.055115071076624986, 0.32448497933342324, 0.8432879389630332, 0.8037595876588889, 1.0]
      [1.0]
      
      scaled_target_npmat = 
      [0.8352300836842569, 0.6229668973991612, 0.47037783364770835, 0.1942252150254483, -0.06945364606145459]
      [1.2086581842168989, 1.2742301877146247, 1.1503252362944103, 1.1307619352630993, 0.88447239798718]
      [0.9404934630223677, 0.8701974062142825, 0.6816786614665502, 0.4939321308059752, 0.143842155904721]
      [0.9203986117729256, 0.9850618002693972, 0.847308741385268, 0.6828706522143898, 0.5792275279958895]
      [5.635977418165948, 5.658116281877606, 8.561929904750741, 10.998147030080538, 11.910690569251393]
      
      scaled, masked input = 
      [[ 0.56089376  0.76830256  0.90322267  1.          0.92282082]
       [-1.          0.02386002  0.44656728  0.798596    1.        ]
       [ 0.30001957  0.58159667  0.81678408  0.89792204  1.        ]
       [ 0.05511507  0.32448498  0.84328794  0.80375959  1.        ]
       [-1.         -1.         -1.         -1.          1.        ]]
      scaled target = 
      [0.8352300836842569, 0.6229668973991612, 0.47037783364770835, 0.1942252150254483, -0.06945364606145459]
      [1.2086581842168989, 1.2742301877146247, 1.1503252362944103, 1.1307619352630993, 0.88447239798718]
      [0.9404934630223677, 0.8701974062142825, 0.6816786614665502, 0.4939321308059752, 0.143842155904721]
      [0.9203986117729256, 0.9850618002693972, 0.847308741385268, 0.6828706522143898, 0.5792275279958895]
      [5.635977418165948, 5.658116281877606, 8.561929904750741, 10.998147030080538, 11.910690569251393]
      
      scaled = [ 6.52972586  4.87027843  3.67735593  1.51842879 -0.54298005], error = 0.0
      scaled = [7.6296558  8.04357912 7.26142902 7.13793566 5.58323275], error = 0.0
      scaled = [8.99753892 8.32502866 6.52150231 4.7253636  1.37611312], error = 0.0
      scaled = [2.27062304 2.4301471  2.09031036 1.68464165 1.4289541 ], error = 0.0
      scaled = [1.898      1.90545559 2.88335842 3.70379111 4.01110384], error = 0.0
      
      input_train = 
      [[ 0.05511507  0.32448498  0.84328794  0.80375959  1.        ]
       [-1.         -1.         -1.         -1.          1.        ]
       [ 0.30001957  0.58159667  0.81678408  0.89792204  1.        ]]
      
      input_test = 
      [[ 0.56089376  0.76830256  0.90322267  1.          0.92282082]
       [-1.          0.02386002  0.44656728  0.798596    1.        ]]
      
      target_train = 
      [[ 0.92039861  0.9850618   0.84730874  0.68287065  0.57922753]
       [ 5.63597742  5.65811628  8.5619299  10.99814703 11.91069057]
       [ 0.94049346  0.87019741  0.68167866  0.49393213  0.14384216]]
      
      target_test = 
      [[ 0.83523008  0.6229669   0.47037783  0.19422522 -0.06945365]
       [ 1.20865818  1.27423019  1.15032524  1.13076194  0.8844724 ]]

       

Phil 5.7.19

7:00 – 8:00 ASRC NASA GOES-R

  • Via CSAIL: “The team’s approach isn’t particularly efficient now – they must train and “prune” the full network several times before finding the successful subnetwork. However, MIT professor Michael Carbin says that his team’s findings suggest that, if we can determine precisely which part of the original network is relevant to the final prediction, scientists might one day be able to skip this expensive process altogether. Such a revelation has the potential to save hours of work and make it easier for meaningful models to be created by individual programmers and not just huge tech companies.”
    • From the abstract of The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
      : We find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively. Based on these results, we articulate the “lottery ticket hypothesis:” dense, randomly-initialized, feed-forward networks contain subnetworks (“winning tickets”) that – when trained in isolation – reach test accuracy comparable to the original network in a similar number of iterations. The winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective. 
    • Sounds like a good opportunity for evolutionary systems
  • Finished with text mods for IEEE letter
  • Added Kaufman and Olfati-Sabir to the discussion on Social Influence Horizon
  • Started the draft deck for the tech summit
  • More MatrixScalar
    • Core functions work
    • Change test and train within the class to input and target
    • Create a coordinating class that loads and creates test and train matrices
  • JuryRoom meeting
    • Progress is good enough to start tracking it. Going to create a set of Google sheets that keep track of tasks and bugs

Phil 5.6.19

7:00 – 5:00 ASRC GOES-R

  • Finished the AI/ML paper with Aaron M over the weekend. I need to have him ping me when it goes in. I think it turned out pretty well, even when cut down to 7 pages (with references!! Why, IEEE, why?)
    • Sent a copy to Wayne, and distributed around work. Need to put in on ArXiv on Thursday
  • Starting to pull parts from phifel.com to make the lit review for the dissertation. Those reviews may have had a reason after all!
    • And oddly (though satisfying), I wound up adding a section on Moby-Dick as a way of setting up the rest of the lit review
  • More Matrix scalar class. Basically a satisfying day of just writing code.
  • Need to fix IEEE letter and take a self-portrait. Need to charge up the good camera

Phil 5.3.19

8:00 – 5:00 ASRC AIMES

  • I may need to do something Star Wars-ish tomorrow
  • Didn’t get the Google AI for Social Good. Sigh
  • Working on the MatrixScalar class
    • use np.array as core type?
    • Good progress. I can build a scaled, square matrix
    • Need to add a log(x)/10X scale/rescale

Phil 5.2.19

7:00 – 9:00 ASRC NASA

  • Wrote up my notes from yesterday
  • Need to make an Akido Drone image, maybe even a sim in Zach’s environment?
  • Changed the title of the Dissertation
  • Need to commit the changes to LMN from the laptop – done
  • Need to create an instance of the JASSS paper in overleaf and make sure it runs
  • Put the jasss.bst file in the svn repo – done
  • Thinking about putting my dict find on stackoverflow, but did see this page on xpath for dict that is making me wonder if I just shouldn’t point there.
  • Did meaningless 2019 goal stuff
  • Adding ragged edge argument and generate a set of curves for eval
  • ML seminar 4:30
  • Meeting with Aaron M at 7:00
    • Spent a good deal of time discussing the structure of the paper and the arguments. Aaron wants the point made that the “arc to full autonomy” is really only the beginning, predictable part of the process. In this part, the humans own the “reflective part” of the process, either as a human in the loop, where they decide to pull the trigger, or in the full autonomy mode where they select the training data and evaluation criteria for the reflexive system that’s built. The next part of that sequence is when machines begin to develop reflective capabilities. When that happens, many of the common assumptions that sets of human adversaries make about conflict (OODA, for example), may well be disrupted by systems that do not share the common background and culture, but have been directed to perform the same mission.

Phil 5.1.19

7:00 – 7:00 ASRC NASA AIMS

  • Added lit review section to the dissertation, and put the seven steps of sectarianism in.
  • Spent most of yesterday helping Aaron with TimeSeriesML. Currently working on a JSON util that will get a value on a provided path
  • Had to set up python at the module and not project level, which was odd. Here’s how: www.jetbrains.com/help/idea/2016.1/configuring-global-project-and-module-sdks.html#module_sdk
  • Done!
        def lfind(self, query_list:List, target_list:List, targ_str:str = "???"):
            for tval in target_list:
                if isinstance(tval, dict):
                    return self.dfind(query_list[0], tval, targ_str)
                elif tval == query_list[0]:
                    return tval
    
        def dfind(self, query_dict:Dict, target_dict:Dict, targ_str:str = "???"):
            for key, qval in query_dict.items():
                # print("key = {}, qval = {}".format(key, qval))
                tval = target_dict[key]
                if isinstance(qval, dict):
                    return self.dfind(qval, tval, targ_str)
                elif isinstance(qval, list):
                    return self.lfind(qval, tval, targ_str)
                else:
                    if qval == targ_str:
                        return tval
                    if qval != tval:
                        return None
    
        def find(self, query_dict:Dict):
            # pprint.pprint(query_dict)
            result = self.dfind(query_dict, self.json_dict)
            return result
    
    
  • It’s called like this:
    ju = JsonUtils("../../data/output_data/lstm_structure.json")
    # ju.pprint()
    result = ju.find({"config":[{"class_name":"Masking", "config":{"batch_input_shape": "???"}}]})
    print("result 1 = {}".format(result))
    result = ju.find({"config":[{"class_name":"Masking", "config":{"mask_value": "???"}}]})
    print("result 2 = {}".format(result))
  • Here’s the results:
    result 1 = [None, 12, 1]
    result 2 = 666.0
  • Got Aaron’s code running!
  • Meeting with Joel
    • A quicker demo that I was expecting, though I was able to walk through how to create and use Corpus Manager and LMN. Also, we got a bug where the column index for the eigenvector didn’t exist. Fixed that in JavaUtils.math.Labeled2DMatrix.java
  • Meeting with Wayne
    • Walked through the JASSS paper. Need to make sure that the lit review is connected and in the proper order
    • Changed the title of the dissertation to
      • Stampede Theory: Mapping Dangerous Misinformation at Scale
    • Solidifying defense over the winter break, with diploma in the Spring
    • Mentioned the “aikido with drones” concept. Need to make an image. Actually, I wonder if there is a way for that model to be used for actually getting a grant to explore weaponized AI in a way that isn’t directly mappable to weapons systems, but is close enough to reality that people will get the point.
    • Also discussed the concept of managing runaway AI with the Sanhedrin-17a concept, where unanimous agreement to convict means acquittal.  Cities had Sanhedrin of 23 Judges and the Great Sanhedrin had 71 Judges en.wikipedia.org/wiki/Sanhedrin
      • Rav Kahana says: In a Sanhedrin where all the judges saw fit to convict the defendant in a case of capital law, they acquit him. The Gemara asks: What is the reasoning for this halakha? It is since it is learned as a tradition that suspension of the trial overnight is necessary in order to create a possibility of acquittal. The halakha is that they may not issue the guilty verdict on the same day the evidence was heard, as perhaps over the course of the night one of the judges will think of a reason to acquit the defendant. And as those judges all saw fit to convict him they will not see any further possibility to acquit him, because there will not be anyone arguing for such a verdict. Consequently, he cannot be convicted.

 

Phil 4.19.19

8:00 – 4:00 ASRC TL

  • Updating working copies of the paper based on the discussion with Aaron M last night.
  • Based on the diagrams of the weights that I could make with the MNIST model, I think I want to try to make a layer neuron/weight visualizer. This one is very pretty
  • Need to start on framework for data generation and analysis with Zach this morning
  • Got Flask working (see above for rant on how).
  • Flask-RESTful provides an extension to Flask for building REST APIs. Flask-RESTful was initially developed as an internal project at Twilio, built to power their public and internal APIs.

Phil 4.18.19

7:00 – ASRC TL

  • Added the Talmud to the implications section: “Rav Kahana says: In a Sanhedrin where all the judges saw fit to convict the defendant in a case of capital law, they acquit him.
  • Changed the title of the dissertation again. Now it’s Stampede Theory: Diversity in Networked Systems.
  • Need to transition Machine Teaching paper to IEEE format before meeting with Aaron
  • Shimei’s ML group – want to talk about narrative embedding
    • Semi-Supervised Classification with Graph Convolutional Networks
      • We present a scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs. We motivate the choice of our convolutional architecture via a localized first-order approximation of spectral graph convolutions. Our model scales linearly in the number of graph edges and learns hidden layer representations that encode both local graph structure and features of nodes. In a number of experiments on citation networks and on a knowledge graph dataset we demonstrate that our approach outperforms related methods by a significant margin.
  • More JASS writing
    • Fixed the prior work image to have all three states
      • Note: In Illustrator, the easiest way to outline an image is to select the image, click on mask, then select the outline stroke, select a color, and you’re done.
    • changed \cite{} to \citep{}, which puts the parens in the right place
  • Meeting with Aaron. Nice LaTex lesson

Phil 4.17.19

7:00 – 5:00 ASRC TL

  • Continuing to read Army of None. Really solid analysis
  • Working on JASS paper
  • The buzz on Twitter about the possible change to topic-based (e.g. Reddit?) rather than person-based, makes me wonder if there should be the ability to follow people in JuryRoom. I’d follow Cricket, for example
    • “Cricket sits down in front of the Troll doing her best to appear completely relaxed and smiles, “come and lay down again while I sing.” Gesturing in front of her she gives a smile, “and if I can just look at the pretty box for a little bit maybe right here in front of me? It will stay really close so you can grab it up once you fall asleep.” At this point Cricket hadn’t actually lied, she currently had no intention of taking the box but if it opened as she suspected she fully intended to open it and hopefully take the contents.”
  • More Grokking MNIST. Here’s a pix of the neurons. The difference about halfway through is the switch from training to testing data: mnist
  • MorphNet: Towards Faster and Smaller Neural Networks
    • Here we describe MorphNet, a sophisticated technique for neural network model refinement, which takes the latter approach. Originally presented in our paper, “MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks”, MorphNet takes an existing neural network as input and produces a new neural network that is smaller, faster, and yields better performance tailored to a new problem. We’ve applied the technique to Google-scale problems to design production-serving networks that are both smaller and more accurate, and now we have open sourced the TensorFlow implementation of MorphNet to the community so that you can use it to make your models more efficient.
  • Had one of the stupidest, brain-damaged meetings I have ever had. Just destructive for destruction’s sake, as near as I can tell.
  • Updated IntelliJ, which was painful this time, requiring finding Java and Python SDKs that were clearly visible in the settings. I guess it’s that kind of day.

Phil 4.15.19

7:00 – ASRC TL

  • I’ve been hunting around for what a core message of the iSchool should be (And I like LAMDA), but I think this sums it up nicely. From The Library Book: Library
  • use arxiv2bibtex to get bibtex information for arXiv submissions for use in BibTeX, on web pages or in Wikis. You can enter:
    • one or several paper IDs like “1510.01797” or “math/0506203”.
    • your arXiv author ID looking similar to “grafvbothmer_h_1” to get a list of all your submitted papers.
    • your ORCID ID looking similar to “0000-0003-0136-444X” which you should register with your arXiv-account.
  • Here’s hoping the proposal goes in. It did!
  • Start on IEEE paper? Nope. Did get back to Grokking Deep learning. Trying to get the system working with MNIST.
  • Something for the arousal potential/Clockwork Muse file: Accelerating dynamics of collective attention
    • With news pushed to smart phones in real time and social media reactions spreading across the globe in seconds, the public discussion can appear accelerated and temporally fragmented. In longitudinal datasets across various domains, covering multiple decades, we find increasing gradients and shortened periods in the trajectories of how cultural items receive collective attention. Is this the inevitable conclusion of the way information is disseminated and consumed? Our findings support this hypothesis. Using a simple mathematical model of topics competing for finite collective attention, we are able to explain the empirical data remarkably well. Our modeling suggests that the accelerating ups and downs of popular content are driven by increasing production and consumption of content, resulting in a more rapid exhaustion of limited attention resources. In the interplay with competition for novelty, this causes growing turnover rates and individual topics receiving shorter intervals of collective attention.
  • Chasing down narrative embedding using force-directed graphs and found Tulip
    • Tulip is an information visualization framework dedicated to the analysis and visualization of relational data. Tulip aims to provide the developer with a complete library, supporting the design of interactive information visualization applications for relational data that can be tailored to the problems he or she is addressing.
    • There are Python bindings. The following are for large layouts
      • FM^3 (OGDF)
        • Implements the FM³ layout algorithm by Hachul and Jünger. It is a multilevel, force-directed layout algorithm that can be applied to very large graphs.
      • H3 (GRIP)
        • Implements the H3 layout technique for drawing large directed graphs as node-link diagrams in 3D hyperbolic space. That algorithm can lay out much larger structures than can be handled using traditional techniques for drawing general graphs because it assumes a hierarchical nature of the data. It was first published as: H3: Laying out Large Directed Graphs in 3D Hyperbolic Space . Tamara Munzner. Proceedings of the 1997 IEEE Symposium on Information Visualization, Phoenix, AZ, pp 2-10, 1997. The implementation in Python (MIT License) has been written by BuzzFeed (https://github.com/buzzfeed/pyh3).
  • Mahzarin R. Banaji
    • Professor Banaji studies thinking and feeling as they unfold in social context, with a focus on mental systems that operate in implicit or unconscious mode. She studies social attitudes and beliefs in adults and children, especially those that have roots in group membership.  She explores the implications of her work for questions of individual responsibility and social justice in democratic societies. Her current research interests focus on the origins of social cognition and applications of implicit cognition to improve individual decisions and organizational policies. 
      • What do Different Beliefs Tell us? An Examination of Factual, Opinion-Based, and Religious Beliefs 
        • Children and adults differentiate statements of religious belief from statements of fact and opinion, but the basis of that differentiation remains unclear. Across three experiments, adults and 8-10-year-old children heard statements of factual, opinion-based, and religious belief. Adults and children judged that statements of factual belief revealed more about the world, statements of opinion revealed more about individuals, and statements of religious belief provided information about both. Children—unlike adults—judged that statements of religious belief revealed more about the world than the believer. These results led to three conclusions. First, judgments concerning the relative amount of information statements of religious belief provide about individuals change across development, perhaps because adults have more experience with diversity. Second, recognizing that statements of religious belief provide information about both the world and the believer does not require protracted learning. Third, statements of religious belief are interpreted as amalgams of factual and opinion-based statements.
          • My sense is that these three regios – factual, religious, and opinion are huge attractors in our belief landscape
      • Studying Implicit Social Cognition with Noninvasive Brain Stimulation