Category Archives: Java

Phil 8.26.19

7:00 – ASRC GOES

  • Dissertation – working my way through the lit review section
  • Antonio sent a note about Software Impacts, which provides a scholarly reference to software that has been used to address a research challenge. The journal disseminates impactful and re-usable scientific software through Original Software Publications (OSP) which describe the application of the software to research and the published outputs.
    • Submissions to Software Impacts consist of two major parts:
      • A short descriptive paper of about three pages including an Impact Overview and references to publications where the software has been used
      • An open source software distribution with support material.
    • So, to get things to fit on GitHub, I worked on getting GPM to work with a smaller library – done
  • Discussions with Aaron about using TF 2.0 xformer on GOES sim data
  • Security training – an hour or so
  • Copied the Waikato JuryRoom proposal to PolarizationGame folder

Phil 3.3.19

Once more, icky weather makes me productive

  • Ingested all the runs into the db. We are at 7,246 posts
  • Reworking the 5 bucket analysis
  • Building better ignore files and rebuilding bucket spreadsheets. It tuns out that for tymora1, names took up 25% of the BOW, so I increased the fraction saved to the trimmed spreadsheets to 50%
  • Building bucket spreadsheets and saving the centrality vector
  • Here’s what I’ve got so far: ThreeRuns
  • Trajectories: Trajectories
  • First map: firstMap
  • Here it is annotated: firstMapAnnotated
  • Some thoughts. I think this is still “zoomed out” too far. Changing the granularity should help some. I need to automate some of my tools though. The other issue is how I’m assembling my sequences.

Phil 3.2.19

Updating SheetToMap to take comma separated cell names. Lines 180 – 193. I think I’ll need an iterating compare function. Nope, wound up doing something simpler

for (String colName : colNames) {
    String curCells = tm.get(colName);
    String[] cellArray = curCells.split("\\|\\|"); <--- new!
    for(String curCell : cellArray) {
        addNode(curCell, rowName);
        if (prevCell != null && !curCell.equals(prevCell)) {
            String edgeName = curCell + "+" + prevCell;
            if (graph.getEdge(edgeName) == null) {
                try {
                    graph.addEdge(edgeName, curCell, prevCell);
                    System.out.println("adding edge [" + edgeName + "]");
                } catch (EdgeRejectedException e) {
                    System.out.println("didn't add edge [" + edgeName + "]");
                }
            }
        }
        prevCell = curCell;
    }

    //System.out.print(curCell + ", ");
    prevCell = cellArray[0];
    col++;
}

Updating GPM to generate comma separated cell names in trajectories

  • need to get the previous n cell names
  • Need to change the cellName val in FlockingBeliefCA to be a stack of tail length. Done.
  • Parsed the strings in SheetToMap. Each cell has a root name (the first) which connects to the roots of the previous cell. The root then links to the subsequent names in the chain of names that are separated by “||” 
    "cell_[4, 5]||cell_[4, 4]||cell_[4, 3]||cell_[4, 2]||cell_[4, 1]"
  • Seems to be working: tailtest

Phil 2.25.19

7:00 – 2:30 ASRC TL

2:30 – 4:30 PhD

  • Fix directory code of LMN so that it remembers the input and output directories – done
  • Add time bucketing capabilities. Do this by taking the complete conversation and splitting the results into N sublists. Take the beginning and ending time from each list and then use those to set the timestamp start and stop for each player’s posts.
  • Thinking about a time-series LMN tool that can chart the relative occurrence of the sorted terms over time. I think this could be done with tkinter. I would need to create and executable as described here, though the easiest answer seems to be pyinstaller.
  • Here are two papers that show the advantages of herding over nomadic behavior:
    • Phagotrophy by a flagellate selects for colonial prey: A possible origin of multicellularity
      • Predation was a powerful selective force promoting increased morphological complexity in a unicellular prey held in constant environmental conditions. The green alga, Chlorella vulgaris, is a well-studied eukaryote, which has retained its normal unicellular form in cultures in our laboratories for thousands of generations. For the experiments reported here, steady-state unicellular C. vulgaris continuous cultures were inoculated with the predator Ochromonas vallescia, a phagotrophic flagellated protist (‘flagellate’). Within less than 100 generations of the prey, a multicellular Chlorella growth form became dominant in the culture (subsequently repeated in other cultures). The prey Chlorella first formed globose clusters of tens to hundreds of cells. After about 10–20 generations in the presence of the phagotroph, eight-celled colonies predominated. These colonies retained the eight-celled form indefinitely in continuous culture and when plated onto agar. These self-replicating, stable colonies were virtually immune to predation by the flagellate, but small enough that each Chlorella cell was exposed directly to the nutrient medium.
    • De novo origins of multicellularity in response to predation
      • The transition from unicellular to multicellular life was one of a few major events in the history of life that created new opportunities for more complex biological systems to evolve. Predation is hypothesized as one selective pressure that may have driven the evolution of multicellularity. Here we show that de novo origins of simple multicellularity can evolve in response to predation. We subjected outcrossed populations of the unicellular green alga Chlamydomonas reinhardtii to selection by the filter-feeding predator Paramecium tetraurelia. Two of five experimental populations evolved multicellular structures not observed in unselected control populations within ~750 asexual generations. Considerable variation exists in the evolved multicellular life cycles, with both cell number and propagule size varying among isolates. Survival assays show that evolved multicellular traits provide effective protection against predation. These results support the hypothesis that selection imposed by predators may have played a role in some origins of multicellularity. SpontaniousClustering\

Phil 8.27.18

7:00 – 5:00 ASRC MKT

  • Good chat with Barbara yesterday. She suggests horse racing podcasts, since the question is always the same “who’s going to win today” and the information to discuss is much more constrained. Additionally, there is the wagering information that could be used to determine the level of consensus?
  • Found an idiom translator! “Swing of the pendulum” occurs at least in French, German and Italian
  • Downloaded the new videos Need to put them in the ppt when the slides stabilize
  • Pinged Wayne about getting together today
  • Changed the questions page to have English, Italian, French and German terms for belief space
  • Another example of diversity injection (twitter)
  • Working on podcast text handling
      • Created the MapsFromPodcasts project in Development
      • Created an new key and downloaded the key json file
      • Installed Google Cloud Tools (213.0.0), following the directions of this page. Wow. Lots of stuff! 
        Output folder: D:\Programs\GoogleCloudAPI
Downloading Google Cloud SDK core.
Extracting Google Cloud SDK core.
Create Google Cloud SDK bat file: D:\Programs\GoogleCloudAPI\cloud_env.bat
Installing components.
Welcome to the Google Cloud SDK!
Your current Cloud SDK version is: 213.0.0
Installing components from version: 213.0.0
+-----------------------------------------------------------------------------+
| These components will be installed. |
+-----------------------------------------------------+------------+----------+
| Name | Version | Size |
+-----------------------------------------------------+------------+----------+
| BigQuery Command Line Tool | 2.0.34 | < 1 MiB |
| BigQuery Command Line Tool (Platform Specific) | 2.0.34 | < 1 MiB |
| Cloud SDK Core Libraries (Platform Specific) | 2018.06.18 | < 1 MiB |
| Cloud Storage Command Line Tool | 4.33 | 3.6 MiB |
| Cloud Storage Command Line Tool (Platform Specific) | 4.32 | < 1 MiB |
| Cloud Tools for PowerShell | | |
| Cloud Tools for PowerShell | 1.0.1.8 | 17.9 MiB |
| Default set of gcloud commands | | |
| Windows command line ssh tools | | |
| Windows command line ssh tools | 2017.09.15 | 1.8 MiB |
| gcloud cli dependencies | 2018.08.03 | 1.3 MiB |
+-----------------------------------------------------+------------+----------+
For the latest full release notes, please visit:
https://cloud.google.com/sdk/release_notes
#============================================================#
#= Creating update staging area =#
#============================================================#
#= Installing: BigQuery Command Line Tool =#
#============================================================#
#= Installing: BigQuery Command Line Tool (Platform Spec... =#
#============================================================#
#= Installing: Cloud SDK Core Libraries (Platform Specific) =#
#============================================================#
#= Installing: Cloud Storage Command Line Tool =#
#============================================================#
#= Installing: Cloud Storage Command Line Tool (Platform... =#
#============================================================#
#= Installing: Cloud Tools for PowerShell =#
#============================================================#
#= Installing: Cloud Tools for PowerShell =#
#============================================================#
#= Installing: Default set of gcloud commands =#
#============================================================#
#= Installing: Windows command line ssh tools =#
#============================================================#
#= Installing: Windows command line ssh tools =#
#============================================================#
#= Installing: gcloud cli dependencies =#
#============================================================#
#= Creating backup and activating new installation =#
#============================================================#
Performing post processing steps...
..............................................................................................................................................................done.
Update done!
This will install all the core command line tools necessary for working with
the Google Cloud Platform.
For more information on how to get started, please visit:
https://cloud.google.com/sdk/docs/quickstarts
Google Cloud SDK has been installed!

         

     

    • Google is sooooooooooooooooooooo Unix/Linux
  • Meeting with Wayne
    • Fix slides some more
    • Email about demo and poster – done

Phil 8.10.18

7:00 – ASRC MKT

  • Finished the first pass through the SASO slides. Need to start working on timing (25 min + 5 min questions)
  • Start on poster (A0 size)
  • Sent Wayne a note to get permission for 899
  • Started setting up laptop. I hate this part. Google drive took hours to synchronize
    • Java
    • Python/Nvidia/Tensorflow
    • Intellij
    • Visual Studio
    • MikTex
    • TexStudio
    • Xampp
    • Vim
    • TortoiseSVN
    • WinSCP
    • 7-zip
    • Creative Cloud
      • Acrobat
      • Reader
      • Illustrator
      • Photoshop
    • Microsoft suite
    • Express VPN

Phil 6.27.18

7:00 – 12:00 ASRC MKT

  • Print out documents! Done. Got passport drive too.
  • Need to write an extractor that lets the user navigate the xml file containing influences of selected agents. This could be a sample-by sample network. Maybe two modes?
    • Select an agent and see all the other agents come in and out of influcene
    • Select an number of agents and only watch the mutual influence.
    • There is an integrated JavaFX charts that I could use, or it could be an uploaded webapp? JavaFX would be easier in the short term, but a webapp would help more with JuryRoom…
    • Another option would be Python, since that’s where the LSTM code will live.
    • On the whole, two days before leaving on travel is probably the wrong time to start coding
  • Fixed a bug in the xml file generation
  • copied the new jar file onto the thumb drive
  • copied the xml file onto the thumb drive

12:00 – 4:00 ASRC A2P

  • Pomoting things to QA – done! Or at least, up to date with the excel files

Phil 6.26.18

7:00 – 5:00 ASRC MKT

  • Started back with the Evolution of Cooperation
  • Social loafing (Scholar results)
    • In social psychology, social loafing is the phenomenon of a person exerting less effort to achieve a goal when they work in a group than when they work alone. This is seen as one of the main reasons groups are sometimes less productive than the combined performance of their members working as individuals, but should be distinguished from the accidental coordination problems that groups sometimes experience. Research on social loafing began with rope pulling experiments by Ringelmann, who found that members of a group tended to exert less effort in pulling a rope than did individuals alone. In more recent research, studies involving modern technology, such as online and distributed groups, have also shown clear evidence of social loafing. Many of the causes of social loafing stem from an individual feeling that his or her effort will not matter to the group.
  • NELA2017 contains almost every news article from 92 sources between April 2017 and October 2017, amounting to over 136K articles. This data set is the first release of NELA datasets. This version of the data set can be found on github and a full description and use cases can be found in our 2018 ICWSM paper.
  • Submitted “One Simple Trick” final to SASO
  • Updated ArXive
  • Fixed a bug that prevented population interactions in FlockingAgentManager.initializeAgents(): 
                // add to the global list
            allBoidsList.add(fs);

            // add a pointer to the global list to each shape
            fs.setFlockingShapeList(allBoidsList);

            // Add to the flock so that we can get flock headings
            List flock = flockListsMap.get(flockName);
            flock.add(fs);

    Seriously, what was I thinking?

  • Continued GUI tweaking. I think it looks pretty good, and it fits (mostly) on my laptop Version6.26.18
  • Verified that the influences record agents from different flocks and sources.
  • Copied all CI 2018 things I can think of onto the thumb drive

Phil 6.25.18

7:00 – 9:00 ASRC MKT

  • Update laptop – Intellij, Java, GroupPolarazation codebase
  • Add XML output for influence – done!
  • Refactored the GUI to work with smaller (laptop) screens)

9:00 – 2:30 ASRC A2P

  • Debug what’s going on with the excel reading. Try a new config file first?
  • Ground slowly through options
    • Replaced the config file
    • Stepped through the debugger, and noticed that the worksheet was null. Tried a different worksheet/config, and that was *not* null
    • Created a new workbook and copied everything over without formatting. That worked on the converter, but didn’t work with A2P
    • Reformatted the new workbook and wound up using the Funding Summary Details data with the formatting, which is *crazy*….
    • Had some issues getting connected to the server. Pageant forgot my key.

3:00 – 4:00 ASRC MKT

  • Fika. No, not really. Wound up chatting with Will

Phil 6.22.18

7:00 – 5:30 ASRC MKT

  • Twitter experiment on a fake Gary Indiana secession. IFTTT retweeting leads to interesting behavior.
  • Fixed FlockingShape casting by adding a customDrawStep(GraphicsContext gc) to the SmartShape base class that’s called from draw().
  • Add records to each agent that store a list of source and agent influences at each time sample. It should include the name of the item and the amount of influence. Probably save as an XML file, since it has too many dimensions. The file could then be used to create terms or spreadsheets.
    • Started on CAInfluence class which will be added to CA classes in an arrayList in BaseCA;
  • More file conversion with Bob – and everything worked great until I try one after Bob leaves. Ka-BOOM!
    • Installed all the packages to get everything to run in the debugger. Found what appears to be a perfectly good line “range” that causes the problem? Will start debugging on Wednesday.
  • Project MERCATOR proposal
  • Meeting with Sy

Phil 6.21.18

7:00 – 4:00 ASRC MKT

  • Add an attractor scalar for agents that’s normally zero. A vector to each agent within the SIH is calculated and scaled by the attractor scalar. That vector is then added to the direction vector to the agent – done
  • Remove the heading influence based on site – done
  • Add a white circle to the center of the agent that is the size of the attraction scalar. Done
  • Add attraction radius slider that is independent of the SIH. -done
  • Add a ‘site trajectory’ to the spreadsheet that will have the site lists (and their percentage?)
  • There is now an opportunity for a poster and a demo at SASO
  • Add stories, lists and maps to implication slides – done
  • Got all my connections set up
  • Successfully converted and deployed cosmos-2
  • Voted!

Phil 6.20.18

7:00 – 9:00 2:00 – 5:00 ASRC MKT

  • Redo doodle for all of August – done
  • Schooling Fish May Offer Insights Into Networked Neurons
    • Iain Couzin is deciphering the rules that govern group behavior. The results might provide a fresh perspective on how networks of neurons work together.
  • City arts and lectures: The New Science Of Psychedelics With Michael Pollan
    • Psychedelics reduce the section of the brain that have to do with the sense of self. Pollan thinks that this also happens with certain types of rhythmic music and in crowd situations. This could be related to stampedes and flocking.
    • LSD May Chip Away at the Brain’s “Sense of Self” Network
      • Brain imaging suggests LSD’s consciousness-altering traits may work by hindering some brain networks and boosting overall connectivity
  • Add an attractor scalar for agents that’s normally zero. A vector to each agent within the SIH is calculated and scaled by the attractor scalar. That vector is then added to the direction vector to the agent – done?
  • Remove the heading influence based on site – done
  • Add a white circle to the center of the agent that is the size of the attraction scalar. Done
  • Add a ‘site trajectory’ to the spreadsheet that will have the site lists (and their percentage?)
  • Worked on A2P white paper with Aaron.
  • Worked on a response to Dr. Li’s response

ASRC IRAD 9:00 – 2:00

  • Mind meld with Bob
    • Revisit Yarn
    • Excel stuff?
    • Connect to AWS using bastion. Look in FoxyProxy how to. I need certs
    • Drop on rabbit to deploy to CI and QA and NESDIS  ONE (production)
    • Don’t want sensitive information in Git. We use sharepoint instead
    • Notes and screenshots in document.

Phil 5.18.18

7:00 – 4:00 ASRC MKT

Phil 5.17.18

7:00 – 4:00 ASRC MKT

  • How artificial intelligence is changing science – This page contains pointers to a bunch of interesting projects:
  • Multi-view Discriminative Learning via Joint Non-negative Matrix Factorization
    • Multi-view learning attempts to generate a classifier with a better performance by exploiting relationship among multiple views. Existing approaches often focus on learning the consistency and/or complementarity among different views. However, not all consistent or complementary information is useful for learning, instead, only class-specific discriminative information is essential. In this paper, we propose a new robust multi-view learning algorithm, called DICS, by exploring the Discriminative and non-discriminative Information existing in Common and view-Specific parts among different views via joint non-negative matrix factorization. The basic idea is to learn a latent common subspace and view-specific subspaces, and more importantly, discriminative and non-discriminative information from all subspaces are further extracted to support a better classification. Empirical extensive experiments on seven real-world data sets have demonstrated the effectiveness of DICS, and show its superiority over many state-of-the-art algorithms.
  • Add Nomadic, Flocking, and Stampede to terms. And a bunch more
  • Slides
  • Embedding navigation
    • Extend SmartShape to SourceShape. It should be a stripped down version of FlockingShape
    • Extend BaseCA to SourceCA, again, it should be a stripped down version of FlockingBeliefCA
    • Add a sourceShapeList for FlockingAgentManager that then passes that to the FlockingShapes
  • And it’s working! Well, drawing. Next is the interactions: Influence
  • Finally went and joined the IEEE

Phil 5.16.18

7:00 – 3:30 ASRC MKT

  • My home box has become very slow. 41 seconds to do a full recompile of GPM, while it takes 3 sec on a nearly identical machine at work. This may help?
  • Working on terms
  • Working on slides
  • Attending talk on Big Data, Security and Privacy – 11 am to 12 pm at ITE 459
    • Bhavani Thiraisingham
    • Big data management and analytics emphasizing GANs  and deep learning<- the new hotness
      • How do you detect attacks?
      • UMBC has real time analytics in cyber? IOCRC
    • Example systems
      • Cloud centric assured information sharing
    • Research challenges:
      • dynamically adapting and evolving policies to maintain privacy under a changing environment
      • Deep learning to detect attacks tat were previously not detectable
      • GANs or attacker and defender?
      • Scaleabe is a big problem, e.g. policies within Hadoop operatinos
      • How much information is being lost by not sharing data?
      • Fine grained access control with Hive RDF?
      • Distributed Search over Encrypted Big Data
    • Data Security & Privacy
      • Honypatching – Kevin xxx on software deception
      • Novel Class detection – novel class embodied in novel malware. There are malware repositories?
    • Lifecycle for IoT
    • Trustworthy analytics
      • Intel SGX
      • Adversarial SVM
      • This resembles hyperparameter tuning. What is the gradient that’s being descended?
      • Binary retrofitting. Some kind of binary man-in-the-middle?
      • Two body problem cybersecurity
    • Question –
      • discuss how a system might recognize an individual from session to session while being unable to identify the individual
      • What about multiple combinatorial attacks
      • What about generating credible false information to attackers, that also has steganographic components for identifying the attacker?
  • I had managed to not commit the embedding xml and the programs that made them, so first I had to install gensim and lxml at home. After that it’s pretty straightforward to recompute with what I currently have.
  • Moving ARFF and XLSX output to the menu choices. – done
  • Get started on rendering
    • Got the data read in and rendering, but it’s very brute force: 
      if(getCurrentEmbeddings().loadSuccess){
    double posScalar = ResizableCanvas.DEFAULT_SCALAR/2.0;
    List<WordEmbedding> weList = currentEmbeddings.getEmbeddings();
    for (WordEmbedding we : weList){
        double size = 10.0 * we.getCount();
        SmartShape ss = new SmartShape(we.getEntry(), Color.WHITE, Color.BLACK);
        ss.setPos(we.getCoordinate(0)*posScalar, we.getCoordinate(1)*posScalar);
        ss.setSize(size, size);
        ss.setAngle(0);
        ss.setType(SmartShape.SHAPE_TYPE.OVAL);
        canvas.addShape(ss);
    }
}

      It took a while to remember how shapes and agents work together. Next steps:

      • Extend SmartShape to SourceShape. It should be a stripped down version of FlockingShape
      • Extend BaseCA to SourceCA, again, it should be a stripped down version of FlockingBeliefCA
      • Add a sourceShapeList for FlockingAgentManager that then passes that to the FlockingShapes