Monthly Archives: August 2017

Phil 8.31.17

7:00 – 9:00, 10:00-4:30 IRAD

  • Something for my Visible from Space collection?
  • Saw this in the letters from CACM this month:

    Causal Connections for Predictive AI

    As part of the “ACM Panels in Print” section “Artificial Intelligence” (Feb. 2017), panelist David Blei said he believes computer science needs to identify the causal connections between data components, concluding that artificial intelligence, with its predictive capabilities, will be enhanced through causal inference. For example, the first step toward using AI in database systems is to analyze and create a data map of the complexity of the causal interconnections between the data components in a problem space. A data map connects a data component to other data components through causal interrelationships. A data map can be created through qualitative analysis of data collected for a particular problem. The qualitative analysis could then take the form of “thematic analysis,” (Braun, V. and Clarke, V. 2006) using systems diagramming to gain greater insight into the data. At that point it might be advantageous to start applying AI directly to the data.

    Researching the complexity of database systems, I have thus created such a data map, which is now ready to move to the next stage of automation where predictive analytics can help improve management of database systems. Using an analogy of the CODEX, or Control of Data Expediently, my research into causal connections has identified a potential role for AI in automating continuously changing best practice, thus representing an agile approach to deciphering the complexity of interconnections and promising to help create an autonomous way to deliver best practice in database management.

    Victoria Holt, Bath, U.K.

  • I looked at hew blog, research page and twitter account. Can’t find anything about mapping. Even looking at the thematic analysis and citing articles, I can’t find much about maps or mapping. Odd
  • Getting back to drawing the database. But first, updating my IDE, it seems…
  • Broke out DimensionInfo to TensorDimensionInfo since I need the mappingLow and mappingHigh to draw the borders
  • Changed the canvas drawBG to use StorageAndRetreival, since I need to know the projection I’m drawing
  • Killed JavaFX for a while. Had to reboot
  • More RFI work
  • Ok, got the background scaled correctly and drawing the border. Had to deal with more hard-coded values. SAD! Anyway, here’s all the parts working: stageWithBorder

Phil 8.30.17

6:30 – 6:00 ASRC Research, proposals and job stuff

  • Redid my resume this morning. The word doc went fine, but when I went to generate the html versions, I had a momentary lapse of good sense and had Word output the pages.  Horrible mistake. I had to go and use the very fine HTML Cleaner website and put everything back together again. So 15 minutes of adding a paragraph and a picture turned into over an hour of work. Sigh.
  • If I have time, the next step today is to get the StorageAndRetrieval code drawing on the canvas. Started setting this up. The actual cell coloring could be done with another evaluateTensor method? Maybe later. Get the basics working first.
  • In a significant miracle, my PhD work is now my job, at least until the end of the year. Sprinkled with some proposal writing.
  • Spent most of the day working on the NNSA RFI

Phil 8.29.17

7:00 – 6:30 ASRC Research

  • Probably not going to get the StorageAndRetreival class done and integrated by Sept 1, so I’ll need to work out the paper based on what I have. But let’s see how far we get…
  • I think I’m going to start simply by placing the agent type in the IR cell and color accordingly. There will be three states, NULL (white) , EXPLORE (red) and EXPLOIT (green)
  • Fixing some things along the way. Stage dimension initial size was hard coded. Oops.
  • Adding a clear() to LabeledTensor and StorageAndRetreival
  • Working out how to recurse though a set of arrays. Putting here for later reference:
    class Recurse{
        int linecount = 0;
        boolean fillMat(int[] curIndices, int[] maxIndices, int curIndex){
            if(curIndex > curIndices.length-1){
                //System.out.printf("curIndex(%d) > %d\n", curIndex, curIndices.length-1);
                return true;
            for(int i = 0; i < maxIndices[curIndex]; ++i) {
                curIndices[curIndex] = i;
                boolean done = fillMat(curIndices, maxIndices, curIndex+1);
                if(done) {
                    System.out.println(linecount+" [" + curIndex + "]: " + Arrays.toString(curIndices) + " = " + done);
            return false;
        public static void main(String[] args){
            int[] initial = {0, 0, 0};
            int[] max = {3, 3, 3};
            Recurse r = new Recurse();
            r.fillMat(initial, max, 0);
  • Here’s the final version, with an Interface callback
    public boolean fillTensor(int[] curIndices, int[] maxIndices, int curIndex, TensorCellEvaluator tce){
        if(curIndex > curIndices.length-1){
            // if we get here, we're done
            return true;
        for(int i = 0; i < maxIndices[curIndex]; ++i) {
            curIndices[curIndex] = i;
            boolean done = fillTensor(curIndices, maxIndices, curIndex+1, tce);
            if(done) {
                tce.evaluate(curIndices, lt);
        // if we get here, we're not none, keep going
        return false;
  • Here’s the call with the implemented interface. It doesn’t do anything yet, but the pieces all work
    stoRet.fillTensor(indicies, maxIndicies, 0, new TensorCellEvaluator() {
        public String evaluate(int[] indicies, LabeledTensor lt) {
            return Arrays.toString(indicies);
  • Interview with USDA team. Not a fit at all

Phil 8.28.17

Another weekend of scrambling to get everything done in time. Plus a good bike ride 🙂

7:00 – 5:00 Research (ASRC IRAD)

  • Started up the computer this morning and it freaked out. After a restart, it rebuilt all the program indices and many of my configs were gone. I’m guessing that it was a bad disk sector. Scary, since this is a pretty new box
  • Adding in the Information Retrieval part that will mediate interactions between agents
  • And now the changes to JavaUtils2 aren’t being recognized. Deleted reference and reimported. And not the project is rebuilding super slow????? This is just not my morning.
  • And now IntelliJ is frozen. Restarted. The error is still there, but it compiles no problem. Had to invalidate caches.
  • Started StorageAndRetreival class and have a rough framing running
  • Long discussion with the interns about using mapping for the HR space
  • Wrote up some more paragraphs on normative mapping for paper justification
  • Fuzzy thought for today. The map is actually a set of pointers to the data used to produce it. For example, a discussion that produces normative poles has a history of collapse that the normative poles could reference back to.

Phil 8.25.17

7:00 – 5:00 ASRC IRAD

  • Got Understanding Ignorance, which has a section on Ignorance as a Place. More on that later
  • Since I’m in the middle of LabledTensor, I’m going to add the ability to string together multiple ArrayLists to accommodate Long indexing
    • Created class LongArrayList, based on part of List. Can’t implement the interface, since the parameters are int, rather than long. Done! Also fixed a few formatting errors in the toString().

Phil 8.24.17

6:45 – 3:00 ASRC IRAD

Phil 8.23.17


  • Started the ball rolling on 899 approval and getting together waith Wayne for a chat
  • 899 is set up, Wayne is going to ECSCW, so sometime after that

8:30 – 5:30 ASRC

  • BRI suspended payment on the contract, so much churn. Lots of discussions with many people. Looks like an interview on Friday?
  • Had to reinstall Office. Getting coffee…
  • Connnected to my SVN and got the LabeledTensor code to work on. Need to add the support for labels and more than 32k entries.

Phil 8.18.17

7:00 – 8:00 Research

  • Got indexFromLocation() working. It took some fooling around with Excel. Here’s the method:
    public int[] indexFromLocation(double[] loc){
        int[] index = new int[loc.length];
        for(int i = 0; i < loc.length; ++i){
            double findex = loc[i]/mappingStep;
            double roundDown = Math.floor(findex);
            double roundUp = Math.ceil(findex);
            double lowdiff = findex - roundDown;
            double highdiff = roundUp - findex;
            if(lowdiff < highdiff){
                index[i] = (int)roundDown;
                index[i] = (int)roundUp;
        return index;
  • And here are the much cleaner results:
    • [0.00, 0.00] = [0, 0]
      [0.00, 0.10] = [0, 0]
      [0.00, 0.20] = [0, 1]
      [0.00, 0.30] = [0, 1]
      [0.00, 0.40] = [0, 2]
      [0.00, 0.50] = [0, 2]
      [0.00, 0.60] = [0, 2]
      [0.00, 0.70] = [0, 3]
      [0.00, 0.80] = [0, 3]
      [0.00, 0.90] = [0, 4]
      [0.00, 1.00] = [0, 4]

      [1.00, 0.00] = [4, 0]
      [1.00, 0.10] = [4, 0]
      [1.00, 0.20] = [4, 1]
      [1.00, 0.30] = [4, 1]
      [1.00, 0.40] = [4, 2]
      [1.00, 0.50] = [4, 2]
      [1.00, 0.60] = [4, 2]
      [1.00, 0.70] = [4, 3]
      [1.00, 0.80] = [4, 3]
      [1.00, 0.90] = [4, 4]
      [1.00, 1.00] = [4, 4]
  • Another thought that struck me as far as the (int) constraint is that I can have a number of ArrayLists that are embedded in a an object that has the first and last index in it. These would be linked together to provide unconstrained (MAX_VALUE or 2,147,483,647 lists) storage

8:30 – 4:30 BRI

  • I realized yesterday that the Ingest and Query microservices need to access the same GeoMesa Spring service. That keeps all the general store/query GeoMesa access code in one place, simplifies testing and allows for DI to provide the correct (hbase, accumulo, etc) implementation through a facade interface.
  • Got tangled up with getting classpaths right and importing the proper libraries
  • Got the maven files behaving, or at least not complaining on mvn clean and mvn compile!
  • Well that’s a new error: Error: Could not create the Java Virtual Machine. I get that running the new installation with the geomesa-quickstart-hbase
    • Ah, that’s what will happen when you paste your command-line arguments into the VM arguments space just above where it should go…
    • Wednesday’s goal will to verify that HBaseQuickStart is running correctly in its new home and start to turn it into a service.

Phil 8.17.17

BRI – one hour chasing down research hours from Jan – May

7:00 – 6:00 Research

  • Found this on negative flocking influences: The rise of negative partisanship and the nationalization of US elections in the 21st century.  Paper saved to Lit Review
    • One of the most important developments affecting electoral competition in the United States has been the increasingly partisan behavior of the American electorate. Yet more voters than ever claim to be independents. We argue that the explanation for these seemingly contradictory trends is the rise of negative partisanship. Using data from the American National Election Studies, we show that as partisan identities have become more closely aligned with social, cultural and ideological divisions in American society, party supporters including leaning independents have developed increasingly negative feelings about the opposing party and its candidates. This has led to dramatic increases in party loyalty and straight-ticket voting, a steep decline in the advantage of incumbency and growing consistency between the results of presidential elections and the results of House, Senate and even state legislative elections. The rise of negative partisanship has had profound consequences for electoral competition, democratic representation and governance.
  • Working on putting together an indexable high-dimension matrix that can contain objects. Generally, I’d expect it to be doubles, but I can see Strings and Objects as well.
  • Starting off by seeing what’s in the newest Apache Commons Math (v 3.6.1)
  • Found SimpleTensor, which uses the Efficient Java Matrix Library (EJML) and creates a 3D block of rows, columns and slices. THought it was what I wanted, but nope
  • Looks like there isn’t a class that would do what I need to do, or that I can even modify. I’m thinking that the best option is to use org.apache.commons.math3.linear.AbstractRealMatrix as a template.
  • Nope, coudn’t figure out how to do things as nested lists. So I’m doing it C-Style, where you really only have one array that you index into. Here’s a 4x4x4x4 Tensor filled with zeroes:
    Total elements = 256
    0.0:[0, 0, 0, 0], 0.0:[1, 0, 0, 0], 0.0:[2, 0, 0, 0], 0.0:[3, 0, 0, 0],
    0.0:[0, 1, 0, 0], 0.0:[1, 1, 0, 0], 0.0:[2, 1, 0, 0], 0.0:[3, 1, 0, 0],
    0.0:[0, 2, 0, 0], 0.0:[1, 2, 0, 0], 0.0:[2, 2, 0, 0], 0.0:[3, 2, 0, 0],
    0.0:[0, 3, 0, 0], 0.0:[1, 3, 0, 0], 0.0:[2, 3, 0, 0], 0.0:[3, 3, 0, 0],
    0.0:[0, 0, 1, 0], 0.0:[1, 0, 1, 0], 0.0:[2, 0, 1, 0], 0.0:[3, 0, 1, 0],
    0.0:[0, 2, 3, 3], 0.0:[1, 2, 3, 3], 0.0:[2, 2, 3, 3], 0.0:[3, 2, 3, 3],
    0.0:[0, 3, 3, 3], 0.0:[1, 3, 3, 3], 0.0:[2, 3, 3, 3], 0.0:[3, 3, 3, 3]
  • The only issue that I currently have is that ArrayLists are indexed by int, so the total size is 32k elements. That should be good enough for now, but it will need to be fixed.
  • set() and get() work nicely:
    lt.set(new int[]{0, 1, 0, 0}, 9.9);
    lt.set(new int[]{3, 3, 3, 3}, 3.3);
    System.out.println("[0, 1, 0, 0] = " + lt.get(new int[]{0, 1, 0, 0}));
    System.out.println("[3, 3, 3, 3] = " + lt.get(new int[]{3, 3, 3, 3}));
    [0, 1, 0, 0] = 9.9
    [3, 3, 3, 3] = 3.3
  • Started the indexFromLocation method, but this is too sloppy:
    index[i] = (int)Math.floor(Math.round(loc[i]/mappingStep));

Phil 8.16.17

7:00 – 8:00 Research

  • Added takeaway thoughts to my C&C writeup.
  • Working out how to add capability to the sim for P&RCH paper. My thoughts from vacation:
    • The agents contribution is the heading and speed
    • The UI is what the agent’s can ‘see’
    • The IR is what is available to be seen
    • An additional part might be to add the ability to store data in the space. Then the behavior of the IR (e.g. empty areas) would b more apparent, as would the effects of UI (only certain data is visible, or maybe only nearby data is visible) Data could be a vector field in Hilbert space, and visualized as color.
  • Updated IntelliJ
  • Working out how to to have a voxel space for the agents to move through that can also be drawn. It’s any number of dimensions, but it has to project to 2D. In the case of the agents, I just choose the first two axis. Each agent has an array of statements that are assembled into a belief vector. The space can be an array of beliefs. Are these just constructed so that they fill a space according to a set of rules? Then the xDimensionName and yDimensionName axis would go from (0, 1), which would scale to stage size? IR would still be a matter of comparing the space to the agent’s vector. Hmm.
  • This looks really good from an information horizon perspective: The Role of the Information Environment in Partisan Voting
    • Voters are often highly dependent on partisanship to structure their preferences toward political candidates and policy proposals. What conditions enable partisan cues to “dominate” public opinion? Here I theorize that variation in voters’ reliance on partisanship results, in part, from the opportunities their environment provides to learn about politics. A conjoint experiment and an observational study of voting in congressional elections both support the expectation that more detailed information environments reduce the role of partisanship in candidate choice

9:00 – 5:00 BRI

  • Good lord, the BoA corporate card comes with SIX seperate documents to read.
  • Onward to Chapter Three and Spring database interaction
  • Well that’s pretty clean. I do like the JdbcTemplate behaviors. Not sure I like the way you specify the values passed to the query, but I can’t think of anything better if you have more than one argument:
    public class EmployeeDaoImpl implements EmployeeDao {
        private DataSource dataSource;
        private JdbcTemplate jdbcTemplate;
        private RowMapper<Employee> employeeRowMapper = new RowMapper<Employee>() {
            public Employee mapRow(ResultSet rs, int i) throws SQLException {
                Employee employee = new EmployeeImpl();
                employee.setEmployeeName(rs.getString("FirstName") + " " + rs.getString("LastName"));
                return employee;
        public Employee getEmployeeById(int id) {
            Employee employee = null;
            employee = jdbcTemplate.queryForObject(
                    "select * from Employee where id = ?",
                    new Object[]{id},
            return employee;
        public List<Employee> getAllEmployees() {
            List<Employee> eList = jdbcTemplate.query(
                    "select * from Employee",
            return eList;
  • Here’s the xml to wire the thing up:
    <context:component-scan base-package="org.springframework.chapter3.dao"/>
    <bean id="employeeDao" class="org.springframework.chapter3.dao.EmployeeDaoImpl"/>
    <bean id="dataSource"
        <property name="driverClassName" value="${jdbc.driverClassName}" />
        <property name="url" value="${jdbc.url}" />
        <property name="username" value="xxx"/>
        <property name="password" value="yyy"/>
    <bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
        <property name="dataSource" ref="dataSource" />
    <context:property-placeholder location="" />
  • And here’s the properties. Note that I had to disable SSL:

Phil 8.15.17

7:00 – 8:00 Research

8:30 – 6:00 BRI

  • Remembered my bike shoes, but it may rain all day
  • Wrote up a note to T about burning PhD hours for the Oct1 CHIIR paper deadline
  • Need to write up the dev plan for GeoMesa into the EIP pattern (include a strawman use case?)
  • Back to learning Spring
  • And here is a way to populate a nested bean:
    <bean id="countryParent" abstract="true" class="org.springframework.chapter2.pojos.EmployeeImpl"><!-- this just gets one abstract class implementation -->
        <property name="country" value="USA"/>
    <bean id="employeeServiceBean"
        <property name="employeeList">
                <bean parent="countryParent" class="org.springframework.chapter2.pojos.EmployeeImpl">
                    <property name="employeeAge" value="10"/>
                    <property name="employeeName" value="Ten Tin"/>
                    <property name="employeeId" value="1"/>
                <bean parent="countryParent" class="org.springframework.chapter2.pojos.EmployeeImpl">
                    <property name="employeeAge" value="11"/>
                    <property name="employeeName" value="Eleven Even"/>
                    <property name="employeeId" value="2"/>
                <bean parent="countryParent" class="org.springframework.chapter2.pojos.EmployeeImpl">
                    <property name="employeeAge" value="12"/>
                    <property name="employeeName" value="Twelve Elve"/>
                    <property name="employeeId" value="3"/>
  • That sends the instanced objects to setEmployeeList that has to take a list of objects. To get it typed, I did this:
    public void setEmployeeList(List<Object> employeeList) {
        for(Object o : employeeList){
            Employee e = (Employee)o;
  • A typed employee is added as follows:
    public void setEmployee(Employee employee) {
  • IntelliJ will extract an interface and rename your class. How cool!
  • Printed, signed and scanned my credit card agreement.

Phil 8.13.17

8:30 – 3:30  BRI

  • Got in at 10:30 last night and am here on time. Kind of surprised. I’ll need to leave a bit early so that I can get shopping and lawn triaged. It rained a LOT while I was gone.
  • getting caught up, updating IDE, etc. It looks like I’m still working on GeoMesaIngest.
  • Since I still need to come up to speed on Spring, and I don’t have tasking, I think I’ll start there
  • Downloaded book
  • installed lamp-server^ and php. Verified that everything is running
  • Got an end-to-end Spring example running with no mysterious incantations. Nice. Now I want to understand testing, but I have to go through a few chapters first…