Author Archives: pgfeldman

Phil 10.17.15

8:00 – 5:00 SR

  • Some progress on dev machine? Submitted (another) ticket.
  • Meeting with Ronda at 3:00.
  • Thinking about weights on the trust matrix. Is adjacency calculated by keyword/entity/host etc., and the weight of that connection based on the trust value? I’m currently thinking that all the support/dispute scores are added together, and used as the weight for (all?) the connections. Then the question is how to build the matrices.
    • Do the urls get connected directly, or do they go through keyword/entity/host etc?
    • Is there a separate matrix for keyword/entity/host/etc?
    • It would probably make sense to try one large heterogeneous matrix and several smaller homogeneous matrices. With and without the intervening non-url item.
  • Finishing up with adding ratings. Done!
  • Need to add ‘Relationship’ that can connect to multiple items. It should be like a rating item that connects to all selected without the rating value. Although ‘relevance’ might be interesting. And allow me to use the ratingPanel html. Done
  • Adding re-centering on the selected items. Discovered I wasn’t adding items to the rootObject. Instead I was adding them to the scene directly. That took a while to fix. Working on centering now..

Phil 10.16.15

8:00 – 5:00 SR

  • Added Ronda as an admin. We need to do something with groups but can’t figure out what/how to do.
  • Fixed Carla’s cert problems
  • Need to add ‘tolower’ on the names/labels of non-url items in the PHP, before the item is stored – TODO
  • Need to check if it’s easy to move the root object to where the selected item(‘s average) position – TODO
  • Need to be able to edit an item – TODO
  • Working on ratings
    • Making a ratingObject to pass between the controller, the rating directive, and the rssFeed directives. Done
    • Wired up the Rating directive to the RssFeed directive.
    • Reworking the rating query.It needs to create the item, get its id_index, then create an association between it and the target that has already been passed in. Done
    • Last thing to do is to enable the query in the rssPull.php and uncomment the submitObj call in RssController.addRatingSubmit. Cool!

Phil 10.15.15

8:00 – 4:00 SR

  • No change on dev admin
  • Need to add ‘tolower’ on the names/labels of non-url items in the PHP, before the item is stored
  • Adding rating directive today.
    • Starting with adding ratings to the data object and data format
    • Added additional shapes so ratings can be distinguished.
    • Killed Eclipse trying to run a PHP test
    • Adding a separate tn_rating to the network is going to be a lot of work. Going to see if I can use tn_items for ratings. They have enough fields, and I have the logic to treat items differently already. Ok, that seems to work. Going to delete the tn_ratings logic. Add rating will be the addition of a RATING item type and an RATING association from the rating item to the target (which can be any item?). Actually, this may turn out better. I can add COMMENT types and other meta information items.
    • Got most of the initial directive layout done and running. Tomorrow we’ll wire it up.
  • Need a show/hide selection capability. All, URLS, ENTITIES, SELECTED, etc.TODO
  • If that gets done I’ve been thinking about the addition of user-specified items. TODO
    • A new item should (must?) have an association with at least one other item.
    • Associations will be made with all selected items
    • Directive dialog to create the new item.
    • In the item list, selected and ‘visible’ items need to be color coded and labeled
    • Items need to be selectable from the item list

Phil 10.14.15

8:00 – 4:00 SR

  • Updated truancy report
  • Seminar today 11:30 – 12:30
  • Automated culling. I think I’m going to start with the cull on the canvas, rather than the server. This way I can still have the items in the list, but not shown on the network. This implies for an item to be visible (and for physics to operate on it) the item should be the source and/or target of one or more associations. DONE!
    • Tried to be fancy by just using the target array but I need a ‘source’ listing of IComponents.
    • This is what I needed to do, basically. The physics only require checks for visibility: 
      var tarray:ITarget[] = this.getTargetArray();
var sarray:IComponentBase[] = this.getSourceArray();
var numLinks = tarray.length + sarray.length;
var makeVisible = false; // or maybe we're also visible when selected?
if(numLinks > 1){
    makeVisible = true;
}
this.setIsVisible(makeVisible);
    • I also added a ‘forceVisibility’ argument to the config object so that items like Queries always show up.
    • And, because it was suddenly easy, I added the list of sources to the item selection code. So now we ‘shift select’ targets and sources, which feels intuitively right: 
      tarray = componentBase.getTargetArray();
for(i = 0; i < tarray.length; ++i){
    targ = tarray[i].target;
    if(!this.isModelSelected(targ)){
        this.selectedModels.push(targ);
        targ.highlight();
    }
}
sarray = componentBase.getSourceArray();
for(i = 0; i < sarray.length; ++i){
    targ = sarray[i];
    if(!this.isModelSelected(targ)){
        this.selectedModels.push(targ);
        targ.highlight();
    }
}
  • Rating. Need a new modal directive. There should also be a way of having additional information about an item, like an annotation. For example, in the query I’m currently working on, “Horse and Jockey”, There was a horse “Tirpitz”, which is also the name of a battleship. There should be a way to resolve this ambiguity.
  • Pagerank. “Links” are determined by shared associations.  TODO
  • Explicit associations TODO
  • Today’s Progress: Weeds

Phil 9.10.15

8:00 – 2:30 SR

  • Taking tomorrow off
  • No progress on dev machine admin
  • Added page rank function to PhilsUtils
  • Discovered that I still needed to pull fresh meta
  • Make it so that a shift-click on a BaseComponent selects all the items that the item is directly connected to. Done.  Bonus points for each shift-click selects an additional hop. TODO
  • Got my first linkage!

DriverF1

  • Next is some automated culling, and then adding rating/pagerank. TODO

Phil 9.9.15

8:00 – 5:00 SR

  • No progress on Dev admin
  • Work on correlation of concepts, keywords, etc. Start with ‘horse’ and ‘rider’?  TODO
  • Work on support/dispute TODO
  • PageRank in PHP TODO
  • Add ‘show/hide unrelated‘. Also, Invisible items should not have physics calculations
  • Add ‘center on selected‘? that would move the center of rotation to be any selected item.
  • Change the CSS so that the style of the buttons is taller It looks funky in some version of FF. Done
  • Clicking on queries in the network display doesn’t cause them to show up in the query window, but CONCEPTS do. Fixed
  • Adding ‘delete network’ button, since the workspace is getting cluttered. Done
  • Ran into a difficult bug to chase down. When I was deleting the last of two items (a query and an url) the url didn’t delete. It turns out that I was using a reference to an array rather than a new array of references. So I was deleting an item from an array while I was still processing the array. Ooops.

Phil 9.8.15

8:00 – 4:00 SR

  • Off yesterday for Labor Day.
  • More dev justification
  • Google Cloud platform java – https://cloud.google.com/appengine/docs/java/. The goal of this would be to calculate overall page ranks
  • Added Save for networks. In reality, this only Creates a network that is then ‘saved live’. TODO: Need to make that more clear
  • Work on correlation of concepts, keywords, etc. Start with ‘horse’ and ‘rider’?  TODO
  • Work on support/dispute 
  • PageRank in PHP 

Phil 9.4.15

8:00 – 5:00 SR

  • Lost my earlier post for today. Odd.
  • Added removeItemAssociations, which is essentially a delete item from the network
  • Fixed a bug that caused the oldes item regardless to be placed at the origin. Now it’s the oldest item of type QUERY
  • TODO: Add the ability to make visible items and their connections based on tn_type
  • Meeting with Ronda at 3:00

Phil 9.3.15

8:00 – 3:00 SR

  • More hoops to jump through for dev machine admin.
  • Training
  • Working with Ronda to get the new servers visible on the lab’s network.
  • Add a test in the NetworkChartInfo.addNetItems() method that checks to see if an item exists in the network but not in the data provider and delete it. Done. Add the appropriate call to the PHP to delete an item and get the new dataProvider back – done
  • Fix ‘+’ sign being added to search terms – done
  • Get the image textures to work – done
  • Get rid of ‘open link here’ and replace with ‘delete’ – done
  • Tried out the app on some lower-end machines and the physics become unstable. Need to add some velocity (and force?) clamping.

Phil 9.2.15

8:00 – 5:00 SR

  • Fixed a project name for Carla
  • Submitted a ticket for admin on the dev machine
  • We can see the new server in http, but not https, which is odd. Told Ronda about that, and and had Lenny send the url to some of the labs to see if it’s visible there as well.
  • Realized that I can use a view for the within network queries, where all the text can be concatenated, and possibly any associations as well? In other words, if the search returns a keyword or concept, all URLs that associate with that keyword can be displayed based on the relevance of the association…? TODO
  • Also, strip off any additional URL cruft? – Done
  • Housekeeping
    • Clear the network – Done
    • Overload the network and make sure that new items and associations don’t get added
      • Added check for items. Need to verify that associations already check. Yup, it’s there already.
  • Adding the pulls for http(s) and queries. Ran into a stupid bug where I was setting network_id with user_id

Phil 9.1.15

8:00 – 5:00 SR

  • No news on my dev box admin.
  • Build the ‘openUrl’ query
    • Send the url to Alchemy for title, Author, text, and image. Take the responses and build the html description that the 
      <item>

<title>Obama to Call for More Icebreakers in Arctic as US Seeks Foothold - New York Times</title>

<link>http://news.google.com/news/url?sa=t&fd=R&ct2=us&usg=AFQjCNEzISdc_7dbACNLX7SfwLZaa1UXBQ&clid=c3a7d30bb8a4878e06b80cf16b898331&cid=52778940195490&ei=hJjlVfCYMcvH3gGGs57IDw&url=http://www.nytimes.com/2015/09/02/us/politics/obama-to-call-for-more-icebreakers-in-arctic-as-us-seeks-foothold.html</link>

<guid isPermaLink="false">tag:news.google.com,2005:cluster=52778940195490</guid>

<category>Top Stories</category>

<pubDate>Tue, 01 Sep 2015 11:15:00 GMT</pubDate>

<description><table border="0" cellpadding="2" cellspacing="7" style="vertical-align:top;"><tr><td width="80" align="center" valign="top"><font style="font-size:85%;font-family:arial,sans-serif"><a href="http://news.google.com/news/url?sa=t&fd=R&ct2=us&usg=AFQjCNEzISdc_7dbACNLX7SfwLZaa1UXBQ&clid=c3a7d30bb8a4878e06b80cf16b898331&cid=52778940195490&ei=hJjlVfCYMcvH3gGGs57IDw&url=http://www.nytimes.com/2015/09/02/us/politics/obama-to-call-for-more-icebreakers-in-arctic-as-us-seeks-foothold.html"><img src="//t0.gstatic.com/images?q=tbn:ANd9GcTb8poiCV7-c6L52plHslVILC4Ti7eb_asJZjOiu7lRPrMAiViJFVC7YHxXBbLyaV2YQthpDI8" alt="" border="1" width="80" height="80"><br><font size="-2">New York Times</font></a></font></div></font></td></tr></table></description>

</item>
    • Obviously, the description part is the trickiest, but it doesn’t have to be as rich in content
    • Tutorial on RSS
    • Encoding and decoding html using PHP
    • In looking at the above, I think the first thing to do is to build the template for displaying the output, then encode it and see if it can be displayed. Here’s an example of a GoogleNews item: 
      <table border="0" cellpadding="2" cellspacing="7" style="vertical-align:top;">

<tr>

<td width="80" align="center" valign="top">

<font style="font-size:85%;font-family:arial,sans-serif">

<a href="http://news.google.com/news/url?sa=t&fd=R&ct2=us&usg=AFQjCNEzISdc_7dbACNLX7SfwLZaa1UXBQ&clid=c3a7d30bb8a4878e06b80cf16b898331&cid=52778940195490&ei=hJjlVfCYMcvH3gGGs57IDw&url=http://www.nytimes.com/2015/09/02/us/politics/obama-to-call-for-more-icebreakers-in-arctic-as-us-seeks-foothold.html">

<img src="//t0.gstatic.com/images?q=tbn:ANd9GcTb8poiCV7-c6L52plHslVILC4Ti7eb_asJZjOiu7lRPrMAiViJFVC7YHxXBbLyaV2YQthpDI8" alt="" border="1" width="80" height="80">

<br>

<font size="-2">New York Times</font>

</a>

</font>

</td></tr></table>
    • Ran into problems with Imagick that seem to be legendary. Rather than dealing with it on the dev box, and since it seems to be there on the main server, I’m using this hacky workaround and just ignoring it for now. What I want is thumbnails so that the asset folder doesn’t get crushed. But that’s not critical right now.
    • Finished building the itemHtml 
      private function constructHtml(){
 $htmlStr = '<table border="0" cellpadding="2" cellspacing="7" style="vertical-align:top;" width="100%"><tr><td width="80" align="center" valign="top">';
 $htmlStr .= '<a href="'.$this->rssLink.'">';
 $htmlStr .= '<img src="assets/'.$this->rssImage.'"height="80"></a></td>';
 $htmlStr .= '<td width="80%" align="left" valign="top"><span style="font-size:85%;font-family:arial,sans-serif">';
 $htmlStr .= '<a href="'.$this->rssLink.'">'.$this->rssTitle.'</a>';
 $htmlStr .= '<p>'.$this->rssDescription.'</p>';
 $htmlStr .= '</span></td></tr></table>';
 $this->rssDescription = $htmlStr;
 }
    • Tomorrow we’ll wire it up so that http and https ‘queries’ get handled from the client

Phil 8.31.15

8:00 – 4:30 SR

  • Status report – Done
  • Timesheets! – Done
  • Get selected items showing up in the feed panel. Done! looks good. Commented out the default addition to the feed. It did make me realize that I need to add a ‘filter network’. Selections should highlight in the GUI and list in the feed.
    • By text search – searching title, description and query
    • By Date range
    • Other?
  • Got the selection working. Query-ish items go in the query box where they are combined into a single query, URL-ish items are listed in the feed section.
  • Get parsing of webpages working so so that they fit into the googleNewsFeed object. This should be just using the Alchemy object with some additional parsing.
  • Deployed to the server, and verified that the page works with Chrome, FF, Edge and IE11

Phil 8.28.15

8:00 – 5:00 SR

  • Commuting in the reverse order. Don’t forget to pick up energy drink mix too!
  • 3:00 meeting with Ronda to work on the servers
  • Adding isAssociation() and isItem() methods to the rssFeed directive. Done. Will probably add isRating() and isTouch() once they start getting used.
  • Deleting ComponentNetwork class from rssController – Done
  • Deleting pendingQueryArray and queryHistory from rSSController. The history can be pulled from the items in the DataProvider. Which works.
    • Add some test QUERY types to validate – Done – replaced the ENTITIES in tn_items. Worked with no other changes.
  • Setting items if their associated queries are clicked and setting the query if a query item in the chart is clicked. Done!
  • Deploying new version to the server.
  • And I am done for the day. Next week needs the following
    • Add items, which will (should?) return the whole data provider. Only differences (additions and deletions) should cause an effect.
    • Have selected items (only?) show up in the list.
    • Have the ‘query’ task change it’s behavior based on what’s sitting in the field. If it’s an url, open it in a new tab.If it’s text (can be multiple queries) combine all the text into a new query and run it.
    • Continue to flesh out the types.
    • Save, Save As, reload (clear and load) models. Verify! Test!

Phil 8.27.15

7:30 – 5:00 SR

    • Ranted about JavaScript’s design (evolution?) flaws here.
    • Adding items to the environment
      • Need to compare the data provider to see what’s in the model but not in the DP and delete it
      • Need to change the data format to include all the tn_types – started
      • Link the pulled-down data to the chart data. Going to try just assigning pointers first.
      • Hey! It loaded from the DB and rendered! Need to anchor the network root. Done. For the time being I’m using the lowest id_index, which will equate to the earliest entry.

loadedNetwork

      • Need to verify that all the connections are good too.
      • Attached the data object.
      • The view that produced the query results was wrong. Fixed now:

loadedNetwork

  • Up on the server and tested in all browsers. Had a heart stoppage when the tn_view_network_items didn’t get added. But it’s all good now.  For tomorrow, try adding items and modifying the network structure with only the new items.

Phil 8.26.15

8:00 – 3:00 SR

  • Can still log in and do things. No progress on the dev box.
  • Make sure that the default network is selected in the initial flyout panel logic. Done
  • Back to building the data provider object. Thought about doing it on the server side, but doing it on the client 1) allows the formatting of the data from the server to remain constant as I muck with the client and 2) I can examine each item and throw away the ones that have not changed WRT the display. At least that’s my thoughts for now.
  • Got the items done, working on the associations. Added a JSON.stringify() to build a version that I can check outside the debugger.
  • Associations are doneand it looks good in JSONLint: 
    {
    "type": "XYZ",
    "bounds": {
        "xmin": -5,
        "xmax": 5,
        "ymin": 0,
        "ymax": 5,
        "zmin": -5,
        "zmax": 5
    },
    "data3D": {
        "URL": {
            "URL_1": {
                "data": {
                    "type": "URL",
                    "idIndex": 1,
                    "guid": "my_url_guid_1",
                    "description": "my url 1",
                    "title": "url 1",
                    "link": "http://url",
                    "image": "rock-512.jpg"
                }
            },
            "URL_2": {
                "data": {
                    "type": "URL",
                    "idIndex": 2,
                    "guid": "my_url_guid_2",
                    "description": "my url 2",
                    "title": "url 2",
                    "link": "http://url",
                    "image": "rock-512.jpg"
                }
            },
            "URL_3": {
                "data": {
                    "type": "URL",
                    "idIndex": 3,
                    "guid": "my_url_guid_3",
                    "description": "my url 3",
                    "title": "url 3",
                    "link": "http://url",
                    "image": "rock-512.jpg"
                }
            },
            "URL_4": {
                "data": {
                    "type": "URL",
                    "idIndex": 4,
                    "guid": "my_url_guid_4",
                    "description": "my url 4",
                    "title": "url 4",
                    "link": "http://url",
                    "image": "rock-512.jpg"
                }
            }
        },
        "ENTITIES": {
            "ENTITIES_5": {
                "data": {
                    "type": "ENTITIES",
                    "idIndex": 5,
                    "guid": "my_entity_guid_5",
                    "description": "my entity 1",
                    "title": "entity 1",
                    "link": "no link for entity",
                    "image": "rock-512.jpg"
                }
            },
            "ENTITIES_6": {
                "data": {
                    "type": "ENTITIES",
                    "idIndex": 6,
                    "guid": "my_entity_guid_6",
                    "description": "my entity 2",
                    "title": "entity 2",
                    "link": "no link for entity",
                    "image": "rock-512.jpg"
                }
            },
            "ENTITIES_7": {
                "data": {
                    "type": "ENTITIES",
                    "idIndex": 7,
                    "guid": "my_entity_guid_7",
                    "description": "my entity 3",
                    "title": "entity 3",
                    "link": "no link for entity",
                    "image": "rock-512.jpg"
                }
            },
            "ENTITIES_8": {
                "data": {
                    "type": "ENTITIES",
                    "idIndex": 8,
                    "guid": "my_entity_guid_8",
                    "description": "my entity 4",
                    "title": "entity 4",
                    "link": "no link for entity",
                    "image": "rock-512.jpg"
                }
            }
        }
    },
    "network": {
        "edges": [
            {
                "source": "URL.URL_1",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_1",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_2",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_2",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_3",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_3",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_4",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            },
            {
                "source": "URL.URL_4",
                "target": "ENTITIES.ENTITIES_5",
                "val": 1,
                "type": "TRUSTWORTHINESS"
            }
        ]
    }
}
  • Will test and try to build networks tomorrow. That will require getting all the types set up, and then some integration of the data items and the rssFeed display list.