Category Archives: Development

Phil 10.23.15

8:00 – 4:00 SR

  • Things look good for the switchover
  • Getting the PHP right for items from text
    • Fixed the author parsing
    • Fixed image save
    • Fixed guid generation for manual add. Alchemy likes my resume. I wonder what makes that more parse-able?
    • Will need to generate some default values for the items used to generate the guid.
  • Add some new tn_types for tn_items so we know what the source is? Ether that or add a ‘source’ field….
    • ANLP_
    • USER_
    • GNEWS_
    • etc
  • 3:00 switching production servers.
    • Looks like everything works.

Phil 10-21-2015

8:00 – 6:00 SR

  • Well, we have hoverboards, after a fashion.
  • Discussed the new approach in some detail with Lenny. It turns out that the current system has around 140 tables and about 15 views spread across 5 databases. So getting it down to one DB with 5 tables makes some sense.
  • Adding direct text interaction to the AlchempNLP code – Looking for a (good?) abstract
    • So here’s kind of an interesting thing. Google has my citation history, and uses that to make recommendations. But since my previous work is surgical simulator heavy, I’m not getting results that are useful to me. I want to be able to adjust weights! I think I’ll call this ‘the self driving car problem’ 🙂
    • Refactoring alchemyNLP pull into a base class and then an URL version and a text version
  • Meeting on campus with Wayne, Shimei and Victor.
    • Since by next week, something should be working, Wayne would like to use the system to try and find mentors. Sheimei suggests using the dblp as a source. It’s structured and can be downloaded
    • I realized that items should(?) have their own(?) networks of items that are attached at a system level. For example, when AlchemyNLP parses a web page, it produces concepts, entities, author etc. Rather than re-running the analysis each time, the system should look to see if there is already an <itemGuid_Alchemy> network produced the first time the item was ingested. Then those items can be attached to the user network as needed.
    • Talked with Victor about collaborating somehow. We also talked about how to present a list of search results in a ‘curated’ context. It could be a sharded display where each item listed has a slider/add/discard control, and hovering over the item shows it’s relationship to the curated collection on the right hand side of the page. Choosing the context could be in a dropdown at the top that is guessed at by context, but can be changed by the user.

Phil 10.20.15

8:00 – 4:00 SR

  • Need to change the QueryHistory list to Visible Ranks. Done
  • Google custom search, and pricing
  • Started on ‘manual’ item. Need to add an IDENTITY association so that items that are connected will still show up in the dataObject query. Added to tn_types. Will need to work on the queries;
  • Added most of the Javascript guts to send the manual item to the PHP. Need to do that side next. And don’t forget to add an IDENTITY link on the PHP or the item won’t come back.
    • Any time a link is added, delete IDENTITY links if the IDENTITY item is the source or target..?
  • Need to make it so that all associations have adjustable weights. It probably should be based on the item that is the source, but maybe independent ‘advanced’ manipulation should be possible?

Phil 10.19.15

8:00 – 12:00 SR

  • Add  the check in the flyout directive that looks at the user_id and read_only coming back. If the user_id is not the current user_id from the session object, force the network to be read_only regardless of its listing in the db
    • Populated network 3 for user_id = 3. Done, and verified on the server.
  • Need to change the QueryHistory list to Visible Ranks.
  • Try running a pdf through alchemy. It returns a ‘non-html’ error. However, I can parse the pdf in PHP and then send the text off to Alchemy for analysis
    • I think though, that the way to do this is to add a ‘manual’ item. This would add the following fields(?):
      • Text_content – cut and paste of the text that matters
      • Authors (comma separated authors – add validation and parsing)
      • The rest of the items could be the RSS2.0 spec, which makes sense anyway.
      • This would require a new button and a new directive. On ‘save, the text_content gets sent to alchemy for the creation of the keyword(?) network. The other items (title, author, etc) get added to the network explicitly.
      • This does mean that when when an item is added to the network, that there are other ‘items’ lie author that should be attached automatically.

Phil 10.16.15

8:00 – 4:00 SR

  • I have my access back!
  • More justification for dev machine
  • Updated truancy reports.
  • Add ‘read only’ to network and use it to disable buttons on the GUI. It should have a checkbox on the flyout next to private.
    • Added. Had to add a read_only field to tn_networks.
    • Next, make the various buttons that cause writes to the DB to be conditional.Done
    • Updated the public server.
    • Changed findNetworks() so $queryString .= “select * from tn_networks where user_id = :user_id or is_private = 0”;. Now I need to verify that I’ve reconciled in the Flyout Directive. Need to make a new network for that. Monday.

Phil 10.15.15

8:00 – 5:00 SR

  • Still don’t have my access back
  • But, was able to update the needed table for FR using the scripting system. So that’s pretty cool.
  • Looking to migrate over to the new servers next Friday.
  • Adding behaviors to support the changes I made to the GUI.
    • Link to wampeter
      • GUI – Done
      • DB – Done. Changed the way that linkSelected worked so that if a sourceID was passed along a wampeter was not created.
    • Update wampter value change DB (means updating association weights)– Done
    • Update Rating value change DB – Done
  • Taadaa! ItAllWorks

Phil 10.14.15

8:00 – 5:00 SR

  • Got an error message after trying to explore an URL (http://www.sportsgrid.com/mlb/back-to-the-future-part-ii-predicted-cubs-as-2015-world-series-champions/):
    • Error: itemTableArray is undefined
      RssController</RssController.prototype.parseDataObject@http://philfeldman.com/WglCharts2/controllers/rssControllerTS.js:195:29
    • Need to flag that so that it shows up as an error message in the feed rather than having the ‘waiting’ message appear
    • Really need to re-weight the Alchemy items. Changed to 0.5 for now. Still A hack
    • Adding behaviors to support the changes I made to the GUI.
      • Unlink (from Wampeter)
        • GUI – done. Lots of oddness with the <select> directive.
          • Fixed the blank first selection by setting the initial values in the guiVars object (I’m using to keep things clean) to null.
          • The ng-model is being set to a JSON string. So I had to get the object pointer from the list used to populate the select. In this case I used the guid. What a hack!
        • DB – Done. Had to add a addDirectedAssociation()
      • Link to wampeter
        • GUI – Done
        • DB – TODO
      • Update wampter value change DB (means updating association weights)– TODO
      • Update Rating value change DB – TODO

Phil 10.13.15

8:00 – 5:00 SR

  • Sent the (hopefully) fixed FA and RA to Bill
  • Think I got my accesses fixed. Again
  • Adding behaviors to support the changes I made to the GUI yesterday.
    • Unlink (from Wampeter)
      • GUI – done. Lots of oddness with the <select> directive.
        • Fixed the blank first selection by setting the initial values in the guiVars object (I’m using to keep things clean) to null.
        • The ng-model is being set to a JSON string. So I had to get the object pointer from the list used to populate the select. In this case I used the guid. What a hack!
      • DB – TODO
    • Link to wampeter
      • GUI – TODO
      • DB – TODO
    • Update wampter DB – TODO
    • Update Rating DB – TODO

Phil 10.12.15

1:30 – 5:30 SR

  • Adding data objects to ITarget – done
  • Adding test in parseDataObject and addNetItems to see if links aren’t loading sometimes. Done
  • Adding a check to see prevent (non-URL, non-Query) items that only have one link aren’t added to pagerank calculation and listing. Done.
  • Adding an ‘add selected’ to the Wampeter feed directive – button added TODO: add functionality
  • Adding unlink from: <wampeter name> to any icon that is, well, linked to a wampeter. Not that this may need to be an ng-repeat. – button added TODO: add functionality
  • May switch over to keywords only. Concepts and entities overlap a lot. Also thinking that Adding the url could do that without exploring. Just open and make sure that it’s not 404? And just running with that thought, Explore could return a ‘Cliff Notes’ Dialog that could have the capability to link to other items? Or just select what gets attached? And merged?…

Phil 9.10.15

8:00 – 4:00 SR

  • Lost my accounts on all the servers. Asked Ronda to look into it.
  • And since I can’t get on the server, asked Ronda to fix the setupModel.xml files to fix login issues.
  • Sent Dong a note to see what’s going on. He’ll take a look this weekend.
  • Some of my demos that use the WebGl3dCharts module were broken. The reason is that I was looking for items in the optional data objects and throwing a null. Two lessons:
    • FF console does not log this error
    • Just because you have static type checking in TypeScript doesn’t mean that you shouldn’t use hasOwnProperty() to check if an item is present. Here’s the fix: 
      var forceVisible:boolean = false;
if(itemData.hasOwnProperty('data') && itemData.data.hasOwnProperty('type')) {
    forceVisible = this.getForceVisible(itemData.data.type);
}

      So now it’s much more robust.

    • Adding ability to modify text. Done. Changing the title also ripples through the display. I also don’t (re)calculate page rank if the weight hasn’t changed.  Also added 
      findModelFromDataObject()

      for WebGlCanvas.

  • Found these guys (http://www.intensityanalytics.com/), who do keyboard biometrics. Trying to contact them.

Phil 10.8.15

8:00 – 4:00 SR

  • Noticed yesterday that the time it takes for the map to center is just too long. When I tried increasing the speed I got a lot of jitter because I was testing for proximity to goal. I wound up changing so that the goal is reached when the distance to the goal is increasing.
  • Add editing of Rating and linkSelected
  • Make sure that items with only one link that are not ‘force visible’ are not shown in lists
  • Bold selected items in lists
  • Change the query list to be the ‘visible’ list
  • Trim the length of the list items so the containing box sizes don’t go crazy.
  • Look for floating panel CSS?

Phil 10.7.15

8:00 – 6:00 SR

  • Gina’s still having problems logging on
  • Write up the DB and programming test for Lenny, since the next interview(?) won’t have me along
  • Banged away at PageRank and finally have it working. The matricies were evaporating, so I wound up layering inbound and outbound links over an identity matrix. That worked great. I need to think about why though.
  • Meeting with Dr. Pan and Dr. Lutters at 4:00(?). Went well.
    • One thing that came up in the discussion was how to feed in enough information to make the analytics work. Dr. Pan suggested using Politifact, since it’s well formatted ground truth that references a source. In essence, I could build a small java program that could iterate over all the PF reports and build a network for each of the Entities (or concepts) referenced. Once nice side effect is that this could be a ‘seed set’ of vetted data that could be used to inform other searches.
    • The other item discussed is whether this is what kind of view this is. Dr. Lutters suggested that this might be the analyst’s (development) view, while a consumer’s view would be more like a traditional  news feed, like inkl.com

Phil 10.6.15

8:00 – 5:00 SR

  • The FA and RA apps are not working right with 2016 data. Pinged Dong.He says he can take a look tonight.
  • After installing Win10, I had a heart stopping moment where Apache wasn’t starting, with an error message of “Problem detected! Port 80 in use by “Unable to open process” with PID 4!” Stack Overflow suggested that this was the “World Wide Web Publishing Service” and you have to stop it. Here’s a screenshot: Services
  • Other than that, Win10 seems just fine. I love the menu bar for each monitor.
  • Figuring out how to add the network to the Ranking code…
    • Will need to sort the array based on the ranking. Doesn’t look like a problem.
    • Added weight to ITarget
    • Reworked Rating.ts so that everything is initialized to a square matrix of zeros. You now have to create the Rankings.PageRank class (appropriately sized), then populate it, then calculate and get results.
    • Construction of the matrix from the network is nearly done. Finish tomorrow.

Phil 10.5.15

  1. 8:00 – 4:00 SR
  • Change the divisor in the PageRank class to be a scalar.
  • Change the ‘History’ list to show the ranked items. Either all, or individually. (tab view)?
  • Add sliders to the rating view in the feed. Will also need a save with associated PHP code
  • Add update rating PHP
  • Add update linkSelected PHP
  • Show the changes in the list of items as the sliders are adjusted.
  • add an ‘annotation’ field to the rating? Better tracking?

Phil 10.2.15

8:00 – 5:00 SR

  • So far today, the roof is leaking, I’ve lost my badge, and the espresso machine is fried. Perfect Friday.
  • Still wondering where to calculate pagerank. Here’s the algorithm in JavaScript: https://github.com/stevemacn/PageRank/blob/master/lib/pagerank.js
  • Looks like there isn’t a TypeScript version. I think I’ll start with the client version. That should allow for more interactivity. Maybe just calculate for all users on the server?
  • Implemented a PageRank algorithm from first principals. First, the versions that are out there are buggy. Second, because they use variable length arrays of links, you can’t put weights. The downside is that there is a performance hit, but on small networks that shouldn’t(?) be a problem.
  • One easy performance mod is to change the normalization process to first calculate the fraction, then create a multiplier of  1/that, so we’re multiplying rather than dividing.