Category Archives: Development

Phil 8.6.15

8:00 – 4:00 SR

  • Still no admin
  • Check out Debra Fisher on LinkedIn
  • Finish rewriting login
  • Start on DataProvider – nope, spent the day on getting login to work right. Sheesh. At least I added Load/Save SearchNet buttons under the ‘history’ panel.

Phil 8.5.15

8:00 – 4:00 SR

  • Still no admin.
  • Need to add checks for http(s):// in the freeform text. If it’s there, open the url, if it’s not then send to GoogleNews
  • I think the best way to get the results from the network queries is to take the raw rows from the server and parse them into graphical network data. I will need to add variables for id_index and type (table at least, maybe more)
  • Query for all items on a network with labels synthesized from id_index and type name: 
    -- get all the items in a network
select ti.id_index, ti.guid, ti.item_type, ty.name as type_name, CONCAT(ty.name, '_', ti.id_index) as label, ti.text, ti.float_val, ti.link, ti.image
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_items ti on a.target_id = ti.id_index
inner join tn_types ty on ty.id_index = ti.item_type
where dat.network_id = 1
union
select si.id_index, si.guid, si.item_type, ty.name as type_name, CONCAT(ty.name, '_', si.id_index) as label, si.text, si.float_val, si.link, si.image
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_items si on a.source_id = si.id_index
inner join tn_types ty on ty.id_index = si.item_type
where dat.network_id = 1;
  • Query for all associations on the same network 
    -- get all the associations in a network
select a.id_index, u.login, a.user_id, si.guid as source_guid, a.source_id, ti.guid as target_guid, a.target_id, at.name as assoc_name, a.assoc_type, a.created_on
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_types at on a.assoc_type = at.id_index
inner join tn_items si on a.source_id = si.id_index
inner join tn_items ti on a.target_id = ti.id_index
inner join tn_users u on a.user_id=u.id_index
where dat.network_id = 1;
  • Ok, let’s see if we can pull this down and parse it!
  • Got into some kind of issue trying to use my old (pre ATSBase) login directives. Rebuilding.

Phil 8.4.15

8:00 – 4:30 SR

  • Still can’t get on the server.
  • Steve’s found a potential SW engineer. I’ve asked for her resume.
  • Cleaning up the networkDbIo.php code. I’ve also realized that ratings can have ratings. Changed tn_ratings to have a rating_type and a target_type.
  • Finished with addAssociation. It’s like item in that if one exists already we don’t want to create an exact copy with a new date.
  • Building the dataprovider object which requires a list of unique items, pulled from associations. So how do you get distinct items from two columns in mySql? StackOverflow has the answer. In my case, it’s: 
    select ti.id_index, ti.guid, ti.item_type, ti.text, ti.float_val, ti.link, ti.image
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_items ti on a.target_id = ti.id_index
where dat.network_id = 1
union
select si.id_index, si.guid, si.item_type, si.text, si.float_val, si.link, si.image
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_items si on a.source_id = si.id_index
where dat.network_id = 1;
  • Ok, that part worked. Now working on building the rest of the data provider. Below is the current structure. Not quite sure how to convert the rows from the database pull to the data3D. This might be multiple queries. The edges, on the other hand are more straightforward. Might need a (unique within the type?) label for the tn_items though. 
    this.dataProvider = {
   type:'XYZ',
   bounds:{xmin: -5, xmax: 5, ymin: 0, ymax: 5, zmin: -5, zmax: 5},
   data3D:{
      Speakers:{
         story_0:{xpos:5, ypos:5, zpos: 0, mass:0.9811469360199058, imageLoc: "./assets/checkerboard.jpg", notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_1:{mass:1.1514476240300544, notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_2:{mass:0.7468225934308907, notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_3:{mass:1.2358769332362016, notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_4:{mass:1.1071240443456865,  notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."}
      },
      Subjects:{
         story_0:{xpos:-5, ypos:5, zpos: 0, mass:1.2374208338984265,  imageLoc: "./assets/checkerboard.jpg", notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_1:{mass:0.4781579303734177,   notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_2:{mass:0.32479382813444624, notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_3:{mass:0.9931577182910525,  notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_4:{mass:1.1995832704567262,  notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."}
      },
      Platforms:{
         story_0:{xpos:5, ypos:-5, zpos: 0, mass:0.2805132286382044, notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_1:{mass:0.9416553793139485, notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_2:{mass:1.10169089403032,  notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_3:{mass:0.3067471426015206,  notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."},
         story_4:{mass:0.6582850955300814,  notes:"Ut erat ante, varius ac elementum eget, cursus pulvinar purus. Nulla venenatis nec odio ut scelerisque. Donec hendrerit porta mauris."}
      }},
   network: {
      edges:[{source:"Speakers.story_1", target:"Speakers.story_2", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_2", target:"Platforms.story_0", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_3", target:"Subjects.story_2", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_3", target:"Speakers.story_1", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_4", target:"Subjects.story_1", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_4", target:"Speakers.story_3", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_4", target:"Speakers.story_3", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Speakers.story_4", target:"Speakers.story_2", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Subjects.story_0", target:"Subjects.story_3", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Subjects.story_2", target:"Subjects.story_0", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Subjects.story_2", target:"Subjects.story_3", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Subjects.story_3", target:"Speakers.story_4", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Platforms.story_0", target:"Platforms.story_4", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Platforms.story_2", target:"Platforms.story_0", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Platforms.story_2", target:"Speakers.story_0", val:1, type:Wgl3dCharts.DEFAULT},
         //{source:"Platforms.story_3", target:"Speakers.story_1", val:1, type:Wgl3dCharts.DEFAULT},
         //{source:"Platforms.story_4", target:"Speakers.story_1", val:1, type:Wgl3dCharts.DEFAULT},
         //{source:"Platforms.story_4", target:"Speakers.story_4", val:1, type:Wgl3dCharts.DEFAULT},
         //{source:"Platforms.story_4", target:"Subjects.story_2", val:1, type:Wgl3dCharts.DEFAULT},
         {source:"Platforms.story_4", target:"Platforms.story_1", val:1, type:Wgl3dCharts.DEFAULT}
      ]
   }
};

Phil 7.31.15

7:30 – 2:30 SR

  • AC service today, so I’m working from home.
  • Got server code running at home, now testing on production server.
  • Had to play around with config files, but everything is working now.
  • Working out a way to identify an item for easy searching when there is no (Google) guid. Trying an sha1 hash of the elements used to make the item.
  • Finished addItem()
  • Finished getItem()
  • Finished addItemRating()
  • Finished getItemRatings()

Phil 7.30.15

8:00 – 4:30 SR

  • Still working on getting software for the new dev machine.
  • Added tn_ratings and tn_touches to handle historical behavior. I realized that items shouldn’t have histories in a relational db. Histories should point at items. Sideways thinking.
  • Setting up base and subclasses for the DbIO.
    • baseDbIo
      • improved fail returns
    • userDbIo
    • networkDbIo
      • getUserNetworks
  • Working on item find/add/change/delete PHP functions

Phil 7.28.15

8:00 – 5:00 SR

  • Fixing charts – everything done but TST. Did we do that one by hand?
  • Had a thought that the entire financial system could be represented as a network, which could be made to fit into the framework I’m trying to develop. It might let me collapse the project into something small enough for one person to manage, since the database would shrink to only a few (5?) tables…
  • Ordering Software – Visual Studio and IDEA Ultimate
  • A Visual Introduction to Machine Learning
  • Starting to build the server classes that will access the network data
    • A new component doesn’t have have an id_index, so we will know when to create and when to update. Deletion may be tricky, since multiple networks may share an item. We may just want to keep deleted items around anyway, since the fact that they were once attached might mean something…?
    • Class is up and doing raw calls to the database, which, of course is dangerous as heck. Working through how to do bound parameters. Staring with the addUser(), checkUser() and changePassword() methods, since they’re needed and unlikely to change.
    • Discovered the very nice PHP Data Objects (PDO) and this helpful tutorial. I have the checkUser() and rawSqlQuery() methods converted. Need to add some error checking, but very happy.

Phil 7.27.15

8:00 – 2:30 SR

  • I’m thinking about how the edges in a network can have certain characteristics
    • Bandwidth – the capacity of the edge. Can be expressed as used and potential
    • Frequency – how often the edge is used. Bandwidth provides a ceiling on this
    • Richness – I’m not sure how to think about this. In the most basic case, this could be an expression of information content (i.e. an infrequent transmission with lots of content is equivalent to a frequent transmission with low content). But when we layer on context and meaning, all kinds of additional information gets ‘compressed’ into the message – a glance can be nothing or everything. Ideally, the description of the edge should contain some way of accessing that context, and shouldn’t be considered simply a link
  • Along this lines, I found this paper: Lexical chains as representations of context for the detection and correction of malapropisms. Lexical chains are certainly one way of representing context.
  • Another way of thinking about richness is in terms of similarity. If the link is carrying the same payload over and over again, then there isn’t much richness. So can we look at similarity as a way of determining how rich a link is? An Information-Theoretic Definition of Similarity. And here’s looking at news headlines, which might be relevant to short posts or tweets: Similarity for news recommender systems
  • And this reminded me of Princeton’s WordNet, which could be really helpful.
  • Ok, back to the database
    • Based on the above thoughts, I’m adding assoc_type to tn_associations
    • Here’s a query that pulls all the parts together. According to everything I’ve read, joins are probably the most efficient way to do this (size and speed): 
      select u.login as `Link Created By`, a.created_on as `Created On`, at.name as `Assoc Type`, si.text as Source, st.name as `Source Type`, ti.text as Target, tt.name as `Target Type`
from  tn_associations a
inner join tn_users u on a.user_id=u.uid
inner join tn_types at on a.assoc_type = at.uid
inner join tn_items si on a.source_id = si.uid
inner join tn_types st on si.item_type = st.uid
inner join tn_items ti on a.target_id = ti.uid
inner join tn_types tt on ti.item_type = tt.uid;
    • Now I need to load and save a named network from a particular user or any variants up to and including all networks from all users, without redundant nodes/edges…

Phil 7.22.25

8:00 – 5:00 SR

  • Filled out contact info for Steve
  • The stricter trust chain did not work. I had Ronda go back to the looser one.
  • Fixed attraction/repulsion/linkScalar it’s 1.0, 10.0, 10.0 for defaults
  • Ran into a weirdness with <input type=”range> and browsers. Chrome is fine. FF and Chrome have differing default widths and margin/padding. Had to add the following to get all ranges to work similarly: 
    .forceRange{
    position: absolute;
    width: 120px;
    right: 10px;
    margin: 0px;
    padding: 5px;
    z-index: 2;
}
  • Back to figuring out the AlchemyNews API. Blew up the limit again, being careful. I think that the news API, aside from behaving poorly (things like ‘&q.enriched.url.concepts.concept.relevance=0.9’ don’t work). Think I’m going to add more user interaction and less machine learning. Store it all for later page ranking?

Phil 7.21.15

8:00 – 4:30 SR

  • Server is behaving with the stricter trustchain.
  • This is the AlchemyNews REST API Documentation, and the list of fields that can be returned. And Twitter access, BTW.
  • Blew through my limits for the day trying to figure out keywords. Asking for academic license.
  • A good example of how to group query elements: http://alchemyapi.readme.io/docs/sentiment-analysis
  • Need to add sliders for attraction and repulsion (network scalars?). Implemented. Now I need to figure out some good values. I think that we might just have to scale attraction for linked items. It should clean things up and cut down the math a bit.

Phil 7.17.15

8:00 – 2:00 SR

  • The production server is behaving again!
  • Wrote up some thoughts on TypeScript interfaces and fat arrow notation.
  • Working out more subtle network linkages. An item is current (selected?) has potentially several other items that will connect to it. Selecting should also bring up the search results associated with that item (create an itemlist of nearest neighbors). Maybe the item should be created before the results of the query come back. That might make things more modal.

Phil 7.16.15

8:00 – 4:30 SR

  • The production server isn’t visible through the browser, though the test server is. It first I thought it was pre-populated keychains, but when the servers are set up identically, the production server still doesn’t cooperate. Ronda’s putting in a ticket.
  • Helped Ronda generate a PKI request for the new dev server, she’s going to try to make it work on the other two.
  • Looking at Dong’s code, I cannot figure out why he needs fat arrow. His: 
    public promiseCaller():void{
   this.promise = this.service.getQueries();
   this.promise.then(this.processData, this.errorData);
}

public processData = (data:any):void => {
   console.log("got data");
};

public errorData = (data:any):void => {
   alert("error getting data");
};
  • And mine 
    private goodUserQuery (response:any) {console.log("got data");}
private goodUserQuery (response:any) {alert("error getting data");}
public promiseCaller():void{
    this.queryService.submit(qstr, this.goodUserQuery, this.errorResponse);
}
  • Which calls (in the service): 
    public submit(query:string, goodResponse:any, errorResponse:any):any {
   return this.httpService(query).then(goodResponse, errorResponse);
}
  • And it looks like Google won’t let me use their images as textures. Working at downloading on the server side and storing in a local directory, using PHP. That works just fine! And to keep from downloading the same pix all the time, added a check as well. PHP has pretty much everything you’d ever want to do stuff on the web. But no typing. Must check out Hack one of these days.
  • Adding connections. Finally! Simple conditions are working and up.

Phil 7.15.15

8:00 – 4:00SR

  • Prime day. May poke at that a bit. Disappointing.
  • Call Porsche for oil change and chin cover repair – nope. Tomorrow?
  • Get network connections working – Almost!
  • Add news API integration – nope

Phil 7/14/15

7:30 – 5:00

  • Write up a description of Dong’s skillset.
  • Call Arhouse.
  • Continue on ComponentNetwork
    • Chain -> History
    • ‘Relevance to Search’ dialog for keywords, etc.
    • Accumulate search terms in some way. Possibly weight them?
    • Rebuilding the network construction so that it’s handled by its own class. The basics are working, need to add connections based on pending requests next. Overall it’s better than using addEnvironment, which really wasn’t all that appropriate. It should be easy to add in textures too. Fingers crossed.
  • Looks like my workstation should be in next week
  • Pluto flyby!

Phil 7.10.15

8:00 – 4:00 SR

  1. Sent Steve F. some background on Visibility and equipment requests
  2. Continue working on integrating dynamic data provider into charts. First pass is done, but kind of crappy. I think there needs to be a recursive way of adding the data. Something like an tree object that recurses through itself to create the data provider which is then rendered. Ponder it over the weekend.
  3. Need to make a self-contained example of the flexbox/ng-repeat interaction.
  4. Redid my tuition reimbursement because Novetta changed it after I submitted it before.
  5. Uploaded the current app and it still works!