Author Archives: pgfeldman

Phil 8.25.15

8:00 – 5:00 SR

Zeno’s admin saga continues…. Hey, I got remote desktop running! Ok, on to getting admin on my dev machine.
Meeting with Ronda on Friday to work on provisioning and testing servers.
Realized that since putting in the responses to a query, that I need associations that are COMPUTED, EXPLICIT, and DEPRECATED. In other words, the query results in 10 URLS. But the user only selects 3 now and 2 later. Only those urls and their connections are explicit in the network. Ones that I don’t want to show I deprecate. They can still be there (and probably should be? so the different networks can be compared)
This means that rather than adding the results of a search, we cull. This may actually work well, since in brings a different (less is more) mindset.
Working on putting the database items in the network, which means making a dataprovider. I think the easiest way to not have duplications is to have the Wgl3dCharts.NetworkChartInfo class manage that?

Phil 8.24.15

8:00 – 3:00 SR

Still working the Zeno’s Administrator problem.
Had a good discussion with Lenny about Low Threshold/High Ceiling coding and also about functional vs declarative programming and how ExtJS and AngularJS relate to those concepts.
Adding items to the live network and then bring down and parse the data object. This means pulling out the code that loads before the DB call.
Added queries and associations to any query that is not “__topNews__”

Phil 8.21.15

8:00 – 4:30 SR

Still no admin, but we’re at the last step. I think.
Meeting with the GMO folks. They have an ExtJS framework (YUI-based!?) for all their user IO.
We will *not* be abandoning Angular:
Long discussion with Mark and Lenny about next steps. Maybe integrate all the efforts? I made the point that what we have is an information navigation system, but the info is questionable. Wheat we need is a data/inference system
More housekeeping for network logging in. Cleaned up the add network code on the client and server. Everything seems to be working as it’s supposed to. Next step will be to add items to the live network and then bring down and parse the data object.
I also realized that the best way to show the items is to have the master list populated by sublists. So the status item would be loaded first, then the selected items, then the search results. Then the various sublists can be managed separately and the other lists won’t care.

Phil 8.20.15

8:30 – 4:30 SR

Working on getting admin on the servers. Then my dev box.
Only clear the data item list when a new query is submitted. Adding an item to the network should show up in the network. Any selected items appear at the head of the itemList, above the previous search results. This way, more than one item can be added from the same search.
Items are added to the network only from the response from the server. The query should also send up a list of item ids and association ids that it already has, so only new items are returned. Ok, that’s mostly done, though at this point the full network is returned.
Network file operations need
- A default state (id == -1?). This is an interesting problem. Since the system needs the server and the DB to build the connections in the network, we have to have a provisional network. I think I’m going to do this by creating networks where the ‘new’ network is created as ‘provisional’. If the user exits without saving and then starts a new session, any prior provisional network is deleted.
- Merge With – adds the current network to the selected network and then downloads the while thing
- Load – deletes anything current
- Save – save as current
I think I have this now working by saving the network the moment that the app comes up as not archive. The id is then used until another network is loaded (then all not archive are deleted for the user anyway. If the user explicitly saves the new network, then it’s set to archive.

Phil 8.19.15

8:00 – 5:00 SR

Admin has been approved. Waiting for my logins to re-activate
Friday, 9:00-ish meeting with GAO-ish folks to talk about ingest.
Continue working on storing items and associations.
These people integrate annotations into the article, which is deeper. But in the reading of an article, this seems overwhelming
Bayes Server – worth getting? John G. Sandiford’s thesis.
Adding a UID to associations, since they need to be unique and findable. Done, and it’s for within a network.
Added a HOST type so that the root of the domain can be stored as an attribute as well. Of course, there was a PHP call to do this.
Tested creation of items and associations against this Miami Herald article.
Need to make sure that NLP items don’t get stored if they don’t return right.

Phil 8.18.15

8:00 – 5:00 SR

Still no admin
I forgot to save the new SQL for the truancy reports. Fixed now.
There is something that’s not quite right in the network physics. When an object is added, sometimes the forces get asymmetric. My guess is that the velocities have to be made non-zero each time an item is added.
Was getting a problem where additional information was getting stuck on the end of the image url, which was contaminating the name. Now, rather than assuming that the image name ends with the extension, I look within the image url for a reasonable extension and return that.
Need a query that returns the associations shared between two items. Done
Working on adding Alchemy NLP results as items with associations

Phil 8.17.15

8:00 – 3:00 SR

Still no admin

Given that I’m using FizzBuzz for the coding side of the SWE interview, I though I should put together a test for the database portion of the interview:

Create three tables:
- One for names and salaries,
- One for roles,
- One that assigns a particular person to one or more roles.
Create a query that prints the name strings with their role strings. Extra points if it’s a view.
Create a query that sums the salaries.

Create a query that sums the salaries for a given role.

DROP TABLE IF EXISTS test_people;
CREATE TABLE test_people (
	id_index 		INT AUTO_INCREMENT PRIMARY KEY NOT NULL,
	name			varchar(255),
	salary			BIGINT
);

insert into test_people (name, salary) values
('Sophia',10000000000),
('Emma',1000000000),
('Olivia',100000000),
('Isabella', 10000000),
('Jackson',1000000),
('Aiden',100000),
('Liam',10000),
('Noah', 1000),
('Charlotte', 100),
('Caden', 10),
('Harper', 1);

DROP TABLE IF EXISTS test_roles;
CREATE TABLE test_roles (
	id_index 		INT AUTO_INCREMENT PRIMARY KEY NOT NULL,
	role			varchar(255)
);

insert into test_roles (role) values
('Minion'),
('Worker'),
('Manager'),
('CEO'),
('Evil Overlord');

DROP TABLE IF EXISTS test_assignments;
CREATE TABLE test_assignments (
	person_id 	INT,
	role_id		INT
);
insert into test_assignments (person_id, role_id) values
(1, 5),
(2, 4),
(3, 3),
(4, 2),
(5, 1),
(6, 1),
(7, 1),
(8, 1);

drop view IF EXISTS test_join;
create view test_join as
select p.name, r.role, p.salary
	from test_assignments a
	inner join test_people p on a.person_id = p.id_index
	inner join test_roles r on a.role_id = r.id_index;

select * from test_join;
select sum(salary) from test_join where role = "Minion";

DROP TABLE IF EXISTS test_people;
DROP TABLE IF EXISTS test_roles;
DROP TABLE IF EXISTS test_assignments;
drop view IF EXISTS test_join;

Add the ability to save a network and then start saving one!
Since the mysqldump file doesn’t work on my server, spit the current schema file into three sections:
- Table creation (incorporate DROP TABLE IF EXISTS table_name) – done
- View creation – done
- Tests – done
Using the linkFn() in ngFlyoutPanel to call getNetworks() on the load of the directive. Wasn’t sure that it would work, but it does. In Chrome, FF, and IE yet..
Uploaded to the main server. Works there, too!

Phil 8.14.15

8:00 – 6:00 SR

Doing the reverse commute today – meeting with Ronda between 3:00 – 4:00.
Could not get the userId over from the login directive over to the flyoutPanel directive. So rather than pass in individual values to the login, I’m now passing the same session object to both. That works fine. Cleaner, too:
```
this.sessionMessageObject = {
    loggedIn:false,
    userId : -1,
    networkId: -1,
    serverTarget:"rssPull.php"
};
```

I also like the way the call in the html looks:

<ng-login-dialog message_obj = "mc.sessionMessageObject" ng-if = "mc.sessionMessageObject.loggedIn === false"></ng-login-dialog>

It tells a better story, I think

Yay! got the service pulling down network names for a particular user!
Working on loading the full network.
For a break, started to migrate code to the server. The database won’t accept sourcing views generated by mysqldump -. I get an access denied. But using the the hand-coded sql in tnTableStructures, everything works.
Meeting with Ronda. The certs for the dev machine are bad? They seem to be OK on the production box. We installed tomcat and mySql on the dev box and verified that they were running.

Phil 8.13.15

8:00 – 5:00 SR

Still no admin
Gave Lenny the FizzBuzz test. He’s trying to solve it in Excel. And he did a fine job. Something to know about the Excel IF function. It’s an implicit ELSEIF. So always start with the narrow case and work out.
Going to add views to chart_data db for pulls, rather than submit a join directly.
- Ok, that was awesome. Views. Are. The. Best.
- Changing queries in the PHP to use the views. Done.
Get network listing, selecting and loading working
- got the interface built and running
Make the query into a flyout where the buttons are still visible in the collapsed state. Will also have to change the size of the webgl div? Or just overlay it?
Updated lib to angular 1.4.3 – no changes that I can see.

Phil 5.12.15

8:00 – 3:00

Still no admin
Interview with Debbie. She’s very comfortable with MySql, but not so much with coding. She didn’t even try to solve the Fizz Buzz test. I told Chris that I’d be fine with her and a coder, but he can only afford everything in one package.
I need to add a MERGE association. It will chain up from one merge to the next going from higher number to lower number and will end at the lowest number and show that. In other words, I might want to merge (1, ‘Donald Trump’), (3, ‘Trump’), (6, ‘The Donald’) and (10, ‘Mr. Trump’). Each of these could have a merge association so we could chain 10->6->3->1.
Since I now know how to use networks, I’m going to build out the open/save network flyout panel. I think I can use ng-style?
And I fixed the CSS problems! It turns out that I was using the wrong CSS class name on the main page. It was being used in the directive HTML as well.
Making the flyout a directive that will share a data object with the controller. Which si going very smoothly. I think one of the patterns with angular and typescript is to limit the number of files that require referencing within many files. If you don’t have to do a lot of wiring, things go much faster. Of course, this is also an argument for automation…
- Working through the styles and loading of the networks.

Phil 8.11.15

8:00 – 5:30 SR

No admin
Got pulled down the CSS rabbit hole and got nowhere, but I spent 2 hours doing it.
Fixed guids for Google News items – top news did not match targeted searches. Now it’s simply the link.
Added saving of pictures from NLP
restructuring the rsstestbed
- The first thing that is always done is that something is added to the network.
  - Search – google news rss.
  - URL – if selected from the list provided from the search, then a link is made. Otherwise, it’s a new start.
  - NLP – any URL that comes back has author, title, keywords, entities and concepts associated with it. These are attached to the URL that is added to the network. Because there is only one item of each type, implicit networks are made that connect URLS
  - Ratings point to an item or another rating. They can also point to an URL (which is then NLP’d for relations)

Phil 8.10.15

8:00 – 3:00

And still no admin
Today we figure out how to link items in the database that are linked in the display. Since a query can actually combine several sub-queries, I need to either send up an array of items or fire off multiple requests.
So first, I think that only the selected item(s) should be able to be the ‘parent’ of a query. That means that I first have to make it so that clicking on the space between items doesn’t unselect. Done. Now making multiple select if you hold down the ctrl key while clicking. Done.

Having a very annoying issue where the text api as used in the Alchemy PHP API is returning one thing and the Alchemy demo is returning something entirely different. I’m getting the former and want the latter:

Array
(
    [status] => OK
    [usage] => By accessing AlchemyAPI or using information generated by AlchemyAPI, you are agreeing to be bound by the AlchemyAPI Terms of Use: http://www.alchemyapi.com/company/terms.html
    [url] => http://krugman.blogs.nytimes.com//2015/06/29/the-awesome-gratuitousness-of-the-greek-crisis/
    [language] => english
    [text] => Barry Eichengreen ... fundamentally unworkable.
)

So why is this different?

  "status": "OK",
  "usage": "By accessing AlchemyAPI or using information generated by AlchemyAPI, you are agreeing to be bound by the AlchemyAPI Terms of Use: http://www.alchemyapi.com/company/terms.html",
  "url": "http://krugman.blogs.nytimes.com/2015/06/29/the-awesome-gratuitousness-of-the-greek-crisis/?_r=0",
  "language": "english",
  "text": [
    {
      "provider": "theconversation.com",
      "link": "https://theconversation.com/path-to-grexit-tragedy-paved-by-political-incompetence-43988"
    },
    {
      "provider": "krugman.blogs.nytimes.com",
      "link": "http://krugman.blogs.nytimes.com/2010/04/28/how-reversible-is-the-euro/"
    },
    {
      "provider": "www.imf.org",
      "link": "http://www.imf.org/external/pubs/ft/fm/2015/01/fmindex.htm"
    }
  ]

A Unifying Framework for Behavior-based Trust Models and who’s cited this

Phil 8.7.15

8:00 – 4:00 SR

And still no admin
Meeting with Debbie next Wednesday at 9:00
Fixed various timesheet issues
Working on storing and retrieving networks
- Items should only be added, not deleted
  - Modifying GoogleNewsRSS so that it can create and retrieve items using networkDbIo – done
  - Sending up userId for queries. Also setting data xfer so that data is sent up as an IPostObject. – done
- Is this the same for associations?
- Associations are added and removed from tn_network_data
- All sessions start with a default name (user+timestamp?), which can be modified
  - Added a description field to tn_networks.
  - Added a title field to tn_items that I somehow overlooked

Phil 8.6.15

8:00 – 4:00 SR

Still no admin
Check out Debra Fisher on LinkedIn
Finish rewriting login
- Ye gods. I spent all morning chasing down this ‘feature’: http://stackoverflow.com/questions/18342917/angularjs-ng-model-doesnt-work-inside-ng-if
- Changed the scope variables to be wrapped in an object and went back to ng-if.
Start on DataProvider – nope, spent the day on getting login to work right. Sheesh. At least I added Load/Save SearchNet buttons under the ‘history’ panel.

Phil 8.5.15

8:00 – 4:00 SR

Still no admin.
Need to add checks for http(s):// in the freeform text. If it’s there, open the url, if it’s not then send to GoogleNews
I think the best way to get the results from the network queries is to take the raw rows from the server and parse them into graphical network data. I will need to add variables for id_index and type (table at least, maybe more)

Query for all items on a network with labels synthesized from id_index and type name:

-- get all the items in a network
select ti.id_index, ti.guid, ti.item_type, ty.name as type_name, CONCAT(ty.name, '_', ti.id_index) as label, ti.text, ti.float_val, ti.link, ti.image
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_items ti on a.target_id = ti.id_index
inner join tn_types ty on ty.id_index = ti.item_type
where dat.network_id = 1
union
select si.id_index, si.guid, si.item_type, ty.name as type_name, CONCAT(ty.name, '_', si.id_index) as label, si.text, si.float_val, si.link, si.image
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_items si on a.source_id = si.id_index
inner join tn_types ty on ty.id_index = si.item_type
where dat.network_id = 1;

Query for all associations on the same network

-- get all the associations in a network
select a.id_index, u.login, a.user_id, si.guid as source_guid, a.source_id, ti.guid as target_guid, a.target_id, at.name as assoc_name, a.assoc_type, a.created_on
from tn_network_data dat 
inner join tn_associations a on dat.assoc_id = a.id_index
inner join tn_types at on a.assoc_type = at.id_index
inner join tn_items si on a.source_id = si.id_index
inner join tn_items ti on a.target_id = ti.id_index
inner join tn_users u on a.user_id=u.id_index
where dat.network_id = 1;

Ok, let’s see if we can pull this down and parse it!
Got into some kind of issue trying to use my old (pre ATSBase) login directives. Rebuilding.

viztales

Dimension reduction, State, Orientation, and Speed

Author Archives: pgfeldman

Phil 8.25.15

Phil 8.24.15

Phil 8.21.15

Phil 8.20.15

Phil 8.19.15

Phil 8.18.15

Phil 8.17.15

Phil 8.14.15

Phil 8.13.15

Phil 5.12.15

Phil 8.11.15

Phil 8.10.15

Phil 8.7.15

Phil 8.6.15

Phil 8.5.15