Category Archives: Phil

Phil 11.30.15

7:00 – 2:30: ???

  • Introduction to Learning, Nearest Neighbors
    • Learning based on observations of regularity (Bulldozer Computing)
      • Nearest Neighbor
        • Pattern Recognition
      • Neural Networks
      • Boosting
    • Learning based on constraint (Human-Like)
      • One Shot Learning
      • Explanation-based learning
    • Pattern Recognition
      • Feature detector produces a vector of values.
      • Fed into a Comparator which tests the new vector against a library of other vectors
      • Can use decision boundaries
      • If something is similar in some respects, it is likely to be similar in other respects.
      • Robotic motion is a search problem these days??
  • Work
    • Standard first-day stuff
    • Discussions with Aaron about design
    • And the interesting thought for the day:
      • Do we need a sort of crowd-sourced weighting determination of machine ethics? Right now, the person that writes the code for the first self-driving car that decides the runaway trolley problem could reasonably be thought of as having committed premeditated murder. But what if we all together set those outcomes, in a way that reflected our current culture and local values?

Phil 11.25.15

7:00 – 1:00 Leave

  • Constraints: Search, Domain Reduction
    • Order from most constrained to least.
    • For a constrained problem, check over and under allocations to see where the gap between fast failure and fast completion lie.
    • Only recurse through neighbors where domain (choices) have been reduced to 1.
  • Dictionary
    • Add an optional ‘source_text’ field to the tn_dictionaries table so that user added words can be compared to the text. Done. There is the issue that the dictionary could be used against a different corpus, at which point this would be little more than a creation artifact
    • Add a ‘source_count’ to the tn_dictionary_entries table that is shown in the directive. Defaults to zero? Done. Same issue as above, when compared to a new corpus, do we recompute the counts?
    • Wire up Attach Dictionary to Network
      • Working on AlchemyDictReflect that will place keywords in the tn_items table and connect them in the tn_associations table.
      • Had to add a few helper methods in networkDbIo.php to handle the modifying of the network tables, since alchemyNLPbase doesn’t extend baseBdIo. Not the cleanest thing I’ve ever done, but not *horrible*.
      • Done and working! Need to deploy.

Phil 11.24.15

7:00 – Leave

  • Constraints: Interpreting Line Drawings
    • Successful research:
      • Finds a problem
      • Finds a method that solves the problem
      • Using some principal (That can be generalized)
  • Gave Aaron M. A subversion account and sent him a description of the structure of the project
  • Back to dictionary creation
    • Wire up Extract into Dictionary
      • I think I’m going to do most of this on the server. If I do a select text from tn_view_network_items where network = X, then I can run that text that is already in the DB through the term extractor, which should be the fastest thing I can do.
      • The next fastest thing would be to pull the text from the url (if it exists) and add that to the text pull.
      • Added a getTextFromNetwork() method to NetworkDbObject.
      • The html was getting extracted badly, so I had to add a call to alchemy to return the cleaned text. TODO: in the future add a ‘clean_text’ column to tn_items so this is done on ingestion. I also added
      • Added all the pieces to the rssPull.php file and tested. And integrated with the client. Looks like it takes about 8 seconds to go through my resume, so some offline processing will probably be needed for ACM papers, for example.
    • Wire up Attach Dictionary to Network
      • The current setup is set so that a new item that is read in will associate with the current network dictionary. Need to add a way to have the items that are already in the network to check themselves against the new dictionary.
      • Added class AlchemyDictReflect that will place keywords in the DB. Still need to debug. And don’t forget that the controller will have to reload the network after all thechanges are made.

 

Phil 11.23.15

7:00 – Leave

  • Search: Games, Minimax, and Alpha-Beta
    • Branching factor (B)
    • Search depth (D)
    • Combining the two gives the number of leaf nodes or B^D
    • Branching factor of chess is approximately 14?
  • Dictionaries
    • Wire up Create New Dictionary – done
    • Wire up Extract into Dictionary
      • I think I’m going to do most of this on the server. If I do a select text from tn_view_network_items where network = X, then I can run that text that is already in the DB through the term extractor, which should be the fastest thing I can do.
      • The next fastest thing would be to pull the text from the url (if it exists) and add that to the text pull.
    • Wire up Attach Dictionary to Network
      • The current setup is set so that a new item that is read in will associate with the current network dictionary. Need to add a way to have the items that are already in the network to check themselves against the new dictionary.

Phil 11.19.15

7:00 – 5:00 Leave

  • Reasoning: Goal Trees and Rule-Based Expert Systems
    • There, now I’m back in order.
    • H. Simon – The complexity of the behavior is max(cplx(prgm), cplx(env))
    • More Genesis – Elaboration graphs
    • Genesis judges similarity in multiple ways: (this presentation, page 25)
      • Using word vectors
      • Using concept vectors: seeing similarities not evident in the words.
    • Genesis aligns similar stories for analogical reasoning (Needleman-Wunch algorithm, which is a way of comparing string similarity using matrices)
  • IRB renewal – ask Wayne
  • In a fit of orderliness, created shortcuts to the cmd windows that the makefiles run in
  • Dictionaries
    • Don’t forget to move the text-extraction calls to the methods that need it and see if that speeds up the others. Done. Much faster. PHP is dumb. Or needs a preloader/compiler
    • Cleaned up the loading and display code a bit. Need to add a tree view (which looks like it can be done in pure CSS), but that can wait.
    • Adding manual entry
      • Finished the form
      • Clear for entry and separate parent is done
      • addition
      • deletion
      • modification (adding parents in particular)
    • Adding text extraction

Phil 11.18.15

7:00 – 5:00 Leave

  • Search: Depth-First, Hill Climbing, Beam
    • Looks like I missed one. Will go back tomorrow.
    • Patrick Winston is the instructor, and he showed a version of Genesis (around 42 minutes in), which maps stories into graphs and then searches through them at different levels of abstraction. It is capable of providing relevant answers to questions. Here’s a white paper. What I think is interesting here is the following:
      • How they describe their hierarchy (revenge, tit-for-tat, etc). This is built, not computed based on what looks to be a rules engine.
      • How text in a story can be looked at as a graph.
      • And that’s without reading the paper!
  • Working on dictionary directive
    • Ramifying the dictionary choice through the controller. It should be set initially when the network loads (done), but should be changed if a new one is selected.
    • There does need to be a second step where the DictPull is run and the text of the items is compared against the selected dictionary and KEYWORDS items are added to the network. This is when the network’s dictionary index should be changed.
    • Downloading the dictionary.
    • Need to upload a new word
  • 12:00 workshop on Grant Writing in ITE 459
  • 4:00 meeting with Dr. Pan.

Phil 11.17.15

7:00 – 4:00 leave

  • Reasoning: Goal Trees and Problem Solving (why is this pertinent to me?)
    • Apply all safe transforms
    • Apply heuristics
    • AND nodes and OR nodes (AND/OR, problem reduction, or goal tree)
  • Realized that I have a redundant user_id in tn_dictionary_entries. This could be used to allow non-owners to add words to the dictionary. Which means that dictionaries could be shared. On the whole, I think that’s a good idea. Adding the change to the dictionary code.
  • Ok, back to text extraction (Is this a safe transform?)
  • Added a _buildExtractor() private method and imported all the extractor parts.
  • It does take a long time. Just to load? I’d like to try profiling…
  • Wow. Extracting, loading into the database, getting the JSON output and deleting the dictionary all work!
  • I think the next step is to either get some definitions or start building the directive. After my lunchtime ride, I decided that the directive is probably the best thing to do next. Adding user functionality is a good way of ensuring that the server functionality makes sense.
  • Added ‘get_user_dictionaries‘ to rssPull.php. We’ll start with that.
  • Got the skeleton of a directive up and retrieving the dictionary list.

 

Phil 11.16.15

7:00 – 6:00 Leave

  • Found a new programmer resource that looks good – I Programmer. They pointed me to an article about Babel, which compiles JavaScript to… other things. It might even be able to monkeypatch modern JS to run on old browsers. Need to test one of these years. It’s based on plugins which really means that it can map from one thing to anything else. My only issue is that it could break debugging unless there is a mapping file like typescript has.
  • Discovered another communication app – Telegram. ISIS used it to announce Paris?
  • Noon – Thad Starner in ITE 459. Very interesting. Met Aaron Massey, who might be good on the Committee.
  • I’ve been reading Tefko Saracevic‘s paper RELEVANCE: A Review of and a Framework for the thinking on the Notion in Information Science. It’s full of really nice stuff, from a time when you couldn’t just throw processing power at problems and brute force an answer. It’s clarified my thinking about the client word-based network:
    • Search engines are pragmatic relevance engines (i.e topic-relatedness, quality, novelty, importance, credibility, etc.). The networks that they produce try to correlate knowledge at the ‘source’ – basically ‘in the world’
    • We, as individuals are pertinence/situational relevance machines (Wilson’s concerns, preferences and stock of knowledge). Our internal knowledge graph represents our view of the world. We are the ‘destination’ for information.
      • “Situational Relevance is relevance to a particular individual’s situation – but to the situation as he sees it, not as others see it, nor as it really is.”
      • The ‘shape’ of our internal knowledge graph, the sources of information that we lean more heavily on, the weights that we give to certain words (or possibly concepts) may be able to determine whether we are dependably credible or dependably counter-credible.
    • By enabling client-side weighting, we let users adjust components of a relevant search so that it becomes pertinent to us.
    • The information that we produce in this process (dictionaries, weights, etc) can be stored so that a well-structured record of what is pertinent to individuals (and more importantly, groups of individuals) becomes part of the world knowledge. Correlations with respect to internal credibility may then in turn be able to infer the credibility (or lack of) of information in the world.
  • Getting back to dictionary integration.
    • Re-upped my IntelliJ subscription for another two years
    • Updated files and DB. All seems to work
    • DbDictionary.removeDictionary returns a fail JSON message. Fixing. Fixed!
    • Adding ability to update an entry – done.
    • Finishing CreateDictionary. Finished and tested
    • Adding DeleteDictionary. Finished and tested
    • Adding ModifyDictionary. Finished and tested
    • Adding term extraction. Started poking, but that’s it. More tomorrow.

Phil 11.13.15

7:00 – 3:30 SR

  • Brought in the flex code and gave to Al.
  • Al thought that I had not brought in all the Java code and was going to have me produce a spreadsheet stating where all the jar files could be found. In reality, I had brought in all the jar files, but the import in his IDE hadn’t done a proper job and he never looked inside the zip file to see what was there. Sigh.
  • Al also doesn’t have the System diagrams that I sent him through Bill on November third. Verifying that they got there. If they didn’t, I’ll burn a new disk and bring that in.
  • Told the folks on site that I was taking off the next 4 weeks to work on my thesis, with a high probability that I would be put on a new position before I got back, since there’s no full time work for me on SR any more apparently (Steve F’s words)
  • Wrote up a description for Lenny to do the truancy reports
  • Burned disks that have the entire Viztool project on them. Gave one to Joe and Bill gets the other one to pass on to Al.

Phil 11.9.15

7:00 – 3:00 SR

  • Training
  • Got all the Java files built and burned to disk the main problem that I had was getting a Tomcat runtime instance showing up. Here was the fix: http://stackoverflow.com/questions/2000078/apache-tomcat-not-showing-in-eclipse-server-runtime-environments

Phil 11.6.15

Burning vacation today

Working on the dictionary server code

  • Make sure that adding a parent doesn’t cause a loop.
  • Adding UpdateStatementObject helper class since I seem to be doing that a lot. And of course it took a few hours longer than I thought it would, but it’s nice and general.

11.2.15

8:00 – 5:00 SR

  • Al came on board today. Showed him around the system, discovering that the scripting system wasn’t working on the production server. Fixed that, and downloaded a copy of the documentation for him to look at. Also gave him accounts on the integration server for him to poke around.
  • Fixed the Reqonciler bug. Had to insert the modified query directly into the reqonciler table to get around odd quote-escaping issues.
  • Updated Friday’s work from the repo. Updated the database and ran the term extraction and dictionary tests.
  • Working on dictionary access methods.
    • Got AddEntry and Remove Entry working. Also removed the tn_dictionary table and stuck the dictionary_id in the tn_dictionary_entries table.
    • Added cascade entry/modification of parent if it doesn’t exist. Otherwise the indices won’t work.

Phil 10.30.15

8:00 – 4:00 SR

  • Working from home today, waiting for people to show up.
  • Here’s the fix for the Reqonciler issue:
    • Open Reqonciler in your browser.
    • click Post-Processing button to see all queries
    • double click the one that you disabled this morning to edit, Order 2100, update month 1 year 2 to 100% from month 12 year 1
    • add ” AND NOT ISNULL(bc.uid)” at the end of the query without the double quotes. Make sure there is a space before.
    • Save, run, and check the data
  • In the process of getting my home dev environment working again. I swear I should just do this once a week so it’s less stressful.
    • Fixed the Imagick load so that there is a test for the extension and whether the extension is installed correctly.
    • Disabled the world wide web service so that apache could run on port 80
    • Updates all the files in the Apache htdocs directory. Forgot that I had updated the server access methods to take an object.
    • It occurs to me that I can load up the DB directly on the server if I don’t get everything done with the dictionary by Wednesday.
  • Examine AlchemyNLP and see if there is a hierarchy that can be used. Not without a lot of work.
  • Buy and download the fivefilters term extractor and see how to integrate.
    • Ordered. Waiting for confirmation to show up.
    • Installed. Time to see if it’ll work. It looks good, though possibly slow? starting to put together a dictionary class to examine more deeply.
  • Add dictionary Flyout directive
    • Name the dictionary
    • Choose the networks (add/remove from list) ()
    • Input html, text or url
    • Get the clean text and show the machine extracted terms. We could look up potential definitions too – from wordnik. Set up an account and applied for a developer key.
    • Show a list of selected terms with checkboxes
      • Checked items can be deleted or grouped
      • Items can be added by typing into a field
    • Show a list of ‘group items’.  This displays a list of the items who’s index appears in the ‘parent’ field
      • Selecting an item in this list reorders the item list to show the appropriate group first
  • There should also be a select dictionary option on the network flyout