7:00 – 5:00 VTX

• Writing
  • Loaded extracted terms into LMT.
    • removed method and datum
  • Working through current IR and group polarization
  • A document retrieval system for man-machine interaction. From 1964, folks. Also, the ACM is mapping concepts to Wikipedia entries now.
• Write up sprint story – done

  • Develop a ‘training’ corpus known bad actors (KBA) for each domain.

    • KBAs will be pulled from http://w3.nyhealth.gov/opmc/factions.nsf, which provides a large list.
    • List of KBAs will be added to the content rating DB for human curation
    • HTML and PDF data will be used to populate a list of documents that will then be scanned and analyzed to prepare TF-IDF and LSI term-document tables.
    • The resulting table will in turn be analyzed using term centrality, with the output being an ordered list of terms to be evaluated for each domain.

• Building view to get person, rating and link from the db – done, or at least V1

  CREATE VIEW view_ratings AS
    select io.link, qo.search_type, po.first_name, po.last_name, po.pp_state, ro.person_characterization from item_object io
      INNER JOIN query_object qo ON io.query_id = qo.id
      INNER JOIN rating_object ro on io.id = ro.result_id
      INNER JOIN poi_object po on qo.provider_id = po.id;

• Took results from w3.nyhealth.gov and ran them through the whole system. The full results are in the Corpus file under w3.nyhealth.gov-PDF-centrality_06_02_16-13_12_09.xlsx and w3.nyhealth.gov-WEB-centrality_06_02_16-13_12_09.xlsx. The results seem to make incredibly specific searches. Here are the two first examples. Note that there are very few .com sites.:
  • PDF-derived: respondent consent committee probation agreement pursuant
    • This is very medical for some reason, though not very region specific
  • Web-derived: physician license professional return number effective
    • This is very medical, but also biased towards NY state.
  • Prepending physician names seems (so far) to lead to consistent hits in the search results on standard Google