Phil 10.5.2009

7:00 – 3:00

  • Ingested data, fixed some logins, and added a superuser that had somehow gotten deleted
  • Thinking about form builder – got a good start. It’s *big*
  • Dong found a bug with the QueryBuilderBase tree builder – fixed

Visiblity – to do

  • configurable html/flash based upload file control
  • return distinct table names for vis2
  • vis2 returning columns when tables are requested and the user doesn’t have any tables??????
  • Make sure parent Role can see the data set for children
  • User who has no tables causes vis2 to generate list of all columns in the database tables(?) when Get Tables button is clicked
  • Set others’ autoIngest data uneditable

Visibility Bug List 10/2/2009

  1. Upload control no longer launches html, fixed
  2. UserID on AutoIngest needs to be a name, not a number
  3. Auto-ingested files appear to be private (They can’t be seen by others)
  4. Unable to set PUBLISH or PRIVATE in AutoIngest, fixed
  5. “Duration” should be set to “Frequency”, or How Often” – fixed
  6. Time popup for Duration disappears after each ingestion
  7. Password disappears for first user?
  8. “temp” appears after remote ingest. Remote file disappears (Figure 1)
  9. Verify that “Add” and “Add as child” check to see that thereis a Question Name. May be also disable the ability to edit the Query text and only have that set by the widget (tooltip to explain)
  10. Mouseup in chart sometimes causes an “object.toString” tootip, which is weird
  11. File seem to be ingesting earlier than specified.
  12. Update time does not appear in table select dropdown (Figure 2)

Phil 10.2.2009

8:00 – 3:00

  • Autoingest appears to be working, though not at exactly the right time
  • Ingested data
  • Working on bug fixes
  • Fixed password issues
  • Added checkbox renderer to user role presentation
  • Added checks to “Add” and “Add Child” to Questions view in Advanced Query Builder

Phil 10.1.2009

7:30 – 4:30

  • Tried deploying but got nowhere. The code kept trying to connect to 127.0.0.1 rather than the actual server, even though I verified that the config files were correct
  • Deployed to laureldemo to see if there was any difference. It seemed to deploy just fine. There was one problem with private data being visible to other users. Dong’s fixing that now. Done
  • Burned  a new disk and going to try again
  • Talked to Anne, Pat can’t remember her password
  • Deployed the new version. Most things work, made a list of 12 bugs to bring in

Phil 9.30.2009

7:30 – 3:30

  • Deployed the new version – there are a few bugs:
    • Although remote files validate, they don’t read in – looks like the problem was that the remote files don’t terminate in .xml
    • Blanks are counted in the UniqueCounts view – fixed
    • Need to clear out the “where clauses” if the last item is unchecked – fixed
    • Need to keep the where clauses in the unique count query construction – fixed
  • Had a good discussion with Mike C about data
  • Scheduled a meeting with Anne to go over the new features
  • Burned a new disc and am ready to deploy tomorrow.

Dong Shin Visibility Status – Sep '09

Week of 09/28

  • IE chokes when URL (endpoint) is not set up with full path; ***.***.com or IP address
  • Fixed tables returning from server not role/user based
  • Added publish status, changed mode to Overwrite to IngestConfirm window
  • Changed date in the table names to return YYYY-MM-DD format
  • Bug found in AutoIngestor
    • found that files with no ext (.xml, .xls, etc.) fails to return columns
    • input validation added to RemoteFileWindow and RoleSelectinCanvas
  • Added dataTips to AutoIngest DataGrid
  • Rearranged New and Remote File buttons in AutoIngestConfig
  • Extending File class to HTTPFile to use remote files
  • Adding HTTPS and HTTP capabilities to other types of files
  • All changes uploaded and downloaded.

Week of 09/21

  • Administrators can now delete any tables from the IngestManager
    • isUserAdmin() methods added to both client and server
  • Changed displaying ID of the user to Names of the Users in IngestPanel and IngestedDataConfiguration
    • TablesInfoObject.java
    • Database Query to join TABLE_USERS and TABLE_DATA tables
    • StyleFunction for the AdvancedDataGrid
  • AutoIngest now checks for existing data (file name and table name) on the client
  • Remote File Ingest to handle http and https
    • Server – HttpUtillities.java and SSLUtilities.java under com.edgeti.utils
    • Flex – added Remote File button in IngestVBox, RemoteFileWindow.mxml to handle remote file ingest
    • remote file added to Files Available after validation
    • change handleFileSelected() in IngestVBox to handle http and https and related events/services
    • columns returned from http and https
    • removed old remote file routines from AutoIngestConfigVBox and put RemoteFileWindow to use http and https autoingest…..
    • changed server codes to use full filename for remote (http) and uploadDir for local files.
    • ingest works!!!! seems nothing is broken….

Week of 09/14

  • disabled Use Remote location for Auto Ingest for now….
  • fixed auto ingest to run specified time, hourly, daily, weekly.
  • creating RoleColumns/RoleTree component so it can be used by IngestVBox and AutoIngest.

Week of 09/07

  • Resolved most of the bugs/fixes/enhancements Phil asked.
  • Reworking Ingestor, AutoIngest Config, and Ingested Data Config to use Roles and Role Columns. This requires changes to the Servlet, too.
  • Ingestor problem resolved
    • moved connect/disconnect for each insert SQL to east ingest. It seems ingestion is a lot faster now. XLS ingestor should be changed
  • Continued troubleshooting ingestor problem
    • Ingestor running too fast?
    • file too large?
  • Remote File ingestion
    • running Tomcat under SYSTEM prevents the Ingestor to read remote files
    • running different user (me) works fine.

Week of 08/31

  • Test Deploy to EdgeDemo – successful
    • change endpoint in assets/setupModel.xml
    • MakeFakeData in ManagementUtilsTest.html fails. It seems there is a problem in populateFakeUsers in UsersAndRoles.java
  • Having problems with ingestor. MySQL reports socket not available error
  • Continue looking into remote file ingesttion

Phil 9.28.2009

7:30 – 3:30

  • Gotta install sp2…. Well that was 20 minutes well spent! Wheee.
  • Working on multiple select = done. Leaving the single select code in place for a while out of, well, superstition and fear, actually.
  • I’m thinking about adding another chart that has the unique counts for each column – that way it would be possible to easily find out how many people we in a query, even if they had multiple skills. – done

Mike 9.27.09

  • Search now automatically scrolls to the selected item in the datagrid if it finds one
  • Saving / loading now once again works with the AdvQueryWidget
  • DynamicPanel may now go directly from minimized to maximized and vice versa
  • Updated all Layout Managers to handle the new panel feature

Phil 9.25.2009

7:30 – 3:30

  • Another meeting with John, Anne, Brian, and Mike this time
  • Based on getting new data (roster2) John is much happier with the tool. Mind you, there have been no changes to the tool.
  • Working on getting an “insert column” capability, *then* we might get to multiple selects – done

Phil 9.24.2009

7:30 – 4:00

  • Long and painful meeting with John and Anne. My notes follow:
    • Data names are not clear from source
    • Should be able to check a box and get a list of names
    • Eliminate redundant columns
    • Fix column names
    • Either the basic tree or the question tree needs to have some checkboxes that represent “where clauses” that would be OR-ed together
    • Collapse redundant entries in columns on Flex side – i.e. how many unique names there are. This may be a new chart view
  • Working on Question interface – first pass is done. Rolling up to bring in.
  • Tomorrow, work on AdvancedQueryWidget.onCreateChartClicked() so that multiple bars can be selected.

Phil 9.23.2001

7:30 – 5:00

  • Ingested data at the fort. Talked to John, who tried to find info with Pat yesterday. I said that the display is only as good as the data, but I don’t think it stuck.
  • Working on “Questions” interface
    • Modify – done
    • Add – done
    • Add as Child – done
    • Delete – done – turned out to be tricky. You have to use the delete operator.
    • Have “Set Query Builder” populate the _questionQuery.text field
    • Add a setTableQuestions(String questions,  String tableName) method in DbRemoteObject that a) Creates a table_questions table if there is none, and b) loads the xml as a blob in the table row (note: only one set of questions per table) See the dashboard entry for how to handle xml – done
    • Add a getTableQuestions(String tableName) method in DbRemoteObject that a) Creates a table_questions table if there is none, and b) either returns the xml if it exists or returns and empty <questions/> xml string. – done

Phil 9.22.2009

7:30 – 3:30

  • Deployed the new version with Mike’s fixes. Everything seems to be working fine
  • Talked to Anne about building special-purpose tables from the master table to make it easier to explore the data
  • Floated the question idea
  • Realized that we could also have all the available strings in a column listed in the (a?) hierachy view, so some level of drilldown could be available…
  • Changed all query double quotes to single quotes
  • Phone call from Anne – Meeting with here and John on Thursday. Might have a version by then.
  • Started on the question tab, mostly because it’s an easier problem to solve that might give us what we want. Go the part where you can click on a tree component “question” that runs a query.
    • Need to be able to interactively build the xml
    • Need to save the xml in a table-specific entry (only one entry per table holding the xml)

Phil 9.21.2009

7:30 – 4:00

  • Hey! It’s Fall! No wonder it’s dark when I get up these days
  • So I come in this morning and find that Mike has fixed half the issues that Anne and John listed. It’s like having pixies do the dishes at night!
  • VIS2 Meeting at 10:30
  • Writing up support for S2Dash
  • Need to make the system more Pat-freindly. Not sure what that means, but I’m leaning towards “Questions” that point to queries