7:30 – 4:30

• Prepping for demo
• Got dong’s war file deployed on my machine. That works, rather than building. He’s going to ask Mike to try building and see where the problem lies
• Problems with ie today – it can’t open multiple Flex programs.
• Spent some time getting the projmanager user working in mySql
• better pull for errors: SELECT p.title as project, f.name as funding, o.* FROM obligations_outlays o, fundings f, projects p;
• IngestManager problems:
  • Can’t parse special characters into column names or the values can’t be inserted properly in the column. For example, ‘2/15/60’ won’t read in at all, ‘2-15-60’ won’t have any data in the columns, and ‘2_15_60’ is a date format that Actionscript can’t recognize
  • If the XML SOLR file has CRs in the text file, they are kept, which screws up VISIBILITY. They need to be stripped.

Script to look for decreasing values in outlays:

# Strip off a column based on its name
def stripColumn(headers, table, columnName):
    newRow = []
    for item in headers:
        if item != columnName:
            newRow.append(item)
    newTable = [];
    newTable.append(newRow)
    for row in table:
        newRow = []
        for i in xrange(len(headers)):
            if headers[i] != columnName:
                newRow.append(row[i])
        newTable.append(newRow)
    return newTable

# Find a month that is less than the previous
def errorsToSolrXml(doc, root, header, table):
    for row in table:
        for i in xrange(len(row)-1):
            if header[i].count("month_") > 0:
                cur = int(float(row[i]))
                next = int(float(row[i+1]))
                #print row[0]+": "+header[i]+" = "+str(cur)
                if(cur > next):
                    #print header[i]+" = "+str(cur)+" > ("+str(next)+")"

                    errorNode = doc.createElement('doc')
                    root.appendChild(errorNode)
                    fieldNode = doc.createElement('field')
                    fieldNode.setAttribute('name', 'type')
                    textNode = doc.createTextNode(str(row[0]))
                    fieldNode.appendChild(textNode)
                    errorNode.appendChild(fieldNode)

                    fieldNode = doc.createElement('field')
                    fieldNode.setAttribute('name', 'cur_month')
                    textNode = doc.createTextNode(header[i])
                    fieldNode.appendChild(textNode)
                    errorNode.appendChild(fieldNode)

                    fieldNode = doc.createElement('field')
                    fieldNode.setAttribute('name', 'next_month')
                    textNode = doc.createTextNode(header[i+1])
                    fieldNode.appendChild(textNode)
                    errorNode.appendChild(fieldNode)

                    fieldNode = doc.createElement('field')
                    fieldNode.setAttribute('name', 'value')
                    textNode = doc.createTextNode(str(cur - next))
                    fieldNode.appendChild(textNode)
                    errorNode.appendChild(fieldNode)

#################### 'main'

impl = getDOMImplementation()

jf = ScriptFacades()
table = jf.dbQuery('project_portfolio', 'SELECT * from obligations_outlays where project_id=1 and year=2010')
tableData = table.getTable()
newTable = stripColumn(tableData[:1][0], tableData[1:], 'uid')

errorDoc = impl.createDocument(None, "add", None)
root = errorDoc.documentElement
errorsToSolrXml(errorDoc, root, newTable[:1][0], newTable[1:])
#print errorDoc.toprettyxml()
jf.publishXml("/obligation_errors.xml", errorDoc.toxml())

7:30 – 5:30

• Working on writing a script that does something with Dong’s data

Inserting data for months – no invoice though

jf = ScriptFacades()

table = jf.dbQuery('project_portfolio', 'SELECT uid from obligations_outlays where project_id=1')
tableData = table.getTable()
print tableData[1:]

for uid in tableData[1:]:
    for month in xrange(1,25):
        monthName = 'month_'+str(month)
        query = 'UPDATE obligations_outlays set '+str(monthName)+' = '+str(random.randint(1000,2000)*month)+' where uid = '+str(uid[0])
        print query
        jf.dbQuery('project_portfolio', query)

Getting data and publishing

# This method converts a DbTable (A list of lists) into an Apache SOLR xml document
def solrXmlFromListList(doc, root, headers, table):
    for row in table:
        rowNode = doc.createElement('doc')
        root.appendChild(rowNode)
        #print row
        for i in xrange(len(headers)):
            #print str(headers[i])+" = "+str(row[i])
            fieldNode = doc.createElement('field')
            fieldNode.setAttribute('name', headers[i])
            textNode = doc.createTextNode(str(row[i]))
            fieldNode.appendChild(textNode)
            rowNode.appendChild(fieldNode)

# Strip off a column based on its name
def stripColumn(headers, table, columnName):
    newRow = []
    for item in headers:
        if item != columnName:
            newRow.append(item)
    newTable = [];
    newTable.append(newRow)
    for row in table:
        newRow = []
        for i in xrange(len(headers)):
            if headers[i] != columnName:
                newRow.append(row[i])
        newTable.append(newRow)
    return newTable

# Change month_xx to a year/date string
def mapDates(row, year):
    for i in xrange(len(row)):
        if row[i].count("month_") > 0:
            splits = row[i].split('_');
            month = (int(splits[1])-1)%12
            addYear = (int(splits[1])-1)/12
            row[i] = str(month+1)+"-1-"+str(year+addYear)
            #print row[i]

#################### 'main'

jf = ScriptFacades()
table = jf.dbQuery('project_portfolio', 'SELECT * from obligations_outlays where project_id=1 and year=2010')
tableData = table.getTable()
newTable = stripColumn(tableData[:1][0], tableData[1:], 'uid')

mapDates(newTable[:1][0], 2010)

impl = getDOMImplementation()
myDoc = impl.createDocument(None, "add", None)
rootElement = myDoc.documentElement
solrXmlFromListList(myDoc, rootElement, newTable[:1][0], newTable[1:])
#print myDoc.toprettyxml()
jf.publishXml("/obligation_outlays_proj_1_year_2010.xml", myDoc.toprettyxml())

7:30 -4:30

• Today, in 1862, the Mexican Army defeated the French at the battle of Puebla. In case you were wondering.
• Richard M is getting Tomcat working better on the LDAT server. We will be ready to continue experimenting in a week or so.
• While waiting for Dong to roll in his changes from yesterday, I’m going to try and fix the case where the JVM holding a jython instance dies from some fatal error. Wound up deleting the ScriptEntity instance from the static map of all script instances and loading a new instance. Seems to work, but we need to test on other servers.
• Got Dong’s data, but the project will not build

7:30 – 3:30

• Some thoughts on Mike’s slideshows
  1. Make sure that the changes/additions to the db happen automatically when the class is first addressed (CREATE TABLE IF NOT EXIST foo()) doesn’t seem to take much overhead. That way, deployment will be easier.
  2. I was thinking that the slide show controls would be another Utils panel that could be hidden as needed.
  3. Selectable transitions (fade, slide vertical, slade horizontal, etc) would be a plus
  4. Maybe the slide controls can live in the top part of the display and be hidden unless needed?
• Brought in EdgeUtils for Clift
• Meeting with Jim M, Trish ? and Dong. Lots of changes, as can be seen is his log entry.

7:30 – 4:30

• Svn is working, but the check in is not working in Eclipse. Attempting to update svn plugin, but things are hanging. May have to say goodby to svn integration. Dong has Subclipse working. Will switch to that later. Tortoise for now
• Call Richard Martin – 443.479.8721
• Got Visibility and VisibilityScripting checked out and running