Phil 11.28.18

7:00 – 4:00 ASRC PhD

    • Made so much progress yesterday that I’m not sure what to do next. Going to see if I can run queries against the DB in Python for a start, and then look at the Stanford tools.
      • installed pymysql (in lowercase. There is also a CamelCase version PyMySQL, that seems to be the same thing…)
      • Piece of cake! Here’s the test code:
        import pymysql
        class forum_reader:
            connection: pymysql.connections.Connection
            def __init__(self, user_name: str, user_password: str, db_name: str):
                self.connection = pymysql.connect(host='localhost', user=user_name, password=user_password, db=db_name)
            def read_data(self, sql_str: str) -> str:
                with self.connection.cursor() as cursor:
                    result = cursor.fetchall()
            def close(self):
        if __name__ == '__main__':
            fr = forum_reader("some_user", "some_pswd", "some_db")
            print(fr.read_data("select topic_id, forum_id, topic_title from phpbb_topics"))
      • And here’s the result:
        ((4, 14, 'SUBJECT: 3 Room Linear Dungeon Test 1'),)
      • Note that this is not an object db, which I prefer, but since this is a pre-existing schema, that’s what I’ll be doing. Going to look for a way to turn a query into an object anyway. But it turns out that you can do this:
        self.connection = pymysql.connect(
            host='localhost', user=user_name, password=user_password, db=db_name,
      • Which returns as an array of JSON objects:
        [{'topic_id': 4, 'forum_id': 14, 'topic_title': 'SUBJECT: 3 Room Linear Dungeon Test 1'}]
    • Built a MySQL view to get all the data back in one shot:
      CREATE or REPLACE VIEW post_view AS
      SELECT p.post_id, FROM_UNIXTIME(p.post_time) as post_time, p.topic_id, t.topic_title, t.forum_id, f.forum_name, u.username, p.poster_ip, p.post_subject, p.post_text
        FROM phpbb_posts p
        INNER JOIN phpbb.phpbb_forums f ON p.forum_id=f.forum_id
        INNER JOIN phpbb.phpbb_topics t ON p.topic_id=t.topic_id
        INNER JOIN phpbb.phpbb_users u ON p.poster_id=u.user_id;
    • And that works like a charm in the Python code:
      	'post_id': 4,
      	'post_time': datetime.datetime(2018, 11, 27, 16, 0, 27),
      	'topic_id': 4,
      	'topic_title': 'SUBJECT: 3 Room Linear Dungeon Test 1',
      	'forum_id': 14,
      	'forum_name': 'DB Test',
      	'username': 'dungeon_master1',
      	'poster_ip': '',
      	'post_subject': 'SUBJECT: 3 Room Linear Dungeon Test 1',
      	'post_text': 'POST: dungeon_master1 says that you are about to take on a 3-room linear dungeon.'


  • Tricia Wang thick data <- add some discussion about this with respect to gathering RPG data
  • Spend some time Grokking as well. Need to nail down backpropagation. Not today
  • Long discussions with Aaron about the structure of TimeSeriesML. Including looking at FFTs for the initial analytics.
  • A2P/AIMS meeting
    • Terrabytes of AIMS data?

Progress for today 🙂 ide