ASRC NASA 9:00 – 4:30

• New schema, as of yesterday:
• Next steps for financial analytics
  • Get the historical data to Aaron’s code. Need to look at Pandas’ read_json
    • That was a terrible experience. Here’s how you take a date, change it to a timestamp, and get it into a Pandas Dataframe
    • Step 1:

      def get_balance_JSON(self, type: 'QuantileName') -> Dict:
          dt = datetime.datetime(self.actual_year, self.actual_month, 1)
          ts = datetime.datetime.timestamp(dt)
          return {"time": ts, type: self.balances[type]}

    • Step 2:

      for ph in self.pred_history_list:
          data = ph.get_history()
          jdata = json.dumps(data)
          df = pd.read_json(jdata)
          df.set_index('time',inplace=True)
          pd.to_datetime(df.index)
          print("\n-----\n", df)

    • Useful links
      • how-can-i-convert-a-datetime-object-to-milliseconds-since-epoch-unix-time-in-p
      • timestamp-index-for-pandas-dataframe-from-read-json
      • pandas-dataframe-datetime-index-doesnt-survive-json-conversion-and-reconversion
  • Get the predictions and intervals back – done!
  • Store the raw data
  • update and insert the lineitems – nope
  • populate PredictedAvailableUDO table
• Found a subtle error with creating the actual date from the fiscal date:

  def override_dates(self, year: int, fmonth: int):
      # handle fiscal math converting months that are greater than 12 to the correct fiscal year and month
      self.fiscal_year = year
      if fmonth > 12:
          fmonth = (fmonth % 12) + 1
          self.fiscal_year += 1
      self.fiscal_month = fmonth
  
      # convert the fiscal month and year to actual
      month = fmonth + 2 # convert from US Gov Fiscal to Actual
      self.actual_year = year
      if month > 12:
          month = (month % 12)
          self.actual_year += 1
      self.actual_month = month

  The issue is how the months are handled. The fiscal month is taking an unbounded number and modding it by 12. That produces a range from 0 – 11, so I add one to the result. The actual month is offset by 2 months (The end of the fiscal year is two months before the end of the actual year). So in this case I mod by 12, but don’t have to add the one because it’s working on a range of 1 – 12, not 0 – 11. Anyway, I think it’s fixed now.