Wouldn't you know that as soon as I'd post a request I'd figure it out. I used a DateDiff funtion within the initial Calculation before trying to run the average.
So I used...
TimeDiff: IIf([Time2]-[Time1]<0, (DateDiff("n",[Time1], [Time2],)+1440), DateDiff("n", [Time1],[Time2]))
Then...