Aggregate function must reference a field, not a control. Otherwise, have to repeat all the calculations within the Sum() function. And those calculations must use fields, not form controls.
Also, problem with subform pulling value from main form is the subform actually loads first, which means the values on main form are not available to the subform calcs.
Change subform RecordSource to:
SELECT CostDetails.*, Costings.TotHrs, Costings.DayHours, Costings.Employees FROM Costings RIGHT JOIN CostDetails ON Costings.ItemID = CostDetails.ItemID;
Then calculate Quantity:
=IIf([Unit]="Hour",[TotHrs],IIf([Unit]="Day",RoundUp([TotHrs]/[DayHours]),IIf([Unit]="Employee",[Employees],IIf([Unit]="Group",1,Null))))
And the Sum():
=Sum(Nz(IIf([Unit]="Hour",[TotHrs],IIf([Unit]="Day",RoundUp([TotHrs]/[DayHours]),IIf([Unit]="Employee",[Employees],IIf([Unit]="Group",1,Null))))*[Rate],0))