First, it should be noted that a Date field actually can store both date AND time in the same field, and Access will handle it correctly. And it has tools to isolate dates from times if you later need to separate the numbers. Call that "food for thought."
For what you have (i.e. separated date/time fields), you have a potential point of confusion. According to your mock-up of the form, you are using the same name of the control as you use in a field in your table. If you were going to enter a date into that and hit "SEARCH" then that control should ALSO be unbound.
If you enter the date into that box and hit your search, your problem is that for Access, your appointments are not normally distributed horizontally as you have shown it. Access likes vertical things but those boxes are horizontal. So you would need to instead open a recordset. From that point you would have to write some code to loop through a group of records for that AppDate and test the Apptime for each. You would then decide which of your 20 text boxes to fill in to show the appointment for that time&date.
The trick of changing background color is that you need to visit each box and do a DCount to see how many appointments have that specific date and time. It would be possible to step through a series of times in 1-hour (or other convenient) intervals looking for zero and non-zero values. If the result of the DCount (of appointments for a specific time) is 0 or not 0, you have the basis for deciding whether to change colors or not. For each of those cases, the same logic would apply, just different names.