On a separate note, it is a bad idea to name table fields 'DateTime' and 'Count'. They are reserved words and can cause issues that are difficult to diagnose. Iwould suggest changing these names if you can.
DELETE
d.*
FROM [Data] d
LEFT JOIN (
SELECT
FinalName,
MAX([DateTime]) AS MaxDateTime
FROM [Data]
GROUP BY
FinalName
ORDER BY
FinalName
) t
ON d.FinalName = t.FinalName
AND d.[DateTime] = t.MaxDateTime
WHERE t.FinalName IS NULL;
SELECT
d.*
FROM [Data] d
LEFT JOIN (
SELECT
FinalName,
MAX([DateTime]) AS MaxDateTime
FROM [Data]
GROUP BY
FinalName
ORDER BY
FinalName
) t
ON d.FinalName = t.FinalName
AND d.[DateTime] = t.MaxDateTime
WHERE t.FinalName IS NOT NULL;
(Note the WHERE clause is different from the DELETE version)