cache: optimise cached files scan

[jiefenghuang]

Why is this needed:
To address performance and memory issues when there are too many cached files:
1. filepath.WalkDir calls os.ReadDir, which by default reads all directory entries at once — this can use a lot of memory. It would be better to read entries in batches.
2. filepath.WalkDir sorts entries, which is unnecessary.
3. We can add concurrency: on a high-performance disk, using parallel directory reading / traversal should improve performance.

[SandyXSD]

cc @simpleqin