If it is intermittent and there are nop errors, then it will be hard to see what is causing it.

See if you can get the same error by repeatedly clearing the cache via the Joomla cache manager page.
It could be something to do with read/write locks on folders from the server, that lock cache files while (and shortly after) they get created.

It is working fine now for me.

You probably get the error when it times out, because of the large amount of cache. So Cache Cleaner can't finish clearing all the cache before the timeout.

You are also querying an external url, which might time out or might sometimes refuse connections.

You could try to investigate why you are getting so much cache that it times out in the first place.
Maybe some extension is filling up your cache folders when it shouldn't.

You can make Cache Cleaner ignore specific folders. There is no option to make it sometimes ignore it.

Prevent the cache folder from getting too large. Clear it more often. You could set it up to clear on interval or via a cronjob.