Fixed by adding a resource_type on the file which is required for the detroy method. It defaults to 'image', so if it isn't defined then it will try to remove an image with the passed public_id.