What does Job mean by destroy? He has not been destroyed. If it has been appointed for all men to die, if naked we come into and leave this life, if the possessions of this world only rust and rot away, what has been destroyed but our false notions and expectations? Being temporarily emptied is not the same as being destroyed. God has spared Job's life and has His purposes for such. Why isn't that the focus?
The language or translation seems to be holding something back here. If Ham only saw (caught a glimps), why does it say Noah knew what the younger son had done to him? There was some type of violation or deeper shame committed to warrant the severe curse that followed. In seeing it may be in how he saw and in telling it may be in how/what he told his brothers. Or else it may be something worse.