Basic Guidelines For English Spellings
READ THESE ARTICLES
Chiefly in predicative use. Of a thief: having stolen goods in the hand; that is in the very act of stealing; ‘red-handed’. Also (in attributive use) of a theft: discovered while the thief was in the act of stealing.
Middle English. From hand + having, probably after Old English aet haebbendre handa.