Begin is the most commonly used. It can be used in not only polite letters but conversation as well. An example would be, "I began to wonder if he was actually coming or not"
Outset is a lot more rare, and is a quite a bit formal than "begin". Personally, I never use it. It means the "beginning" or "start" of something, and an example would be, "Tom's political skill was apparent from the outset"
Set off is less common than "begin", and means, "to start a journey" An example would be "He set off to London" It might be more common in British English.
Set out is also less common than "begin", and also means, "to start a journey" Set off and Set out mean the same thing, but "set out" is more common in American English I think. "I set out to mail some things at the post office" would be an example.
To be honest, I would just use "begin" a lot of the time. "Set off" and "set out" sounds a bit too formal for regular usage.
Begin: "I begin to cry" 「今、泣いて始める。」 it means "to start".
Outset: 「始めにの所」 "I wanted to explain myself at the outset." It is a noun for "the beginning", normally said when talking about the past in hindsight. Note: No one really uses this word.
Set off: "I set off to sea" 「海の中に行く。」 It means to leave for somewhere. Can also mean "to activate" as in, "I set off an alarm".
Set out: "I set out to sea." 「海の中に行く」 日本語で set off と set outは基本的には同じだ。 Same as "set off to", but more emphasis on the place you're going to. "Set off" has more emphasis on the act of going. They are more or less interchangeable.