"Looks like the vascular system has been compromised and only a bit is left to function. The scraping bit is worrisome, the black might just be honey dew, bacteria growing on insect excrement and scraping could ruin the epidermis..."
"Black stuff is usually fracas the poop of the insects...whatever insect problem you have. The reason plants become infested is the plant itself is not happy. Not vigorous. More vulnerable. That major pruning job might have ruined a big chunk of the vascular system...it was cut way too close to the main trunk/stem."
"Yup, your tree was buried too deeply. The soil and chips kept the moisture next to the bark long enough for the bacteria to damage the cambium just beneath...the vascular system. Only a third is viable thus the green growth you have left. Pull the soil back until you see the tops of the roots, take a picture. A weak plant is susceptible to disease and insects. Insects are not your problem, the damaged trunk is your problem..."
"I see no sign of any boring insect. I do see a compromised vascular system. The bark with bacteria and rot would be attractive to boring insects. ... I am so happy that you used potting soil!! Good job! ... The health and vigor of your tree has been compromised by soil and moisture up on your trunk."
"I am almost sure that these 'chewed' portions were after the pruning, during the last 3 years. I don't see sawdust. Sawdust would mean recent damage. That would still not mean the cause of your problem was because of borers, this might be a secondary problem. The dead branches would be good to cut off right at the trunk. I'd love to see what is going on between the trunk and one of those dead branches. Clean pruners with alcohol [before cutting] the end of the dead branch ... off the trunk."
~stormy (from comments)
"I'd suspect too much fertilizer to be the cause. A build-up of excess nitrogen may weaken the plant, and inhibit potassium (potassium strengthens stems and such; too much potassium is bad, too, though)."
~Shule (from comments)