The geometry of the diff mounts is probably a bit wrong. I don't know what the deal is with Skyline diffs, maybe the mounts are custom, or maybe you just put it in like it is. It's likely that the nose of the diff is pointing up, causing the tailshaft to be too close to the tunnel.
The fix would be to get adjustable top links. Ones with rod ends would also help reduce movement. To start with adjust them so the pinion is closer to horizontal, or maybe pointing down. (Not 100% sure on this, there is lots of theory about).
If you have axle tramp, get those bolt on addapters for the lower mounts on the diff. This brings the lower links back to horizontal. Also, check the condition of your shocks or adjust them.