@Arsalan Ehsan like @Gaurav bhai put it, you know your financial challenges better and you need to make that decision, however the Prado is definitely a good buy. You can always ask and make the necessary modifications if you determine off-roading is the sport for you.
Just to answer your questions, I am a big fan of the Pajero and yet it puts me off for desert driving for a few reasons. Mainly it is the low over hang bumpers that they come with, but that is an easy a cheap fix. Another thing is that the pre-2014 models have an up-shifting problem which means you cannot keep it in the 1st gear when required, it automatically shifts to the second gear thus you loose your momentum, however it is brilliant on-road and a very safe car. Maintaining it with an external garage is much cheaper than maintaining it with the Dealership. An example, I found out that the major service with Al Habtoor Motors would cost me AED 4000+. I got all the jobs to be done, purchased the necessary parts with them and fixed a rate with the garage to change all oils, coolant,timing belt, and it cost me extra because a couple of pulleys had also given in. The whole thing cost me less than AED 1000/-. Carnity has been amazing for me to find out more about my car and carry the services out at a much lower cost saving at least AED 1000s of Dirhams in maintenance and repair costs.
I love the Xterra as well and now we have 4 in the club, the off-road version is the one with the difflock and is definitely better than the stock version, bit if the price difference is too much and if you cannot afford it is not a deal breaker. However the rear shock for the X-Terra are the leaf springs which makes it slightly uncomfortable and a bit loud with regular off-roading, @Emmanuel did spray with some solution which did make a quieter for a bit, maybe he can share his experience better. @Shamil, @Jamy B. and @Asif Hussain are the other X-Terra owners and can give you the low down on their experiences. Don't be in a hurry and you will find plenty of deals going around soon.